* net: a few small cleanups before replacing boost threads
- Drop the interruption point directly after the pnode allocation. This would
be leaky if hit.
- Rearrange thread creation so that the socket handler comes first
* net: add CThreadInterrupt and InterruptibleSleep
* net: make net interruptible
Also now that net threads are interruptible, switch them to use std
threads/binds/mutexes/condvars.
* net: make net processing interruptible
* net: remove thread_interrupted catch
This is now a std::thread, so there's no hope of catching a boost interruption
point.
* net: make proxy receives interruptible
* net: misc header cleanups
* Remove orphan state wipe from UnloadBlockIndex.
As orphan state is now "network state", like in
d6ea737be1,
UnloadBlockIndex is only used during init if we end up reindexing
to clear our block state so that we can start over. However, at
that time no connections have been brought up as CConnman hasn't
been started yet, so all of the network processing state logic is
empty when its called.
* Move network-msg-processing code out of main to its own file
* Rename the remaining main.{h,cpp} to validation.{h,cpp}
* net: move CBanDB and CAddrDB out of net.h/cpp
This will eventually solve a circular dependency
* net: Create CConnman to encapsulate p2p connections
* net: Move socket binding into CConnman
* net: move OpenNetworkConnection into CConnman
* net: move ban and addrman functions into CConnman
* net: Add oneshot functions to CConnman
* net: move added node functions to CConnman
* net: Add most functions needed for vNodes to CConnman
* net: handle nodesignals in CConnman
* net: Pass CConnection to wallet rather than using the global
* net: Add rpc error for missing/disabled p2p functionality
* net: Pass CConnman around as needed
* gui: add NodeID to the peer table
* net: create generic functor accessors and move vNodes to CConnman
* net: move whitelist functions into CConnman
* net: move nLastNodeId to CConnman
* net: move nLocalHostNonce to CConnman
This behavior seems to have been quite racy and broken.
Move nLocalHostNonce into CNode, and check received nonces against all
non-fully-connected nodes. If there's a match, assume we've connected
to ourself.
* net: move messageHandlerCondition to CConnman
* net: move send/recv statistics to CConnman
* net: move SendBufferSize/ReceiveFloodSize to CConnman
* net: move nLocalServices/nRelevantServices to CConnman
These are in-turn passed to CNode at connection time. This allows us to offer
different services to different peers (or test the effects of doing so).
* net: move semOutbound and semMasternodeOutbound to CConnman
* net: SocketSendData returns written size
* net: move max/max-outbound to CConnman
* net: Pass best block known height into CConnman
CConnman then passes the current best height into CNode at creation time.
This way CConnman/CNode have no dependency on main for height, and the signals
only move in one direction.
This also helps to prevent identity leakage a tiny bit. Before this change, an
attacker could theoretically make 2 connections on different interfaces. They
would connect fully on one, and only establish the initial connection on the
other. Once they receive a new block, they would relay it to your first
connection, and immediately commence the version handshake on the second. Since
the new block height is reflected immediately, they could attempt to learn
whether the two connections were correlated.
This is, of course, incredibly unlikely to work due to the small timings
involved and receipt from other senders. But it doesn't hurt to lock-in
nBestHeight at the time of connection, rather than letting the remote choose
the time.
* net: pass CClientUIInterface into CConnman
* net: Drop StartNode/StopNode and use CConnman directly
* net: Introduce CConnection::Options to avoid passing so many params
* net: add nSendBufferMaxSize/nReceiveFloodSize to CConnection::Options
* net: move vNodesDisconnected into CConnman
* Made the ForEachNode* functions in src/net.cpp more pragmatic and self documenting
* Convert ForEachNode* functions to take a templated function argument rather than a std::function to eliminate std::function overhead
* net: move MAX_FEELER_CONNECTIONS into connman
* Build against system UniValue when available
* doc: Add UniValue to build instructions
* Bugfix: The var is LIBUNIVALUE,not LIBBITCOIN_UNIVALUE
* Change default configure option --with-system-univalue to "no"
* Bugfix: Always include univalue in DIST_SUBDIRS
* LDADD dependency order shuffling
* build-unix: Update UniValue build conditions
* Implement proposal validation
Includes commits:
Implemented CProposalValidator
Use CProposalValidator to check proposals at prepare and submit stages
Modify proposal validator to support numerical data in string format
Multiple bug fixes in governance-validators.cpp
Fixed bug in CheckURL
Fixed stream state check
Increase strictness of payment address validation for compatibility with sentinel
Improved error reporting
Implemented "check" rpc command to validate proposals
Fixes to RPC check command
Fix error message
Unit test and data files for proposal validator
Added test cases
Removed debugging code
* Fix name validation
* Changes to address code review comments
* HD wallet
Minimal set of changes (no refactoring) backported from Bitcoin upstream to make HD wallets work in Dash 0.12.1.x+
* minimal bip44 (hardcoded account and change)
* minimal bip39
Additional cmd-line options for new wallet:
-mnemonic
-mnemonicpassphrase
* Do not recreate HD wallet on encryption
Adjusted keypool.py test
* Do not store any private keys for hd wallet besides the master one
Derive all keys on the fly.
Original idea/implementation - btc PR9298, backported and improved
* actually use bip39
* pbkdf2 test
* backport wallet-hd.py test
* Allow specifying hd seed, add dumphdseed rpc, fix bugs
- -hdseed cmd-line param to specify HD seed on wallet creation
- dumphdseed rpc to dump HD seed
- allow seed of any size
- fix dumpwallet rpc bug (wasn't decrypting HD seed)
- print HD seed and extended public masterkey on dumpwallet
* top up keypool on HD wallet encryption
* split HD chain: external/internal
* add missing cs_wallet lock in init.cpp
* fix `const char *` issues (use strings)
* default mnemonic passphrase is an empty string in all cases
* store mnemonic/mnemonicpassphrase
replace dumphdseed with dumphdinfo
* Add fCrypted flag to CHDChain
* prepare internal structures for multiple HD accounts
(plus some code cleanup)
* use secure allocator for storing sensitive HD data
* use secure strings for mnemonic(passphrase)
* small fix in GenerateNewHDChain
* use 24 words for mnemonic by default
* make sure mnemonic passphrase provided by user does not exceed 256 symbols
* more usage of secure allocators and memory_cleanse
* code cleanup
* rename: CSecureVector -> SecureVector
* add missing include
* fix warning in rpcdump.cpp
* refactor mnemonic_check (also fix a bug)
* move bip39 functions to CMnemonic
* Few fixes for CMnemonic:
- use `SecureVector` for data, bits, seed
- `Check` should return bool
* init vectors with desired size where possible
All names containing bitcoinconsensus remaned to contain dashconsensus.
This is needed to avoid conflicts with real bitcoinconsensus library
shipped with Bitcoin Core.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
068c178 Added DBG macro in util.h to facilitate debugging
- This macro allows debugging statements (typically printf's or cout's) to
be activated or deactivated with a single comment. Uncomment the line:
//#define ENABLE_DASH_DEBUG
in util.h to enable debugging statements.
- When commented any code wrapped with the DBG() macro will simply be removed
by the preprocessor. When not commented all such wrapped statements will
be present.
- For maximum effectiveness it is best that util.h be the first effective include
in all source files. It is also possible to enable the macro for a single file
by temporarily adding #define ENABLE_DASH_DEBUG to the top of the file.
- Code committed to non-development branches should always have the define
commented.
d125d9b V0.12.1.x -- merging trigger/generic object/superblock changes for testnet phase II
- This commit contains the core governance system changes for 0.12.1. Any unrelated
changes have either been removed or moved to separate commits.
120724c File mode fixes
- Changed mode 0755->0644 on several source files.
c7f9e11 Updated todo reminders
- Added reminder to revert temporary reduction of number of votes
required to trigger superblock to 1 for testing
92adc98 Made CSuperblockManager::IsValidSuperblockHeight an inline function
- This is for efficiency since this function is called often and is
only 1 line of code.
c050ed7 Added comment explaining rationale for no LOCK(cs) in CSuperblock::IsValid
dc933fe Removed unused CSuperblockManager::IsBlockValid function
decec88 Moved calls to SuperblockManager::IsValidSuperblockHeight into IsSuperblockTriggered.
- Since calls to the later function are always protected by the former there's
no reason to keep these separate and this simplifies the code in
masternode-payments.cpp.
8672885 Reestablished expected value check for non-superblocks in IsBlockValueValid
b01cbe0 Changes to IsBlockValueValid to fix rpc test failure
a937c76 Changed include order to allow per file activation of the DBG macro
d116aa5 Fixed IsValidSuperblockHeight logic
- Note this has an effect on testing because we can now only create
1 superblock per day. Devs may need to temporarily change testnet params
for easier testing.
2d0c2de Convert superblock payments to CAmount
- We assume that payment values in JSON are in units of DASH
for consistency with other RPC functions, such as
createrawtransaction.
376b833 Revert temporary testing value for nAbsVoteReq
- Also ensure that number of votes required is never smaller than 1
8c89f4b Cleaned up CSuperblock error handling
- Exceptions are now thrown consistently rather than using a mix of
exceptions and return code checking. Exceptions are now caught only
in AddNewTrigger when the CSuperblock constructor is called. Unnecessary object
status members have been removed.
d7c8a6b Removed utilstrencodings header
- This appears to help with travis tests, for unknown reasons.
c4dfc7a Fixed some minor code review issues
63c3580 Reverted locking change in miner.
- This should have been done in the original PR but was overlooked.
4ab72de Fixed variable name to match common practice and bracket formatting
886a678 Improvements to vote conversion code
- Replaced redundantly defined function with inclusion of governance-vote.h
- Replaced magic numbers with their corresponding constant symbols
0a37966 Reordered governance message handling
This is a cherry-pick of a4625acb with minor conflict
resolution.
Conflicts:
src/Makefile.am
=====
leveldb: integrate leveldb into our buildsystem
leveldb's buildsystem causes us a few problems:
- breaks out-of-tree builds
- forces flags used for some tools
- limits cross builds
Rather than continuing to add wrappers around it, simply integrate it into our
build.
- fix parsing of BIND_NOW with older readelf
- add _IO_stdin_used to ignored exports
For details see: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=634261#109
- add check-symbols and check-security make targets
These are not added to the default checks because some of them depend on
release-build configs.
- always link librt for glibc back-compat builds
glibc absorbed clock_gettime in 2.17. librt (its previous location) is safe to
link in anyway for back-compat.
Fixes#7420
- add security/symbol checks to gitian
Github-Pull: #7424
Rebased-From: cd27bf51e0475813ba5bf3d3eaf78ea8ce872118a81c87fafc
Add "bip125-replaceable" output field to listtransactions and gettransaction
which indicates if an unconfirmed transaction, or any unconfirmed parent, is
signaling opt-in RBF according to BIP 125.
Github-Pull: #7286
Rebased-From: eaa8d2754b
58ef0ff doc: update docs for Tor listening (Wladimir J. van der Laan)
68ccdc4 doc: Mention Tor listening in release notes (Wladimir J. van der Laan)
09c1ae1 torcontrol improvements and fixes (Wladimir J. van der Laan)
2f796e5 Better error message if Tor version too old (Peter Todd)
8f4e67f net: Automatically create hidden service, listen on Tor (Wladimir J. van der Laan)
Starting with Tor version 0.2.7.1 it is possible, through Tor's control socket
API, to create and destroy 'ephemeral' hidden services programmatically.
https://stem.torproject.org/api/control.html#stem.control.Controller.create_ephemeral_hidden_service
This means that if Tor is running (and proper authorization is available),
bitcoin automatically creates a hidden service to listen on, without user
manual configuration. This will positively affect the number of available
.onion nodes.
- When the node is started, connect to Tor through control socket
- Send `ADD_ONION` command
- First time:
- Make it create a hidden service key
- Save the key in the data directory for later usage
- Make it redirect port 8333 to the local port 8333 (or whatever port we're listening on).
- Keep control socket connection open for as long node is running. The hidden service will
(by default) automatically go away when the connection is closed.
Until now there were quite a few leftovers, and only the coverage
related files in `src/` were cleaned, while the ones in the other dirs
remained. `qa/tmp/` is related to the BitcoinJ tests, and `cache/` is
related to RPC tests.
Benchmarking framework, loosely based on google's micro-benchmarking
library (https://github.com/google/benchmark)
Wny not use the Google Benchmark framework? Because adding Even More Dependencies
isn't worth it. If we get a dozen or three benchmarks and need nanosecond-accurate
timings of threaded code then switching to the full-blown Google Benchmark library
should be considered.
The benchmark framework is hard-coded to run each benchmark for one wall-clock second,
and then spits out .csv-format timing information to stdout. It is left as an
exercise for later (or maybe never) to add command-line arguments to specify which
benchmark(s) to run, how long to run them for, how to format results, etc etc etc.
Again, see the Google Benchmark framework for where that might end up.
See src/bench/MilliSleep.cpp for a sanity-test benchmark that just benchmarks
'sleep 100 milliseconds.'
To compile and run benchmarks:
cd src; make bench
Sample output:
Benchmark,count,min,max,average
Sleep100ms,10,0.101854,0.105059,0.103881
Continues Johnathan Corgan's work.
Publishing multipart messages
Bugfix: Add missing zmq header includes
Bugfix: Adjust build system to link ZeroMQ code for Qt binaries
d528025 Revert "rpc-tests: re-enable rpc-tests for Windows" (Wladimir J. van der Laan)
1e700c9 doc: update deps in build-unix.md after libevent (Wladimir J. van der Laan)
26c9b83 Move windows socket init to utility function (Wladimir J. van der Laan)
4be0b08 libevent: Windows reuseaddr workaround in depends (Cory Fields)
3a174cd Fix race condition between starting HTTP server thread and setting EventBase() (Wladimir J. van der Laan)
6d2bc22 Document options for new HTTP/RPC server in --help (Wladimir J. van der Laan)
be33f3f Implement RPCTimerHandler for Qt RPC console (Wladimir J. van der Laan)
57d85d9 doc: mention SSL support dropped for RPC in release notes (Wladimir J. van der Laan)
40b556d evhttpd implementation (Wladimir J. van der Laan)
ee2a42b tests: GET requests cannot have request body, use POST in rest.py (Wladimir J. van der Laan)
6e996d3 tests: fix qt payment test (Cory Fields)
3140ef9 build: build-system changes for libevent (Wladimir J. van der Laan)
a9af234 libevent: add depends (Cory Fields)
6a21dd5 Remove rpc_boostasiotocnetaddr test (Wladimir J. van der Laan)
8f9301c qa: Remove -rpckeepalive tests from httpbasics (Wladimir J. van der Laan)
51fcfc0 doc: remove documentation for rpcssl (Wladimir J. van der Laan)
- *Replace usage of boost::asio with [libevent2](http://libevent.org/)*.
boost::asio is not part of C++11, so unlike other boost there is no
forwards-compatibility reason to stick with it. Together with #4738 (convert
json_spirit to UniValue), this rids Bitcoin Core of the worst offenders with
regard to compile-time slowness.
- *Replace spit-and-duct-tape http server with evhttp*. Front-end http handling
is handled by libevent, a work queue (with configurable depth and parallelism)
is used to handle application requests.
- *Wrap HTTP request in C++ class*; this makes the application code mostly
HTTP-server-neutral
- *Refactor RPC to move all http-specific code to a separate file*.
Theoreticaly this can allow building without HTTP server but with another RPC
backend, e.g. Qt's debug console (currently not implemented) or future RPC
mechanisms people may want to use.
- *HTTP dispatch mechanism*; services (e.g., RPC, REST) register which URL
paths they want to handle.
By using a proven, high-performance asynchronous networking library (also used
by Tor) and HTTP server, problems such as #5674, #5655, #344 should be avoided.
What works? bitcoind, bitcoin-cli, bitcoin-qt. Unit tests and RPC/REST tests
pass. The aim for now is everything but SSL support.
Configuration options:
- `-rpcthreads`: repurposed as "number of work handler threads". Still
defaults to 4.
- `-rpcworkqueue`: maximum depth of work queue. When this is reached, new
requests will return a 500 Internal Error.
- `-rpctimeout`: inactivity time, in seconds, after which to disconnect a
client.
- `-debug=http`: low-level http activity logging
- Replaced coinbase cache in favor of using the masternode payments voting system only
- Syncing masternode payments now supports up to the syncing the entire payment list
- With nodes coming and going on the network, the network could come to different opinions about who should get paid next in line due to some nodes being flagged as failing a PoSe check. This will have to be fixed by introducing a blockchain based PoSe system, but that's out of the scope of this release. To fix the issues in the interrim, I'm removing PoSe checks for the time being.
- Masternode nLastPaid is removed and a new caching system that keeps the last 30 days of coinbase payees replaces it
- To deal with some significant attack vectors, the masternode donation feature was removed. The donation feature was added to support developement anyway, so this will be replaced by the budgeting code.
- This code should allow the network to come to consensus about who should be paid pretty effectively
- implement find_value() function for UniValue
- replace all Array/Value/Object types with UniValues, remove JSON Spirit to UniValue wrapper
- remove JSON Spirit sources
86a5f4b Relocate calls to CheckDiskSpace (Alex Morcos)
67708ac Write block index more frequently than cache flushes (Pieter Wuille)
b3ed423 Cache tweak and logging improvements (Pieter Wuille)
fc684ad Use accurate memory for flushing decisions (Pieter Wuille)
046392d Keep track of memory usage in CCoinsViewCache (Pieter Wuille)
540629c Add memusage.h (Pieter Wuille)
This class groups transactions that have been confirmed in blocks into buckets, based on either their fee or their priority. Then for each bucket, the class calculates what percentage of the transactions were confirmed within various numbers of blocks. It does this by keeping an exponentially decaying moving history for each bucket and confirm block count of the percentage of transactions in that bucket that were confirmed within that number of blocks.
-Eliminate txs which didn't have all inputs available at entry from fee/pri calcs
-Add dynamic breakpoints and tracking of confirmation delays in mempool transactions
-Remove old CMinerPolicyEstimator and CBlockAverage code
-New smartfees.py
-Pass a flag to the estimation code, using IsInitialBlockDownload as a proxy for when we are still catching up and we shouldn't be counting how many blocks it takes for transactions to be included.
-Add a policyestimator unit test
- Removed of reference node and replaced with decentralized quorums that pick the masternodes who get paid each block.
- Made a budgeting system, where masternodes can vote on individual budgets and the data is stored perminently on each clients computer
