* Implement proposal validation
Includes commits:
Implemented CProposalValidator
Use CProposalValidator to check proposals at prepare and submit stages
Modify proposal validator to support numerical data in string format
Multiple bug fixes in governance-validators.cpp
Fixed bug in CheckURL
Fixed stream state check
Increase strictness of payment address validation for compatibility with sentinel
Improved error reporting
Implemented "check" rpc command to validate proposals
Fixes to RPC check command
Fix error message
Unit test and data files for proposal validator
Added test cases
Removed debugging code
* Fix name validation
* Changes to address code review comments
* HD wallet
Minimal set of changes (no refactoring) backported from Bitcoin upstream to make HD wallets work in Dash 0.12.1.x+
* minimal bip44 (hardcoded account and change)
* minimal bip39
Additional cmd-line options for new wallet:
-mnemonic
-mnemonicpassphrase
* Do not recreate HD wallet on encryption
Adjusted keypool.py test
* Do not store any private keys for hd wallet besides the master one
Derive all keys on the fly.
Original idea/implementation - btc PR9298, backported and improved
* actually use bip39
* pbkdf2 test
* backport wallet-hd.py test
* Allow specifying hd seed, add dumphdseed rpc, fix bugs
- -hdseed cmd-line param to specify HD seed on wallet creation
- dumphdseed rpc to dump HD seed
- allow seed of any size
- fix dumpwallet rpc bug (wasn't decrypting HD seed)
- print HD seed and extended public masterkey on dumpwallet
* top up keypool on HD wallet encryption
* split HD chain: external/internal
* add missing cs_wallet lock in init.cpp
* fix `const char *` issues (use strings)
* default mnemonic passphrase is an empty string in all cases
* store mnemonic/mnemonicpassphrase
replace dumphdseed with dumphdinfo
* Add fCrypted flag to CHDChain
* prepare internal structures for multiple HD accounts
(plus some code cleanup)
* use secure allocator for storing sensitive HD data
* use secure strings for mnemonic(passphrase)
* small fix in GenerateNewHDChain
* use 24 words for mnemonic by default
* make sure mnemonic passphrase provided by user does not exceed 256 symbols
* more usage of secure allocators and memory_cleanse
* code cleanup
* rename: CSecureVector -> SecureVector
* add missing include
* fix warning in rpcdump.cpp
* refactor mnemonic_check (also fix a bug)
* move bip39 functions to CMnemonic
* Few fixes for CMnemonic:
- use `SecureVector` for data, bits, seed
- `Check` should return bool
* init vectors with desired size where possible
All names containing bitcoinconsensus remaned to contain dashconsensus.
This is needed to avoid conflicts with real bitcoinconsensus library
shipped with Bitcoin Core.
Signed-off-by: Oleg Girko <ol@infoserver.lv>
068c178 Added DBG macro in util.h to facilitate debugging
- This macro allows debugging statements (typically printf's or cout's) to
be activated or deactivated with a single comment. Uncomment the line:
//#define ENABLE_DASH_DEBUG
in util.h to enable debugging statements.
- When commented any code wrapped with the DBG() macro will simply be removed
by the preprocessor. When not commented all such wrapped statements will
be present.
- For maximum effectiveness it is best that util.h be the first effective include
in all source files. It is also possible to enable the macro for a single file
by temporarily adding #define ENABLE_DASH_DEBUG to the top of the file.
- Code committed to non-development branches should always have the define
commented.
d125d9b V0.12.1.x -- merging trigger/generic object/superblock changes for testnet phase II
- This commit contains the core governance system changes for 0.12.1. Any unrelated
changes have either been removed or moved to separate commits.
120724c File mode fixes
- Changed mode 0755->0644 on several source files.
c7f9e11 Updated todo reminders
- Added reminder to revert temporary reduction of number of votes
required to trigger superblock to 1 for testing
92adc98 Made CSuperblockManager::IsValidSuperblockHeight an inline function
- This is for efficiency since this function is called often and is
only 1 line of code.
c050ed7 Added comment explaining rationale for no LOCK(cs) in CSuperblock::IsValid
dc933fe Removed unused CSuperblockManager::IsBlockValid function
decec88 Moved calls to SuperblockManager::IsValidSuperblockHeight into IsSuperblockTriggered.
- Since calls to the later function are always protected by the former there's
no reason to keep these separate and this simplifies the code in
masternode-payments.cpp.
8672885 Reestablished expected value check for non-superblocks in IsBlockValueValid
b01cbe0 Changes to IsBlockValueValid to fix rpc test failure
a937c76 Changed include order to allow per file activation of the DBG macro
d116aa5 Fixed IsValidSuperblockHeight logic
- Note this has an effect on testing because we can now only create
1 superblock per day. Devs may need to temporarily change testnet params
for easier testing.
2d0c2de Convert superblock payments to CAmount
- We assume that payment values in JSON are in units of DASH
for consistency with other RPC functions, such as
createrawtransaction.
376b833 Revert temporary testing value for nAbsVoteReq
- Also ensure that number of votes required is never smaller than 1
8c89f4b Cleaned up CSuperblock error handling
- Exceptions are now thrown consistently rather than using a mix of
exceptions and return code checking. Exceptions are now caught only
in AddNewTrigger when the CSuperblock constructor is called. Unnecessary object
status members have been removed.
d7c8a6b Removed utilstrencodings header
- This appears to help with travis tests, for unknown reasons.
c4dfc7a Fixed some minor code review issues
63c3580 Reverted locking change in miner.
- This should have been done in the original PR but was overlooked.
4ab72de Fixed variable name to match common practice and bracket formatting
886a678 Improvements to vote conversion code
- Replaced redundantly defined function with inclusion of governance-vote.h
- Replaced magic numbers with their corresponding constant symbols
0a37966 Reordered governance message handling
This is a cherry-pick of a4625acb with minor conflict
resolution.
Conflicts:
src/Makefile.am
=====
leveldb: integrate leveldb into our buildsystem
leveldb's buildsystem causes us a few problems:
- breaks out-of-tree builds
- forces flags used for some tools
- limits cross builds
Rather than continuing to add wrappers around it, simply integrate it into our
build.
- fix parsing of BIND_NOW with older readelf
- add _IO_stdin_used to ignored exports
For details see: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=634261#109
- add check-symbols and check-security make targets
These are not added to the default checks because some of them depend on
release-build configs.
- always link librt for glibc back-compat builds
glibc absorbed clock_gettime in 2.17. librt (its previous location) is safe to
link in anyway for back-compat.
Fixes#7420
- add security/symbol checks to gitian
Github-Pull: #7424
Rebased-From: cd27bf51e0475813ba5bf3d3eaf78ea8ce872118a81c87fafc
Add "bip125-replaceable" output field to listtransactions and gettransaction
which indicates if an unconfirmed transaction, or any unconfirmed parent, is
signaling opt-in RBF according to BIP 125.
Github-Pull: #7286
Rebased-From: eaa8d2754b
58ef0ff doc: update docs for Tor listening (Wladimir J. van der Laan)
68ccdc4 doc: Mention Tor listening in release notes (Wladimir J. van der Laan)
09c1ae1 torcontrol improvements and fixes (Wladimir J. van der Laan)
2f796e5 Better error message if Tor version too old (Peter Todd)
8f4e67f net: Automatically create hidden service, listen on Tor (Wladimir J. van der Laan)
Starting with Tor version 0.2.7.1 it is possible, through Tor's control socket
API, to create and destroy 'ephemeral' hidden services programmatically.
https://stem.torproject.org/api/control.html#stem.control.Controller.create_ephemeral_hidden_service
This means that if Tor is running (and proper authorization is available),
bitcoin automatically creates a hidden service to listen on, without user
manual configuration. This will positively affect the number of available
.onion nodes.
- When the node is started, connect to Tor through control socket
- Send `ADD_ONION` command
- First time:
- Make it create a hidden service key
- Save the key in the data directory for later usage
- Make it redirect port 8333 to the local port 8333 (or whatever port we're listening on).
- Keep control socket connection open for as long node is running. The hidden service will
(by default) automatically go away when the connection is closed.
Until now there were quite a few leftovers, and only the coverage
related files in `src/` were cleaned, while the ones in the other dirs
remained. `qa/tmp/` is related to the BitcoinJ tests, and `cache/` is
related to RPC tests.
Benchmarking framework, loosely based on google's micro-benchmarking
library (https://github.com/google/benchmark)
Wny not use the Google Benchmark framework? Because adding Even More Dependencies
isn't worth it. If we get a dozen or three benchmarks and need nanosecond-accurate
timings of threaded code then switching to the full-blown Google Benchmark library
should be considered.
The benchmark framework is hard-coded to run each benchmark for one wall-clock second,
and then spits out .csv-format timing information to stdout. It is left as an
exercise for later (or maybe never) to add command-line arguments to specify which
benchmark(s) to run, how long to run them for, how to format results, etc etc etc.
Again, see the Google Benchmark framework for where that might end up.
See src/bench/MilliSleep.cpp for a sanity-test benchmark that just benchmarks
'sleep 100 milliseconds.'
To compile and run benchmarks:
cd src; make bench
Sample output:
Benchmark,count,min,max,average
Sleep100ms,10,0.101854,0.105059,0.103881
Continues Johnathan Corgan's work.
Publishing multipart messages
Bugfix: Add missing zmq header includes
Bugfix: Adjust build system to link ZeroMQ code for Qt binaries
d528025 Revert "rpc-tests: re-enable rpc-tests for Windows" (Wladimir J. van der Laan)
1e700c9 doc: update deps in build-unix.md after libevent (Wladimir J. van der Laan)
26c9b83 Move windows socket init to utility function (Wladimir J. van der Laan)
4be0b08 libevent: Windows reuseaddr workaround in depends (Cory Fields)
3a174cd Fix race condition between starting HTTP server thread and setting EventBase() (Wladimir J. van der Laan)
6d2bc22 Document options for new HTTP/RPC server in --help (Wladimir J. van der Laan)
be33f3f Implement RPCTimerHandler for Qt RPC console (Wladimir J. van der Laan)
57d85d9 doc: mention SSL support dropped for RPC in release notes (Wladimir J. van der Laan)
40b556d evhttpd implementation (Wladimir J. van der Laan)
ee2a42b tests: GET requests cannot have request body, use POST in rest.py (Wladimir J. van der Laan)
6e996d3 tests: fix qt payment test (Cory Fields)
3140ef9 build: build-system changes for libevent (Wladimir J. van der Laan)
a9af234 libevent: add depends (Cory Fields)
6a21dd5 Remove rpc_boostasiotocnetaddr test (Wladimir J. van der Laan)
8f9301c qa: Remove -rpckeepalive tests from httpbasics (Wladimir J. van der Laan)
51fcfc0 doc: remove documentation for rpcssl (Wladimir J. van der Laan)
- *Replace usage of boost::asio with [libevent2](http://libevent.org/)*.
boost::asio is not part of C++11, so unlike other boost there is no
forwards-compatibility reason to stick with it. Together with #4738 (convert
json_spirit to UniValue), this rids Bitcoin Core of the worst offenders with
regard to compile-time slowness.
- *Replace spit-and-duct-tape http server with evhttp*. Front-end http handling
is handled by libevent, a work queue (with configurable depth and parallelism)
is used to handle application requests.
- *Wrap HTTP request in C++ class*; this makes the application code mostly
HTTP-server-neutral
- *Refactor RPC to move all http-specific code to a separate file*.
Theoreticaly this can allow building without HTTP server but with another RPC
backend, e.g. Qt's debug console (currently not implemented) or future RPC
mechanisms people may want to use.
- *HTTP dispatch mechanism*; services (e.g., RPC, REST) register which URL
paths they want to handle.
By using a proven, high-performance asynchronous networking library (also used
by Tor) and HTTP server, problems such as #5674, #5655, #344 should be avoided.
What works? bitcoind, bitcoin-cli, bitcoin-qt. Unit tests and RPC/REST tests
pass. The aim for now is everything but SSL support.
Configuration options:
- `-rpcthreads`: repurposed as "number of work handler threads". Still
defaults to 4.
- `-rpcworkqueue`: maximum depth of work queue. When this is reached, new
requests will return a 500 Internal Error.
- `-rpctimeout`: inactivity time, in seconds, after which to disconnect a
client.
- `-debug=http`: low-level http activity logging
- Replaced coinbase cache in favor of using the masternode payments voting system only
- Syncing masternode payments now supports up to the syncing the entire payment list
- With nodes coming and going on the network, the network could come to different opinions about who should get paid next in line due to some nodes being flagged as failing a PoSe check. This will have to be fixed by introducing a blockchain based PoSe system, but that's out of the scope of this release. To fix the issues in the interrim, I'm removing PoSe checks for the time being.
- Masternode nLastPaid is removed and a new caching system that keeps the last 30 days of coinbase payees replaces it
- To deal with some significant attack vectors, the masternode donation feature was removed. The donation feature was added to support developement anyway, so this will be replaced by the budgeting code.
- This code should allow the network to come to consensus about who should be paid pretty effectively
- implement find_value() function for UniValue
- replace all Array/Value/Object types with UniValues, remove JSON Spirit to UniValue wrapper
- remove JSON Spirit sources
86a5f4b Relocate calls to CheckDiskSpace (Alex Morcos)
67708ac Write block index more frequently than cache flushes (Pieter Wuille)
b3ed423 Cache tweak and logging improvements (Pieter Wuille)
fc684ad Use accurate memory for flushing decisions (Pieter Wuille)
046392d Keep track of memory usage in CCoinsViewCache (Pieter Wuille)
540629c Add memusage.h (Pieter Wuille)
This class groups transactions that have been confirmed in blocks into buckets, based on either their fee or their priority. Then for each bucket, the class calculates what percentage of the transactions were confirmed within various numbers of blocks. It does this by keeping an exponentially decaying moving history for each bucket and confirm block count of the percentage of transactions in that bucket that were confirmed within that number of blocks.
-Eliminate txs which didn't have all inputs available at entry from fee/pri calcs
-Add dynamic breakpoints and tracking of confirmation delays in mempool transactions
-Remove old CMinerPolicyEstimator and CBlockAverage code
-New smartfees.py
-Pass a flag to the estimation code, using IsInitialBlockDownload as a proxy for when we are still catching up and we shouldn't be counting how many blocks it takes for transactions to be included.
-Add a policyestimator unit test
- Removed of reference node and replaced with decentralized quorums that pick the masternodes who get paid each block.
- Made a budgeting system, where masternodes can vote on individual budgets and the data is stored perminently on each clients computer
- Ensures ports remain open and client are responsive to IX requests.
- Completely 100% decentralized. This farms out the work of checking the masternode network to the masternode network. 1% of the network is determistically selected to check another 1% of the network each block. It takes six separate checks to deactivate a node, thus making it tamper proof.
- Nodes are kept in the masternode list if they fail enough PoSe checks to deactivate. They will continue to be checked until the operator fixes them. However they will not be paid while they're failing checks.
- Detect endian instead of stopping configure on big-endian
- Add `byteswap.h` and `endian.h` header for compatibility with
Windows and other operating systems that don't come with them
- Update `crypto/common.h` functions to use compat
endian header
This was added a while ago for testing purposes, but was never intended to be
used. Remove it until upstream libsecp256k1 decides that verification is
stable/ready.
Backwards-compatibility for libstdc++ is not limited to straightforward abi
changes. Symbol visibility also needs to be taken into consideration, and
that really can't be addressed simply.
Instead, just static-link libstdc++ for backwards-compat.
This makes it easier for us to replace it if desired, since it's now only in
one spot. Also, it avoids the openssl include from allocators.h, which
essentially forced openssl to be included from every compilation unit.
2ecd294 Bugfix: configure: Correctly detect "nothing to build" condition (Luke Dashjr)
b7a4ecc Bugfix: Only check for boost when building code that requires it (Luke Dashjr)
a19eeac Bugfix: configure: Check for openssl/ec.h (Luke Dashjr)
fe925e2 Use EXTRA_LIBRARIES instead of noinst_LIBRARIES so we can avoid building unused code (Cory Fields)
More info regarding KeePass: http://keepass.info/
KeePass integration will use KeePassHttp (https://github.com/pfn/keepasshttp/) to facilitate communications between the client and KeePass. KeePassHttp is a plugin for KeePass 2.x and provides a secure means of exposing KeePass entries via HTTP for clients to consume.
The implementation is dependent on the following:
- crypter.h for AES encryption helper functions.
- rpcprotocol.h for handling RPC communications. Could only be used partially however due some static values in the code.
- OpenSSL for base64 encoding. regular util.h libraries were not used for base64 encoding/decoding since they do not use secure allocation.
- JSON Spirit for reading / writing RPC communications
The following changes were made:
- Added CLI options in help
- Added RPC commands: keepass <genkey|init|setpassphrase>
- Added keepass.h and keepass.cpp which hold the integration routines
- Modified rpcwallet.cpp to support RPC commands
The following new options are available for darkcoind and darkcoin-qt:
-keepass Use KeePass 2 integration using KeePassHttp plugin (default: 0)
-keepassport=<port> Connect to KeePassHttp on port <port> (default: 19455)
-keepasskey=<key> KeePassHttp key for AES encrypted communication with KeePass
-keepassid=<name> KeePassHttp id for the established association
-keepassname=<name> Name to construct url for KeePass entry that stores the wallet passphrase
The following rpc commands are available:
- keepass genkey: generates a base64 encoded 256 bit AES key that can be used for the communication with KeePassHttp. Only necessary for manual configuration. Use init for automatic configuration.
- keepass init: sets up the association between darkcoind and keepass by generating an AES key and sending an association message to KeePassHttp. This will trigger KeePass to ask for an Id for the association. Returns the association and the base64 encoded string for the AES key.
- keepass setpassphrase <passphrase>: updates the passphrase in KeePassHttp to a new value. This should match the passphrase you intend to use for the wallet. Please note that the standard RPC commands walletpassphrasechange and the wallet encrption from the QT GUI already send the updates to KeePassHttp, so this is only necessary for manual manipulation of the password.
Sample initialization flow from darkcoin-qt console (this needs to be done only once to set up the association):
- Have KeePass running with an open database
- Start darkcoin-qt
- Open console
- type: "keepass init" in darkcoin-qt console
- (keepass pops up and asks for an association id, fill that in). Example: mydrkwallet
- response: Association successful. Id: mydrkwalletdarkcoin - Key: AgQkcs6cI7v9tlSYKjG/+s8wJrGALHl3jLosJpPLzUE=
- Edit darkcoin.conf and fill in these values
keepass=1
keepasskey=AgQkcs6cI7v9tlSYKjG/+s8wJrGALHl3jLosJpPLzUE=
keepassid=mydrkwallet
keepassname=testwallet
- Restart darkcoin-qt
At this point, the association is made. The next action depends on your particular situation:
- current wallet is not yet encrypted. Encrypting the wallet will trigger the integration and stores the password in KeePass (Under the 'KeePassHttp Passwords' group, named after keepassname.
- current wallet is already encrypted: use "keepass setpassphrase <passphrase>" to store the passphrase in KeePass.
At this point, the passphrase is stored in KeePassHttp. When Unlocking the wallet, one can use keepass as the passphrase to trigger retrieval of the password. This works from the RPC commands as well as the GUI.
Attempt to codify the possible error statuses associated with script
validation. script/types.h has been created with the expectation that it will
be part of the public lib interface. The other flag enums will be moved here in
a future commit.
Logging has also been removed in order to drop the dependency on core.h. It can
be re-added to bitcoind as-needed. This makes script verification finally free
of application state and boost!
99f41b9 MOVEONLY: core.o -> core/block.o (jtimon)
561e9e9 MOVEONLY: Move script/compressor out of script and put CTxOutCompressor (from core) with it (jtimon)
999a2ab MOVEONLY: separate CTxUndo out of core (jtimon)
4a3587d MOVEONLY: Separate CTransaction and dependencies from core (jtimon)
eda3733 MOVEONLY: Move CFeeRate and Amount constants to amount.o (jtimon)
50f71cd boost: code movement only: split CECKey into separate files (Cory Fields)
bdaec6a boost: remove CPubKey dependency from CECKey. Follow-up of e405aa48 (Cory Fields)
Similar to the INCLUDES changes in 6b099402b4, split out LIBS into individual
entries for more fine-grained control.
Also add MINIUPNPC_LIBS which was missing before, and hook it up to
executables.
