* Fix duplicate headers download in initial sync
Now that initial block download is delayed until the headers sync is done,
it was noticed that the initial headers sync may happen multiple times in
parallel in the case new blocks are announced. This happens because for
every block in INV that is received, a getheaders message is immediately
sent out resulting in a full download of the headers chain starting from
the point of where the initial headers sync is currently at. This happens
once for each peer that announces the new block. This slows down the
initial headers sync and increases the chance of another block being
announced before it is finished, probably leading to the same behavior
as already described, slowing down the sync even more...and so on.
This commit delays sending of GETHEADERS to later in case the chain is too
far behind while a new block gets announced. Header chains will still be
downloaded multiple times, but the downloading will start much closer
to the tip of the chain, so the damage is not that bad anymore.
This ensures that we get all headers from all peers, even if any of them
is on another chain. This should avoid what happened in
https://github.com/bitcoin/bitcoin/pull/8054
which needed to be reverted later.
This fixes the Bitcoin issue https://github.com/bitcoin/bitcoin/issues/6755
* Introduce DelayGetHeadersTime chain param and fix tests
The delaying of GETHEADERS in combination with very old block times in
test cases resulted in the delaying being triggered when the first newly
mined block arrives. This results in a completely stalled sync.
This is fixed by avoiding delaying in when running tests.
* Disconnect peers which are not catched up
Peers which stop sending us headers too early are very likely peers which
did not catch up before and stalled for some reason. We should disconnect
these peers and chose another one to continue.
* IBD check uses minimumchain work instead of checkpoints.
This introduces a 'minimum chain work' chainparam which is intended
to be the known amount of work in the chain for the network at the
time of software release. If you don't have this much work, you're
not yet caught up.
This is used instead of the count of blocks test from checkpoints.
This criteria is trivial to keep updated as there is no element of
subjectivity, trust, or position dependence to it. It is also a more
reliable metric of sync status than a block count.
* Remove GetTotalBlocksEstimate and checkpoint tests that test nothing.
GetTotalBlocksEstimate is no longer used and it was the only thing
the checkpoint tests were testing.
Since checkpoints are on their way out it makes more sense to remove
the test file than to cook up a new pointless test.
# Conflicts:
# src/Makefile.test.include
# src/test/Checkpoints_tests.cpp
* IsInitialBlockDownload no longer uses header-only timestamps.
This avoids a corner case (mostly visible on testnet) where bogus
headers can keep nodes in IsInitialBlockDownload.
* Delay parallel block download until chain has sufficient work
nMinimumChainWork is an anti-DoS threshold; wait until we have a proposed
tip with more work than that before downloading blocks towards that tip.
* Add timeout for headers sync
At startup, we choose one peer to serve us the headers chain, until
our best header is close to caught up. Disconnect this peer if more
than 15 minutes + 1ms/expected_header passes and our best header
is still more than 1 day away from current time.
* Introduce assumevalid setting to skip presumed valid scripts.
This disentangles the script validation skipping from checkpoints.
A new option is introduced "assumevalid" which specifies a block whos
ancestors we assume all have valid scriptsigs and so we do not check
them when they are also burried under the best header by two weeks
worth of work.
Unlike checkpoints this has no influence on consensus unless you set
it to a block with an invalid history. Because of this it can be
easily be updated without risk of influencing the network consensus.
This results in a massive IBD speedup.
This approach was independently recommended by Peter Todd and Luke-Jr
since POW based signature skipping (see PR#9180) does not have the
verifiable properties of a specific hash and may create bad incentives.
The downside is that, like checkpoints, the defaults bitrot and older
releases will sync slower. On the plus side users can provide their
own value here, and if they set it to something crazy all that will
happen is more time will be spend validating signatures.
Checkblocks and checklevel are also moved to the hidden debug options:
Especially now that checkblocks has a low default there is little need
to change these settings, and users frequently misunderstand them as
influencing security or IBD speed. By hiding them we offset the
space added by this new option.
* Add consensusParams to FindNextBlocksToDownload
* Adjust check in headers timeout logic to align with 144 blocks in Dash
* HD wallet
Minimal set of changes (no refactoring) backported from Bitcoin upstream to make HD wallets work in Dash 0.12.1.x+
* minimal bip44 (hardcoded account and change)
* minimal bip39
Additional cmd-line options for new wallet:
-mnemonic
-mnemonicpassphrase
* Do not recreate HD wallet on encryption
Adjusted keypool.py test
* Do not store any private keys for hd wallet besides the master one
Derive all keys on the fly.
Original idea/implementation - btc PR9298, backported and improved
* actually use bip39
* pbkdf2 test
* backport wallet-hd.py test
* Allow specifying hd seed, add dumphdseed rpc, fix bugs
- -hdseed cmd-line param to specify HD seed on wallet creation
- dumphdseed rpc to dump HD seed
- allow seed of any size
- fix dumpwallet rpc bug (wasn't decrypting HD seed)
- print HD seed and extended public masterkey on dumpwallet
* top up keypool on HD wallet encryption
* split HD chain: external/internal
* add missing cs_wallet lock in init.cpp
* fix `const char *` issues (use strings)
* default mnemonic passphrase is an empty string in all cases
* store mnemonic/mnemonicpassphrase
replace dumphdseed with dumphdinfo
* Add fCrypted flag to CHDChain
* prepare internal structures for multiple HD accounts
(plus some code cleanup)
* use secure allocator for storing sensitive HD data
* use secure strings for mnemonic(passphrase)
* small fix in GenerateNewHDChain
* use 24 words for mnemonic by default
* make sure mnemonic passphrase provided by user does not exceed 256 symbols
* more usage of secure allocators and memory_cleanse
* code cleanup
* rename: CSecureVector -> SecureVector
* add missing include
* fix warning in rpcdump.cpp
* refactor mnemonic_check (also fix a bug)
* move bip39 functions to CMnemonic
* Few fixes for CMnemonic:
- use `SecureVector` for data, bits, seed
- `Check` should return bool
* init vectors with desired size where possible
* Multi-quorum InstantSend, complete refactoring
+ cleanup for IS and partial protobump
* more changes:
- allow InstantSend tx to have 10 inputs max
- store many unique tx hashes in mapVotedOutpoints
- more checks in AcceptToMemoryPoolWorker (moved from ProcessMessage + CTxLockRequest(tx).IsValid() )
* More changes:
- let multiple lock candidates compete for votes
- fail to vote on the same outpoint twice early
* More changes:
- notify CInstantSend on UpdatedBlockTip -> remove cs_main from CheckAndRemove()
- notify CInstantSend on SyncTransaction -> count expiration block starting from the block corresponding tx was confirmed instead of the block lock candidate/vote was created
- fixed few locks
* add comments about nConfirmedHeight
* Fix "Block vs Lock" edge case
* Fix "Block vs Lock" edge case, p2
* Fix issues:
- fix logic for locking inputs and notifying - see UpdateLockedTransaction, TryToFinalizeLockCandidate
- add missing hash inserting in ProcessTxLockVote
- add nMaxBlocks param to ResolveConflicts to limit max depth allowed to disconnect blocks recursively
- fix false positive mempool conflict
- add missing mutex locks
- fix fRequireUnspent logic in CTxLockRequest::IsValid
b6b6d6c Added nSuperblockStartBlock, adjusted testnet/regtest params
15a3c64 More for governance block checks, p1 (non-compilable):
- add GetPaymentsLimit() and GetPaymentsTotalAmount()
- IsValidBlockHeight() should check nSuperblockStartBlock
- CSuperblock::IsValid should check payment limit and miner payout
- no cs_main
- slightly refactored related things
e8f9e5d More for governance checks, p2 (compilable):
- IsBlockValueValid(), IsBlockPayeeValid() and FillBlockPayee() rewritten, no cs_main for them
- CreateNewBlock adjusted, need more work on CBlockTemplate (see TODO)
- moved (and simplified) IsBlockPayeeValid() call from CheckBlock() to ConnectBlock()
51434cf Add ability to calculate only superblock part of subsidy in GetBlockSubsidy()
aa74200 Fix GetPaymentsLimit()
f7b6234 braces and comment
ade8f64 more checks for IsValidBlockHeight()
068c178 Added DBG macro in util.h to facilitate debugging
- This macro allows debugging statements (typically printf's or cout's) to
be activated or deactivated with a single comment. Uncomment the line:
//#define ENABLE_DASH_DEBUG
in util.h to enable debugging statements.
- When commented any code wrapped with the DBG() macro will simply be removed
by the preprocessor. When not commented all such wrapped statements will
be present.
- For maximum effectiveness it is best that util.h be the first effective include
in all source files. It is also possible to enable the macro for a single file
by temporarily adding #define ENABLE_DASH_DEBUG to the top of the file.
- Code committed to non-development branches should always have the define
commented.
d125d9b V0.12.1.x -- merging trigger/generic object/superblock changes for testnet phase II
- This commit contains the core governance system changes for 0.12.1. Any unrelated
changes have either been removed or moved to separate commits.
120724c File mode fixes
- Changed mode 0755->0644 on several source files.
c7f9e11 Updated todo reminders
- Added reminder to revert temporary reduction of number of votes
required to trigger superblock to 1 for testing
92adc98 Made CSuperblockManager::IsValidSuperblockHeight an inline function
- This is for efficiency since this function is called often and is
only 1 line of code.
c050ed7 Added comment explaining rationale for no LOCK(cs) in CSuperblock::IsValid
dc933fe Removed unused CSuperblockManager::IsBlockValid function
decec88 Moved calls to SuperblockManager::IsValidSuperblockHeight into IsSuperblockTriggered.
- Since calls to the later function are always protected by the former there's
no reason to keep these separate and this simplifies the code in
masternode-payments.cpp.
8672885 Reestablished expected value check for non-superblocks in IsBlockValueValid
b01cbe0 Changes to IsBlockValueValid to fix rpc test failure
a937c76 Changed include order to allow per file activation of the DBG macro
d116aa5 Fixed IsValidSuperblockHeight logic
- Note this has an effect on testing because we can now only create
1 superblock per day. Devs may need to temporarily change testnet params
for easier testing.
2d0c2de Convert superblock payments to CAmount
- We assume that payment values in JSON are in units of DASH
for consistency with other RPC functions, such as
createrawtransaction.
376b833 Revert temporary testing value for nAbsVoteReq
- Also ensure that number of votes required is never smaller than 1
8c89f4b Cleaned up CSuperblock error handling
- Exceptions are now thrown consistently rather than using a mix of
exceptions and return code checking. Exceptions are now caught only
in AddNewTrigger when the CSuperblock constructor is called. Unnecessary object
status members have been removed.
d7c8a6b Removed utilstrencodings header
- This appears to help with travis tests, for unknown reasons.
c4dfc7a Fixed some minor code review issues
63c3580 Reverted locking change in miner.
- This should have been done in the original PR but was overlooked.
4ab72de Fixed variable name to match common practice and bracket formatting
886a678 Improvements to vote conversion code
- Replaced redundantly defined function with inclusion of governance-vote.h
- Replaced magic numbers with their corresponding constant symbols
0a37966 Reordered governance message handling
42bdf42 Refactor/fix spork:
- move ProcessSpork, GetSporkValue, IsSporkActive, ExecuteSpork and mapSporksActive to CSporkManager
- move Sign, CheckSignature, Relay to CSporkMessage
- move ReprocessBlocks out of sporks to main.cpp / rename DisconnectBlocksAndReprocess to DisconnectBlocks
- rename SporkKey to SporkPubKey
- bugfix: only set strMasterPrivKey if spork signature produced by that key was verified successfully
- few log format changes, cleaned up includes
- use "" account address instead
- remove `collateralPubKey` and `SetCollateralAddress()` from darksend and create `dummyScriptPubkey` and `InitDummyScriptPubkey()` in masternodeman instead
- initialize `dummyScriptPubkey` before loading cache otherwise MNs could be removed because of the invalid tx
get rid of most `if(Params().NetworkIDString() == CBaseChainParams::MAIN)` and alike to unify formulas.
Further improvements will require testnet restart (see TODO notes in changed code) so I'm keeping few things as they are till then.
This switches the Merkle tree logic for blocks to one that runs in constant (small) space.
The old code is moved to tests, and a new test is added that for various combinations of
block sizes, transaction positions to compute a branch for, and mutations:
* Verifies that the old code and new code agree for the Merkle root.
* Verifies that the old code and new code agree for the Merkle branch.
* Verifies that the computed Merkle branch is valid.
* Verifies that mutations don't change the Merkle root.
* Verifies that mutations are correctly detected.
Assume that when a wallet transaction has a valid block hash and transaction position
in it, the transaction is actually there. We're already trusting wallet data in a
much more fundamental way anyway.
To prevent backward compatibility issues, a new record is used for storing the
block locator in the wallet. Old wallets will see a wallet file synchronized up
to the genesis block, and rescan automatically.
- use map.insert instead of [] (should be safer)
- debug output / comments / spaces / names
- fix few long strings / make translatable one more label in UI
Clean up the code in chainparams a bit more after the recent
refactorings. In particular, make sure the structure of the "RegTest"
params matches the structure of the other classes. This makes the code
clearer to read.
Also remove redundant values of the genesis block in always-specified
optional arguments and mark variable/argument as "const".
I've never liked the chain-specific exception to having to set a
password. It gives issues with #6388 which makes it valid to
set no password in every case (as it enables random cookie authentication).
This pull removes the flag, so that all chains are regarded the same.
It also removes the username==password test, which doesn't provide any
substantial extra security.
No longer relevant after #5957. This hack existed because of another
hack where the numthreads parameter, on regtest, doubled as how many
blocks to generate.
If/when CTransaction::CURRENT_VERSION is incremented, this will break CChainParams and the miner tests. This fix sets the transaction version explicitly where we depend on the hash value (genesis block, proof of work checks).
a8cdaf5 checkpoints: move the checkpoints enable boolean into main (Cory Fields)
11982d3 checkpoints: Decouple checkpoints from Params (Cory Fields)
6996823 checkpoints: make checkpoints a member of CChainParams (Cory Fields)
9f13a10 checkpoints: store mapCheckpoints in CCheckpointData rather than a pointer (Cory Fields)
This adds a -prune=N option to bitcoind, which if set to N>0 will enable block
file pruning. When pruning is enabled, block and undo files will be deleted to
try to keep total space used by those files to below the prune target (N, in
MB) specified by the user, subject to some constraints:
- The last 288 blocks on the main chain are always kept (MIN_BLOCKS_TO_KEEP),
- N must be at least 550MB (chosen as a value for the target that could
reasonably be met, with some assumptions about block sizes, orphan rates,
etc; see comment in main.h),
- No blocks are pruned until chainActive is at least 100,000 blocks long (on
mainnet; defined separately for mainnet, testnet, and regtest in chainparams
as nPruneAfterHeight).
This unsets NODE_NETWORK if pruning is enabled.
Also included is an RPC test for pruning (pruning.py).
Thanks to @rdponticelli for earlier work on this feature; this is based in
part off that work.
This adds a -checkblockindex (defaulting to true for regtest), which occasionally
does a full consistency check for mapBlockIndex, setBlockIndexCandidates, chainActive, and
mapBlocksUnlinked.
This adds a -checkblockindex (defaulting to true for regtest), which occasionally
does a full consistency check for mapBlockIndex, setBlockIndexCandidates, chainActive, and
mapBlocksUnlinked.
UNITTEST parameter are not used by any current tests, and the model
(modifyable parameters) is inconvenient when unit-testing. As
they are stored in a global structure eevery test
would have to (re)set up its own parameters.
For consistency it is also better to test with MAIN parameters.
856e862 namespace: drop most boost namespaces and a few header cleanups (Cory Fields)
9b1ab86 namespace: drop boost::assign altogether here (Cory Fields)
a324199 namespace: remove boost namespace pollution (Cory Fields)
If uint256() constructor takes a string, uint256(0) will become
dangerous when uint256 does not take integers anymore (it will go
through std::string(const char*) making a NULL string, and the explicit
keyword is no help).
85c579e script: add a slew of includes all around and drop includes from script.h (Cory Fields)
db8eb54 script: move ToString and ValueString out of the header (Cory Fields)
e9ca428 script: add ToByteVector() for converting anything with begin/end (Cory Fields)
066e2a1 script: move CScriptID to standard.h and add a ctor for creating them from CScripts (Cory Fields)
It runs sipas crawler, but rather than using its custom nameserver implementation it serves a generated zonefile via bind9. The zone always contains 25 IPv4 and 25 IPv6 peers. FWIW, the zone is secured using DNSSEC.
This makes it possible for a node with `-onlynet=tor` to bootstrap
itself.
It also adds the base infrastructure for adding IPv6 seed nodes.
Also represent IPv4 fixed seed addresses in 16-byte format.
4eedf4f make RandAddSeed() use OPENSSL_cleanse() (Philip Kaufmann)
6354935 move rand functions from util to new random.h/.cpp (Philip Kaufmann)
001a53d add GetRandBytes() as wrapper for RAND_bytes() (Philip Kaufmann)
This commit removes all the unnecessary dependencies (key, core,
netbase, sync, ...) from bitcoin-cli.
To do this it shards the chain parameters into BaseParams, which
contains just the RPC port and data directory (as used by utils and
bitcoin-cli) and Params, with the rest.
contrib/devtools/fix-copyright-headers.py script to be able to perform this maintenance task with ease during the rest of the year, every year. Modifications to contrib/devtools/README.md to document what fix-copyright-headers.py does.
Use misc methods of avoiding unnecesary header includes.
Replace int typedefs with int##_t from stdint.h.
Replace PRI64[xdu] with PRI[xdu]64 from inttypes.h.
Normalize QT_VERSION ifs where possible.
Resolve some indirect dependencies as direct ones.
Remove extern declarations from .cpp files.
