* Merge #8068: Compact Blocks
48efec8 Fix some minor compact block issues that came up in review (Matt Corallo)
ccd06b9 Elaborate bucket size math (Pieter Wuille)
0d4cb48 Use vTxHashes to optimize InitData significantly (Matt Corallo)
8119026 Provide a flat list of txid/terators to txn in CTxMemPool (Matt Corallo)
678ee97 Add BIP 152 to implemented BIPs list (Matt Corallo)
56ba516 Add reconstruction debug logging (Matt Corallo)
2f34a2e Get our "best three" peers to announce blocks using cmpctblocks (Matt Corallo)
927f8ee Add ability to fetch CNode by NodeId (Matt Corallo)
d25cd3e Add receiver-side protocol implementation for CMPCTBLOCK stuff (Matt Corallo)
9c837d5 Add sender-side protocol implementation for CMPCTBLOCK stuff (Matt Corallo)
00c4078 Add protocol messages for short-ids blocks (Matt Corallo)
e3b2222 Add some blockencodings tests (Matt Corallo)
f4f8f14 Add TestMemPoolEntryHelper::FromTx version for CTransaction (Matt Corallo)
85ad31e Add partial-block block encodings API (Matt Corallo)
5249dac Add COMPACTSIZE wrapper similar to VARINT for serialization (Matt Corallo)
cbda71c Move context-required checks from CheckBlockHeader to Contextual... (Matt Corallo)
7c29ec9 If AcceptBlockHeader returns true, pindex will be set. (Matt Corallo)
96806c3 Stop trimming when mapTx is empty (Pieter Wuille)
* Merge #8408: Prevent fingerprinting, disk-DoS with compact blocks
1d06e49 Ignore CMPCTBLOCK messages for pruned blocks (Suhas Daftuar)
1de2a46 Ignore GETBLOCKTXN requests for unknown blocks (Suhas Daftuar)
* Merge #8418: Add tests for compact blocks
45c7ddd Add p2p test for BIP 152 (compact blocks) (Suhas Daftuar)
9a22a6c Add support for compactblocks to mininode (Suhas Daftuar)
a8689fd Tests: refactor compact size serialization in mininode (Suhas Daftuar)
9c8593d Implement SipHash in Python (Pieter Wuille)
56c87e9 Allow changing BIP9 parameters on regtest (Suhas Daftuar)
* Merge #8505: Trivial: Fix typos in various files
1aacfc2 various typos (leijurv)
* Merge #8449: [Trivial] Do not shadow local variable, cleanup
a159f25 Remove redundand (and shadowing) declaration (Pavel Janík)
cce3024 Do not shadow local variable, cleanup (Pavel Janík)
* Merge #8739: [qa] Fix broken sendcmpct test in p2p-compactblocks.py
157254a Fix broken sendcmpct test in p2p-compactblocks.py (Suhas Daftuar)
* Merge #8854: [qa] Fix race condition in p2p-compactblocks test
b5fd666 [qa] Fix race condition in p2p-compactblocks test (Suhas Daftuar)
* Merge #8393: Support for compact blocks together with segwit
27acfc1 [qa] Update p2p-compactblocks.py for compactblocks v2 (Suhas Daftuar)
422fac6 [qa] Add support for compactblocks v2 to mininode (Suhas Daftuar)
f5b9b8f [qa] Fix bug in mininode witness deserialization (Suhas Daftuar)
6aa28ab Use cmpctblock type 2 for segwit-enabled transfer (Pieter Wuille)
be7555f Fix overly-prescriptive p2p-segwit test for new fetch logic (Matt Corallo)
06128da Make GetFetchFlags always request witness objects from witness peers (Matt Corallo)
* Merge #8882: [qa] Fix race conditions in p2p-compactblocks.py and sendheaders.py
b55d941 [qa] Fix race condition in sendheaders.py (Suhas Daftuar)
6976db2 [qa] Another attempt to fix race condition in p2p-compactblocks.py (Suhas Daftuar)
* Merge #8904: [qa] Fix compact block shortids for a test case
4cdece4 [qa] Fix compact block shortids for a test case (Dagur Valberg Johannsson)
* Merge #8637: Compact Block Tweaks (rebase of #8235)
3ac6de0 Align constant names for maximum compact block / blocktxn depth (Pieter Wuille)
b2e93a3 Add cmpctblock to debug help list (instagibbs)
fe998e9 More agressively filter compact block requests (Matt Corallo)
02a337d Dont remove a "preferred" cmpctblock peer if they provide a block (Matt Corallo)
* Merge #8975: Chainparams: Trivial: In AppInit2(), s/Params()/chainparams/
6f2f639 Chainparams: Trivial: In AppInit2(), s/Params()/chainparams/ (Jorge Timón)
* Merge #8968: Don't hold cs_main when calling ProcessNewBlock from a cmpctblock
72ca7d9 Don't hold cs_main when calling ProcessNewBlock from a cmpctblock (Matt Corallo)
* Merge #8995: Add missing cs_main lock to ::GETBLOCKTXN processing
dfe7906 Add missing cs_main lock to ::GETBLOCKTXN processing (Matt Corallo)
* Merge #8515: A few mempool removal optimizations
0334430 Add some missing includes (Pieter Wuille)
4100499 Return shared_ptr<CTransaction> from mempool removes (Pieter Wuille)
51f2783 Make removed and conflicted arguments optional to remove (Pieter Wuille)
f48211b Bypass removeRecursive in removeForReorg (Pieter Wuille)
* Merge #9026: Fix handling of invalid compact blocks
d4833ff Bump the protocol version to distinguish new banning behavior. (Suhas Daftuar)
88c3549 Fix compact block handling to not ban if block is invalid (Suhas Daftuar)
c93beac [qa] Test that invalid compactblocks don't result in ban (Suhas Daftuar)
* Merge #9039: Various serialization simplifcations and optimizations
d59a518 Use fixed preallocation instead of costly GetSerializeSize (Pieter Wuille)
25a211a Add optimized CSizeComputer serializers (Pieter Wuille)
a2929a2 Make CSerAction's ForRead() constexpr (Pieter Wuille)
a603925 Avoid -Wshadow errors (Pieter Wuille)
5284721 Get rid of nType and nVersion (Pieter Wuille)
657e05a Make GetSerializeSize a wrapper on top of CSizeComputer (Pieter Wuille)
fad9b66 Make nType and nVersion private and sometimes const (Pieter Wuille)
c2c5d42 Make streams' read and write return void (Pieter Wuille)
50e8a9c Remove unused ReadVersion and WriteVersion (Pieter Wuille)
* Merge #9058: Fixes for p2p-compactblocks.py test timeouts on travis (#8842)
dac53b5 Modify getblocktxn handler not to drop requests for old blocks (Russell Yanofsky)
55bfddc [qa] Fix stale data bug in test_compactblocks_not_at_tip (Russell Yanofsky)
47e9659 [qa] Fix bug in compactblocks v2 merge (Russell Yanofsky)
* Merge #9160: [trivial] Fix hungarian variable name
ec34648 [trivial] Fix hungarian variable name (Russell Yanofsky)
* Merge #9159: [qa] Wait for specific block announcement in p2p-compactblocks
dfa44d1 [qa] Wait for specific block announcement in p2p-compactblocks (Russell Yanofsky)
* Merge #9125: Make CBlock a vector of shared_ptr of CTransactions
b4e4ba4 Introduce convenience type CTransactionRef (Pieter Wuille)
1662b43 Make CBlock::vtx a vector of shared_ptr<CTransaction> (Pieter Wuille)
da60506 Add deserializing constructors to CTransaction and CMutableTransaction (Pieter Wuille)
0e85204 Add serialization for unique_ptr and shared_ptr (Pieter Wuille)
* Merge #8872: Remove block-request logic from INV message processing
037159c Remove block-request logic from INV message processing (Matt Corallo)
3451203 [qa] Respond to getheaders and do not assume a getdata on inv (Matt Corallo)
d768f15 [qa] Make comptool push blocks instead of relying on inv-fetch (mrbandrews)
* Merge #9199: Always drop the least preferred HB peer when adding a new one.
ca8549d Always drop the least preferred HB peer when adding a new one. (Gregory Maxwell)
* Merge #9233: Fix some typos
15fa95d Fix some typos (fsb4000)
* Merge #9260: Mrs Peacock in The Library with The Candlestick (killed main.{h,cpp})
76faa3c Rename the remaining main.{h,cpp} to validation.{h,cpp} (Matt Corallo)
e736772 Move network-msg-processing code out of main to its own file (Matt Corallo)
87c35f5 Remove orphan state wipe from UnloadBlockIndex. (Matt Corallo)
* Merge #9014: Fix block-connection performance regression
dd0df81 Document ConnectBlock connectTrace postconditions (Matt Corallo)
2d6e561 Switch pblock in ProcessNewBlock to a shared_ptr (Matt Corallo)
2736c44 Make the optional pblock in ActivateBestChain a shared_ptr (Matt Corallo)
ae4db44 Create a shared_ptr for the block we're connecting in ActivateBCS (Matt Corallo)
fd9d890 Keep blocks as shared_ptrs, instead of copying txn in ConnectTip (Matt Corallo)
6fdd43b Add struct to track block-connect-time-generated info for callbacks (Matt Corallo)
* Merge #9240: Remove txConflicted
a874ab5 remove internal tracking of mempool conflicts for reporting to wallet (Alex Morcos)
bf663f8 remove external usage of mempool conflict tracking (Alex Morcos)
* Merge #9344: Do not run functions with necessary side-effects in assert()
da9cdd2 Do not run functions with necessary side-effects in assert() (Gregory Maxwell)
* Merge #9273: Remove unused CDiskBlockPos* argument from ProcessNewBlock
a13fa4c Remove unused CDiskBlockPos* argument from ProcessNewBlock (Matt Corallo)
* Merge #9352: Attempt reconstruction from all compact block announcements
813ede9 [qa] Update compactblocks test for multi-peer reconstruction (Suhas Daftuar)
7017298 Allow compactblock reconstruction when block is in flight (Suhas Daftuar)
* Merge #9252: Release cs_main before calling ProcessNewBlock, or processing headers (cmpctblock handling)
bd02bdd Release cs_main before processing cmpctblock as header (Suhas Daftuar)
680b0c0 Release cs_main before calling ProcessNewBlock (cmpctblock handling) (Suhas Daftuar)
* Merge #9283: A few more CTransactionRef optimizations
91335ba Remove unused MakeTransactionRef overloads (Pieter Wuille)
6713f0f Make FillBlock consume txn_available to avoid shared_ptr copies (Pieter Wuille)
62607d7 Convert COrphanTx to keep a CTransactionRef (Pieter Wuille)
c44e4c4 Make AcceptToMemoryPool take CTransactionRef (Pieter Wuille)
* Merge #9375: Relay compact block messages prior to full block connection
02ee4eb Make most_recent_compact_block a pointer to a const (Matt Corallo)
73666ad Add comment to describe callers to ActivateBestChain (Matt Corallo)
962f7f0 Call ActivateBestChain without cs_main/with most_recent_block (Matt Corallo)
0df777d Use a temp pindex to avoid a const_cast in ProcessNewBlockHeaders (Matt Corallo)
c1ae4fc Avoid holding cs_most_recent_block while calling ReadBlockFromDisk (Matt Corallo)
9eb67f5 Ensure we meet the BIP 152 old-relay-types response requirements (Matt Corallo)
5749a85 Cache most-recently-connected compact block (Matt Corallo)
9eaec08 Cache most-recently-announced block's shared_ptr (Matt Corallo)
c802092 Relay compact block messages prior to full block connection (Matt Corallo)
6987219 Add a CValidationInterface::NewPoWValidBlock callback (Matt Corallo)
180586f Call AcceptBlock with the block's shared_ptr instead of CBlock& (Matt Corallo)
8baaba6 [qa] Avoid race in preciousblock test. (Matt Corallo)
9a0b2f4 [qa] Make compact blocks test construction using fetch methods (Matt Corallo)
8017547 Make CBlockIndex*es in net_processing const (Matt Corallo)
* Merge #9486: Make peer=%d log prints consistent
e6111b2 Make peer id logging consistent ("peer=%d" instead of "peer %d") (Matt Corallo)
* Merge #9400: Set peers as HB peers upon full block validation
d4781ac Set peers as HB peers upon full block validation (Gregory Sanders)
* Merge #9499: Use recent-rejects, orphans, and recently-replaced txn for compact-block-reconstruction
c594580 Add braces around AddToCompactExtraTransactions (Matt Corallo)
1ccfe9b Clarify comment about mempool/extra conflicts (Matt Corallo)
fac4c78 Make PartiallyDownloadedBlock::InitData's second param const (Matt Corallo)
b55b416 Add extra_count lower bound to compact reconstruction debug print (Matt Corallo)
863edb4 Consider all (<100k memusage) txn for compact-block-extra-txn cache (Matt Corallo)
7f8c8ca Consider all orphan txn for compact-block-extra-txn cache (Matt Corallo)
93380c5 Use replaced transactions in compact block reconstruction (Matt Corallo)
1531652 Keep shared_ptrs to recently-replaced txn for compact blocks (Matt Corallo)
edded80 Make ATMP optionally return the CTransactionRefs it replaced (Matt Corallo)
c735540 Move ORPHAN constants from validation.h to net_processing.h (Matt Corallo)
* Merge #9587: Do not shadow local variable named `tx`.
44f2baa Do not shadow local variable named `tx`. (Pavel Janík)
* Merge #9510: [trivial] Fix typos in comments
cc16d99 [trivial] Fix typos in comments (practicalswift)
* Merge #9604: [Trivial] add comment about setting peer as HB peer.
dd5b011 [Trivial] add comment about setting peer as HB peer. (John Newbery)
* Fix using of AcceptToMemoryPool in PrivateSend code
* add `override`
* fSupportsDesiredCmpctVersion
* bring back tx ressurection in DisconnectTip
* Fix delayed headers
* Remove unused CConnman::FindNode overload
* Fix typos and comments
* Fix minor code differences
* Don't use rejection cache for corrupted transactions
Partly based on https://github.com/bitcoin/bitcoin/pull/8525
* Backport missed cs_main locking changes
Missed from 58a215ce8c
* Backport missed comments and mapBlockSource.emplace call
Missed from two commits:
88c35491ab7c98ce584e
* Add CheckPeerHeaders() helper and check in (nCount == 0) too
* Drop nBudgetProposalEstablishingTime
* Refactor: replace `== COutPoint()` with `.IsNull()`
* Refactor: add `operator bool()` to CMasternodePing
* Refactor: actually use `operator bool()` for CPendingDsaRequest
* Refactor: fixing code style in TrafficGraphData
* Fix some comments and whitespaces
* Drop CGovernanceVote::GetTypeHash()
* Drop legacy X11 code
No longer used... if it ever was used at all.
* Move `<boost/foreach.hpp>` out of coins.h
* Simplify CMasternodeBlockPayees::GetRequiredPaymentsString()
Also less of boost::lexical_cast
* Drop CTxDSIn::nSentTimes
* Fix few warnings
* fix warning (timer)
* fix nit
d59a518 Use fixed preallocation instead of costly GetSerializeSize (Pieter Wuille)
25a211a Add optimized CSizeComputer serializers (Pieter Wuille)
a2929a2 Make CSerAction's ForRead() constexpr (Pieter Wuille)
a603925 Avoid -Wshadow errors (Pieter Wuille)
5284721 Get rid of nType and nVersion (Pieter Wuille)
657e05a Make GetSerializeSize a wrapper on top of CSizeComputer (Pieter Wuille)
fad9b66 Make nType and nVersion private and sometimes const (Pieter Wuille)
c2c5d42 Make streams' read and write return void (Pieter Wuille)
50e8a9c Remove unused ReadVersion and WriteVersion (Pieter Wuille)
1df3111 protocol.h: Make enums in GetDataMsg concrete values (Wladimir J. van der Laan)
2c09a52 protocol.h: Move MESSAGE_START_SIZE into CMessageHeader (Wladimir J. van der Laan)
f9bd92d version.h: s/shord/short/ in comment (Wladimir J. van der Laan)
* net: Split resolving out of CNetAddr
* net: Split resolving out of CService
* net: Split resolving out of CSubNet
* net: move CNetAddr/CService/CSubNet out of netbase
* net: narrow include scope after moving to netaddress
Net functionality is no longer needed for CAddress/CAddrman/etc. now that
CNetAddr/CService/CSubNet are dumb storage classes.
* net: Add direct tests for new CSubNet constructors
* net: Have LookupNumeric return a CService directly
Also fix up a few small issues:
- Lookup with "badip:port" now sets the port to 0
- Don't allow assert to have side-effects
* net: fixup nits
* net: Consistent checksum handling
In principle, the checksums of P2P packets are simply 4-byte blobs which
are the first four bytes of SHA256(SHA256(payload)).
Currently they are handled as little-endian 32-bit integers half of the
time, as blobs the other half, sometimes copying the one to the other,
resulting in somewhat confused code.
This PR changes the handling to be consistent both at packet creation
and receiving, making it (I think) easier to understand.
* net: Hardcode protocol sizes and use fixed-size types
The P2P network uses a fixed protocol, these sizes shouldn't change
based on what happens to be the architecture.
* Only store and connect to NODE_NETWORK nodes
* Keep addrman's nService bits consistent with outbound observations
* Verify that outbound connections have expected services
* Don't require services in -addnode
* Introduce enum ServiceFlags for service flags
* Introduce REQUIRED_SERVICES constant
Followup for the recent winner_block PR (#1028) and name confusion discovered during code review.
"Mostly" because also:
- CMasternodeBlockPayees::GetPayee -> CMasternodeBlockPayees::GetBestPayee which describes what this function is actually doing a bit better imo;
- fixing constructor CMasternodePaymentVote empty constructor
* store vote hashes in CMasternodePayee and use them in CMasternodePayments::Sync
* Request low data payment blocks in batches directly from some node instead of/after preliminary Sync.
* remove nVotes
- mn hash compatibility with 70103
- ignore some requests while syncing
- fix locking/initializing in sync
- do not ban for old mnw
- split budget/governance messages/invs
487674f Governance object/vote syncing fixes
- disable fCached values
- use two maps for storing votes, by hash and parent-hash/type
- disable part of flatdb.dump (still overwriting)
- fixed govobj/votes relay and sync
15821fe various fixes
- Added const where possible
- Uncommented sync block
- Protocol min 70201
- Fixed bug which flags invalid votes incorrectly
- Formatting
aa8fdd7 fix curly braces
d8e39b1 Fix GetTypeHash bug
- Should not collide based on the outcome
732a8a3 fixed mismatched index for vote map
If we want to be compatible to 12.0 on network layer we need them back in ppszTypeName[] to keep inv type consistent (allNetMessageTypes[] should not have them, we shouldn't accept non-implmented messages)
- Avoids string typos (by making the compiler check)
- Makes it easier to grep for handling/generation of a certain message type
- Refer directly to documentation by following the symbol in IDE
- Move list of valid message types to protocol.cpp:
protocol.cpp is a more appropriate place for this, and having
the array there makes it easier to keep things consistent.
Github-Pull: #7181
Rebased-From: 9bbe71b641
Lets nodes advertise that they offer bloom filter support explicitly.
The protocol version bump allows SPV nodes to assume that NODE_BLOOM is
set if NODE_NETWORK is set for pre-70011 nodes.
Also adds an option to turn bloom filter support off for nodes which
advertise a version number >= 70011. Nodes attempting to use bloom
filters on such protocol versions are banned, and a later upgade
should drop nodes of an older version which attempt to use bloom
filters.
Much code stolen from Peter Todd.
Implements BIP 111
- Added commands for using budgets "mnbudget" and "mnfinalbudget"
- Supports 100% decentralized budget control and view-only site with json meta data object
- Removed of reference node and replaced with decentralized quorums that pick the masternodes who get paid each block.
- Made a budgeting system, where masternodes can vote on individual budgets and the data is stored perminently on each clients computer
- Ensures ports remain open and client are responsive to IX requests.
- Completely 100% decentralized. This farms out the work of checking the masternode network to the masternode network. 1% of the network is determistically selected to check another 1% of the network each block. It takes six separate checks to deactivate a node, thus making it tamper proof.
- Nodes are kept in the masternode list if they fail enough PoSe checks to deactivate. They will continue to be checked until the operator fixes them. However they will not be paid while they're failing checks.
Thanks to Pieter Wuille for most of the work on this commit.
I did not fixup the overhaul commit, because a rebase conflicted
with "remove fields of ser_streamplaceholder".
I prefer not to risk making a mistake while resolving it.
The implementation of each class' serialization/deserialization is no longer
passed within a macro. The implementation now lies within a template of form:
template <typename T, typename Stream, typename Operation>
inline static size_t SerializationOp(T thisPtr, Stream& s, Operation ser_action, int nType, int nVersion) {
size_t nSerSize = 0;
/* CODE */
return nSerSize;
}
In cases when codepath should depend on whether or not we are just deserializing
(old fGetSize, fWrite, fRead flags) an additional clause can be used:
bool fRead = boost::is_same<Operation, CSerActionUnserialize>();
The IMPLEMENT_SERIALIZE macro will now be a freestanding clause added within
class' body (similiar to Qt's Q_OBJECT) to implement GetSerializeSize,
Serialize and Unserialize. These are now wrappers around
the "SerializationOp" template.
Use misc methods of avoiding unnecesary header includes.
Replace int typedefs with int##_t from stdint.h.
Replace PRI64[xdu] with PRI[xdu]64 from inttypes.h.
Normalize QT_VERSION ifs where possible.
Resolve some indirect dependencies as direct ones.
Remove extern declarations from .cpp files.
The new class is accessed via the Params() method and holds
most things that vary between main, test and regtest networks.
The regtest mode has two purposes, one is to run the
bitcoind/bitcoinj comparison tool which compares two separate
implementations of the Bitcoin protocol looking for divergence.
The other is that when run, you get a local node which can mine
a single block instantly, which is highly convenient for testing
apps during development as there's no need to wait 10 minutes for
a block on the testnet.
Design goals:
* Only keep a limited number of addresses around, so that addr.dat does not grow without bound.
* Keep the address tables in-memory, and occasionally write the table to addr.dat.
* Make sure no (localized) attacker can fill the entire table with his nodes/addresses.
See comments in addrman.h for more detailed information.
This introduces CNetAddr and CService, respectively wrapping an
(IPv6) IP address and an IP+port combination. This functionality used
to be part of CAddress, which also contains network flags and
connection attempt information. These extra fields are however not
always necessary.
These classes, along with logic for creating connections and doing
name lookups, are moved to netbase.{h,cpp}, which does not depend on
headers.h.
Furthermore, CNetAddr is mostly IPv6-ready, though IPv6
functionality is not yet enabled for the application itself.
This commit does *not* and should not modify *any* code, it only moves
it from net.h and splits it across protocol.cpp and protocol.hpp.
Signed-off-by: Giel van Schijndel <me@mortis.eu>
This commit does *not* and should not modify *any* code, it only moves
it from net.h and splits it across protocol.cpp and protocol.hpp.
Signed-off-by: Giel van Schijndel <me@mortis.eu>
Move CMessageHeader from net.h to protocol.[ch]pp, with the
implementation in the .cpp compilation unit (compiling once is enough).
This commit does *not* and should not modify *any* code, it only moves
it from net.h and splits it across protocol.cpp and protocol.hpp.
Indentation changes aside the closest thing to a modification of code is
the addition of the 'TODO' comment (the execution of which requires code
modifications and thus doesn't belong in this commit).
Signed-off-by: Giel van Schijndel <me@mortis.eu>
