Commit Graph

17693 Commits

Author SHA1 Message Date
Alexander Block
9e57c35e82 Don't even try to send/receive blocks for fMasternode connections
This avoids a lot of overhead in SendMessages, which is called very often.
2020-04-16 18:06:59 +02:00
Alexander Block
a808ff3330 Temporary hack for unnecessary invocations of Broadcast signal 2020-04-16 18:06:59 +02:00
Alexander Block
24ead62905 Only call SendMessages when ProcessMessages did some work or when some time passed
Invoking SendMessages for hundreds of peers while we know that there is no
work to do give a lot of overhead on the message handler thread.
2020-04-16 18:06:59 +02:00
Alexander Block
79f0bb1033
Fix crash in validateaddress (#3418)
The only reason this was not detected in integration tests is that
mapHdPubKeys was empty.
2020-04-16 18:01:40 +02:00
Alexander Block
d032d02f10
Merge pull request #3410 from codablock/pr_fix_tests
Multiple fixes for failing tests
2020-04-14 16:58:12 +02:00
Alexander Block
2a87aa9d4f Bump mocktime by 60 secs after calling setnetworkactive(True)
When we bumped before the call, it sometimes happened that in the following
loop one of the early nodes which got the network reactivated tried to
connect to one of the latter nodes with still disabled network. Such
connection attempts were dropped by the latter nodes and the retry
throttling then prevented the connection to be retried later. Doing the
bumping after setnetworkactive(True) should fix this.
2020-04-14 15:16:51 +02:00
Alexander Block
4118b77249 Better clarification for future conflict resolution 2020-04-14 15:16:51 +02:00
Alexander Block
d9911e51fc Bump mocktime while waiting for getblocktemplate 2020-04-14 15:16:51 +02:00
Wladimir J. van der Laan
d4065e79ac Merge #15324: test: Make bloom tests deterministic
fae169c95e09ddf068dcaebc8170c4f41b02cf66 test: Make bloom tests deterministic (MarcoFalke)

Pull request description:

  non-deterministic tests are useless, since a failing test could not be reproduced unless the seed is known.

Tree-SHA512: 4f634ff0c6adf663444f1ac504f6dbceaa46b78d697b840531977ba30006453ac559d5c21cc3eaef6d92b87d46008a34b0db6331ea3318001987fcfaec634acf
2020-04-14 15:16:51 +02:00
Alexander Block
8dc7ec7e56 Call set_node_times by default in bump_mocktime
And remove manual calls accordingly.
2020-04-14 15:16:51 +02:00
Alexander Block
d80212c47c Fix use of mocktime when calling PoissonNextSendInbound
This seems to have been wrong already in the backported PR.
2020-04-14 15:16:51 +02:00
Alexander Block
12ae7e171e Sync mempool before generating blocks 2020-04-14 15:16:51 +02:00
Alexander Block
36d909aa30 Fix compilation error 2020-04-14 13:32:50 +02:00
MarcoFalke
c52c1e1ce5 Merge #17243: p2p: add PoissonNextSend method that returns mockable time
1a8f0d5a74d5cc0000456932babf35301f5c1686 [tools] update nNextInvSend to use mockable time (Amiti Uttarwar)
4de630354fc6808b9b13b9e82da1a82f2f50f26a [tools] add PoissonNextSend method that returns mockable time (Amiti Uttarwar)

Pull request description:

  Introduce a Poisson helper method that wraps the existing method to return `std::chrono::duration` type, which is mockable.

  Needed for https://github.com/bitcoin/bitcoin/pull/16698.

ACKs for top commit:
  ajtowns:
    ACK 1a8f0d5a74d5cc0000456932babf35301f5c1686
  MarcoFalke:
    re-ACK 1a8f0d5a74d5cc0000456932babf35301f5c1686
  naumenkogs:
    ACK 1a8f0d5, and let's merge it and come back to it later.

Tree-SHA512: 7e2325d7c55fc0b4357cb86b83e0c218ba269f678c1786342d8bc380bfd9696373bc24ff124b9ff17a6e761c62b2b44ff5247c3911e2afdc7cc5c20417e8290b
2020-04-14 13:32:50 +02:00
Pieter Wuille
ea9f97c53b Merge #13298: Net: Bucketing INV delays (1 bucket) for incoming connections to hide tx time
d45b344ffd Bucket for inbound when scheduling invs to hide tx time (Gleb)

Pull request description:

  It has been brought up to my attention that current random delays mechanism (originally intended to obfuscate transaction metadata) allows to easily estimate the time a transaction was received by a node.

  It may be done by connecting multiple observer nodes to the same node. Each of those nodes will generate its own schedule of delays. Combined metadata regarding those events from different sources allows an observer to estimate transaction time.

  After this patch a spy won't gain additional information by just creating multiple connections to a target.

Tree-SHA512: c71dae5ff350b614cb40a8e201fd0562d3e03e3e72a5099718cd451f0d84c66d5e52bbaf0d5b4b75137514c8efdedcc6ef4df90142b360153f04ad0721545ab1
2020-04-14 13:32:50 +02:00
Alexander Block
8a0f702f39 Use verifiedProRegTxHash.IsNull() instead of fMasternode to check for MN connections
fMasternode connections don't even relay anything anymore.
2020-04-14 13:32:50 +02:00
PastaPastaPasta
8ade2f7957
Merge bitcoin#11281: Avoid permanent cs_main/cs_wallet lock during RescanFromTime (#3411)
* Merge #11281: Avoid permanent cs_main/cs_wallet lock during RescanFromTime

7f81250 Mention that other RPC calls report keys as "imported" while txns are still missing (Jonas Schnelli)
ccd8ef6 Reduce cs_main lock in ReadBlockFromDisk, only read GetBlockPos under the lock (Jonas Schnelli)
bc356b4 Make sure WalletRescanReserver has successfully reserved the rescan (Jonas Schnelli)
dbf8556 Add RAII wallet rescan reserver (Jonas Schnelli)
8d0b610 Avoid pemanent cs_main/cs_wallet lock during wallet rescans (Jonas Schnelli)

Pull request description:

  Right now, we are holding `cs_main`/`cs_wallet` during the whole rescan process (which can take a couple of hours).
  This was probably only done because of laziness and it is an important show-stopper for #11200 (GUI rescan abort).

Tree-SHA512: 0fc3f82d0ee9b2f013e6bacba8d59f7334306660cd676cd64c47bb305c4cb7c7a36219d6a6f76023b74e5fe87f3ab9fc7fd2439e939f71aef653fddb0a1e23b1
Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/wallet/rpcdump.cpp
#	src/wallet/test/wallet_tests.cpp
#	src/wallet/wallet.h

* remove "LearnAllRelatedScripts"

Signed-off-by: Pasta <pasta@dashboost.org>

* fix importelectrumwallet

Signed-off-by: Pasta <pasta@dashboost.org>

Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
2020-04-14 13:01:03 +03:00
Alexander Block
54198ce739
Backport bitcoin#15773: test: Add BitcoinTestFramework::sync_* methods (incomplete backport) (#3409)
* scripted-diff: Rename sync_blocks to send_blocks to avoid name collisions and confusion

-BEGIN VERIFY SCRIPT-
FILE_LIST=$(git grep -l 'def sync_blocks' ./test/functional/*.py)
sed -i -e 's/def sync_blocks/def send_blocks/g'   $FILE_LIST
sed -i -e 's/self.sync_blocks/self.send_blocks/g' $FILE_LIST
-END VERIFY SCRIPT-

* test: Pass at most one node group to sync_all

* test: Add BitcoinTestFramework::sync_* methods

* Move bumping of mocktime into BitcoinTestFramework.sync_mempool

Instead of doing it in every place it's needed

* scripted-diff: use self.sync_* methods

-BEGIN VERIFY SCRIPT-
sed -i -e 's/sync_blocks(self.nodes)/self.sync_blocks()/g'     $(git grep -l 'sync_blocks(self.nodes)'   ./test/functional/*.py)
sed -i -e 's/sync_mempools(self.nodes)/self.sync_mempools()/g' $(git grep -l 'sync_mempools(self.nodes)' ./test/functional/*.py)

sed -i -e 's/  sync_blocks(/  self.sync_blocks(/g'     $(git grep -l sync_blocks   ./test/functional/*.py)
sed -i -e 's/  sync_mempools(/  self.sync_mempools(/g' $(git grep -l sync_mempools ./test/functional/*.py)
-END VERIFY SCRIPT-

* Pass nodes into bump_mocktime

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-04-14 13:00:16 +03:00
PastaPastaPasta
93513d1296
Backport 10279 (#3394)
* Merge #10279: Add a CChainState class to validation.cpp to take another step towards clarifying internal interfaces

22fddde Avoid calling GetSerializeSize on each tx in a block if !fTxIndex (Matt Corallo)
2862aca Move some additional variables into CChainState private (Matt Corallo)
fd4d80a Create initial CChainState to hold chain state information (Matt Corallo)
e104f0f Move block writing out of AcceptBlock (Matt Corallo)
50701ba Move txindex/undo data disk location stuff out of ConnectBlock (Matt Corallo)
93a34cf Make DisconnectBlock unaware of where undo data resides on disk (Matt Corallo)

Pull request description:

  CChainState should eventually, essentially, be our exposed "libconsensus", but we're probably a few releases away, so the real goal is to clarify our internal interfaces. The main split was a big step, but validation.cpp is still a somewhat ranomly-mixed bag of functions that are pure functions which validate inputs (which should probably either merge with their callers or move into another file in consensus/), read/write data from disk, manipulate our current chain state (which moves into CChainState), and do mempool transaction validation.

  Obviously this is only a small step, but some effort is made to clean up what functions the functions in CChainState call, and obviously as things are added its easy to keep clear "CChainState::* cannot call anything except via callbacks through CValidationInterface, pure functions, or disk read/write things". Right now there are some glaring violations in mempool callbacks, and general flushing logic needs cleaning up (FlushStateToDisk maybe shouldnt be called, and there should be an API towards setDirtyBlockIndex, but I'll leave that for after @sipa's current changesets land).

Tree-SHA512: 69b8ec191b36b19c9492b4dee74c8057621fb6ec98ad219e8da0b2ed5c3ad711b10b5af9ff1117e8807ccf88918eeeab573be8448baecc9a59f099c53095985b

* fix

Signed-off-by: Pasta <pasta@dashboost.org>

* fix

Signed-off-by: Pasta <pasta@dashboost.org>

* More of 10279

Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-04-11 23:02:26 +03:00
UdjinM6
d2756fb210
Merge pull request #3395 from PastaPastaPasta/backports-0.16-pr16
Backports 0.16 pr16
2020-04-10 13:11:03 +03:00
Alexander Block
26dec64e79
Merge pull request #3397 from codablock/pr_backport_txrequests
Backport bitcoin#14897 and bitcoin#15834 and modify it to work with Dash messages
2020-04-09 00:12:44 +02:00
Alexander Block
410d4a68c2
Merge pull request #3396 from codablock/pr_backport_poll
Backport poll() code from Bitcoin
2020-04-09 00:12:20 +02:00
Alexander Block
9401366e8e Set wakeupSelectNeeded=true while poll() is running 2020-04-08 23:19:17 +02:00
Wladimir J. van der Laan
1524185462 Merge #13212: Net: Fixed a race condition when disabling the network.
793290f940a9af18d4f0292a263d976a066dff65 Net: Fixed a race condition when disabling the network. (lmanners)

Pull request description:

  This change addresses a race condition where setnetworkactive=false wouldn't always disconnect all peers.

  Before this change, the following could happen:
  1. Thread A -- Begins connecting to a node.
  2. Thread B -- Sets kNetworkActive=false and disconnects connected nodes.
  3. Thread A -- Finishes connecting and adds node to list of connected nodes.

  The node that was connected from Thread A remains connected and active,
  even though kNetworkActive=false.

  To fix the race, disconnections when kNetworkActive=false are now handled in the main network loop.

  fixes #13038

Tree-SHA512: 6d0b7a78ae956358e796efcc034cb532c2e0d824a52ae822a3899eefc7df76327519d1f2d77362c9fddf34ef860abd095d7490d7cc02d0ba7543bf1e8c8f2535
2020-04-08 22:46:34 +02:00
Alexander Block
f2364b7e5c Wait for ping/pong before starting testing in net.py 2020-04-08 22:43:56 +02:00
MarcoFalke
351ddf938d Merge #15069: test: Fix rpc_net.py "pong" race condition
de23739b22 test: Fix rpc_net.py "pong" race condition (Ben Woosley)

Pull request description:

  Prior to this change, the test fails with KeyError if pong has
  a zero value at the time this is called, as getpeerinfo's
  bytesrecv_per_msg result excludes zero-values.

  Combined these to a single wait_until as well, which will be a bit more
  forgiving re the timeout while still enforcing the same 2 seconds
  overall.

  https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/21310881#L62

Tree-SHA512: dc60f95a0e139c104fd81c8a7e0c9b3c25907de26c9d4e5976ae490e8ed5db0f0c492cd0e996ef6b5eb02cae82a62d4551ed36f95601871b19472050b3247bc0
2020-04-08 22:43:56 +02:00
MarcoFalke
62dc0bbdae Merge #16459: [qa] Fix race condition in example_test.py
d9ab0ffa386f2dd8d05852563d5011b3cb501df8 [qa] Fix race condition in example_test.py (Suhas Daftuar)

Pull request description:

  There's a race between sending a getdata for a bunch of blocks with the node receiving those blocks from a peer, which could cause test failure.  Fix this.

ACKs for top commit:
  MarcoFalke:
    ACK d9ab0ffa386f2dd8d05852563d5011b3cb501df8
  laanwj:
    ACK  d9ab0ffa386f2dd8d05852563d5011b3cb501df8
  promag:
    ACK d9ab0ffa386f2dd8d05852563d5011b3cb501df8.

Tree-SHA512: c891f209eb2492f44e47da52ee6df950ff874ae26d2739011aca940d1caff6cedbac032b6509adbed07044c14fd711ba9d4d0e35c0f70bb2691f2ea4a46672ed
2020-04-08 22:43:56 +02:00
Alexander Block
575d89fcf4 Introduce USE_WAKEUP_PIPE
Instead of checking for WIN32
2020-04-08 22:43:56 +02:00
Patrick Strateman
bc2c70f44a Increase maxconnections limit when using poll. 2020-04-08 22:43:56 +02:00
Alexander Block
efec6cc409 Implement wakeup pipe handling in poll based SocketEvents 2020-04-08 22:43:56 +02:00
Patrick Strateman
c0b10497bf Implement poll() on systems which support it properly.
This eliminates the restriction on maximum socket descriptor number.
2020-04-08 22:43:56 +02:00
Alexander Block
8481d6c110 --- Begin backporting bitcoin#14336 2020-04-08 22:43:56 +02:00
Patrick Strateman
7c022a20a1 Move SocketEvents logic to private method.
This separates the select() logic from the socket handling logic, setting up
for a switch to poll().
2020-04-08 22:43:56 +02:00
Patrick Strateman
ccc3f267ba Move GenerateSelectSet logic to private method.
This separates the socket event collection logic from the logic
deciding which events we're interested in at all.
2020-04-08 22:43:56 +02:00
Patrick Strateman
7a38df4f76 Introduce and use constant SELECT_TIMEOUT_MILLISECONDS. 2020-04-08 22:43:56 +02:00
Patrick Strateman
4e00e435e9 Move SocketHandler logic to private method. 2020-04-08 22:43:56 +02:00
Patrick Strateman
cbb6c9a4f9 Move InactivityCheck logic to private method. 2020-04-08 22:43:56 +02:00
Patrick Strateman
07f6d92e36 Move DisconnectNodes logic to private method. 2020-04-08 22:43:55 +02:00
Patrick Strateman
6eef194443 Move NotifyNumConnectionsChanged logic to private method. 2020-04-08 22:38:04 +02:00
Alexander Block
f9bf204829 --- Begin backporting bitcoin#14335 2020-04-08 22:38:04 +02:00
MarcoFalke
9591199629 Merge #13503: Document FreeBSD quirk. Fix FreeBSD build: Use std::min<int>(...) to allow for compilation under certain FreeBSD versions.
629a47a154 Document FreeBSD quirk. Fix FreeBSD build. (practicalswift)

Pull request description:

  * Document FreeBSD quirk.
  * Fix FreeBSD build: Cast to `int` to allow `std::min` to work under FreeBSD.

  Context: https://github.com/bitcoin/bitcoin/pull/9598#issuecomment-398353904

Tree-SHA512: 5ca7a5fa9e1f3efae241b9be64c9b019ec713c11dcc3edaaed383477ea48ac0dc82549ffebbe9069e8c3f6eff30acd6e4542b4aa31d307f022f4f51e5851a82c
2020-04-08 22:38:04 +02:00
Alexander Block
5cf417bc3a Bump mocktime while syncing mempools
Otherwise the inv/getdata logic won't work with inbound connections due to
the added delay of 2 seconds.
2020-04-08 22:28:39 +02:00
Alexander Block
e20c63f535
Merge pull request #3398 from codablock/pr_speedups
A few simple/trivial optimizations
2020-04-08 22:19:14 +02:00
Alexander Block
9bcdeaea57
Merge pull request #3399 from codablock/pr_speedups2
Avoid unnecessary processing/verification of reconstructed recovered signatures
2020-04-08 22:19:02 +02:00
Alexander Block
38556a3d49
Don't try to connect to masternodes that we already have a connection to (#3401) 2020-04-08 22:18:45 +02:00
Alexander Block
0e56e32c22
Add cache for CBlockTreeDB::HasTxIndex (#3402)
* Add cache for CBlockTreeDB::HasTxIndex

* Also update mapHasTxIndexCache in WriteTxIndex
2020-04-08 22:18:33 +02:00
Alexander Block
2dff0501e9
Remove semaphore for masternode connections (#3403)
We now have hundreds of MN connections due to the new intra quorum connections,
which means having a limiting semaphore here doesn't make sense anymore. It
actually causes DKG and signing failures as not all necessary connections
can be established.

Masternode connections are still implicitely limited as there is always
only a limited set of LLMQs active.
2020-04-08 22:18:21 +02:00
Peter Bushnell
c1d9dd553a
FindDevNetGenesisBlock remove unused arg (#3405) 2020-04-08 22:18:04 +02:00
Alexander Block
24b25e13da Add wait_func to sync_mempool 2020-04-08 22:13:54 +02:00
Wladimir J. van der Laan
6724a1ba64
Merge #13507: RPC: Fix parameter count check for importpubkey
3f72d04e29caf247a3fee92f86fe0d250d578914 Fix parameter count check for importpubkey. (Kristaps Kaupe)

Pull request description:

  Found this while working on #13464. Parameter count check for `importpubkey` was wrong.

Tree-SHA512: aba41b666c6493379f320be5e3e438a6cad1a96429102ff4428c092c48f29c2eead2195792c0b018296f20e1c42eb091dd5b9886c42cecbb1f0d03d5def14705
2020-04-08 11:46:27 -05:00