Commit Graph

22669 Commits

Author SHA1 Message Date
UdjinM6
d47af2c4bf
feat/fix: ScanQuorums improvements (#4945)
* feat: Revive quorum scan caching

* refactor: split quorumsCacheCs mutex into two

* fix: Avoid extra work in quorum scanning

Non-rotation quorums do not become rotation ones (anymore), use `useRotation` to pick the right method only. This brings CPU load for `d-isman` thread (while being idle) from ~5% down to ~1% on testnet for me.

* apply suggestions
2022-08-08 19:09:21 +03:00
UdjinM6
63d9c6abf7
fix: 4946 follow-up (#4957) 2022-08-08 19:06:10 +03:00
Konstantin Akimov
ae051bb6e0
Merge #17260: Split some CWallet functions into new LegacyScriptPubKeyMan (#4938)
* Move wallet enums to walletutil.h

* MOVEONLY: Move key handling code out of wallet to keyman file

Start moving wallet and ismine code to scriptpubkeyman.h, scriptpubkeyman.cpp

The easiest way to review this commit is to run:

   git log -p -n1 --color-moved=dimmed_zebra

And check that everything is a move (other than includes and copyrights comments).

This commit is move-only and doesn't change code or affect behavior.

* Refactor: Split up CWallet and LegacyScriptPubKeyMan and classes

This moves CWallet members and methods dealing with keys to a new
LegacyScriptPubKeyMan class, and updates calling code to reference the new
class instead of CWallet.

Most of the changes are simple text replacements and variable substitutions
easily verified with:

    git log -p -n1 -U0 --word-diff-regex=.

The only nontrivial chunk of code added is the new LegacyScriptPubKeyMan class
declaration, but this code isn't new and is just selectively copied and moved
from the previous CWallet class declaration. This can be verified with:

    git log -p -n1 --color-moved=dimmed_zebra src/wallet/scriptpubkeyman.h src/wallet/wallet.h

or

    git diff HEAD~1:src/wallet/wallet.h HEAD:src/wallet/scriptpubkeyman.h

This commit does not change behavior.

* Renamed classes in scriptpubkeyman

* Fixes for conflicts, compilation and linkage errors due to previous commits

* Reordered methods in scriptpubkeyman to make further backports easier

* Reordered methods in scriptpubkeyman to make further backports easier (part II)

* Remove HDChain copy from SigningProvider class

* fixes/suggestions

Co-authored-by: Andrew Chow <achow101-github@achow101.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-08-08 11:05:21 -05:00
UdjinM6
8882a7377e
feat: Avoid starting useless DKG threads on regular non-watching nodes (#4946)
* refactor: Initialize masternode mode related variables and objects earlier

* feat: Avoid starting useless DKG threads on regular nodes
2022-08-06 11:50:11 +03:00
UdjinM6
bdfa322f1f
fix: Restore caching in CalcCbTxMerkleRootMNList (#4951) 2022-08-06 11:45:02 +03:00
UdjinM6
abcc14b51e
feat: Implement caching in CalcCbTxMerkleRootQuorums (#4952) 2022-08-06 11:44:21 +03:00
Konstantin Akimov
7b9f0acbdd
fix: resolve some memory leaks (#4939)
- In wallet/rpcdump memory the leak would happen, if wallet is not correct JSON
 - In BLS the leak would happen, if object hasn't been pushed in queue by some reason. Need to remove temporary object also
2022-08-05 12:13:42 -05:00
UdjinM6
0835e4c307
fix(rpc): Fix off-by-1 in protx diff (#4950) 2022-08-05 11:35:46 +03:00
Kittywhiskers Van Gogh
bb7d6aed99
refactor(llmq): substitute memberless class llmq::CLLMQUtils with namespace llmq::utils (#4931)
* refactor(llmq): substitute memberless class llmq::CLLMQUtils with namespace llmq::utils

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* chore: mark functions internal to `llmq::utils` as `static`

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-08-02 12:14:25 -05:00
UdjinM6
c1a7a591c1
Merge pull request #4944 from PastaPastaPasta/18/rc12
[18.x] backport: rc12 backport and bump
2022-07-29 19:52:37 +03:00
pasta
ec4f9fd9b6
chore: bump to rc12 2022-07-29 11:06:21 -05:00
Odysseas Gabrielides
3ca5f25a38
Merge pull request #4942 from UdjinM6/fix_cqmbqr
fix(llmq): Calculate quorum members while not debugging llmq
2022-07-29 11:05:56 -05:00
Odysseas Gabrielides
7588c5a441
Merge pull request #4942 from UdjinM6/fix_cqmbqr
fix(llmq): Calculate quorum members while not debugging llmq
2022-07-28 13:43:02 +03:00
UdjinM6
39bf079f54
fix(llmq): Calculate quorum members while not debugging llmq
¯\_(ツ)_/¯
2022-07-27 23:34:29 +03:00
Konstantin Akimov
a6e9e50845
Merge bitcoin/bitcoin#23315: [22.x] build: explicitly disable libsecp256k1 openssl based tests (#4941)
e959b46aa933856e7636557f4ec6fce0efbc76aa build: explicitly disable libsecp256k1 openssl based tests (fanquake)

Pull request description:

  Backport of #23314

  These tests are failing when run against OpenSSL 3, and have been
  removed upstream, bitcoin-core/secp256k1#983, so
  disabled them for now to avoid `make check` failures.

  Note that this will also remove warning output from our build, due to
  the use of deprecated OpenSSL API functions. See #23048.

Top commit has no ACKs.

Tree-SHA512: ab3213dc82e7a64a005ce237710009bb447dee2702c4c02245e70df62063a00add73c4e80e9c619ce57345d4a2808fd4dc08e2e02a319b0f3d9285b8b0056599

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2022-07-27 21:25:14 +03:00
UdjinM6
b504b97a81
Merge pull request #4940 from PastaPastaPasta/backport-rc10
[v18.x] rc10 backports
2022-07-26 23:45:39 +03:00
pasta
d238549dd0
chore: bump rc to 11 2022-07-26 15:32:10 -05:00
Odysseas Gabrielides
b5ae437ca7
fix!: Fix on QuorumDataRequests and refactoring (#4937)
* qdata typo fixes and refactoring

* code style fix

* Add LOCK2 back

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-07-26 15:31:54 -05:00
Odysseas Gabrielides
6041e25722
fix!: Fix on QuorumDataRequests and refactoring (#4937)
* qdata typo fixes and refactoring

* code style fix

* Add LOCK2 back

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-07-26 15:31:03 -05:00
UdjinM6
1fa81c6a04
Merge pull request #4936 from PastaPastaPasta/v18.x-rc10
[V18.x] backport: rc10 backports
2022-07-26 02:48:10 +03:00
pasta
cd56e5002b
chore: bump rc to 10 2022-07-25 17:08:53 -05:00
Odysseas Gabrielides
71c0bbeaf2
Store QuorumDataRequests per {ProTx, quorumHash, llmqType} (#4935) 2022-07-25 17:08:30 -05:00
UdjinM6
b9dbd6da02
fix(dkg/net): Drop outdated connections to nodes that became masternodes recently (#4934)
* feat: switch nTimeFirstMessageReceived from microseconds to seconds

Was acting more like a bool until now, so nothing should change really. Align it with nTimeConnected.

* fix(dkg/net): Drop outdated connections to nodes that became masternodes recently

Such nodes won't be seen as masternodes by RelayInvToParticipants otherwise so no contributions will be sent to them when they are picked as relay members which in its turn may result in other nodes PoSe-punishing us.
2022-07-25 17:08:22 -05:00
UdjinM6
f7427d9c13
fix: Handle quorum watch connections correctly (#4933)
We add them via EnsureQuorumConnections+ThreadOpenMasternodeConnections so they are clearly masternode connections and they are dropped regularly which is annoying. But also, we don't want every masternode connection to be a qwatch one, we want only the ones we added via that algo.
2022-07-25 17:08:10 -05:00
UdjinM6
0ca9fc087c
fix(llmq): use keepOldConnections (#4932)
We must scan/cache keepOldConnections quorums or we won't be able to process sig shares signed in the 8 blocks window (signig offset) once new quorum(s) are mined.
2022-07-25 17:07:34 -05:00
UdjinM6
8a1a86b063
fix/test: Count MN connections properly, add more tests for getnetworkinfo (#4928)
* fix: Count MN connections properly

* tests: check extended connections info returned via getnetworkinfo
2022-07-25 17:07:25 -05:00
UdjinM6
e9500f597a
fix(llmq): use keepOldConnections (#4932)
We must scan/cache keepOldConnections quorums or we won't be able to process sig shares signed in the 8 blocks window (signig offset) once new quorum(s) are mined.
2022-07-26 00:39:04 +03:00
UdjinM6
5a5278d398
fix/test: Count MN connections properly, add more tests for getnetworkinfo (#4928)
* fix: Count MN connections properly

* tests: check extended connections info returned via getnetworkinfo
2022-07-26 00:38:07 +03:00
UdjinM6
17b15efd61
fix(dkg/net): Drop outdated connections to nodes that became masternodes recently (#4934)
* feat: switch nTimeFirstMessageReceived from microseconds to seconds

Was acting more like a bool until now, so nothing should change really. Align it with nTimeConnected.

* fix(dkg/net): Drop outdated connections to nodes that became masternodes recently

Such nodes won't be seen as masternodes by RelayInvToParticipants otherwise so no contributions will be sent to them when they are picked as relay members which in its turn may result in other nodes PoSe-punishing us.
2022-07-26 00:36:17 +03:00
UdjinM6
1f161e520d
fix: Handle quorum watch connections correctly (#4933)
We add them via EnsureQuorumConnections+ThreadOpenMasternodeConnections so they are clearly masternode connections and they are dropped regularly which is annoying. But also, we don't want every masternode connection to be a qwatch one, we want only the ones we added via that algo.
2022-07-26 00:35:45 +03:00
Odysseas Gabrielides
5dbf375d31
Store QuorumDataRequests per {ProTx, quorumHash, llmqType} (#4935) 2022-07-26 00:34:33 +03:00
PastaPastaPasta
4bdb9ac95d
refactor(llmq): misc refactors (#4922)
* refactor: remove c-style casts

* refactor: avoid logging initialization when not needed

* refactor: require ';' after LogPrintfFinalCommitment

* refactor: prefer lambda to std::bind

* refactor: explicit nullptr comparison

* refactor: compress a lock and set

* refactor: don't use else after return

* refactor: more const

* refactor: avoid implicit bool conversions

* refactor: add more const

* refactor: use structured binding

* refactor: compact lambda into inline

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* move scheduler_thread to initialization list

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-07-18 23:55:47 +03:00
UdjinM6
fa12c9685c
Merge pull request #4921 from PastaPastaPasta/v18.x-rc9
[v18.x] rc9 backports
2022-07-18 22:44:37 +03:00
pasta
3950dcd295
chore: bump to rc9 2022-07-18 14:31:52 -05:00
UdjinM6
9e22801878
feat(llmq): Ensure connections between IS quorums (#4917)
* fix(llmq): Ensure connections between quorums

Every masternode will now "watch" a single node from _every other_ quorum in addition to intra-quorum connections. This should make propagation of recsigs produced by one quorum to other quorums much more reliable.

* fix: Do this only for masternodes which participate in IS quorums

* refactor: rename `CQuorumManager::EnsureQuorumConnections` to better match the actual behaviour

(and avoid confusion with `CLLMQUtils::EnsureQuorumConnections`)

* refactor: move IS quorums watch logic into `CQuorumManager::CheckQuorumConnections`

avoid calling slow `ScanQuorums` (no caching atm) inside the loop

* tests: check that inter-quorum connections are added

* use `ranges::any_of`
2022-07-18 14:31:52 -05:00
UdjinM6
666859b47f
feat(llmq): Ensure connections between IS quorums (#4917)
* fix(llmq): Ensure connections between quorums

Every masternode will now "watch" a single node from _every other_ quorum in addition to intra-quorum connections. This should make propagation of recsigs produced by one quorum to other quorums much more reliable.

* fix: Do this only for masternodes which participate in IS quorums

* refactor: rename `CQuorumManager::EnsureQuorumConnections` to better match the actual behaviour

(and avoid confusion with `CLLMQUtils::EnsureQuorumConnections`)

* refactor: move IS quorums watch logic into `CQuorumManager::CheckQuorumConnections`

avoid calling slow `ScanQuorums` (no caching atm) inside the loop

* tests: check that inter-quorum connections are added

* use `ranges::any_of`
2022-07-18 22:26:51 +03:00
UdjinM6
0d67aa1915
fix: Pass KeyOriginInfo instead of CKeyMetadata to DeriveChildExtKey (#4918)
Resolves "hdchain -> wallet/walletdb -> ... -> hdchain" circular dependencies
2022-07-18 16:56:33 +03:00
PastaPastaPasta
30d6584cb6
Merge pull request #4886 from kittywhiskers/fuzz3
backport: bitcoin#18867, #19247, #19222, #18363, #18190, #18393, partial #18047, #18314, #19143 (fuzzing harness backports: part 3)
2022-07-17 16:34:37 -04:00
Kittywhiskers Van Gogh
38781fc114 merge bitcoin#17989: Add one specialized ProcessMessage(...) fuzzing binary per message type for optimal results when using coverage-guided fuzzing 2022-07-15 21:24:21 +05:30
Kittywhiskers Van Gogh
141c6e2920 merge bitcoin#19934: Add fuzzing harness for Keccak and SHA3_256 2022-07-15 21:24:21 +05:30
Kittywhiskers Van Gogh
e17767ccf1 merge bitcoin#19286: Add fuzzing harness for CHash{160,256}, C{HMAC_,}SHA{1,256,512}, CRIPEMD160, CSipHasher, etc. 2022-07-15 21:24:21 +05:30
Kittywhiskers Van Gogh
bf04694e73 merge bitcoin#18393: Don't assume presence of __builtin_mul_overflow in MultiplicationOverflow(...) fuzzing harness 2022-07-15 21:24:21 +05:30
Kittywhiskers Van Gogh
b50f00a2d5 merge bitcoin#18190: Add fuzzing harness for Golomb-Rice coding (GolombRiceEncode/GolombRiceDecode) 2022-07-15 21:24:21 +05:30
Kittywhiskers Van Gogh
93d3c26063 merge bitcoin#18363: Add fuzzing harness for HTTPRequest, libevent's evhttp and related functions 2022-07-15 21:24:21 +05:30
Kittywhiskers Van Gogh
01367c944e fix(fuzz): statically cast to consistent type to account for type similarity
AdditionOverflow is a template function that expects both inputs to explicitly be of the same type
2022-07-15 21:24:20 +05:30
Kittywhiskers Van Gogh
eb2fa81288 partial bitcoin#19143: Add fuzzing harnesses for CAutoFile, CBufferedFile, LoadExternalBlockFile and other FILE* consumers 2022-07-15 21:23:01 +05:30
Kittywhiskers Van Gogh
da690542f4 merge bitcoin#19222: Add fuzzing harness for BanMan 2022-07-15 21:09:53 +05:30
Kittywhiskers Van Gogh
cd9b83c15e merge bitcoin#19247: Add fuzzing harness for {Read,Write}{LE,BE}{16,32,64} 2022-07-15 21:09:53 +05:30
Kittywhiskers Van Gogh
a7109d8724 partial bitcoin#18314: Add deserialization fuzzing of SnapshotMetadata (utxo_snapshot). Increase fuzzing coverage 2022-07-15 21:09:53 +05:30
Kittywhiskers Van Gogh
0d5a7929ac partial bitcoin#18047: Add basic fuzzing harness for CNetAddr/CService/CSubNet related functions (netaddress.h) 2022-07-15 21:09:53 +05:30