Commit Graph

18941 Commits

Author SHA1 Message Date
UdjinM6
c631d042c7
llmq: Use thread pool for quorum cache and data recovery threads (#4008)
* llmq: Use thread pool for quorum cache and data recovery threads

Move quorum data and cache thread handling into CQuorumManager.

* llmq: Fix explicit capture list

* llmq: Directly push the lamdas to workerPool

Co-authored-by: xdustinface <xdustinfacex@gmail.com>
2021-02-20 15:43:43 -05:00
PastaPastaPasta
10b0c70079
Merge pull request #4003 from xdustinface/pr-backport-some-dependencies
depends|build: Upgrade boost to 1.70, fix macOS (10.15) depends build
2021-02-20 14:40:18 -06:00
UdjinM6
44e9ba50cc
instantsend: Ignore IS while reindexing/loading blocks from file, bail out early when IS is off (#3985)
* instantsend: Bail out early on disabled IS in more places

* instantsend: Disable InstantSend while reindexing and importing blocks

* Drop extra checks in GetInstantSendLockHashByTxid, GetInstantSendLockByInput and GetInstantSendLockByHash
2021-02-20 15:37:25 -05:00
dustinface
2ec8f21c4a
gitlab: Typo nowalet -> nowallet (#4004) 2021-02-17 21:13:33 +01:00
UdjinM6
1a61c5d9aa
wallet: Always skip conflicting txes in SelectCoinsGroupedByAddresses (#4002)
* wallet: Always skip conflicting txes in SelectCoinsGroupedByAddresses

* tests: add tests for SelectCoinsGroupedByAddresses
2021-02-17 12:02:17 -05:00
UdjinM6
5d9c9089bb
instantsend: Introduce KnownInstantSendLock, optimize ProcessInstantSendLock a bit (#3984)
* instantsend: Introduce (and use) KnownInstantSendLock

Check islock hash against both current and archived islocks

* instantsend: Optimize ProcessInstantSendLock, check for known islocks first

Two reasons:
1. GetTransaction is a potentially much heavier one
2. Doesn't make much sense to look for a tx if we have a known islock already

* Drop extra pendingInstantSendLocks.count(hash)

* Check pindexMined != nullptr

* Update src/llmq/quorums_instantsend.cpp

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update src/llmq/quorums_instantsend.h

Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
2021-02-17 10:59:16 -05:00
Wladimir J. van der Laan
53ad08710c Merge #20195: build: fix mutex detection when building bdb on macOS
d0a829e9632379e42f0be5c554e3b692f0d14a95 build: fix mutex detection when building bdb on macOS (fanquake)

Pull request description:

  Starting with the Apple Clang shipped with Xcode 12, [Apple has enabled -Werror=implicit-function-declaration by default](https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes):
  > Clang now reports an error when you use a function without an explicit declaration when building C or Objective-C code for macOS (-Werror=implicit-function-declaration flag is on). This additional error detection unifies Clang’s behavior for iOS/tvOS and macOS 64-bit targets for this diagnostic. (49917738)

  This causes bdbs mutex detection to fail when building on macOS (not cross-compiling):
  ```bash
  checking for mutexes... UNIX/fcntl
  configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS PLATFORM.
  configure: error: Unable to find a mutex implementation
  ```

  as previously emitted warnings are being turned into errors. i.e:
  ```bash
  configure:18704: checking for mutexes
  configure:18815: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -mmacosx-version-min=10.12 --sysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o conftest -pipe -O2  -I/Users/michael/github/fanquake-bitcoin/depends/x86_64-apple-darwin19.6.0/include -L/Users/michael/github/fanquake-bitcoin/depends/x86_64-apple-darwin19.6.0/lib conftest.c  -lpthread >&5
  conftest.c:46:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
  main() {
  ^
  conftest.c:51:2: error: implicitly declaring library function 'exit' with type 'void (int) __attribute__((noreturn))' [-Werror,-Wimplicit-function-declaration]
          exit (
          ^
  conftest.c:51:2: note: include the header <stdlib.h> or explicitly provide a declaration for 'exit'
  1 warning and 1 error generated.
  ```

  Append `-Wno-error=implicit-function-declaration` to `cflags` so that `-Wimplicit-function-declaration` [returns to being a warning](https://clang.llvm.org/docs/UsersManual.html#cmdoption-wno-error), and the configure checks succeed.

  Fixes #19411.

ACKs for top commit:
  laanwj:
    Code review ACK d0a829e9632379e42f0be5c554e3b692f0d14a95

Tree-SHA512: 7813005b1fc0b370f843b6c0672acab32c999416e92c3f02b75d866e9c7aa41fe5822704fc74de6b65f0d7d94f2cdd05cc7c3ee83295ff1ecbc71d8492b9a2bf
2021-02-17 15:21:53 +01:00
Wladimir J. van der Laan
badc91f98e Merge #17231: depends: fix boost mac cross build with clang 9+
50037e97d11356218c4b36767232e47b74742b0b depends: fix boost mac cross build with clang 9+ (Cory Fields)

Pull request description:

  The ancient "darwin-4.9.1" profile has long been used to match against clang, which prior to version 9, reported 4.9.1 as its version when invoking "clang++ -dumpversion". Presumably this was a historical compatibility quirk related to Apple's switch from gcc to clang.

  This was "fixed" in clang 9.0, so that -dumpversion reports the real version. Unfortunately that had the side-effect of breaking the (brittle) boost compiler detection.

  Move to the seemingly more-correct "clang-darwin" profile, which passes the checks and builds correctly.

  Also switch to using ar rather than libtool for archiving, as it's what the clang-darwin profile expects to be using.

  Note that because this is using a different profile, some of the final command-line arguments end up changing. Those changes look sane at a glance.

ACKs for top commit:
  fanquake:
    ACK 50037e97d11356218c4b36767232e47b74742b0b - tested on on macOS, will wait for the gitian build.

Tree-SHA512: eac1f353513a445add6fbece7fc78dd3dbdde5e2219bfb7739b82f40bb14de449667a94d2e303d43c67d9b38e7ceb0ba5f0d8fe20b40be2017b1ca0875467c2c
2021-02-17 14:53:57 +01:00
Wladimir J. van der Laan
d390a1412e Merge #15461: [depends] update to Boost 1.70
2620e24b83d16bf0f2bfe360dee1e98b4be59ca5 [depends] boost: update to 1.70 (Sjors Provoost)

Pull request description:

  Version [1.70](https://www.boost.org/users/history/version_1_70_0.html) is most recent.

  Versions needed for:
  * 1.66: #12557: fixes the single arm64 configuration ([06ee5b5](06ee5b54ef))

ACKs for commit 2620e2:

Tree-SHA512: 6e0174f1d92c2c24314c0689d4809e048914f8f42d17aa73799f5ee232169e0dd0ed71f5f973903c44c08309f2837c629c493f15e5c31ec6c7bd1daae5f3b25f
2021-02-17 14:22:09 +01:00
UdjinM6
77e9a4a83b
chainlocks: Relay clsig inv only after bestChainLock(Hash) are assigned (#3990)
Fixes a race condidion when we might reply with `notfound` to other nodes because we might still be waiting for locks here.
2021-02-16 16:54:19 +01:00
UdjinM6
ebea362d06
qt: Decouple transaction label updates from transaction status updates (#3994)
* qt: Decouple tx label updates from tx status updates

* Update src/qt/transactionrecord.cpp

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2021-02-14 17:27:28 -06:00
dustinface
6ec8b9a5c8
llmq: Some refactoring in CChainLocksHandler::ProcessNewChainLock (#3976)
* llmq: Drop InternalHasConflictingChainLock in ProcessNewChainLock

* llmq: Directly use clsig.blockHash instead of copying it into msgHash

* llmq: Reuse CInv(MSG_CLSIG, hash)

* llmq: Add const in two places
2021-02-14 16:12:40 -06:00
UdjinM6
054be30344
llmq: Sleep less in quorum data recovery threads (#3989)
Speeds up feature_llmq_data_recovery.py test ~25% and makes sense overall imo.
2021-02-14 16:11:20 -06:00
UdjinM6
f5e74c8a10
instantsend: Mark blocks as conflicting when there is a CL vs IS conflict (#3987)
* instantsend: Mark a block with IS-locks which conflict with txes in a CL-ed block as conflicting and not as invalid

* tests: Tweak feature_llmq_is_cl_conflicts.py to test CL overriding a block with conflicting IS-locks
2021-02-14 16:09:21 -06:00
UdjinM6
ee3e175722
instantsend: Refactor various parts (#3983)
* refactor: Add `const` qualifier to various InstantSend related methods

* refactor: Drop ProcessNewTransaction

* refactor: Simplify GetInstantSendLock(*) methods

* refactor: Check height inside Remove(Archived/Confirmed)InstantSendLocks

* refactor: Squash two loops in AddNonLockedTx into one

`!info.tx == res.second` is always `true`

* Update src/llmq/quorums_instantsend.cpp

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update src/llmq/quorums_instantsend.cpp

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2021-02-14 16:07:54 -06:00
UdjinM6
5ce5e35580
qt: Change Node Type to recognize 3 types: Regular, Masternode and Verified Masternode (#3995) 2021-02-11 11:36:54 -05:00
UdjinM6
260cc09de2
qt: Fix labels in transaction list (#3993)
* qt: Fix labels in transaction list

The issue was introduced in #3682

* qt: Always use labels from TransactionStatus for transaction list

Missed this in #3155

* Update src/qt/transactiontablemodel.cpp

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2021-02-11 11:36:09 -05:00
UdjinM6
899c124c76
instantsend|sigs: Sleep when there is no more work (#3988)
* instantsend|sigs: Sleep when there is no more work

Instead of sleeping only when no work has been done.
Avoids useless cycles, improves batching.

* llmq: Add and use nMaxBatchSize

* llmq: Compare to what we got in return, not what we verified at the end

It might happen that we get 32 pending but do only verify less than 32 and in this case we would assume there is no more work but it could still be more in the pipeline from my understanding.

* llmq: Rename more_work -> fMoreWork

* llmq: Be consistent with the other fMoreWork initialization

Co-authored-by: xdustinface <xdustinfacex@gmail.com>
2021-02-11 11:32:30 -05:00
thephez
317353deb3
docs: cli option help additions (#3981)
* docs: add help messages for missing devnet options

* docs: add help for llmq recovery

* docs: add help for regtest llmq

* docs: add help for sporkkey

* Apply suggestions from code review

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2021-02-11 11:22:54 -05:00
UdjinM6
67145ccb57
Merge pull request #3977 from PastaPastaPasta/backports-0.17-pr26
Backports 0.17 pr26
2021-02-08 18:34:45 +03:00
UdjinM6
f22d5e2aaa
tests: Apply --timeoutscale option to rpc timeouts (#3982) 2021-02-08 14:39:05 +01:00
UdjinM6
e029f24c1c Fix some circular dependencies, update EXPECTED_CIRCULAR_DEPENDENCIES to make linter happy 2021-02-04 14:58:20 -06:00
MarcoFalke
e6bbb2f2b3 Merge #13924: tests: Simplify comparison in rpc_blockchain.py
1f87c372b5 Simplify comparison in rpc_blockchain.py. (Daniel Kraft)

Pull request description:

  The test for `gettxoutsetinfo` in `rpc_blockchain.py` verifies that the result is the same as before after invalidating and reconsidering a block.  The comparison has to exclude the `disk_size` field, though, as it is not deterministic.

  Instead of comparing all the other fields for equality, this change explicitly removes the `disk_size` field and then compares the full objects.  This makes the intent more explicit (compare everything except for `disk_size`, not compare just a given list of fields) and also the code simpler.

Tree-SHA512: 3c376a8836b62988fb2f0117c9ca65de64a33bf3cd4980a123de30bf5e7b7a48eda477b25e03d672ff076e205c698e83432469156caa0f0f3ebbb0480f0dd77d
2021-02-04 14:58:13 -06:00
Wladimir J. van der Laan
2f301fdb09 Merge #13925: Merge leveldb subtree
ec749b1bcdf2483b642fb51d635800e272c68ba6 Squashed 'src/leveldb/' changes from 64052c76c5..524b7e36a8 (MarcoFalke)

Pull request description:

  For review:

  ```sh
  git fetch https://github.com/bitcoin-core/leveldb
  ./test/lint/git-subtree-check.sh src/leveldb
  ```

  Closes #13860

Tree-SHA512: 9d13384fe35e7144b4a7fca57efe77b0cc5295952da4a397e4c6d8aa3f8043d5113fccedd3ae1dcaa3d2649e732e5f57a71504847946e055aa4dc8c3780e29fc
2021-02-04 14:58:13 -06:00
MarcoFalke
b08e60e529 Merge #13876: wallet: Catch filesystem_error and raise InitError
fa8527ffec wallet: Catch filesystem_error and raise InitError (MarcoFalke)

Pull request description:

  Fixes #13754 by restoring the previous behaviour

Tree-SHA512: f64052e89f6b332be395df2a5ea6f227c213fa2f38e415e83f30a4fad0938e947e5cddff7902368a43c07be135955a31b90f7eac5a46875c58d54ea95f87f6e6
2021-02-04 14:58:13 -06:00
Wladimir J. van der Laan
a9ba11c343 Merge #13717: docs: Link to python style guidelines from developer notes
a9e898a4ad547ad344671db2b942b99925f0c732 docs: Link to python style guidelines from developer notes (Mason Simon)

Pull request description:

  Initially I moved the python style guidelines from the functional test README, but some of the python rules are test-specific, and most of the developer notes doc is C++ centric, so just dropping a link seemed better.

Tree-SHA512: 9d4d5cc45526319a118595d90fcfad2c9aced22007aa096d8af04ba1b963312822804f4c15b0b227d66af49565034437691b7760e7ff6d1e3f8b10b898906362
2021-02-04 14:58:13 -06:00
MarcoFalke
4a7387fba9 Merge #13786: refactor: Avoid locking tx pool cs thrice
fa5ed4f8d2 refactor: Avoid locking tx pool cs thrice (MarcoFalke)

Pull request description:

  `addUnchecked` is (outside the tests) only called by ATMP, which already takes the tx pool read lock. So locking it twice more in both `addUnchecked` methods seems redundant.

  Similarly `CalculateMemPoolAncestors` is (beside once in the wallet) only called in contexts, where the tx pool lock is already taken. So remove the lock there as well.

Tree-SHA512: fcf603b570da0fc529fe6db8add218663eae52845510732bee0d4611263d2429d3d3c9c8ae68493d67287d13504500ed51905ccbe711eb15a0af3b019edad543
2021-02-04 14:58:13 -06:00
pasta
cc9c47071a fix
Signed-off-by: pasta <pasta@dashboost.org>
2021-02-04 14:58:13 -06:00
Wladimir J. van der Laan
a8863dc3b5 Merge #12634: [refactor] Make TransactionWithinChainLimit more flexible
f77e1d34fd5f17304ce319b5f962b8005592501a test: Add MempoolAncestryTests (Karl-Johan Alm)
a08d76bcfee6f563a268933357931abe32060668 mempool: Calculate descendant maximum thoroughly (Karl-Johan Alm)
6d3568371eb2559d65a3e2334252d36a262319e8 wallet: Switch to using ancestor/descendant limits (Karl-Johan Alm)
6888195b062c8c58dd776fd10b44b25554eb1f15 wallet: Strictly greater than for ancestor caps (Karl-Johan Alm)
322b12ac4e0a8c892e81a760ff7225619248b74f Remove deprecated TransactionWithinChainLimit (Karl-Johan Alm)
47847515473b054929af0c8de3d54b6672500cab Switch to GetTransactionAncestry() in OutputEligibleForSpending (Karl-Johan Alm)
475a385a80198a46a6d99846f99b968f04e9b470 Add GetTransactionAncestry to CTxMemPool for general purpose chain limit checking (Karl-Johan Alm)
46847d69d2c1cc908fd779daac7884e365955dbd mempool: Fix max descendants check (Karl-Johan Alm)
b9ef21dd727dde33f5bd3c33226b05d07eb12aac mempool: Add explicit max_descendants (Karl-Johan Alm)

Pull request description:

  Currently, `TransactionWithinChainLimit` is restricted to single-output use, and needs to be called every time for different limits. If it is replaced with a chain limit value calculator, that can be called once and reused, and is generally more flexible (see e.g. #12257).

  Update: this PR now corrects usage of max ancestors / max descendants, including calculating the correct max descendant value, as advertised for the two limits.

  ~~This change also makes `nMaxAncestors` signed, as the replacement method will return `-1` for "not in the mempool", which is different from "0", which means "no ancestors/descendants in mempool".~~

  ~~This is a subset of #12257.~~

Tree-SHA512: aa59c849360542362b3126c0e29d44d3d58f11898e277d38c034dc4b86a5b4500f77ac61767599ce878c876b5c446fec9c02699797eb2fa41e530ec863a00cf9
2021-02-04 14:58:13 -06:00
MarcoFalke
cab82cfe18 Merge #13691: Remove redundant variables, statements and forward declarations
3dee4cc509 Remove redundant statement (practicalswift)
99be644966 Remove redundant unused variables (practicalswift)
66ed242343 Remove redundant forward declaration (practicalswift)

Pull request description:

  Remove redundant …
  * ~access modifiers,~
  * forward declarations,
  * unused variables,
  * statements, and
  * ~return types from lambdas.~

Tree-SHA512: 328bb7d9c45398e44ecbee32095b6376879470dfddbc2180e037620d8390d524b51d7fda112fd58a078715e04432b24dd6998a2459f3550aa0498aa68de866d4
2021-02-04 14:58:13 -06:00
MarcoFalke
7b8eb59178 Merge #13683: wallet: Introduce assertion to document the assumption that cache and cache_used are always set in tandem
d06330396f wallet: Avoid potential null pointer dereference in CWalletTx::GetAvailableCredit(...) (practicalswift)

Pull request description:

  Avoid potential null pointer dereference in `CWalletTx::GetAvailableCredit(...)`.

  Introduced in 4279da47855ec776f8d57c6579fe89afc9cbe8c1.

Tree-SHA512: 92d1da9682f0bab11f6f96395ca30549331b0a056cbceb7e1a7f98b3d17d10082aaeed0907cafd46f4164b0e0f4b77e01f78bfd7d24e0503a66c6942ae842aa5
2021-02-04 14:58:13 -06:00
Wladimir J. van der Laan
cd59867e7b Merge #13695: lint: Add linter for circular dependencies
5c613aadd64453c75cb2373c6fcc1326c3cf0b7a lint: Add linter for circular dependencies (Ben Woosley)

Pull request description:

  Protects against added circular depencies, makes it explicit in the
  code when circular dependencies have been removed.

  Modeled after EXPECTED_BOOST_INCLUDES in lint-includes.sh

  Example output:
  ```
  $ test/lint/lint-circular-dependencies.sh
  A new circular dependency in the form of "qt/paymentserver -> qt/walletmodel -> qt/paymentserver" appears to have been introduced.

  $ echo $?
  1
  $ test/lint/lint-circular-dependencies.sh
  Good job! The circular dependency "Fake" is no longer present.
  Please remove it from EXPECTED_CIRCULAR_DEPENDENCIES in test/lint/lint-circular-dependencies.sh
  to make sure this circular dependency is not accidentally reintroduced.

  $ echo $?
  1
  $ test/lint/lint-circular-dependencies.sh
  $ echo $?
  0

  ```

Tree-SHA512: 4519434de29f6d50859daed1480e531c01c1cdbc3f0a5f093251daf62ae2b5b9073fb274b86f541a985e06837aa1165b76558c5f35fb51a759d72e83f1b61e44
2021-02-04 14:58:13 -06:00
UdjinM6
4c1a04a254
llmq: Avoid writing commitments to evodb and altering caches when all we want is to check block candidate validity (#3980)
* llmq: Avoid writing commitments to evodb and altering caches when all we want is to check block candidate validity

* tests: call `getblocktemplate` to trigger `CreateNewBlock` before quorum commitment is mined
2021-02-03 22:01:26 -05:00
dustinface
bbf8eac53f
test: Wait for recovered sig in rpc_verifyislock.py (#3975)
Should fix failures like https://gitlab.com/dashpay/dash/-/jobs/997361410.

Looks like the reason for the failures is queuing recovered sigs here for processing 2493d851a7/src/llmq/quorums_instantsend.cpp (L876).
2021-02-03 11:09:08 -05:00
UdjinM6
2493d851a7
tests: fix feature_pruning.py (#3967)
* tests: postpone dip3 and dip8 activation in feature_pruning.py

* tests: Mine one block before restart to avoid automatic recovery from forks
2021-02-01 11:17:29 -05:00
UdjinM6
7188004905
Fix two issues with ResetBlockFailureFlags, add tests (#3965)
* Merge #13199: Bugfix: ensure consistency of m_failed_blocks after reconsiderblock

11fa6bb66e Bugfix: ensure consistency of m_failed_blocks after reconsiderblock (Suhas Daftuar)

Pull request description:

  This was introduced in 015a5258ad and could cause a node to crash (due to assertion failure) when using the `reconsiderblock` rpc.

Tree-SHA512: 820dcd761bf983e36f5d0f16777ed75c833daaf62a6b3a4dbd17f6caaf9287223e3a202d06540ac62f8ba72926b73b0873bb76c6273ddcb19d9408f4c1cd325e

* bugfix: Mark all nearest BLOCK_FAILED_CHILD descendants (if any) as BLOCK_FAILED_VALID while removing the invalidity flag from all ancestors in ResetBlockFailureFlags

Fixes `Assertion failed: ((pindex->nStatus & BLOCK_FAILED_MASK) == 0), function CheckBlockIndex`

* tests: Make sure ResetBlockFailureFlags does the job correctly

* Wait for the expected block height, check the final chain tip hash

Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2021-02-01 11:15:46 -05:00
dustinface
d6c6174958
llmq|init|test: Implement DKG data recovery / quorum verification vector sync (#3964)
* llmq: Implement automated DKG recovery threads

* llmq: Implement quorum verification vector sync

* init: Validiate quorum data recovery related command line parameter

* test: Add quorum_data_request_timeout_seconds in DashTestFramework

* test: Test quorum data recovery in feature_llmq_data_recovery.py

* test: Add feature_llmq_data_recovery.py to BASE_SCRIPTS

* test: Fix quorum_data_request_expiration_timeout in wait_for_quorum_data

* test: Always test the existence of secretKeyShare in test_mn_quorum_data

With this change it also validates that "secretKeyShare" is not in `quorum_info` if its not expected to be in there. Before this was basically just not tested.

* llmq|test: Use bool as argument type for -llmq-data-recovery

* llmq: Always set nTimeLastSuccess to 0

* test: Set -llmq-data-recovery=0 in p2p_quorum_data.py

* test: Simplify test_mns

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* refactor: pass CQuorumCPtr to StartQuorumDataRecoveryThread

* test: Fix thread name in comment

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-02-01 17:10:19 +01:00
dustinface
d9105d8de1
llmq|rpc: Avoid some quorum data related copying (#3974)
* llmq|rpc: Return const reference in CQuorum::GetSkShare

* llmq: Refactor BuildQuorumContributions
2021-02-01 10:24:51 -05:00
dustinface
edc286be46
governance: Fix signature check in CGovernanceObject::Sign (#3973) 2021-02-01 10:23:58 -05:00
UdjinM6
7c5710f64b
chainlocks: Fix potential deadlock (#3972)
Should fix this:
```
Previous lock order was:
 pnode->cs_sendProcessing  net.cpp:2854
 cs_main  net_processing.cpp:3946 (TRY)
 mempool.cs  net_processing.cpp:4183
 (1) pto->cs_inventory  net_processing.cpp:4183
 pto->cs_filter  net_processing.cpp:4233
 (2) cs  llmq/quorums_chainlocks.cpp:87
Current lock order is:
 cs_main  llmq/quorums_chainlocks.cpp:138
 (2) cs  llmq/quorums_chainlocks.cpp:138
 cs_vNodes  net.cpp:3662
 (1) cs_inventory  ./net.h:1085
```
2021-02-01 10:21:52 -05:00
UdjinM6
7099dea2e4
Merge pull request #3939 from PastaPastaPasta/backport-addrv2
Backport partial addrv2 support
2021-01-29 11:39:00 +03:00
UdjinM6
3c768739fc
chainlocks: Send an inv for the best ChainLock we have when replying to mempool p2p message (#3962) 2021-01-28 21:55:58 -05:00
UdjinM6
2aa584ba33
rpc: Loop through all nodes when looking for one to override mnauth data for (#3971) 2021-01-28 21:16:28 -05:00
dustinface
21cfb4c934
llmq|rpc|test|version: Implement P2P messages QGETDATA <-> QDATA (#3953)
* version: Bump PROTOCOL_VERSION and MIN_MASTERNODE_PROTO_VERSION

* version: Introduce LLMQ_DATA_MESSAGES_VERSION for QGETDATA/QDATA support

* test: Bump MY_VERSION to 70219 (LLMQ_DATA_MESSAGES_VERSION)

* llmq: Introduce CQuorumDataRequest as wrapper for QGETDATA requests

* llmq: Implement CQuorum::{SetVerificationVector, SetSecretKeyShare}

* llmq|net|protocol: Implement QGETDATA/QDATA P2P messages

* llmq: Restrict processing QGETDATA/QDATA to masternodes only

* llmq: Implement request limiting for QGETDATA/QDATA

* llmq: Implement CQuorumManger::RequestQuorumData

* rpc: Implement "quorum getdata" as wrapper around QGETDATA

Allows to trigger sending QGETDATA messages to connected peers by RPC.

* test: Handle QGETDATA/QDATA messages in mininode

* test: Add data structures to support QGETDATA/QDATA

* test: Add some helper in test_framework.py

* test: Implement tests for QGETDATA/QDATA in p2p_quorum_data.py

* test: Add p2p_quorum_data.py to BASE_SCRIPTS

* llmq|test: Add QWATCH support for QGETDATA/QDATA

* llmq: Store CQuorumPtr in cache, not CQuorumCPtr

* llmq: Fix cache usage after recent changes

* Use uacomment to create/find specific p2ps

* No need to use network adjusted time here, GetTime should be enough

* rpc: check proTxHash

* minor tweaks

* test: Adjustments after 4e27d6513e

* llmq: Rename and improve error lambda in CQuorumManager::ProcessMessage

* llmq: Process QDATA if -watchquorums is enabled

* test: Handle qwatch messages in mininode

* test: Add test for -watchquorums support

* test: Just some empty lines

* test: Properly stop the p2p network thread at the end of the test

* rpc: Adjust "quorum getdata" parameter descriptions

Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>

* rpc: Fix optionality of proTxHash in "quorum getdata" command

* test: Test optionality of proTxHash for "quorum getdata" command

* test: Be more specific about imports in p2p_quorum_data.py

* llmq|rpc: Add some comments about the request.GetDataMask checks

* test: Some more empty lines

* rpc: One more parameter description

Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>

* test: Unify assert statements / drop parentheses for all of them

* fix typo

Signed-off-by: pasta <pasta@dashboost.org>

* adjust some line wrapping to 80 chars

Signed-off-by: pasta <pasta@dashboost.org>

* tests: Seperate out into dif atomic methods, add logging

Signed-off-by: pasta <pasta@dashboost.org>

* test: Avoid restarting masternodes, just let available requests expire

Just takes a lot time and isn't required imo.

* test: Drop redundant code/tests after separation

This was introduced in 9e224ec2f2

* test: Merge three tests

"test_mnauth_restriction", "test_invalid_messages" and "test_invalid_unexpected_qdata" with the resulting name "test_basics" because i don't feel like DKG recovery thing should be part of a test called "test_invalid_messages" and giving it an own test probably wouldn't make a lot sense because it would still depend on "test_invalid_messages". I also think there is no need for a separated "test_invalid_unexpected_qdata".

* test: Rename test_ratelimiting_banscore -> test_request_limit

* test: Apply python style

* test: Wrap all at 120 characters

Thats the default "draw annoying warnings" setting for PyCharm (and IMO a reasonable line length).

* test: Move some variables

* test: Optimize for speed

* tests: use wait_until in get_mininode_id

* test: Don't use `!=` to check for `None`

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
Co-authored-by: pasta <pasta@dashboost.org>
2021-01-28 17:33:18 -05:00
UdjinM6
5d4431c017
refactor: Add (and use) queueAndMaybePushInv helper (#3961) 2021-01-25 22:35:18 -05:00
PastaPastaPasta
671b336a34
refactor: misc. governance refactoring (#3958)
* fix typos

Signed-off-by: pasta <pasta@dashboost.org>

* remove unneeded initialization

Signed-off-by: pasta <pasta@dashboost.org>

* make method const

Signed-off-by: pasta <pasta@dashboost.org>

* use default for trivial destructor

Signed-off-by: pasta <pasta@dashboost.org>

* governance: remove all typedefs for iterators, use auto instead

Signed-off-by: pasta <pasta@dashboost.org>

* remove redundant size_type

Signed-off-by: pasta <pasta@dashboost.org>

* Remove unused and singly used typedefs

Signed-off-by: pasta <pasta@dashboost.org>

* remove unused code

Signed-off-by: pasta <pasta@dashboost.org>

* mark constructor as explicit

Signed-off-by: pasta <pasta@dashboost.org>

* remove unused typedef

Signed-off-by: pasta <pasta@dashboost.org>

* remove unneeded initialization

Signed-off-by: pasta <pasta@dashboost.org>

* remove singly used typedef

Signed-off-by: pasta <pasta@dashboost.org>

* pass const reference, and don't copy for no reason

Signed-off-by: pasta <pasta@dashboost.org>

* make method const

Signed-off-by: pasta <pasta@dashboost.org>

* make method const

Signed-off-by: pasta <pasta@dashboost.org>

* make method const

Signed-off-by: pasta <pasta@dashboost.org>

* typo

Signed-off-by: pasta <pasta@dashboost.org>

* make constructor explicit

Signed-off-by: pasta <pasta@dashboost.org>

* Clang-Tidy: 'virtual' is redundant since the function is already declared 'override'

Signed-off-by: pasta <pasta@dashboost.org>

* Clang-Tidy: Prefer using 'override' or (rarely) 'final' instead of 'virtual'

Signed-off-by: pasta <pasta@dashboost.org>

* use default for trivial destructor

Signed-off-by: pasta <pasta@dashboost.org>

* remove unused include

Signed-off-by: pasta <pasta@dashboost.org>

* remove unneeded semicolon

Signed-off-by: pasta <pasta@dashboost.org>

* fix typos

Signed-off-by: pasta <pasta@dashboost.org>

* fix typo

Signed-off-by: pasta <pasta@dashboost.org>

* mark constructor explicit

Signed-off-by: pasta <pasta@dashboost.org>

* remove unused typedef

Signed-off-by: pasta <pasta@dashboost.org>

* remove commented out code

Signed-off-by: pasta <pasta@dashboost.org>

* mark constructor explicit

Signed-off-by: pasta <pasta@dashboost.org>

* remove unused spork

Signed-off-by: pasta <pasta@dashboost.org>

* remove boolean check where always true

Signed-off-by: pasta <pasta@dashboost.org>

* make method const

Signed-off-by: pasta <pasta@dashboost.org>

* Remove nCount completely

Signed-off-by: pasta <pasta@dashboost.org>

* Use default path

Signed-off-by: pasta <pasta@dashboost.org>
2021-01-25 22:31:31 -05:00
UdjinM6
783f07e9f7
tests: Add uacomment in P2PConnection and use it to create (correct network specific) strSubVer (#3969) 2021-01-25 19:35:17 +01:00
dustinface
dc2473cd04
llmq: Some fixes/improvements (#3943)
* llmq: Detach dash-q-cachepop from caller

There should be no reason to keep this tread attached
to its parent, if so, let me know.

* llmq: Avoid nullptr access for pindexStart in ScanQuorums

* llmq: Add cacheKey in ProcessCommitment

* llmq: Erase minable commitments if they have been processed

* llmq: Add CLLMQUtils::InitQuorumsCache

* llmq: Use unordered_lru_cache for quorumsCache and rename it

* llmq: Use unordered_lru_cache for hasMinedCommitmentCache and rename it

* llmq: Drop redundant check

* llmq: Rename nMaxCount2 -> nScanCommitments

* llmq: Refactor storeCache -> fCacheExists

* llmq: Rename maxCount -> nCountRequested

* llmq: Rename result -> vecResultQuorums

* llmq: Return an empty vector if the are zero elements requested

* unordered_lru_cache: Add max_size()

* llmq: Partially reuse existing cache if more than max is requested

* llmq: std::map<LLMQType, unordered_lru_cache<...>> for scanQuoumsCache

* llmq: Drop params

* llmq: Only emplace to cache if there is something available
2021-01-25 10:22:28 +01:00
UdjinM6
f69d0f4d64
instantsend: Relay IS-locks for matching txes when replying to merkleblock inv in getdata (#3959) 2021-01-25 00:01:08 -05:00
UdjinM6
a328756396
Flush callbacks one more time at shutdown (#3963)
This is needed to be able to process SetBestChain callback generated by the first FlushStateToDisk call in PrepareShutdown

Partially reverts #3378
2021-01-24 23:56:18 -05:00