Commit Graph

18193 Commits

Author SHA1 Message Date
Wladimir J. van der Laan
7a33600dcd
Merge #9740: Add friendly output to dumpwallet
164019d Add dumpwallet output test (aideca)
9f82134 Add friendly output to dumpwallet refs #9564 (aideca)

Tree-SHA512: 913fcf18d42eebe34173f1f2519973494b1ad2d86d125ff4bf566d6c64aa501c02f8831e6f44812cd87a46916f61c6f510146af406865b31856d8336c173569f
2020-06-18 11:41:53 -05:00
Jonas Schnelli
14166c052d
Merge #12143: [Doc] Fix link for BIP-159 pull request
91769d6e2 [Doc] Fix link for bip 159 pull request (azuchi)

Pull request description:

  The link of the pull request for BIP-159 described in bips.md was a different link.

Tree-SHA512: 818ea29259ff84a55282df8b0c59fc4ccd3af08d124a104005ac48e67da4155a8b071b980b1d12c35af3f4a008ba77e5b4ee3242292f6c034cb0006b5532ce69
2020-06-18 11:41:53 -05:00
Wladimir J. van der Laan
a022cc5ec7
Merge #12206: qa: Sync with validationinterface queue in sync_mempools
fa1e69e qa: Sync with validationinterface queue in sync_mempools (MarcoFalke)

Pull request description:

  Commit e545dedf72 moved `TransactionAddedToMempool` to the background scheduler thread. Thus, adding a transaction to the mempool will no longer add it to the wallet immediately. Functional tests, that `sync_mempools` and then call into wallet rpcs will race against the scheduler thread.

  Fix that race by flushing the scheduler queue.

  Fixes #12205; Fixes #12171;
  References #9584;

Tree-SHA512: 14d99cff9c4756de9fad412f04e6d8e25bb9a0938f24ed8348de79df5b4ee67763dac5214b1a69e77e60787d81ee642976d1482b1b5637edfc4892a238ed22af
2020-06-18 11:41:53 -05:00
MarcoFalke
50c5439ac3
Merge #12264: Fix versionbits warning test
1e2e09e2f6 Fix intermittent failure in p2p-versionbits-warning.py (John Newbery)
3bbd843708 Improve comments/logging in p2p-versionbits-warning.py (John Newbery)
ef2beb2c13 Fix flake8 warnings in p2p-versionbits-warning.py (John Newbery)

Pull request description:

  fixes #12259 (and tidies up the test)

  The problem was that the node was still in IBD at the point the last block was generated. UpdateTip() will not generate a warning if the node is still in IBD:

  cc5870a405/src/validation.cpp (L2151)

  The 'proper' fix would be to remove the overenthusiastic latching in DoWarning:

  cc5870a405/src/validation.cpp (L2135)

  so that more than one warning message can be output to `alertnotify`. Really we should suppress multiple messages of the same type, but allow messages to be output if they're for different warnings. That would mean the test wouldn't need to stop-start the node.

Tree-SHA512: 5c9aa5af7eba3c1350ea28482d57d3d79e3166c6224ceddb5d5a631090081d890d7403015e41f413c22990959a488cf1231f88bb825c54a609b24f89c450a1f6
2020-06-18 11:41:53 -05:00
Wladimir J. van der Laan
fa9fbbaab0
Merge #12197: Log debug build status and warn when running benchmarks
34328b4 Use PACKAGE_NAME instead of hardcoding application name in log message (Wladimir J. van der Laan)
0c74e2e Log debug build status and warn when running benchmarks (Wladimir J. van der Laan)

Pull request description:

  Log whether the starting instance of bitcoin core is a debug or release build (--enable-debug).

  Also warn when running the benchmarks with a debug build, to prevent mistakes comparing debug to non-debug results.

Tree-SHA512: f612dcb7d0a8435016cff0df8aef4942144dfb88be8a00df45cc8830d2aba4b167f6d397b83f8f57d57685888babd04ba88d4dac5a202d3dbd91bcbea3708ef0
2020-06-18 11:41:53 -05:00
Wladimir J. van der Laan
ab07a468b0
Merge #12217: qa: Add missing syncwithvalidationinterfacequeue to tests
fa796bb qa: Add missing syncwithvalidationinterfacequeue to tests (MarcoFalke)

Pull request description:

  Fixes intermittent travis failures with those tests caused by a missing flush of mempool txes to the wallet.

Tree-SHA512: 4f57c93a81af9c07b36c16996bf3e6bbb2af61779f0d6ae0126b64563eb4ec4b53f64241c9cf4c3f322db56f4339fd939319747653bebc93bbc7e3d5dceedda6
2020-06-18 11:41:53 -05:00
Wladimir J. van der Laan
98f6284873
Merge #12619: doc: Give hint about gitian not able to download
08e0855b9 Give hint about gitian not able to download (kallewoof)

Pull request description:

  Gitian fails to perform downloads right now on my set up. This can be circumvented by first checking out the tag being built and then doing the depends download step before running `gbuild`.

  This should of course be fixed in gitian, but having this note until it's fixed is definitely useful.

Tree-SHA512: ae9d0eb44ecfdae44d35aecc6e5fd6db7d9e95b8e0badc76a1d9aaf8fe70bc00a2914dfcb4f516d030560835af411515ca13736ebf8b49b7040b340457882779
Signed-off-by: pasta <pasta@dashboost.org>
2020-06-18 11:41:53 -05:00
Wladimir J. van der Laan
e9b42fd5c3
Merge #12098: [scripts] lint-whitespace: add param to check last N commits
8dbf740f8 [scripts] lint-whitespace: check last N commits or unstaged changes (Sjors Provoost)

Pull request description:

  E.g. before you push three commits to Github and upset Travis, check if you didn't make any whitespace mistakes:
  ```sh
  contrib/devtools/lint-whitespace.sh 3
  ```

  This is slightly more convenient than doing:
  ```sh
  TRAVIS_COMMIT_RANGE=HEAD~3...HEAD contrib/devtools/lint-whitespace.sh
  ```

Tree-SHA512: 5d9c1ae978ccbe59477e8cf53391e9bd697d2da87f417a2519264af560d4768138e0b2d320dd497a1f1e704e18ab279d724f523b57c17a80ccd753133a5445bf
2020-06-18 11:41:53 -05:00
Wladimir J. van der Laan
a88ad9384f
Merge #12097: [scripts] lint-whitespace: use perl instead of grep -P
40b17f5f9 [scripts] lint-whitespace: use perl instead of grep -P (Sjors Provoost)

Pull request description:

  MacOS does not support `grep -P` out of the box. This change makes
  it easier for developers to check for whitespace problems locally.

  Based on [this](https://stackoverflow.com/a/16658690) and [this](https://serverfault.com/a/504387) Stack Exchange answer.

  Tested with:
  ```sh
  export TRAVIS_COMMIT_RANGE='fe78c9a...62e0453'
  contrib/devtools/lint-whitespace.sh
  This diff appears to have added new lines with tab characters instead of spaces.
  The following changes were suspected:

  diff --git a/src/test/bignum_tests.cpp b/src/test/bignum_tests.cpp
  @@ -0,0 +1,110 @@
  +	num.setint64(n);
  ```

Tree-SHA512: 37c342a0ca2580289cf326a278a051a7c21ba918d6b2143fd9987f159fab85f1de3d770fcf532a642cd5d1957afc8595678128196e102dc473924758f133db7f
2020-06-18 11:41:53 -05:00
PastaPastaPasta
3f8a7b2068
Merge #12193: RPC: Consistently use UniValue.pushKV instead of push_back(Pair()) (karel-3d) (#3532)
* Begin Merge 12193 Squashed 'src/univalue/' changes from 07947ff2da..51d3ab34ba

51d3ab34ba Merge #10: Add pushKV(key, boolean) function (replaces #5)
129bad96d5 [tests] test pushKV for boolean values
b3c44c947f Pushing boolean value to univalue correctly

git-subtree-dir: src/univalue
git-subtree-split: 51d3ab34ba2857f0d03dc07250cb4a2b5e712e67

* scripted-diff: Use UniValue.pushKV instead of push_back(Pair()) (end #12193)

-BEGIN VERIFY SCRIPT-
git grep -l "push_back(Pair" | xargs sed -i "s/push_back(Pair(\(.*\)));/pushKV(\1);/g"
-END VERIFY SCRIPT-

Signed-off-by: pasta <pasta@dashboost.org>

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-06-18 12:17:23 +03:00
UdjinM6
ed057dadd0
Sleep 1 sec in mininode's wait_for_disconnect (#3538)
* Sleep 1 sec in mininode's wait_for_disconnect

* Add a comment about a future backport that might potentially fix related issues
2020-06-18 11:41:27 +03:00
Alexander Block
737ed479bc
Don'd send SENDXXX messages to fMasternode connections (#3537)
And respect a received QSENDRECSIGS no matter if fMasternode or not. We
assume that fMasternode peers won't send QSENDRECSIGS from now on.
2020-06-18 11:41:18 +03:00
Alexander Block
0fa058deb5
Only relay DKG messages to intra quorum connection members (#3536)
This reuses the old intra-quorum connections algorithm.
2020-06-18 11:40:33 +03:00
UdjinM6
f7a40a45a4
Merge pull request #3530 from PastaPastaPasta/backports-0.17-pr5
Backports 0.17 pr5
2020-06-18 11:37:19 +03:00
MarcoFalke
3d252a72b2 Merge #12710: Append scripts to new test_list array to fix bad assignment
b0fec8d623 Append scripts to new test_list array to fix bad assignment (Jeff Rade)

Pull request description:

  Fixes review by @MarcoFalke in PR [#12437](https://github.com/bitcoin/bitcoin/pull/12437)

  Assignment of `test_list` would point to the same array object as `BASE_SCRIPT` or `ALL_SCRIPTS` which we do not want.

Tree-SHA512: 57d6c1f4563aaffbac68e96782283799b10be687292152d70ffbbc22e7b52c11c1af0c483acb01c69fbaa99bdae01431b65a5d1d0a913d549f58dfd95d0d28d9
2020-06-17 14:29:55 -05:00
MarcoFalke
19e2de1b22 Merge #12437: [Trivial] Simplify if-else blocks and more descriptive variable naming
97bcd36811 [Trivial] Simplify if-else blocks and more descriptive variable naming (Jeff Rade)

Pull request description:

  Was looking through `test_runner.py` to start work on [#11964](https://github.com/bitcoin/bitcoin/issues/11964).  Made a few changes to make the file more readable and keep these separate from future PR.

Tree-SHA512: 7508f4ee39672d18718d8f80b61b89918eac7b4c75953682b812b73013f18ebd81adc7953f3b6c98c5c598adeb1998f5455f123b5566d1cc03631c7924b4103a
2020-06-17 14:29:55 -05:00
Wladimir J. van der Laan
8adade97a4 Merge #12603: [docs] PeerLogicValidation interface
b7cd08b71 Add documentation to PeerLogicValidation interface and related functions (James O'Beirne)

Pull request description:

  Adds docs for PeerLogicValidation's public interface and two related functions.

Tree-SHA512: b4c2f47e9baa9396d2b6faf3792e46b371c50cd91b9ac890f263f4d14eb24a71e7b40ceb4cbb41e254f5008eff357f417b842618e7ebece9039802ab2a5dd728
2020-06-17 14:29:55 -05:00
Wladimir J. van der Laan
95edea4e3e Merge #12615: doc: allow for SIGNER containing spaces
e2b2e48b6 doc: SIGNER can contains space inside now. (Ken Lee)

Pull request description:

  SIGNER can contains space inside now. mentioned in https://github.com/bitcoin/bitcoin/pull/12527#issuecomment-370506416

Tree-SHA512: 8da1e8146751457c351058c0142fa3d474a338fe7304a31ebed4726202202724aaca94441806458512259238a703601b87961196abc3fdd57b5eb0f062ff0c12
2020-06-17 14:29:55 -05:00
Wladimir J. van der Laan
e6a1cc63d1 Merge #12477: test: Plug memory leaks and stack-use-after-scope
fadb39c test: Plug memory leaks and stack-use-after-scope (MarcoFalke)

Pull request description:

Tree-SHA512: 7bd6bbba43c7870bbd9732d73ecfc520f21701168e6fb4ad099a08ea5b21d9cd09215e70d22fb92a1af03993204ef89ad74b3e80d9fa5a10831c3e7cf2dd04cd
2020-06-17 14:29:55 -05:00
MarcoFalke
1cf2fff1c4 Merge #12308: contrib: Add support for out-of-tree builds in gen-manpages.sh
526e28220a contrib: Add support for out-of-tree builds in gen-manpages.sh (Wladimir J. van der Laan)

Pull request description:

  This adds support for setting the environment variable `BUILDDIR` to point to executables that are outside the source directory.

  E.g. to invoke the tool when the build is in $PWD/build:

  ```bash
  BUILDDIR=$PWD/build contrib/devtools/gen-manpages.sh
  ```

  This avoids having to manually copy the generated manpages after they end up in the build instead of source path, when setting TOPDIR instead.

Tree-SHA512: 8dc6dd7a47a0c014ae7d27f0ac9d86f69238ec6bac8a3007b975bb88c9f37014755c716c5e62604dd91baad2f8a41fd1544cdca3ba4b59bc76602e6593f4a4a7
2020-06-17 14:29:55 -05:00
Jonas Schnelli
a0dac26647 Merge #12029: Build: Add a makefile target for Doxygen documentation
a777244e4 Build: Add a makefile target for Doxygen documentation (Andrea Comand)

Pull request description:

  You can now build the doxygen documentation with `make docs` and clean it with `make clean-docs`.

  Fixes: #11949

Tree-SHA512: f2361ec7f771227367dd04bba1a444b44e59f13901463a678a5f2f579a10a56d67db2e29552e754e312a1c472a31593b6af189cbaac5cd351a428c57baf5ace7
2020-06-17 14:29:55 -05:00
Wladimir J. van der Laan
128e14640e Merge #12425: Add some script tests
be45a67 Add some script tests related to BOOL ops and odd values like negative 0. (Richard Kiss)

Pull request description:

  Add some script tests related to BOOL ops and odd values like negative 0.

Tree-SHA512: 8e633f7ea5eea39e31016994baf60f295fa1dc8cae27aa5fcfc741ea97136bfb3ddc57bb62b9c6bf9fe256fc09cdd184906ba8e611e297cf8d2d363da2bbf1d4
2020-06-17 14:29:55 -05:00
MarcoFalke
10a27192b1 Merge #12442: devtools: Exclude patches from lint-whitespace
fafbf7f74e devtools: Exclude patches from lint-whitespace (MarcoFalke)

Pull request description:

  By default, unified patches have trailing whitespace in all context lines. Thus, exclude patches from linting.

Tree-SHA512: 8f89f1584581e94dd4e34bd522cba21602bafe7933b4631a3abc5da5a8f25568810862d696618fe63c15edf3e046869ad5077d09373f09792985503c6a415538
2020-06-17 14:29:55 -05:00
Wladimir J. van der Laan
145f6798a3 Merge #12426: qt: Initialize members in WalletModel
fa27623 qt: Initialize members in WalletModel (MarcoFalke)

Pull request description:

  This prevents segfaults (or errors when running qt in valgrind)

  ```
  Conditional jump or move depends on uninitialised value(s)
      WalletModel::checkBalanceChanged() (walletmodel.cpp:156)

Tree-SHA512: 38c8c03c7fa947edb3f1c13eab2ac7a62ef8f8141603c2329a7dc5821a887a349af8014dc739b762e046f410f44a9c6653b6930f08b53496cf66381cadc06246
2020-06-17 14:29:55 -05:00
Wladimir J. van der Laan
2a24d84bd2 Merge #12409: rpc: Reject deprecated reserveChangeKey in fundrawtransaction
fa5f518 rpc: Reject deprecated reserveChangeKey in fundrawtransaction (MarcoFalke)

Pull request description:

Tree-SHA512: 8506d1494b13c4582b1379e3b8c3906016f1980ebe847727a43a90e7bb9f71b896a1792bc97a8dc7320ccce0534050eb04f92a6f82f811d08efa74a98b3e43f0
2020-06-17 14:29:55 -05:00
Wladimir J. van der Laan
9b0c1f7733 Merge #12393: Fix a-vs-an typos
11376b5 Fix a-vs-an typos (practicalswift)

Pull request description:

  Fix a-vs-an typos.

Tree-SHA512: 2cf74c15656a20ec13d2da7d86a39d14e634db368833d92da06a78d1266950accfc4fcc89cfecdaadd46e6b48b17e6fad29080428e564871e78482c53f3e855c
2020-06-17 14:29:55 -05:00
Wladimir J. van der Laan
4025a5b9af Merge #12282: wallet: Disallow abandon of conflicted txes
fa795cf wallet: Disallow abandon of conflicted txes (MarcoFalke)

Pull request description:

  Abandon transactions that are already conflicted is a noop, so don't try and return false/throw instead.

Tree-SHA512: fd2af4149bd2323f7f31fe18685c763790b8589319b4e467b464ab456d5e8971501ab16d124e57a22693666b06ae433ac3e59f0fd6dfbd2be2c6cae8be5bcbd8
2020-06-17 14:29:55 -05:00
Pasta
b40dc8f1ec fix lint-python.sh after 11835 and 12295
Signed-off-by: Pasta <pasta@dashboost.org>
2020-06-17 14:29:55 -05:00
MarcoFalke
89cf527e57 Merge #12295: Enable flake8 warnings for all currently non-violated rules
a9d0ebc262 Enable flake8 warnings for all currently non-violated rules (practicalswift)
4cbab15e75 tests: Fix accidental redefinition of previously defined variable via list comprehension (practicalswift)
0b9207efbe Enable flake8 warning for "list comprehension redefines 'foo' from line N" (F812) (practicalswift)

Pull request description:

  * Enable `flake8` warnings for all currently non-violated rules
  * Fix accidental redefinition via list comprehension

Tree-SHA512: 738b87789e99d02abb2c6b8ff58f65c0cbfeb93e3bf320763e033e510ebd0a4f72861bc8faaf42c14a056a5d4659c33dc70a63730a32cc15159559427bf21193
2020-06-17 14:29:55 -05:00
MarcoFalke
cd1fdb5b01 Partial Merge #11835: Add Travis check for unused Python imports
d60b32074 Add Travis check for unused Python imports (practicalswift)
c7399e708 Remove unused Python imports (practicalswift)

Pull request description:

  Add Travis check for unused Python imports.

  ```
  $ contrib/devtools/lint-python.sh
  ./test/functional/example_test.py:18:1: F401 'test_framework.mininode.NODE_NETWORK' imported but unused
  ./test/functional/test_framework/messages.py:27:1: F401 'test_framework.util.wait_until' imported but unused
  ./test/functional/test_framework/test_framework.py:16:1: F401 'traceback' imported but unused
  ```

Tree-SHA512: 78e50fb1488abe3ebe365e766cb8d6d448cf1bd16c8691e102cb9bf7c202988bdf6e10b25ff772c62e05c72568168462e88cdc7ad98069d9eb3be727735b2d56
2020-06-17 14:29:55 -05:00
MarcoFalke
9e71390d39 Merge #12438: [Tests] Fix trivial typo in test_runner.py causing error
ada1af6d8f Fix typo in test_runner.py causing error (MeshCollider)

Pull request description:

  In the case that a test fails, the typo in run_tests() (introduced in #11858) will cause an error rather than printing out the combined logs, hiding the cause of the failure.

Tree-SHA512: 7d7aa406d92750320ed20610cc5f174cdc94086f630af8c0c4db2003497132e0c56d59b94312fb42ad4507904a2fa858226a4a9337450930bf206183fc35c0a0
2020-06-17 14:29:54 -05:00
Wladimir J. van der Laan
a8da932ff0 Merge #11858: qa: Prepare tests for Windows
faefd29 qa: Prepare functional tests for Windows (MarcoFalke)

Pull request description:

  * Pass `sys.executable` when calling a python script via the subprocess
    module
  * Don't remove the log file while it is still open and written to
  * Properly use os.pathsep and os.path.sep when modifying the PATH
    environment variable
  * util-tests: Use os.path.join for Windows compatibility

  Ref:  #8227

Tree-SHA512: c507a536af104b3bde4366b6634099db826532bd3e7c35d694b5883c550920643b3eab79c76703ca67e1044ed09979e855088f7324321c8d52112514e334d614
2020-06-17 14:29:54 -05:00
John Newbery
219ff3901f Partial Merge #11789 2020-06-17 14:29:54 -05:00
UdjinM6
f12c592b1f
Merge pull request #3416 from PastaPastaPasta/backport-12254
Backport 12254 and 14073 (BIP158)
2020-06-16 12:20:23 +03:00
UdjinM6
593e8b2e2a
Merge pull request #3527 from PastaPastaPasta/backports-0.17-pr4
Backports 0.17 pr4
2020-06-16 12:19:57 +03:00
UdjinM6
085bd7fd0a
Merge pull request #3533 from PastaPastaPasta/backport-13352
Merge #13352: qa: Avoid checking reject code for now
2020-06-16 12:19:20 +03:00
UdjinM6
8a3d82dd3c
Merge pull request #3519 from PastaPastaPasta/backports-0.17-pr3
Backports 0.17 pr3
2020-06-16 11:25:02 +03:00
Wladimir J. van der Laan
329c56721b
Merge #13352: qa: Avoid checking reject code for now
faac7a2db4f9f511c901cb1b4d4e7c599b92884f qa: Avoid checking reject code for now (MarcoFalke)

Pull request description:

  The node will often disconnect before sending a reject code. A more
  robust solution would be to read from the debug log. See  #13006

Tree-SHA512: 1dabf8a43dabbc722f4ffe4fbc1f870090253a66290b2d1a95e7a24e14c6442b493c314480c0314587164eb65e5d468aa9eb5e107ad90bb3ca821a97ea4d373c
2020-06-14 12:22:23 -05:00
Wladimir J. van der Laan
78ef3fdd3a
Merge #12373: Build: Add build support for profiling.
cfaac2a60 Add build support for 'gprof' profiling. (murrayn)

Pull request description:

  Support for profiling build: `./configure --enable-profiling`

Tree-SHA512: ea983cfce385f1893bb4ab7f94ac141b7d620951dc430da3bbc92ae1357fb05521eac689216e66dc87040171a8a57e76dd7ad98036e12a2896cfe5ab544347f0
2020-06-14 11:41:08 -05:00
Wladimir J. van der Laan
a00da7fb0e
Merge #10271: Use std:🧵:hardware_concurrency, instead of Boost, to determine available cores
937bf4335 Use std:🧵:hardware_concurrency, instead of Boost, to determine available cores (fanquake)

Pull request description:

  Following discussion on IRC about replacing Boost usage for detecting available system cores, I've opened this to collect some benchmarks + further discussion.

  The current method for detecting available cores was introduced in #6361.

  Recap of the IRC chat:
  ```
  21:14:08 fanquake: Since we seem to be giving Boost removal a good shot for 0.15, does anyone have suggestions for replacing GetNumCores?
  21:14:26 fanquake: There is std:🧵:hardware_concurrency(), but that seems to count virtual cores, which I don't think we want.
  21:14:51 BlueMatt: fanquake: I doubt we'll do boost removal for 0.15
  21:14:58 BlueMatt: shit like BOOST_FOREACH, sure
  21:15:07 BlueMatt: but all of boost? doubtful, there are still things we need
  21:16:36 fanquake: Yea sorry, not the whole lot, but we can remove a decent chunk. Just looking into what else needs to be done to replace some of the less involved Boost usage.
  21:16:43 BlueMatt: fair
  21:17:14 wumpus: yes, it makes sense to plan ahead a bit, without immediately doing it
  21:18:12 wumpus: right, don't count virtual cores, that used to be the case but it makes no sense for our usage
  21:19:15 wumpus: it'd create a swarm of threads overwhelming any machine with hyperthreading (+accompanying thread stack overhead), for script validation, and there was no gain at all for that
  21:20:03 sipa: BlueMatt: don't worry, there is no hurry
  21:59:10 morcos: wumpus: i don't think that is correct
  21:59:24 morcos: suppose you have 4 cores (8 virtual cores)
  21:59:24 wumpus: fanquake: indeed seems that std has no equivalent to physical_concurrency, on any standard. That's annoying as it is non-trivial to implement
  21:59:35 morcos: i think running par=8 (if it let you) would be notably faster
  21:59:59 morcos: jeremyrubin and i discussed this at length a while back... i think i commented about it on irc at the time
  22:00:21 wumpus: morcos: I think the conclusion at the time was that it made no difference, but sure would make sense to benchmark
  22:00:39 morcos: perhaps historical testing on the virtual vs actual cores was polluted by concurrency issues that have now improved
  22:00:47 wumpus: I think there are not more ALUs, so there is not really a point in having more threads
  22:01:40 wumpus: hyperthreads are basically just a stored register state right?
  22:02:23 sipa: wumpus: yes but it helps the scheduler
  22:02:27 wumpus: in which case the only speedup using "number of cores" threads would give you is, possibly, excluding other software from running on the cores on the same time
  22:02:37 morcos: well this is where i get out of my depth
  22:02:50 sipa: if one of the threads is waiting on a read from ram, the other can use the arithmetic unit for example
  22:02:54 morcos: wumpus: i'm pretty sure though that the speed up is considerably more than what you might expect from that
  22:02:59 wumpus: sipa: ok, I back down, I didn't want to argue this at all
  22:03:35 morcos: the reason i haven't tested it myself, is the machine i usually use has 16 cores... so not easy due to remaining concurrency issues to get much more speedup
  22:03:36 wumpus: I'm fine with restoring it to number of virtual threads if that's faster
  22:03:54 morcos: we should have somene with 4 cores (and  actually test it though, i agree
  22:03:58 sipa: i would expect (but we should benchmark...) that if 8 scriot validation threads instead of 4 on a quadcore hyperthreading is not faster, it's due to lock contention
  22:04:20 morcos: sipa: yeah thats my point, i think lock contention isn't that bad with 8 now
  22:04:22 wumpus: on 64-bit systems the additional thread overhead wouldn't be important at least
  22:04:23 gmaxwell: I previously benchmarked, a long time ago, it was faster.
  22:04:33 gmaxwell: (to use the HT core count)
  22:04:44 wumpus: why was this changed at all then?
  22:04:47 wumpus: I'm confused
  22:05:04 sipa: good question!
  22:05:06 gmaxwell: I had no idea we changed it.
  22:05:25 wumpus: sigh 
  22:05:54 gmaxwell: What PR changed it?
  22:06:51 gmaxwell: In any case, on 32-bit it's probably a good tradeoff... the extra ram overhead is worth avoiding.
  22:07:22 wumpus: https://github.com/bitcoin/bitcoin/pull/6361
  22:07:28 gmaxwell: PR 6461 btw.
  22:07:37 gmaxwell: er lol at least you got it right.
  22:07:45 wumpus: the complaint was that systems became unsuably slow when using that many thread
  22:07:51 wumpus: so at least I got one thing right, woohoo
  22:07:55 sipa: seems i even acked it!
  22:07:57 BlueMatt: wumpus: there are more alus
  22:08:38 BlueMatt: but we need to improve lock contention first
  22:08:40 morcos: anywya, i think in the past the lock contention made 8 threads regardless of cores a bit dicey.. now that is much better (although more still to be done)
  22:09:01 BlueMatt: or we can just merge #10192, thats fee
  22:09:04 gribble: https://github.com/bitcoin/bitcoin/issues/10192 | Cache full script execution results in addition to signatures by TheBlueMatt · Pull Request #10192 · bitcoin/bitcoin · GitHub
  22:09:11 BlueMatt: s/fee/free/
  22:09:21 morcos: no, we do not need to improve lock contention first.   but we should probably do that before we increase the max beyond 16
  22:09:26 BlueMatt: then we can toss concurrency issues out the window and get more speedup anyway
  22:09:35 gmaxwell: wumpus: yea, well in QT I thought we also diminished the count by 1 or something?  but yes, if the motivation was to reduce how heavily the machine was used, thats fair.
  22:09:56 sipa: the benefit of using HT cores is certainly not a factor 2
  22:09:58 wumpus: gmaxwell: for the default I think this makes a lot of sense, yes
  22:10:10 gmaxwell: morcos: right now on my 24/28 physical core hosts going beyond 16 still reduces performance.
  22:10:11 wumpus: gmaxwell: do we also restrict the maximum par using this? that'd make less sense
  22:10:51 wumpus: if someone *wants* to use the virtual cores they should be able to by setting -par=
  22:10:51 sipa: *flies to US*
  22:10:52 BlueMatt: sipa: sure, but the shared cache helps us get more out of it than some others, as morcos points out
  22:11:30 BlueMatt: (because it means our thread contention issues are less)
  22:12:05 morcos: gmaxwell: yeah i've been bogged down in fee estimation as well (and the rest of life) for a while now.. otherwise i would have put more effort into jeremy's checkqueue
  22:12:36 BlueMatt: morcos: heh, well now you can do other stuff while the rest of us get bogged down in understanding fee estimation enough to review it 
  22:12:37 wumpus: [to answer my own question: no, the limit for par is MAX_SCRIPTCHECK_THREADS, or 16]
  22:12:54 morcos: but to me optimizing for more than 16 cores is pretty valuable as miners could use beefy machines and be less concerned by block validation time
  22:14:38 BlueMatt: morcos: i think you may be surprised by the number of mining pools that are on VPSes that do not have 16 cores 
  22:15:34 gmaxwell: I assume right now most of the time block validation is bogged in the parts that are not as concurrent. simple because caching makes the concurrent parts so fast. (and soon to hopefully increase with bluematt's patch)
  22:17:55 gmaxwell: improving sha2 speed, or transaction malloc overhead are probably bigger wins now for connection at the tip than parallelism beyond 16 (though I'd like that too).
  22:18:21 BlueMatt: sha2 speed is big
  22:18:27 morcos: yeah lots of things to do actually...
  22:18:57 gmaxwell: BlueMatt: might be a tiny bit less big if we didn't hash the block header 8 times for every block. 
  22:21:27 BlueMatt: ehh, probably, but I'm less rushed there
  22:21:43 BlueMatt: my new cache thing is about to add a bunch of hashing
  22:21:50 BlueMatt: 1 sha round per tx
  22:22:25 BlueMatt: and sigcache is obviously a ton
  ```

Tree-SHA512: a594430e2a77d8cc741ea8c664a2867b1e1693e5050a4bbc8511e8d66a2bffe241a9965f6dff1e7fbb99f21dd1fdeb95b826365da8bd8f9fab2d0ffd80d5059c
2020-06-14 11:41:08 -05:00
Wladimir J. van der Laan
f52fa3465e
Merge #12616: Set modal overlay hide button as default
cfdd89589 qt: Set modal overlay hide button as default (João Barbosa)

Pull request description:

  Without this change the only way to close the modal overlay is to click the hide button. Setting the button to default allows to activate it with the ENTER key.

  Before:
  <img width="849" alt="screen shot 2018-03-06 at 15 14 23" src="https://user-images.githubusercontent.com/3534524/37040276-58af9ce0-2151-11e8-8c55-50acdea669d9.png">

  After:
  <img width="848" alt="screen shot 2018-03-06 at 15 12 41" src="https://user-images.githubusercontent.com/3534524/37040294-650d1c9c-2151-11e8-8245-2da250a71b3d.png">

Tree-SHA512: a93ef440a507843ed7870fd07a693af93dd97c8fce2fb6824c69a227b5dee258f340bf1ae344da32a9dd6e6cb2330f72db9dac9635bbd34184c3e7f8476a472e
2020-06-14 11:41:07 -05:00
Wladimir J. van der Laan
9b825c8bb7
Merge #12604: Add DynamicMemoryUsage() to CDBWrapper to estimate LevelDB memory use
741f0177c Add DynamicMemoryUsage() to LevelDB (Evan Klitzke)

Pull request description:

  This adds a new method `CDBWrapper::DynamicMemoryUsage()` similar to Bitcoin's existing methods of the same name. It's implemented by asking LevelDB for the information, and then parsing the string response. I've also added logging to `CDBWrapper::WriteBatch()` to track this information:

  ```
  $ tail -f ~/.bitcoin/testnet3/debug.log | grep WriteBatch
  2018-03-05 19:34:55 WriteBatch memory usage: db=chainstate, before=0.0MiB, after=0.0MiB
  2018-03-05 19:35:17 WriteBatch memory usage: db=index, before=0.0MiB, after=0.0MiB
  2018-03-05 19:35:17 WriteBatch memory usage: db=chainstate, before=0.0MiB, after=8.0MiB
  2018-03-05 19:35:22 WriteBatch memory usage: db=index, before=0.0MiB, after=0.0MiB
  2018-03-05 19:35:22 WriteBatch memory usage: db=chainstate, before=8.0MiB, after=17.0MiB
  2018-03-05 19:35:26 WriteBatch memory usage: db=index, before=0.0MiB, after=0.0MiB
  2018-03-05 19:35:27 WriteBatch memory usage: db=chainstate, before=9.0MiB, after=18.0MiB
  2018-03-05 19:35:40 WriteBatch memory usage: db=index, before=0.0MiB, after=0.0MiB
  2018-03-05 19:35:41 WriteBatch memory usage: db=chainstate, before=9.0MiB, after=7.0MiB
  2018-03-05 19:35:52 WriteBatch memory usage: db=index, before=0.0MiB, after=0.0MiB
  2018-03-05 19:35:52 WriteBatch memory usage: db=chainstate, before=7.0MiB, after=9.0MiB
  ^C
  ```

  As LevelDB doesn't seem to provide a way to get the database name, I've also added a new `m_name` field to the `CDBWrapper`. This is necessary because we have multiple LevelDB databases (two now, and possibly more later, e.g. #11857).

  I am using this information in other branches where I'm experimenting with changing LevelDB buffer sizes.

Tree-SHA512: 7ea8ff5484bb07ef806af17d000c74ccca27d2e0f6c3229e12d93818f00874553335d87428482bd8acbcae81ea35aef2a243326f9fccbfac25989323d24391b4
2020-06-14 11:41:07 -05:00
Wladimir J. van der Laan
1b4d89f319
Merge #12568: Allow dustrelayfee to be set to zero
874e81808 Allow dustrelayfee to be set to zero (Luke Dashjr)

Pull request description:

  I don't see and can't think of any rationale for forbidding this configuration.

Tree-SHA512: df09441f4aec63e79bea94838b7f8e336cebaeb0a22b5e58d27937bbeb1377f229921aeae43674e0b63fc40a39ae51a264d48aa1cdb4cbd0e3339d32856698bf
2020-06-14 11:41:07 -05:00
Wladimir J. van der Laan
4e26e4e72c
Merge #11880: Stop special-casing phashBlock handling in validation for TBV
9c5a4a6ed Stop special-casing phashBlock handling in validation for TBV (Matt Corallo)

Pull request description:

  There is no reason to do this, really, we already have "ignore PoW" flags. Motivated by https://github.com/bitcoin/bitcoin/pull/11739#discussion_r155841721

Tree-SHA512: 37cb1ae5b11c9e8ed7a679bb07ad3b119a2a014744b26d197d67ba21beb19fe6815271df935e40f7c7bd5f2e4d7ae4dad7bd4d00fa230a8d789f37e9de31a769
2020-06-14 11:41:06 -05:00
Wladimir J. van der Laan
9dd01a17a3
Merge #12260: [Trivial] link mentioned scripted-diff-commit (developer-doc)
7eb665fc8 [Trivial] link mentioned scripted-diff-commit (Felix Wolfsteller)

Pull request description:

  Make it easier for people who do not operate on a cloned repository to access the example mentioned.

Tree-SHA512: 1c06e551c68cad03e6bd541bf0e0076cdf0b48ef9b8b4e4a61435367c3435e2e4ccb934112e8dc29d3d70217d8834924704aaf839e25d1133312df86848ca1a1
2020-06-14 11:41:06 -05:00
Wladimir J. van der Laan
28879fc052
Merge #12434: [doc] dev-notes: Members should be initialized
fa9461473 [doc] dev-notes: Members should be initialized (MarcoFalke)

Pull request description:

  Also, remove mention of threads that were removed long ago.

  Motivation:
  Make it easier to spot bugs such as #11654 and  #12426

Tree-SHA512: 8ca1cb54e830e9368803bd98a8b08c39bf2d46f079094ed7e070b32ae15a6e611ce98d7a614f897803309f4728575e6bc9357fab1157c53d2536417eb8271653
2020-06-14 11:41:05 -05:00
Wladimir J. van der Laan
c939de0256
Merge #12570: Add test cases for HexStr (std::reverse_iterator and corner cases)
ac48861 Add tests for HexStr std::reverse_iterator cases (Kosta Zertsekel)
90eac8c Add tests for HexStr corner cases (Kosta Zertsekel)

Pull request description:

Tree-SHA512: 6298d6fdc344e67a9ea6dc74eadb04e68f4f49fc4511d4a8765cafce7eeb8603f96ebedd82c13811326bcaf1ee511946419b651ca411f711baca91bec51947d6
2020-06-14 11:41:05 -05:00
Wladimir J. van der Laan
eaf79feda0
Merge #12421: [qt] navigate to transaction history page after send
e7d9fc5 [qt] navigate to  transaction history page after send (Sjors Provoost)

Pull request description:

  Before this change QT just remained on the Send tab, which I found confusing. Now it switches to the Transactions tab. This makes it more clear to the user that the send actually succeeded, and here they can monitor progress.

  Ideally I would like to highlight the transaction, e.g. by refactoring `TransactionView::focusTransaction(const QModelIndex &idx)` to accept a transaction hash, but I'm not sure how to do that.

Tree-SHA512: 8aa93e03874de8434e18951f8aec47377814c0bcaf7eda4766fc41d5a4e32806346e12e4139e4d45468dfdf0b786f5a7faa393a31b8cd6c65ccac21fb3782c33
2020-06-14 11:41:04 -05:00
Wladimir J. van der Laan
169db16be5
Merge #12083: Improve getchaintxstats test coverage
57e6786 qa: Improve getchaintxstats functional test (João Barbosa)
501b439 rpc: Refactor blockhash parse in getchaintxstats (João Barbosa)

Pull request description:

Tree-SHA512: 61dec5cb68122998df7ec7b5239830f3caf0fe7185c107a66f27653ab2531a800db19a09050671b6fa8dbb5b53181da861eb31199c79d8635f246ccfa0d10efd
2020-06-14 11:41:04 -05:00
Wladimir J. van der Laan
82f04481f1
Merge #11733: qt: Remove redundant locks
d6f3a73 Remove redundant locks (practicalswift)

Pull request description:

  Remove redundant locks:
  * ~~`FindNode(...)` is locking `cs_vNodes` internally~~
  * `SetAddressBook(...)` is locking `cs_wallet` internally
  * `DelAddressBook(...)` is locking `cs_wallet` internally

  **Note to reviewers:** From what I can tell these locks are redundantly held from a data integrity perspective (guarding specific variables), and they do not appear to be needed from a data consistency perspective (ensuring a consistent state at the right points). Review thoroughly and please let me know if I'm mistaken :-)

Tree-SHA512: 7e3ca2d52fecb16385dc65051b5b20d81b502c0025d70b0c489eb3881866bdd57947a9c96931f7b213f5a8a76b6d2c7b084dff0ef2028a1e9ca9ccfd83e5b91e
2020-06-14 11:41:03 -05:00