* Backporting Merge #18584: test: Check that the version message does not leak the local address
fa404f1e4718e8155581f23826480086dfbcfaa6 test: Check that the version message does not leak the local address of the node (MarcoFalke)
Pull request description:
Add test for #8740
ACKs for top commit:
theStack:
ACK fa404f1e47
Tree-SHA512: 4d1c10d1c02fba4b51bd8b9eb3a0d9a682b6aac8c3f6924e295fdca3faefa5ecc3eaa87d347cfec5d2b2bc49963c10fe0a37c463f36088ed0304a2e3716b963b
* Merge #18584: test: Check that the version message does not leak the local address
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
6f6514a08090b37b5e8c086015ee4881813ef867 Correct units for "-dbcache" and "-prune" (Hennadii Stepanov)
Pull request description:
Actually, all `dbcache`-related values in the code are measured in MiB (not in megabytes, MB) or in bytes (e.g., `nTotalCache`).
See: https://github.com/bitcoin/bitcoin/blob/master/src/txdb.hba8c8b2227/src/init.cpp (L1405-L1424)
Also, "-prune" is fixed:
1. The GUI values in GB are translated to the node values in MiB correctly.
2. The maximum of the "prune" `QSpinBox` is not limited by default value of 99 (GB).
Fix: #15106
Tree-SHA512: 151ec43b31b1074db8b345fedb1dcc10bde225899a5296bfc183f57e1553d13ac27db8db100226646769ad03c9fcab29d88763065a471757c6c41ac51108459d
948d8f4f10c31220ba4b6779cc862e2b6a0af5f6 lint: Enable python linters via an array (Ben Woosley)
Pull request description:
This assures consistent recording of the enabled linters.
This applies the same fix as #15170 to lint-python.sh
Tree-SHA512: 6d03f919e86e7c2465475c88b25dd84391282bcc11728078024daf0432a7dccddf9e4a2cdae35d6ef374971cb4e12f0fa21b58f757e25f2fe7c12ceb4f4b2c57
fa21568208 qa: Avoid race in p2p_invalid_block by waiting for the block request (MarcoFalke)
6c787d340c tests: Make feature_block pass on centos (MarcoFalke)
Pull request description:
This hopefully fixes#14661, which I believe is caused by a race in `send_blocks_and_test`. By setting `request_block=False` we only effectively check `node.getbestblockhash() != blocks[-1].hash` before returning and checking the debug.log. By setting `request_block=True` (the default) we make sure that we send the block, then sync with a ping before asserting on the debug.log.
Even if this patch doesn't fix the issue, it is good cleanup: There is no reason to not wait for the blocks to be requested, since in all these cases the header gives no indication that the block is consensus invalid. So this patch makes the test also a bit stricter and more useful.
Unrelated to this, I also include a fix that makes the tests pass on latest CentOS.
Tree-SHA512: c7abee3b7dc790a8af6c289159a7751bd962f6fa16c1537e7e21a0a0ef05b9596d1f4eb75319614603c05cb803e021314fa3596508ba443edd03046b25527e0f
9b4a36effcf642f3844c6696b757266686ece11a [qa] Test for duplicate inputs within a transaction (Suhas Daftuar)
b8f801964f59586508ea8da6cf3decd76bc0e571 Fix crash bug with duplicate inputs within a transaction (Suhas Daftuar)
Pull request description:
Tree-SHA512: 8c7ea34c7fa44188d86c04a690a7cbf8e9deda71ab1f7ca6d11de1f2abb3dd7222627071f86d0d39689a8b302ba9af142f0202466a67e30cd54aed3a08d4eb14
0385109444646561a718f34ae437b7e0e4d4d5bc Add test for rpcpassword hash error (MeshCollider)
13fe258e91e7a92326aedf151c571994166a06d4 Error if rpcpassword in conf contains a hash character (MeshCollider)
Pull request description:
Fixes#13143 now #13482 was merged
Tree-SHA512: e7d00c8df1657f6b8d0eee1e06b9ce2b1b0a2de487377699382c1b057836e1571dac313ca878b5877c862f0461ba789a50b239d2a9f34accd8a6321f126e3d2a
5a1f57646b qa: clean up assert_memory_usage_stable utility (James O'Beirne)
0cf1632f03 qa: fix p2p_invalid_messages on macOS (James O'Beirne)
Pull request description:
Infinite mea culpa for the number of problems with this test.
This change bumps the acceptable RSS increase threshold from 3% to 50% when spamming the test node with junk 4MB messages. On [@MarcoFalke's macOS test build](https://travis-ci.org/MarcoFalke/btc_nightly) we see RSS grow ~14% from ~71MB to 81MB, so a 50% increase threshold should be more than sufficient to avoid spurious failures.
Tree-SHA512: 150a7b88080fd883c7a5d0b9ffa470f61a97c4885fccc1a06fde6260aaef15640a7c1de7e89c581b245df7807d617ec3d86775330386ec5149ad567492fc5d31
3d305e3b89 Send fewer spam messages in p2p_invalid_messages (James O'Beirne)
Pull request description:
Builds on travis are failing because the test node isn't
able to drop all the bad messages sent within the given
timeout. Reduce the number of bad messages we're sending
and increase the timeout to avoid failures on travis.
Tree-SHA512: 11c389619d9590caf7eca74e0efe6d950469415d34220072770689024b350cc08a2d5ec90634237d87ff71ba8b638c1152b8a45ffbb2815a48bde6a88fbb8fc6
d20a9fa13d1c13f552e879798c0508be70190e71 tests: add tests for invalid P2P messages (James O'Beirne)
62f94d39f8de88a44bb0a8a2837d864f777aaacc tests: add P2PConnection.send_raw_message (James O'Beirne)
5aa31f6ef26f51ce461c917654dd1cfbbdd1409a tests: add utility to assert node memory usage hasn't increased (James O'Beirne)
Pull request description:
- Adds `p2p_invalid_messages.py`: tests based on behavior for dealing with invalid and malformed P2P messages. Includes a test verifying that we can't DoS a node by spamming it with large invalid messages.
- Adds `TestNode.assert_memory_usage_stable`: a context manager that allows us to ensure memory usage doesn't significantly increase on a node during some test.
- Adds `P2PConnection.send_raw_message`: which allows us to construct and send messages with tweaked headers.
Tree-SHA512: 720a4894c1e6d8f1551b2ae710e5b06c9e4f281524623957cb01599be9afea82671dc26d6152281de0acb87720f0c53b61e2b27d40434d30e525dd9e31fa671f
13782b8ba8 docs: add perf section to developer docs (James O'Beirne)
58180b5fd4 tests: add utility to easily profile node performance with perf (James O'Beirne)
Pull request description:
Adds a context manager to easily (and selectively) profile node performance during functional test execution using `perf`.
While writing some tests, I encountered some odd bitcoind slowness. I wrote up a utility (`TestNode.profile_with_perf`) that generates performance diagnostics for a node by running `perf` during the execution of a particular region of test code.
`perf` usage is detailed in the excellent (and sadly unmerged) https://github.com/bitcoin/bitcoin/pull/12649; all due props to @eklitzke.
### Example
```python
with node.profile_with_perf("large-msgs"):
for i in range(200):
node.p2p.send_message(some_large_msg)
node.p2p.sync_with_ping()
```
This generates a perf data file in the test node's datadir (`/tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data`).
Running `perf report` generates nice output about where the node spent most of its time while running that part of the test:
```bash
$ perf report -i /tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data --stdio \
| c++filt \
| less
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 135 of event 'cycles:pp'
# Event count (approx.): 1458205679493582
#
# Children Self Command Shared Object Symbol
# ........ ........ ............... ................... ........................................................................................................................................................................................................................................................................
#
70.14% 0.00% bitcoin-net bitcoind [.] CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
|
---CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
70.14% 0.00% bitcoin-net bitcoind [.] CNetMessage::readData(char const*, unsigned int)
|
---CNetMessage::readData(char const*, unsigned int)
CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
35.52% 0.00% bitcoin-net bitcoind [.] std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
|
---std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
CNetMessage::readData(char const*, unsigned int)
CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
...
```
Tree-SHA512: 9ac4ceaa88818d5eca00994e8e3c8ad42ae019550d6583972a0a4f7b0c4f61032e3d0c476b4ae58756bc5eb8f8015a19a7fc26c095bd588f31d49a37ed0c6b3e
fa7d36b8e7 test: Move UBSAN suppressions to test/sanitizer_suppressions/ubsan (MarcoFalke)
fa36d4e456 travis: --disable-hardening for xenial thread sanitizer (MarcoFalke)
89bf196c88 travis: Run thread sanitizer (MarcoFalke)
Pull request description:
On unit tests only for now. Disabled for the gui unit tests and all functional tests.
Tree-SHA512: 56f7d3b44e7cb68c76a2dc5abd85658955b1c2188932e988667c5a1cbcdd6be995d37bb949d62c6eb08a4aebfc43ff0370b7da1719d4e4f322a3495c1941a5e0
* Merge #13743: refactor: Replace boost::bind with std::bind
cb53b825c2 scripted-diff: Replace boost::bind with std::bind (Chun Kuan Lee)
2196c51821 refactor: Use boost::scoped_connection in signal/slot, also prefer range-based loop instead of std::transform (Chun Kuan Lee)
Pull request description:
Replace boost::bind with std::bind
- In `src/rpc/server.cpp`, replace `std::transform` with simple loop.
- In `src/validation.cpp`, store the `boost::signals2::connection` object and use it to disconnect.
- In `src/validationinterface.cpp`, use 2 map to store the `boost::signals2::scoped_connection` object.
Tree-SHA512: 6653cbe00036fecfc495340618efcba6d7be0227c752b37b81a27184433330f817e8de9257774e9b35828026cb55f11ee7f17d6c388aebe22c4a3df13b5092f0
* Replace boost::bind with std::bind and remove Boost.Bind includes
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
* Merge 9a3a984bb
Merge #14272: init: Remove deprecated args from hidden args
fa910e4301 init: Remove deprecated args from hidden args (MarcoFalke)
Pull request description:
The args have been deprecated since 0.17 (maybe longer) and since we reject unknown args, there is no need to add deprecated args to the list of hidden args and then hand-craft an error message if a user provides them.
Tree-SHA512: 3a3191439ab0d7969fb72801d097bd86998524f84b3819380224f746cbe4b0f57beec1ad34744424f6587038035b0ddf418ad13171a8d9c3b97b4f3b7b3222a3
* Address review comment
Modified hidden_args in SetupServerArgs()
Removed usehd from list of unsupported/deprecated args
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
fa5278a419 qa: Use wallet to retrieve raw transactions (MarcoFalke)
fa2198328e qa: Style-only fixes in touched files (MarcoFalke)
Pull request description:
Instead of asking the coin database and block storage about a transaction, pull it directly from the wallet in wallet related tests.
This refactoring only makes sense in light of #15159.
<sub>This product may contain minor stylistic cleanups
Tree-SHA512: ec34c7150d873da9f19fead3f7e3f758baba5ef10061942384c470a47a6f320690109be9c5160f0c8bc228272a729653d44c78471455337318f657d6c164ba23
e6c58d3b014ab8ef5cca4be68764af4b79685fcb Do not import private keys to wallets with private keys disabled (Andrew Chow)
b5c5021b644731d14a6ef04961320a99466f035a Refactor importwallet to extract data from the file and then import (Andrew Chow)
1f77f6754ce724493b0cb084ae0b35107d58605f tests: unify RPC argument to cli argument conversion and handle dicts and lists (Andrew Chow)
Pull request description:
Fixes a bug where private keys could be imported to wallets with private keys disabled. Now every RPC which can import private keys checks for whether the wallet has private keys are disabled and errors if it is. Also added an belt-and-suspenders check to `AddKeyPubkeyWithDB` to have it assert that the wallet has private keys enabled.
Tree-SHA512: 5cd04febce9aa2bd9bfd02f312c6ff8705e37278cae59efd3895f6d6e2f1b477aefd297e2dd0860791bdd3d4f3cad8eb1a404f8f3d4e2035b91314ad2c1028ae
dash changes
4999992c34 whitespace: Split ~300 char line into multiple ones (MarcoFalke)
fa71b38168 scripted-diff: Rename rpc_timewait to rpc_timeout (MarcoFalke)
fa3e5786d0 scripted-diff: Remove unused 'split' parameter to setup_network (MarcoFalke)
Pull request description:
This is a bugfix, since wallet_dump currently uses the wrong name:
18857b4c40/test/functional/wallet_dump.py (L89-L92)
Rename all to the same name with a scripted diff (and some unrelated cleanups).
Tree-SHA512: 338ddd20dae12e6cf7aa7adbcfb239cf648017a1572b373f8431fecb184bd2a65492846d81e75a023864d9e41c94afb53044c16b79651a5937d34a5a6b772f81
d6b3790d1a tests: check readability of cookie file (Chun Kuan Lee)
Pull request description:
This PR would wait until the `.cookie` file is readable
Possible fix no. 5 `PermissionError` in #14446
Tree-SHA512: e7055c7ca26a6eadbbe19e4eef08ffee61cd17de79b30af2f0d090f0ad81ca24815e3c7e034e5e30d47c580bb0b221b3955e9ff2fcec2274fbf7b9232ab0cdc7
688f665a5e526fda0fb797bf617412fe9cbe64fd Scripts and tools & Docs: Used #!/usr/bin/env bash instead of obsolete #!/bin/bash, added linting for .sh files shebang and updated the Developer Notes. (vim88)
Pull request description:
As it was discussed in [#13510](https://github.com/bitcoin/bitcoin/pull/13510), it is better to use `#!/usr/bin/env bash` instead of `#!/bin/bash`.
Tree-SHA512: 25f71eb9a6a0cdc91568b5c6863205c5fe095f77a69e633503a2ac7805bd9013af8538e538c0c666ce96a28e3f43ce7a8df5f08d4ff007723bb588d85674f2da
43f909990190b3ff7883f0b2c117daa876d8fd99 scripted-diff: Run scripted-diff in subshell (Carl Dong)
Pull request description:
scripted-diffs should be run in subshells so that their execution does not
affect the shell variables of commit-script-check. Shell variables are not
unset before evaluating the scripted-diff, so that they might be used in
the subshell. To this end, the variable previously named i is now more
descriptively named commit, this also allows scripted-diffs to use the
commonly used variable i without fear of losing a reference to the
commit.
Tree-SHA512: 0d86c069c2a978ca07d71bcd2b1b273e9bfabfe7e31a50c7b1b860e04f178b81c65814c3a38fb01e50b41a5065b646f0dab5b05d9be71138e72d4baba607e37b
2012d4df2 Add CScriptNum decode python implementation in functional suite (Gregory Sanders)
Pull request description:
I needed this for reasons and thought it'd be good to upsteam it.
Tree-SHA512: 6ea89fa2a5f5a7759ba722f2b4ed5cd6423ebfff4e83ac8b8b5c935e6aa479684e626c5f41fa020816d2a9079a99af5564e30808594d5c13e3b51ec9b474926d
ed2e18398b3ab657e98e3e1fe135cbf8dd94fda3 Remove fs::relative call and fix listwalletdir tests (João Barbosa)
Pull request description:
The implementation of `fs::relative` resolves symlinks which is not
intended in ListWalletDir. The replacement does what is required, and
`listwalletdir` RPC tests are fixed accordingly.
Also, `fs::recursive_directory_iterator` iteration is fixed to build
with boost 1.47.
Based on #14559
Tree-SHA512: 1da516226073f195285d10d9d9648c90cce0158c5d1eb9c31217bb4abb575cd37f07c00787c5a850554d6120bbc5a3cbc5cb47d4488b32ac6bcb52bc1882d600
4dca7d0a98 appveyor: Enable multiwallet test (Chun Kuan Lee)
Pull request description:
Based on #14320
This PR enable multiwallet test on appveyor. Also re-enable symlink
tests on Windows which is available after Windows Vista.
I disable these tests in #13964 because I suppose that Windows does
not support symlink, but I was wrong.
Tree-SHA512: 852cd4dedf36ec9c34aff8926cb34e6a560aea0bb9170c7a2264fc292dbb605622d561568d8df39aeb90d3d2bb700901d218ea7e7c5e21d84827c40d6370b369
ca6d86c322 tests: Stop node before removing the notification file (Chun Kuan Lee)
Pull request description:
Stop node before removing the notification file to make sure the
command has been terminated. After then we could removing those files
safely and do not receive any permission error. (See #14446)
The permission error is Windows specific, documented in python doc:
>On Windows, attempting to remove a file that is in use causes an
exception to be raised
See https://docs.python.org/3/library/os.html#os.remove
Tree-SHA512: fbdabf3a9a838bb59ba207dd9e9fbdd87c702a99ad66bee0b2b1537f80f8630d22d9d5e9c4ded23a82a66bfc10989227fb024b27393425abe0e5a2ad4e4cbb82
67654b6405 tests: write the notification to different files to avoid race condition (Chun Kuan Lee)
Pull request description:
This PR change the behavior that `feature_notifications.py` would
write to different files instead of writing to the same file to avoid
race condition.
Tree-SHA512: 78406167cc6a3f570134b0ee76d2be1440bc1498cd7b1be72fae16d0ab86950e26ef3bf6008796016e5418231400c6492f0e062909dd882646541ecb7a70fb30
d56a0689354fb814510c6c393f3e07ac9362dc1f docs: Add release notes for listwalletdir RPC (João Barbosa)
0cb3cad166bbeb75e9cc1512286453f8e7d4f717 qa: Add tests for listwalletdir RPC (João Barbosa)
cc3377360c417780f5cbd7bd69b438817a9d60be rpc: Add listwalletdir RPC (João Barbosa)
d1b03b8e5f04a2cc9ebb985bd9a1aebd2068f757 interfaces: Add getWalletDir and listWalletDir to Node (João Barbosa)
fc4db35bfd78d85d6b52d5da3d89696160658450 wallet: Add ListWalletDir utility (João Barbosa)
Pull request description:
`ListWalletDir` returns all available wallets in the current wallet
directory.
Based on MeshCollider work in pull #11485.
Tree-SHA512: 5843e3dbd1e0449f55bb8ea7c241a536078ff6ffcaad88ce5fcf8963971d48c78600fbc4f44919523b8a92329d5d8a5f567a3e0ccb0270fdd27366e19603a716
c82190cdb6 tests: Add Python dead code linter (vulture) (practicalswift)
590a57fdec tests: Remove unused testing code (practicalswift)
Pull request description:
Add Python dead code linter (`vulture`) to Travis.
Rationale for allowing dead code only after explicit opt-in (via `--ignore-names`):
* Less is more :-)
* Unused code is by definition "untested"
* Unused code can be an indication of bugs/logical errors. By making the contributor aware of newly introduced unused code it gives him/her an opportunity to investigate if the unused code they introduce is malignant or benign :-)
* Unused code is hard to spot for humans and is thus often missed during manual review
* [YAGNI](https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it)
Based on #14312 to make linter job pass.
Tree-SHA512: 4c581df7c34986e226e4ade479e0d3c549daf38f4a4dc4564b25564d63e773a1830ba55d1289c771b1fa325483e8855b82b56e61859fe8e4b7dfa54034b093b6
8340763dc3278c828ee7d525de4c71b4615d17cc Check for codespell in lint-spelling.sh (Kristaps Kaupe)
Pull request description:
Similar check for `spellcheck` already exists in `lint-shell.sh`
ACKs for top commit:
practicalswift:
ACK 8340763dc3278c828ee7d525de4c71b4615d17cc -- diff looks correct
fanquake:
ACK 8340763dc3278c828ee7d525de4c71b4615d17cc - Test is the same as in `lint-shell.sh`.
Tree-SHA512: f590fc4eae391c5246e86cb70cf79431bd3763ac872f82d3b2222c3a095235ca046a7072c5e61bbaeb3ed6ecd5ff85979769ee8b585de4fb97ac8cfbbf73b956
b168dd30cf71ac176e271bc610b0b1a79ceaf075 Bugfix: QA: Run tests with UPnP disabled (Luke Dashjr)
Pull request description:
This replaces #16560 by adding `upnp=0` to `bitcoin.conf` rather than passing it to nodes.
> Needed for builds configured with --enable-upnp-default
You can test this change using:
```bash
./configure --enable-upnp-default && make -j6 && test/functional/test_runner.py feature_config_args.py
```
on master the test will fail without this change.
ACKs for top commit:
practicalswift:
ACK b168dd30cf71ac176e271bc610b0b1a79ceaf075 -- diff looks correct
Tree-SHA512: e639dd480dda2cffa19a679018c4bd7e4bd4d0f5e3001d6b407b833e3c166bde98b201063e267b8e45f8a20b0d53ec8bc028bec806b2357f9a7ba314cc4e2d07
fac2e6a6045e4ddd6b473f4f3ddbb69d9d6921f6 test: Fail early on disconnect in mininode.wait_for_* (MarcoFalke)
Pull request description:
The node might crash or disconnect when our mininode waits for data. Due to the crash, the data is guaranteed to never arrive and we can fail early with an assert
ACKs for top commit:
laanwj:
ACK fac2e6a6045e4ddd6b473f4f3ddbb69d9d6921f6
Tree-SHA512: 32ca844eb66bd70ea49103d51c76b953242b1886e0834d96fca8840fc984ff40346d0a799adf8f76b03514a783cb9cec69d45e00bdd328c5192c31b5d8d17af2
fa7dd88b71 test: Add test for unknown args (MarcoFalke)
Pull request description:
Currently uncovered.
Further reading:
* https://marcofalke.github.io/btc_cov/total.coverage/src/util/system.cpp.gcov.html
* Fail on unknown config file options #15021
ACKs for commit fa7dd8:
promag:
ACK fa7dd88b71a1c6641bd450fae29a4a31849b1afd, tests looks good to me.
hebasto:
ACK fa7dd88b71a1c6641bd450fae29a4a31849b1afd, I have tested the code.
Tree-SHA512: 86ab370ce8e85925f945a52e81457b5678d71bbabcef01205a97782b780003f363552e0bad1ff678bccc784f82c6b511c3b88de3f8f25f62b0b713c387950564
2e5d482659 tests: Print remaining jobs in test_runner.py (Steven Roose)
Pull request description:
This helps finding out which tests fail to finish.
Tree-SHA512: d22beb82beecd33aaa50731c83075e49577842d29fd21aa63bcb859df5da99069eba9cc16eed5d91dbba8fb0fdc317fb88b3b370c4d3917e9da1cd13b0a622dc
