50cede3f5a4d4fbfbb7c420b94e661a6a159bced [mempool] Allow one extra single-ancestor transaction per package (Matt Corallo)
Pull request description:
This implements the proposed policy change from [1], which allows
certain classes of contract protocols involving revocation
punishments to use CPFP. Note that some such use-cases may still
want some form of one-deep package relay, though even this alone
may greatly simplify some lightning fee negotiation.
[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-November/016518.html
ACKs for top commit:
ajtowns:
ACK 50cede3f5a4d4fbfbb7c420b94e661a6a159bced -- looked over code again, compared with previous commit, compiles, etc.
sdaftuar:
ACK 50cede3f5a4d4fbfbb7c420b94e661a6a159bced
ryanofsky:
utACK 50cede3f5a4d4fbfbb7c420b94e661a6a159bced. Changes since last review: adding EXTRA_DESCENDANT_TX_SIZE_LIMIT constant, changing max ancestor size from 1,000,000 to nLimitAncestorSize constant (101,000), fixing test comment and getting rid of unused test node.
Tree-SHA512: b052c2a0f384855572b4579310131897b612201214b5abbb225167224e4f550049e300b471dbf320928652571e92ca2d650050b7cf39ac92b3bc1d2bcd386c1c
* test: Add BRR and DAT unit tests
* test: Drop feature_block_reward_reallocation.py
* change copyright
* remove trivially removable includes
* use constexpr, remove empty statement
* Don't use BOOST_ASSERT, fix bug?
Not sure if this was a bug, as it does an assignment. If this isn't a bug please add a comment / explanation
```
BOOST_ASSERT(::ChainActive().Tip()->nVersion = 536870912);
```
* deduplicate all the things (also test all activation periods)
* use try_emplace, and remove some tempararies
* update threshold to be inline with dynamic
* explicitly include map, vector, remove now unneeded base58.h
* remove unused param, and replace raw loop with range loop
* re: Don't use BOOST_ASSERT, fix bug?
* Make TestChain<smth>Setup in dynamic_activation_thresholds_tests more general
* Specify min level activation tests correctly
Co-authored-by: Pasta <pasta@dashboost.org>
fa08c5cb99 test_runner: Move pruning back to extended (MarcoFalke)
Pull request description:
This reverts fafb55e2c2b257efd4e584f72adf62401c01d573, since the test is still too slow to run with asan enabled on a network hdd
ACKs for commit fa08c5:
jnewbery:
utACK fa08c5cb993f07fd4309f2a6bd9ef4696f07e24c
jonasschnelli:
utACK fa08c5cb993f07fd4309f2a6bd9ef4696f07e24c
Tree-SHA512: de16786b9d507a72210805c3e9eef360e5fc3d4bc3a81f7175b6cc70d1bc426cde7ac97bc0d1a0d4e0813067e1e251c2dd49256552cc6b52446b475251b7c32b
e91f0a7af2 doc: Remove travis badge from readme (MarcoFalke)
Pull request description:
The readme(s) are shipped in the released source-code archive, in which case the travis badge is useless since it doesn't link to the travis result of the correct commit/tag/branch. GitHub embeds the correct links for each tag or commit that ci ran on, so we don't need this link in the readme.
ACKs for commit e91f0a:
hebasto:
ACK e91f0a7af2aec7d924f00da25c69d8f46e0dd33d
Tree-SHA512: 860435a58b38a9bd0bc62a1e74b3a63c138c9a2f09008a090d5ecc7fd86fa908d2e5eda41d16606507a238d9488fa5323405364a9556b670684a2e4838aead2d
3c3e31c3a4 [tests] Add wallet-tool test (João Barbosa)
49d2374acf [tools] Add wallet inspection and modification tool (Jonas Schnelli)
Pull request description:
Adds an offline tool `bitcoin-wallet-tool` for wallet creation and maintenance.
Currently this tool can create a new wallet file, display information on an existing wallet, and run the salvage and zapwallettxes maintenance tasks on an existing wallet. It can later be extended to support other common wallet maintenance tasks.
Doing wallet maintenance tasks in an offline tool makes much more sense (and is potentially safer) than having to spin up a full node.
Tree-SHA512: 75a28b8a58858d9d76c7532db40eacdefc5714ea5aab536fb1dc9756e2f7d750d69d68d59c50a68e633ce38fb5b8c3e3d4880db30fe01561e07ce58d42bceb2b
fad0ce59e9 tests: Fail if RPC has been added without tests (MarcoFalke)
Pull request description:
Need to be run with --coverage
ACKs for commit fad0ce:
ryanofsky:
utACK fad0ce59e9154f9b7e61907a71c740a942c60282. New comment in travis.yml is the only change since last review.
Tree-SHA512: b53632dfe9865ec06991bfcba2fd67238bebbb866b355f09624eaf233257b2bca902caac6c24abb358b2f4c1c43f28ca75e30982765911e1a117102df65276d9
369244f654 utils: Fix broken Windows filelock (Chun Kuan Lee)
Pull request description:
Fix broken filelock on Windows, also add a test for this. It's a regression introduced by #13862.
Tree-SHA512: 15665b1930cf39ec71f3ab07def8e2897659f6fd4d2de749d63a5a8ec920e4a04282f12bc262f242b1b3d14d2dd9fa191ddbcf16a46fb927b5b2b14d9f6b5d01
c7b3e487f2 tests: exclude all tests with difference parameters (Chun Kuan Lee)
Pull request description:
Fix broken exclusion list in functional tests. See https://github.com/bitcoin/bitcoin/pull/14007#pullrequestreview-158309105
Tree-SHA512: b6c2b86fef13e3c00c695adaeeb3e47ee9b48877c71bc605d24201ce931b2ef3ae9f5f199071fa1ec5de2d7aadc478410094c380cc297922e683e9b2569cda03
fac95398366f644911b58f1605e6bc37fb76782d qa: Run all tests even if wallet is not compiled (MarcoFalke)
faa669cbcd1fc799517b523b0f850e01b11bf40a qa: Premine to deterministic address with -disablewallet (MarcoFalke)
Pull request description:
Currently the test_runner would exit if the wallet was not compiled into the Bitcoin Core executable. However, a lot of the tests run without the wallet just fine and there is no need to globally require the wallet to run the tests.
Tree-SHA512: 63177260aa29126fd20f0be217a82b10b62288ab846f96f1cbcc3bd2c52702437703475d91eae3f8d821a3149fc62b725a4c5b2a7b3657b67ffcbc81532a03bb
661ac15a4a appveyor: Run functional tests on appveyor (Chun Kuan Lee)
2148c36b6e tests: Make it possible to run functional tests on Windows (Chun Kuan Lee)
Pull request description:
This PR do the following things:
- Make functional tests compatible with Windows
- Print color output in functional tests for Windows 10
- Run util and functional tests on appveyor
- Do not run symlink tests on Windows
Note:
- The wallet_multiwallet.py fail is unrelated to the test framework, it's a bug related to c++ code or maybe dependencies. `bitcoind` would exit with 0xC0000005(Access violation) during shutdown occasionally. Disable this for now.
- Not using `--failfast` because this is still in experimental. We should track if there is any other error.
- Disable ZMQ tests because the python zmq library could cause access violation sometimes.
- Disable `feature_notifications` because Bitcoin Core handles the command in different thread, whicha can cause a race condition.
Tree-SHA512: b76db137d264e62a5c130e1cbca7a2ca002a7a0f4153fa0b92c1ea6c9c09ef0533e11c49bdbd566c472d8ff59f245758feb5e5a6ec6cb6bb66a1c67bab5fa48a
f13ad1cae0 modify test for memory locked in case locking pages failed at some point (Adam Jonas)
2fa85ebd1c add rpc_misc.py, mv test getmemoryinfo, add test mallocinfo (Adam Jonas)
Pull request description:
Creating the `rpc_misc.py` functional test file to add space for adding tests to a file that doesn't have a lot of coverage.
- Removing the `getmemoryinfo()` smoke test from wallet basic rather than moving it to keep the wallet decoupled. Feel like testing for reasonable memory allocation values should suffice.
- Adding coverage for `mallocinfo()`. Introduced standard lib XML parser since the function exports an XML string that describes the current state of the memory-allocation implementation in the caller.
Tree-SHA512: ced30115622916c88d1e729969ee331272ec9f2881eb36dee4bb7331bf633a6810a57fed63a0cfaf86de698edb5162e6a035efd07c89ece1df56b69d61288072
fae8b8bb1a qa: Add tool-prefix to functional test readme (MarcoFalke)
faf3d22725 test_runner: Remove unused --force option (MarcoFalke)
Pull request description:
When someone calls the script they already have all intention to call it, no need to specify a redundant `--force`.
The functional tests are still disabled on the travis windows cross builds, where they'd run into issues when run under Wine.
Tree-SHA512: ada0dd9b3c0cd28c5832a12c5e04c029dc3bfe5ddf366fd0abc24fb7914d2e0f0a873fe756ade7ba780a561abe9bc731838c289accc421deda481269e08514cd
d20a9fa13d1c13f552e879798c0508be70190e71 tests: add tests for invalid P2P messages (James O'Beirne)
62f94d39f8de88a44bb0a8a2837d864f777aaacc tests: add P2PConnection.send_raw_message (James O'Beirne)
5aa31f6ef26f51ce461c917654dd1cfbbdd1409a tests: add utility to assert node memory usage hasn't increased (James O'Beirne)
Pull request description:
- Adds `p2p_invalid_messages.py`: tests based on behavior for dealing with invalid and malformed P2P messages. Includes a test verifying that we can't DoS a node by spamming it with large invalid messages.
- Adds `TestNode.assert_memory_usage_stable`: a context manager that allows us to ensure memory usage doesn't significantly increase on a node during some test.
- Adds `P2PConnection.send_raw_message`: which allows us to construct and send messages with tweaked headers.
Tree-SHA512: 720a4894c1e6d8f1551b2ae710e5b06c9e4f281524623957cb01599be9afea82671dc26d6152281de0acb87720f0c53b61e2b27d40434d30e525dd9e31fa671f
2e5d482659 tests: Print remaining jobs in test_runner.py (Steven Roose)
Pull request description:
This helps finding out which tests fail to finish.
Tree-SHA512: d22beb82beecd33aaa50731c83075e49577842d29fd21aa63bcb859df5da99069eba9cc16eed5d91dbba8fb0fdc317fb88b3b370c4d3917e9da1cd13b0a622dc
3d2c7d6f94 Add regtest for JSON-RPC batch calls. (Daniel Kraft)
Pull request description:
This adds a new regtest file `interface_rpc.py`, containing a test for batch JSON-RPC requests. Those were previously not tested at all. Tests for basic requests are not really necessary, as those are used anyway in lots of other regtests.
The existing `interface_http.py` file is more about the underlying HTTP connection, so adding a new interface file for the JSON-RPC specific things makes sense.
Tree-SHA512: 7c7576004c8474e23c98f4bf25fb655328ba6bb73ea06744ebee1c0ffbb26bc132e621ae52955d51dab0803b322f8d711667626a777ac9b26003339c2484502f
fa43626611 test_runner: Remove travis specific code (MarcoFalke)
Pull request description:
The tests are no longer run on travis, but in a docker, developer machines or a windows vm.
The code was essentially dead for months now. Fix that by explicitly passing in `--ci` to the test runner on our docker and appveyor windows vm.
Tree-SHA512: 5d48693c03e8eb27536658ccf9ba738fe93a72abd4b72c80caac084b5b2cdffa77a1031a671eeefe70b71d63500f55917803d4be54d01849722afdccb700a9e6
fa78a2fc67 [tests] Test that nodes respond to getdata with notfound (MarcoFalke)
Pull request description:
If a node has not announced a tx at all, then it should respond to
getdata messages for that tx with notfound, to avoid leaking tx
origination privacy.
In the future this could be adjusted such that a node responds with
notfound when a tx has not been announced to us, but that seems
to be a more involved change. See e.g.
https://github.com/jnewbery/bitcoin/commits/pr14220.1
Tree-SHA512: 6244afa5bd5d8fec9b89dfc02c9958bc370195145a0f3715f33200d6cf73a376c94193d44bf4523867196e6591c53ede8f9b6a77cb296b48c114a117b8c8b1fa
42a995ae48 [tests] Remove rpc_zmq.py (John Newbery)
Pull request description:
rpc_zmq.py is racy and fails intermittently. Remove that test file and
move the getzmqnotifications RPC test into interface_zmq.py.
Tree-SHA512: 666c8f252f8a392deda1bd531e84fdc04bdae4eab09407657ade2b5fc0aeffa247735e20314236f56e4e3402476673f3b7538d6e09f5af6976021ba2377ce63c
fa8ced32a60dea37ac169241cf9a1f708ef46c4b doc: Mention blocksonly in reduce-traffic.md, unhide option (MarcoFalke)
fa320de79faaca2b088fcbe7f76701faa9bff236 test: Add test for p2p_blocksonly (MarcoFalke)
fa3872e7b4540857261aed948b94b6b2bfdbc3d1 test: Format predicate source as multiline on error (MarcoFalke)
fa1dce7329d3e74d46ab98b93772b1832a3f1819 net: Rename ::fRelayTxes to ::g_relay_txes (MarcoFalke)
Pull request description:
This is de-facto no longer hidden
ACKs for commit fa8ced:
jamesob:
utACK fa8ced32a6
Tree-SHA512: 474fbdee6cbd035ed9068a066b6056c1f909ec7520be0417820fcd1672ab3069b53f55c5147968978d9258fd3a3933fe1a9ef8e4f6e14fb6ebbd79701a0a1245
d117f4541d4717e83c9396273e92960723622030 Add test for setban (nicolas.dorier)
dc7529abf0197dccb876dc4a93cbdd2ad9f03e5c [Fix] Allow connection of a noban banned peer (nicolas.dorier)
Pull request description:
Reported by @MarcoFalke on https://github.com/bitcoin/bitcoin/pull/16248#discussion_r314026195
The bug would mean that if the peer connecting to you is banned, but whitelisted without specific permissions, it would not be able to connect to the node.
The solution is just to move the same line below.
ACKs for top commit:
Sjors:
Agree inline is more clear. utACK d117f45
MarcoFalke:
ACK d117f4541d4717e83c9396273e92960723622030
Tree-SHA512: 0fed39acb1e8db67bb0bf4c4de3ad034ae776f38d55bd661f1ae0e1a4c6becaf1824ab46ed8279f2f31df3f4b29ff56461d8b167d3e9cece62cfe58b5a912811
c5b404e8f1973afe071a07c63ba1038eefe13f0f Add functional tests for flexible whitebind/list (nicolas.dorier)
d541fa391844f658bd7035659b5b16695733dd56 Replace the use of fWhitelisted by permission checks (nicolas.dorier)
ecd5cf7ea4c3644a30092100ffc399e30e193275 Do not disconnect peer for asking mempool if it has NO_BAN permission (nicolas.dorier)
e5b26deaaa6842f7dd7c4537ede000f965ea0189 Make whitebind/whitelist permissions more flexible (nicolas.dorier)
Pull request description:
# Motivation
In 0.19, bloom filter will be disabled by default. I tried to make [a PR](https://github.com/bitcoin/bitcoin/pull/16176) to enable bloom filter for whitelisted peers regardless of `-peerbloomfilters`.
Bloom filter have non existent privacy and server can omit filter's matches. However, both problems are completely irrelevant when you connect to your own node. If you connect to your own node, bloom filters are the most bandwidth efficient way to synchronize your light client without the need of some middleware like Electrum.
It is also a superior alternative to BIP157 as it does not require to maintain an additional index and it would work well on pruned nodes.
When I attempted to allow bloom filters for whitelisted peer, my proposal has been NACKed in favor of [a more flexible approach](https://github.com/bitcoin/bitcoin/pull/16176#issuecomment-500762907) which should allow node operator to set fine grained permissions instead of a global `whitelisted` attribute.
Doing so will also make follow up idea very easy to implement in a backward compatible way.
# Implementation details
The PR propose a new format for `--white{list,bind}`. I added a way to specify permissions granted to inbound connection matching `white{list,bind}`.
The following permissions exists:
* ForceRelay
* Relay
* NoBan
* BloomFilter
* Mempool
Example:
* `-whitelist=bloomfilter@127.0.0.1/32`.
* `-whitebind=bloomfilter,relay,noban@127.0.0.1:10020`.
If no permissions are specified, `NoBan | Mempool` is assumed. (making this PR backward compatible)
When we receive an inbound connection, we calculate the effective permissions for this peer by fetching the permissions granted from `whitelist` and add to it the permissions granted from `whitebind`.
To keep backward compatibility, if no permissions are specified in `white{list,bind}` (e.g. `--whitelist=127.0.0.1`) then parameters `-whitelistforcerelay` and `-whiterelay` will add the permissions `ForceRelay` and `Relay` to the inbound node.
`-whitelistforcerelay` and `-whiterelay` are ignored if the permissions flags are explicitly set in `white{bind,list}`.
# Follow up idea
Based on this PR, other changes become quite easy to code in a trivially review-able, backward compatible way:
* Changing `connect` at rpc and config file level to understand the permissions flags.
* Changing the permissions of a peer at RPC level.
ACKs for top commit:
laanwj:
re-ACK c5b404e8f1973afe071a07c63ba1038eefe13f0f
Tree-SHA512: adfefb373d09e68cae401247c8fc64034e305694cdef104bdcdacb9f1704277bd53b18f52a2427a5cffdbc77bda410d221aed252bc2ece698ffbb9cf1b830577
d6d2602a32251c1017da88b47c801b7283c66ce3 add: test that transactions expire from mempool (0xb10c)
Pull request description:
This adds the functional test `mempool_expiry.py` covering mempool transaction expiry. Both the default `DEFAULT_MEMPOOL_EXPIRY` of 336 hours (two weeks, set in #9312) and the user definable mempool expiry via the `-mempoolexpiry=<n>` command line option are tested. The test checks that descendants of expired transactions are removed as well.
*Notes for reviewers*
- `LimitMempoolSize()` (which is the only caller of `CTxMemPool::Expire()`) is only called when a transaction is added to the mempool. In order to test expiry of a transaction-that-should-expire, the mocktime is set and a random transaction is broadcast to trigger `LimitMempoolSize()`. The transaction-that-should-expire is then checked for expiry. LMK if there is another way, but I don't think there is.
ACKs for top commit:
MarcoFalke:
ACK d6d2602a32251c1017da88b47c801b7283c66ce3
theStack:
ACK d6d2602a32
promag:
Code review ACK d6d2602a32251c1017da88b47c801b7283c66ce3.
Tree-SHA512: eb68cd9e2d870872b8e8e1522fed8954fb99cc9e4edda4b28bb2a4e41cddbc53fe6f7d9c090f1e0e98ab49beb24bf37ff3787a9e9801a95e8ae9ca9eb34fe6f0
a47df13471e3168e2e02023fb20cdf2414141b36 [qa] Test disconnect block failure -> shutdown (Suhas Daftuar)
4433ed0f730cfd60eeba3694ff3c283ce2c0c8ee [validation] Crash if disconnecting a block fails (Suhas Daftuar)
Pull request description:
If we're unable to disconnect a block during normal operation, then that is a
failure of our local system (such as disk failure) or the chain that we are on
(eg CVE-2018-17144), but cannot be due to failure of the (more work) chain that
we're trying to validate.
We should abort rather than stay on a less work chain.
Fixes#14341.
ACKs for top commit:
practicalswift:
utACK a47df13471e3168e2e02023fb20cdf2414141b36
TheBlueMatt:
utACK a47df13471e3168e2e02023fb20cdf2414141b36. Didn't bother to review the test in detail, it looked fine. Debated whether invalidateblock should ever crash the node, but *not* crashing in the case of hitting a pruned block (which is the only change here) is clearly better, even if there are other cases I'd argue we should crash in.
ryanofsky:
utACK a47df13471e3168e2e02023fb20cdf2414141b36. Only change since last review is new comment.
promag:
ACK a47df1347, it takes awhile to quit (RPC connection timeouts) but that's unrelated - hope to fix that soon.
fanquake:
ACK a47df13471e3168e2e02023fb20cdf2414141b36
Tree-SHA512: 4dec8cef6e7dbbe513c138fc5821a7ceab855e603ece3c16185b51a3830ab7ebbc844a28827bf64e75326f45325991dcb672f13bd7baede53304f27289c4af8d
e142ee03e7a139168aa1dbf5910c616f60d25042 doc: describe how to pass wildcard names to test runner (Jon Atack)
6a7a70b8cf05a82737c72020fd2b0eebc97cb5e4 test: enable passing wildcards with path to test runner (Jon Atack)
Pull request description:
Currently, passing wildcard testname args to the test runner from outside the test/functional/ directory does not work, even though developers expect it to. See these recent IRC discussions for more background: http://www.erisian.com.au/bitcoin-core-dev/log-2019-07-10.html#l-262 (lines 262 to 323) and http://www.erisian.com.au/bitcoin-core-dev/log-2019-07-11.html#l-134.
1. [BUGFIX] Enable passing wildcards with paths. Examples:
- `test/functional/test_runner.py test/functional/wallet*`
- `functional/test_runner.py functional/wallet*`
- `test/functional/test_runner.py ./test/functional/tool* test/functional/mempool*`
- A current limitation this PR does not change: 9 test files with arguments in their filename are not picked up by wildcard search.
2. [Docs] Describe how to pass wildcard names (multiple and with paths) to the test runner in test/README.md.
ACKs for top commit:
jnewbery:
tested ACK e142ee03e7a139168aa1dbf5910c616f60d25042
jachiang:
Tested ACK e142ee03e7. Thanks a lot for this fix!
MarcoFalke:
ACK e142ee03e7a139168aa1dbf5910c616f60d25042, fine with me
Tree-SHA512: cb3d994880cdc9b8918546b573a25faa5b4c7339826ac7cfe20f076aac6e731a34271609c0cf5a7ee5e4a2d5ae205298319d24bf36ef5b5d569a1a0c57883e54
5c40e7b91a test: allows test_runner command line to receive parameters for each test (marcoagner)
Pull request description:
Fixes#14791.
Seems to address the asked behaviour in a simple way, but could address more if a more complex behaviour is found to be needed (e.g. call `rpc_bind --ipv4` without the ".py" and have it added).
Tree-SHA512: d7e21dd0ada36dd97dac71196bb97702a92986b181beb3753e37e3294b899fb65129aff5f9a45fe92b06fbf7c74e605ccb87a422f874f7a85d30401f4c2228c8
fa85c985ed qa: Add p2p_invalid_locator test (MarcoFalke)
Pull request description:
Should not be merged *before* #13907
Tree-SHA512: a67ca407854c421ed20a184d0b0dc90085aed3e3431d9652a107fa3022244767e67f67e50449b7e95721f56906836b134615875f28a21e8a012eb22cfe6a66a5
5a05aa2db2 Add metavar to match var name in help text + Change wording for better readability (Martin Erlandsson)
Pull request description:
The help text given by `test/functional/test_runner.py -h` refers to the value `n`, which is defined as `COMBINEDLOGSLEN` in the list of commands.
To make the help text consistent, this PR changes the display name `COMBINEDLOGSLEN` to `n` by setting the argparse [`metavar`](https://docs.python.org/3/library/argparse.html#metavar) attribute. (`metavar` only changes the _displayed_ name)
Alternatively: Do the opposite and change the help text to use `COMBINEDLOGSLEN`.
---
Before PR:
```
➜ bitcoin > test/functional/test_runner.py -h | grep -A 1 combinedlogslen
--combinedlogslen COMBINEDLOGSLEN, -c COMBINEDLOGSLEN
print a combined log (of length n lines) from all test nodes and test framework to the console on failure.
```
After PR:
```
➜ bitcoin > test/functional/test_runner.py -h | grep -A 1 combinedlogslen
--combinedlogslen n, -c n
print a combined log (of length n lines) from all test nodes and test frameworks to the console on failure.
```
---
Also, fixed pluralization typo.
Tree-SHA512: a1124a4976d29fae1e8ecd7fa2ac523b7f05d541c611166532f44692995691a96faf797fa71582d78634f328b500cbee49c6ef296c8f1a898a57c050cc4e721d
