* Add .gitlab-ci.yml
* Use | instead of > for multiline commands
This honor new-lines and makes ; unnecessary
* Use ubuntu:bionic as base image
* Move cache initialization before apt-get installs
* Cache apt packages
* Move installation of wget and unzip up as we need it for the cache
* Prevent apt from deleting caches
* Collect test logs into artifact
* Make combine_logs.py always look for the template in the correct dir
* Move final cache stuff into after_script
* Reintroduce PYTHON_DEBUG=1, but only for .travis.yml
* Install jinja2 in Travis builder image
* Enable ChainLocks after quorums have been created
Creating 4 quorums causes a lot of blocks to be created and signed by
ChainLocks, which then causes timeouts later.
* Increase timeout in wallet-dump.py test
The first dumpwallet is quite slow sometimes, which then makes the
later called dumpwallet throw a wallet locked exception.
28479f926f21f2a91bec5a06671c60e5b0c55532 qa: Test bitcond shutdown (João Barbosa)
8d3f46ec3938e2ba17654fecacd1d2629f9915fd http: Remove timeout to exit event loop (João Barbosa)
e98a9eede2fb48ff33a020acc888cbcd83e24bbf http: Remove unnecessary event_base_loopexit call (João Barbosa)
6b13580f4e3842c11abd9b8bee7255fb2472b6fe http: Unlisten sockets after all workers quit (João Barbosa)
18e968581697078c36a3c3818f8906cf134ccadd http: Send "Connection: close" header if shutdown is requested (João Barbosa)
02e1e4eff6cda0bfc24b455a7c1583394cbff6eb rpc: Add wait argument to stop (João Barbosa)
Pull request description:
Fixes#11777. Reverts #11006. Replaces #13501.
With this change the HTTP server will exit gracefully, meaning that all requests will finish processing and sending the response, even if this means to wait more than 2 seconds (current time allowed to exit the event loop).
Another small change is that connections are accepted even when the server is stopping, but HTTP requests are rejected. This can be improved later, especially if chunked replies are implemented.
Briefly, before this PR, this is the order or events when a request arrives (RPC `stop`):
1. `bufferevent_disable(..., EV_READ)`
2. `StartShutdown()`
3. `evhttp_del_accept_socket(...)`
4. `ThreadHTTP` terminates (event loop exits) because there are no active or pending events thanks to 1. and 3.
5. client doesn't get the response thanks to 4.
This can be verified by applying
```diff
// Event loop will exit after current HTTP requests have been handled, so
// this reply will get back to the client.
StartShutdown();
+ MilliSleep(2000);
return "Bitcoin server stopping";
}
```
and checking the log output:
```
Received a POST request for / from 127.0.0.1:62443
ThreadRPCServer method=stop user=__cookie__
Interrupting HTTP server
** Exited http event loop
Interrupting HTTP RPC server
Interrupting RPC
tor: Thread interrupt
Shutdown: In progress...
torcontrol thread exit
Stopping HTTP RPC server
addcon thread exit
opencon thread exit
Unregistering HTTP handler for / (exactmatch 1)
Unregistering HTTP handler for /wallet/ (exactmatch 0)
Stopping RPC
RPC stopped.
Stopping HTTP server
Waiting for HTTP worker threads to exit
msghand thread exit
net thread exit
... sleep 2 seconds ...
Waiting for HTTP event thread to exit
Stopped HTTP server
```
For this reason point 3. is moved right after all HTTP workers quit. In that moment HTTP replies are queued in the event loop which keeps spinning util all connections are closed. In order to trigger the server side close with keep alive connections (implicit in HTTP/1.1) the header `Connection: close` is sent if shutdown was requested. This can be tested by
```
bitcoind -regtest
nc localhost 18443
POST / HTTP/1.1
Authorization: Basic ...
Content-Type: application/json
Content-Length: 44
{"jsonrpc": "2.0","method":"stop","id":123}
```
Summing up, this PR:
- removes explicit event loop exit — event loop exits once there are no active or pending events
- changes the moment the listening sockets are removed — explained above
- sends header `Connection: close` on active requests when shutdown was requested which is relevant when it's a persistent connection (default in HTTP 1.1) — libevent is aware of this header and closes the connection gracefully
- removes event loop explicit break after 2 seconds timeout
Tree-SHA512: 4dac1e86abe388697c1e2dedbf31fb36a394cfafe5e64eadbf6ed01d829542785a8c3b91d1ab680d3f03f912d14fc87176428041141441d25dcb6c98a1e069d8
* Use `wait_until` in sporks.py
* Drop unused spork functions in `dip3-deterministicmns.py`
* Use `wait_until` in multikeysporks.py
* scripted-diff: Rename `*_test_spork_state` to `*_test_spork_value` in multikeysporks.py to better match actual functionality
-BEGIN VERIFY SCRIPT-
sed -i 's/_test_spork_state/_test_spork_value/g' test/functional/multikeysporks.py
-END VERIFY SCRIPT-
* Drop `get_mnsync_status`, `wait_to_sync` and `sync_masternodes` and introduce `force_finish_mnsync` for MNs only
* Use `force_finish_mnsync` from util.py in dip3-deterministicmns.py and drop local unused functions
Also move the call, `force_finish_mnsync` should be called before `connect_nodes_bi`
e97b113b04 [tests] Change invalidblockrequest to use BitcoinTestFramework (John Newbery)
2b7064eda7 [tests] Fix flake8 warnings in invalidblockrequest (John Newbery)
54b8c580b7 [test] Fix nits leftover from 11771 (Conor Scott)
Pull request description:
Builds on #11771. Please review that PR first
Next step in #10603.
- first commit tidies up invalidblockrequest.py
- second commit removes usage of ComparisonTestFramework
Tree-SHA512: 14b10c09c8c0ebef4a9176eb5b883a275d04c096785ee31b84ef594eed346ec6344d7ed32184c5fb397e744725df3911f45cdfadd0810e5a52eaa256084e3456
95e2e9a [tests] Change invalidtxrequest to use BitcoinTestFramework (John Newbery)
359d067 [tests] Fix flake8 warnings in invalidtxrequest (John Newbery)
c32cf9f [tests] Add P2PDataStore class (John Newbery)
cc046f6 [tests] Reduce NodeConn connection logging from info to debug (John Newbery)
Pull request description:
Next step in #10603
- first commit changes log level for an internal log from INFO to DEBUG. (Not really related, but I started finding the INFO level logging annoying when debuging test failures)
- second commit introduces a `P2PStub` class - a subclass of `NodeConnCB` which has its own block and tx store and responds appropriately to getdata requests. Not all the functionality is used in `invalidtxrequest.py`, but will be used in `invalidblockrequest.py` and `p2p-fullblocktest` when those are changed to use `BitcoinTestFramework`
- third commit tidies up `invalidtxrequest.py`
- fourth commit removes usage of `ComparisonTestFramework`
Tree-SHA512: f3085c73c15d6ce894e401490bce8a7fa7cf52b0c9d135ff7e351f1f6f517c99accab8588fcdc443f39ea8315329aaabd66b2baa32499df5a774737882030373
5c8ff26 [tests] Add NetworkThread assertions (John Newbery)
34e08b3 [tests] Fix network threading in functional tests (John Newbery)
74e64f2 [tests] Use network_thread_start() in tests. (John Newbery)
5fc6e71 [tests] Add network_thread_ utility functions. (John Newbery)
Pull request description:
Add assert that only one NetworkThread exists at any time in functional tests, and fix cases where that wasn't true.
fixes#11776
Tree-SHA512: fe5d1c59005f94bf66e11bb23ccf274b1cd9913741b56ea11dbcd21db4cc0b53b4413c0c4c16dbcd6ac611adad5e5cc2baaa39720598ce7b6393889945d06298
faaa7db qa: Only allow disconnecting all NodeConns (MarcoFalke)
Pull request description:
Disconnecting the connection with `index=0` makes no sense when there are more than one connections, as the list "rotates around" and populates index 0 after `del`.
Just disconnect all NodeConns in any case.
Tree-SHA512: e5cf540823fccb31634b5a11501f54222be89862e80ccafc28bc06726480f8d2153b8c1b6f859fa6a6d087876251d48a6c6035bccdaaf16831e300bc17ff613d
32ae82f5c [tests] use TestNode p2p connection in tests (John Newbery)
5e5725cc2 [tests] Add p2p connection to TestNode (John Newbery)
b86c1cd20 [tests] fix TestNode.__getattr__() method (John Newbery)
Pull request description:
Final two steps of #10082 : Adding the "mininode" P2P interface to `TestNode`
This PR adds the mininode P2P interface to `TestNode`. It simplifies the process for opening a P2P connection to the node-under-test from this:
```python
node0 = NodeConnCB()
connections = []
connections.append(NodeConn('127.0.0.1', p2p_port(0), self.nodes[0], node0))
node0.add_connection(connections[0])
```
to this:
```python
self.nodes[0].add_p2p_connection(p2p_conn_type=NodeConnCB)
```
The first commit adds the infrastructure to `test_node.py`. The second updates the individual test cases to use it. Can be separated if this is too much review for one PR.
Tree-SHA512: 44f1a6320f44eefc70489ae8350c6a16ad1a6035e4b9b7bafbdf19f5905ed0e2db85beaaf4758eec3059dd89a375a47a45352a029f39f57a86ab38a9ae66650e
* Add timeout params to wait_for*_chainlock methods
* Give chainlocks more time in specific case
* Add logs to llmq-chainlock.py
* Replace wait_for_chainlocked_tip_all_nodes with wait_for_chainlocked_block_all_nodes
wait_for_chainlocked_tip_all_nodes did wait for the tip of each individual
node, which would not necessarily be the same. We should only allow to
explicitly specify which block to wait for.
* Get rid of wait_for_chainlocked_tip
Same as with wait_for_chainlocked_tip_all_nodes
This speeds up assumevalid.py from 22s to 7s on my machine. On travis, this
should be an improvement of a few minutes. Without this, Travis actually
fails due to block download timeouts.
* scripted-diff: Rename `wait_for_chainlock*` test functions
-BEGIN VERIFY SCRIPT-
sed -i 's/wait_for_chainlock_tip_all_nodes(/wait_for_chainlocked_tip_all_nodes(/g' test/functional/*.py
sed -i 's/wait_for_chainlock_tip(/wait_for_chainlocked_tip(/g' test/functional/*.py
sed -i 's/wait_for_chainlock(/wait_for_chainlocked_block(/g' test/functional/*.py
sed -i 's/wait_for_chainlock /wait_for_chainlocked_block /g' test/functional/*.py
-END VERIFY SCRIPT-
* Move `wait_for_*chainlock*` functions from individual tests to DashTestFramework
* Use `wait_until` in most Dash-specific `wait_for*` functions instead of custom timers
* Cleanup p2p-instantsend.py
Bump mocktime before generating new blocks and generate a few blocks at the end of test_mempool_doublespend to clean things up
* Update test/functional/p2p-instantsend.py
Co-Authored-By: PastaPastaPasta <6443210+PastaPastaPasta@users.noreply.github.com>
* Fix `wait_for_instantlock` to make it fail if instantlock wasn't aquired, use `wait_until`
Currently it simply returns False if islock failed but that's not the way we use it (we never check results).
* Wait for txes to propagate before checking for instantlock
4526d21 Add test for multiwallet batch RPC calls (Russell Yanofsky)
74182f2 Add missing batch rpc calls to python coverage logs (Russell Yanofsky)
505530c Add missing multiwallet rpc calls to python coverage logs (Russell Yanofsky)
9f67646 Make AuthServiceProxy._batch method usable (Russell Yanofsky)
e02007a Limit AuthServiceProxyWrapper.__getattr__ wrapping (Russell Yanofsky)
edafc71 Fix uninitialized URI in batch RPC requests (Russell Yanofsky)
Pull request description:
This fixes "Wallet file not specified" errors when making batch wallet RPC calls with more than one wallet loaded. This issue was reported by @NicolasDorier in https://github.com/bitcoin/bitcoin/issues/11257
Request URI is not used for anything except multiwallet request dispatching, so this change has no other effect.
Tree-SHA512: b3907af48a6323f864bb045ee2fa56b604188b835025ef82ba3d81673244c04228d796323cec208a676e7cd578a95ec7c7ba1e84d0158b93844d5dda8f6589b9
659b206 Make listsinceblock refuse unknown block hash (Russell Yanofsky)
Pull request description:
Change suggested by @theuni who noticed listsinceblock would ignore invalid block hashes causing it to return a completely unfiltered list of transactions.
Tree-SHA512: 3c8fb160265780d1334e856e853ab48e2e18372b8f1fc71ae480c3f45317048cc1fee0055d5c58031981a91b9c2bdbeb8e49a889d04ecba61729ce8109f2ce3f
f3d4adf Make p2p-acceptablock not an extended test (Matt Corallo)
00dcda6 [qa] test that invalid blocks on an invalid chain get a disconnect (Matt Corallo)
015a525 Reject headers building on invalid chains by tracking invalidity (Matt Corallo)
932f118 Accept unrequested blocks with work equal to our tip (Matt Corallo)
3d9c70c Stop always storing blocks from whitelisted peers (Matt Corallo)
3b4ac43 Rewrite p2p-acceptblock in preparation for slight behavior changes (Matt Corallo)
Pull request description:
@sdaftuar pointed out that the version in #11487 was somewhat DoS-able as someone could feed you a valid chain that forked off the the last checkpoint block and force you to do lots of work just walking backwards across blocks for each new block they gave you. We came up with a few proposals but settled on the one implemented here as likely the simplest without obvious DoS issues. It uses our existing on-load mapBlockIndex walk to make sure everything that descends from an invalid block is marked as such, and then simply caches blocks which we attempted to connect but which were found to be invalid. To avoid DoS issues during IBD, this will need to depend on #11458.
Includes tests from #11487.
Tree-SHA512: 46aff8332908e122dae72ceb5fe8cd241902c2281a87f58a5fb486bf69d46458d84a096fdcb5f3e8e07fbcf7466232b10c429f4d67855425f11b38ac0bf612e1
e065249 Add unit test for outbound peer eviction (Suhas Daftuar)
5a6d00c Permit disconnection of outbound peers on bad/slow chains (Suhas Daftuar)
c60fd71 Disconnecting from bad outbound peers in IBD (Suhas Daftuar)
Pull request description:
The first commit will disconnect an outbound peer that serves us a headers chain with insufficient work while we're in IBD.
The second commit introduces a way to disconnect outbound peers whose chains fall out of sync with ours:
For a given outbound peer, we check whether their best known block (which is known from the blocks they announce to us) has at least as much work as our tip. If it doesn't, we set a 20 minute timeout, and if we still haven't heard about a block with as much work as our tip had when we set the timeout, then we send a single getheaders message, and wait 2 more minutes. If after two minutes their best known block has insufficient work, we disconnect that peer.
We protect 4 of our outbound peers (who provide some "good" headers chains, ie a chain with at least as much work as our tip at some point) from being subject to this logic, to prevent excessive network topology changes as a result of this algorithm, while still ensuring that we have a reasonable number of nodes not known to be on bogus chains.
We also don't require our peers to be on the same chain as us, to prevent accidental partitioning of the network in the event of a chain split. Note that if our peers are ever on a more work chain than our tip, then we will download and validate it, and then either reorg to it, or learn of a consensus incompatibility with that peer and disconnect. This PR is designed to protect against peers that are on a less work chain which we may never try to download and validate.
Tree-SHA512: 2e0169a1dd8a7fb95980573ac4a201924bffdd724c19afcab5efcef076fdbe1f2cec7dc5f5d7e0a6327216f56d3828884f73642e00c8534b56ec2bb4c854a656
01b52ce Add comment explaining forced processing of compact blocks (Suhas Daftuar)
08fd822 qa: add test for minchainwork use in acceptblock (Suhas Daftuar)
ce8cd7a Don't process unrequested, low-work blocks (Suhas Daftuar)
Pull request description:
A peer could try to waste our resources by sending us unrequested blocks with
low work (eg to fill up our disk). Since e265200 we no longer request blocks until we
know we're on a chain with more than nMinimumChainWork (our anti-DoS
threshold), but we would still process unrequested blocks that had more work
than our tip (which generally has low-work during IBD), even though we may not
yet have found a headers chain with sufficient work.
Fix this and add a test.
Tree-SHA512: 1a4fb0bbd78054b84683f995c8c3194dd44fa914dc351ae4379c7c1a6f83224f609f8b9c2d9dde28741426c6af008ffffea836d21aa31a5ebaa00f8e0f81229e
eac64bb7a [qa] Test nMinimumChainWork (Suhas Daftuar)
0311836f6 Allow setting nMinimumChainWork on command line (Suhas Daftuar)
Pull request description:
As discussed briefly here: https://botbot.me/freenode/bitcoin-core-dev/2017-02-28/?msg=81712308&page=4
This adds a hidden command line option for setting `nMinimumChainWork`, which allows us to test this parameter in our functional tests, as well as allowing for niche use cases like syncing nodes that are otherwise disconnected from the network.
See also #10345, which proposes a new use of `nMinimumChainWork`.
Tree-SHA512: fe4d8f4f289697615c98d8760f1cc74c076110310ea0b5b875fcab78c127da9195b4eb84148aebacc7606c246e5773d3f13bd5d9559d0a8bffac20a3a28c62df
fafa003 qa: Remove never used return value of sync_with_ping (MarcoFalke)
fa9de37 qa: Make tmpdir option an absolute path (MarcoFalke)
Pull request description:
This should fix issues with the multiwallet test and its symlinks
when the tmpdir is a relative path.
Rather than fixing os.symlink to work with paths relative to a
directory descriptor, which does not work on Windows, normalize
the path instead.
Tree-SHA512: 189690f3d065ea2f0f48e06775c86d513d0916c7c86312432e8e16df160e65539e288c2bd53d49a4180735fa940f6fcd52b506ccd7d9815651a9b1a69850dda6
478a89c Avoid opening copied wallet databases simultaneously (Russell Yanofsky)
Pull request description:
Make sure wallet databases have unique fileids. If they don't, throw an error. BDB caches do not work properly when more than one open database has the same fileid, because values written to one database may show up in reads to other databases.
Bitcoin will never create different databases with the same fileid, but users can create them by manually copying database files.
BDB caching bug was reported by @dooglus in https://github.com/bitcoin/bitcoin/issues/11429
Tree-SHA512: e7635dc81a181801f42324b72fe9e0a2a7dd00b1dcf5abcbf27fa50938eb9a1fc3065c2321326c3456c48c29ae6504353b02f3d46e6eb2f7b09e46d8fe24388d
5d465e396 Ensure backupwallet fails when attempting to backup to source file (Tomas van der Wansem)
Pull request description:
Previous behaviour was to destroy the wallet (to zero-length)
This fixes#11375
Tree-SHA512: bfd1738659b15e3f23b6bbdf55ec12269c62c820bf701daec19500b52bd5845bb5516733c6f76f36197eb155182a8a35dc239ad4de2ef1e59bbb0f124a455759
* Make sure chainlocks and blocks are propagated in llmq-is-cl-conflicts.py before moving to next steps
Add few comments here and there to clarify what's actually going on under the hood
* Simplify
aa57590d7 Update importprivkey named args documentation (Dusty Williams)
Pull request description:
Addresses issue #11462 by updating the documentation for the importprivkey arguments to the correct names, and updates the functional test importprunedfunds.py to use named arguments when calling importprivkey.
Tree-SHA512: 64e14bf89c8c6eec9c37f6ec0c9fc0012fdb035d9ec32cd652110c75abaa922ec5c7523d6ec5098c8a7b42124159b5e330e070974eb79b8b92816f8d61074523
0cd9273 rpc: Prevent `dumpwallet` from overwriting files (Wladimir J. van der Laan)
Pull request description:
Prevent arbitrary files from being overwritten by `dumpwallet`. There have been reports that users have overwritten wallet files this way. It may also avoid other security issues.
Fixes#9934. Adds mention to release notes and adds a test.
Tree-SHA512: 268c98636d40924d793b55a685a0b419bafd834ad369edaec08227ebe26ed4470ddea73008d1c4beb10ea445db1b0bb8e3546ba8fc2d1a411ebd4a0de8ce9120
fafff1220 qa: Restore bitcoin-util-test py2 compatibility (MarcoFalke)
Pull request description:
Currently `./configure && make check` will look for python3, then python2. As long as we support python2 (and use it as fallback), `make check` should run fine with both python2 and python3.
Fixes#11352 by @Zenitur
Tree-SHA512: a335ebdd224328d6f924fe52a9b97de196926476c9ee04ce3280743ea93bcae355eb2d5d4bed4050c01b2e904105595eac7db2eaa9307207581caa0a98ebcc0b
49f869f Fix bip68-sequence rpc test (Johnson Lau)
Pull request description:
The test mined 1 extra block for the ACTIVE state. Test added to catch the right moment of LOCKED_IN->ACTIVE transaction
Tree-SHA512: a42477cf0b137e7e3b7c6c7b2530101cfad4e4f59866170b8fc0d655c43b3144aad6bca4287a4a8df4c28d7cf08d3f8df166975ad2e8dcb7d2cc15de60cf11cd
fb1f32582 Add listwallets RPC test to multiwallet.py (Cristian Mircea Messel)
Pull request description:
Tree-SHA512: acf3377ac86fcc4e7fe36e3b97120f3e51b7bd94cb7136ce22e7e2043ce8ec5555c246c41f8501272aeb1e56ad16d05eb8c4eeb6acf3315a218c1ed4e5844871
8fdb6f912 [tests] fixup dbcrash interaction with add_nodes() (John Newbery)
Pull request description:
Another conflict with #11121. Apologies - this is entirely my fault. I didn't run the extended test suite after rebasing on master.
@MarcoFalke @sdaftuar
Tree-SHA512: eefce1d1c63dc4a63c5e030a541e046ad4832e8a709c0a8aad40ffdc4712b2065486778b406dfa57cfd34e66db86064278ee3fea8f2c2afd2390772875e6fa3e
faa8d9581 [qa] TestNode: Add wait_until_stopped helper method (MarcoFalke)
Pull request description:
This adds a helper method `wait_until_stopped` to the `TestNode` class. This should prevent numerous `time.sleep()` over all places.
Additionally, the timeout behavior is restored. (Was removed by the introduction of `TestNode`.)
This should prevent tests from running indefinitely by accident.
Tree-SHA512: 7133fc64d55711869c4e372e9d30625c98f1237fb3578c24a26900d9319831f10eb95592d7b08e536fa706158dffb0abf9197f11c5d9ef605c880628e1a6533f
b3d6fc654 Improve signmessages functional test (Cristian Mircea Messel)
Pull request description:
This patch improves branch coverage of the test, making sure a message can not be verified with the wrong address or signature.
Tree-SHA512: 984eb66af8ba1caaccfb8ce2c5cc89c634a50d2fa587bfaf6e196aaf5b89e77accd76522ac33105621ead795521ec6d90f0afcb3b6064d22e53b54d2b41c7991
3918d93f3 [tests] fixups from set_test_params() (John Newbery)
Pull request description:
#11121 had a silent merge conflict in `bitcoin_cli.py`. This fixes it.
Also fixes a comment in `example_test.py`
Tree-SHA512: f22a645c51c9aeda005526338ad6f2ee07f2bab172847fc54f51fecf1c79e778970be61e5e637d4e0aba3a02e5aba0737b0b57f1bc11a514a1acd94c221b54d6
7148b74dc [tests] Functional tests must explicitly set num_nodes (John Newbery)
5448a1471 [tests] don't override __init__() in individual tests (John Newbery)
6cf094a02 [tests] Avoid passing around member variables in test_framework (John Newbery)
36b626867 [tests] TestNode: separate add_node from start_node (John Newbery)
be2a2ab6a [tests] fix - use rpc_timeout as rpc timeout (John Newbery)
Pull request description:
Some additional tidyups after the introduction of TestNode:
- commit 1 makes TestNode use the correct rpc timeout. This should have been included in #11077
- commit 2 separates `add_node()` from `start_node()` as originally discussed here: https://github.com/bitcoin/bitcoin/pull/10556#discussion_r121161453 with @kallewoof . The test writer no longer needs to assign to `self.nodes` when starting/stopping nodes.
- commit 3 adds a `set_test_params()` method, so individual tests don't need to override `__init__()` and call `super().__init__()`
Tree-SHA512: 0adb030623b96675b5c29e2890ce99ccd837ed05f721d0c91b35378c5ac01b6658174aac12f1f77402e1d38b61f39b3c43b4df85c96952565dde1cda05b0db84
4f2905b76 Add getmininginfo functional test (Cristian Mircea Messel)
Pull request description:
Add `getmininginfo` functional test in `mining.py`
Tree-SHA512: 12be9cfb37e9ac4c6625fc06051704c8a8dfd7271c2654f994c7659c8810e4b7a4335105ae159315308bcd45b65589bab1829bd134d2f4cabf74d63f2e5d22fe
6d2d2eb49 RPC: gettxout: Slightly improve doc and tests (Jorge Timón)
Pull request description:
Slightly related to https://github.com/bitcoin/bitcoin/pull/10822 in the sense that I felt the documentation and testing wasn't as good as it could be while writing it.
Ping @sipa since we discussed this on IRC.
Tree-SHA512: a0b3ffdac65245a0429e772fc2d8bcc1e829b02c70fb2af6ee0b7578cae46683f6c51a824b4d703d4dc3f99b6f03a658d6bbc818bf32981516f24124249a211d
c6ec4358a [tests] Add bitcoin_cli.py test script (John Newbery)
b23549f6e [tests] add TestNodeCLI class for calling bitcoin-cli for a node (John Newbery)
Pull request description:
We don't test bitcoin-cli at all. That means that we can miss inconsistencies between the bitcoin-cli client and the RPC interface, such as #10698 and #10747. It also means that the various bitcoin-cli options and features are untested and regressions could be silently introduced.
Let's fix that.
This PR adds bitcoin-cli testing in the python functional test_framework:
1. Add a bitcoin_cli.py test script that tests bitcoin-cli. At the moment it only tests that the result of `getinfo` is the same if you run it as an RPC or through bitcoin-cli, but can easily be extended to test additional bitcoin-cli features
**EDIT: `--usecli` option is moved to a separate PR. This PR now only covers the bitcoin_cli.py test.**
2. ~Add a `--usecli` option to the test framework. This changes the test to use bitcoin-cli for all RPC calls instead of using direct HTTP requests. This is somewhat experimental. It works for most tests, but there are some cases where it can't work transparently because:~
- ~the testcase is asserting on a specific error code, and bitcoin-cli returns a different error code from the direct RPC~
- ~we're sending a very large RPC request (eg `submitblock`) and it can't be serialized into a shell bitcoin-cli call.~
~I think that even though `--usecli` doesn't work on all tests, it's still a useful experimental feature. Future potential enhancements:~
- ~enhance the framework to automatically skip tests that are known to fail with bitcoin-cli if the `--usecli` option is used.~
- ~run a subset of tests in Travis with `-usecli`~
This builds on and requires the `TestNode` PR #10711 . As an aside, this is a good demonstration of how tidy it is to add additional features/interfaces now that test node logic/state is encapsulated in a TestNode class.
Addresses #10791
Tree-SHA512: a1e6be12e8e007f6f67b3d3bbcd142d835787300831eb38e6027a1ad25ca9d79c4bc99a41b19e31ee95205cba1b3b2d21a688b5909316aad70bfc2b4eb6d8a52
08ce33f8e qa: Move wait_until to util (MarcoFalke)
Pull request description:
This moves `wait_until` to `util.py` to make it generally available to python tests.
Also, `wait_until` now takes an optional lock that is acquired while testing the predicate.
Previously the lock was always acquired, even when it was not necessary, cf. `disconnect_ban.py`.
Tree-SHA512: 18e452a017a6566fa8ad09bde058e1b841e167039dc63299e70cfa7a6dcbc779581e60ca3e8eb2f1b610767d5208b9376c203eb11015b250fd0542b5eb4215a8
2b4ea52 [tests] fix timeout issues from TestNode (John Newbery)
Pull request description:
Fixes a couple of bugs from the introduction of TestNode:
- test scripts were no longer able to specify a custom timeout for
starting a node. Therefore tests with nodes that take a long time to
start up (eg pruning.py) would fail.
- the test for whether a node has failed on start up was broken
by changing 'assert x is None' to 'assert not x'. Since
subprocess.poll() can return None (indicating the node is still running)
or 0 (indicating the node exited with return code 0), this was a
regression.
Tree-SHA512: 42a62a5459eea2e5d83b44dae2a5ccc7b15eb7fef8f8745ff04884dbba8f79d66ffdd65c67d37f6865b36da3f522bcdd0d6ea99861d7ce86dd8a56dc29cd643f
c1470a0 test: Increase initial RPC timeout to 60 seconds (Wladimir J. van der Laan)
Pull request description:
When running the tests locally with a parallelism of 4 on an otherwise busy system, RPC can take quite a wait to come up.
With the current timeout tests often fail with "Unable to connect to bitcoind".
Change the timeout to 60 seconds just to be safe.
Tree-SHA512: 0c08cc8ce3f25ba2882beac2a50d1fcdd7c8c3bd6e3a8707813f94f2d39c14e2139ba1ddf7f9b66013d4c7f55db92d3f4aa88b433d855fd21e82842e350e459a
6b4f231 Move transaction combining from signrawtransaction to new RPC (Andrew Chow)
Pull request description:
Create a combinerawtransaction RPC which accepts a json array of hex raw transactions to combine them into one transaction. Signrawtransaction is changed to no longer combine transactions and only accept one transaction at a time.
The tests have been updated to test this. Tests for the signrawtransaction merge have also been removed.
This is part of #10570
Tree-SHA512: 035aebbd6537c1c017d5c8e06d309228b4c23fe52d5b31ffde19741c81a11a6346ddbbdc582b77b02a47f4c22b1952b69d3c2ee1109c29b3f0f1b612d8de53ed
* Introduce getbestchainlock rpc and fix llmq-is-cl-conflicts.py
* Add `known_block` field and move `getbestchainlock` to `blockchain` rpc category
* Add CChainLockSig::IsNull() and throw an exception in getbestchainlock if there is no known chainlock yet
* drop blockHash initializer
"subtractFeeFromOutputs" is applied to the ordering of the input
transaction and after that BIP69 sorting is performed. This causes
flakiness in tests.
d34957e [wallet] [tests] Add keypool topup functional test (Jonas Schnelli)
095142d [wallet] keypool mark-used and topup (John Newbery)
c25d90f [wallet] Add HasUnusedKeys() helper (John Newbery)
f2123e3 [wallet] Cache keyid -> keypool id mappings (John Newbery)
83f1ec3 [wallet] Don't hold cs_LastBlockFile while calling setBestChain (John Newbery)
2376bfc [wallet] [moveonly] Move LoadKeyPool to cpp (Matt Corallo)
cab8557 [wallet] [moveonly] Move CAffectedKeysVisitor (Jonas Schnelli)
Pull request description:
This PR contains the first part of #10882 :
- if a key from the keypool is used, mark all keys up to that key as used, and then try to top up the keypool
- top up the keypool on startup
Notably, it does not stop the node or prevent the best block from advancing if the keypool drops below a threshold (which means that transactions may be missed and funds lost if restoring from an old HD wallet backup).
Tree-SHA512: ac681fefeaf7ec2aab2fa1da93d12273ea80bd05eb48d7b3b551ea6e5d975dd97ba7de52b7fba52993823280ac4079cc36cf78a27dac708107ebf8fb6326142b
faa24441ec047ec336b86f586016b9d318c1c0ad policy: Remove promiscuousmempoolflags (MarcoFalke)
Pull request description:
It seems odd to clutter validation code with features that can only ever be used for testing (testnet or regtest). Removing that test-only code makes the mempool logic less painful to understand and easier to reason about when changed or refactored in the future.
Tree-SHA512: 3b897aa9604ac8d82ebe9573c6efd468c93ddaa08d378ebc902e247b7aa6c68fcde71e5b449c08f17a067146cdc66dc50a67ce06d07607c27e5189a49c3fba3f
4ccc12a [qa] Rewrite BIP66 functional tests (Suhas Daftuar)
d4f0d87 [qa] Rewrite BIP65 functional tests (Suhas Daftuar)
Pull request description:
After 122786d, BIP65 and BIP66 activate at
particular fixed heights (without regard to version numbers of blocks
below those heights). Rewrite the functional tests to take
this into account, and remove two tests that weren't really testing anything.
Moves the rewritten functional tests out of the extended test suite, so that they run in travis regularly.
Note: I discovered that the ComparisonTestFramework (which the original versions of these p2p tests were written is, has a bug that caused them to not catch obvious errors, eg if you just comment out setting the script flags for these softforks in ConnectBlock, the versions of these tests in master do not fail(!) -- will separately PR a fix for the comparison test framework).
cf82a9e Do not allow users to get keys from keypool without reserving them (Matt Corallo)
Pull request description:
fundrawtransaction allows users to add a change output and then
not have it removed from keypool. While it would be nice to have
users follow the normal CreateTransaction/CommitTransaction process
we use internally, there isnt much benefit in exposing this option,
especially with HD wallets, while there is ample room for users to
misunderstand or misuse this option.
This partially reverts #9377. Would be nice to get this for 15 since its kinda crazy we have this option to begin with IMO, will need release notes as an RPC option is now ignored.
Tree-SHA512: 72b5ee9c4a229b84d799dfb00c56fe80d8bba914ce81a433c3f5ab325bf9bf2b839ee658c261734f0ee183ab19435039481014d09c41dbe155e6323e63beb01d
* Refactor `HandleFullyConfirmedBlock()`
* Pass `pindexMined` into `AddNonLockedTx()`
* Unify time import in Dash specific tests
* Refactor UpdateSpork and SetPrivKey
* Merge #10757: RPC: Introduce getblockstats to plot things
41d0476f62269027ec2193a5f80d508d789de8aa Tests: Add data file (Anthony Towns)
4cbfb6aad9ba8fa17b5e7ed3e9a36dc8a24f1fcf Tests: Test new getblockstats RPC (Jorge Timón)
35e77a0288bcac5594ff25c10c9679a161cb730b RPC: Introduce getblockstats (Jorge Timón)
cda8e36f019dd181e5c3774961b4f1335e5602cb Refactor: RPC: Separate GetBlockChecked() from getblock() (Jorge Timón)
Pull request description:
It returns per block statistics about several things. It should be easy to add more if people think of other things to add or remove some if I went too far (but once written, why not keep it? EDIT: answer: not to test or maintain them).
The currently available options are: minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs (EDIT: see updated list in the rpc call documentation)
For the x axis, one can use height or block.nTime (I guess I could add mediantime if there's interest [EDIT: nobody showed interest but I implemented mediantime nonetheless, in fact there's no distinction between x or y axis anymore, that's for the caller to judge]).
To calculate fees, -txindex is required.
Tree-SHA512: 2b2787a3c7dc4a11df1fce62c8a4c748f5347d7f7104205d5f0962ffec1e0370c825b49fd4d58ce8ce86bf39d8453f698bcd46206eea505f077541ca7d59b18c
* Replace get_mocktime() usage with self.mocktime
