6af6d9b23d test: Add tests for RPC help (João Barbosa)
Pull request description:
At the moment the new test checks for:
- invalid usages
- expected output for unknown command
- current RPC command titles (derived from command categories) — this prevents adding wrong RPC categories and new categories must be added to the test
Tree-SHA512: f987535d001b1cd300656588602b1634099ea68a1dd2282180c30fa56caf7f990be9e2dc86c7431dfcf7fd686d0299a8d4935df178a2c9f0fb6fbebcba748eb5
16e288acdd61fa5fa5e39f3936fb50499f82c085 test padding non micro timestamps (John Newbery)
995dd89d884bda3fb5ca1885c5887d989cd2cad3 [Tests] Make combine_logs.py handle multi-line logs (John Newbery)
Pull request description:
combine_logs.py currently inserts additional newlines into multi-line
log messages, and doesn't color them properly. Fix both of those.
Tree-SHA512: dbe2f3ecc7cfbc95ee4350e648d127538c79cb6555257d4aeec12fe3d159366742b68e90e620c8ed7219a44b973395c7e5929ba374fae115fbee25560db645f6
8845c8aea65897637c330f5893461c0da180eaf8 tests: Replace usage of tostring() with tobytes() (Carl Dong)
Pull request description:
tostring() is deprecated as of python 3.7 and results in stderr output
causing tests to fail
Tree-SHA512: 8c5bbd6c6127490922add98543ee7719d19e11200e081784adef2f026ddf90d7735da7d0fb41fa4307d0d3450a27e126752c2b01cbd79b0c8a695855aed080ac
4441ad677a Make format string linter understand basic template parameter syntax (practicalswift)
Pull request description:
Make format string linter understand basic template parameter syntax.
Fixes issue described in https://github.com/bitcoin/bitcoin/pull/13705#issuecomment-412046126.
Thanks to @ken2812221 for reporting!
Tree-SHA512: 8fb995bc6b29d8b9926ef5969e02cf71c494e829434fcdeb4ed5fabad5ab96e86e5b8eea705e8a416927757b4fa4e58abc0fd4f483daa58c94e2c6fdcb8ee822
38040c34e1 [tests] Remove accounts from wallet_importprunedfunds.py (John Newbery)
Pull request description:
This was split from #13075 to not block review/merge of that PR.
Tree-SHA512: 631d7139ed2bda5222ec395cc75720261e2e1f741dba04723d09fe04ef6cf92222a3679d886026ec33e2db2d1e2fa1a0f36c2451581d0f733a9939a98c7118ab
f618c58b75 Docs: Update python docs to reflect that wildcard imports are disallowed (Ben Woosley)
Pull request description:
These have been disallowed via flake8 since: #13054
Tree-SHA512: f41651fd883e3786a7e87c4aa4c54749308e576287f2f88da3f1d8d0f59e14519d99061f1efd05b8745f495f87a14286cea576f1507d10ccd226f8cf2f2b3cc8
42ff30ec6 [Docs] add short documentation for /rest/blockhashbyheight (Jonas Schnelli)
579d418f7 [QA] add rest tests for /rest/blockhashbyheight/<HEIGHT>.<FORMAT> (Jonas Schnelli)
eb9ef04c4 REST: add "blockhashbyheight" call, fetch blockhash by height (Jonas Schnelli)
Pull request description:
Completes the REST interface for trivial block exploring by adding a call that allows to fetch the blockhash in the main chain by a given height.
Tree-SHA512: 94be9e56718f857279b11cc16dfa8d04f3b5a762e87ae54281b4d87247c71c844895f4944d5a47f09056bf851f4c4761ac4fbdbaaee957265d14de5c1c73e8d2
7cf994d5cfd53dcff76ebd0e0007e3477a7570e8 qa: Improve tests of /rest/headers and /rest/block (João Barbosa)
0825b86b280c684c32c60bac9e862298c7279f27 doc: /rest/block responds with 404 if block does not exist (João Barbosa)
be625f7c5562afed517ff51d2d85268ba5ce6017 doc: Explain empty result of /rest/headers (João Barbosa)
Pull request description:
Follow up of #15107.
Tree-SHA512: a7fdeed05216e3eda9604664db529237c2d0ddf422cfac139d6345a22b6e00bfe870d4e3f177423db7d4efb295ac2dc0ca2eb20c9c27c0719b89fd5428860d03
e4a0c3547ed886871f8b3d51c6b4ffdb181a8b9c Improve blocksdir functional test. (Hennadii Stepanov)
c3f1821ac788e522e7558e3575150433450dcb8c Make blockdir always net specific (Hennadii Stepanov)
Pull request description:
The blocks directory is net specific by definition.
Also this prevents the side effect of calling `GetBlocksDir(false)` in the non-mainnet environment.
Currently a new node creates an unused `blocks\` directory in the root of the data directory when `-testnet` or `-regtest` is specified.
Refs:
- #12653
- https://github.com/bitcoin/bitcoin/pull/12653#discussion_r174784834 by @laanwj
- https://github.com/bitcoin/bitcoin/issues/14595#issuecomment-436011186
Tree-SHA512: c9957a68a4a200ebd2010823a56db7e61563afedcb7c9828e86b13f3af2990e07854b622c1f3374756f94574acb3ea32de7d2a399eef6c0623f0e11265155627
cbd9091ed5a76bb2e1e57cd0d8db035c15529341 refactor/lint: Add ignored suggestions to an array (Vidar Holen)
Pull request description:
By adding excluded shellcheck suggestions to an array, you can avoid the current duplication
between command and comments. This ensures that they never go out of sync, makes it easier to
add new ones, and improves the readability of related diffs.
Tree-SHA512: 04afced1d27fda940cc5e61d7f9ed04507c8f7f7dfd0031c09898a599c6de93695923a80cb3d515a0f0bf728847592d8680c15ac2e376b48726c03ca744f13a5
f652f85d0c88629b4284f69b179362254307fb55 qa: Ignore shellcheck warning SC2236 (João Barbosa)
Pull request description:
With shellcheck 0.6.0 the warning `SC2236 - Use -n instead of ! -z` is raised. This change adds that warning to the ignored list.
Tree-SHA512: 7b0dfbce55e5da4efb927e251257993cdf67cd1c90f8490d99fa75bf6e233bbee79ea1c59d28774994c59862c5eac061313aa154833284950fc844bda60a54e9
ef5ebc6d8f docs: Clarifying testing instructions (benthecarman)
Pull request description:
This statement confused me on my first time reading through. Hopefully, this addition will help someone else on their first time.
Tree-SHA512: 17f421275adb7586eca954910269d29fcd3bacc42fab4bc2e01110f9e13ca6f8c1ca178246f7192e1131f14ced7f7dc0b57e7aec324898807c1813a2ebc513de
c9066f07c94e3610f7762d6406851cd135790511 Allow running rpc_bind.py --nonloopback test without IPv6 (Kristaps Kaupe)
Pull request description:
Don't see a reason why this can't be tested with IPv4 only.
Tree-SHA512: 515bdf700fad420e4b1798fd4978b53e2da3ddb26e43b16d68b43071bc912c325f1ceb10046ba3d0494dab289a53c45ddc2de9064117d8c1d6bf11e88323f490
fa6edfef358518022ee86c0abc77c1c068f106a3 qa: Remove portseed_offset from test runner (MarcoFalke)
Pull request description:
The portseed_offset is no longer needed in the test runner, since we already kill leftover processes (see #12904). This "fixes" #10869 because we deterministically pick ports starting at 11000
Tree-SHA512: 1ee22e19e02acd3afadc7c6a2b391fd3b5cfcec22c0fe194f3207251e7b1264a04e47d90a3ff8be4aca7d0ec33219a2f5855076acb3565291767939bc2f2fa17
fa4760fbb3f1099dcd3c43ebc53c2a761a2170e8 qa: Increase includeconf test coverage (MarcoFalke)
Pull request description:
This adds some missing `return false` for error conditions and adds test coverage [1] for those.
Also, extend recursion warning when the chain was set in one of the includeconfs.
[1] See the red lines in https://marcofalke.github.io/btc_cov/total.coverage/src/util.cpp.gcov.html for missing coverage.
Tree-SHA512: d32563c9bb277879895a173e699034db5ecdb4061a1ec8890c566d61e36a09efa5eda19a029baf952ff6d568f8b9684a13a0bb90827850075470975e2088fee4
fa26cf015658ac2aa52b5e5656e38af9a12160cc qa: Fixup setting of PATH env var (MarcoFalke)
Pull request description:
This was an oversight of mine in #13188
Can be trivially tested with `BITCOIND=bitcoin-qt ./test/functional/wallet_disable.py` before and after this fix.
Tree-SHA512: 06c7b2f12158855eb2b6392861943821bd7ad3152cf0dd49ac4abd878e5b937ebee55e256ce5bdc1c2a9c775a452112c34533366c934ff5f0f412b3a7e1c8118
c2dfbb4a97 Add unavailable options to hidden options category (Andrew Chow)
Pull request description:
From IRC:
```
<ossifrage> FYI, bitcoin-qt from the head I built today won't start if you have "daemon=0" in the config file, so you can't use the same config for either bitcoind or bitcoin-qt
<ossifrage> Seems like bitcoin-qt should ignore this option?
<provoostenator> ossifrage: probably caused by 13112. Another problem is disablewallet=1 will prevent a launch if you compile bitcoind without wallet. It probably needs to be relaxed slightly.
```
Adds all of the options that are unavailable due to compiling options to the hidden category so that shared config files do not break with the alternative binaries.
Tree-SHA512: 1ef43f5f7ad46ecc2865d22ee683ef22831e8f131ec99b732bb36d90381f7964bf64829595e993c2d435823fe4425a20323c8e65307cf2463a9e40b8049ab559
903055730b Test gArgs erroring on unknown args (Andrew Chow)
4f8704d57f Give an error and exit if there are unknown parameters (Andrew Chow)
174f7c8080 Use a struct for arguments and nested map for categories (Andrew Chow)
Pull request description:
Following #13190, gArgs is aware of all of the command line arguments. This PR has gArgs check whether the arguments provided are actually valid arguments. When an unknown argument is encountered, an error is printed to stderr and the program exist.
Since gArgs is used for everything that has command line arguments, `bitcoind`, `bitcoin-cli`, `bitcoin-qt`, `bitcoin-tx`, and `bench_bitcoin` are all effected by this change and all now have the same argument checking behavior.
Closes#1044
Tree-SHA512: 388201319a7d6493204bb5433da47e8e6c8266882e809f6df45f86d925f1f320f2fd13edb3e57ffc6a37415dfdfc689f83929452bca224229783accb367032e7
6249021d1 [docs] Add release notes for HD master key -> HD seed rename (John Newbery)
79053a5f2 [rpc] [wallet] Add 'hdmasterkeyid' alias return values. (John Newbery)
c75c35141 [refactor] manually change remaining instances of master key to seed. (John Newbery)
131d4450b scripted-diff: Rename master key to seed (John Newbery)
Pull request description:
Addresses #12084 and #8684
This renames a couple of functions and members (no functional changes, expect log prints):
- Rename CKey::SetMaster to CKey::SetSeed
- Rename CHDChain::masterKeyId to CHDChain::seedID
- Rename CHDChain::hdMasterKeyID to CHDChain::hdSeedID
- Rename CWallet::GenerateNewHDMasterKey to CWallet::GenerateNewHDSeed
- Rename CWallet::SetHDMasterKey to CWallet::SetHDSeed
As well it introduces a tiny API change:
- RPC API change: Rename "hdmasterkeyid" to "hdseedid", rename "hdmaster" in wallet-dump output to "hdseed"
Fixes also a bug:
- Bugfix: use "s" instead of the incorrect "m" for the seed-key hd-keypath key metadata
Tree-SHA512: c913252636f213135a3b64df5de5d21844fb9c2d646567c1aad0ec65745188587de26119de99492c67e559bd49fdd9606b54276f00dddb84301785beba58f281
2352aa9 test: Ensure that recursive -includeconf produces appropriate warnings (Karl-Johan Alm)
c5bcc7d util: warn about recursive -includeconf arguments in configuration files (Karl-Johan Alm)
Pull request description:
This is a follow-up PR to #10267, and addresses https://github.com/bitcoin/bitcoin/pull/10267#issuecomment-387546144.
~~I am adding extra work for @jnewbery in #12755 here -- maybe I should just rebase on top of that, but not sure what the appropriate approach is here.~~
Tree-SHA512: 87f0c32436b70424e33616ffb88d7cb699f90d6a583a10237e224b28fc936d6a9df95536c8c52ee8546b3942da92b2a357e61bf87e00d1462bc10d46d3bee352
beee49b [tests] Allow stderr to be tested against specified string (John Newbery)
e503671 [Tests] Use LIBC_FATAL_STDERR_=1 in tests (John Newbery)
c22ce8a [Tests] Write stdout/stderr to datadir instead of temp file. (John Newbery)
Pull request description:
**Due to a merge conflict, this is now based on #10267. Please review that PR first!**
Subset of #12379 now that parts of that PR have been merged.
#12362 was only observed when running the functional tests locally because:
- by defatul libc logs to `/dev/tty` instead of stderr
- the functional tests only check for substring inclusion in stderr when we're expecting bitcoind to fail.
This PR tightens our checking of stderr and will cause tests to fail if there is any unexpected message in stderr:
- commit *Write stdout/stderr to datadir instead of temp file* writes stderr to a file in the datadir instead of a temporary file. This helps with debugging in the case of failure.
- commit *Use LIBC_FATAL_STDERR=1 in tests* ensures that libc failures are logged to stderr instead of the terminal.
commit *Assert that bitcoind stdout is empty on shutdown* asserts that stderr is empty on bitcoind shutdown.
Tree-SHA512: 21111030e667b3b686f2a7625c2b625ebcfb6998e1cccb4f3932e8b5d21fb514b19a73ac971595d049343430e9a63155986a7f5648cad55b8f36f3c58b1c7048
232f96f5c8a3920c09db92f4dbac2ad7d10ce8cf doc: Add release notes for -avoidpartialspends (Karl-Johan Alm)
e00b4699cc6d2ee5697d38dd6607eb2631c9b77a clean-up: Remove no longer used ivars from CInputCoin (Karl-Johan Alm)
43e04d13b1ffc02b1082176e87f420198b40c7b1 wallet: Remove deprecated OutputEligibleForSpending (Karl-Johan Alm)
0128121101fb3ee82f3abd3973a967a4226ffe0e test: Add basic testing for wallet groups (Karl-Johan Alm)
59d6f7b4e2f847ec1f2ff46c84e6157655984f85 wallet: Switch to using output groups instead of coins in coin selection (Karl-Johan Alm)
87ebce25d66952f5ce565bb5130dcf5e24049872 wallet: Add output grouping (Karl-Johan Alm)
bb629cb9dc567cc819724d9f4852652926e60cbf Add -avoidpartialspends and m_avoid_partial_spends (Karl-Johan Alm)
65b3eda458221644616d0fdd6ba0fe01bdbce893 wallet: Add input bytes to CInputCoin (Karl-Johan Alm)
a443d7a0ca333b0bae63e04b5d476f9ad9c7aeac moveonly: CoinElegibilityFilter into coinselection.h (Karl-Johan Alm)
173e18a289088c6087ba6fac708e322aa63b7a94 utils: Add insert() convenience templates (Karl-Johan Alm)
Pull request description:
This PR adds an optional (off by default) `-avoidpartialspends` flag, which changes coin select to use output groups rather than outputs, where each output group corresponds to all outputs with the same destination.
It is a privacy improvement, as each time you spend some output, any other output that is publicly associated with the destination (address) will also be spent at the same time, at the cost of fee increase for cases where coin select without group restriction would find a more optimal set of coins (see example below).
For regular use without address reuse, this PR should have no effect on the user experience whatsoever; it only affects users who, for some reason, have multiple outputs with the same destination (i.e. address reuse).
Nodes with this turned off will still try to avoid partial spending, if the fee of the resulting transaction is not greater than the fee of the original transaction.
Example: a node has four outputs linked to two addresses `A` and `B`:
* 1.0 btc to `A`
* 0.5 btc to `A`
* 1.0 btc to `B`
* 0.5 btc to `B`
The node sends 0.2 btc to `C`. Without `-avoidpartialspends`, the following coin selection will occur:
* 0.5 btc to `A` or `B` is picked
* 0.2 btc is output to `C`
* 0.3 - fee is output to (unique change address)
With `-avoidpartialspends`, the following will instead happen:
* Both of (0.5, 1.0) btc to `A` or `B` is picked (one or the other pair)
* 0.2 btc is output to `C`
* 1.3 - fee is output to (unique change address)
As noted, the pro here is that, assuming nobody sends to the address after you spend from it, you will only ever use one address once. The con is that the transaction becomes slightly larger in this case, because it is overpicking outputs to adhere to the no partial spending rule.
This complements #10386, in particular it addresses @luke-jr and @gmaxwell's concerns in https://github.com/bitcoin/bitcoin/pull/10386#issuecomment-300667926 and https://github.com/bitcoin/bitcoin/pull/10386#issuecomment-302361381.
Together with `-avoidreuse`, this fully addresses the concerns in #10065 I believe.
Tree-SHA512: 24687a4490ba59cf4198ed90052944ff4996653a4257833bb52ed24d058b3e924800c9b3790aeb6be6385b653b49e304453e5d7ff960e64c682fc23bfc447621
# Conflicts:
# src/Makefile.am
# src/bench/coin_selection.cpp
# src/wallet/coincontrol.h
# src/wallet/coinselection.cpp
# src/wallet/coinselection.h
# src/wallet/init.cpp
# src/wallet/test/coinselector_tests.cpp
# src/wallet/wallet.cpp
# src/wallet/wallet.h
# test/functional/test_runner.py
d641c29a5a travis: Run feature_dbcrash functional tests in cron job (Chun Kuan Lee)
c0d947d725 tests: Reorder tests and move most of extended tests up to normal tests (Chun Kuan Lee)
Pull request description:
The travis should run almost all jobs unless it takes really long time, however it does not take too long for now. So it's time for moving it to normal job.
(The test sort is to see how many conflict will this cause, will drop it if there are too many)
The first commit can be reviewed by `git diff --color-moved=plain`
Tree-SHA512: db6bd5b1f19de2f729012adda6ed00ca989071fd40a20710c0ff2579b5bd008edcf7421c1ad56d5f0752354e7df408f58351129d35a1ab7f4a6caa9d315df2ec
# Conflicts:
# .travis/test_06_script.sh
# test/functional/test_runner.py
364bae5 qa: Pad scriptPubKeys to get minimum sized txs (MarcoFalke)
7485488 Policy to reject extremely small transactions (Johnson Lau)
0f8719b Add transaction tests for constant scriptCode (Johnson Lau)
9dabfe4 Add constant scriptCode policy in non-segwit scripts (Johnson Lau)
Pull request description:
This disables `OP_CODESEPARATOR` in non-segwit scripts (even in an unexecuted branch), and makes a positive `FindAndDelete` result invalid. This ensures that the `scriptCode` serialized in `SignatureHash` is always the same as the script passing to the `EvalScript`.
Tree-SHA512: a0552cb920294d130251c48053fa2ff1fbdd26332e62b52147d918837852750f0ce35ce2cd1cbdb86588943312f8154ccb4925e850dbb7c2254bc353070cd5f8
67d6ee1 remove redundant tests in p2p-segwit.py (Johnson Lau)
9260085 test segwit uncompressed key fixes (Johnson Lau)
248f3a7 Fix ismine and addwitnessaddress: no uncompressed keys in segwit (Pieter Wuille)
b811124 [qa] Add tests for uncompressed pubkeys in segwit (Suhas Daftuar)
9f0397a Make test framework produce lowS signatures (Johnson Lau)
4c0c25a Require compressed keys in segwit as policy and disable signing with uncompressed keys for segwit scripts (Johnson Lau)
3ade2f6 Add standard limits for P2WSH with tests (Johnson Lau)
fac1e1f qa: Remove unused option --srcdir (MarcoFalke)
Pull request description:
The `srcdir` option was both unused and misleading; It should have been called `builddir`. So remove it.
Tree-SHA512: 2c24dcf2aa82219158b8cbbf03dd3f0f51f805f1f5f670faa1fd59e5a8d60fda120ffddadeccb058d8d3f20583b4952be7afd2df6bbefb9367d35c0f0a9fda3c
25b7ab9 doc: Add release notes for -includeconf (Karl-Johan Alm)
0f0badd test: Test includeconf parameter. (Karl-Johan Alm)
629ff8c -includeconf=<path> support in config handler, for including external configuration files (Karl-Johan Alm)
Pull request description:
Fixes: #10071.
Done:
- adds `-includeconf=<path>`, where `<path>` is relative to `datadir` or to the path of the file being read, if in a file
- protects against circular includes
- updates help docs
~~~Thoughts:~~~
- ~~~I am not sure how to test this in a neat manner. Feedback on this would be nice. Will dig/think though.~~~
Tree-SHA512: cb31f1b2f69fbc0890d264948eb2e501ac05cf12f5e06a5942f9c1539eb15ea8dc3cae817f4073aecb2fcc21d0386747f14f89d990772003a76e2a6d25642553
a533834d50 [tests] Fix flake8 warnings in several wallet functional tests (John Newbery)
Pull request description:
Fixes flake8 warnings in several wallet functional tests.
Several wallet functional tests need rewrite to remove the accounts API (#13075). To prepare for that, I fixed all the flake8 warnings in those tests.
#13075 is blocked on a bitcoind bug. This PR is just the flake8 fixes so we're not completely blocked.
Tree-SHA512: 2dc1d589b2f8f4318083a681e487532d0f8f3d57e8bc8f37b660b728ffc33329b88e9251eb223104aea89f293c3f4074ca700fe690e645617326b859da3e93c3
9b2704777c [doc] Include txindex changes in the release notes. (Jim Posen)
ed77dd6b30 [test] Simple unit test for TxIndex. (Jim Posen)
6d772a3d44 [rpc] Public interfaces to GetTransaction block until synced. (Jim Posen)
a03f804f2a [index] Move disk IO logic from GetTransaction to TxIndex::FindTx. (Jim Posen)
e0a3b80033 [validation] Replace tx index code in validation code with TxIndex. (Jim Posen)
8181db88f6 [init] Initialize and start TxIndex in init code. (Jim Posen)
f90c3a62f5 [index] TxIndex method to wait until caught up. (Jim Posen)
70d510d93c [index] Allow TxIndex sync thread to be interrupted. (Jim Posen)
94b4f8bbb9 [index] TxIndex initial sync thread. (Jim Posen)
34d68bf3a3 [index] Create new TxIndex class. (Jim Posen)
c88bcec93f [db] Migration for txindex data to new, separate database. (Jim Posen)
0cb8303241 [db] Create separate database for txindex. (Jim Posen)
Pull request description:
I'm re-opening #11857 as a new pull request because the last one stopped loading for people
-------------------------------
This refactors the tx index code to be in it's own class and get built concurrently with validation code. The main benefit is decoupling and moving the txindex into a separate DB. The primary motivation is to lay the groundwork for other indexers that might be desired (such as the [compact filters](https://github.com/bitcoin/bips/pull/636)). The basic idea is that the TxIndex spins up its own thread, which first syncs the txindex to the current block index, then once in sync the BlockConnected ValidationInterface hook writes new blocks.
### DB changes
At the suggestion of some other developers, the txindex has been split out into a separate database. A data migration runs at startup on any nodes with a legacy txindex. Currently the migration blocks node initialization until complete.
### Open questions
- Should the migration of txindex data from the old DB to the new DB block in init or should it happen in a background thread? The downside to backgrounding it is that `getrawtransaction` would return an error message saying the txindex is syncing while the migration is running.
### Impact
In a sample size n=1 test where I synced nodes from scratch, the average time [Index writing](https://github.com/bitcoin/bitcoin/blob/master/src/validation.cpp#L1903) was 3.36ms in master and 1.72ms in this branch. The average time between `UpdateTip` log lines for sequential blocks between 400,000 and IBD end on mainnet was 0.297204s in master and 0.286134s in this branch. Most likely this is just variance in IBD times, but I can try with some more trials if people want.
Tree-SHA512: 451fd7d95df89dfafceaa723cdf0f7b137615b531cf5c5035cfb54e9ccc2026cec5ac85edbcf71b7f4e2f102e36e9202b8b3a667e1504a9e1a9976ab1f0079c4
