Commit Graph

192 Commits

Author SHA1 Message Date
Kittywhiskers Van Gogh
21c7e57493 bitcoin#14121: Index for BIP 157 block filters
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-08-12 12:44:03 +05:30
Kittywhiskers Van Gogh
4d3e3d7b07 bitcoin#15118: Refactor block file logic 2021-08-12 12:32:29 +05:30
Kittywhiskers Van Gogh
4e6aca0ac1 bitcoin#17051: Add deserialization fuzzing harnesses 2021-08-11 09:48:41 +05:30
Kittywhiskers Van Gogh
f47768b94c bitcoin#17018: Add descriptor Parse(...) fuzzing harness 2021-08-11 09:48:41 +05:30
Kittywhiskers Van Gogh
19b94c6dfa bitcoin#17076: Add fuzzing harness for CheckTransaction(...), IsStandardTx(...) and other CTransaction related functions 2021-08-11 09:48:41 +05:30
Kittywhiskers Van Gogh
f202a902bf bitcoin#15504: Link BasicTestingSetup (shared with unit tests) 2021-08-11 09:48:40 +05:30
Kittywhiskers Van Gogh
0dde23949b bitcoin#15399: Script validation flags 2021-08-11 08:50:43 +05:30
Kittywhiskers Van Gogh
d967c39222 bitcoin#15295: Add test/fuzz/test_runner.py and run it in travis 2021-08-11 08:50:43 +05:30
Kittywhiskers Van Gogh
0e7fe9e6ab bitcoin#15043: Build fuzz targets into separate executables 2021-08-11 08:50:43 +05:30
Kittywhiskers Van Gogh
fc8952aa19
build: add libgmp detection, make immer a package (#4311)
* build: detect the presence of libgmp before generating Makefile

* depends: add arximboldi/immer@v0.6.2 as a package and add detection

* depends: remove immer from source tree, build using package only

* Drop immer refs from tools

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-08-10 22:35:55 +03:00
PastaPastaPasta
edf0552c0c
Merge pull request #4187 from kittywhiskers/scantxout
merge #12196, #13697: Add scantxoutset RPC method
2021-07-25 19:10:11 -05:00
Kittywhiskers Van Gogh
2bdb8540b6 Merge #13697: Support output descriptors in scantxoutset
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-07-21 11:58:00 +05:30
MarcoFalke
c4a60148b9 Merge #14455: build: unbreak make clean
a48e44bd4f Fix CLEAN_BITCOIN_TEST to remove .log files for all BITCOIN_TESTS files (James O'Beirne)

Pull request description:

  My preferred alternative to https://github.com/bitcoin/bitcoin/pull/14440 (I'm too lazy to review an entire file move). This just applies a `.log` suffix addition to all `BITCOIN_TESTS` files for the purposes of cleaning, and thus doesn't erroneously remove test/scriptnum10.h.

Tree-SHA512: 06a0d9fb2c4851f4e74215aa4bca9cf81f49787f16c46b0172e5c1cf18ca744d747c5b0c1564cac433e0dea29b0afabd2d95f0f58a0157d1e05b2b7cb7800e00
2021-07-17 13:52:11 -05:00
MarcoFalke
9d0fb5c295 Merge #14253: Build: during 'make clean', remove some files that are currently missed.
3f5ac27205 Include some files currently missed by 'make distclean'. (murrayn)

Pull request description:

  `make clean` currently leaves behind some cache and test log files that should be removed.

Tree-SHA512: a1877e776e24232f6dd1468d7f392ea0bd1e93fdd975e623897d48c4b23a080a2e84ebb199f5482abd6b8c9ddd036850325e7b7ed07e2f9fe7a32f83cc99da4a
2021-07-17 13:50:02 -05:00
UdjinM6
85b8a05690
Merge pull request #4263 from UdjinM6/bp11911
Merge #11911: Free BerkeleyEnvironment instances when not in use
2021-07-16 03:37:08 +03:00
Wladimir J. van der Laan
e681ffc3bb
Merge #11911: Free BerkeleyEnvironment instances when not in use
14bc2a17dd03ccd89f65a302328763ff22c710c2 Trivial: add doxygen-compatible comments relating to BerkeleyEnvironment (Pierre Rochard)
88b1d956fe3e38f2d2dd805feee9dadb0be9e8a9 Tests: add unit tests for GetWalletEnv (Pierre Rochard)
f1f4bb7345b90853ec5037478173601035593d26 Free BerkeleyEnvironment instances when not in use (Russell Yanofsky)

Pull request description:

  Instead of adding BerkeleyEnvironment objects permanently to the g_dbenvs map, use reference counted shared pointers and remove map entries when the last BerkeleyEnvironment reference goes out of scope.

  This change was requested by @TheBlueMatt and makes code that sets up mock databases cleaner. The mock database environment will now go out of scope and be reset on destruction so there is no need to call BerkeleyEnvironment::Reset() during wallet construction to clear out prior state.

  This change does affect bitcoin behavior slightly. On startup, instead of same wallet environments staying open throughout VerifyWallets() and OpenWallets() calls, VerifyWallets() will open and close an environment once for each wallet, and OpenWallets() will create its own environment(s) later.

Tree-SHA512: 219d77a9e2268298435b86088f998795e059fdab1d2050ba284a9ab8d8a44961c9b5cf96e94ee521688108d23c6db680e3e3a999b8cb2ac2a8590f691d50668b
2021-07-16 01:37:51 +03:00
MarcoFalke
d74366e54b Merge #18183: test: Set catch_system_errors=no on boost unit tests
fac52dafa013047b051ca7163cc30ac69ad35531 test: Set catch_system_errors=no on boost unit tests (MarcoFalke)

Pull request description:

  Closes #16700

  Can be tested by adding an `assert(0)` and then running either `make check` or `./src/test/test_bitcoin -t bla_tests --catch_system_errors=no/yes`

ACKs for top commit:
  practicalswift:
    ACK fac52dafa013047b051ca7163cc30ac69ad35531
  Empact:
    Tested ACK fac52dafa0

Tree-SHA512: ec00636951b2c1137aaf43610739d78d16f823f7da76a726d47f93b8b089766fb66b21504b3c5413bcf8b6b5c3db0ad74027d677db24a44487d6d79a6bdee2e0
2021-07-13 21:19:43 -05:00
Kittywhiskers Van Gogh
c00b3e942f Merge #13557: BIP 174 PSBT Serializations and RPCs 2021-07-13 22:00:17 +05:30
Wladimir J. van der Laan
dbb0c39f93
Merge #13355: Fix "gmake check" under OpenBSD 6.3 (probably *BSD): Avoid using GNU grep specific regexp handling
db56755ca4a0e53785b5bf322d3b65ffe328b60a Fix "gmake check" under OpenBSD 6.3 (probably *BSD): Avoid using GNU grep specific regexp handling (practicalswift)

Pull request description:

  Fixes #13337 (!)

  GNU grep and BSD grep differs in the way they handle regexps when extended regular expressions are not enabled via the `-E` flag:

  ```
  $ grep --version | head -1
  grep (GNU grep) 3.1
  $ echo "BOOST_FIXTURE_TEST_SUITE(foo)" | grep "BOOST_FIXTURE_TEST_SUITE(\|BOOST_AUTO_TEST_SUITE("
  BOOST_FIXTURE_TEST_SUITE(foo)
  $
  ```

  ```
  $ grep --version | head -1
  grep version 0.9
  $ echo "BOOST_FIXTURE_TEST_SUITE(foo)" | grep "BOOST_FIXTURE_TEST_SUITE(\|BOOST_AUTO_TEST_SUITE("
  $
  ```

  The portable way to do it is:

  ```
  $ echo "BOOST_FIXTURE_TEST_SUITE(foo)" | grep -E "(BOOST_FIXTURE_TEST_SUITE\\(|BOOST_AUTO_TEST_SUITE\\()"
  BOOST_FIXTURE_TEST_SUITE(foo)
  $
  ```

Tree-SHA512: d83c78f34421504dd8efc3921c98527f499045b702bd34715a5bc78e04ef2a5f49f601a55ad08632e870f137b1edada94a3f530291bc9107d8d6b16fe11e640b
2021-06-28 02:00:51 +03:00
UdjinM6
c5cc285d0e
Merge pull request #4191 from kittywhiskers/checkqueue
partial #15842, merge #15849, #17342, #18710: Add local thread pool to CCheckQueue
2021-06-26 16:08:02 +03:00
Kittywhiskers Van Gogh
3f8caf00bc merge #15849: Thread names in logs and deadlock debug tools
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-06-25 12:25:10 +05:30
PastaPastaPasta
e8875967b5
Merge pull request #4199 from kittywhiskers/fstreamWrapper
merge bitcoin#14209, #13878, #14192, #17086: logging and filesystem updates
2021-06-24 20:43:42 -05:00
Kittywhiskers Van Gogh
93d7ecabf6 bitcoin#13878: Add fstream wrapper to allow to pass unicode filename on Windows 2021-06-24 23:43:00 +05:30
Kittywhiskers Van Gogh
0206ca2dcb bitcoin#16805: add timing information to FlushStateToDisk() 2021-06-17 10:57:37 +05:30
UdjinM6
a8aee57447
Merge pull request #4190 from kittywhiskers/tlocks
merge #11640, #11599, #16112, #16127, #18635, #19249: thread safety and locking improvements
2021-06-11 15:23:35 +03:00
Kittywhiskers Van Gogh
c5c3dee308 merge #11640: Make LOCK, LOCK2, TRY_LOCK work with CWaitableCriticalSection 2021-06-06 16:22:17 +05:30
Wladimir J. van der Laan
bb44ac04c9
Merge #13023: Fix some concurrency issues in ActivateBestChain()
dd435ad Add unit tests for signals generated by ProcessNewBlock() (Jesse Cohen)
a3ae8e6 Fix concurrency-related bugs in ActivateBestChain (Jesse Cohen)
ecc3c4a Do not unlock cs_main in ABC unless we've actually made progress. (Matt Corallo)

Pull request description:

  Originally this PR was just to add tests around concurrency in block validation - those tests seem to have uncovered another bug in ActivateBestChain - this now fixes that bug and adds tests.

  ActivateBestChain (invoked after a new block is validated) proceeds in steps - acquiring and releasing cs_main while incrementally disconnecting and connecting blocks to sync to the most work chain known (FindMostWorkChain()). Every time cs_main is released the result of FindMostWorkChain() can change - but currently that value is cached across acquisitions of cs_main and only refreshed when an invalid chain is explored. It needs to be refreshed every time cs_main is reacquired. The test added in 6094ce7304 will occasionally fail without the commit fixing this issue 26bfdbaddb

  Original description below
  --

  After a bug discovered where UpdatedBlockTip() notifications could be triggered out of order (#12978), these unit tests check certain invariants about these signals.

  The scheduler test asserts that a SingleThreadedSchedulerClient processes callbacks fully and sequentially.

  The block validation test generates a random chain and calls ProcessNewBlock from multiple threads at random and in parallel. ValidationInterface callbacks verify that the ordering of BlockConnected BlockDisconnected and UpdatedBlockTip events occur as expected.

Tree-SHA512: 4102423a03d2ea28580c7a70add8a6bdb22ef9e33b107c3aadef80d5af02644cdfaae516c44933924717599c81701e0b96fbf9cf38696e9e41372401a5ee1f3c
2021-06-05 22:13:02 +03:00
Pieter Wuille
7ff6515c88
Merge #13033: Build txindex in parallel with validation
9b2704777c [doc] Include txindex changes in the release notes. (Jim Posen)
ed77dd6b30 [test] Simple unit test for TxIndex. (Jim Posen)
6d772a3d44 [rpc] Public interfaces to GetTransaction block until synced. (Jim Posen)
a03f804f2a [index] Move disk IO logic from GetTransaction to TxIndex::FindTx. (Jim Posen)
e0a3b80033 [validation] Replace tx index code in validation code with TxIndex. (Jim Posen)
8181db88f6 [init] Initialize and start TxIndex in init code. (Jim Posen)
f90c3a62f5 [index] TxIndex method to wait until caught up. (Jim Posen)
70d510d93c [index] Allow TxIndex sync thread to be interrupted. (Jim Posen)
94b4f8bbb9 [index] TxIndex initial sync thread. (Jim Posen)
34d68bf3a3 [index] Create new TxIndex class. (Jim Posen)
c88bcec93f [db] Migration for txindex data to new, separate database. (Jim Posen)
0cb8303241 [db] Create separate database for txindex. (Jim Posen)

Pull request description:

  I'm re-opening #11857 as a new pull request because the last one stopped loading for people

  -------------------------------

  This refactors the tx index code to be in it's own class and get built concurrently with validation code. The main benefit is decoupling and moving the txindex into a separate DB. The primary motivation is to lay the groundwork for other indexers that might be desired (such as the [compact filters](https://github.com/bitcoin/bips/pull/636)). The basic idea is that the TxIndex spins up its own thread, which first syncs the txindex to the current block index, then once in sync the BlockConnected ValidationInterface hook writes new blocks.

  ### DB changes

  At the suggestion of some other developers, the txindex has been split out into a separate database. A data migration runs at startup on any nodes with a legacy txindex. Currently the migration blocks node initialization until complete.

  ### Open questions

  - Should the migration of txindex data from the old DB to the new DB block in init or should it happen in a background thread? The downside to backgrounding it is that `getrawtransaction` would return an error message saying the txindex is syncing while the migration is running.

  ### Impact

  In a sample size n=1 test where I synced nodes from scratch, the average time [Index writing](https://github.com/bitcoin/bitcoin/blob/master/src/validation.cpp#L1903) was 3.36ms in master and 1.72ms in this branch. The average time between `UpdateTip` log lines for sequential blocks between 400,000 and IBD end on mainnet was 0.297204s in master and 0.286134s in this branch. Most likely this is just variance in IBD times, but I can try with some more trials if people want.

Tree-SHA512: 451fd7d95df89dfafceaa723cdf0f7b137615b531cf5c5035cfb54e9ccc2026cec5ac85edbcf71b7f4e2f102e36e9202b8b3a667e1504a9e1a9976ab1f0079c4
2021-05-25 13:48:04 +03:00
Kittywhiskers Van Gogh
4b2b5f78d4 Merge #16702: supplying and using asmap to improve IP bucketing 2021-05-20 00:05:09 +05:30
Wladimir J. van der Laan
460c1ee359
Merge #12926: Run unit tests in parallel
7ef9cd8 Increase entropy in test temp directory name (Pieter Wuille)
f6dfb0f Reorder travis builds (Pieter Wuille)
156db42 tests: run tests in parallel (Cory Fields)
66f3255 tests: split up actual tests and helper files (Cory Fields)

Pull request description:

  This runs the unit tests (`src/test/test_bitcoin`) in 4 separate simultaneous processes, significantly speeding up some Travis runs (over 2x for win32).

  This uses an approach by @theuni that relies on `make` as the mechanism for distributing tests over processes (through `-j`). For every test .cpp file, we search for `BOOST_FIXTURE_TEST_SUITE` or `BOOST_AUTO_TEST_SUITE`, and then invoke the test binary for just that suite (using `-t`). The (verbose) output is stored in a temporary file, and only shown in the case of failure.

  Some makefile reshuffling is necessary to avoid trying to run tests from `src/test/test_bitcoin.cpp` for example, which contains framework/utility code but no real tests.

  Finally, order the Travis jobs from slow to fast (apart from the arm/doc job which goes first, for fast failure). This should help reducing the total wall clock time before opening a PR and finishing Travis, in case where not all jobs are started simultaneously.

  This is an alternative to #12831.

Tree-SHA512: 9f82eb4ade14ac859618da533c7d9df2aa9f5592a076dcc4939beeffd109eda33f7d5480d8f50c0d8b23bf3099759e9f3a2d4c78efb5b66b04569b39b354c185
2021-04-17 16:22:01 -04:00
dustinface
ae506bae66
refactor: PrivateSend -> CoinJoin + Move the tab (#4038)
* qt|wallet|privatesend: Rename PrivateSend to CoinJoin in GUI strings

* qt: Move CoinJoin next to Transactions

* qt: Adjust status tip of privateSendCoinsMenuAction

Co-authored-by: thephez <thephez@users.noreply.github.com>

* rename: privateSend -> coinJoin

* rename: privatesend -> coinjoin

* rename: PrivateSend -> CoinJoin

* rename: use_ps -> use_cj

* rename: PRIVATESEND -> COINJOIN

* rename: privatesend -> coinjoin for files and folders

* refactor: Re-order coinjoin files in cmake/make files

* refactor: Re-order coinjoin includes where it makes sense

* test: Update lint-circular-dependencies.sh

* Few cleanups

* test: test/coinjoin_tests.cpp -> wallet/test/coinjoin_test.cpp

* s/AdvancedPSUI/AdvancedCJUI/g

* s/privateSentAmountChanged/coinJoinAmountChanged/g

* wallet: Rename "ps_salt" backwards compatible

* Minimal PrivateSend -> CoinJoin migration for settings and cmd-line

* wallet: Fix privatesendrounds -> coinjoinrounds migration

* qt: Migrate nPrivateSendAmount -> nCoinJoinAmount

* `-coinjoindenoms` never existed

* Migrate all PS options/settings

* rpc: Formatting only

* qt: Make Send/CoinJoin tabs a bit more distinguishable

Co-authored-by: thephez <thephez@users.noreply.github.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-03-17 18:36:11 -04:00
tomthoros
4ce3635d99
Consensus: DIP-0020: Dash opcode updates. Remaining dip0020 opcodes (#3893)
* DIP-0020: Dash opcode updates - enable AND, OR, XOR

* DIP-0020: Dash opcode updates - enable DIV, MOD

* DIP-0020: Dash opcode updates - enable BIN2NUM, NUM2BIN

* DIP-0020: Dash opcode updates - enable CHECKDATASIG, CHECKDATASIGVERIFY (no tests)

* DIP-0020: Dash opcode updates - fix whitespace

* DIP-0020: Dash opcode updates - add checkdatasig and datacheckdatasigverify json tests

* More AND_OR_XOR tests

* DIP-0020: Dash opcode updates - move opcodes enabled tests around. Add unit tests back

* DIP-0020: Dash opcode updates - sort BITCOIN_TESTS aplhabetically

* DIP-0020: Dash opcode updates - formatting

* DIP-0020: Dash opcode updates - formatting

* DIP-0020: Dash opcode updates - formatting references alignment

* DIP-0020: Dash opcode updates - formatting references alignment

* Update src/script/interpreter.h

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update src/script/interpreter.h

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

* Update src/test/script_tests.cpp

Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
2021-01-13 14:45:04 -06:00
Wladimir J. van der Laan
70c82570b2
Merge #11372: Address encoding cleanup
92f1f8b31 Split off key_io_tests from base58_tests (Pieter Wuille)
119b0f85e Split key_io (address/key encodings) off from base58 (Pieter Wuille)
ebfe217b1 Stop using CBase58Data for ext keys (Pieter Wuille)
32e69fa0d Replace CBitcoinSecret with {Encode,Decode}Secret (Pieter Wuille)

Pull request description:

  This PR contains some of the changes left as TODO in #11167 (and built on top of that PR). They are not intended for backporting.

  This removes the `CBase58`, `CBitcoinSecret`, `CBitcoinExtKey`, and `CBitcoinExtPubKey` classes, in favor of simple `Encode`/`Decode` functions. Furthermore, all Bitcoin-specific logic (addresses, WIF, BIP32) is moved to `key_io.{h,cpp}`, leaving `base58.{h,cpp}` as a pure utility that implements the base58 encoding/decoding logic.

Tree-SHA512: a5962c0ed27ad53cbe00f22af432cf11aa530e3efc9798e25c004bc9ed1b5673db5df3956e398ee2c085e3a136ac8da69fe7a7d97a05fb2eb3be0b60d0479655

Make linter happy

Dashify
2021-01-08 22:35:34 +03:00
Pieter Wuille
c9936a5115
Partial merge bitcoin#11167: Full BIP173 (Bech32) support
Implement {Encode,Decode}Destination without CBitcoinAddress

Import Bech32 C++ reference code & tests

This includes a reformatted version of the Bech32 reference code
(see https://github.com/sipa/bech32/tree/master/ref/c%2B%2B), with
extra documentation.

Convert base58_tests from type/payload to scriptPubKey comparison

Add regtest testing to base58_tests

Implement ConvertBits

A part of "Implement BIP173 addresses and tests"

Make linter happy
2021-01-08 22:35:13 +03:00
UdjinM6
6ae4d32b4b
Backport bitcoin#10637 (partial) (#3878)
* Calculate and store the number of bytes required to spend an input

* Store effective value, fee, and long term fee in CInputCoin

Have CInputCOin store effective value information. This includes the effective
value itself, the fee, and the long term fee for the input

* Implement Branch and Bound coin selection in a new file

Create a new file for coin selection logic and implement the BnB algorithm in it.

* Move output eligibility to a separate function

* Use a struct for output eligibility

Instead of specifying 3 parameters, use a struct for those parameters
in order to reduce the number of arguments to SelectCoinsMinConf.

* Remove coinselection.h -> wallet.h circular dependency

Changes CInputCoin to coinselection and to use CTransactionRef in
order to avoid a circular dependency. Also moves other coin selection
specific variables out of wallet.h to coinselectoin.h

* Add tests for the Branch and Bound algorithm

* Move current coin selection algorithm to coinselection.{cpp,h}

Moves the current coin selection algorithm out of SelectCoinsMinConf
and puts it in coinselection.{cpp,h}. The new function, KnapsackSolver,
instead of taking a vector of COutputs, will take a vector of CInputCoins
that is prepared by SelectCoinsMinConf.

* Move original knapsack solver tests to coinselector_tests.cpp

* Add a GetMinimumFeeRate function which is wrapped by GetMinimumFee

* Have SelectCoinsMinConf and SelectCoins use BnB or Knapsack and use it (partial)

Allows SelectCoinsMinConf and SelectCoins be able to switch between
using BnB or Knapsack for choosing coins.

Has SelectCoinsMinConf do the preprocessing necessary to support either
BnB or Knapsack. This includes calculating the filtering the effective
values for each input.

Uses BnB in CreateTransaction to find an exact match for the output.
If BnB fails, it will fallback to the Knapsack solver.

Dash specific note: just always use Knapsack in CreateTransaction.

* Benchmark BnB in the worst case where it exhausts

* Add a test to make sure that negative effective values are filtered

* More of 12747: Fix typos

Co-authored-by: Andrew Chow <achow101-github@achow101.com>
2020-12-18 11:43:48 -06:00
dustinface
c207e0c953
test: Implement unit tests for CTransactionBuilder (#3677)
* makefile.test.include: Let privatesend_tests.cpp depend on ENABLE_WALLET

* test: Implement unit tests for CTransactionBuilder

* Check that we can decrease the amount and GetAmountLeft() is updated accordingly

* Check if resulting tx has a change output when expected

* Avoid pushing nullptr into vecOutputs

* Add few notes about size calculations

* nit: better readability (imo)

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-09-05 00:25:16 +03:00
Wladimir J. van der Laan
46c368b8f1
Merge #13056: [tests] Make rpcauth.py testable and add unit tests
6674a75 [tests] Make rpcauth.py testable and add unit tests (Qasim Javed)

Pull request description:

  refs #12995

Tree-SHA512: 609a85a75dafe46723f050db1e96098f6859519afd5e1b141a332f34ab715144d32fdd8f10d245d6dc0765e77f82158fe7b8aa2e4317f0e893de491d2e0acf1f
2020-07-22 12:12:46 -05:00
pasta
8a86c23e96 continued 13312
Signed-off-by: pasta <pasta@dashboost.org>
2020-07-07 11:27:55 -05:00
MarcoFalke
f77b58e89f Merge #13312: docs: Add a note about the source code filename naming convention
e56771365b Do not use uppercase characters in source code filenames (practicalswift)
419a1983ca docs: Add a note about the source code filename naming convention (practicalswift)

Pull request description:

  Add a note about the source code filename naming convention.

Tree-SHA512: 8d329bd9e19bcd26e74b0862fb0bc2369b46095dbd3e69d34859908632763abd7c3d00ccc44ee059772ad4bae4460c2bcc1c0e22fd9d8876d57e5fcd346cea4b
2020-07-07 11:17:45 -05:00
Wladimir J. van der Laan
17929071d4 Merge #12254: BIP 158: Compact Block Filters for Light Clients
254c85b68794ada713dbdae415db72adf5fcbaf3 bench: Benchmark GCS filter creation and matching. (Jim Posen)
f33b717a85363e067316c133a542559d2f4aaeca blockfilter: Optimization on compilers with int128 support. (Jim Posen)
97b64d67daf0336dfb64b132f3e4d6a4c1967da4 blockfilter: Unit test against BIP 158 test vectors. (Jim Posen)
a4afb9cadbaecb0676e6475ab8d32a52faecb47a blockfilter: Additional helper methods to compute hash and header. (Jim Posen)
cd09c7925b5af4104834971cfe072251e3ac2bda blockfilter: Serialization methods on BlockFilter. (Jim Posen)
c1855f6052aca806fdb51be01b30dfeee8b55f40 blockfilter: Construction of basic block filters. (Jim Posen)
53e7874e079f9ddfe8b176f11d46e6b59c7283d5 blockfilter: Simple test for GCSFilter construction and Match. (Jim Posen)
558c536e35a25594881693e6ff01d275c88d7af1 blockfilter: Implement GCSFilter Match methods. (Jim Posen)
cf70b550054eed36f194eaa13f4a9cb31e32df38 blockfilter: Implement GCSFilter constructors. (Jim Posen)
c454f0ac63c6028f54c7eb51683b3ccdb475b19b blockfilter: Declare GCSFilter class for BIP 158 impl. (Jim Posen)
9b622dc72279b027c59d6541cddff53800fc689b streams: Unit tests for BitStreamReader and BitStreamWriter. (Jim Posen)
fe943f99bf0a2bbb12e30bc4803c0337e3c95b93 streams: Implement BitStreamReader/Writer classes. (Jim Posen)
87f2d9ee43a9220076b1959d1ca65245d9591be9 streams: Unit test for VectorReader class. (Jim Posen)
947133dec92cd25ec2b3358c09b8614ba6fb40d4 streams: Create VectorReader stream interface for vectors. (Jim Posen)

Pull request description:

  This implements the compact block filter construction in [BIP 158](https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki). The code is not used anywhere in the Bitcoin Core code base yet. The next step towards [BIP 157](https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki) support would be to create an indexing module similar to `TxIndex` that constructs the basic and extended filters for each validated block.

  ### Filter Sizes

  [Here](https://gateway.ipfs.io/ipfs/QmRqaAAQZ5ZX5eqxP7J2R1MzFrc2WDdKSWJEKtQzyawqog) is a CSV of filter sizes for blocks in the main chain.

  As you can see below, the ratio of filter size to block size drops after the first ~150,000 blocks:

  ![filter_sizes](https://user-images.githubusercontent.com/881253/42900589-299772d4-8a7e-11e8-886d-0d4f3f4fbe44.png)

  The reason for the relatively large filter sizes is that Golomb-coded sets only achieve good compression with a sufficient number of elements. Empirically, the average element size with 100 elements is 14% larger than with 10,000 elements.

  The ratio of filter size to block size is computed without witness data for basic filters. Here is a summary table of filter size ratios *for blocks after height 150,000*:

  | Stat | Filter Type |
  |-------|--------------|
  | Weighted Size Ratio Mean | 0.0198 |
  | Size Ratio Mean | 0.0224 |
  | Size Ratio Std Deviation | 0.0202 |
  | Mean Element Size (bits) | 21.145 |
  | Approx Theoretical Min Element Size (bits) | 21.025 |

Tree-SHA512: 2d045fbfc3fc45490ecb9b08d2f7e4dbbe7cd8c1c939f06bbdb8e8aacfe4c495cdb67c820e52520baebbf8a8305a0efd8e59d3fa8e367574a4b830509a39223f
2020-06-13 14:08:18 -05:00
Wladimir J. van der Laan
9b9005e395
Merge #12895: tests: Add note about test suite name uniqueness requirement to developer notes
d1b622b tests: Add check for test suite name uniqueness in lint-tests.sh (practicalswift)
dc8067b tests: Add note about uniqueness requirement for test suite names (practicalswift)
3ebfb2d tests: Avoid test suite name collision in wallet crypto_tests (MarcoFalke)

Pull request description:

  * Add documentation: Add note about test suite name uniqueness requirement in developer notes
  * Add regression test: Update `lint-tests.sh` to make it check also for test suite name uniqueness

  Context: #12894 (`tests: Avoid test suite name collision in wallet crypto_tests`)

Tree-SHA512: 3c8502db069ef3d753f534976a86a997b12bac539e808a7285193bf81c9dd8c1b06821c3dd1bdf870ab87722b02c8aa9574c62ace70c2a1b8091785cb8c9aace
2020-06-11 23:20:48 -05:00
MarcoFalke
a9ac795173
Merge #11293: Deduplicate CMerkleBlock construction code, add test coverage
46ce223d1 Add tests for CMerkleBlock usage with txids specified (James O'Beirne)
5ab586f90 Consolidate CMerkleBlock constructor into a single method (James O'Beirne)

Pull request description:

  What started as a simple task to add test coverage ended up giving way to a light refactoring. This consolidates the mostly-identical `CMerkleBlock` constructors into one (using C++11 constructor delegation) and adds coverage for the by-txids construction case.

  ### Before

  ![selection_006](https://user-images.githubusercontent.com/73197/30242104-0f381fe4-9545-11e7-9617-83b87fce0456.png)

  ### After

  ![selection_008](https://user-images.githubusercontent.com/73197/30242107-1425dfaa-9545-11e7-9e6b-2c3432517dd1.png)

Tree-SHA512: eed84ed3e8bfc43473077b575c8252759a857e37275e4b36ca7cc2c17a65895e5f494bfd9d4aeab09fc6e98fc6a9c641ac7ecc0ddbeefe01a9e4308e7909e529
2020-04-08 11:46:27 -05:00
UdjinM6
9efa0e3d82
Merge pull request #3361 from PastaPastaPasta/backports-0.16-pr11
Backports 0.16 pr11
2020-03-20 01:47:53 +03:00
PastaPastaPasta
8a1ec935a0
Backport 11651 (#3358)
* scripted-diff: Replace #include "" with #include <> (ryanofsky)

-BEGIN VERIFY SCRIPT-
for f in \
  src/*.cpp \
  src/*.h \
  src/bench/*.cpp \
  src/bench/*.h \
  src/compat/*.cpp \
  src/compat/*.h \
  src/consensus/*.cpp \
  src/consensus/*.h \
  src/crypto/*.cpp \
  src/crypto/*.h \
  src/crypto/ctaes/*.h \
  src/policy/*.cpp \
  src/policy/*.h \
  src/primitives/*.cpp \
  src/primitives/*.h \
  src/qt/*.cpp \
  src/qt/*.h \
  src/qt/test/*.cpp \
  src/qt/test/*.h \
  src/rpc/*.cpp \
  src/rpc/*.h \
  src/script/*.cpp \
  src/script/*.h \
  src/support/*.cpp \
  src/support/*.h \
  src/support/allocators/*.h \
  src/test/*.cpp \
  src/test/*.h \
  src/wallet/*.cpp \
  src/wallet/*.h \
  src/wallet/test/*.cpp \
  src/wallet/test/*.h \
  src/zmq/*.cpp \
  src/zmq/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>

* scripted-diff: Replace #include "" with #include <> (Dash Specific)

-BEGIN VERIFY SCRIPT-
for f in \
  src/bls/*.cpp \
  src/bls/*.h \
  src/evo/*.cpp \
  src/evo/*.h \
  src/governance/*.cpp \
  src/governance/*.h \
  src/llmq/*.cpp \
  src/llmq/*.h \
  src/masternode/*.cpp \
  src/masternode/*.h \
  src/privatesend/*.cpp \
  src/privatesend/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>

* build: Remove -I for everything but project root

Remove -I from build system for everything but the project root,
and built-in dependencies.

Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/Makefile.test.include

* qt: refactor: Use absolute include paths in .ui files

* qt: refactor: Changes to make include paths absolute

This makes all include paths in the GUI absolute.

Many changes are involved as every single source file in
src/qt/ assumes to be able to use relative includes.

Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/qt/dash.cpp
#	src/qt/optionsmodel.cpp
#	src/qt/test/rpcnestedtests.cpp

* test: refactor: Use absolute include paths for test data files

* Recommend #include<> syntax in developer notes

* refactor: Include obj/build.h instead of build.h

* END BACKPORT #11651 Remove trailing whitespace causing travis failure

* fix backport 11651

Signed-off-by: Pasta <pasta@dashboost.org>

* More of 11651

* fix blockchain.cpp

Signed-off-by: pasta <pasta@dashboost.org>

* Add missing "qt/" in includes

* Add missing "test/" in includes

* Fix trailing whitespaces

Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-03-20 01:46:56 +03:00
MarcoFalke
d8d18d6657
Merge #11714: [tests] Test that mempool rejects coinbase transactions
65e91f5ed [tests] Test that mempool rejects coinbase transactions (James O'Beirne)

Pull request description:

  ![selection_063](https://user-images.githubusercontent.com/73197/32978622-b0fa9d70-cbfa-11e7-9a72-1997409e5ba8.png)

  Neither the unit nor functional tests appear to cover rejecting a transaction from acceptance to the mempool on the basis of it being a coinbase. Seems like a decent thing to have a test for.

Tree-SHA512: 53af53c975cad5d7a21c443d71a1c0ced5c70a7799b75bb44d9b7dd6ab2afbcdcaab14571540efeb848f3a1daee5e1dd856530d8f2b50582595219a1c17555ff
2020-03-14 00:57:48 -05:00
Wladimir J. van der Laan
e288330e03
Merge #11748: [Tests] Adding unit tests for GetDifficulty in blockchain.cpp.
3e1ee31 [Tests] Adding unit tests for GetDifficulty in blockchain.cpp. (sean)

Pull request description:

  blockchain.cpp has low unit test coverage. This commit is intended
  to start improving its code coverage to reasonable levels. One or more
  follow up commits will complete the task that this commit is starting
  (though the usefulness of this commit is not dependent upon later
  commits).

  Note that these tests were not written based upon a specification of how
  GetDifficulty *should* work, but rather how it actually *does* work. As
  a result, if there are any bugs in the current GetDifficulty
  implementation, these unit tests serve to lock them in rather than
  expose them.

  -- Why has blockchain.cpp been modified if this is a unit testing change?

  Since the existing GetDifficulty function relies on a global variable,
  chainActive, it was not suitable for unit testing purposes. Both the
  existing GetDifficulty function and the unit tests now call through to
  a new, more modular version of GetDifficulty that can work on any chain,
  not just chainActive.

  -- Why does blockchain_tests.cpp directly include blockchain.cpp instead
  of blockchain.h?

  While the new GetDifficulty function's signature is arguably better than
  the old one's, it still isn't great, and doesn't seem to warrant inclusion
  as part of the blockchain.h API, especially since only test code is
  directly using it. If a better way of exposing the new GetDifficulty
  function to unit tests exists, please mention it and the commit will be
  updated accordingly.

  -- Why is the test fixture named blockchain_difficulty_tests rather than
  blockchain_tests?

  The Bitcoin Core policy for naming unit test files is to match the the
  file under test ("blockchain" becomes "blockchain_tests"). While this
  commit complies with that, blockchain.cpp is a massive file, such that
  having all of the unit tests in one file will tend towards disorder.
  Since there will be a lot more tests added to this file, the intention
  is to divide up different types of tests into different test fixtures
  within the same file.

Tree-SHA512: a7dda9c2a9414d4819b4d2911f5637891dc19cecbecfc1463846161d2a78793151927a5ab911c69a5d3013f7668e75a1d78a65667cb9d83910cda439cbe84d62
2020-01-31 07:43:43 -06:00
Wladimir J. van der Laan
3654f15c91
Merge #11879: [tests] remove redundant univalue_tests.cpp
2862b56 [tests] remove redundant univalue_tests.cpp (John Newbery)

Pull request description:

  univalue unit tests were added in #4730 , and exist at `/src/test/univalue_tests.cpp` (outside the univalue tree). That test was brought into the univalue repository in https://github.com/bitcoin-core/univalue/pull/4 , which was pulled into the github repository in https://github.com/bitcoin/bitcoin/pull/11420.

  That means that the univalue test exists in two places:
  1. `/src/test/univalue_tests.cpp`
  2. `/src/univalue/test/object.cpp`

  (2) is a strict superset of (1). It adds some macros to work around boost not being a univalue dependency, and adds a few extra lines of test.

  Therefore remove `/src/test/univalue_tests.cpp`

Tree-SHA512: 3747b10bbf62e9f12363905488b29945ad559ddca68c5c03d8a362de612a51f408f41a04d3712c6889bfc1632fb1a5fa0d7df0fbf02c322b3981a6d698f501b0
2020-01-31 07:43:43 -06:00
PastaPastaPasta
4af4432cb9
Add unit tests for CPrivateSend::IsCollateralAmount (#3310)
* add privatesend collateral tests

Signed-off-by: Pasta <pasta@dashboost.org>

* move privatesend_tests.cpp in Makefile.test.include

Signed-off-by: Pasta <pasta@dashboost.org>
2020-01-31 13:57:21 +03:00
Wladimir J. van der Laan
1b2252c28c Merge #13386: SHA256 implementations based on Intel SHA Extensions
66b2cf1ccfad545a8ec3f2a854e23f647322bf30 Use immintrin.h everywhere for intrinsics (Pieter Wuille)
4c935e2eee456ff66cdfb908b0edffdd1e8a6c04 Add SHA256 implementation using using Intel SHA intrinsics (Pieter Wuille)
268400d3188200c9e3dcd3482c4853354388a721 [Refactor] CPU feature detection logic for SHA256 (Pieter Wuille)

Pull request description:

  Based on #13191.

  This adds SHA256 implementations that use Intel's SHA Extension instructions (using intrinsics). This needs GCC 4.9 or Clang 3.4.

  In addition to #13191, two extra implementations are provided:
  * (a) A variable-length SHA256 implementation using SHA extensions.
  * (b) A 2-way 64-byte input double-SHA256 implementation using SHA extensions.

  Benchmarks for 9001-element Merkle tree root computation on an AMD Ryzen 1800X system:
  * Using generic C++ code (pre-#10821): 6.1ms
  * Using SSE4 (master, #10821): 4.6ms
  * Using 4-way SSE4 specialized for 64-byte inputs (#13191): 2.8ms
  * Using 8-way AVX2 specialized for 64-byte inputs (#13191): 2.1ms
  * Using 2-way SHA-NI specialized for 64-byte inputs (this PR): 0.56ms

  Benchmarks for 32-byte SHA256 on the same system:
  * Using SSE4 (master, #10821): 190ns
  * Using SHA-NI (this PR): 53ns

  Benchmarks for 1000000-byte SHA256 on the same system:
  * Using SSE4 (master, #10821): 2.5ms
  * Using SHA-NI (this PR): 0.51ms

Tree-SHA512: 2b319e33b22579f815d91f9daf7994a5e1e799c4f73c13e15070dd54ba71f3f6438ccf77ae9cbd1ce76f972d9cbeb5f0edfea3d86f101bbc1055db70e42743b7
2019-10-01 23:20:06 +02:00
Wladimir J. van der Laan
adfd2e1a83 Merge #13408: crypto: cleanup sha256 build
f68049dd879c216d1e98b6635eec488f8e936ed4 crypto: cleanup sha256 build (Cory Fields)

Pull request description:

  Requested by @sipa in #13386.

  Rather than appending all possible cpu variants to all targets, create a convenience variable that encompasses all.

Tree-SHA512: 8e9ab2185515672b79bb7925afa4f3fbfe921bfcbe61456833d15457de4feba95290de17514344ce42ee81cc38b252476cd0c29432ac48c737c2225ed515a4bd
2019-10-01 23:20:06 +02:00