Commit Graph

129 Commits

Author SHA1 Message Date
Kittywhiskers Van Gogh
808842b1a3
refactor: define all key/value pairings as *Entry and use them instead 2024-06-28 08:14:30 +00:00
Kittywhiskers Van Gogh
9a6503d9e8
refactor: make CBlockTreeDB::Read*Index arguments const
With bonus code formatting. We cannot make the functions themselves
const as the iterators are non-const.
2024-06-28 08:14:30 +00:00
Kittywhiskers Van Gogh
e933d78a88
merge bitcoin#23438: Use spans of std::byte in serialize
continuation of de54b878 from dash#5574
2024-02-28 13:37:34 -06:00
Kittywhiskers Van Gogh
2c32a09f4e
merge bitcoin#21969: Switch serialize to uint8_t 2024-02-28 13:37:33 -06:00
Wladimir J. van der Laan
13188e1d92
Merge #19331: build: Do not include server symbols in wallet
faca73000fa8975c28f6be8be01957c1ae94ea62 ci: Install fixed version of clang-format for linters (MarcoFalke)
fa4695da4c69646b58a8fa0b6b30146bb234deb8 build: Sort Makefile.am after renaming file (MarcoFalke)
cccc2784a3bb10fa8e43be7e68207cafb12bd915 scripted-diff: Move ui_interface to the node lib (MarcoFalke)
fa72ca6a9d90d66012765b0043fd819698b94ba8 qt: Remove unused includes (MarcoFalke)
fac96e6450d595fe67168cb7afa7692da6cc9973 wallet: Do not include server symbols (MarcoFalke)
fa0f6c58c1c6d10f04c4e65a424cc51ebca50a8c Revert "Fix link error with --enable-debug" (MarcoFalke)

Pull request description:

  This reverts a hacky workaround from commit b83cc0f, which only happens to work due to compiler optimizations. Then, it actually fixes the linker error.

  The underlying problem is that the wallet includes symbols from the server (ui_interface), which usually results in linker failures. Though, in this specific case the linker failures have not been observed (unless `-O0`) because our compilers were smart enough to strip unused symbols.

  Fix the underlying problem by creating a new header-only with the needed symbol and move ui_interface to node to clarify that this is part of libbitcoin_server.

ACKs for top commit:
  Sjors:
    ACK faca730
  laanwj:
    ACK faca73000fa8975c28f6be8be01957c1ae94ea62
  hebasto:
    re-ACK faca73000fa8975c28f6be8be01957c1ae94ea62, since the [previous](https://github.com/bitcoin/bitcoin/pull/19331#pullrequestreview-434420539) review:

Tree-SHA512: e9731f249425aaea50b6db5fc7622e10078cf006721bb87989cac190a2ff224412f6f8a7dd83efd018835302337611f5839e29e15bef366047ed591cef58dfb4
2024-01-10 12:06:05 -06:00
MarcoFalke
52c8ef2feb
Merge bitcoin/bitcoin#22263: refactor: wrap CCoinsViewCursor in unique_ptr
7ad414f4bfa74595ee5726e66f3527045c02a977 doc: add comment about CCoinsViewDBCursor constructor (James O'Beirne)
0f8a5a4dd530549d37c43da52c923ac3b2af1a03 move-only(ish): don't expose CCoinsViewDBCursor (James O'Beirne)
615c1adfb07b9b466173166dc2e53ace540e4b32 refactor: wrap CCoinsViewCursor in unique_ptr (James O'Beirne)

Pull request description:

  I tripped over this one for a few hours at the beginning of the week, so I've sort of got a personal vendetta against `CCoinsView::Cursor()` returning a raw pointer.

  Specifically in the case of CCoinsViewDB, if a raw cursor is allocated and not freed, a cryptic leveldb assertion failure occurs on CCoinsViewDB destruction (`Assertion 'dummy_versions_.next_ == &dummy_versions_' failed.`).

  This is a pretty simple change.

  Related to: https://github.com/bitcoin/bitcoin/issues/21766
  See also: https://github.com/google/leveldb/issues/142#issuecomment-414418135

ACKs for top commit:
  MarcoFalke:
    review ACK 7ad414f4bfa74595ee5726e66f3527045c02a977 🔎
  jonatack:
    re-ACK 7ad414f4bfa74595ee5726e66f3527045c02a977 modulo suggestion
  ryanofsky:
    Code review ACK 7ad414f4bfa74595ee5726e66f3527045c02a977. Two new commits look good and thanks for clarifying constructor comment

Tree-SHA512: 6471d03e2de674d84b1ea0d31e25f433d52aa1aa4996f7b4aab1bd02b6bc340b15e64cc8ea07bbefefa3b5da35384ca5400cc230434e787c30931b8574c672f9
2023-12-03 20:45:01 -06:00
Kittywhiskers Van Gogh
f011c31b1a refactor: make AddressType a strong enum, remove uint8_t for address_type
Co-authored-by: Konstantin Akimov <knstqq@gmail.com>
2023-09-25 22:57:42 +05:30
Kittywhiskers Van Gogh
5c34da0675 refactor: cleanup spentindex structs, use fixed-length integers 2023-09-25 20:21:07 +05:30
Konstantin Akimov
4aa197dbdb Merge #18673: scripted-diff: Sort test includes
fa4632c41714dfaa699bacc6a947d72668a4deef test: Move boost/stdlib includes last (MarcoFalke)
fa488f131fd4f5bab0d01376c5a5013306f1abcd scripted-diff: Bump copyright headers (MarcoFalke)
fac5c373006a9e4bcbb56843bb85f1aca4d87599 scripted-diff: Sort test includes (MarcoFalke)

Pull request description:

  When writing tests, often includes need to be added or removed. Currently the list of includes is not sorted, so developers that write tests and have `clang-format` installed will either have an unrelated change (sorting) included in their commit or they will have to manually undo the sort.

  This pull preempts both issues by just sorting all includes in one commit.

  Please be aware that this is **NOT** a change to policy to enforce clang-format or any other developer guideline or process. Developers are free to use whatever tool they want, see also #18651.

  Edit: Also includes a commit to bump the copyright headers, so that the touched files don't need to be touched again for that.

ACKs for top commit:
  practicalswift:
    ACK fa4632c41714dfaa699bacc6a947d72668a4deef
  jonatack:
    ACK fa4632c41714dfaa, light review and sanity checks with gcc build and clang fuzz build

Tree-SHA512: 130a8d073a379ba556b1e64104d37c46b671425c0aef0ed725fd60156a95e8dc83fb6f0b5330b2f8152cf5daaf3983b4aca5e75812598f2626c39fd12b88b180
2023-08-29 22:00:59 -05:00
UdjinM6
48a1632688
fix: implement missing logic for additional indexes, fix bugs and logging (#5477)
## Issue being fixed or feature implemented
The logic for additional indexes is incomplete, handling of P2PK on
block disconnect is broken (luckily no one is using P2PK and reorgs are
rare) and there are a few other small issues that would be nice to have
fixed.

## What was done?
Pls see individual commits

## How Has This Been Tested?
Run `feature_dbcrash.py`, it should succeed (NOTE: it takes ~30 minutes
to complete, that's normal).
Run `feature_addressindex.py`, `feature_timestampindex.py` and
`feature_spentindex.py` (and other tests) should still succeed too.

## Breaking Changes
n/a

## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-07-24 20:54:24 +03:00
Kittywhiskers Van Gogh
6bf39d7632 merge bitcoin#19806: UTXO snapshot activation 2023-06-06 22:38:56 +05:30
fanquake
497ad8d4b5 Merge #18087: Get rid of VARINT default argument
0e0fa27acb74b4f0075afcf59a0dff51a21baddb Get rid of VARINT default argument (Pieter Wuille)

Pull request description:

  This removes the need for the non-strandard use of variadic macros.

ACKs for top commit:
  ryanofsky:
    Code review ACK 0e0fa27acb74b4f0075afcf59a0dff51a21baddb. Only change since last review reverting outdated documentation change from earlier version of pr
  jonatack:
    ACK 0e0fa27 code review, built/ran tests/bitcoind
  practicalswift:
    ACK 0e0fa27acb74b4f0075afcf59a0dff51a21baddb -- diff looks correct
  MarcoFalke:
    ACK 0e0fa27acb74b4f0075afcf59a0dff51a21baddb 📯

Tree-SHA512: 6e335e4b586d62112b7260a12481cd949d1b3bbdb83edf8db690348f0a01852e68504336ff3e072e5131a7c8cb404ef11a2f786f842b8d08bbf6ea0e688777b1
2023-05-24 12:43:57 -05:00
Kittywhiskers Van Gogh
39ee56ab68 merge bitcoin#21016: remove boost::thread_group usage 2023-01-19 03:42:51 +00:00
Kittywhiskers Van Gogh
a52eb1e72f merge bitcoin#18758: Remove unused boost/thread 2023-01-19 03:42:50 +00:00
Kittywhiskers Van Gogh
898fef5c01 merge bitcoin#21404: Remove MakeUnique<T>() 2022-10-20 16:08:45 -05:00
Kittywhiskers Van Gogh
2fb3a9fbf5 merge bitcoin#18637: allow cache resize after init
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-05-23 10:39:43 +05:30
Kittywhiskers Van Gogh
c25b668125 merge bitcoin#16362: Add bilingual_str type 2022-04-06 00:08:01 +05:30
Vijay Manikpuri
3148450032
Merge #16129: refactor: Remove unused includes (#4623)
67f4e9c522 Include core_io.h from core_read.cpp (practicalswift)
eca9767673 Make reasoning about dependencies easier by not including unused dependencies (practicalswift)

Pull request description:

  Make reasoning about dependencies easier by not including unused dependencies.

  Please note that the removed headers are _not_ "transitively included" by other still included headers. Thus the removals are real.

  As an added bonus this change means less work for the preprocessor/compiler. At least 51 393 lines of code no longer needs to be processed:

  ```
  $ git diff -u HEAD~1 | grep -E '^\-#include ' | cut -f2 -d"<" | cut -f1 -d">" | \
        sed 's%^%src/%g' | xargs cat | wc -l
  51393
  ```

  Note that 51 393 is the lower bound: the real number is likely much higher when taking into account transitively included headers :-)

ACKs for commit 67f4e9:

Tree-SHA512: 0c8868aac59813f099ce53d5307eed7962dd6f2ff3546768ef9e5c4508b87f8210f1a22c7e826c3c06bebbf28bdbfcf1628ed354c2d0fdb9a31a42cefb8fdf13

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2022-01-03 13:10:02 -06:00
Wladimir J. van der Laan
29f354e8ec
Merge #17059: util: Simplify path argument for CBlockTreeDB ctor
c2bb3919a8cc52ea35d424c1f31f9474f1ea4911 util: Simplify path argument for CBlockTreeDB ctor (Hennadii Stepanov)

Pull request description:

  This PR:
  - simplifies path argument (`datadir/blocks/index`) for `CBlockTreeDB`  constructor
  - does not change behavior as `GetBlocksDir()` with unset "-blocksdir" returns the same path
  - improves code readability

ACKs for top commit:
  MarcoFalke:
    ACK c2bb3919a8cc52ea35d424c1f31f9474f1ea4911
  laanwj:
    ACK c2bb3919a8cc52ea35d424c1f31f9474f1ea4911
  promag:
    ACK c2bb3919a8cc52ea35d424c1f31f9474f1ea4911.

Tree-SHA512: 646a0a3a31e2f419b05f696cbdfb7d8987f1d89ec0797b72464ae05680fd5f95f6469be0ea5b56f772434c49d48504cd9cf9760c05d4054d11349d502e157ee2
2021-11-03 12:24:05 -04:00
Kittywhiskers Van Gogh
84339a4b84 merge bitcoin#16443: have CCoins* data managed under CChainState 2021-10-22 08:15:38 +05:30
Kittywhiskers Van Gogh
a620d941d3 merge bitcoin#16194: share blockmetadata with BlockManager
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-10-22 08:15:38 +05:30
Wladimir J. van der Laan
d0724b5ee1
Merge #13258: uint256: Remove unnecessary crypto/common.h dependency
bf2e01097 uint256: Remove unnecessary crypto/common.h use (Karl-Johan Alm)

Pull request description:

  This is an alternative to #13242 which keeps the `ReadLE64` part, but moves the `crypto/common.h` dependency into `crypto/common.h` as a function outside of `uint256`.

  **Reason:** this change will remove dependencies for `uint256` to `crypto/common.h`, `compat/endian.h`, and `compat/byteswap.h`.

  This PR removes the need to update tests to be endian-aware/-independent, but keeps the (arguably dubious) `ReadLE64` part (which was only introduced to fix the tests, not for any functionality).

Tree-SHA512: 78b35123cdb185b3b3ec59aba5ca8a5db72624d147f2d6a5484ffa5ce626a72f782a01dc6893fc8f5619b03e2eae7b5a03b0df5d43460f3bda428e719e188aec
2021-10-21 17:41:33 +05:30
Jonas Schnelli
756af776fd Merge #12980: Allow quicker shutdowns during LoadBlockIndex()
af5fa82b6 Allow quicker shutdowns during LoadBlockIndex() (Jonas Schnelli)

Pull request description:

ACKs for commit af5fa8:
  promag:
    utACK af5fa82b676a36e60eda080ca0a946bdfffefd49.
  practicalswift:
    utACK af5fa82b676a36e60eda080ca0a946bdfffefd49

Tree-SHA512: 1c64dcc5d8a9d3411553257cd5a598dcd29be981660e5bca9283c1d957dc56798abcf41d9969cd573088137597a23e48e62a8c476c463d3f176b86a10048f47b
2021-07-12 17:11:08 -05:00
MarcoFalke
570d573786
Merge #13774: Return void instead of bool for functions that cannot fail
d78a8dc3e8 Return void instead of bool for functions that cannot fail (practicalswift)

Pull request description:

  Return `void` instead of `bool` for functions that cannot fail:
  * `CBlockTreeDB::ReadReindexing(...)`
  * `CChainState::ResetBlockFailureFlags(...)`
  * `CTxMemPool::addUnchecked(...)`
  * `CWallet::CommitTransaction(...)`
  * `CWallet::LoadDestData(...)`
  * `CWallet::LoadKeyMetadata(...)`
  * `CWallet::LoadScriptMetadata(...)`
  * `CWallet::LoadToWallet(...)`
  * `CWallet::SetHDChain(...)`
  * `CWallet::SetHDSeed(...)`
  * `PendingWalletTx::commit(...)`
  * `RemoveLocal(...)`
  * `SetMinVersion(...)`
  * `StartHTTPServer(...)`
  * `StartRPC(...)`
  * `TorControlConnection::Disconnect(...)`

  Some of the functions can fail by throwing.

  Found by manually inspecting the following candidate functions:

  ```
  $ git grep -E '(^((static|virtual|inline|friend)[^a-z])*[^a-z]*bool [^=]*\(|return true|return false)' -- "*.cpp" "*.h"
  ```

Tree-SHA512: c0014e045362dbcd1a0cc8f69844e7b8cbae4f538e7632028daeca3a797ac11d8d3d86ebc480bedcb8626df3e96779d592747d52a12556fc49921b114fa0ccc6
2021-07-02 12:59:28 +03:00
Wladimir J. van der Laan
4cd32e43e4
Merge #13235: Break circular dependency: init -> * -> init by extracting shutdown.h
1fabd59e7 Break circular dependency: init -> * -> init by extracting shutdown.h (Ben Woosley)
e62fdfeea Drop unused init.h includes (Ben Woosley)

Pull request description:

  Most includers just wanted to react to pending shutdown.

  This isolates access to `fRequestShutdown` and limits access to the shutdown api functions, including the new `CancelShutdown` for setting it to `false`.

Tree-SHA512: df42f75dfbba163576710e9a67cf1228531fd99d70a2f187bfba0bcc476d6749cf88180a97e66a81bb5b6c3c7f0917de7402d26039ba7b644cb7509b02f7e267
2021-07-02 00:42:18 +03:00
Wladimir J. van der Laan
22e4729496
Merge #13243: Make reusable base class for auxiliary indices
ec3073a274bf7affe1b8c87a10f75d126f5ac027 index: Move index DBs into index/ directory. (Jim Posen)
89eddcd365e9a2218648f5cc5b9f22b28023f50a index: Remove TxIndexDB from public interface of TxIndex. (Jim Posen)
2318affd27de436ddf9d866a4b82eed8ea2e738b MOVEONLY: Move BaseIndex to its own file. (Jim Posen)
f376a4924109af2496b5fd16a787299eb039f1c8 index: Generalize logged statements in BaseIndex. (Jim Posen)
61a1226d87d80234b2be123c5cad07534c318cfb index: Extract logic from TxIndex into reusable base class. (Jim Posen)
e5af5fc6fb4658599b940d1d50853129b31b8766 db: Make reusable base class for index databases. (Jim Posen)
9b0ec1a7f9ffae816fd5ca32ff7e7559640b6f6d db: Remove obsolete methods from CBlockTreeDB. (Jim Posen)

Pull request description:

  This refactors most of the logic in TxIndex into a reusable base class for other indices. There are two commits moving code between files, which may be be more easily reviewed using `git diff --color-moved` (https://blog.github.com/2018-04-05-git-217-released/).

  The motivation for this is to support BIP 157 by indexing block filters.

  <!-- Reviewable:start -->
  ---
  This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/bitcoin/bitcoin/13243)
  <!-- Reviewable:end -->

Tree-SHA512: 0857f04df2aa920178dab2eb8e57984d8eb4d5010deca9971190358479e05b6672ccca2a08af0a7ac9fe02afb947be84cf35a3693204d0667263c6add2959cbf
2021-07-02 00:42:14 +03:00
Kittywhiskers Van Gogh
46151acdb5 partial bitcoin#16889: Add some general std::vector utility functions 2021-06-27 12:03:14 +05:30
Kittywhiskers Van Gogh
ef4dfa8524 merge bitcoin#14555: Move util files to directory
(script modified to account for Dash backports, doesn't account for rebasing)

------------- BEGIN SCRIPT ---------------
mkdir -p src/util
git mv src/util.h src/util/system.h
git mv src/util.cpp src/util/system.cpp
git mv src/utilmemory.h src/util/memory.h
git mv src/utilmoneystr.h src/util/moneystr.h
git mv src/utilmoneystr.cpp src/util/moneystr.cpp
git mv src/utilstrencodings.h src/util/strencodings.h
git mv src/utilstrencodings.cpp src/util/strencodings.cpp
git mv src/utiltime.h src/util/time.h
git mv src/utiltime.cpp src/util/time.cpp
git mv src/utilasmap.h src/util/asmap.h
git mv src/utilasmap.cpp src/util/asmap.cpp
git mv src/utilstring.h src/util/string.h
git mv src/utilstring.cpp src/util/string.cpp

gsed -i 's/<util\.h>/<util\/system\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilmemory\.h>/<util\/memory\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilmoneystr\.h>/<util\/moneystr\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilstrencodings\.h>/<util\/strencodings\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utiltime\.h>/<util\/time\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilasmap\.h>/<util\/asmap\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilstring\.h>/<util\/string\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')

gsed -i 's/BITCOIN_UTIL_H/BITCOIN_UTIL_SYSTEM_H/g' src/util/system.h
gsed -i 's/BITCOIN_UTILMEMORY_H/BITCOIN_UTIL_MEMORY_H/g' src/util/memory.h
gsed -i 's/BITCOIN_UTILMONEYSTR_H/BITCOIN_UTIL_MONEYSTR_H/g' src/util/moneystr.h
gsed -i 's/BITCOIN_UTILSTRENCODINGS_H/BITCOIN_UTIL_STRENCODINGS_H/g' src/util/strencodings.h
gsed -i 's/BITCOIN_UTILTIME_H/BITCOIN_UTIL_TIME_H/g' src/util/time.h
gsed -i 's/BITCOIN_UTILASMAP_H/BITCOIN_UTIL_ASMAP_H/g' src/util/asmap.h
gsed -i 's/BITCOIN_UTILSTRING_H/BITCOIN_UTIL_STRING_H/g' src/util/string.h

gsed -i 's/ util\.\(h\|cpp\)/ util\/system\.\1/g' src/Makefile.am
gsed -i 's/utilmemory\.\(h\|cpp\)/util\/memory\.\1/g' src/Makefile.am
gsed -i 's/utilmoneystr\.\(h\|cpp\)/util\/moneystr\.\1/g' src/Makefile.am
gsed -i 's/utilstrencodings\.\(h\|cpp\)/util\/strencodings\.\1/g' src/Makefile.am
gsed -i 's/utiltime\.\(h\|cpp\)/util\/time\.\1/g' src/Makefile.am
gsed -i 's/utilasmap\.\(h\|cpp\)/util\/asmap\.\1/g' src/Makefile.am
gsed -i 's/utilstring\.\(h\|cpp\)/util\/string\.\1/g' src/Makefile.am

gsed -i 's/-> util ->/-> util\/system ->/' test/lint/lint-circular-dependencies.sh
gsed -i 's/src\/util\.cpp/src\/util\/system\.cpp/g' test/lint/lint-format-strings.py test/lint/lint-locale-dependence.sh
gsed -i 's/src\/utilmoneystr\.cpp/src\/util\/moneystr\.cpp/g' test/lint/lint-locale-dependence.sh
gsed -i 's/src\/utilstrencodings\.\(h\|cpp\)/src\/util\/strencodings\.\1/g' test/lint/lint-locale-dependence.sh
------------- END   SCRIPT ---------------
2021-06-27 12:03:13 +05:30
UdjinM6
bcc8b35194
Merge pull request #4178 from UdjinM6/backports-0.17-pr30
Merge #13033: Build txindex in parallel with validation
2021-06-05 22:08:47 +03:00
MarcoFalke
6a36c4e4b1
partial merge #18317: Serialization improvements step 6 (all except wallet/gui)
f9ee0f37c28f604bc82dab502ce229c66ef5b3b9 Add comments to CustomUintFormatter (Pieter Wuille)
4eb5643e3538863c9d2ff261f49a9a1b248de243 Convert everything except wallet/qt to new serialization (Pieter Wuille)
2b1f85e8c52c8bc5a17eae4c809eaf61d724af98 Convert blockencodings_tests to new serialization (Pieter Wuille)
73747afbbeb013669faf4c4d2c0903cec4526fb0 Convert merkleblock to new serialization (Pieter Wuille)
d06fedd1bc26bf5bf2b203d4445aeaebccca780e Add SER_READ and SER_WRITE for read/write-dependent statements (Russell Yanofsky)
6f9a1e5ad0a270d3b5a715f3e3ea0911193bf244 Extend CustomUintFormatter to support enums (Russell Yanofsky)
769ee5fa0011ae658770586442715452a656559d Merge BigEndian functionality into CustomUintFormatter (Pieter Wuille)

Pull request description:

  The next step of changes from #10785.

  This:
  * Adds support for enum serialization to `CustomUintFormatter`, used in `CAddress` for service flags.
  * Merges `BigEndian` into `CustomUintFormatter`, used in `CNetAddr` for port numbers.
  * Converts everything (except wallet and gui) to use the new serialization framework.

ACKs for top commit:
  MarcoFalke:
    re-ACK f9ee0f37c2, only change is new documentation commit for CustomUintFormatter 📂
  ryanofsky:
    Code review ACK f9ee0f37c28f604bc82dab502ce229c66ef5b3b9. Just new commit adding comment since last review
  jonatack:
    Code review re-ACK f9ee0f37c28f604bc82dab502ce229c6 only change since last review is an additional commit adding Doxygen documentation for `CustomUintFormatter`.

Tree-SHA512: e7a0a36afae592d5a4ff8c81ae04d858ac409388e361f2bc197d9a78abca45134218497ab2dfd6d031e0cce0ca586cf857077b7c6ce17fccf67e2d367c1b6cd4
2021-05-28 11:39:24 +03:00
Pieter Wuille
7ff6515c88
Merge #13033: Build txindex in parallel with validation
9b2704777c [doc] Include txindex changes in the release notes. (Jim Posen)
ed77dd6b30 [test] Simple unit test for TxIndex. (Jim Posen)
6d772a3d44 [rpc] Public interfaces to GetTransaction block until synced. (Jim Posen)
a03f804f2a [index] Move disk IO logic from GetTransaction to TxIndex::FindTx. (Jim Posen)
e0a3b80033 [validation] Replace tx index code in validation code with TxIndex. (Jim Posen)
8181db88f6 [init] Initialize and start TxIndex in init code. (Jim Posen)
f90c3a62f5 [index] TxIndex method to wait until caught up. (Jim Posen)
70d510d93c [index] Allow TxIndex sync thread to be interrupted. (Jim Posen)
94b4f8bbb9 [index] TxIndex initial sync thread. (Jim Posen)
34d68bf3a3 [index] Create new TxIndex class. (Jim Posen)
c88bcec93f [db] Migration for txindex data to new, separate database. (Jim Posen)
0cb8303241 [db] Create separate database for txindex. (Jim Posen)

Pull request description:

  I'm re-opening #11857 as a new pull request because the last one stopped loading for people

  -------------------------------

  This refactors the tx index code to be in it's own class and get built concurrently with validation code. The main benefit is decoupling and moving the txindex into a separate DB. The primary motivation is to lay the groundwork for other indexers that might be desired (such as the [compact filters](https://github.com/bitcoin/bips/pull/636)). The basic idea is that the TxIndex spins up its own thread, which first syncs the txindex to the current block index, then once in sync the BlockConnected ValidationInterface hook writes new blocks.

  ### DB changes

  At the suggestion of some other developers, the txindex has been split out into a separate database. A data migration runs at startup on any nodes with a legacy txindex. Currently the migration blocks node initialization until complete.

  ### Open questions

  - Should the migration of txindex data from the old DB to the new DB block in init or should it happen in a background thread? The downside to backgrounding it is that `getrawtransaction` would return an error message saying the txindex is syncing while the migration is running.

  ### Impact

  In a sample size n=1 test where I synced nodes from scratch, the average time [Index writing](https://github.com/bitcoin/bitcoin/blob/master/src/validation.cpp#L1903) was 3.36ms in master and 1.72ms in this branch. The average time between `UpdateTip` log lines for sequential blocks between 400,000 and IBD end on mainnet was 0.297204s in master and 0.286134s in this branch. Most likely this is just variance in IBD times, but I can try with some more trials if people want.

Tree-SHA512: 451fd7d95df89dfafceaa723cdf0f7b137615b531cf5c5035cfb54e9ccc2026cec5ac85edbcf71b7f4e2f102e36e9202b8b3a667e1504a9e1a9976ab1f0079c4
2021-05-25 13:48:04 +03:00
Kittywhiskers Van Gogh
bfe5971359
Merge #17957: Convert compression.h to new serialization framework
4de934b9b5

Partial #17957: Add FORMATTER_METHODS, similar to SERIALIZE_METHODS, but for formatters

ca34c5cba5
2021-04-18 17:10:25 -04:00
Wladimir J. van der Laan
6ca09aaae3
Merge #9753: Add static_assert to prevent VARINT(<signed value>)
499d95e27 Add static_assert to prevent VARINT(<signed value>) (Russell Yanofsky)

Pull request description:

  Using VARINT with signed types is dangerous because negative values will appear to serialize correctly, but then deserialize as positive values mod 128.

  This commit changes the VARINT macro to trigger a compile error by default if called with an signed value, and it updates existing broken uses of VARINT to pass a special flag that lets them keep working with no changes in behavior.

  There is some discussion about this issue here: https://github.com/bitcoin/bitcoin/pull/9693#issuecomment-278701473. I think another good change along these lines would be to make `GetSizeOfVarInt` and `WriteVarInt` throw exceptions if they are passed numbers less than 0 to serialize. But unlike this change, that would be a change in runtime behavior, and need more consideration.

Tree-SHA512: 082c65598cfac6dc1da042bdb47dbc9d5d789fc849fe52921cc238578588f4e5ff976c8b4b2ce42cb75290eb14f3b42ea76e26202c223c5b2aa63ef45c2ea3cc
2020-12-17 15:20:31 +03:00
Pieter Wuille
ec71097801
Merge #12683: Fix more constness violations in serialization code
172f5fa738 Support deserializing into temporaries (Pieter Wuille)
2761bca997 Merge READWRITEMANY into READWRITE (Pieter Wuille)

Pull request description:

  This is another fragment of improvements from #10785.

  The current serialization code does not support serializing/deserializing from/to temporaries (like `s >> CFlatData(script)`). As a result, there are many invocations of the `REF` macro which in addition to changing the reference type also changes the constness. This is unnecessary in C++11 as we can use rvalue references now instead.

  The first commit is an extra simplification we can make that removes the duplication of code between `READWRITE` and `READWRITEMANY` (and related functions).

Tree-SHA512: babfa9cb268cc3bc39917e4f0a90e4651c33d85032161e16547a07f3b257b7ca7940e0cbfd69f09439d26fafbb1a6cf6359101043407e2c7aeececf7f20b6eed
2020-12-17 05:07:07 +03:00
Wladimir J. van der Laan
ba32ad8620
Merge #12653: Allow to optional specify the directory for the blocks storage
a192636 -blocksdir: keep blockindex leveldb database in datadir (Jonas Schnelli)
f38e4fd QA: Add -blocksdir test (Jonas Schnelli)
386a6b6 Allow to optional specify the directory for the blocks storage (Jonas Schnelli)

Pull request description:

  Since the actual block files taking up more and more space, it may be desirable to have them stored in a different location then the data directory (use case: SSD for chainstate, etc., HD for blocks).

  This PR adds a `-blocksdir` option that allows one to keep the blockfiles and the blockindex external from the data directory (instead of creating symlinks).

  I fist had an option to keep the blockindex within the datadir, but seems to make no sense since accessing the index will (always) lead to access (r/w) the block files.

Tree-SHA512: f8b9e1a681679eac25076dc30e45e6e12d4b2d9ac4be907cbea928a75af081dbcb0f1dd3e97169ab975f73d0bd15824c00c2a34638f3b284b39017171fce2409
2020-10-14 13:26:32 -04:00
pasta
c49e506176 scripted-diff: Merge #13753: Remove trailing whitespaces
-BEGIN VERIFY SCRIPT-

sed --in-place'' --regexp-extended 's/[[:space:]]+$//g' $(git grep -I --files-with-matches --extended-regexp '[[:space:]]+$' -- src test  ':!*.svg' ':!src/crypto/sha256_sse4*' ':!src/leveldb' ':!src/qt/locale' ':!src/secp256k1' ':!src/univalue')

-END VERIFY SCRIPT-

Signed-off-by: pasta <pasta@dashboost.org>
2020-07-28 20:23:12 -05:00
Wladimir J. van der Laan
101f73bb6d
Merge #12891: [logging] add lint-logs.sh to check for newline termination.
d207207 [logging] add lint-logs.sh to check for newline termination. (John Newbery)
5c21e6c [logging] Comment all continuing logs. (John Newbery)

Pull request description:

  Check that all calls to LogPrintf() are terminated by a newline,
  except those that are explicitly marked as 'continued' logs.

Tree-SHA512: fe5162b2b2df1e8a4c807da87584fa9af97a6b8377e4090fe0caa136d90bf29a487a123cde94569bdce7101fee3478196d99aa13f1212e24bfe5f41c773604fc
2020-06-11 23:20:48 -05:00
Alexander Block
0e56e32c22
Add cache for CBlockTreeDB::HasTxIndex (#3402)
* Add cache for CBlockTreeDB::HasTxIndex

* Also update mapHasTxIndexCache in WriteTxIndex
2020-04-08 22:18:33 +02:00
PastaPastaPasta
8a1ec935a0
Backport 11651 (#3358)
* scripted-diff: Replace #include "" with #include <> (ryanofsky)

-BEGIN VERIFY SCRIPT-
for f in \
  src/*.cpp \
  src/*.h \
  src/bench/*.cpp \
  src/bench/*.h \
  src/compat/*.cpp \
  src/compat/*.h \
  src/consensus/*.cpp \
  src/consensus/*.h \
  src/crypto/*.cpp \
  src/crypto/*.h \
  src/crypto/ctaes/*.h \
  src/policy/*.cpp \
  src/policy/*.h \
  src/primitives/*.cpp \
  src/primitives/*.h \
  src/qt/*.cpp \
  src/qt/*.h \
  src/qt/test/*.cpp \
  src/qt/test/*.h \
  src/rpc/*.cpp \
  src/rpc/*.h \
  src/script/*.cpp \
  src/script/*.h \
  src/support/*.cpp \
  src/support/*.h \
  src/support/allocators/*.h \
  src/test/*.cpp \
  src/test/*.h \
  src/wallet/*.cpp \
  src/wallet/*.h \
  src/wallet/test/*.cpp \
  src/wallet/test/*.h \
  src/zmq/*.cpp \
  src/zmq/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>

* scripted-diff: Replace #include "" with #include <> (Dash Specific)

-BEGIN VERIFY SCRIPT-
for f in \
  src/bls/*.cpp \
  src/bls/*.h \
  src/evo/*.cpp \
  src/evo/*.h \
  src/governance/*.cpp \
  src/governance/*.h \
  src/llmq/*.cpp \
  src/llmq/*.h \
  src/masternode/*.cpp \
  src/masternode/*.h \
  src/privatesend/*.cpp \
  src/privatesend/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>

* build: Remove -I for everything but project root

Remove -I from build system for everything but the project root,
and built-in dependencies.

Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/Makefile.test.include

* qt: refactor: Use absolute include paths in .ui files

* qt: refactor: Changes to make include paths absolute

This makes all include paths in the GUI absolute.

Many changes are involved as every single source file in
src/qt/ assumes to be able to use relative includes.

Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/qt/dash.cpp
#	src/qt/optionsmodel.cpp
#	src/qt/test/rpcnestedtests.cpp

* test: refactor: Use absolute include paths for test data files

* Recommend #include<> syntax in developer notes

* refactor: Include obj/build.h instead of build.h

* END BACKPORT #11651 Remove trailing whitespace causing travis failure

* fix backport 11651

Signed-off-by: Pasta <pasta@dashboost.org>

* More of 11651

* fix blockchain.cpp

Signed-off-by: pasta <pasta@dashboost.org>

* Add missing "qt/" in includes

* Add missing "test/" in includes

* Fix trailing whitespaces

Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-03-20 01:46:56 +03:00
Wladimir J. van der Laan
fdf3f25a0a
Merge #10969: Declare single-argument (non-converting) constructors "explicit"
64fb0ac Declare single-argument (non-converting) constructors "explicit" (practicalswift)

Pull request description:

  Declare single-argument (non-converting) constructors `explicit`.

  In order to avoid unintended implicit conversions.

  For a more thorough discussion, see ["C.46: By default, declare single-argument constructors explicit"](http://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#c46-by-default-declare-single-argument-constructors-explicit) in the C++ Core Guidelines (Stroustrup & Sutter).

Tree-SHA512: e0c6922e56b11fa402621a38656d8b1122d16dd8f160e78626385373cf184ac7f26cb4c1851eca47e9b0dbd5e924e39a85c3cbdcb627a05ee3a655ecf5f7a0f1
2020-01-10 10:33:57 -06:00
Jonas Schnelli
2aa7ca5723
Merge #10770: Drop upgrade-cancel callback registration for a generic "cancelable"
ee4d1493e Drop upgrade-cancel callback registration for a generic "resumeable" (Matt Corallo)

Pull request description:

  Instead of passing a StartShutdown reference all the way up from
  txdb, give ShowProgress a "cancelable" boolean, as StartShutdown
  is pretty much always what you'll want to use to cancel. Use the
  same boolean to allow cancel during initial block verification.

Tree-SHA512: 515817aaa4b9e3e856200e00be9c2d44ecfa2d4f288fe3e02116105fe85de2650c13076ee7e45396ec1ce6ab45e53b0477cddda7cfdee5b3bd0589cb81a4c346
2020-01-03 08:35:55 -06:00
Alexander Block
b4aefb513d Also consider txindex for transactions in AlreadyHave() (#3126)
This avoids many false negatives where TXs are announced to us which are
already mined and then were spent later.
2019-10-01 17:24:42 +03:00
Pasta
60b3ae1821
add gArgs
Signed-off-by: Pasta <pasta@dashboost.org>
2019-08-02 12:41:37 -05:00
Wladimir J. van der Laan
453d756571
Merge #10148: Use non-atomic flushing with block replay
176c021 [qa] Test non-atomic chainstate writes (Suhas Daftuar)
d6af06d Dont create pcoinsTip until after ReplayBlocks. (Matt Corallo)
eaca1b7 Random db flush crash simulator (Pieter Wuille)
0580ee0 Adapt memory usage estimation for flushing (Pieter Wuille)
013a56a Non-atomic flushing using the blockchain as replay journal (Pieter Wuille)
b3a279c [MOVEONLY] Move LastCommonAncestor to chain (Pieter Wuille)

Tree-SHA512: 47ccc62303f9075c44d2a914be75bd6969ff881a857a2ff1227f05ec7def6f4c71c46680c5a28cb150c814999526797dc05cf2701fde1369c06169f46eccddee
2019-08-01 14:57:28 -05:00
Pieter Wuille
dc97d48bbb
Merge #10760: Avoid dereference-of-casted-pointer
0aadc11fd Avoid dereference-of-casted-pointer (Pieter Wuille)

Pull request description:

  And prefer a static_cast to the intended reference type.

Tree-SHA512: e83b20023a4dca6029b46f7040a8a6fd54e1b42112ec0c87c3c3b567ed641de97a9e2335b57a2efb075491f641e5b977bc226a474276bea0c3c3c71d8d6ac54d
2019-07-24 11:59:09 -05:00
Wladimir J. van der Laan
8677d7599b
Merge #9176: Globals: Pass Consensus::Params through CBlockTreeDB::LoadBlockIndexGuts()
b324b28 Globals: Pass Consensus::Params through CBlockTreeDB::LoadBlockIndexGuts() (Jorge Timón)

Tree-SHA512: 9ef561a026960dc69df52603f4d60246c913bdd127465e8238f0954c6bbbc6b8b3b039bab70df38be1cbf5d06788fb631a2236b49c0167633d54366103c7c055
2019-07-11 10:34:46 -05:00
Pieter Wuille
2e050ea744
Merge #10395: Replace boost::function with std::function (C++11)
1b936f5 Replace boost::function with std::function (C++11) (practicalswift)

Tree-SHA512: c4faec8cf3f801842010976115681f68ffa08fbc97ba50b22e95c936840f47e1b3bd8d7fd2f5b4e094b5a46bf3d29fc90b69d975a99e77322c0d19f8a00d53d3
2019-06-24 11:39:34 -05:00
PastaPastaPasta
29194b1f5a Backport Bitcoin#9424, Bitcoin#10123 and Bitcoin#10153 (#2918)
* Contains dashification. disables `-debug dash`
Merge #9424: Change LogAcceptCategory to use uint32_t rather than sets of strings.

6b3bb3d Change LogAcceptCategory to use uint32_t rather than sets of strings. (Gregory Maxwell)

Tree-SHA512: ebb5bcf9a7d00a32dd1390b727ff4d29330a038423611da01268d8e1d2c0229e52a1098e751d4e6db73ef4ae862e1e96d38249883fcaf12b68f55ebb01035b34
Signed-off-by: Pasta <Pasta@dash.org>

31 -> 32

Signed-off-by: Pasta <Pasta@dash.org>

* Merge #10123: Allow debug logs to be excluded from specified component

3bde556 Add -debugexclude option to switch off logging for specified components (John Newbery)

Tree-SHA512: 30202e3f2085fc2fc5dd4bedb92988f4cb162c612a42cf8f6395a7da326f34975ddc347f82bc4ddca6c84c438dc0cc6e87869f90c7ff88105dbeaa52a947fa43

* bump to uint64_t due to added Dash codes

Signed-off-by: Pasta <Pasta@dash.org>

* bump to uint64_t due to added Dash codes cont.

Signed-off-by: Pasta <Pasta@dash.org>

* string -> BCLog format

Signed-off-by: Pasta <Pasta@dash.org>

* uint32_t -> uint64_t

Signed-off-by: Pasta <Pasta@dash.org>

* Fix CBatchedLogger

* Fix most fDebug-s

* Fix `debug` rpc

* Fix BENCH and RAND conflicts

* Add ALERT and use it

* Update LogPrint-s in dash-specific code

* Tweak few log categories

Specifically:
- use PRIVATESEND in `CPrivateSendClientManager::GetRandomNotUsedMasternode()`
- use ZMQ in `CZMQPublishRawGovernanceVoteNotifier::NotifyGovernanceVote()` and `CZMQPublishRawGovernanceObjectNotifier::NotifyGovernanceObject()`

* Drop no longer used MASTERNODE category

* Merge #10153: logging: Fix off-by-one for shrinkdebugfile default

faab624 logging: Fix off-by-one for shrinkdebugfile (MarcoFalke)

Tree-SHA512: d6153e06067906172ff0611af9e585a3ecf0a7d56925b6ad7c12e75aa802441047059b9b6f6c78e79916c3f2abc8f1998bfd2d5b84201ec6421f727c08da3c21

* Shift dash-specific log categories to start from `1ul << 32` to avoid potential future conflicts with bitcoin ones

* Fix `dash` category

* remove debugCategories

Signed-off-by: Pasta <Pasta@dash.org>

* Prepend "std::" to find call

* Check for BCLog::PRIVATESEND instead of logCategories != BCLog::NONE

* Use BCLog::MNPAYMENTS category instead of checking for logCategories != BCLog::NONE

* Move "End Dash" comment below "ALERT"

When adding new entries here, we'll otherwise get confused with ordering
and might end up forgetting that adding something Dash specific must
continue with the bit after 43.
2019-05-23 00:51:39 +03:00
MarcoFalke
25074aeb43 Merge #9644: [refactor] Remove using namespace <xxx> from src/
b7b48c8 Refactor: Remove using namespace <xxx> from src/*.cpp. (Karl-Johan Alm)
2018-01-23 09:24:03 +01:00
Wladimir J. van der Laan
19a2d668cf Merge #9039: Various serialization simplifcations and optimizations
d59a518 Use fixed preallocation instead of costly GetSerializeSize (Pieter Wuille)
25a211a Add optimized CSizeComputer serializers (Pieter Wuille)
a2929a2 Make CSerAction's ForRead() constexpr (Pieter Wuille)
a603925 Avoid -Wshadow errors (Pieter Wuille)
5284721 Get rid of nType and nVersion (Pieter Wuille)
657e05a Make GetSerializeSize a wrapper on top of CSizeComputer (Pieter Wuille)
fad9b66 Make nType and nVersion private and sometimes const (Pieter Wuille)
c2c5d42 Make streams' read and write return void (Pieter Wuille)
50e8a9c Remove unused ReadVersion and WriteVersion (Pieter Wuille)
2018-01-15 06:14:15 +01:00