Commit Graph

17165 Commits

Author SHA1 Message Date
Andrew Chow
f293c046f4
Merge #16528: Native Descriptor Wallets using DescriptorScriptPubKeyMan
223588b1bbc63dc57098bbd0baa48635e0cc0b82 Add a --descriptors option to various tests (Andrew Chow)
869f7ab30aeb4d7fbd563c535b55467a8a0430cf tests: Add RPCOverloadWrapper which overloads some disabled RPCs (Andrew Chow)
cf060628590fab87d73f278e744d70ef2d5d81db Correctly check for default wallet (Andrew Chow)
886e0d75f5fea2421190aa4812777d89f68962cc Implement CWallet::IsSpentKey for non-LegacySPKMans (Andrew Chow)
3c19fdd2a2fd5394fcfa75b2ba84ab2277cbdabf Return error when no ScriptPubKeyMan is available for specified type (Andrew Chow)
388ba94231f2f10a0be751c562cdd4650510a90a Change wallet_encryption.py to use signmessage instead of dumpprivkey (Andrew Chow)
1346e14831489f9c8f53a08f9dfed61d55d53c6f Functional tests for descriptor wallets (Andrew Chow)
f193ea889ddb53d9a5c47647966681d525e38368 add importdescriptors RPC and tests for native descriptor wallets (Hugo Nguyen)
ce24a944940019185efebcc5d85eac458ed26016 Add IsLegacy to CWallet so that the GUI knows whether to show watchonly (Andrew Chow)
1cb42b22b11c27e64462afc25a94b2fc50bfa113 Generate new descriptors when encrypting (Andrew Chow)
82ae02b1656819f4bd5023b8955447e1d4ea8692 Be able to create new wallets with DescriptorScriptPubKeyMans as backing (Andrew Chow)
b713baa75a62335ab9c0eed9ef76a95bfec30668 Implement GetMetadata in DescriptorScriptPubKeyMan (Andrew Chow)
8b9603bd0b443e2f7984eb72bf2e21cf02af0bcb Change GetMetadata to use unique_ptr<CKeyMetadata> (Andrew Chow)
72a9540df96ffdb94f039b9c14eaacdc7d961196 Implement FillPSBT in DescriptorScriptPubKeyMan (Andrew Chow)
84b4978c02102171775c77a45f6ec198930f0a88 Implement SignMessage for descriptor wallets (Andrew Chow)
bde7c9fa38775a81d53ac0484fa9c98076a0c7d1 Implement SignTransaction in DescriptorScriptPubKeyMan (Andrew Chow)
d50c8ddd4190f20bf0debd410348b73408ec3143 Implement GetSolvingProvider for DescriptorScriptPubKeyMan (Andrew Chow)
f1ca5feb4ad668a3e1ae543d0addd5f483f1a88f Implement GetKeypoolOldestTime and only display it if greater than 0 (Andrew Chow)
586b57a9a6b4b12a78f792785b63a5a1743bce0c Implement ReturnDestination in DescriptorScriptPubKeyMan (Andrew Chow)
f866957979c23cefd41efa9dae9e53b9177818dc Implement GetReservedDestination in DescriptorScriptPubKeyMan (Andrew Chow)
a775f7c7fd0b9094fcbeee6ba92206d5bbb19164 Implement Unlock and Encrypt in DescriptorScriptPubKeyMan (Andrew Chow)
bfdd0734869a22217c15858d7a76d0dacc2ebc86 Implement GetNewDestination for DescriptorScriptPubKeyMan (Andrew Chow)
58c7651821b0eeff0a99dc61d78d2e9e07986580 Implement TopUp in DescriptorScriptPubKeyMan (Andrew Chow)
e014886a342508f7c8d80323eee9a5f314eaf94c Implement SetupGeneration for DescriptorScriptPubKeyMan (Andrew Chow)
46dfb99768e7d03a3cf552812d5b41ceaebc06be Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file (Andrew Chow)
4cb9b69be031e1dc65d8964794781b347fd948f5 Implement several simple functions in DescriptorScriptPubKeyMan (Andrew Chow)
d1ec3e4f19487b4b100f80ad02eac063c571777d Add IsSingleType to Descriptors (Andrew Chow)
953feb3d2724f5398dd48990c4957a19313d2c8c Implement loading of keys for DescriptorScriptPubKeyMan (Andrew Chow)
2363e9fcaa41b68bf11153f591b95f2d41ff9a1a Load the descriptor cache from the wallet file (Andrew Chow)
46c46aebb7943e1e2e96755e94dc6c197920bf75 Implement GetID for DescriptorScriptPubKeyMan (Andrew Chow)
ec2f9e1178c8e38c0a5ca063fe81adac8f916348 Implement IsHDEnabled in DescriptorScriptPubKeyMan (Andrew Chow)
741122d4c1a62ced3e96d16d67f4eeb3a6522d99 Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan (Andrew Chow)
2db7ca765c8fb2c71dd6f7c4f29ad70e68ff1720 Implement IsMine for DescriptorScriptPubKeyMan (Andrew Chow)
db7177af8c159abbcc209f2caafcd45d54c181c5 Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet (Andrew Chow)
78f8a92910d34247fa5d04368338c598d9908267 Implement SetType in DescriptorScriptPubKeyMan (Andrew Chow)
834de0300cde57ca3f662fb7aa5b1bdaed68bc8f Store WalletDescriptor in DescriptorScriptPubKeyMan (Andrew Chow)
d8132669e10c1db9ae0c2ea0d3f822d7d2f01345 Add a lock cs_desc_man for DescriptorScriptPubKeyMan (Andrew Chow)
3194a7f88ac1a32997b390b4f188c4b6a4af04a5 Introduce WalletDescriptor class (Andrew Chow)
6b13cd3fa854dfaeb9e269bff3d67cacc0e5b5dc Create LegacyScriptPubKeyMan when not a descriptor wallet (Andrew Chow)
aeac157c9dc141546b45e06ba9c2e641ad86083f Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet (Andrew Chow)
96accc73f067c7c95946e9932645dd821ef67f63 Add WALLET_FLAG_DESCRIPTORS (Andrew Chow)
6b8119af53ee2fdb4c4b5b24b4e650c0dc3bd27c Introduce DescriptorScriptPubKeyMan as a dummy class (Andrew Chow)
06620302c713cae65ee8e4ff9302e4c88e2a1285 Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it (Andrew Chow)

Pull request description:

  Introducing the wallet of the glorious future (again): native descriptor wallets. With native descriptor wallets, addresses are generated from descriptors. Instead of generating keys and deriving addresses from keys, addresses come from the scriptPubKeys produced by a descriptor. Native descriptor wallets will be optional for now and can only be created by using `createwallet`.

  Descriptor wallets will store descriptors, master keys from the descriptor, and descriptor cache entries. Keys are derived from descriptors on the fly. In order to allow choosing different address types, 6 descriptors are needed for normal use. There is a pair of primary and change descriptors for each of the 3 address types. With the default keypool size of 1000, each descriptor has 1000 scriptPubKeys and descriptor cache entries pregenerated. This has a side effect of making wallets large since 6000 pubkeys are written to the wallet by default, instead of the current 2000. scriptPubKeys are kept only in memory and are generated every time a descriptor is loaded. By default, we use the standard BIP 44, 49, 84 derivation paths with an external and internal derivation chain for each.

  Descriptors can also be imported with a new `importdescriptors` RPC.

  Native descriptor wallets use the `ScriptPubKeyMan` interface introduced in #16341 to add a `DescriptorScriptPubKeyMan`. This defines a different IsMine which uses the simpler model of "does this scriptPubKey exist in this wallet". Furthermore, `DescriptorScriptPubKeyMan` does not have watchonly, so with native descriptor wallets, it is not possible to have a wallet with both watchonly and non-watchonly things. Rather a wallet with `disable_private_keys` needs to be used for watchonly things.

  A `--descriptor` option was added to some tests (`wallet_basic.py`, `wallet_encryption.py`, `wallet_keypool.py`, `wallet_keypool_topup.py`, and `wallet_labels.py`) to allow for these tests to use descriptor wallets. Additionally, several RPCs are disabled for descriptor wallets (`importprivkey`, `importpubkey`, `importaddress`, `importmulti`, `addmultisigaddress`, `dumpprivkey`, `dumpwallet`, `importwallet`, and `sethdseed`).

ACKs for top commit:
  Sjors:
    utACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82 (rebased, nits addressed)
  jonatack:
    Code review re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82.
  fjahr:
    re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
  instagibbs:
    light re-ACK 223588b
  meshcollider:
    Code review ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82

Tree-SHA512: 59bc52aeddbb769ed5f420d5d240d8137847ac821b588eb616b34461253510c1717d6a70bab8765631738747336ae06f45ba39603ccd17f483843e5ed9a90986

Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it

Introduce DescriptorScriptPubKeyMan as a dummy class

Add WALLET_FLAG_DESCRIPTORS

Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet

Create LegacyScriptPubKeyMan when not a descriptor wallet

Introduce WalletDescriptor class

WalletDescriptor is a Descriptor with other wallet metadata

Add a lock cs_desc_man for DescriptorScriptPubKeyMan

Store WalletDescriptor in DescriptorScriptPubKeyMan

Implement SetType in DescriptorScriptPubKeyMan

Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet

Implement IsMine for DescriptorScriptPubKeyMan

Adds a set of scriptPubKeys that DescriptorScriptPubKeyMan tracks.
If the given script is in that set, it is considered ISMINE_SPENDABLE

Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan

Implement IsHDEnabled in DescriptorScriptPubKeyMan

Implement GetID for DescriptorScriptPubKeyMan

Load the descriptor cache from the wallet file

Implement loading of keys for DescriptorScriptPubKeyMan

Add IsSingleType to Descriptors

IsSingleType will return whether the descriptor will give one or multiple scriptPubKeys

Implement several simple functions in DescriptorScriptPubKeyMan

Implements a bunch of one liners: UpgradeKeyMetadata, IsFirstRun, HavePrivateKeys,
KeypoolCountExternalKeys, GetKeypoolSize, GetTimeFirstKey, CanGetAddresses,
RewriteDB

Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file

Implement SetupGeneration for DescriptorScriptPubKeyMan

Implement TopUp in DescriptorScriptPubKeyMan

Implement GetNewDestination for DescriptorScriptPubKeyMan

Implement Unlock and Encrypt in DescriptorScriptPubKeyMan

Implement GetReservedDestination in DescriptorScriptPubKeyMan

Implement ReturnDestination in DescriptorScriptPubKeyMan

Implement GetKeypoolOldestTime and only display it if greater than 0

Implement GetSolvingProvider for DescriptorScriptPubKeyMan

Internally, a GetSigningProvider function is introduced which allows for
some private keys to be optionally included. This can be called with a
script as the argument (i.e. a scriptPubKey from our wallet when we are
signing) or with a pubkey. In order to know what index to expand the
private keys for that pubkey, we need to also cache all of the pubkeys
involved when we expand the descriptor. So SetCache and TopUp are
updated to do this too.

Implement SignTransaction in DescriptorScriptPubKeyMan

Implement SignMessage for descriptor wallets

Implement FillPSBT in DescriptorScriptPubKeyMan

FillPSBT will add our own scripts to the PSBT if those inputs are ours.
If an input also lists pubkeys that we happen to know the private keys
for, we will sign those inputs too.

Change GetMetadata to use unique_ptr<CKeyMetadata>

Implement GetMetadata in DescriptorScriptPubKeyMan

Be able to create new wallets with DescriptorScriptPubKeyMans as backing

Generate new descriptors when encrypting

Add IsLegacy to CWallet so that the GUI knows whether to show watchonly

add importdescriptors RPC and tests for native descriptor wallets

Co-authored-by: Andrew Chow <achow101-github@achow101.com>

Functional tests for descriptor wallets

Change wallet_encryption.py to use signmessage instead of dumpprivkey

Return error when no ScriptPubKeyMan is available for specified type

When a CWallet doesn't have a ScriptPubKeyMan for the requested type
in GetNewDestination, give a meaningful error. Also handle this in
Qt which did not do anything with errors.

Implement CWallet::IsSpentKey for non-LegacySPKMans

tests: Add RPCOverloadWrapper which overloads some disabled RPCs

RPCOverloadWrapper overloads some deprecated or disabled RPCs with
an implementation using other RPCs to avoid having a ton of code churn
around replacing those RPCs.

Add a --descriptors option to various tests

Adds a --descriptors option globally to the test framework. This will
make the test create and use descriptor wallets. However some tests may
not work with this.

Some tests are modified to work with --descriptors and run with that
option in test_runer:
* wallet_basic.py
* wallet_encryption.py
* wallet_keypool.py <---- wallet_keypool_hd.py actually
* wallet_keypool_topup.py
* wallet_labels.py
* wallet_avoidreuse.py
2024-03-07 01:23:15 +07:00
Konstantin Akimov
4064334732
fix: get receiving address for Descriptor Wallets
Expected to use wallet's interface instead LegacyScriptPubKeyMan to get a new address
2024-03-07 01:22:37 +07:00
UdjinM6
bdbd0b14a7
chore: dashification of descriptor implementation in dash
It removes mentioning of wpkh
2024-03-07 01:22:37 +07:00
Konstantin Akimov
b02fc0b2ce
fix: counting calculation of internal keys for Descriptor Wallets
KeypoolCountInternalKeys is not working as expected if there is
more than one ScriptPubKeyManager
2024-03-07 01:22:37 +07:00
Kittywhiskers Van Gogh
b60c493265
merge bitcoin#28100: more Span<std::byte> modernization & follow-ups 2024-03-05 21:43:22 +00:00
Kittywhiskers Van Gogh
7c5edf772a
merge bitcoin#28267: BIP324 ciphersuite follow-up 2024-03-05 21:43:22 +00:00
Kittywhiskers Van Gogh
1b1924e3d5
merge bitcoin#28008: BIP324 ciphersuite 2024-03-05 21:43:21 +00:00
Kittywhiskers Van Gogh
ff542199bc
merge bitcoin#27993: Make poly1305 support incremental computation + modernize 2024-03-05 21:42:49 +00:00
Kittywhiskers Van Gogh
d7482eb8a6
merge bitcoin#27985: Add support for RFC8439 variant of ChaCha20 2024-03-05 21:42:49 +00:00
Kittywhiskers Van Gogh
a1636d429a
refactor: subsume CGovernanceTriggerManager into CGovernanceManager
Having a dedicated manager to manipulate one variable, that relies
almost exclusively on another manager doesn't make much sense. Rather
than converting it into a unique_ptr, creating an alias and deglob'ing
it, it's much easier to subsume it into its dependent manager.
2024-03-05 13:15:59 -06:00
pasta
8459820478
Merge #5854: refactor: move HD code from CWallet to LegacyScriptPubKeyMan
b5f4411d11 fix: extra logs to distinct WriteHDChain for encrypted/raw batches (Konstantin Akimov)
e8f84afd3e refactor: move BIP39 related code to wallet (Konstantin Akimov)
fa6519f320 refactor: move hdchain to wallet/ because it belongs there (Konstantin Akimov)
392b51b197 refactor: obsolete hdCryptedChain from ScriptPubKeyMan (Konstantin Akimov)
a0389e181f refactor: rename EncryptHDChain to EncryptAndSetHDChain (Konstantin Akimov)
b2143f9346 refactor: re-order public and private methods in scriptpubkeyman (Konstantin Akimov)
a219748f5e refactor: make SetHDChain private in ScriptPubKeyManager (Konstantin Akimov)
f418b9ac62 refactor: move Encrypt chain call inside ScriptPubKeyMan::Encrypt (Konstantin Akimov)
c3754d5183 refactor: move DecryptHDChain from public to private (Konstantin Akimov)
e08b64a1bb refactor: unify SetHDChainSingle and SetCryptedHDChainSingle (Konstantin Akimov)
59a998843b refactor: unify ScriptPubKeyMan's SetHDChain nad SetHDCryptedChain (Konstantin Akimov)
a180d73e5c refactor: unify WalletBatch's WriteHDChain and WriteCryptedHDChain (Konstantin Akimov)
9c8e4584ae refactor: unify SetHDChain and SetCryptedHDChain helpers (Konstantin Akimov)
7b72726b3e refactor: SetCryptedHDChain moved to scriptpubkeyman (Konstantin Akimov)
6993d112f3 feat: extra check failure case CWallet::GenerateNewHDChainEncrypted (Konstantin Akimov)

Pull request description:

  ## Issue being fixed or feature implemented
  During backport bitcoin#17261 significant part of HD chain has been forgotten in CWallet due to our own implementation.

  This PR do not change behaviour of HD wallets, it's just refactoring.

  ## What was done?
  This PR refactor HD wallets implementation:
   - key related stuff is moved from CWallet to LegacyScriptPubKeyMan (to follow-up bitcoin#17261)
   - refactored duplicated code between hdChain and hdCryptedChain
   - modules hdchain and bip39 related moved to wallet/ module

  ## How Has This Been Tested?
  Run unit/functional tests

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

Top commit has no ACKs.

Tree-SHA512: 763ea6dd9af902dfc4dd99896c1cb16a75e3c0322e806e37ff05bc9229923057c544732ebf9fca705da870568643370867d3f81a46cab1e9d229b2949016c484
2024-03-05 13:10:56 -06:00
Konstantin Akimov
b5f4411d11
fix: extra logs to distinct WriteHDChain for encrypted/raw batches 2024-03-06 02:04:25 +07:00
Konstantin Akimov
e8f84afd3e
refactor: move BIP39 related code to wallet 2024-03-06 02:04:25 +07:00
Konstantin Akimov
fa6519f320
refactor: move hdchain to wallet/ because it belongs there 2024-03-06 02:04:25 +07:00
Konstantin Akimov
392b51b197
refactor: obsolete hdCryptedChain from ScriptPubKeyMan 2024-03-06 02:04:24 +07:00
Konstantin Akimov
a0389e181f
refactor: rename EncryptHDChain to EncryptAndSetHDChain 2024-03-06 02:04:24 +07:00
Konstantin Akimov
b2143f9346
refactor: re-order public and private methods in scriptpubkeyman 2024-03-06 02:04:24 +07:00
Konstantin Akimov
a219748f5e
refactor: make SetHDChain private in ScriptPubKeyManager 2024-03-06 02:04:23 +07:00
Konstantin Akimov
f418b9ac62
refactor: move Encrypt chain call inside ScriptPubKeyMan::Encrypt 2024-03-06 02:04:23 +07:00
Konstantin Akimov
c3754d5183
refactor: move DecryptHDChain from public to private 2024-03-06 02:04:23 +07:00
Konstantin Akimov
e08b64a1bb
refactor: unify SetHDChainSingle and SetCryptedHDChainSingle 2024-03-06 02:04:22 +07:00
Konstantin Akimov
59a998843b
refactor: unify ScriptPubKeyMan's SetHDChain nad SetHDCryptedChain 2024-03-06 02:04:22 +07:00
Konstantin Akimov
a180d73e5c
refactor: unify WalletBatch's WriteHDChain and WriteCryptedHDChain 2024-03-06 02:04:21 +07:00
Konstantin Akimov
9c8e4584ae
refactor: unify SetHDChain and SetCryptedHDChain helpers 2024-03-06 02:04:21 +07:00
Konstantin Akimov
7b72726b3e
refactor: SetCryptedHDChain moved to scriptpubkeyman 2024-03-06 02:04:21 +07:00
Konstantin Akimov
6993d112f3
feat: extra check failure case CWallet::GenerateNewHDChainEncrypted
To unify code with GenerateNewHDChain
2024-03-06 02:04:20 +07:00
MarcoFalke
4e46e6101c
Merge #19478: Remove CTxMempool::mapLinks data structure member
296be8f58e02b39a58f017c52294aceed22c3ffd Get rid of unused functions CTxMemPool::GetMemPoolChildren, CTxMemPool::GetMemPoolParents (Jeremy Rubin)
46d955d196043cc297834baeebce31ff778dff80 Remove mapLinks in favor of entry inlined structs with iterator type erasure (Jeremy Rubin)

Pull request description:

  Currently we have a peculiar data structure in the mempool called maplinks. Maplinks job is to track the in-pool children and parents of each transaction. This PR can be primarily understood and reviewed as a simple refactoring to remove this extra data structure, although it comes with a nice memory and performance improvement for free.

  Maplinks is particularly peculiar because removing it is not as simple as just moving it's inner structure to the owning CTxMempoolEntry. Because TxLinks (the class storing the setEntries for parents and children) store txiters to each entry in the mempool corresponding to the parent or child, it means that the TxLinks type is "aware" of the boost multiindex (mapTx) it's coming from, which is in turn, aware of the entry type stored in mapTx. Thus we used maplinks to store this entry associated data we in an entirely separate data structure just to avoid a circular type reference caused by storing a txiter inside a CTxMempoolEntry.

  It turns out, we can kill this circular reference by making use of iterator_to multiindex function and std::reference_wrapper. This allows us to get rid of the maplinks data structure and move the ownership of the parents/child sets to the entries themselves.

  The benefit of this good all around, for any of the reasons given below the change would be acceptable, and it doesn't make the code harder to reason about or worse in any respect (as far as I can tell, there's no tradeoff).

  ### Simpler ownership model
  No longer having to consistency check that mapLinks did have records for our CTxMempoolEntry, impossible to have a mapLinks entry outlive or incorrectly die before a CTxMempoolEntry.

  ### Memory Usage
  We get rid of a O(Transactions) sized map in the mempool, which is a long lived data structure.

  ### Performance
  If you have a CTxMemPoolEntry, you immediately know the address of it's children/parents, rather than having to do a O(log(Transactions)) lookup via maplinks (which we do very often). We do it in *so many* places that a true benchmark has to look at a full running node, but it is easy enough to show an improvement in this case.

  The ComplexMemPool shows a good coherence check that we see the expected result of it being 12.5% faster / 1.14x faster.
  ```
  Before:
  # Benchmark, evals, iterations, total, min, max, median
  ComplexMemPool, 5, 1, 1.40462, 0.277222, 0.285339, 0.279793

  After:
  # Benchmark, evals, iterations, total, min, max, median
  ComplexMemPool, 5, 1, 1.22586, 0.243831, 0.247076, 0.244596
  ```
  The ComplexMemPool benchmark only checks doing addUnchecked and TrimToSize for 800 transactions. While this bench does a good job of hammering the relevant types of function, it doesn't test everything.

  Subbing in 5000 transactions shows a that the advantage isn't completely wiped out by other asymptotic factors (this isn't the only bottleneck in growing the mempool), but it's only a bit proportionally slower (10.8%, 1.12x), which adds evidence that this will be a good change for performance minded users.

  ```
  # Benchmark, evals, iterations, total, min, max, median
  ComplexMemPool, 5, 1, 59.1321, 11.5919, 12.235, 11.7068

  # Benchmark, evals, iterations, total, min, max, median
  ComplexMemPool, 5, 1, 52.1307, 10.2641, 10.5206, 10.4306
  ```
  I don't think it's possible to come up with an example of where a maplinks based design would have better performance, but it's something for reviewers to consider.

  # Discussion
  ## Why maplinks in the first place?

  I spoke with the author of mapLinks (sdaftuar) a while back, and my recollection from our conversation was that it was implemented because he did not know how to resolve the circular dependency at the time, and there was no other reason for making it a separate map.

  ## Is iterator_to weird?

  iterator_to is expressly for this purpose, see https://www.boost.org/doc/libs/1_51_0/libs/multi_index/doc/tutorial/indices.html#iterator_to

  >  iterator_to provides a way to retrieve an iterator to an element from a pointer to the element, thus making iterators and pointers interchangeable for the purposes of element pointing (not so for traversal) in many situations. This notwithstanding, it is not the aim of iterator_to to promote the usage of pointers as substitutes for real iterators: the latter are specifically designed for handling the elements of a container, and not only benefit from the iterator orientation of container interfaces, but are also capable of exposing many more programming bugs than raw pointers, both at compile and run time. iterator_to is thus meant to be used in scenarios where access via iterators is not suitable or desireable:
  >
  >     - Interoperability with preexisting APIs based on pointers or references.
  >     - Publication of pointer-based interfaces (for instance, when designing a C-compatible library).
  >     - The exposure of pointers in place of iterators can act as a type erasure barrier effectively decoupling the user of the code from the implementation detail of which particular container is being used. Similar techniques, like the famous Pimpl idiom, are used in large projects to reduce dependencies and build times.
  >     - Self-referencing contexts where an element acts upon its owner container and no iterator to itself is available.

  In other words, iterator_to is the perfect tool for the job by the last reason given. Under the hood it should just be a simple pointer cast and have no major runtime overhead (depending on if the function call is inlined).

  Edit by laanwj: removed at sign from the description

ACKs for top commit:
  jonatack:
    re-ACK 296be8f per `git range-diff ab338a19 3ba1665 296be8f`, sanity check gcc 10.2 debug build is clean.
  hebasto:
    re-ACK 296be8f58e02b39a58f017c52294aceed22c3ffd, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/19478#pullrequestreview-482400727) review (verified with `git range-diff`).

Tree-SHA512: f5c30a4936fcde6ae32a02823c303b3568a747c2681d11f87df88a149f984a6d3b4c81f391859afbeb68864ef7f6a3d8779f74a58e3de701b3d51f78e498682e
2024-03-06 02:00:40 +07:00
Wladimir J. van der Laan
be5662a785
Merge #19596: Deduplicate parent txid loop of requested transactions and missing parents of orphan transactions
4c0731f9c50b0556f8a57b912c8f295c7a9ea89c Deduplicate missing parents of orphan transactions (Suhas Daftuar)
81961762439fb72fc2ef168164689ddc29d7ef94 Rewrite parent txid loop of requested transactions (Suhas Daftuar)

Pull request description:

  I noticed a couple of places recently where we loop over all inputs of a transaction in order to do some processing on the txids we find in those inputs.  There may be thousands of inputs in a transaction, and the same txid may appear many times.  In a couple of places in particular, we loop over those txids and add them to a rolling bloom filter; doing that multiple times for the same txid wastes entries in that filter.

  This PR fixes that in two places relating to transaction relay: one on the server side, where we look for parent transactions of a tx that we are delivering to a peer to ensure that getdata requests for those parents will succeed; and the other on the client side, where when we process an orphan tx we want to loop over the parent txids and ensure that all are eventually requested from the peer who provided the orphan.

  This addresses a couple of [related](https://github.com/bitcoin/bitcoin/pull/19109#discussion_r455197217) [comments](https://github.com/bitcoin/bitcoin/pull/19109#discussion_r456820373) left in #19109.

ACKs for top commit:
  laanwj:
    Code review ACK 4c0731f9c50b0556f8a57b912c8f295c7a9ea89c
  jonatack:
    ACK 4c0731f9c50b0556f8a57b912c8f295c7a9ea89c
  ajtowns:
    ACK 4c0731f9c50b0556f8a57b912c8f295c7a9ea89c

Tree-SHA512: 8af9df7f56c6e54b5915519d7d5465e081473ceb1bcc89bbebf83e78722cf51ff58145e588cf57126bce17071a8053273f4bcef0ad8166bec83ba14352e40f5d
2024-03-06 02:00:40 +07:00
MarcoFalke
9c3ecd167e
Merge #19473: net: Add -networkactive option
2aac093a3d60e446b85eebdf170ea6bed77bec92 test: Add test coverage for -networkactive option (Hennadii Stepanov)
3c58129b1293742a49aa196cb210ff345a7339e6 net: Log network activity status change unconditionally (Hennadii Stepanov)
62fe6aa87e4cdd8b06207abc1387c68d7bfc04c1 net: Add -networkactive option (Hennadii Stepanov)

Pull request description:

  Some Bitcoin Core activity is completely local (offline), e.g., reindexing.

  The `setnetworkactive` RPC command is already present. This PR adds the corresponding command-line argument / config option, and allows to start the client with disabled p2p network by providing `-networkactive=0` or `-nonetworkactive`.

  This was done while reviewing #16981.

ACKs for top commit:
  MarcoFalke:
    re-ACK 2aac093a3d60e446b85eebdf170ea6bed77bec92 🏠
  LarryRuane:
    ACK 2aac093a3d60e446b85eebdf170ea6bed77bec92

Tree-SHA512: 446d791b46d7b556d7694df7b1f88cd4fbc09301fe4eaf036b45cb8166ed806156353cc03788a07b633d5887d5eee30a7c02a2d4307141c8ccc75e0a88145636
2024-03-06 02:00:39 +07:00
MarcoFalke
941b39d9a9
Merge #18937: refactor: s/command/msg_type/ in CNetMsgMaker and CSerializedNetMsg
51e9393c1f6c9eaac554f821f5327f63bd09c8cf refactor: s/command/msg_type/ in CNetMsgMaker and CSerializedNetMsg (Sebastian Falbesoner)

Pull request description:

  Follow-up PR for #18533 -- another small step towards getting rid of the confusing "command" terminology. Also see PR #18610 which tackled the functional tests.

ACKs for top commit:
  MarcoFalke:
    ACK 51e9393c1f6c9eaac554f821f5327f63bd09c8cf

Tree-SHA512: bb6f05a7be6823d5c4eab1d05b31fee944e700946827ad9425d59a3957fd879776c88c606319cbe9832d9451b275baedf913b71429ea3e01e4e82bf2d419e819
2024-03-06 02:00:38 +07:00
MarcoFalke
ad3f424b4d
partial Merge #18638: net: Use mockable time for ping/pong, add tests
fa3365430c5fb57d7c0b5f2bce9fbbe290be93c3 net: Use mockable time for ping/pong, add tests (MarcoFalke)
faab4aaf2fa1153c6d76efc8113fa01b06943ece util: Add count_microseconds helper (MarcoFalke)

Pull request description:

  Switch `CNode::m_ping_start` and `CNetMessage::m_time` to mockable time, so that tests can be added.

  Mockable time is also type-safe, since it uses `std::chrono`

ACKs for top commit:
  jonatack:
    Code review re-ACK fa33654 re-read code, verified rebase per `git range-diff 4b5c919 fa94d6f fa33654`, previous tested ACKs still valid
  troygiorshev:
    ACK fa3365430c5fb57d7c0b5f2bce9fbbe290be93c3

Tree-SHA512: 7d632bd6019ce7c882029e71b667a61517e783af82755a85dd979ef09380934e172dec8b8f91d57b200a30a6e096aeaf01f19fee7f3aed0e0e871c72eb44d70e
2024-03-06 02:00:30 +07:00
Hennadii Stepanov
2018b459cc
Merge bitcoin-core/gui#683: doc: Drop no longer relevant comment
5d332da2cfdc67d69a165119c6ff598e29b28afa doc: Drop no longer relevant comment (Hennadii Stepanov)

Pull request description:

  The comment was introduced in 4cf3411056, and since 7e4bd19785ff9120b7242ff9f15231868aaf7db4 it has been no longer relevant.

ACKs for top commit:
  jarolrod:
    ACK 5d332da2cfdc67d69a165119c6ff598e29b28afa

Tree-SHA512: 6d32561336993b1ff7d7c524d090ac52aefb40078ed706ca4c6d5026cc3f63244c49c0e00e45ff192ba0e9f1527faf63249aa18bc8aa677b9e053d387e0f4027
2024-03-05 10:40:38 -06:00
fanquake
5d666eba1b
Merge bitcoin/bitcoin#25548: gui: Check for readlink buffer overflow and handle gracefully
e049fd76f0d57c1e6400fbfbaf4cc6ebe540f16f Bugfix: Check for readlink buffer overflow and handle gracefully (Luke Dashjr)

Pull request description:

  If readlink returns the size of the buffer, an overflow may have (safely) occurred.
  Pass a buffer size of MAX_PATH+1 (the size of the actual buffer) to detect this scenario.

ACKs for top commit:
  hebasto:
    ACK e049fd76f0d57c1e6400fbfbaf4cc6ebe540f16f.

Tree-SHA512: 188bace79cbe556efe7782e46b870c02729b07b104a9316b0f7d50013504972e85baf507403d2d6060bb2bf3e13f40d735bddd18255d97a60810208c3de87691
2024-03-05 10:40:38 -06:00
MacroFake
76b31fcf60
Merge bitcoin/bitcoin#26377: test: Make system_tests/run_command test locale and platform agnostic
884304e6c6a250719a64752fe52d1283cd5e36ab test: Make `system_tests/run_command` locale agnostic (Hennadii Stepanov)

Pull request description:

  Fixes bitcoin/bitcoin#26368.

ACKs for top commit:
  Sjors:
    tACK 884304e6c6a250719a64752fe52d1283cd5e36ab

Tree-SHA512: 76d4941e02b3b119dcf4dacbe60ef45a9dc8cf775bdb31b5291cd8147665285d41caaf1f5688abdfc9a47c393ddb535af7b11af839660d30ef30f1ca0d936133
2024-03-05 10:40:38 -06:00
glozow
462498a365
Merge bitcoin/bitcoin#26394: Fix typo in comment SHA256->SHA512
0cc23fc60374512b3c4be888d98a7dbdd3a0c931 Fix typo in comment SHA256->SHA512 (Elichai Turkel)

Pull request description:

  The comment says it's the SHA-256 state, while it's actually the SHA-512 state

ACKs for top commit:
  andrewtoth:
    ACK 0cc23fc60374512b3c4be888d98a7dbdd3a0c931
  aureleoules:
    ACK 0cc23fc60374512b3c4be888d98a7dbdd3a0c931

Tree-SHA512: 4e390ceefb847d3bbe4f5caab390a4fdd14892fe443f58c32b08b3444fccd611cff22938c3dfa611dfd2497736f779fae4165497b4208e48aa8fc9d2236f943b
2024-03-05 10:40:37 -06:00
fanquake
c70aa4218e
Merge bitcoin/bitcoin#26188: test: silence TSAN false positive in coinstatsindex_initial_sync
861cb3fadce88cfaee27088185a48f03fb9dafe7 test: move SyncWithValidationInterfaceQueue() before Stop() in txindex_tests (Vasil Dimov)
6526dc3b78d9ca2b5c67564b04dcacbc75b857e1 test: silence TSAN false positive in coinstatsindex_initial_sync (Vasil Dimov)

Pull request description:

  Silence false positives from TSAN about unsynchronized calls to `BaseIndex::~BaseIndex()` and `BaseIndex::SetBestBlockIndex()`. They are synchronized, but beyond the comprehension of TSAN - by `SyncWithValidationInterfaceQueue()`, called from `BaseIndex::BlockUntilSyncedToCurrentChain()`.

  Fixes https://github.com/bitcoin/bitcoin/issues/25365

ACKs for top commit:
  MarcoFalke:
    review ACK 861cb3fadce88cfaee27088185a48f03fb9dafe7
  ryanofsky:
    Code review ACK 861cb3fadce88cfaee27088185a48f03fb9dafe7. Just comment change since last review.

Tree-SHA512: 8c30fdf2fd11d54e9adfa68a67185ab820bd7bd9f7f3ad6456e7e6d219fa9cf6d34b41e98e723eae86cb0c1baef7f3fc57b1b011a13dc3fe3d78334b9b5596de
2024-03-05 10:40:37 -06:00
MacroFake
ee44732376
Merge bitcoin/bitcoin#26284: Fix comment typos
adb1714426365fb72f73097610ba217ac94ea560 Fix comment typos in scriptpubkeyman.cpp, wallet.cpp, wallet.h (Dimitris Tsapakidis)

Pull request description:

  Fixes a number of comment typos found in the code.

Top commit has no ACKs.

Tree-SHA512: c2c996b66d33ecf0ee734b76303a0f2444e184d2f3ff6931768712ca51011ad51e54336c33a2ff55133766d20ae6adcbb14ddc754dde58b1fe9167d68f54fec5
2024-03-05 10:40:36 -06:00
fanquake
f9b526bd31
Merge bitcoin/bitcoin#26209: Update leveldb subtree
1a463c70a368fb20316e03efac273438cf47baa3 Squashed 'src/leveldb/' changes from 22f1e4a02f..e2f10b4e47 (fanquake)

Pull request description:

  Pulls in https://github.com/bitcoin-core/leveldb-subtree/pull/34: win32: fix -Wmissing-field-initializers warnings
  Related to #26090, #25972.

  Guix Build:
  ```bash
  a8c49e700309e9268ad0a503e7813775f58daf37813501775ae5831eab0cf26a  guix-build-13601da17e7a/output/aarch64-linux-gnu/SHA256SUMS.part
  3a1722403f8daf8ea152c72004fb1021f8a7241ee7f51fb0e6adeeb50881f001  guix-build-13601da17e7a/output/aarch64-linux-gnu/bitcoin-13601da17e7a-aarch64-linux-gnu-debug.tar.gz
  3b7c1f2361ef4832c305ff4b0d9eb7419f787f8cf537952d7c20be549debe0af  guix-build-13601da17e7a/output/aarch64-linux-gnu/bitcoin-13601da17e7a-aarch64-linux-gnu.tar.gz
  329cb5bc932621785a6aeca43af307279c47d87a4983ee3d59636f8e3dc27afa  guix-build-13601da17e7a/output/arm-linux-gnueabihf/SHA256SUMS.part
  4030f344c4c31cc0877c11d3b89f6ea93f930a4ae7e7976d49ad5123cc31a367  guix-build-13601da17e7a/output/arm-linux-gnueabihf/bitcoin-13601da17e7a-arm-linux-gnueabihf-debug.tar.gz
  ab50945c416b256b4c92ea0a50c0b7ec241e4c1f5803a4e7cb4ba83a6c5db888  guix-build-13601da17e7a/output/arm-linux-gnueabihf/bitcoin-13601da17e7a-arm-linux-gnueabihf.tar.gz
  604557cc73c4065697255c6971547a46cd0d6ad1cdf74261d4c73d002abe8c46  guix-build-13601da17e7a/output/arm64-apple-darwin/SHA256SUMS.part
  11e781c64fab7933df4a9b4f56357ad18dd8f149b3c9195bf2d310b58355851d  guix-build-13601da17e7a/output/arm64-apple-darwin/bitcoin-13601da17e7a-arm64-apple-darwin-unsigned.dmg
  e3fe9e278310f3dc97f463bb8cf2b930b587dba5095e18a7f38c4ef76338daad  guix-build-13601da17e7a/output/arm64-apple-darwin/bitcoin-13601da17e7a-arm64-apple-darwin-unsigned.tar.gz
  00828a9bfd74013a439b7f0254bd72904b99ff1e3a9503ef6307ee57de81c5eb  guix-build-13601da17e7a/output/arm64-apple-darwin/bitcoin-13601da17e7a-arm64-apple-darwin.tar.gz
  d80032d2509a982dc81790b3ef3cd8d44551b41047082ebc506f01bbb8830de1  guix-build-13601da17e7a/output/dist-archive/bitcoin-13601da17e7a.tar.gz
  61655cdf7c081c43275da57e5b42c21e33d7864378f3784652969ffdf7c3c2b2  guix-build-13601da17e7a/output/powerpc64-linux-gnu/SHA256SUMS.part
  3d213e2f25ca72dc0c3d372c0ad3f6803865825a233b29dd80b5cff3e3963621  guix-build-13601da17e7a/output/powerpc64-linux-gnu/bitcoin-13601da17e7a-powerpc64-linux-gnu-debug.tar.gz
  ef8cad747428b6b46c5d2c598fb7896b320a5321cf9a0c9dba2209c638bfbc7b  guix-build-13601da17e7a/output/powerpc64-linux-gnu/bitcoin-13601da17e7a-powerpc64-linux-gnu.tar.gz
  2302e022394ca46570c70bc5977f35bbed258bfbb19c2c6d80f1982a725c9e35  guix-build-13601da17e7a/output/powerpc64le-linux-gnu/SHA256SUMS.part
  5ca3104b7d546c7d905f82dfd92b5350af754e3973b533adc50c6cbde8f59f96  guix-build-13601da17e7a/output/powerpc64le-linux-gnu/bitcoin-13601da17e7a-powerpc64le-linux-gnu-debug.tar.gz
  2781824d8d08f8ac6663925db8b2d1b50b80165e67ed69df86580438284f0c3b  guix-build-13601da17e7a/output/powerpc64le-linux-gnu/bitcoin-13601da17e7a-powerpc64le-linux-gnu.tar.gz
  24a73f91c5ab9e72f2471cd7b31482c702a3de8dade9aff6b76af34253748cec  guix-build-13601da17e7a/output/riscv64-linux-gnu/SHA256SUMS.part
  b09658af4bd24eb5e76bacbfc7f8750a8941f117511b5c55a44ef1beffec8367  guix-build-13601da17e7a/output/riscv64-linux-gnu/bitcoin-13601da17e7a-riscv64-linux-gnu-debug.tar.gz
  56bde3f3df115dbf801944e4f923a08752fd05cda0bbf0e0e3c915387f98e4a8  guix-build-13601da17e7a/output/riscv64-linux-gnu/bitcoin-13601da17e7a-riscv64-linux-gnu.tar.gz
  ca9a5acb332fb2ffda42e44d84b58e273a769e94f0abaaa0623ed1d1c5267dde  guix-build-13601da17e7a/output/x86_64-apple-darwin/SHA256SUMS.part
  ccae805bca5d4075d1bccbaeca7ac40bb4dac0a97f4fd4c3c80b2d6134d6acb0  guix-build-13601da17e7a/output/x86_64-apple-darwin/bitcoin-13601da17e7a-x86_64-apple-darwin-unsigned.dmg
  b12c9682b69b3662150abd6680bb60f9fef656a1fdc3cd21951c2da571be40cc  guix-build-13601da17e7a/output/x86_64-apple-darwin/bitcoin-13601da17e7a-x86_64-apple-darwin-unsigned.tar.gz
  fd3378e4d60f0ef3ae1b18f526e96ed9e838df4def021de332dd0b6cd20ed0c4  guix-build-13601da17e7a/output/x86_64-apple-darwin/bitcoin-13601da17e7a-x86_64-apple-darwin.tar.gz
  47c7af42153850a94d70de6489389f6faf8772d6adc88269705a12e6b894e84e  guix-build-13601da17e7a/output/x86_64-linux-gnu/SHA256SUMS.part
  876898d116041680ea6fa3bbce1becb05e45ea5bdbaf973a1465fde4bd2df26d  guix-build-13601da17e7a/output/x86_64-linux-gnu/bitcoin-13601da17e7a-x86_64-linux-gnu-debug.tar.gz
  56671e22f25ba695c7315de916c6a2d1b57d5f00982b331c68a43912ac3ebc44  guix-build-13601da17e7a/output/x86_64-linux-gnu/bitcoin-13601da17e7a-x86_64-linux-gnu.tar.gz
  f5b9b7d83ad34b7b6c3cb98b9d7878795a1f6fb7b422ffb02b23266beb76d3a6  guix-build-13601da17e7a/output/x86_64-w64-mingw32/SHA256SUMS.part
  a72cfb4ce41cad3bb24df2e4bb318edd05370e12fc31f9ba21637b3fd412dccd  guix-build-13601da17e7a/output/x86_64-w64-mingw32/bitcoin-13601da17e7a-win64-debug.zip
  4b175eac7355715884be279dd4595665912b62815f1d0186ca9874cb9e6424cb  guix-build-13601da17e7a/output/x86_64-w64-mingw32/bitcoin-13601da17e7a-win64-setup-unsigned.exe
  c45d927f0adedc3eb0a01cee0f9fa734c17d9e9a6692b0f416a3830c007b52cc  guix-build-13601da17e7a/output/x86_64-w64-mingw32/bitcoin-13601da17e7a-win64-unsigned.tar.gz
  360a5b8806c72dbfcfed616d5cec78606461d664e21fb28a44bd473a2b7c9326  guix-build-13601da17e7a/output/x86_64-w64-mingw32/bitcoin-13601da17e7a-win64.zip
  ```

ACKs for top commit:
  hebasto:
    ACK 13601da17e7af08133861a86c295356c173b5bf5, I have reviewed the code and it looks OK, I agree it can be merged.
  theuni:
    ACK 13601da17e7af08133861a86c295356c173b5bf5

Tree-SHA512: 1450618714a456d8969fa5bfc3ed700452e0737213af50656a0a1e0764e6063390fb528eb1889d8bf1e02e451f601f0a5bc63a02ac34ef10aeb7dc80fe41acd1
2024-03-05 10:40:36 -06:00
PastaPastaPasta
a1ff8d3e54
refactor: add const
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2024-03-04 20:34:04 -06:00
PastaPastaPasta
521fefb98a
refactor: simplify if block
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2024-03-04 20:34:04 -06:00
pasta
66351aaaf9
refactor: use try_emplace over emplace 2024-03-04 20:34:04 -06:00
pasta
b6907d2bea
refactor: additional if-init 2024-03-04 20:34:03 -06:00
pasta
c1ee599c74
refactor: remove unneeded potentially problematic std::move
std::move implies the moved from variable is now clobbered and unusable; but we continue to use the `pendingRetryTxs` structure; simply remove std::move; this likely doesn't change code generation, but is clearer that no move is happening and resolves a use-after-move warning

calling std::move on a const ref does absolutely nothing; remove to avoid confusion
2024-03-04 20:34:03 -06:00
pasta
f77a1f7bce
refactor: use if-init; ensure we std::move; use try_emplace 2024-03-04 20:34:03 -06:00
pasta
37080c880e
refactor: remove a temporary; this should enable better optimization as the type is && insteas of & now 2024-03-04 20:34:00 -06:00
pasta
a701a8d5c9
refactor: actually move out of previousQuarters.quarterHMinus*C and newQuarterMembers into quorumMembers 2024-03-04 09:02:03 -06:00
UdjinM6
65f55b8aa5
99%+: it 2024-03-04 08:54:52 -06:00
UdjinM6
9b441132fc
80%+: bg 2024-03-04 08:54:52 -06:00
UdjinM6
19ecd79c2e
88%+: ar, de, es, fi, fr, ja, ko, nl, pt, sk, tr, zh_TW 2024-03-04 08:54:52 -06:00
UdjinM6
d541448eee
90%+: it, pl, th, zh_CN 2024-03-04 08:54:51 -06:00