2016-05-06 11:23:48 +02:00
#!/usr/bin/env python3
2020-12-31 18:50:11 +01:00
# Copyright (c) 2014-2020 The Bitcoin Core developers
2014-12-13 05:09:33 +01:00
# Distributed under the MIT software license, see the accompanying
2014-11-19 21:55:40 +01:00
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
2019-01-07 10:55:35 +01:00
""" Test the wallet. """
2021-04-12 16:58:49 +02:00
from decimal import Decimal
2019-08-09 01:21:11 +02:00
2021-05-31 11:26:16 +02:00
from test_framework . blocktools import COINBASE_MATURITY
2015-05-02 12:53:35 +02:00
from test_framework . test_framework import BitcoinTestFramework
2021-04-12 16:58:49 +02:00
from test_framework . util import (
assert_array_result ,
assert_equal ,
assert_fee_amount ,
assert_raises_rpc_error ,
count_bytes ,
)
2020-01-07 23:23:58 +01:00
from test_framework . wallet_util import test_address
2014-11-19 21:55:40 +01:00
2019-02-01 15:35:08 +01:00
2017-04-29 14:47:00 +02:00
class WalletTest ( BitcoinTestFramework ) :
2017-09-01 18:47:13 +02:00
def set_test_params ( self ) :
2016-05-20 15:16:51 +02:00
self . num_nodes = 4
Merge #16528: Native Descriptor Wallets using DescriptorScriptPubKeyMan
223588b1bbc63dc57098bbd0baa48635e0cc0b82 Add a --descriptors option to various tests (Andrew Chow)
869f7ab30aeb4d7fbd563c535b55467a8a0430cf tests: Add RPCOverloadWrapper which overloads some disabled RPCs (Andrew Chow)
cf060628590fab87d73f278e744d70ef2d5d81db Correctly check for default wallet (Andrew Chow)
886e0d75f5fea2421190aa4812777d89f68962cc Implement CWallet::IsSpentKey for non-LegacySPKMans (Andrew Chow)
3c19fdd2a2fd5394fcfa75b2ba84ab2277cbdabf Return error when no ScriptPubKeyMan is available for specified type (Andrew Chow)
388ba94231f2f10a0be751c562cdd4650510a90a Change wallet_encryption.py to use signmessage instead of dumpprivkey (Andrew Chow)
1346e14831489f9c8f53a08f9dfed61d55d53c6f Functional tests for descriptor wallets (Andrew Chow)
f193ea889ddb53d9a5c47647966681d525e38368 add importdescriptors RPC and tests for native descriptor wallets (Hugo Nguyen)
ce24a944940019185efebcc5d85eac458ed26016 Add IsLegacy to CWallet so that the GUI knows whether to show watchonly (Andrew Chow)
1cb42b22b11c27e64462afc25a94b2fc50bfa113 Generate new descriptors when encrypting (Andrew Chow)
82ae02b1656819f4bd5023b8955447e1d4ea8692 Be able to create new wallets with DescriptorScriptPubKeyMans as backing (Andrew Chow)
b713baa75a62335ab9c0eed9ef76a95bfec30668 Implement GetMetadata in DescriptorScriptPubKeyMan (Andrew Chow)
8b9603bd0b443e2f7984eb72bf2e21cf02af0bcb Change GetMetadata to use unique_ptr<CKeyMetadata> (Andrew Chow)
72a9540df96ffdb94f039b9c14eaacdc7d961196 Implement FillPSBT in DescriptorScriptPubKeyMan (Andrew Chow)
84b4978c02102171775c77a45f6ec198930f0a88 Implement SignMessage for descriptor wallets (Andrew Chow)
bde7c9fa38775a81d53ac0484fa9c98076a0c7d1 Implement SignTransaction in DescriptorScriptPubKeyMan (Andrew Chow)
d50c8ddd4190f20bf0debd410348b73408ec3143 Implement GetSolvingProvider for DescriptorScriptPubKeyMan (Andrew Chow)
f1ca5feb4ad668a3e1ae543d0addd5f483f1a88f Implement GetKeypoolOldestTime and only display it if greater than 0 (Andrew Chow)
586b57a9a6b4b12a78f792785b63a5a1743bce0c Implement ReturnDestination in DescriptorScriptPubKeyMan (Andrew Chow)
f866957979c23cefd41efa9dae9e53b9177818dc Implement GetReservedDestination in DescriptorScriptPubKeyMan (Andrew Chow)
a775f7c7fd0b9094fcbeee6ba92206d5bbb19164 Implement Unlock and Encrypt in DescriptorScriptPubKeyMan (Andrew Chow)
bfdd0734869a22217c15858d7a76d0dacc2ebc86 Implement GetNewDestination for DescriptorScriptPubKeyMan (Andrew Chow)
58c7651821b0eeff0a99dc61d78d2e9e07986580 Implement TopUp in DescriptorScriptPubKeyMan (Andrew Chow)
e014886a342508f7c8d80323eee9a5f314eaf94c Implement SetupGeneration for DescriptorScriptPubKeyMan (Andrew Chow)
46dfb99768e7d03a3cf552812d5b41ceaebc06be Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file (Andrew Chow)
4cb9b69be031e1dc65d8964794781b347fd948f5 Implement several simple functions in DescriptorScriptPubKeyMan (Andrew Chow)
d1ec3e4f19487b4b100f80ad02eac063c571777d Add IsSingleType to Descriptors (Andrew Chow)
953feb3d2724f5398dd48990c4957a19313d2c8c Implement loading of keys for DescriptorScriptPubKeyMan (Andrew Chow)
2363e9fcaa41b68bf11153f591b95f2d41ff9a1a Load the descriptor cache from the wallet file (Andrew Chow)
46c46aebb7943e1e2e96755e94dc6c197920bf75 Implement GetID for DescriptorScriptPubKeyMan (Andrew Chow)
ec2f9e1178c8e38c0a5ca063fe81adac8f916348 Implement IsHDEnabled in DescriptorScriptPubKeyMan (Andrew Chow)
741122d4c1a62ced3e96d16d67f4eeb3a6522d99 Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan (Andrew Chow)
2db7ca765c8fb2c71dd6f7c4f29ad70e68ff1720 Implement IsMine for DescriptorScriptPubKeyMan (Andrew Chow)
db7177af8c159abbcc209f2caafcd45d54c181c5 Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet (Andrew Chow)
78f8a92910d34247fa5d04368338c598d9908267 Implement SetType in DescriptorScriptPubKeyMan (Andrew Chow)
834de0300cde57ca3f662fb7aa5b1bdaed68bc8f Store WalletDescriptor in DescriptorScriptPubKeyMan (Andrew Chow)
d8132669e10c1db9ae0c2ea0d3f822d7d2f01345 Add a lock cs_desc_man for DescriptorScriptPubKeyMan (Andrew Chow)
3194a7f88ac1a32997b390b4f188c4b6a4af04a5 Introduce WalletDescriptor class (Andrew Chow)
6b13cd3fa854dfaeb9e269bff3d67cacc0e5b5dc Create LegacyScriptPubKeyMan when not a descriptor wallet (Andrew Chow)
aeac157c9dc141546b45e06ba9c2e641ad86083f Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet (Andrew Chow)
96accc73f067c7c95946e9932645dd821ef67f63 Add WALLET_FLAG_DESCRIPTORS (Andrew Chow)
6b8119af53ee2fdb4c4b5b24b4e650c0dc3bd27c Introduce DescriptorScriptPubKeyMan as a dummy class (Andrew Chow)
06620302c713cae65ee8e4ff9302e4c88e2a1285 Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it (Andrew Chow)
Pull request description:
Introducing the wallet of the glorious future (again): native descriptor wallets. With native descriptor wallets, addresses are generated from descriptors. Instead of generating keys and deriving addresses from keys, addresses come from the scriptPubKeys produced by a descriptor. Native descriptor wallets will be optional for now and can only be created by using `createwallet`.
Descriptor wallets will store descriptors, master keys from the descriptor, and descriptor cache entries. Keys are derived from descriptors on the fly. In order to allow choosing different address types, 6 descriptors are needed for normal use. There is a pair of primary and change descriptors for each of the 3 address types. With the default keypool size of 1000, each descriptor has 1000 scriptPubKeys and descriptor cache entries pregenerated. This has a side effect of making wallets large since 6000 pubkeys are written to the wallet by default, instead of the current 2000. scriptPubKeys are kept only in memory and are generated every time a descriptor is loaded. By default, we use the standard BIP 44, 49, 84 derivation paths with an external and internal derivation chain for each.
Descriptors can also be imported with a new `importdescriptors` RPC.
Native descriptor wallets use the `ScriptPubKeyMan` interface introduced in #16341 to add a `DescriptorScriptPubKeyMan`. This defines a different IsMine which uses the simpler model of "does this scriptPubKey exist in this wallet". Furthermore, `DescriptorScriptPubKeyMan` does not have watchonly, so with native descriptor wallets, it is not possible to have a wallet with both watchonly and non-watchonly things. Rather a wallet with `disable_private_keys` needs to be used for watchonly things.
A `--descriptor` option was added to some tests (`wallet_basic.py`, `wallet_encryption.py`, `wallet_keypool.py`, `wallet_keypool_topup.py`, and `wallet_labels.py`) to allow for these tests to use descriptor wallets. Additionally, several RPCs are disabled for descriptor wallets (`importprivkey`, `importpubkey`, `importaddress`, `importmulti`, `addmultisigaddress`, `dumpprivkey`, `dumpwallet`, `importwallet`, and `sethdseed`).
ACKs for top commit:
Sjors:
utACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82 (rebased, nits addressed)
jonatack:
Code review re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82.
fjahr:
re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
instagibbs:
light re-ACK 223588b
meshcollider:
Code review ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
Tree-SHA512: 59bc52aeddbb769ed5f420d5d240d8137847ac821b588eb616b34461253510c1717d6a70bab8765631738747336ae06f45ba39603ccd17f483843e5ed9a90986
Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it
Introduce DescriptorScriptPubKeyMan as a dummy class
Add WALLET_FLAG_DESCRIPTORS
Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet
Create LegacyScriptPubKeyMan when not a descriptor wallet
Introduce WalletDescriptor class
WalletDescriptor is a Descriptor with other wallet metadata
Add a lock cs_desc_man for DescriptorScriptPubKeyMan
Store WalletDescriptor in DescriptorScriptPubKeyMan
Implement SetType in DescriptorScriptPubKeyMan
Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet
Implement IsMine for DescriptorScriptPubKeyMan
Adds a set of scriptPubKeys that DescriptorScriptPubKeyMan tracks.
If the given script is in that set, it is considered ISMINE_SPENDABLE
Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan
Implement IsHDEnabled in DescriptorScriptPubKeyMan
Implement GetID for DescriptorScriptPubKeyMan
Load the descriptor cache from the wallet file
Implement loading of keys for DescriptorScriptPubKeyMan
Add IsSingleType to Descriptors
IsSingleType will return whether the descriptor will give one or multiple scriptPubKeys
Implement several simple functions in DescriptorScriptPubKeyMan
Implements a bunch of one liners: UpgradeKeyMetadata, IsFirstRun, HavePrivateKeys,
KeypoolCountExternalKeys, GetKeypoolSize, GetTimeFirstKey, CanGetAddresses,
RewriteDB
Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file
Implement SetupGeneration for DescriptorScriptPubKeyMan
Implement TopUp in DescriptorScriptPubKeyMan
Implement GetNewDestination for DescriptorScriptPubKeyMan
Implement Unlock and Encrypt in DescriptorScriptPubKeyMan
Implement GetReservedDestination in DescriptorScriptPubKeyMan
Implement ReturnDestination in DescriptorScriptPubKeyMan
Implement GetKeypoolOldestTime and only display it if greater than 0
Implement GetSolvingProvider for DescriptorScriptPubKeyMan
Internally, a GetSigningProvider function is introduced which allows for
some private keys to be optionally included. This can be called with a
script as the argument (i.e. a scriptPubKey from our wallet when we are
signing) or with a pubkey. In order to know what index to expand the
private keys for that pubkey, we need to also cache all of the pubkeys
involved when we expand the descriptor. So SetCache and TopUp are
updated to do this too.
Implement SignTransaction in DescriptorScriptPubKeyMan
Implement SignMessage for descriptor wallets
Implement FillPSBT in DescriptorScriptPubKeyMan
FillPSBT will add our own scripts to the PSBT if those inputs are ours.
If an input also lists pubkeys that we happen to know the private keys
for, we will sign those inputs too.
Change GetMetadata to use unique_ptr<CKeyMetadata>
Implement GetMetadata in DescriptorScriptPubKeyMan
Be able to create new wallets with DescriptorScriptPubKeyMans as backing
Generate new descriptors when encrypting
Add IsLegacy to CWallet so that the GUI knows whether to show watchonly
add importdescriptors RPC and tests for native descriptor wallets
Co-authored-by: Andrew Chow <achow101-github@achow101.com>
Functional tests for descriptor wallets
Change wallet_encryption.py to use signmessage instead of dumpprivkey
Return error when no ScriptPubKeyMan is available for specified type
When a CWallet doesn't have a ScriptPubKeyMan for the requested type
in GetNewDestination, give a meaningful error. Also handle this in
Qt which did not do anything with errors.
Implement CWallet::IsSpentKey for non-LegacySPKMans
tests: Add RPCOverloadWrapper which overloads some disabled RPCs
RPCOverloadWrapper overloads some deprecated or disabled RPCs with
an implementation using other RPCs to avoid having a ton of code churn
around replacing those RPCs.
Add a --descriptors option to various tests
Adds a --descriptors option globally to the test framework. This will
make the test create and use descriptor wallets. However some tests may
not work with this.
Some tests are modified to work with --descriptors and run with that
option in test_runer:
* wallet_basic.py
* wallet_encryption.py
* wallet_keypool.py <---- wallet_keypool_hd.py actually
* wallet_keypool_topup.py
* wallet_labels.py
* wallet_avoidreuse.py
2019-07-16 19:34:35 +02:00
if self . options . descriptors :
self . extra_args = [ [
" -acceptnonstdtxn=1 "
] for i in range ( self . num_nodes ) ]
else :
self . extra_args = [ [
" -acceptnonstdtxn=1 " ,
' -usehd= {:d} ' . format ( i % 2 == 0 )
] for i in range ( self . num_nodes ) ]
2017-09-01 18:47:13 +02:00
self . setup_clean_chain = True
2019-12-09 19:52:38 +01:00
self . supports_cli = False
2014-11-19 21:55:40 +01:00
2018-09-13 12:33:15 +02:00
def skip_test_if_missing_module ( self ) :
self . skip_if_no_wallet ( )
2017-05-02 20:02:55 +02:00
def setup_network ( self ) :
2018-11-02 16:41:29 +01:00
self . setup_nodes ( )
# Only need nodes 0-2 running at start of test
self . stop_node ( 3 )
2022-09-24 14:36:35 +02:00
self . connect_nodes ( 0 , 1 )
self . connect_nodes ( 1 , 2 )
self . connect_nodes ( 0 , 2 )
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2014-11-19 21:55:40 +01:00
2017-09-01 18:47:13 +02:00
def check_fee_amount ( self , curr_balance , balance_with_fee , fee_per_byte , tx_size ) :
""" Return curr_balance after asserting the fee was in range """
fee = balance_with_fee - curr_balance
assert_fee_amount ( fee , tx_size , fee_per_byte * 1000 )
return curr_balance
2016-04-19 13:22:11 +02:00
2017-09-01 18:47:13 +02:00
def run_test ( self ) :
Merge #16528: Native Descriptor Wallets using DescriptorScriptPubKeyMan
223588b1bbc63dc57098bbd0baa48635e0cc0b82 Add a --descriptors option to various tests (Andrew Chow)
869f7ab30aeb4d7fbd563c535b55467a8a0430cf tests: Add RPCOverloadWrapper which overloads some disabled RPCs (Andrew Chow)
cf060628590fab87d73f278e744d70ef2d5d81db Correctly check for default wallet (Andrew Chow)
886e0d75f5fea2421190aa4812777d89f68962cc Implement CWallet::IsSpentKey for non-LegacySPKMans (Andrew Chow)
3c19fdd2a2fd5394fcfa75b2ba84ab2277cbdabf Return error when no ScriptPubKeyMan is available for specified type (Andrew Chow)
388ba94231f2f10a0be751c562cdd4650510a90a Change wallet_encryption.py to use signmessage instead of dumpprivkey (Andrew Chow)
1346e14831489f9c8f53a08f9dfed61d55d53c6f Functional tests for descriptor wallets (Andrew Chow)
f193ea889ddb53d9a5c47647966681d525e38368 add importdescriptors RPC and tests for native descriptor wallets (Hugo Nguyen)
ce24a944940019185efebcc5d85eac458ed26016 Add IsLegacy to CWallet so that the GUI knows whether to show watchonly (Andrew Chow)
1cb42b22b11c27e64462afc25a94b2fc50bfa113 Generate new descriptors when encrypting (Andrew Chow)
82ae02b1656819f4bd5023b8955447e1d4ea8692 Be able to create new wallets with DescriptorScriptPubKeyMans as backing (Andrew Chow)
b713baa75a62335ab9c0eed9ef76a95bfec30668 Implement GetMetadata in DescriptorScriptPubKeyMan (Andrew Chow)
8b9603bd0b443e2f7984eb72bf2e21cf02af0bcb Change GetMetadata to use unique_ptr<CKeyMetadata> (Andrew Chow)
72a9540df96ffdb94f039b9c14eaacdc7d961196 Implement FillPSBT in DescriptorScriptPubKeyMan (Andrew Chow)
84b4978c02102171775c77a45f6ec198930f0a88 Implement SignMessage for descriptor wallets (Andrew Chow)
bde7c9fa38775a81d53ac0484fa9c98076a0c7d1 Implement SignTransaction in DescriptorScriptPubKeyMan (Andrew Chow)
d50c8ddd4190f20bf0debd410348b73408ec3143 Implement GetSolvingProvider for DescriptorScriptPubKeyMan (Andrew Chow)
f1ca5feb4ad668a3e1ae543d0addd5f483f1a88f Implement GetKeypoolOldestTime and only display it if greater than 0 (Andrew Chow)
586b57a9a6b4b12a78f792785b63a5a1743bce0c Implement ReturnDestination in DescriptorScriptPubKeyMan (Andrew Chow)
f866957979c23cefd41efa9dae9e53b9177818dc Implement GetReservedDestination in DescriptorScriptPubKeyMan (Andrew Chow)
a775f7c7fd0b9094fcbeee6ba92206d5bbb19164 Implement Unlock and Encrypt in DescriptorScriptPubKeyMan (Andrew Chow)
bfdd0734869a22217c15858d7a76d0dacc2ebc86 Implement GetNewDestination for DescriptorScriptPubKeyMan (Andrew Chow)
58c7651821b0eeff0a99dc61d78d2e9e07986580 Implement TopUp in DescriptorScriptPubKeyMan (Andrew Chow)
e014886a342508f7c8d80323eee9a5f314eaf94c Implement SetupGeneration for DescriptorScriptPubKeyMan (Andrew Chow)
46dfb99768e7d03a3cf552812d5b41ceaebc06be Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file (Andrew Chow)
4cb9b69be031e1dc65d8964794781b347fd948f5 Implement several simple functions in DescriptorScriptPubKeyMan (Andrew Chow)
d1ec3e4f19487b4b100f80ad02eac063c571777d Add IsSingleType to Descriptors (Andrew Chow)
953feb3d2724f5398dd48990c4957a19313d2c8c Implement loading of keys for DescriptorScriptPubKeyMan (Andrew Chow)
2363e9fcaa41b68bf11153f591b95f2d41ff9a1a Load the descriptor cache from the wallet file (Andrew Chow)
46c46aebb7943e1e2e96755e94dc6c197920bf75 Implement GetID for DescriptorScriptPubKeyMan (Andrew Chow)
ec2f9e1178c8e38c0a5ca063fe81adac8f916348 Implement IsHDEnabled in DescriptorScriptPubKeyMan (Andrew Chow)
741122d4c1a62ced3e96d16d67f4eeb3a6522d99 Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan (Andrew Chow)
2db7ca765c8fb2c71dd6f7c4f29ad70e68ff1720 Implement IsMine for DescriptorScriptPubKeyMan (Andrew Chow)
db7177af8c159abbcc209f2caafcd45d54c181c5 Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet (Andrew Chow)
78f8a92910d34247fa5d04368338c598d9908267 Implement SetType in DescriptorScriptPubKeyMan (Andrew Chow)
834de0300cde57ca3f662fb7aa5b1bdaed68bc8f Store WalletDescriptor in DescriptorScriptPubKeyMan (Andrew Chow)
d8132669e10c1db9ae0c2ea0d3f822d7d2f01345 Add a lock cs_desc_man for DescriptorScriptPubKeyMan (Andrew Chow)
3194a7f88ac1a32997b390b4f188c4b6a4af04a5 Introduce WalletDescriptor class (Andrew Chow)
6b13cd3fa854dfaeb9e269bff3d67cacc0e5b5dc Create LegacyScriptPubKeyMan when not a descriptor wallet (Andrew Chow)
aeac157c9dc141546b45e06ba9c2e641ad86083f Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet (Andrew Chow)
96accc73f067c7c95946e9932645dd821ef67f63 Add WALLET_FLAG_DESCRIPTORS (Andrew Chow)
6b8119af53ee2fdb4c4b5b24b4e650c0dc3bd27c Introduce DescriptorScriptPubKeyMan as a dummy class (Andrew Chow)
06620302c713cae65ee8e4ff9302e4c88e2a1285 Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it (Andrew Chow)
Pull request description:
Introducing the wallet of the glorious future (again): native descriptor wallets. With native descriptor wallets, addresses are generated from descriptors. Instead of generating keys and deriving addresses from keys, addresses come from the scriptPubKeys produced by a descriptor. Native descriptor wallets will be optional for now and can only be created by using `createwallet`.
Descriptor wallets will store descriptors, master keys from the descriptor, and descriptor cache entries. Keys are derived from descriptors on the fly. In order to allow choosing different address types, 6 descriptors are needed for normal use. There is a pair of primary and change descriptors for each of the 3 address types. With the default keypool size of 1000, each descriptor has 1000 scriptPubKeys and descriptor cache entries pregenerated. This has a side effect of making wallets large since 6000 pubkeys are written to the wallet by default, instead of the current 2000. scriptPubKeys are kept only in memory and are generated every time a descriptor is loaded. By default, we use the standard BIP 44, 49, 84 derivation paths with an external and internal derivation chain for each.
Descriptors can also be imported with a new `importdescriptors` RPC.
Native descriptor wallets use the `ScriptPubKeyMan` interface introduced in #16341 to add a `DescriptorScriptPubKeyMan`. This defines a different IsMine which uses the simpler model of "does this scriptPubKey exist in this wallet". Furthermore, `DescriptorScriptPubKeyMan` does not have watchonly, so with native descriptor wallets, it is not possible to have a wallet with both watchonly and non-watchonly things. Rather a wallet with `disable_private_keys` needs to be used for watchonly things.
A `--descriptor` option was added to some tests (`wallet_basic.py`, `wallet_encryption.py`, `wallet_keypool.py`, `wallet_keypool_topup.py`, and `wallet_labels.py`) to allow for these tests to use descriptor wallets. Additionally, several RPCs are disabled for descriptor wallets (`importprivkey`, `importpubkey`, `importaddress`, `importmulti`, `addmultisigaddress`, `dumpprivkey`, `dumpwallet`, `importwallet`, and `sethdseed`).
ACKs for top commit:
Sjors:
utACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82 (rebased, nits addressed)
jonatack:
Code review re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82.
fjahr:
re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
instagibbs:
light re-ACK 223588b
meshcollider:
Code review ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
Tree-SHA512: 59bc52aeddbb769ed5f420d5d240d8137847ac821b588eb616b34461253510c1717d6a70bab8765631738747336ae06f45ba39603ccd17f483843e5ed9a90986
Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it
Introduce DescriptorScriptPubKeyMan as a dummy class
Add WALLET_FLAG_DESCRIPTORS
Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet
Create LegacyScriptPubKeyMan when not a descriptor wallet
Introduce WalletDescriptor class
WalletDescriptor is a Descriptor with other wallet metadata
Add a lock cs_desc_man for DescriptorScriptPubKeyMan
Store WalletDescriptor in DescriptorScriptPubKeyMan
Implement SetType in DescriptorScriptPubKeyMan
Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet
Implement IsMine for DescriptorScriptPubKeyMan
Adds a set of scriptPubKeys that DescriptorScriptPubKeyMan tracks.
If the given script is in that set, it is considered ISMINE_SPENDABLE
Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan
Implement IsHDEnabled in DescriptorScriptPubKeyMan
Implement GetID for DescriptorScriptPubKeyMan
Load the descriptor cache from the wallet file
Implement loading of keys for DescriptorScriptPubKeyMan
Add IsSingleType to Descriptors
IsSingleType will return whether the descriptor will give one or multiple scriptPubKeys
Implement several simple functions in DescriptorScriptPubKeyMan
Implements a bunch of one liners: UpgradeKeyMetadata, IsFirstRun, HavePrivateKeys,
KeypoolCountExternalKeys, GetKeypoolSize, GetTimeFirstKey, CanGetAddresses,
RewriteDB
Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file
Implement SetupGeneration for DescriptorScriptPubKeyMan
Implement TopUp in DescriptorScriptPubKeyMan
Implement GetNewDestination for DescriptorScriptPubKeyMan
Implement Unlock and Encrypt in DescriptorScriptPubKeyMan
Implement GetReservedDestination in DescriptorScriptPubKeyMan
Implement ReturnDestination in DescriptorScriptPubKeyMan
Implement GetKeypoolOldestTime and only display it if greater than 0
Implement GetSolvingProvider for DescriptorScriptPubKeyMan
Internally, a GetSigningProvider function is introduced which allows for
some private keys to be optionally included. This can be called with a
script as the argument (i.e. a scriptPubKey from our wallet when we are
signing) or with a pubkey. In order to know what index to expand the
private keys for that pubkey, we need to also cache all of the pubkeys
involved when we expand the descriptor. So SetCache and TopUp are
updated to do this too.
Implement SignTransaction in DescriptorScriptPubKeyMan
Implement SignMessage for descriptor wallets
Implement FillPSBT in DescriptorScriptPubKeyMan
FillPSBT will add our own scripts to the PSBT if those inputs are ours.
If an input also lists pubkeys that we happen to know the private keys
for, we will sign those inputs too.
Change GetMetadata to use unique_ptr<CKeyMetadata>
Implement GetMetadata in DescriptorScriptPubKeyMan
Be able to create new wallets with DescriptorScriptPubKeyMans as backing
Generate new descriptors when encrypting
Add IsLegacy to CWallet so that the GUI knows whether to show watchonly
add importdescriptors RPC and tests for native descriptor wallets
Co-authored-by: Andrew Chow <achow101-github@achow101.com>
Functional tests for descriptor wallets
Change wallet_encryption.py to use signmessage instead of dumpprivkey
Return error when no ScriptPubKeyMan is available for specified type
When a CWallet doesn't have a ScriptPubKeyMan for the requested type
in GetNewDestination, give a meaningful error. Also handle this in
Qt which did not do anything with errors.
Implement CWallet::IsSpentKey for non-LegacySPKMans
tests: Add RPCOverloadWrapper which overloads some disabled RPCs
RPCOverloadWrapper overloads some deprecated or disabled RPCs with
an implementation using other RPCs to avoid having a ton of code churn
around replacing those RPCs.
Add a --descriptors option to various tests
Adds a --descriptors option globally to the test framework. This will
make the test create and use descriptor wallets. However some tests may
not work with this.
Some tests are modified to work with --descriptors and run with that
option in test_runer:
* wallet_basic.py
* wallet_encryption.py
* wallet_keypool.py <---- wallet_keypool_hd.py actually
* wallet_keypool_topup.py
* wallet_labels.py
* wallet_avoidreuse.py
2019-07-16 19:34:35 +02:00
2016-04-19 13:22:11 +02:00
# Check that there's no UTXO on none of the nodes
assert_equal ( len ( self . nodes [ 0 ] . listunspent ( ) ) , 0 )
assert_equal ( len ( self . nodes [ 1 ] . listunspent ( ) ) , 0 )
assert_equal ( len ( self . nodes [ 2 ] . listunspent ( ) ) , 0 )
2017-03-09 21:16:20 +01:00
self . log . info ( " Mining blocks... " )
2014-11-19 21:55:40 +01:00
2015-04-01 05:28:28 +02:00
self . nodes [ 0 ] . generate ( 1 )
2014-11-19 21:55:40 +01:00
2014-12-25 08:57:16 +01:00
walletinfo = self . nodes [ 0 ] . getwalletinfo ( )
2016-03-06 16:14:39 +01:00
assert_equal ( walletinfo [ ' immature_balance ' ] , 500 )
2014-12-25 08:57:16 +01:00
assert_equal ( walletinfo [ ' balance ' ] , 0 )
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2021-05-31 11:26:16 +02:00
self . nodes [ 1 ] . generate ( COINBASE_MATURITY + 1 )
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2014-11-19 21:55:40 +01:00
2016-03-06 16:14:39 +01:00
assert_equal ( self . nodes [ 0 ] . getbalance ( ) , 500 )
assert_equal ( self . nodes [ 1 ] . getbalance ( ) , 500 )
2014-11-19 21:55:40 +01:00
assert_equal ( self . nodes [ 2 ] . getbalance ( ) , 0 )
2016-04-19 13:22:11 +02:00
# Check that only first and second nodes have UTXOs
2017-04-29 14:47:00 +02:00
utxos = self . nodes [ 0 ] . listunspent ( )
assert_equal ( len ( utxos ) , 1 )
2016-04-19 13:22:11 +02:00
assert_equal ( len ( self . nodes [ 1 ] . listunspent ( ) ) , 1 )
assert_equal ( len ( self . nodes [ 2 ] . listunspent ( ) ) , 0 )
2017-08-29 01:45:11 +02:00
self . log . info ( " test gettxout " )
confirmed_txid , confirmed_index = utxos [ 0 ] [ " txid " ] , utxos [ 0 ] [ " vout " ]
# First, outputs that are unspent both in the chain and in the
# mempool should appear with or without include_mempool
txout = self . nodes [ 0 ] . gettxout ( txid = confirmed_txid , n = confirmed_index , include_mempool = False )
assert_equal ( txout [ ' value ' ] , 500 )
txout = self . nodes [ 0 ] . gettxout ( txid = confirmed_txid , n = confirmed_index , include_mempool = True )
assert_equal ( txout [ ' value ' ] , 500 )
2020-07-07 19:37:53 +02:00
2016-03-06 16:14:39 +01:00
# Send 210 DASH from 0 to 2 using sendtoaddress call.
2014-11-19 21:55:40 +01:00
# Second transaction will be child of first, and will require a fee
2016-03-06 16:14:39 +01:00
self . nodes [ 0 ] . sendtoaddress ( self . nodes [ 2 ] . getnewaddress ( ) , 110 )
2017-04-29 14:47:00 +02:00
mempool_txid = self . nodes [ 0 ] . sendtoaddress ( self . nodes [ 2 ] . getnewaddress ( ) , 100 )
2017-08-29 01:45:11 +02:00
self . log . info ( " test gettxout (second part) " )
2017-04-29 14:47:00 +02:00
# utxo spent in mempool should be visible if you exclude mempool
# but invisible if you include mempool
txout = self . nodes [ 0 ] . gettxout ( confirmed_txid , confirmed_index , False )
assert_equal ( txout [ ' value ' ] , 500 )
2021-04-17 07:50:48 +02:00
txout = self . nodes [ 0 ] . gettxout ( confirmed_txid , confirmed_index ) # by default include_mempool=True
assert txout is None
2017-04-29 14:47:00 +02:00
txout = self . nodes [ 0 ] . gettxout ( confirmed_txid , confirmed_index , True )
assert txout is None
# new utxo from mempool should be invisible if you exclude mempool
# but visible if you include mempool
txout = self . nodes [ 0 ] . gettxout ( mempool_txid , 0 , False )
assert txout is None
txout1 = self . nodes [ 0 ] . gettxout ( mempool_txid , 0 , True )
txout2 = self . nodes [ 0 ] . gettxout ( mempool_txid , 1 , True )
# note the mempool tx will have randomly assigned indices
# but 10 will go to node2 and the rest will go to node0
balance = self . nodes [ 0 ] . getbalance ( )
assert_equal ( set ( [ txout1 [ ' value ' ] , txout2 [ ' value ' ] ] ) , set ( [ 100 , balance ] ) )
2014-11-19 21:55:40 +01:00
2014-12-25 08:57:16 +01:00
walletinfo = self . nodes [ 0 ] . getwalletinfo ( )
assert_equal ( walletinfo [ ' immature_balance ' ] , 0 )
2015-04-28 16:48:28 +02:00
# Have node0 mine a block, thus it will collect its own fee.
2015-04-01 05:28:28 +02:00
self . nodes [ 0 ] . generate ( 1 )
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2014-11-19 21:55:40 +01:00
2016-03-06 18:30:51 +01:00
# Exercise locking of unspent outputs
unspent_0 = self . nodes [ 2 ] . listunspent ( ) [ 0 ]
unspent_0 = { " txid " : unspent_0 [ " txid " ] , " vout " : unspent_0 [ " vout " ] }
2017-11-16 12:23:55 +01:00
assert_raises_rpc_error ( - 8 , " Invalid parameter, expected locked output " , self . nodes [ 2 ] . lockunspent , True , [ unspent_0 ] )
2016-03-06 18:30:51 +01:00
self . nodes [ 2 ] . lockunspent ( False , [ unspent_0 ] )
2017-11-16 12:23:55 +01:00
assert_raises_rpc_error ( - 8 , " Invalid parameter, output already locked " , self . nodes [ 2 ] . lockunspent , False , [ unspent_0 ] )
2019-09-25 11:34:51 +02:00
assert_raises_rpc_error ( - 4 , " Insufficient funds " , self . nodes [ 2 ] . sendtoaddress , self . nodes [ 2 ] . getnewaddress ( ) , 200 )
2016-03-06 18:30:51 +01:00
assert_equal ( [ unspent_0 ] , self . nodes [ 2 ] . listlockunspent ( ) )
self . nodes [ 2 ] . lockunspent ( True , [ unspent_0 ] )
assert_equal ( len ( self . nodes [ 2 ] . listlockunspent ( ) ) , 0 )
2018-09-24 20:54:10 +02:00
assert_raises_rpc_error ( - 8 , " txid must be of length 64 (not 34, for ' 0000000000000000000000000000000000 ' ) " ,
2021-04-12 16:58:49 +02:00
self . nodes [ 2 ] . lockunspent , False ,
[ { " txid " : " 0000000000000000000000000000000000 " , " vout " : 0 } ] )
2018-09-24 20:54:10 +02:00
assert_raises_rpc_error ( - 8 , " txid must be hexadecimal string (not ' ZZZ0000000000000000000000000000000000000000000000000000000000000 ' ) " ,
self . nodes [ 2 ] . lockunspent , False ,
[ { " txid " : " ZZZ0000000000000000000000000000000000000000000000000000000000000 " , " vout " : 0 } ] )
assert_raises_rpc_error ( - 8 , " Invalid parameter, unknown transaction " ,
self . nodes [ 2 ] . lockunspent , False ,
[ { " txid " : " 0000000000000000000000000000000000000000000000000000000000000000 " , " vout " : 0 } ] )
2017-11-16 12:23:55 +01:00
assert_raises_rpc_error ( - 8 , " Invalid parameter, vout index out of bounds " ,
2021-04-12 16:58:49 +02:00
self . nodes [ 2 ] . lockunspent , False ,
[ { " txid " : unspent_0 [ " txid " ] , " vout " : 999 } ] )
2016-03-06 18:30:51 +01:00
2020-08-31 13:19:50 +02:00
# The lock on a manually selected output is ignored
2018-06-24 18:52:27 +02:00
unspent_0 = self . nodes [ 1 ] . listunspent ( ) [ 0 ]
self . nodes [ 1 ] . lockunspent ( False , [ unspent_0 ] )
tx = self . nodes [ 1 ] . createrawtransaction ( [ unspent_0 ] , { self . nodes [ 1 ] . getnewaddress ( ) : 1 } )
2020-08-31 13:19:50 +02:00
self . nodes [ 1 ] . fundrawtransaction ( tx , { " lockUnspents " : True } )
# fundrawtransaction can lock an input
self . nodes [ 1 ] . lockunspent ( True , [ unspent_0 ] )
assert_equal ( len ( self . nodes [ 1 ] . listlockunspent ( ) ) , 0 )
tx = self . nodes [ 1 ] . fundrawtransaction ( tx , { " lockUnspents " : True } ) [ ' hex ' ]
assert_equal ( len ( self . nodes [ 1 ] . listlockunspent ( ) ) , 1 )
# Send transaction
2020-12-11 03:25:55 +01:00
tx = self . nodes [ 1 ] . signrawtransactionwithwallet ( tx ) [ " hex " ]
2018-06-24 18:52:27 +02:00
self . nodes [ 1 ] . sendrawtransaction ( tx )
assert_equal ( len ( self . nodes [ 1 ] . listlockunspent ( ) ) , 0 )
2014-11-19 21:55:40 +01:00
# Have node1 generate 100 blocks (so node0 can recover the fee)
2021-05-31 11:26:16 +02:00
self . nodes [ 1 ] . generate ( COINBASE_MATURITY )
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2014-11-19 21:55:40 +01:00
2016-03-06 16:14:39 +01:00
# node0 should end up with 1000 DASH in block rewards plus fees, but
# minus the 210 plus fees sent to node2
2021-04-12 16:58:49 +02:00
assert_equal ( self . nodes [ 0 ] . getbalance ( ) , 1000 - 210 )
2016-03-06 16:14:39 +01:00
assert_equal ( self . nodes [ 2 ] . getbalance ( ) , 210 )
2014-11-19 21:55:40 +01:00
# Node0 should have two unspent outputs.
2015-07-14 21:13:16 +02:00
# Create a couple of transactions to send them to node2, submit them through
# node1, and make sure both node0 and node2 pick them up properly:
2014-11-19 21:55:40 +01:00
node0utxos = self . nodes [ 0 ] . listunspent ( 1 )
assert_equal ( len ( node0utxos ) , 2 )
2019-03-15 07:18:16 +01:00
fee_per_input = Decimal ( ' 0.00001 ' )
totalfee = 0
2014-11-19 21:55:40 +01:00
# create both transactions
txns_to_send = [ ]
2015-07-14 21:13:16 +02:00
for utxo in node0utxos :
2014-11-19 21:55:40 +01:00
inputs = [ ]
outputs = { }
2021-04-12 16:58:49 +02:00
inputs . append ( { " txid " : utxo [ " txid " ] , " vout " : utxo [ " vout " ] } )
2018-05-11 00:08:57 +02:00
outputs [ self . nodes [ 2 ] . getnewaddress ( ) ] = utxo [ " amount " ] - fee_per_input
2014-11-19 21:55:40 +01:00
raw_tx = self . nodes [ 0 ] . createrawtransaction ( inputs , outputs )
2018-02-20 03:29:22 +01:00
txns_to_send . append ( self . nodes [ 0 ] . signrawtransactionwithwallet ( raw_tx ) )
2019-03-15 07:18:16 +01:00
totalfee + = fee_per_input
2014-11-19 21:55:40 +01:00
# Have node 1 (miner) send the transactions
2019-09-25 11:51:26 +02:00
self . nodes [ 1 ] . sendrawtransaction ( hexstring = txns_to_send [ 0 ] [ " hex " ] , maxfeerate = 0 )
self . nodes [ 1 ] . sendrawtransaction ( hexstring = txns_to_send [ 1 ] [ " hex " ] , maxfeerate = 0 )
2014-11-19 21:55:40 +01:00
# Have node1 mine a block to confirm transactions:
2015-04-01 05:28:28 +02:00
self . nodes [ 1 ] . generate ( 1 )
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2014-11-19 21:55:40 +01:00
assert_equal ( self . nodes [ 0 ] . getbalance ( ) , 0 )
2019-03-15 07:18:16 +01:00
assert_equal ( self . nodes [ 2 ] . getbalance ( ) , 1000 - totalfee )
2014-11-19 21:55:40 +01:00
2017-11-16 12:23:55 +01:00
# Verify that a spent output cannot be locked anymore
spent_0 = { " txid " : node0utxos [ 0 ] [ " txid " ] , " vout " : node0utxos [ 0 ] [ " vout " ] }
assert_raises_rpc_error ( - 8 , " Invalid parameter, expected unspent output " , self . nodes [ 0 ] . lockunspent , False , [ spent_0 ] )
2016-03-06 16:14:39 +01:00
# Send 100 DASH normal
2014-07-23 14:34:36 +02:00
address = self . nodes [ 0 ] . getnewaddress ( " test " )
2017-12-07 10:43:23 +01:00
fee_per_byte = Decimal ( ' 0.00001 ' ) / 1000
2015-09-21 12:49:13 +02:00
self . nodes [ 2 ] . settxfee ( fee_per_byte * 1000 )
2016-03-06 16:14:39 +01:00
txid = self . nodes [ 2 ] . sendtoaddress ( address , 100 , " " , " " , False )
2015-04-01 05:28:28 +02:00
self . nodes [ 2 ] . generate ( 1 )
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2019-02-01 15:35:08 +01:00
node_2_bal = self . check_fee_amount ( self . nodes [ 2 ] . getbalance ( ) , Decimal ( ' 900 ' ) - totalfee , fee_per_byte , count_bytes ( self . nodes [ 2 ] . gettransaction ( txid ) [ ' hex ' ] ) )
2016-03-06 16:14:39 +01:00
assert_equal ( self . nodes [ 0 ] . getbalance ( ) , Decimal ( ' 100 ' ) )
2014-07-23 14:34:36 +02:00
2016-03-06 16:14:39 +01:00
# Send 100 DASH with subtract fee from amount
txid = self . nodes [ 2 ] . sendtoaddress ( address , 100 , " " , " " , True )
2015-04-01 05:28:28 +02:00
self . nodes [ 2 ] . generate ( 1 )
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2016-03-06 16:14:39 +01:00
node_2_bal - = Decimal ( ' 100 ' )
2015-09-21 12:49:13 +02:00
assert_equal ( self . nodes [ 2 ] . getbalance ( ) , node_2_bal )
2019-02-01 15:35:08 +01:00
node_0_bal = self . check_fee_amount ( self . nodes [ 0 ] . getbalance ( ) , Decimal ( ' 200 ' ) , fee_per_byte , count_bytes ( self . nodes [ 2 ] . gettransaction ( txid ) [ ' hex ' ] ) )
2014-11-19 21:55:40 +01:00
2016-03-06 16:14:39 +01:00
# Sendmany 100 DASH
2018-05-11 00:08:57 +02:00
txid = self . nodes [ 2 ] . sendmany ( ' ' , { address : 100 } , 0 , False , " " , [ ] )
2015-04-01 05:28:28 +02:00
self . nodes [ 2 ] . generate ( 1 )
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2016-03-06 16:14:39 +01:00
node_0_bal + = Decimal ( ' 100 ' )
2019-02-01 15:35:08 +01:00
node_2_bal = self . check_fee_amount ( self . nodes [ 2 ] . getbalance ( ) , node_2_bal - Decimal ( ' 100 ' ) , fee_per_byte , count_bytes ( self . nodes [ 2 ] . gettransaction ( txid ) [ ' hex ' ] ) )
2015-09-21 12:49:13 +02:00
assert_equal ( self . nodes [ 0 ] . getbalance ( ) , node_0_bal )
2015-03-05 14:59:19 +01:00
2016-03-06 16:14:39 +01:00
# Sendmany 100 DASH with subtract fee from amount
2018-05-11 00:08:57 +02:00
txid = self . nodes [ 2 ] . sendmany ( ' ' , { address : 100 } , 0 , False , " " , [ address ] )
2015-04-01 05:28:28 +02:00
self . nodes [ 2 ] . generate ( 1 )
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2016-03-06 16:14:39 +01:00
node_2_bal - = Decimal ( ' 100 ' )
2015-09-21 12:49:13 +02:00
assert_equal ( self . nodes [ 2 ] . getbalance ( ) , node_2_bal )
2019-02-01 15:35:08 +01:00
node_0_bal = self . check_fee_amount ( self . nodes [ 0 ] . getbalance ( ) , node_0_bal + Decimal ( ' 100 ' ) , fee_per_byte , count_bytes ( self . nodes [ 2 ] . gettransaction ( txid ) [ ' hex ' ] ) )
2015-03-05 14:59:19 +01:00
Merge #16528: Native Descriptor Wallets using DescriptorScriptPubKeyMan
223588b1bbc63dc57098bbd0baa48635e0cc0b82 Add a --descriptors option to various tests (Andrew Chow)
869f7ab30aeb4d7fbd563c535b55467a8a0430cf tests: Add RPCOverloadWrapper which overloads some disabled RPCs (Andrew Chow)
cf060628590fab87d73f278e744d70ef2d5d81db Correctly check for default wallet (Andrew Chow)
886e0d75f5fea2421190aa4812777d89f68962cc Implement CWallet::IsSpentKey for non-LegacySPKMans (Andrew Chow)
3c19fdd2a2fd5394fcfa75b2ba84ab2277cbdabf Return error when no ScriptPubKeyMan is available for specified type (Andrew Chow)
388ba94231f2f10a0be751c562cdd4650510a90a Change wallet_encryption.py to use signmessage instead of dumpprivkey (Andrew Chow)
1346e14831489f9c8f53a08f9dfed61d55d53c6f Functional tests for descriptor wallets (Andrew Chow)
f193ea889ddb53d9a5c47647966681d525e38368 add importdescriptors RPC and tests for native descriptor wallets (Hugo Nguyen)
ce24a944940019185efebcc5d85eac458ed26016 Add IsLegacy to CWallet so that the GUI knows whether to show watchonly (Andrew Chow)
1cb42b22b11c27e64462afc25a94b2fc50bfa113 Generate new descriptors when encrypting (Andrew Chow)
82ae02b1656819f4bd5023b8955447e1d4ea8692 Be able to create new wallets with DescriptorScriptPubKeyMans as backing (Andrew Chow)
b713baa75a62335ab9c0eed9ef76a95bfec30668 Implement GetMetadata in DescriptorScriptPubKeyMan (Andrew Chow)
8b9603bd0b443e2f7984eb72bf2e21cf02af0bcb Change GetMetadata to use unique_ptr<CKeyMetadata> (Andrew Chow)
72a9540df96ffdb94f039b9c14eaacdc7d961196 Implement FillPSBT in DescriptorScriptPubKeyMan (Andrew Chow)
84b4978c02102171775c77a45f6ec198930f0a88 Implement SignMessage for descriptor wallets (Andrew Chow)
bde7c9fa38775a81d53ac0484fa9c98076a0c7d1 Implement SignTransaction in DescriptorScriptPubKeyMan (Andrew Chow)
d50c8ddd4190f20bf0debd410348b73408ec3143 Implement GetSolvingProvider for DescriptorScriptPubKeyMan (Andrew Chow)
f1ca5feb4ad668a3e1ae543d0addd5f483f1a88f Implement GetKeypoolOldestTime and only display it if greater than 0 (Andrew Chow)
586b57a9a6b4b12a78f792785b63a5a1743bce0c Implement ReturnDestination in DescriptorScriptPubKeyMan (Andrew Chow)
f866957979c23cefd41efa9dae9e53b9177818dc Implement GetReservedDestination in DescriptorScriptPubKeyMan (Andrew Chow)
a775f7c7fd0b9094fcbeee6ba92206d5bbb19164 Implement Unlock and Encrypt in DescriptorScriptPubKeyMan (Andrew Chow)
bfdd0734869a22217c15858d7a76d0dacc2ebc86 Implement GetNewDestination for DescriptorScriptPubKeyMan (Andrew Chow)
58c7651821b0eeff0a99dc61d78d2e9e07986580 Implement TopUp in DescriptorScriptPubKeyMan (Andrew Chow)
e014886a342508f7c8d80323eee9a5f314eaf94c Implement SetupGeneration for DescriptorScriptPubKeyMan (Andrew Chow)
46dfb99768e7d03a3cf552812d5b41ceaebc06be Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file (Andrew Chow)
4cb9b69be031e1dc65d8964794781b347fd948f5 Implement several simple functions in DescriptorScriptPubKeyMan (Andrew Chow)
d1ec3e4f19487b4b100f80ad02eac063c571777d Add IsSingleType to Descriptors (Andrew Chow)
953feb3d2724f5398dd48990c4957a19313d2c8c Implement loading of keys for DescriptorScriptPubKeyMan (Andrew Chow)
2363e9fcaa41b68bf11153f591b95f2d41ff9a1a Load the descriptor cache from the wallet file (Andrew Chow)
46c46aebb7943e1e2e96755e94dc6c197920bf75 Implement GetID for DescriptorScriptPubKeyMan (Andrew Chow)
ec2f9e1178c8e38c0a5ca063fe81adac8f916348 Implement IsHDEnabled in DescriptorScriptPubKeyMan (Andrew Chow)
741122d4c1a62ced3e96d16d67f4eeb3a6522d99 Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan (Andrew Chow)
2db7ca765c8fb2c71dd6f7c4f29ad70e68ff1720 Implement IsMine for DescriptorScriptPubKeyMan (Andrew Chow)
db7177af8c159abbcc209f2caafcd45d54c181c5 Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet (Andrew Chow)
78f8a92910d34247fa5d04368338c598d9908267 Implement SetType in DescriptorScriptPubKeyMan (Andrew Chow)
834de0300cde57ca3f662fb7aa5b1bdaed68bc8f Store WalletDescriptor in DescriptorScriptPubKeyMan (Andrew Chow)
d8132669e10c1db9ae0c2ea0d3f822d7d2f01345 Add a lock cs_desc_man for DescriptorScriptPubKeyMan (Andrew Chow)
3194a7f88ac1a32997b390b4f188c4b6a4af04a5 Introduce WalletDescriptor class (Andrew Chow)
6b13cd3fa854dfaeb9e269bff3d67cacc0e5b5dc Create LegacyScriptPubKeyMan when not a descriptor wallet (Andrew Chow)
aeac157c9dc141546b45e06ba9c2e641ad86083f Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet (Andrew Chow)
96accc73f067c7c95946e9932645dd821ef67f63 Add WALLET_FLAG_DESCRIPTORS (Andrew Chow)
6b8119af53ee2fdb4c4b5b24b4e650c0dc3bd27c Introduce DescriptorScriptPubKeyMan as a dummy class (Andrew Chow)
06620302c713cae65ee8e4ff9302e4c88e2a1285 Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it (Andrew Chow)
Pull request description:
Introducing the wallet of the glorious future (again): native descriptor wallets. With native descriptor wallets, addresses are generated from descriptors. Instead of generating keys and deriving addresses from keys, addresses come from the scriptPubKeys produced by a descriptor. Native descriptor wallets will be optional for now and can only be created by using `createwallet`.
Descriptor wallets will store descriptors, master keys from the descriptor, and descriptor cache entries. Keys are derived from descriptors on the fly. In order to allow choosing different address types, 6 descriptors are needed for normal use. There is a pair of primary and change descriptors for each of the 3 address types. With the default keypool size of 1000, each descriptor has 1000 scriptPubKeys and descriptor cache entries pregenerated. This has a side effect of making wallets large since 6000 pubkeys are written to the wallet by default, instead of the current 2000. scriptPubKeys are kept only in memory and are generated every time a descriptor is loaded. By default, we use the standard BIP 44, 49, 84 derivation paths with an external and internal derivation chain for each.
Descriptors can also be imported with a new `importdescriptors` RPC.
Native descriptor wallets use the `ScriptPubKeyMan` interface introduced in #16341 to add a `DescriptorScriptPubKeyMan`. This defines a different IsMine which uses the simpler model of "does this scriptPubKey exist in this wallet". Furthermore, `DescriptorScriptPubKeyMan` does not have watchonly, so with native descriptor wallets, it is not possible to have a wallet with both watchonly and non-watchonly things. Rather a wallet with `disable_private_keys` needs to be used for watchonly things.
A `--descriptor` option was added to some tests (`wallet_basic.py`, `wallet_encryption.py`, `wallet_keypool.py`, `wallet_keypool_topup.py`, and `wallet_labels.py`) to allow for these tests to use descriptor wallets. Additionally, several RPCs are disabled for descriptor wallets (`importprivkey`, `importpubkey`, `importaddress`, `importmulti`, `addmultisigaddress`, `dumpprivkey`, `dumpwallet`, `importwallet`, and `sethdseed`).
ACKs for top commit:
Sjors:
utACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82 (rebased, nits addressed)
jonatack:
Code review re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82.
fjahr:
re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
instagibbs:
light re-ACK 223588b
meshcollider:
Code review ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
Tree-SHA512: 59bc52aeddbb769ed5f420d5d240d8137847ac821b588eb616b34461253510c1717d6a70bab8765631738747336ae06f45ba39603ccd17f483843e5ed9a90986
Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it
Introduce DescriptorScriptPubKeyMan as a dummy class
Add WALLET_FLAG_DESCRIPTORS
Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet
Create LegacyScriptPubKeyMan when not a descriptor wallet
Introduce WalletDescriptor class
WalletDescriptor is a Descriptor with other wallet metadata
Add a lock cs_desc_man for DescriptorScriptPubKeyMan
Store WalletDescriptor in DescriptorScriptPubKeyMan
Implement SetType in DescriptorScriptPubKeyMan
Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet
Implement IsMine for DescriptorScriptPubKeyMan
Adds a set of scriptPubKeys that DescriptorScriptPubKeyMan tracks.
If the given script is in that set, it is considered ISMINE_SPENDABLE
Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan
Implement IsHDEnabled in DescriptorScriptPubKeyMan
Implement GetID for DescriptorScriptPubKeyMan
Load the descriptor cache from the wallet file
Implement loading of keys for DescriptorScriptPubKeyMan
Add IsSingleType to Descriptors
IsSingleType will return whether the descriptor will give one or multiple scriptPubKeys
Implement several simple functions in DescriptorScriptPubKeyMan
Implements a bunch of one liners: UpgradeKeyMetadata, IsFirstRun, HavePrivateKeys,
KeypoolCountExternalKeys, GetKeypoolSize, GetTimeFirstKey, CanGetAddresses,
RewriteDB
Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file
Implement SetupGeneration for DescriptorScriptPubKeyMan
Implement TopUp in DescriptorScriptPubKeyMan
Implement GetNewDestination for DescriptorScriptPubKeyMan
Implement Unlock and Encrypt in DescriptorScriptPubKeyMan
Implement GetReservedDestination in DescriptorScriptPubKeyMan
Implement ReturnDestination in DescriptorScriptPubKeyMan
Implement GetKeypoolOldestTime and only display it if greater than 0
Implement GetSolvingProvider for DescriptorScriptPubKeyMan
Internally, a GetSigningProvider function is introduced which allows for
some private keys to be optionally included. This can be called with a
script as the argument (i.e. a scriptPubKey from our wallet when we are
signing) or with a pubkey. In order to know what index to expand the
private keys for that pubkey, we need to also cache all of the pubkeys
involved when we expand the descriptor. So SetCache and TopUp are
updated to do this too.
Implement SignTransaction in DescriptorScriptPubKeyMan
Implement SignMessage for descriptor wallets
Implement FillPSBT in DescriptorScriptPubKeyMan
FillPSBT will add our own scripts to the PSBT if those inputs are ours.
If an input also lists pubkeys that we happen to know the private keys
for, we will sign those inputs too.
Change GetMetadata to use unique_ptr<CKeyMetadata>
Implement GetMetadata in DescriptorScriptPubKeyMan
Be able to create new wallets with DescriptorScriptPubKeyMans as backing
Generate new descriptors when encrypting
Add IsLegacy to CWallet so that the GUI knows whether to show watchonly
add importdescriptors RPC and tests for native descriptor wallets
Co-authored-by: Andrew Chow <achow101-github@achow101.com>
Functional tests for descriptor wallets
Change wallet_encryption.py to use signmessage instead of dumpprivkey
Return error when no ScriptPubKeyMan is available for specified type
When a CWallet doesn't have a ScriptPubKeyMan for the requested type
in GetNewDestination, give a meaningful error. Also handle this in
Qt which did not do anything with errors.
Implement CWallet::IsSpentKey for non-LegacySPKMans
tests: Add RPCOverloadWrapper which overloads some disabled RPCs
RPCOverloadWrapper overloads some deprecated or disabled RPCs with
an implementation using other RPCs to avoid having a ton of code churn
around replacing those RPCs.
Add a --descriptors option to various tests
Adds a --descriptors option globally to the test framework. This will
make the test create and use descriptor wallets. However some tests may
not work with this.
Some tests are modified to work with --descriptors and run with that
option in test_runer:
* wallet_basic.py
* wallet_encryption.py
* wallet_keypool.py <---- wallet_keypool_hd.py actually
* wallet_keypool_topup.py
* wallet_labels.py
* wallet_avoidreuse.py
2019-07-16 19:34:35 +02:00
self . start_node ( 3 , self . nodes [ 3 ] . extra_args )
2022-09-24 14:36:35 +02:00
self . connect_nodes ( 0 , 3 )
2020-06-25 19:25:46 +02:00
# Sendmany with explicit fee (DASH/kB)
# Throw if no conf_target provided
assert_raises_rpc_error ( - 8 , " Selected estimate_mode requires a fee rate " ,
self . nodes [ 2 ] . sendmany ,
amounts = { address : 10 } ,
estimate_mode = ' dash/kB ' )
# Throw if negative feerate
assert_raises_rpc_error ( - 3 , " Amount out of range " ,
self . nodes [ 2 ] . sendmany ,
amounts = { address : 10 } ,
conf_target = - 1 ,
estimate_mode = ' dash/kB ' )
fee_per_kb = 0.0002500
explicit_fee_per_byte = Decimal ( fee_per_kb ) / 1000
txid = self . nodes [ 2 ] . sendmany (
amounts = { address : 10 } ,
conf_target = fee_per_kb ,
estimate_mode = ' dash/kB ' ,
)
self . nodes [ 2 ] . generate ( 1 )
self . sync_all ( self . nodes [ 0 : 3 ] )
node_2_bal = self . check_fee_amount ( self . nodes [ 2 ] . getbalance ( ) , node_2_bal - Decimal ( ' 10 ' ) , explicit_fee_per_byte , count_bytes ( self . nodes [ 2 ] . gettransaction ( txid ) [ ' hex ' ] ) )
assert_equal ( self . nodes [ 2 ] . getbalance ( ) , node_2_bal )
node_0_bal + = Decimal ( ' 10 ' )
assert_equal ( self . nodes [ 0 ] . getbalance ( ) , node_0_bal )
# Sendmany with explicit fee (DUFF/B)
# Throw if no conf_target provided
assert_raises_rpc_error ( - 8 , " Selected estimate_mode requires a fee rate " ,
self . nodes [ 2 ] . sendmany ,
amounts = { address : 10 } ,
estimate_mode = ' duff/b ' )
# Throw if negative feerate
assert_raises_rpc_error ( - 3 , " Amount out of range " ,
self . nodes [ 2 ] . sendmany ,
amounts = { address : 10 } ,
conf_target = - 1 ,
estimate_mode = ' duff/b ' )
fee_duff_per_b = 2
fee_per_kb = fee_duff_per_b / 100000.0
explicit_fee_per_byte = Decimal ( fee_per_kb ) / 1000
txid = self . nodes [ 2 ] . sendmany (
amounts = { address : 10 } ,
conf_target = fee_duff_per_b ,
estimate_mode = ' duff/b ' ,
)
self . nodes [ 2 ] . generate ( 1 )
self . sync_all ( self . nodes [ 0 : 3 ] )
balance = self . nodes [ 2 ] . getbalance ( )
node_2_bal = self . check_fee_amount ( balance , node_2_bal - Decimal ( ' 10 ' ) , explicit_fee_per_byte , count_bytes ( self . nodes [ 2 ] . gettransaction ( txid ) [ ' hex ' ] ) )
assert_equal ( balance , node_2_bal )
node_0_bal + = Decimal ( ' 10 ' )
assert_equal ( self . nodes [ 0 ] . getbalance ( ) , node_0_bal )
2015-07-14 21:13:16 +02:00
2021-04-12 16:58:49 +02:00
# check if we can list zero value tx as available coins
# 1. create raw_tx
# 2. hex-changed one output to 0.0
# 3. sign and send
# 4. check if recipient (node0) can list the zero value tx
2020-07-07 19:37:53 +02:00
usp = self . nodes [ 1 ] . listunspent ( query_options = { ' minimumAmount ' : ' 499.998 ' } ) [ 0 ]
2021-04-12 16:58:49 +02:00
inputs = [ { " txid " : usp [ ' txid ' ] , " vout " : usp [ ' vout ' ] } ]
2016-03-06 16:14:39 +01:00
outputs = { self . nodes [ 1 ] . getnewaddress ( ) : 499.998 , self . nodes [ 0 ] . getnewaddress ( ) : 11.11 }
2015-07-14 21:13:16 +02:00
2021-04-12 16:58:49 +02:00
raw_tx = self . nodes [ 1 ] . createrawtransaction ( inputs , outputs ) . replace ( " c0833842 " , " 00000000 " ) # replace 11.11 with 0.0 (int32)
signed_raw_tx = self . nodes [ 1 ] . signrawtransactionwithwallet ( raw_tx )
decoded_raw_tx = self . nodes [ 1 ] . decoderawtransaction ( signed_raw_tx [ ' hex ' ] )
zero_value_txid = decoded_raw_tx [ ' txid ' ]
self . nodes [ 1 ] . sendrawtransaction ( signed_raw_tx [ ' hex ' ] )
2015-07-14 21:13:16 +02:00
2015-04-21 10:27:35 +02:00
self . sync_all ( )
2021-04-12 16:58:49 +02:00
self . nodes [ 1 ] . generate ( 1 ) # mine a block
2015-04-21 10:27:35 +02:00
self . sync_all ( )
2015-07-14 21:13:16 +02:00
2021-04-12 16:58:49 +02:00
unspent_txs = self . nodes [ 0 ] . listunspent ( ) # zero value tx must be in listunspents output
2015-04-21 10:27:35 +02:00
found = False
2021-04-12 16:58:49 +02:00
for uTx in unspent_txs :
if uTx [ ' txid ' ] == zero_value_txid :
2015-04-21 10:27:35 +02:00
found = True
2015-12-15 17:15:13 +01:00
assert_equal ( uTx [ ' amount ' ] , Decimal ( ' 0 ' ) )
2019-02-01 15:35:08 +01:00
assert found
2015-07-14 21:13:16 +02:00
2021-04-12 16:58:49 +02:00
# do some -walletbroadcast tests
2017-06-02 12:08:48 +02:00
self . stop_nodes ( )
2017-09-01 18:47:13 +02:00
self . start_node ( 0 , [ " -walletbroadcast=0 " ] )
self . start_node ( 1 , [ " -walletbroadcast=0 " ] )
self . start_node ( 2 , [ " -walletbroadcast=0 " ] )
2022-09-24 14:36:35 +02:00
self . connect_nodes ( 0 , 1 )
self . connect_nodes ( 1 , 2 )
self . connect_nodes ( 0 , 2 )
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2015-03-23 18:47:18 +01:00
2021-04-12 16:58:49 +02:00
txid_not_broadcast = self . nodes [ 0 ] . sendtoaddress ( self . nodes [ 2 ] . getnewaddress ( ) , 2 )
tx_obj_not_broadcast = self . nodes [ 0 ] . gettransaction ( txid_not_broadcast )
self . nodes [ 1 ] . generate ( 1 ) # mine a block, tx should not be in there
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2021-04-12 16:58:49 +02:00
assert_equal ( self . nodes [ 2 ] . getbalance ( ) , node_2_bal ) # should not be changed because tx was not broadcasted
2015-07-14 21:13:16 +02:00
2021-04-12 16:58:49 +02:00
# now broadcast from another node, mine a block, sync, and check the balance
self . nodes [ 1 ] . sendrawtransaction ( tx_obj_not_broadcast [ ' hex ' ] )
2015-04-09 17:36:10 +02:00
self . nodes [ 1 ] . generate ( 1 )
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2015-12-15 17:15:13 +01:00
node_2_bal + = 2
2021-04-12 16:58:49 +02:00
tx_obj_not_broadcast = self . nodes [ 0 ] . gettransaction ( txid_not_broadcast )
2015-12-15 17:15:13 +01:00
assert_equal ( self . nodes [ 2 ] . getbalance ( ) , node_2_bal )
2015-07-14 21:13:16 +02:00
2021-04-12 16:58:49 +02:00
# create another tx
self . nodes [ 0 ] . sendtoaddress ( self . nodes [ 2 ] . getnewaddress ( ) , 2 )
2015-07-14 21:13:16 +02:00
2021-04-12 16:58:49 +02:00
# restart the nodes with -walletbroadcast=1
2017-06-02 12:08:48 +02:00
self . stop_nodes ( )
2017-09-01 18:47:13 +02:00
self . start_node ( 0 )
self . start_node ( 1 )
self . start_node ( 2 )
2022-09-24 14:36:35 +02:00
self . connect_nodes ( 0 , 1 )
self . connect_nodes ( 1 , 2 )
self . connect_nodes ( 0 , 2 )
2020-04-14 12:00:16 +02:00
self . sync_blocks ( self . nodes [ 0 : 3 ] )
2015-07-14 21:13:16 +02:00
2015-04-09 17:36:10 +02:00
self . nodes [ 0 ] . generate ( 1 )
2020-04-14 12:00:16 +02:00
self . sync_blocks ( self . nodes [ 0 : 3 ] )
2015-12-15 17:15:13 +01:00
node_2_bal + = 2
2015-07-14 21:13:16 +02:00
2021-04-12 16:58:49 +02:00
# tx should be added to balance because after restarting the nodes tx should be broadcast
2015-12-15 17:15:13 +01:00
assert_equal ( self . nodes [ 2 ] . getbalance ( ) , node_2_bal )
2015-07-14 21:13:16 +02:00
2021-04-12 16:58:49 +02:00
# send a tx with value in a string (PR#6380 +)
txid = self . nodes [ 0 ] . sendtoaddress ( self . nodes [ 2 ] . getnewaddress ( ) , " 2 " )
tx_obj = self . nodes [ 0 ] . gettransaction ( txid )
assert_equal ( tx_obj [ ' amount ' ] , Decimal ( ' -2 ' ) )
2015-07-14 21:13:16 +02:00
2021-04-12 16:58:49 +02:00
txid = self . nodes [ 0 ] . sendtoaddress ( self . nodes [ 2 ] . getnewaddress ( ) , " 0.0001 " )
tx_obj = self . nodes [ 0 ] . gettransaction ( txid )
assert_equal ( tx_obj [ ' amount ' ] , Decimal ( ' -0.0001 ' ) )
2015-07-14 21:13:16 +02:00
2021-04-12 16:58:49 +02:00
# check if JSON parser can handle scientific notation in strings
txid = self . nodes [ 0 ] . sendtoaddress ( self . nodes [ 2 ] . getnewaddress ( ) , " 1e-4 " )
tx_obj = self . nodes [ 0 ] . gettransaction ( txid )
assert_equal ( tx_obj [ ' amount ' ] , Decimal ( ' -0.0001 ' ) )
2015-07-14 21:13:16 +02:00
2019-02-12 21:27:13 +01:00
# General checks for errors from incorrect inputs
2017-03-16 11:57:09 +01:00
# This will raise an exception because the amount type is wrong
2019-09-25 11:34:51 +02:00
assert_raises_rpc_error ( - 3 , " Invalid amount " , self . nodes [ 0 ] . sendtoaddress , self . nodes [ 2 ] . getnewaddress ( ) , " 1f-4 " )
2015-07-14 21:13:16 +02:00
2017-03-16 11:57:09 +01:00
# This will raise an exception since generate does not accept a string
2019-09-25 11:34:51 +02:00
assert_raises_rpc_error ( - 1 , " not an integer " , self . nodes [ 0 ] . generate , " 2 " )
2015-07-14 21:13:16 +02:00
Merge #16528: Native Descriptor Wallets using DescriptorScriptPubKeyMan
223588b1bbc63dc57098bbd0baa48635e0cc0b82 Add a --descriptors option to various tests (Andrew Chow)
869f7ab30aeb4d7fbd563c535b55467a8a0430cf tests: Add RPCOverloadWrapper which overloads some disabled RPCs (Andrew Chow)
cf060628590fab87d73f278e744d70ef2d5d81db Correctly check for default wallet (Andrew Chow)
886e0d75f5fea2421190aa4812777d89f68962cc Implement CWallet::IsSpentKey for non-LegacySPKMans (Andrew Chow)
3c19fdd2a2fd5394fcfa75b2ba84ab2277cbdabf Return error when no ScriptPubKeyMan is available for specified type (Andrew Chow)
388ba94231f2f10a0be751c562cdd4650510a90a Change wallet_encryption.py to use signmessage instead of dumpprivkey (Andrew Chow)
1346e14831489f9c8f53a08f9dfed61d55d53c6f Functional tests for descriptor wallets (Andrew Chow)
f193ea889ddb53d9a5c47647966681d525e38368 add importdescriptors RPC and tests for native descriptor wallets (Hugo Nguyen)
ce24a944940019185efebcc5d85eac458ed26016 Add IsLegacy to CWallet so that the GUI knows whether to show watchonly (Andrew Chow)
1cb42b22b11c27e64462afc25a94b2fc50bfa113 Generate new descriptors when encrypting (Andrew Chow)
82ae02b1656819f4bd5023b8955447e1d4ea8692 Be able to create new wallets with DescriptorScriptPubKeyMans as backing (Andrew Chow)
b713baa75a62335ab9c0eed9ef76a95bfec30668 Implement GetMetadata in DescriptorScriptPubKeyMan (Andrew Chow)
8b9603bd0b443e2f7984eb72bf2e21cf02af0bcb Change GetMetadata to use unique_ptr<CKeyMetadata> (Andrew Chow)
72a9540df96ffdb94f039b9c14eaacdc7d961196 Implement FillPSBT in DescriptorScriptPubKeyMan (Andrew Chow)
84b4978c02102171775c77a45f6ec198930f0a88 Implement SignMessage for descriptor wallets (Andrew Chow)
bde7c9fa38775a81d53ac0484fa9c98076a0c7d1 Implement SignTransaction in DescriptorScriptPubKeyMan (Andrew Chow)
d50c8ddd4190f20bf0debd410348b73408ec3143 Implement GetSolvingProvider for DescriptorScriptPubKeyMan (Andrew Chow)
f1ca5feb4ad668a3e1ae543d0addd5f483f1a88f Implement GetKeypoolOldestTime and only display it if greater than 0 (Andrew Chow)
586b57a9a6b4b12a78f792785b63a5a1743bce0c Implement ReturnDestination in DescriptorScriptPubKeyMan (Andrew Chow)
f866957979c23cefd41efa9dae9e53b9177818dc Implement GetReservedDestination in DescriptorScriptPubKeyMan (Andrew Chow)
a775f7c7fd0b9094fcbeee6ba92206d5bbb19164 Implement Unlock and Encrypt in DescriptorScriptPubKeyMan (Andrew Chow)
bfdd0734869a22217c15858d7a76d0dacc2ebc86 Implement GetNewDestination for DescriptorScriptPubKeyMan (Andrew Chow)
58c7651821b0eeff0a99dc61d78d2e9e07986580 Implement TopUp in DescriptorScriptPubKeyMan (Andrew Chow)
e014886a342508f7c8d80323eee9a5f314eaf94c Implement SetupGeneration for DescriptorScriptPubKeyMan (Andrew Chow)
46dfb99768e7d03a3cf552812d5b41ceaebc06be Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file (Andrew Chow)
4cb9b69be031e1dc65d8964794781b347fd948f5 Implement several simple functions in DescriptorScriptPubKeyMan (Andrew Chow)
d1ec3e4f19487b4b100f80ad02eac063c571777d Add IsSingleType to Descriptors (Andrew Chow)
953feb3d2724f5398dd48990c4957a19313d2c8c Implement loading of keys for DescriptorScriptPubKeyMan (Andrew Chow)
2363e9fcaa41b68bf11153f591b95f2d41ff9a1a Load the descriptor cache from the wallet file (Andrew Chow)
46c46aebb7943e1e2e96755e94dc6c197920bf75 Implement GetID for DescriptorScriptPubKeyMan (Andrew Chow)
ec2f9e1178c8e38c0a5ca063fe81adac8f916348 Implement IsHDEnabled in DescriptorScriptPubKeyMan (Andrew Chow)
741122d4c1a62ced3e96d16d67f4eeb3a6522d99 Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan (Andrew Chow)
2db7ca765c8fb2c71dd6f7c4f29ad70e68ff1720 Implement IsMine for DescriptorScriptPubKeyMan (Andrew Chow)
db7177af8c159abbcc209f2caafcd45d54c181c5 Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet (Andrew Chow)
78f8a92910d34247fa5d04368338c598d9908267 Implement SetType in DescriptorScriptPubKeyMan (Andrew Chow)
834de0300cde57ca3f662fb7aa5b1bdaed68bc8f Store WalletDescriptor in DescriptorScriptPubKeyMan (Andrew Chow)
d8132669e10c1db9ae0c2ea0d3f822d7d2f01345 Add a lock cs_desc_man for DescriptorScriptPubKeyMan (Andrew Chow)
3194a7f88ac1a32997b390b4f188c4b6a4af04a5 Introduce WalletDescriptor class (Andrew Chow)
6b13cd3fa854dfaeb9e269bff3d67cacc0e5b5dc Create LegacyScriptPubKeyMan when not a descriptor wallet (Andrew Chow)
aeac157c9dc141546b45e06ba9c2e641ad86083f Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet (Andrew Chow)
96accc73f067c7c95946e9932645dd821ef67f63 Add WALLET_FLAG_DESCRIPTORS (Andrew Chow)
6b8119af53ee2fdb4c4b5b24b4e650c0dc3bd27c Introduce DescriptorScriptPubKeyMan as a dummy class (Andrew Chow)
06620302c713cae65ee8e4ff9302e4c88e2a1285 Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it (Andrew Chow)
Pull request description:
Introducing the wallet of the glorious future (again): native descriptor wallets. With native descriptor wallets, addresses are generated from descriptors. Instead of generating keys and deriving addresses from keys, addresses come from the scriptPubKeys produced by a descriptor. Native descriptor wallets will be optional for now and can only be created by using `createwallet`.
Descriptor wallets will store descriptors, master keys from the descriptor, and descriptor cache entries. Keys are derived from descriptors on the fly. In order to allow choosing different address types, 6 descriptors are needed for normal use. There is a pair of primary and change descriptors for each of the 3 address types. With the default keypool size of 1000, each descriptor has 1000 scriptPubKeys and descriptor cache entries pregenerated. This has a side effect of making wallets large since 6000 pubkeys are written to the wallet by default, instead of the current 2000. scriptPubKeys are kept only in memory and are generated every time a descriptor is loaded. By default, we use the standard BIP 44, 49, 84 derivation paths with an external and internal derivation chain for each.
Descriptors can also be imported with a new `importdescriptors` RPC.
Native descriptor wallets use the `ScriptPubKeyMan` interface introduced in #16341 to add a `DescriptorScriptPubKeyMan`. This defines a different IsMine which uses the simpler model of "does this scriptPubKey exist in this wallet". Furthermore, `DescriptorScriptPubKeyMan` does not have watchonly, so with native descriptor wallets, it is not possible to have a wallet with both watchonly and non-watchonly things. Rather a wallet with `disable_private_keys` needs to be used for watchonly things.
A `--descriptor` option was added to some tests (`wallet_basic.py`, `wallet_encryption.py`, `wallet_keypool.py`, `wallet_keypool_topup.py`, and `wallet_labels.py`) to allow for these tests to use descriptor wallets. Additionally, several RPCs are disabled for descriptor wallets (`importprivkey`, `importpubkey`, `importaddress`, `importmulti`, `addmultisigaddress`, `dumpprivkey`, `dumpwallet`, `importwallet`, and `sethdseed`).
ACKs for top commit:
Sjors:
utACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82 (rebased, nits addressed)
jonatack:
Code review re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82.
fjahr:
re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
instagibbs:
light re-ACK 223588b
meshcollider:
Code review ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
Tree-SHA512: 59bc52aeddbb769ed5f420d5d240d8137847ac821b588eb616b34461253510c1717d6a70bab8765631738747336ae06f45ba39603ccd17f483843e5ed9a90986
Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it
Introduce DescriptorScriptPubKeyMan as a dummy class
Add WALLET_FLAG_DESCRIPTORS
Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet
Create LegacyScriptPubKeyMan when not a descriptor wallet
Introduce WalletDescriptor class
WalletDescriptor is a Descriptor with other wallet metadata
Add a lock cs_desc_man for DescriptorScriptPubKeyMan
Store WalletDescriptor in DescriptorScriptPubKeyMan
Implement SetType in DescriptorScriptPubKeyMan
Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet
Implement IsMine for DescriptorScriptPubKeyMan
Adds a set of scriptPubKeys that DescriptorScriptPubKeyMan tracks.
If the given script is in that set, it is considered ISMINE_SPENDABLE
Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan
Implement IsHDEnabled in DescriptorScriptPubKeyMan
Implement GetID for DescriptorScriptPubKeyMan
Load the descriptor cache from the wallet file
Implement loading of keys for DescriptorScriptPubKeyMan
Add IsSingleType to Descriptors
IsSingleType will return whether the descriptor will give one or multiple scriptPubKeys
Implement several simple functions in DescriptorScriptPubKeyMan
Implements a bunch of one liners: UpgradeKeyMetadata, IsFirstRun, HavePrivateKeys,
KeypoolCountExternalKeys, GetKeypoolSize, GetTimeFirstKey, CanGetAddresses,
RewriteDB
Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file
Implement SetupGeneration for DescriptorScriptPubKeyMan
Implement TopUp in DescriptorScriptPubKeyMan
Implement GetNewDestination for DescriptorScriptPubKeyMan
Implement Unlock and Encrypt in DescriptorScriptPubKeyMan
Implement GetReservedDestination in DescriptorScriptPubKeyMan
Implement ReturnDestination in DescriptorScriptPubKeyMan
Implement GetKeypoolOldestTime and only display it if greater than 0
Implement GetSolvingProvider for DescriptorScriptPubKeyMan
Internally, a GetSigningProvider function is introduced which allows for
some private keys to be optionally included. This can be called with a
script as the argument (i.e. a scriptPubKey from our wallet when we are
signing) or with a pubkey. In order to know what index to expand the
private keys for that pubkey, we need to also cache all of the pubkeys
involved when we expand the descriptor. So SetCache and TopUp are
updated to do this too.
Implement SignTransaction in DescriptorScriptPubKeyMan
Implement SignMessage for descriptor wallets
Implement FillPSBT in DescriptorScriptPubKeyMan
FillPSBT will add our own scripts to the PSBT if those inputs are ours.
If an input also lists pubkeys that we happen to know the private keys
for, we will sign those inputs too.
Change GetMetadata to use unique_ptr<CKeyMetadata>
Implement GetMetadata in DescriptorScriptPubKeyMan
Be able to create new wallets with DescriptorScriptPubKeyMans as backing
Generate new descriptors when encrypting
Add IsLegacy to CWallet so that the GUI knows whether to show watchonly
add importdescriptors RPC and tests for native descriptor wallets
Co-authored-by: Andrew Chow <achow101-github@achow101.com>
Functional tests for descriptor wallets
Change wallet_encryption.py to use signmessage instead of dumpprivkey
Return error when no ScriptPubKeyMan is available for specified type
When a CWallet doesn't have a ScriptPubKeyMan for the requested type
in GetNewDestination, give a meaningful error. Also handle this in
Qt which did not do anything with errors.
Implement CWallet::IsSpentKey for non-LegacySPKMans
tests: Add RPCOverloadWrapper which overloads some disabled RPCs
RPCOverloadWrapper overloads some deprecated or disabled RPCs with
an implementation using other RPCs to avoid having a ton of code churn
around replacing those RPCs.
Add a --descriptors option to various tests
Adds a --descriptors option globally to the test framework. This will
make the test create and use descriptor wallets. However some tests may
not work with this.
Some tests are modified to work with --descriptors and run with that
option in test_runer:
* wallet_basic.py
* wallet_encryption.py
* wallet_keypool.py <---- wallet_keypool_hd.py actually
* wallet_keypool_topup.py
* wallet_labels.py
* wallet_avoidreuse.py
2019-07-16 19:34:35 +02:00
if not self . options . descriptors :
# This will raise an exception for the invalid private key format
assert_raises_rpc_error ( - 5 , " Invalid private key encoding " , self . nodes [ 0 ] . importprivkey , " invalid " )
# This will raise an exception for importing an address with the PS2H flag
temp_address = self . nodes [ 1 ] . getnewaddress ( )
assert_raises_rpc_error ( - 5 , " Cannot use the p2sh flag with an address - use a script instead " , self . nodes [ 0 ] . importaddress , temp_address , " label " , False , True )
# This will raise an exception for attempting to dump the private key of an address you do not own
assert_raises_rpc_error ( - 4 , " Private key for address %s is not known " % temp_address , self . nodes [ 0 ] . dumpprivkey , temp_address )
# This will raise an exception for attempting to get the private key of an invalid Dash address
assert_raises_rpc_error ( - 5 , " Invalid Dash address " , self . nodes [ 0 ] . dumpprivkey , " invalid " )
# This will raise an exception for attempting to set a label for an invalid Dash address
assert_raises_rpc_error ( - 5 , " Invalid Dash address " , self . nodes [ 0 ] . setlabel , " invalid address " , " label " )
# This will raise an exception for importing an invalid address
assert_raises_rpc_error ( - 5 , " Invalid Dash address or script " , self . nodes [ 0 ] . importaddress , " invalid " )
# This will raise an exception for attempting to import a pubkey that isn't in hex
assert_raises_rpc_error ( - 5 , " Pubkey must be a hex string " , self . nodes [ 0 ] . importpubkey , " not hex " )
# This will raise an exception for importing an invalid pubkey
assert_raises_rpc_error ( - 5 , " Pubkey is not a valid public key " , self . nodes [ 0 ] . importpubkey , " 5361746f736869204e616b616d6f746f " )
# Import address and private key to check correct behavior of spendable unspents
# 1. Send some coins to generate new UTXO
address_to_import = self . nodes [ 2 ] . getnewaddress ( )
txid = self . nodes [ 0 ] . sendtoaddress ( address_to_import , 1 )
self . nodes [ 0 ] . generate ( 1 )
self . sync_all ( self . nodes [ 0 : 3 ] )
# send with explicit dash/kb fee
self . log . info ( " test explicit fee (sendtoaddress as dash/kb) " )
self . nodes [ 0 ] . generate ( 1 )
self . sync_all ( self . nodes [ 0 : 3 ] )
prebalance = self . nodes [ 2 ] . getbalance ( )
assert prebalance > 2
address = self . nodes [ 1 ] . getnewaddress ( )
# Throw if no conf_target provided
assert_raises_rpc_error ( - 8 , " Selected estimate_mode requires a fee rate " ,
self . nodes [ 2 ] . sendtoaddress ,
address = address ,
amount = 1.0 ,
estimate_mode = ' dash/Kb ' )
# Throw if negative feerate
assert_raises_rpc_error ( - 3 , " Amount out of range " ,
self . nodes [ 2 ] . sendtoaddress ,
address = address ,
amount = 1.0 ,
conf_target = - 1 ,
estimate_mode = ' dash/kb ' )
txid = self . nodes [ 2 ] . sendtoaddress (
address = address ,
amount = 1.0 ,
conf_target = 0.00002500 ,
estimate_mode = ' dash/kb ' ,
)
tx_size = count_bytes ( self . nodes [ 2 ] . gettransaction ( txid ) [ ' hex ' ] )
self . sync_all ( self . nodes [ 0 : 3 ] )
self . nodes [ 0 ] . generate ( 1 )
self . sync_all ( self . nodes [ 0 : 3 ] )
postbalance = self . nodes [ 2 ] . getbalance ( )
fee = prebalance - postbalance - Decimal ( ' 1 ' )
assert_fee_amount ( fee , tx_size , Decimal ( ' 0.00002500 ' ) )
# send with explicit duff/b fee
self . sync_all ( self . nodes [ 0 : 3 ] )
self . log . info ( " test explicit fee (sendtoaddress as duff/b) " )
self . nodes [ 0 ] . generate ( 1 )
prebalance = self . nodes [ 2 ] . getbalance ( )
assert prebalance > 2
address = self . nodes [ 1 ] . getnewaddress ( )
# Throw if no conf_target provided
assert_raises_rpc_error ( - 8 , " Selected estimate_mode requires a fee rate " ,
self . nodes [ 2 ] . sendtoaddress ,
address = address ,
amount = 1.0 ,
estimate_mode = ' duff/b ' )
# Throw if negative feerate
assert_raises_rpc_error ( - 3 , " Amount out of range " ,
self . nodes [ 2 ] . sendtoaddress ,
address = address ,
amount = 1.0 ,
conf_target = - 1 ,
estimate_mode = ' duff/b ' )
txid = self . nodes [ 2 ] . sendtoaddress (
address = address ,
amount = 1.0 ,
conf_target = 2 ,
estimate_mode = ' duff/B ' ,
)
tx_size = count_bytes ( self . nodes [ 2 ] . gettransaction ( txid ) [ ' hex ' ] )
self . sync_all ( self . nodes [ 0 : 3 ] )
self . nodes [ 0 ] . generate ( 1 )
self . sync_all ( self . nodes [ 0 : 3 ] )
postbalance = self . nodes [ 2 ] . getbalance ( )
fee = prebalance - postbalance - Decimal ( ' 1 ' )
assert_fee_amount ( fee , tx_size , Decimal ( ' 0.00002000 ' ) )
# 2. Import address from node2 to node1
self . nodes [ 1 ] . importaddress ( address_to_import )
# 3. Validate that the imported address is watch-only on node1
assert self . nodes [ 1 ] . getaddressinfo ( address_to_import ) [ " iswatchonly " ]
# 4. Check that the unspents after import are not spendable
assert_array_result ( self . nodes [ 1 ] . listunspent ( ) ,
{ " address " : address_to_import } ,
{ " spendable " : False } )
# 5. Import private key of the previously imported address on node1
priv_key = self . nodes [ 2 ] . dumpprivkey ( address_to_import )
self . nodes [ 1 ] . importprivkey ( priv_key )
# 6. Check that the unspents are now spendable on node1
assert_array_result ( self . nodes [ 1 ] . listunspent ( ) ,
{ " address " : address_to_import } ,
{ " spendable " : True } )
2015-12-15 17:15:13 +01:00
2016-03-23 13:24:34 +01:00
# Mine a block from node0 to an address from node1
2021-04-12 16:58:49 +02:00
coinbase_addr = self . nodes [ 1 ] . getnewaddress ( )
block_hash = self . nodes [ 0 ] . generatetoaddress ( 1 , coinbase_addr ) [ 0 ]
coinbase_txid = self . nodes [ 0 ] . getblock ( block_hash ) [ ' tx ' ] [ 0 ]
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2016-03-23 13:24:34 +01:00
# Check that the txid and balance is found by node1
2021-04-12 16:58:49 +02:00
self . nodes [ 1 ] . gettransaction ( coinbase_txid )
2016-03-23 13:24:34 +01:00
2016-05-17 09:26:42 +02:00
# check if wallet or blockchain maintenance changes the balance
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2016-03-06 18:30:51 +01:00
blocks = self . nodes [ 0 ] . generate ( 2 )
2020-04-14 12:00:16 +02:00
self . sync_all ( self . nodes [ 0 : 3 ] )
2015-12-15 17:15:13 +01:00
balance_nodes = [ self . nodes [ i ] . getbalance ( ) for i in range ( 3 ) ]
2016-03-28 21:47:13 +02:00
block_count = self . nodes [ 0 ] . getblockcount ( )
2015-12-15 17:15:13 +01:00
2016-04-28 15:18:01 +02:00
# Check modes:
# - True: unicode escaped as \u....
# - False: unicode directly as UTF-8
for mode in [ True , False ] :
2018-05-11 00:08:57 +02:00
self . nodes [ 0 ] . rpc . ensure_ascii = mode
2016-04-28 15:18:01 +02:00
# unicode check: Basic Multilingual Plane, Supplementary Plane respectively
2018-05-11 00:08:57 +02:00
for label in [ u ' рыба ' , u ' 𝅘𝅥𝅯 ' ] :
addr = self . nodes [ 0 ] . getnewaddress ( )
self . nodes [ 0 ] . setlabel ( addr , label )
2020-02-02 09:35:24 +01:00
test_address ( self . nodes [ 0 ] , addr , labels = [ label ] )
2019-02-01 15:35:08 +01:00
assert label in self . nodes [ 0 ] . listlabels ( )
2018-05-11 00:08:57 +02:00
self . nodes [ 0 ] . rpc . ensure_ascii = True # restore to default
2016-04-28 15:18:01 +02:00
# maintenance tests
2015-12-15 17:15:13 +01:00
maintenance = [
' -rescan ' ,
' -reindex ' ,
]
2016-12-20 13:12:46 +01:00
chainlimit = 6
2015-12-15 17:15:13 +01:00
for m in maintenance :
2017-03-09 21:16:20 +01:00
self . log . info ( " check " + m )
2017-06-02 12:08:48 +02:00
self . stop_nodes ( )
2016-12-20 13:12:46 +01:00
# set lower ancestor limit for later
2018-05-11 00:08:57 +02:00
self . start_node ( 0 , [ m , " -limitancestorcount= " + str ( chainlimit ) ] )
self . start_node ( 1 , [ m , " -limitancestorcount= " + str ( chainlimit ) ] )
self . start_node ( 2 , [ m , " -limitancestorcount= " + str ( chainlimit ) ] )
2018-03-19 17:10:29 +01:00
if m == ' -reindex ' :
2016-03-28 21:47:13 +02:00
# reindex will leave rpc warm up "early"; Wait for it to finish
2020-08-15 15:38:51 +02:00
self . wait_until ( lambda : [ block_count ] * 3 == [ self . nodes [ i ] . getblockcount ( ) for i in range ( 3 ) ] )
2015-12-15 17:15:13 +01:00
assert_equal ( balance_nodes , [ self . nodes [ i ] . getbalance ( ) for i in range ( 3 ) ] )
2015-07-14 21:13:16 +02:00
2016-03-06 18:30:51 +01:00
# Exercise listsinceblock with the last two blocks
coinbase_tx_1 = self . nodes [ 0 ] . listsinceblock ( blocks [ 0 ] )
assert_equal ( coinbase_tx_1 [ " lastblock " ] , blocks [ 1 ] )
assert_equal ( len ( coinbase_tx_1 [ " transactions " ] ) , 1 )
assert_equal ( coinbase_tx_1 [ " transactions " ] [ 0 ] [ " blockhash " ] , blocks [ 1 ] )
assert_equal ( len ( self . nodes [ 0 ] . listsinceblock ( blocks [ 1 ] ) [ " transactions " ] ) , 0 )
2015-07-14 21:13:16 +02:00
2016-12-20 13:12:46 +01:00
# ==Check that wallet prefers to use coins that don't exceed mempool limits =====
# Get all non-zero utxos together
chain_addrs = [ self . nodes [ 0 ] . getnewaddress ( ) , self . nodes [ 0 ] . getnewaddress ( ) ]
singletxid = self . nodes [ 0 ] . sendtoaddress ( chain_addrs [ 0 ] , self . nodes [ 0 ] . getbalance ( ) , " " , " " , True )
self . nodes [ 0 ] . generate ( 1 )
node0_balance = self . nodes [ 0 ] . getbalance ( )
# Split into two chains
Merge #16528: Native Descriptor Wallets using DescriptorScriptPubKeyMan
223588b1bbc63dc57098bbd0baa48635e0cc0b82 Add a --descriptors option to various tests (Andrew Chow)
869f7ab30aeb4d7fbd563c535b55467a8a0430cf tests: Add RPCOverloadWrapper which overloads some disabled RPCs (Andrew Chow)
cf060628590fab87d73f278e744d70ef2d5d81db Correctly check for default wallet (Andrew Chow)
886e0d75f5fea2421190aa4812777d89f68962cc Implement CWallet::IsSpentKey for non-LegacySPKMans (Andrew Chow)
3c19fdd2a2fd5394fcfa75b2ba84ab2277cbdabf Return error when no ScriptPubKeyMan is available for specified type (Andrew Chow)
388ba94231f2f10a0be751c562cdd4650510a90a Change wallet_encryption.py to use signmessage instead of dumpprivkey (Andrew Chow)
1346e14831489f9c8f53a08f9dfed61d55d53c6f Functional tests for descriptor wallets (Andrew Chow)
f193ea889ddb53d9a5c47647966681d525e38368 add importdescriptors RPC and tests for native descriptor wallets (Hugo Nguyen)
ce24a944940019185efebcc5d85eac458ed26016 Add IsLegacy to CWallet so that the GUI knows whether to show watchonly (Andrew Chow)
1cb42b22b11c27e64462afc25a94b2fc50bfa113 Generate new descriptors when encrypting (Andrew Chow)
82ae02b1656819f4bd5023b8955447e1d4ea8692 Be able to create new wallets with DescriptorScriptPubKeyMans as backing (Andrew Chow)
b713baa75a62335ab9c0eed9ef76a95bfec30668 Implement GetMetadata in DescriptorScriptPubKeyMan (Andrew Chow)
8b9603bd0b443e2f7984eb72bf2e21cf02af0bcb Change GetMetadata to use unique_ptr<CKeyMetadata> (Andrew Chow)
72a9540df96ffdb94f039b9c14eaacdc7d961196 Implement FillPSBT in DescriptorScriptPubKeyMan (Andrew Chow)
84b4978c02102171775c77a45f6ec198930f0a88 Implement SignMessage for descriptor wallets (Andrew Chow)
bde7c9fa38775a81d53ac0484fa9c98076a0c7d1 Implement SignTransaction in DescriptorScriptPubKeyMan (Andrew Chow)
d50c8ddd4190f20bf0debd410348b73408ec3143 Implement GetSolvingProvider for DescriptorScriptPubKeyMan (Andrew Chow)
f1ca5feb4ad668a3e1ae543d0addd5f483f1a88f Implement GetKeypoolOldestTime and only display it if greater than 0 (Andrew Chow)
586b57a9a6b4b12a78f792785b63a5a1743bce0c Implement ReturnDestination in DescriptorScriptPubKeyMan (Andrew Chow)
f866957979c23cefd41efa9dae9e53b9177818dc Implement GetReservedDestination in DescriptorScriptPubKeyMan (Andrew Chow)
a775f7c7fd0b9094fcbeee6ba92206d5bbb19164 Implement Unlock and Encrypt in DescriptorScriptPubKeyMan (Andrew Chow)
bfdd0734869a22217c15858d7a76d0dacc2ebc86 Implement GetNewDestination for DescriptorScriptPubKeyMan (Andrew Chow)
58c7651821b0eeff0a99dc61d78d2e9e07986580 Implement TopUp in DescriptorScriptPubKeyMan (Andrew Chow)
e014886a342508f7c8d80323eee9a5f314eaf94c Implement SetupGeneration for DescriptorScriptPubKeyMan (Andrew Chow)
46dfb99768e7d03a3cf552812d5b41ceaebc06be Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file (Andrew Chow)
4cb9b69be031e1dc65d8964794781b347fd948f5 Implement several simple functions in DescriptorScriptPubKeyMan (Andrew Chow)
d1ec3e4f19487b4b100f80ad02eac063c571777d Add IsSingleType to Descriptors (Andrew Chow)
953feb3d2724f5398dd48990c4957a19313d2c8c Implement loading of keys for DescriptorScriptPubKeyMan (Andrew Chow)
2363e9fcaa41b68bf11153f591b95f2d41ff9a1a Load the descriptor cache from the wallet file (Andrew Chow)
46c46aebb7943e1e2e96755e94dc6c197920bf75 Implement GetID for DescriptorScriptPubKeyMan (Andrew Chow)
ec2f9e1178c8e38c0a5ca063fe81adac8f916348 Implement IsHDEnabled in DescriptorScriptPubKeyMan (Andrew Chow)
741122d4c1a62ced3e96d16d67f4eeb3a6522d99 Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan (Andrew Chow)
2db7ca765c8fb2c71dd6f7c4f29ad70e68ff1720 Implement IsMine for DescriptorScriptPubKeyMan (Andrew Chow)
db7177af8c159abbcc209f2caafcd45d54c181c5 Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet (Andrew Chow)
78f8a92910d34247fa5d04368338c598d9908267 Implement SetType in DescriptorScriptPubKeyMan (Andrew Chow)
834de0300cde57ca3f662fb7aa5b1bdaed68bc8f Store WalletDescriptor in DescriptorScriptPubKeyMan (Andrew Chow)
d8132669e10c1db9ae0c2ea0d3f822d7d2f01345 Add a lock cs_desc_man for DescriptorScriptPubKeyMan (Andrew Chow)
3194a7f88ac1a32997b390b4f188c4b6a4af04a5 Introduce WalletDescriptor class (Andrew Chow)
6b13cd3fa854dfaeb9e269bff3d67cacc0e5b5dc Create LegacyScriptPubKeyMan when not a descriptor wallet (Andrew Chow)
aeac157c9dc141546b45e06ba9c2e641ad86083f Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet (Andrew Chow)
96accc73f067c7c95946e9932645dd821ef67f63 Add WALLET_FLAG_DESCRIPTORS (Andrew Chow)
6b8119af53ee2fdb4c4b5b24b4e650c0dc3bd27c Introduce DescriptorScriptPubKeyMan as a dummy class (Andrew Chow)
06620302c713cae65ee8e4ff9302e4c88e2a1285 Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it (Andrew Chow)
Pull request description:
Introducing the wallet of the glorious future (again): native descriptor wallets. With native descriptor wallets, addresses are generated from descriptors. Instead of generating keys and deriving addresses from keys, addresses come from the scriptPubKeys produced by a descriptor. Native descriptor wallets will be optional for now and can only be created by using `createwallet`.
Descriptor wallets will store descriptors, master keys from the descriptor, and descriptor cache entries. Keys are derived from descriptors on the fly. In order to allow choosing different address types, 6 descriptors are needed for normal use. There is a pair of primary and change descriptors for each of the 3 address types. With the default keypool size of 1000, each descriptor has 1000 scriptPubKeys and descriptor cache entries pregenerated. This has a side effect of making wallets large since 6000 pubkeys are written to the wallet by default, instead of the current 2000. scriptPubKeys are kept only in memory and are generated every time a descriptor is loaded. By default, we use the standard BIP 44, 49, 84 derivation paths with an external and internal derivation chain for each.
Descriptors can also be imported with a new `importdescriptors` RPC.
Native descriptor wallets use the `ScriptPubKeyMan` interface introduced in #16341 to add a `DescriptorScriptPubKeyMan`. This defines a different IsMine which uses the simpler model of "does this scriptPubKey exist in this wallet". Furthermore, `DescriptorScriptPubKeyMan` does not have watchonly, so with native descriptor wallets, it is not possible to have a wallet with both watchonly and non-watchonly things. Rather a wallet with `disable_private_keys` needs to be used for watchonly things.
A `--descriptor` option was added to some tests (`wallet_basic.py`, `wallet_encryption.py`, `wallet_keypool.py`, `wallet_keypool_topup.py`, and `wallet_labels.py`) to allow for these tests to use descriptor wallets. Additionally, several RPCs are disabled for descriptor wallets (`importprivkey`, `importpubkey`, `importaddress`, `importmulti`, `addmultisigaddress`, `dumpprivkey`, `dumpwallet`, `importwallet`, and `sethdseed`).
ACKs for top commit:
Sjors:
utACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82 (rebased, nits addressed)
jonatack:
Code review re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82.
fjahr:
re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
instagibbs:
light re-ACK 223588b
meshcollider:
Code review ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
Tree-SHA512: 59bc52aeddbb769ed5f420d5d240d8137847ac821b588eb616b34461253510c1717d6a70bab8765631738747336ae06f45ba39603ccd17f483843e5ed9a90986
Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it
Introduce DescriptorScriptPubKeyMan as a dummy class
Add WALLET_FLAG_DESCRIPTORS
Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet
Create LegacyScriptPubKeyMan when not a descriptor wallet
Introduce WalletDescriptor class
WalletDescriptor is a Descriptor with other wallet metadata
Add a lock cs_desc_man for DescriptorScriptPubKeyMan
Store WalletDescriptor in DescriptorScriptPubKeyMan
Implement SetType in DescriptorScriptPubKeyMan
Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet
Implement IsMine for DescriptorScriptPubKeyMan
Adds a set of scriptPubKeys that DescriptorScriptPubKeyMan tracks.
If the given script is in that set, it is considered ISMINE_SPENDABLE
Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan
Implement IsHDEnabled in DescriptorScriptPubKeyMan
Implement GetID for DescriptorScriptPubKeyMan
Load the descriptor cache from the wallet file
Implement loading of keys for DescriptorScriptPubKeyMan
Add IsSingleType to Descriptors
IsSingleType will return whether the descriptor will give one or multiple scriptPubKeys
Implement several simple functions in DescriptorScriptPubKeyMan
Implements a bunch of one liners: UpgradeKeyMetadata, IsFirstRun, HavePrivateKeys,
KeypoolCountExternalKeys, GetKeypoolSize, GetTimeFirstKey, CanGetAddresses,
RewriteDB
Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file
Implement SetupGeneration for DescriptorScriptPubKeyMan
Implement TopUp in DescriptorScriptPubKeyMan
Implement GetNewDestination for DescriptorScriptPubKeyMan
Implement Unlock and Encrypt in DescriptorScriptPubKeyMan
Implement GetReservedDestination in DescriptorScriptPubKeyMan
Implement ReturnDestination in DescriptorScriptPubKeyMan
Implement GetKeypoolOldestTime and only display it if greater than 0
Implement GetSolvingProvider for DescriptorScriptPubKeyMan
Internally, a GetSigningProvider function is introduced which allows for
some private keys to be optionally included. This can be called with a
script as the argument (i.e. a scriptPubKey from our wallet when we are
signing) or with a pubkey. In order to know what index to expand the
private keys for that pubkey, we need to also cache all of the pubkeys
involved when we expand the descriptor. So SetCache and TopUp are
updated to do this too.
Implement SignTransaction in DescriptorScriptPubKeyMan
Implement SignMessage for descriptor wallets
Implement FillPSBT in DescriptorScriptPubKeyMan
FillPSBT will add our own scripts to the PSBT if those inputs are ours.
If an input also lists pubkeys that we happen to know the private keys
for, we will sign those inputs too.
Change GetMetadata to use unique_ptr<CKeyMetadata>
Implement GetMetadata in DescriptorScriptPubKeyMan
Be able to create new wallets with DescriptorScriptPubKeyMans as backing
Generate new descriptors when encrypting
Add IsLegacy to CWallet so that the GUI knows whether to show watchonly
add importdescriptors RPC and tests for native descriptor wallets
Co-authored-by: Andrew Chow <achow101-github@achow101.com>
Functional tests for descriptor wallets
Change wallet_encryption.py to use signmessage instead of dumpprivkey
Return error when no ScriptPubKeyMan is available for specified type
When a CWallet doesn't have a ScriptPubKeyMan for the requested type
in GetNewDestination, give a meaningful error. Also handle this in
Qt which did not do anything with errors.
Implement CWallet::IsSpentKey for non-LegacySPKMans
tests: Add RPCOverloadWrapper which overloads some disabled RPCs
RPCOverloadWrapper overloads some deprecated or disabled RPCs with
an implementation using other RPCs to avoid having a ton of code churn
around replacing those RPCs.
Add a --descriptors option to various tests
Adds a --descriptors option globally to the test framework. This will
make the test create and use descriptor wallets. However some tests may
not work with this.
Some tests are modified to work with --descriptors and run with that
option in test_runer:
* wallet_basic.py
* wallet_encryption.py
* wallet_keypool.py <---- wallet_keypool_hd.py actually
* wallet_keypool_topup.py
* wallet_labels.py
* wallet_avoidreuse.py
2019-07-16 19:34:35 +02:00
rawtx = self . nodes [ 0 ] . createrawtransaction ( [ { " txid " : singletxid , " vout " : 0 } ] , { chain_addrs [ 0 ] : node0_balance / / 2 - Decimal ( ' 0.01 ' ) , chain_addrs [ 1 ] : node0_balance / / 2 - Decimal ( ' 0.01 ' ) } )
2018-02-20 03:29:22 +01:00
signedtx = self . nodes [ 0 ] . signrawtransactionwithwallet ( rawtx )
2019-09-25 11:51:26 +02:00
singletxid = self . nodes [ 0 ] . sendrawtransaction ( hexstring = signedtx [ " hex " ] , maxfeerate = 0 )
2016-12-20 13:12:46 +01:00
self . nodes [ 0 ] . generate ( 1 )
# Make a long chain of unconfirmed payments without hitting mempool limit
2017-01-04 20:48:14 +01:00
# Each tx we make leaves only one output of change on a chain 1 longer
# Since the amount to send is always much less than the outputs, we only ever need one output
# So we should be able to generate exactly chainlimit txs for each original output
sending_addr = self . nodes [ 1 ] . getnewaddress ( )
2016-12-20 13:12:46 +01:00
txid_list = [ ]
2020-08-11 02:50:34 +02:00
for _ in range ( chainlimit * 2 ) :
2017-01-04 20:48:14 +01:00
txid_list . append ( self . nodes [ 0 ] . sendtoaddress ( sending_addr , Decimal ( ' 0.0001 ' ) ) )
2021-04-12 16:58:49 +02:00
assert_equal ( self . nodes [ 0 ] . getmempoolinfo ( ) [ ' size ' ] , chainlimit * 2 )
assert_equal ( len ( txid_list ) , chainlimit * 2 )
2016-12-20 13:12:46 +01:00
2017-01-04 20:48:14 +01:00
# Without walletrejectlongchains, we will still generate a txid
# The tx will be stored in the wallet but not accepted to the mempool
extra_txid = self . nodes [ 0 ] . sendtoaddress ( sending_addr , Decimal ( ' 0.0001 ' ) )
2019-02-01 15:35:08 +01:00
assert extra_txid not in self . nodes [ 0 ] . getrawmempool ( )
assert extra_txid in [ tx [ " txid " ] for tx in self . nodes [ 0 ] . listtransactions ( ) ]
2017-01-04 20:48:14 +01:00
self . nodes [ 0 ] . abandontransaction ( extra_txid )
2021-04-12 16:58:49 +02:00
total_txs = len ( self . nodes [ 0 ] . listtransactions ( " * " , 99999 ) )
2017-01-04 20:48:14 +01:00
# Try with walletrejectlongchains
# Double chain limit but require combining inputs, so we pass SelectCoinsMinConf
Merge #16528: Native Descriptor Wallets using DescriptorScriptPubKeyMan
223588b1bbc63dc57098bbd0baa48635e0cc0b82 Add a --descriptors option to various tests (Andrew Chow)
869f7ab30aeb4d7fbd563c535b55467a8a0430cf tests: Add RPCOverloadWrapper which overloads some disabled RPCs (Andrew Chow)
cf060628590fab87d73f278e744d70ef2d5d81db Correctly check for default wallet (Andrew Chow)
886e0d75f5fea2421190aa4812777d89f68962cc Implement CWallet::IsSpentKey for non-LegacySPKMans (Andrew Chow)
3c19fdd2a2fd5394fcfa75b2ba84ab2277cbdabf Return error when no ScriptPubKeyMan is available for specified type (Andrew Chow)
388ba94231f2f10a0be751c562cdd4650510a90a Change wallet_encryption.py to use signmessage instead of dumpprivkey (Andrew Chow)
1346e14831489f9c8f53a08f9dfed61d55d53c6f Functional tests for descriptor wallets (Andrew Chow)
f193ea889ddb53d9a5c47647966681d525e38368 add importdescriptors RPC and tests for native descriptor wallets (Hugo Nguyen)
ce24a944940019185efebcc5d85eac458ed26016 Add IsLegacy to CWallet so that the GUI knows whether to show watchonly (Andrew Chow)
1cb42b22b11c27e64462afc25a94b2fc50bfa113 Generate new descriptors when encrypting (Andrew Chow)
82ae02b1656819f4bd5023b8955447e1d4ea8692 Be able to create new wallets with DescriptorScriptPubKeyMans as backing (Andrew Chow)
b713baa75a62335ab9c0eed9ef76a95bfec30668 Implement GetMetadata in DescriptorScriptPubKeyMan (Andrew Chow)
8b9603bd0b443e2f7984eb72bf2e21cf02af0bcb Change GetMetadata to use unique_ptr<CKeyMetadata> (Andrew Chow)
72a9540df96ffdb94f039b9c14eaacdc7d961196 Implement FillPSBT in DescriptorScriptPubKeyMan (Andrew Chow)
84b4978c02102171775c77a45f6ec198930f0a88 Implement SignMessage for descriptor wallets (Andrew Chow)
bde7c9fa38775a81d53ac0484fa9c98076a0c7d1 Implement SignTransaction in DescriptorScriptPubKeyMan (Andrew Chow)
d50c8ddd4190f20bf0debd410348b73408ec3143 Implement GetSolvingProvider for DescriptorScriptPubKeyMan (Andrew Chow)
f1ca5feb4ad668a3e1ae543d0addd5f483f1a88f Implement GetKeypoolOldestTime and only display it if greater than 0 (Andrew Chow)
586b57a9a6b4b12a78f792785b63a5a1743bce0c Implement ReturnDestination in DescriptorScriptPubKeyMan (Andrew Chow)
f866957979c23cefd41efa9dae9e53b9177818dc Implement GetReservedDestination in DescriptorScriptPubKeyMan (Andrew Chow)
a775f7c7fd0b9094fcbeee6ba92206d5bbb19164 Implement Unlock and Encrypt in DescriptorScriptPubKeyMan (Andrew Chow)
bfdd0734869a22217c15858d7a76d0dacc2ebc86 Implement GetNewDestination for DescriptorScriptPubKeyMan (Andrew Chow)
58c7651821b0eeff0a99dc61d78d2e9e07986580 Implement TopUp in DescriptorScriptPubKeyMan (Andrew Chow)
e014886a342508f7c8d80323eee9a5f314eaf94c Implement SetupGeneration for DescriptorScriptPubKeyMan (Andrew Chow)
46dfb99768e7d03a3cf552812d5b41ceaebc06be Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file (Andrew Chow)
4cb9b69be031e1dc65d8964794781b347fd948f5 Implement several simple functions in DescriptorScriptPubKeyMan (Andrew Chow)
d1ec3e4f19487b4b100f80ad02eac063c571777d Add IsSingleType to Descriptors (Andrew Chow)
953feb3d2724f5398dd48990c4957a19313d2c8c Implement loading of keys for DescriptorScriptPubKeyMan (Andrew Chow)
2363e9fcaa41b68bf11153f591b95f2d41ff9a1a Load the descriptor cache from the wallet file (Andrew Chow)
46c46aebb7943e1e2e96755e94dc6c197920bf75 Implement GetID for DescriptorScriptPubKeyMan (Andrew Chow)
ec2f9e1178c8e38c0a5ca063fe81adac8f916348 Implement IsHDEnabled in DescriptorScriptPubKeyMan (Andrew Chow)
741122d4c1a62ced3e96d16d67f4eeb3a6522d99 Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan (Andrew Chow)
2db7ca765c8fb2c71dd6f7c4f29ad70e68ff1720 Implement IsMine for DescriptorScriptPubKeyMan (Andrew Chow)
db7177af8c159abbcc209f2caafcd45d54c181c5 Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet (Andrew Chow)
78f8a92910d34247fa5d04368338c598d9908267 Implement SetType in DescriptorScriptPubKeyMan (Andrew Chow)
834de0300cde57ca3f662fb7aa5b1bdaed68bc8f Store WalletDescriptor in DescriptorScriptPubKeyMan (Andrew Chow)
d8132669e10c1db9ae0c2ea0d3f822d7d2f01345 Add a lock cs_desc_man for DescriptorScriptPubKeyMan (Andrew Chow)
3194a7f88ac1a32997b390b4f188c4b6a4af04a5 Introduce WalletDescriptor class (Andrew Chow)
6b13cd3fa854dfaeb9e269bff3d67cacc0e5b5dc Create LegacyScriptPubKeyMan when not a descriptor wallet (Andrew Chow)
aeac157c9dc141546b45e06ba9c2e641ad86083f Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet (Andrew Chow)
96accc73f067c7c95946e9932645dd821ef67f63 Add WALLET_FLAG_DESCRIPTORS (Andrew Chow)
6b8119af53ee2fdb4c4b5b24b4e650c0dc3bd27c Introduce DescriptorScriptPubKeyMan as a dummy class (Andrew Chow)
06620302c713cae65ee8e4ff9302e4c88e2a1285 Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it (Andrew Chow)
Pull request description:
Introducing the wallet of the glorious future (again): native descriptor wallets. With native descriptor wallets, addresses are generated from descriptors. Instead of generating keys and deriving addresses from keys, addresses come from the scriptPubKeys produced by a descriptor. Native descriptor wallets will be optional for now and can only be created by using `createwallet`.
Descriptor wallets will store descriptors, master keys from the descriptor, and descriptor cache entries. Keys are derived from descriptors on the fly. In order to allow choosing different address types, 6 descriptors are needed for normal use. There is a pair of primary and change descriptors for each of the 3 address types. With the default keypool size of 1000, each descriptor has 1000 scriptPubKeys and descriptor cache entries pregenerated. This has a side effect of making wallets large since 6000 pubkeys are written to the wallet by default, instead of the current 2000. scriptPubKeys are kept only in memory and are generated every time a descriptor is loaded. By default, we use the standard BIP 44, 49, 84 derivation paths with an external and internal derivation chain for each.
Descriptors can also be imported with a new `importdescriptors` RPC.
Native descriptor wallets use the `ScriptPubKeyMan` interface introduced in #16341 to add a `DescriptorScriptPubKeyMan`. This defines a different IsMine which uses the simpler model of "does this scriptPubKey exist in this wallet". Furthermore, `DescriptorScriptPubKeyMan` does not have watchonly, so with native descriptor wallets, it is not possible to have a wallet with both watchonly and non-watchonly things. Rather a wallet with `disable_private_keys` needs to be used for watchonly things.
A `--descriptor` option was added to some tests (`wallet_basic.py`, `wallet_encryption.py`, `wallet_keypool.py`, `wallet_keypool_topup.py`, and `wallet_labels.py`) to allow for these tests to use descriptor wallets. Additionally, several RPCs are disabled for descriptor wallets (`importprivkey`, `importpubkey`, `importaddress`, `importmulti`, `addmultisigaddress`, `dumpprivkey`, `dumpwallet`, `importwallet`, and `sethdseed`).
ACKs for top commit:
Sjors:
utACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82 (rebased, nits addressed)
jonatack:
Code review re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82.
fjahr:
re-ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
instagibbs:
light re-ACK 223588b
meshcollider:
Code review ACK 223588b1bbc63dc57098bbd0baa48635e0cc0b82
Tree-SHA512: 59bc52aeddbb769ed5f420d5d240d8137847ac821b588eb616b34461253510c1717d6a70bab8765631738747336ae06f45ba39603ccd17f483843e5ed9a90986
Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it
Introduce DescriptorScriptPubKeyMan as a dummy class
Add WALLET_FLAG_DESCRIPTORS
Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet
Create LegacyScriptPubKeyMan when not a descriptor wallet
Introduce WalletDescriptor class
WalletDescriptor is a Descriptor with other wallet metadata
Add a lock cs_desc_man for DescriptorScriptPubKeyMan
Store WalletDescriptor in DescriptorScriptPubKeyMan
Implement SetType in DescriptorScriptPubKeyMan
Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet
Implement IsMine for DescriptorScriptPubKeyMan
Adds a set of scriptPubKeys that DescriptorScriptPubKeyMan tracks.
If the given script is in that set, it is considered ISMINE_SPENDABLE
Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan
Implement IsHDEnabled in DescriptorScriptPubKeyMan
Implement GetID for DescriptorScriptPubKeyMan
Load the descriptor cache from the wallet file
Implement loading of keys for DescriptorScriptPubKeyMan
Add IsSingleType to Descriptors
IsSingleType will return whether the descriptor will give one or multiple scriptPubKeys
Implement several simple functions in DescriptorScriptPubKeyMan
Implements a bunch of one liners: UpgradeKeyMetadata, IsFirstRun, HavePrivateKeys,
KeypoolCountExternalKeys, GetKeypoolSize, GetTimeFirstKey, CanGetAddresses,
RewriteDB
Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file
Implement SetupGeneration for DescriptorScriptPubKeyMan
Implement TopUp in DescriptorScriptPubKeyMan
Implement GetNewDestination for DescriptorScriptPubKeyMan
Implement Unlock and Encrypt in DescriptorScriptPubKeyMan
Implement GetReservedDestination in DescriptorScriptPubKeyMan
Implement ReturnDestination in DescriptorScriptPubKeyMan
Implement GetKeypoolOldestTime and only display it if greater than 0
Implement GetSolvingProvider for DescriptorScriptPubKeyMan
Internally, a GetSigningProvider function is introduced which allows for
some private keys to be optionally included. This can be called with a
script as the argument (i.e. a scriptPubKey from our wallet when we are
signing) or with a pubkey. In order to know what index to expand the
private keys for that pubkey, we need to also cache all of the pubkeys
involved when we expand the descriptor. So SetCache and TopUp are
updated to do this too.
Implement SignTransaction in DescriptorScriptPubKeyMan
Implement SignMessage for descriptor wallets
Implement FillPSBT in DescriptorScriptPubKeyMan
FillPSBT will add our own scripts to the PSBT if those inputs are ours.
If an input also lists pubkeys that we happen to know the private keys
for, we will sign those inputs too.
Change GetMetadata to use unique_ptr<CKeyMetadata>
Implement GetMetadata in DescriptorScriptPubKeyMan
Be able to create new wallets with DescriptorScriptPubKeyMans as backing
Generate new descriptors when encrypting
Add IsLegacy to CWallet so that the GUI knows whether to show watchonly
add importdescriptors RPC and tests for native descriptor wallets
Co-authored-by: Andrew Chow <achow101-github@achow101.com>
Functional tests for descriptor wallets
Change wallet_encryption.py to use signmessage instead of dumpprivkey
Return error when no ScriptPubKeyMan is available for specified type
When a CWallet doesn't have a ScriptPubKeyMan for the requested type
in GetNewDestination, give a meaningful error. Also handle this in
Qt which did not do anything with errors.
Implement CWallet::IsSpentKey for non-LegacySPKMans
tests: Add RPCOverloadWrapper which overloads some disabled RPCs
RPCOverloadWrapper overloads some deprecated or disabled RPCs with
an implementation using other RPCs to avoid having a ton of code churn
around replacing those RPCs.
Add a --descriptors option to various tests
Adds a --descriptors option globally to the test framework. This will
make the test create and use descriptor wallets. However some tests may
not work with this.
Some tests are modified to work with --descriptors and run with that
option in test_runer:
* wallet_basic.py
* wallet_encryption.py
* wallet_keypool.py <---- wallet_keypool_hd.py actually
* wallet_keypool_topup.py
* wallet_labels.py
* wallet_avoidreuse.py
2019-07-16 19:34:35 +02:00
self . stop_node ( 0 )
extra_args = [ " -walletrejectlongchains " , " -limitancestorcount= " + str ( 2 * chainlimit ) ]
self . start_node ( 0 , extra_args = extra_args )
2017-01-04 20:48:14 +01:00
2022-09-07 20:48:44 +02:00
# wait until the wallet has submitted all transactions to the mempool
2020-08-15 15:38:51 +02:00
self . wait_until ( lambda : len ( self . nodes [ 0 ] . getrawmempool ( ) ) == chainlimit * 2 )
2017-01-04 20:48:14 +01:00
2020-09-06 13:03:39 +02:00
# Prevent potential race condition when calling wallet RPCs right after restart
self . nodes [ 0 ] . syncwithvalidationinterfacequeue ( )
2017-01-04 20:48:14 +01:00
node0_balance = self . nodes [ 0 ] . getbalance ( )
# With walletrejectlongchains we will not create the tx and store it in our wallet.
2019-09-25 11:34:51 +02:00
assert_raises_rpc_error ( - 4 , " Transaction has too long of a mempool chain " , self . nodes [ 0 ] . sendtoaddress , sending_addr , node0_balance - Decimal ( ' 0.01 ' ) )
2017-01-04 20:48:14 +01:00
# Verify nothing new in wallet
2021-04-12 16:58:49 +02:00
assert_equal ( total_txs , len ( self . nodes [ 0 ] . listtransactions ( " * " , 99999 ) ) )
2017-01-04 20:48:14 +01:00
2018-11-04 23:22:35 +01:00
# Test getaddressinfo on external address. Note that these addresses are taken from disablewallet.py
2021-01-26 00:38:38 +01:00
assert_raises_rpc_error ( - 5 , " Invalid prefix for Base58-encoded address " , self . nodes [ 0 ] . getaddressinfo , " 3J98t1WpEZ73CNmQviecrnyiWrnqRhWNLy " )
2020-12-17 13:46:20 +01:00
address_info = self . nodes [ 0 ] . getaddressinfo ( " yjQ5gLvGRtmq1cwc4kePLCrzQ8GVCh9Gaz " )
assert_equal ( address_info [ ' address ' ] , " yjQ5gLvGRtmq1cwc4kePLCrzQ8GVCh9Gaz " )
assert_equal ( address_info [ " scriptPubKey " ] , " 76a914fd2b4d101724a76374fccbc5b6df7670a75d7cd088ac " )
assert not address_info [ " ismine " ]
assert not address_info [ " iswatchonly " ]
assert not address_info [ " isscript " ]
2018-11-04 23:22:35 +01:00
assert not address_info [ " ischange " ]
# Test getaddressinfo 'ischange' field on change address.
self . nodes [ 0 ] . generate ( 1 )
destination = self . nodes [ 1 ] . getnewaddress ( )
txid = self . nodes [ 0 ] . sendtoaddress ( destination , 0.123 )
2019-02-01 15:35:08 +01:00
tx = self . nodes [ 0 ] . decoderawtransaction ( self . nodes [ 0 ] . gettransaction ( txid ) [ ' hex ' ] )
2018-11-04 23:22:35 +01:00
output_addresses = [ vout [ ' scriptPubKey ' ] [ ' addresses ' ] [ 0 ] for vout in tx [ " vout " ] ]
assert len ( output_addresses ) > 1
for address in output_addresses :
ischange = self . nodes [ 0 ] . getaddressinfo ( address ) [ ' ischange ' ]
assert_equal ( ischange , address != destination )
if ischange :
change = address
self . nodes [ 0 ] . setlabel ( change , ' foobar ' )
assert_equal ( self . nodes [ 0 ] . getaddressinfo ( change ) [ ' ischange ' ] , False )
2020-12-17 13:46:20 +01:00
Merge #16873: rpc: fix regression in gettransaction
1b41c2c8a126ef4be183e1d800a17d85cab8837b test: improve gettransaction test coverage (Jon Atack)
0f34f54888f680bfbe7a29ac278636d7178a99bb rpc: fix regression in gettransaction (Jon Atack)
Pull request description:
Closes #16872.
PR #16866 renamed the `decode` argument in gettransaction to `verbose` to make it more consistent with other RPC calls like getrawtransaction. However, it inadvertently overloaded the "details" field when `verbose` is passed. The result is that the original "details" field is no longer returned correctly, which seems to be a breaking API change.
This PR:
- takes the simplest path to restoring the "details" field by renaming the decoded one back to "decoded" while leaving the `verbose` argument for API consistency, which was the main intent of #16866,
- addresses [this comment](https://github.com/bitcoin/bitcoin/pull/16185#discussion_r320740413) by mentioning in the RPC help that the new decoded field is equivalent to decoderawtransaction, and
- updates the help, functional test, and release note.
Reviewers, to test this manually, build and run `bitcoin-cli help gettransaction` and `bitcoin-cli gettransaction <wallet txid> false true`, and verify that the command returns both `details` and `decoded` fields.
ACKs for top commit:
jnewbery:
tACK 1b41c2c8a126ef4be183e1d800a17d85cab8837b
Tree-SHA512: 287edd5db7ed58fe8b548975aba58628bd45ed708b28f40174f10a35a455d89f796fbf27430aa881fc376f47aabda8803f74d4d100683bd86577a02279091cf3
2019-09-15 23:44:06 +02:00
# Test gettransaction response with different arguments.
self . log . info ( " Testing gettransaction response with different arguments... " )
self . nodes [ 0 ] . setlabel ( change , ' baz ' )
baz = self . nodes [ 0 ] . listtransactions ( label = " baz " , count = 1 ) [ 0 ]
expected_receive_vout = { " label " : " baz " ,
" address " : baz [ " address " ] ,
" amount " : baz [ " amount " ] ,
" category " : baz [ " category " ] ,
" vout " : baz [ " vout " ] }
expected_fields = frozenset ( { ' amount ' , ' chainlock ' , ' confirmations ' , ' details ' , ' fee ' ,
' instantlock ' , ' instantlock_internal ' ,
' hex ' , ' timereceived ' , ' time ' , ' trusted ' , ' txid ' , ' walletconflicts ' } )
verbose_field = " decoded "
expected_verbose_fields = expected_fields | { verbose_field }
self . log . debug ( " Testing gettransaction response without verbose " )
tx = self . nodes [ 0 ] . gettransaction ( txid = txid )
assert_equal ( set ( [ * tx ] ) , expected_fields )
assert_array_result ( tx [ " details " ] , { " category " : " receive " } , expected_receive_vout )
self . log . debug ( " Testing gettransaction response with verbose set to False " )
tx = self . nodes [ 0 ] . gettransaction ( txid = txid , verbose = False )
assert_equal ( set ( [ * tx ] ) , expected_fields )
assert_array_result ( tx [ " details " ] , { " category " : " receive " } , expected_receive_vout )
self . log . debug ( " Testing gettransaction response with verbose set to True " )
2019-09-14 11:42:34 +02:00
tx = self . nodes [ 0 ] . gettransaction ( txid = txid , verbose = True )
Merge #16873: rpc: fix regression in gettransaction
1b41c2c8a126ef4be183e1d800a17d85cab8837b test: improve gettransaction test coverage (Jon Atack)
0f34f54888f680bfbe7a29ac278636d7178a99bb rpc: fix regression in gettransaction (Jon Atack)
Pull request description:
Closes #16872.
PR #16866 renamed the `decode` argument in gettransaction to `verbose` to make it more consistent with other RPC calls like getrawtransaction. However, it inadvertently overloaded the "details" field when `verbose` is passed. The result is that the original "details" field is no longer returned correctly, which seems to be a breaking API change.
This PR:
- takes the simplest path to restoring the "details" field by renaming the decoded one back to "decoded" while leaving the `verbose` argument for API consistency, which was the main intent of #16866,
- addresses [this comment](https://github.com/bitcoin/bitcoin/pull/16185#discussion_r320740413) by mentioning in the RPC help that the new decoded field is equivalent to decoderawtransaction, and
- updates the help, functional test, and release note.
Reviewers, to test this manually, build and run `bitcoin-cli help gettransaction` and `bitcoin-cli gettransaction <wallet txid> false true`, and verify that the command returns both `details` and `decoded` fields.
ACKs for top commit:
jnewbery:
tACK 1b41c2c8a126ef4be183e1d800a17d85cab8837b
Tree-SHA512: 287edd5db7ed58fe8b548975aba58628bd45ed708b28f40174f10a35a455d89f796fbf27430aa881fc376f47aabda8803f74d4d100683bd86577a02279091cf3
2019-09-15 23:44:06 +02:00
assert_equal ( set ( [ * tx ] ) , expected_verbose_fields )
assert_array_result ( tx [ " details " ] , { " category " : " receive " } , expected_receive_vout )
assert_equal ( tx [ verbose_field ] , self . nodes [ 0 ] . decoderawtransaction ( tx [ " hex " ] ) )
2019-09-02 13:29:42 +02:00
2019-02-01 15:35:08 +01:00
2014-11-19 21:55:40 +01:00
if __name__ == ' __main__ ' :
2016-05-17 09:26:42 +02:00
WalletTest ( ) . main ( )