sikkienl/dash
1
0
Fork 0
mirror of https://github.com/dashpay/dash.git synced 2024-12-25 12:02:48 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge #6017: backport: bitcoin#18836, #19046, #19490, #20403, #21127, #21238, #21329 - descriptor wallets part V & sethdseed

Browse Source 
d3ad11d056 chore: add release notes for sethdseed RPC (Konstantin Akimov)
ad25d54300 Merge bitcoin/bitcoin#21329: descriptor wallet: Cache last hardened xpub and use in normalized descriptors (Samuel Dobson)
24b1f6bb27 Merge #21238: A few descriptor improvements to prepare for Taproot support (W. J. van der Laan)
14ac2b77f3 Merge #21127: wallet: load flags before everything else (Wladimir J. van der Laan)
19b2b27785 Merge #20403: wallet: upgradewallet fixes, improvements, test coverage (MarcoFalke)
708586c77e Merge #18836: wallet: upgradewallet fixes and additional tests (Andrew Chow)
752e4ca048 Merge #19490: wallet: Fix typo in comments; Simplify assert (Samuel Dobson)
63895fde23 Merge #19046: Replace CWallet::Set* functions that use memonly with Add/Load variants (Andrew Chow)
2c0d5b7c71 refactor: rename hdChain to m_hd_chain (Konstantin Akimov)
266aefc544 feat: sethdseed rpc added. Based on bitcoin#12560 and the newest related changes (Konstantin Akimov)

Pull request description:

  ## Issue being fixed or feature implemented
  Next batch of backports related to descriptor wallets.
  See related issue to track a progress: https://github.com/dashpay/dash-issues/issues/59
  Changes in this PR also used in "hardware signing" feature (coming later)

  ## What was done?
  1. It implements a new rpc `sethdseed` that is based on bitcoin#12560 and newer related changes.
  2. refactoring to rename `hdChain` to `m_hd_chain` (see bitcoin#17681 which is DNM).
  3. Bitcoin backports (some of them uses sethdseed, and requires m_hd_chain to reduce conflicts):
   - bitcoin/bitcoin#19046
   - bitcoin/bitcoin#19490
   - bitcoin/bitcoin#18836
   - bitcoin/bitcoin#20403
   - bitcoin/bitcoin#21127
   - bitcoin/bitcoin#21238
   - bitcoin/bitcoin#21329

  ## How Has This Been Tested?
  Run unit/functional tests.
  The backports #18836 and #20403 are heavily modified to adopt `wallet_upgradewallet.py` to our codebase.

  ## Breaking Changes
  N/A
  Though, `sethdseed` implementation is not a final version as it is now, can be removed (superseeded by `upgradetohd` or got mnemonic-feature and super-seed `upgradetohd`).

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK d3ad11d056

Tree-SHA512: 02182182ec7a5f89eb7d3bc34072d894a86cc89c5eea124e702cc5ed527f76863469b1fd9313b3ea643a8774a358031be927d7b78ec7cd39df0a9ca77559d66d
This commit is contained in:
pasta 2024-05-10 09:14:29 -05:00
commit 6028e37ad9
No known key found for this signature in database
GPG Key ID: 52527BEDABE87984
25 changed files with 1031 additions and 330 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
doc/release-notes-6017.md Normal file
View File

@ -0,0 +1,3 @@
RPC changes
-----------
- A new `sethdseed` RPC allows users to initialize their blank HD wallets with an HD seed. **A new backup must be made when a new HD seed is set.** This command cannot replace an existing HD seed if one is already set. `sethdseed` uses WIF private key as a seed. If you have a mnemonic, use the `upgradetohd` RPC.

1
src/qt/rpcconsole.cpp
View File

@ -70,6 +70,7 @@ namespace {
const QStringList historyFilter = QStringList() const QStringList historyFilter = QStringList()
<< "importprivkey" << "importprivkey"
<< "importmulti" << "importmulti"
<< "sethdseed"
<< "signmessagewithprivkey" << "signmessagewithprivkey"
<< "signrawtransactionwithkey" << "signrawtransactionwithkey"
<< "upgradetohd" << "upgradetohd"

1
src/rpc/client.cpp
View File

@ -45,6 +45,7 @@ static const CRPCConvertParam vRPCConvertParams[] =
{ "sendtoaddress", 7, "conf_target" }, { "sendtoaddress", 7, "conf_target" },
{ "sendtoaddress", 9, "avoid_reuse" }, { "sendtoaddress", 9, "avoid_reuse" },
{ "settxfee", 0, "amount" }, { "settxfee", 0, "amount" },
{ "sethdseed", 0, "newkeypool" },
{ "getreceivedbyaddress", 1, "minconf" }, { "getreceivedbyaddress", 1, "minconf" },
{ "getreceivedbyaddress", 2, "addlocked" }, { "getreceivedbyaddress", 2, "addlocked" },
{ "getreceivedbylabel", 1, "minconf" }, { "getreceivedbylabel", 1, "minconf" },

331
src/script/descriptor.cpp
View File

@ -166,7 +166,7 @@ public:
* write_cache is the cache to write keys to (if not nullptr) * write_cache is the cache to write keys to (if not nullptr)
* Caches are not exclusive but this is not tested. Currently we use them exclusively * Caches are not exclusive but this is not tested. Currently we use them exclusively
*/ */
virtual bool GetPubKey(int pos, const SigningProvider& arg, CPubKey& key, KeyOriginInfo& info, const DescriptorCache* read_cache = nullptr, DescriptorCache* write_cache = nullptr) = 0; virtual bool GetPubKey(int pos, const SigningProvider& arg, CPubKey& key, KeyOriginInfo& info, const DescriptorCache* read_cache = nullptr, DescriptorCache* write_cache = nullptr) const = 0;
/** Whether this represent multiple public keys at different positions. */ /** Whether this represent multiple public keys at different positions. */
virtual bool IsRange() const = 0; virtual bool IsRange() const = 0;
@ -181,7 +181,7 @@ public:
virtual bool ToPrivateString(const SigningProvider& arg, std::string& out) const = 0; virtual bool ToPrivateString(const SigningProvider& arg, std::string& out) const = 0;
/** Get the descriptor string form with the xpub at the last hardened derivation */ /** Get the descriptor string form with the xpub at the last hardened derivation */
virtual bool ToNormalizedString(const SigningProvider& arg, std::string& out, bool priv) const = 0; virtual bool ToNormalizedString(const SigningProvider& arg, std::string& out, const DescriptorCache* cache = nullptr) const = 0;
/** Derive a private key, if private data is available in arg. */ /** Derive a private key, if private data is available in arg. */
virtual bool GetPrivKey(int pos, const SigningProvider& arg, CKey& key) const = 0; virtual bool GetPrivKey(int pos, const SigningProvider& arg, CKey& key) const = 0;
@ -199,7 +199,7 @@ class OriginPubkeyProvider final : public PubkeyProvider
public: public:
OriginPubkeyProvider(uint32_t exp_index, KeyOriginInfo info, std::unique_ptr<PubkeyProvider> provider) : PubkeyProvider(exp_index), m_origin(std::move(info)), m_provider(std::move(provider)) {} OriginPubkeyProvider(uint32_t exp_index, KeyOriginInfo info, std::unique_ptr<PubkeyProvider> provider) : PubkeyProvider(exp_index), m_origin(std::move(info)), m_provider(std::move(provider)) {}
bool GetPubKey(int pos, const SigningProvider& arg, CPubKey& key, KeyOriginInfo& info, const DescriptorCache* read_cache = nullptr, DescriptorCache* write_cache = nullptr) override bool GetPubKey(int pos, const SigningProvider& arg, CPubKey& key, KeyOriginInfo& info, const DescriptorCache* read_cache = nullptr, DescriptorCache* write_cache = nullptr) const override
{ {
if (!m_provider->GetPubKey(pos, arg, key, info, read_cache, write_cache)) return false; if (!m_provider->GetPubKey(pos, arg, key, info, read_cache, write_cache)) return false;
std::copy(std::begin(m_origin.fingerprint), std::end(m_origin.fingerprint), info.fingerprint); std::copy(std::begin(m_origin.fingerprint), std::end(m_origin.fingerprint), info.fingerprint);
@ -216,10 +216,10 @@ public:
ret = "[" + OriginString() + "]" + std::move(sub); ret = "[" + OriginString() + "]" + std::move(sub);
return true; return true;
} }
bool ToNormalizedString(const SigningProvider& arg, std::string& ret, bool priv) const override bool ToNormalizedString(const SigningProvider& arg, std::string& ret, const DescriptorCache* cache) const override
{ {
std::string sub; std::string sub;
if (!m_provider->ToNormalizedString(arg, sub, priv)) return false; if (!m_provider->ToNormalizedString(arg, sub, cache)) return false;
// If m_provider is a BIP32PubkeyProvider, we may get a string formatted like a OriginPubkeyProvider // If m_provider is a BIP32PubkeyProvider, we may get a string formatted like a OriginPubkeyProvider
// In that case, we need to strip out the leading square bracket and fingerprint from the substring, // In that case, we need to strip out the leading square bracket and fingerprint from the substring,
// and append that to our own origin string. // and append that to our own origin string.
@ -244,7 +244,7 @@ class ConstPubkeyProvider final : public PubkeyProvider
public: public:
ConstPubkeyProvider(uint32_t exp_index, const CPubKey& pubkey) : PubkeyProvider(exp_index), m_pubkey(pubkey) {} ConstPubkeyProvider(uint32_t exp_index, const CPubKey& pubkey) : PubkeyProvider(exp_index), m_pubkey(pubkey) {}
bool GetPubKey(int pos, const SigningProvider& arg, CPubKey& key, KeyOriginInfo& info, const DescriptorCache* read_cache = nullptr, DescriptorCache* write_cache = nullptr) override bool GetPubKey(int pos, const SigningProvider& arg, CPubKey& key, KeyOriginInfo& info, const DescriptorCache* read_cache = nullptr, DescriptorCache* write_cache = nullptr) const override
{ {
key = m_pubkey; key = m_pubkey;
info.path.clear(); info.path.clear();
@ -262,9 +262,8 @@ public:
ret = EncodeSecret(key); ret = EncodeSecret(key);
return true; return true;
} }
bool ToNormalizedString(const SigningProvider& arg, std::string& ret, bool priv) const override bool ToNormalizedString(const SigningProvider& arg, std::string& ret, const DescriptorCache* cache) const override
{ {
if (priv) return ToPrivateString(arg, ret);
ret = ToString(); ret = ToString();
return true; return true;
} }
@ -287,9 +286,6 @@ class BIP32PubkeyProvider final : public PubkeyProvider
CExtPubKey m_root_extkey; CExtPubKey m_root_extkey;
KeyPath m_path; KeyPath m_path;
DeriveType m_derive; DeriveType m_derive;
// Cache of the parent of the final derived pubkeys.
// Primarily useful for situations when no read_cache is provided
CExtPubKey m_cached_xpub;
bool GetExtKey(const SigningProvider& arg, CExtKey& ret) const bool GetExtKey(const SigningProvider& arg, CExtKey& ret) const
{ {
@ -304,11 +300,14 @@ class BIP32PubkeyProvider final : public PubkeyProvider
} }
// Derives the last xprv // Derives the last xprv
bool GetDerivedExtKey(const SigningProvider& arg, CExtKey& xprv) const bool GetDerivedExtKey(const SigningProvider& arg, CExtKey& xprv, CExtKey& last_hardened) const
{ {
if (!GetExtKey(arg, xprv)) return false; if (!GetExtKey(arg, xprv)) return false;
for (auto entry : m_path) { for (auto entry : m_path) {
xprv.Derive(xprv, entry); xprv.Derive(xprv, entry);
if (entry >> 31) {
last_hardened = xprv;
}
} }
return true; return true;
} }
@ -326,7 +325,7 @@ public:
BIP32PubkeyProvider(uint32_t exp_index, const CExtPubKey& extkey, KeyPath path, DeriveType derive) : PubkeyProvider(exp_index), m_root_extkey(extkey), m_path(std::move(path)), m_derive(derive) {} BIP32PubkeyProvider(uint32_t exp_index, const CExtPubKey& extkey, KeyPath path, DeriveType derive) : PubkeyProvider(exp_index), m_root_extkey(extkey), m_path(std::move(path)), m_derive(derive) {}
bool IsRange() const override { return m_derive != DeriveType::NO; } bool IsRange() const override { return m_derive != DeriveType::NO; }
size_t GetSize() const override { return 33; } size_t GetSize() const override { return 33; }
bool GetPubKey(int pos, const SigningProvider& arg, CPubKey& key_out, KeyOriginInfo& final_info_out, const DescriptorCache* read_cache = nullptr, DescriptorCache* write_cache = nullptr) override bool GetPubKey(int pos, const SigningProvider& arg, CPubKey& key_out, KeyOriginInfo& final_info_out, const DescriptorCache* read_cache = nullptr, DescriptorCache* write_cache = nullptr) const override
{ {
// Info of parent of the to be derived pubkey // Info of parent of the to be derived pubkey
KeyOriginInfo parent_info; KeyOriginInfo parent_info;
@ -342,6 +341,7 @@ public:
// Derive keys or fetch them from cache // Derive keys or fetch them from cache
CExtPubKey final_extkey = m_root_extkey; CExtPubKey final_extkey = m_root_extkey;
CExtPubKey parent_extkey = m_root_extkey; CExtPubKey parent_extkey = m_root_extkey;
CExtPubKey last_hardened_extkey;
bool der = true; bool der = true;
if (read_cache) { if (read_cache) {
if (!read_cache->GetCachedDerivedExtPubKey(m_expr_index, pos, final_extkey)) { if (!read_cache->GetCachedDerivedExtPubKey(m_expr_index, pos, final_extkey)) {
@ -351,16 +351,17 @@ public:
final_extkey = parent_extkey; final_extkey = parent_extkey;
if (m_derive == DeriveType::UNHARDENED) der = parent_extkey.Derive(final_extkey, pos); if (m_derive == DeriveType::UNHARDENED) der = parent_extkey.Derive(final_extkey, pos);
} }
} else if (m_cached_xpub.pubkey.IsValid() && m_derive != DeriveType::HARDENED) {
parent_extkey = final_extkey = m_cached_xpub;
if (m_derive == DeriveType::UNHARDENED) der = parent_extkey.Derive(final_extkey, pos);
} else if (IsHardened()) { } else if (IsHardened()) {
CExtKey xprv; CExtKey xprv;
if (!GetDerivedExtKey(arg, xprv)) return false; CExtKey lh_xprv;
if (!GetDerivedExtKey(arg, xprv, lh_xprv)) return false;
parent_extkey = xprv.Neuter(); parent_extkey = xprv.Neuter();
if (m_derive == DeriveType::UNHARDENED) der = xprv.Derive(xprv, pos); if (m_derive == DeriveType::UNHARDENED) der = xprv.Derive(xprv, pos);
if (m_derive == DeriveType::HARDENED) der = xprv.Derive(xprv, pos | 0x80000000UL); if (m_derive == DeriveType::HARDENED) der = xprv.Derive(xprv, pos | 0x80000000UL);
final_extkey = xprv.Neuter(); final_extkey = xprv.Neuter();
if (lh_xprv.key.IsValid()) {
last_hardened_extkey = lh_xprv.Neuter();
}
} else { } else {
for (auto entry : m_path) { for (auto entry : m_path) {
der = parent_extkey.Derive(parent_extkey, entry); der = parent_extkey.Derive(parent_extkey, entry);
@ -375,15 +376,14 @@ public:
final_info_out = final_info_out_tmp; final_info_out = final_info_out_tmp;
key_out = final_extkey.pubkey; key_out = final_extkey.pubkey;
// We rely on the consumer to check that m_derive isn't HARDENED as above
// But we can't have already cached something in case we read something from the cache
// and parent_extkey isn't actually the parent.
if (!m_cached_xpub.pubkey.IsValid()) m_cached_xpub = parent_extkey;
if (write_cache) { if (write_cache) {
// Only cache parent if there is any unhardened derivation // Only cache parent if there is any unhardened derivation
if (m_derive != DeriveType::HARDENED) { if (m_derive != DeriveType::HARDENED) {
write_cache->CacheParentExtPubKey(m_expr_index, parent_extkey); write_cache->CacheParentExtPubKey(m_expr_index, parent_extkey);
// Cache last hardened xpub if we have it
if (last_hardened_extkey.pubkey.IsValid()) {
write_cache->CacheLastHardenedExtPubKey(m_expr_index, last_hardened_extkey);
}
} else if (final_info_out.path.size() > 0) { } else if (final_info_out.path.size() > 0) {
write_cache->CacheDerivedExtPubKey(m_expr_index, pos, final_extkey); write_cache->CacheDerivedExtPubKey(m_expr_index, pos, final_extkey);
} }
@ -411,11 +411,10 @@ public:
} }
return true; return true;
} }
bool ToNormalizedString(const SigningProvider& arg, std::string& out, bool priv) const override bool ToNormalizedString(const SigningProvider& arg, std::string& out, const DescriptorCache* cache) const override
{ {
// For hardened derivation type, just return the typical string, nothing to normalize // For hardened derivation type, just return the typical string, nothing to normalize
if (m_derive == DeriveType::HARDENED) { if (m_derive == DeriveType::HARDENED) {
if (priv) return ToPrivateString(arg, out);
out = ToString(); out = ToString();
return true; return true;
} }
@ -428,33 +427,42 @@ public:
} }
// Either no derivation or all unhardened derivation // Either no derivation or all unhardened derivation
if (i == -1) { if (i == -1) {
if (priv) return ToPrivateString(arg, out);
out = ToString(); out = ToString();
return true; return true;
} }
// Derive the xpub at the last hardened step // Get the path to the last hardened stup
CExtKey xprv;
if (!GetExtKey(arg, xprv)) return false;
KeyOriginInfo origin; KeyOriginInfo origin;
int k = 0; int k = 0;
for (; k <= i; ++k) { for (; k <= i; ++k) {
// Derive
xprv.Derive(xprv, m_path.at(k));
// Add to the path // Add to the path
origin.path.push_back(m_path.at(k)); origin.path.push_back(m_path.at(k));
// First derivation element, get the fingerprint for origin
if (k == 0) {
std::copy(xprv.vchFingerprint, xprv.vchFingerprint + 4, origin.fingerprint);
}
} }
// Build the remaining path // Build the remaining path
KeyPath end_path; KeyPath end_path;
for (; k < (int)m_path.size(); ++k) { for (; k < (int)m_path.size(); ++k) {
end_path.push_back(m_path.at(k)); end_path.push_back(m_path.at(k));
} }
// Get the fingerprint
CKeyID id = m_root_extkey.pubkey.GetID();
std::copy(id.begin(), id.begin() + 4, origin.fingerprint);
CExtPubKey xpub;
CExtKey lh_xprv;
// If we have the cache, just get the parent xpub
if (cache != nullptr) {
cache->GetCachedLastHardenedExtPubKey(m_expr_index, xpub);
}
if (!xpub.pubkey.IsValid()) {
// Cache miss, or nor cache, or need privkey
CExtKey xprv;
if (!GetDerivedExtKey(arg, xprv, lh_xprv)) return false;
xpub = lh_xprv.Neuter();
}
assert(xpub.pubkey.IsValid());
// Build the string // Build the string
std::string origin_str = HexStr(origin.fingerprint) + FormatHDKeypath(origin.path); std::string origin_str = HexStr(origin.fingerprint) + FormatHDKeypath(origin.path);
out = "[" + origin_str + "]" + (priv ? EncodeExtKey(xprv) : EncodeExtPubKey(xprv.Neuter())) + FormatHDKeypath(end_path); out = "[" + origin_str + "]" + EncodeExtPubKey(xpub) + FormatHDKeypath(end_path);
if (IsRange()) { if (IsRange()) {
out += "/*"; out += "/*";
assert(m_derive == DeriveType::UNHARDENED); assert(m_derive == DeriveType::UNHARDENED);
@ -464,7 +472,8 @@ public:
bool GetPrivKey(int pos, const SigningProvider& arg, CKey& key) const override bool GetPrivKey(int pos, const SigningProvider& arg, CKey& key) const override
{ {
CExtKey extkey; CExtKey extkey;
if (!GetDerivedExtKey(arg, extkey)) return false; CExtKey dummy;
if (!GetDerivedExtKey(arg, extkey, dummy)) return false;
if (m_derive == DeriveType::UNHARDENED) extkey.Derive(extkey, pos); if (m_derive == DeriveType::UNHARDENED) extkey.Derive(extkey, pos);
if (m_derive == DeriveType::HARDENED) extkey.Derive(extkey, pos | 0x80000000UL); if (m_derive == DeriveType::HARDENED) extkey.Derive(extkey, pos | 0x80000000UL);
key = extkey.key; key = extkey.key;
@ -481,34 +490,42 @@ class DescriptorImpl : public Descriptor
const std::string m_name; const std::string m_name;
protected: protected:
//! The sub-descriptor argument (nullptr for everything but SH and WSH). //! The sub-descriptor arguments (empty for everything but SH and WSH).
//! In doc/descriptors.m this is referred to as SCRIPT expressions sh(SCRIPT) //! In doc/descriptors.m this is referred to as SCRIPT expressions sh(SCRIPT)
//! and wsh(SCRIPT), and distinct from KEY expressions and ADDR expressions. //! and wsh(SCRIPT), and distinct from KEY expressions and ADDR expressions.
const std::unique_ptr<DescriptorImpl> m_subdescriptor_arg; //! Subdescriptors can only ever generate a single script.
const std::vector<std::unique_ptr<DescriptorImpl>> m_subdescriptor_args;
//! Return a serialization of anything except pubkey and script arguments, to be prepended to those. //! Return a serialization of anything except pubkey and script arguments, to be prepended to those.
virtual std::string ToStringExtra() const { return ""; } virtual std::string ToStringExtra() const { return ""; }
/** A helper function to construct the scripts for this descriptor. /** A helper function to construct the scripts for this descriptor.
* *
* This function is invoked once for every CScript produced by evaluating * This function is invoked once by ExpandHelper.
* m_subdescriptor_arg, or just once in case m_subdescriptor_arg is nullptr. *
* @param pubkeys The evaluations of the m_pubkey_args field. * @param pubkeys The evaluations of the m_pubkey_args field.
* @param scripts The evaluation of m_subdescriptor_arg (or nullptr when m_subdescriptor_arg is nullptr). * @param scripts The evaluations of m_subdescriptor_args (one for each m_subdescriptor_args element).
* @param out A FlatSigningProvider to put scripts or public keys in that are necessary to the solver. * @param out A FlatSigningProvider to put scripts or public keys in that are necessary to the solver.
* The script arguments to this function are automatically added, as is the origin info of the provided pubkeys. * The origin info of the provided pubkeys is automatically added.
* @return A vector with scriptPubKeys for this descriptor. * @return A vector with scriptPubKeys for this descriptor.
*/ */
virtual std::vector<CScript> MakeScripts(const std::vector<CPubKey>& pubkeys, const CScript* script, FlatSigningProvider& out) const = 0; virtual std::vector<CScript> MakeScripts(const std::vector<CPubKey>& pubkeys, Span<const CScript> scripts, FlatSigningProvider& out) const = 0;
public: public:
DescriptorImpl(std::vector<std::unique_ptr<PubkeyProvider>> pubkeys, std::unique_ptr<DescriptorImpl> script, const std::string& name) : m_pubkey_args(std::move(pubkeys)), m_name(name), m_subdescriptor_arg(std::move(script)) {} DescriptorImpl(std::vector<std::unique_ptr<PubkeyProvider>> pubkeys, const std::string& name) : m_pubkey_args(std::move(pubkeys)), m_name(name), m_subdescriptor_args() {}
DescriptorImpl(std::vector<std::unique_ptr<PubkeyProvider>> pubkeys, std::unique_ptr<DescriptorImpl> script, const std::string& name) : m_pubkey_args(std::move(pubkeys)), m_name(name), m_subdescriptor_args(Vector(std::move(script))) {}
enum class StringType
{
PUBLIC,
PRIVATE,
NORMALIZED,
};
bool IsSolvable() const override bool IsSolvable() const override
{ {
if (m_subdescriptor_arg) { for (const auto& arg : m_subdescriptor_args) {
if (!m_subdescriptor_arg->IsSolvable()) return false; if (!arg->IsSolvable()) return false;
} }
return true; return true;
} }
@ -518,13 +535,25 @@ public:
for (const auto& pubkey : m_pubkey_args) { for (const auto& pubkey : m_pubkey_args) {
if (pubkey->IsRange()) return true; if (pubkey->IsRange()) return true;
} }
if (m_subdescriptor_arg) { for (const auto& arg : m_subdescriptor_args) {
if (m_subdescriptor_arg->IsRange()) return true; if (arg->IsRange()) return true;
} }
return false; return false;
} }
bool ToStringHelper(const SigningProvider* arg, std::string& out, bool priv, bool normalized) const virtual bool ToStringSubScriptHelper(const SigningProvider* arg, std::string& ret, const StringType type, const DescriptorCache* cache = nullptr) const
{
size_t pos = 0;
for (const auto& scriptarg : m_subdescriptor_args) {
if (pos++) ret += ",";
std::string tmp;
if (!scriptarg->ToStringHelper(arg, tmp, type, cache)) return false;
ret += std::move(tmp);
}
return true;
}
bool ToStringHelper(const SigningProvider* arg, std::string& out, const StringType type, const DescriptorCache* cache = nullptr) const
{ {
std::string extra = ToStringExtra(); std::string extra = ToStringExtra();
size_t pos = extra.size() > 0 ? 1 : 0; size_t pos = extra.size() > 0 ? 1 : 0;
@ -532,42 +561,43 @@ public:
for (const auto& pubkey : m_pubkey_args) { for (const auto& pubkey : m_pubkey_args) {
if (pos++) ret += ","; if (pos++) ret += ",";
std::string tmp; std::string tmp;
if (normalized) { switch (type) {
if (!pubkey->ToNormalizedString(*arg, tmp, priv)) return false; case StringType::NORMALIZED:
} else if (priv) { if (!pubkey->ToNormalizedString(*arg, tmp, cache)) return false;
if (!pubkey->ToPrivateString(*arg, tmp)) return false; break;
} else { case StringType::PRIVATE:
tmp = pubkey->ToString(); if (!pubkey->ToPrivateString(*arg, tmp)) return false;
break;
case StringType::PUBLIC:
tmp = pubkey->ToString();
break;
} }
ret += std::move(tmp); ret += std::move(tmp);
} }
if (m_subdescriptor_arg) { std::string subscript;
if (pos++) ret += ","; if (!ToStringSubScriptHelper(arg, subscript, type, cache)) return false;
std::string tmp; if (pos && subscript.size()) ret += ',';
if (!m_subdescriptor_arg->ToStringHelper(arg, tmp, priv, normalized)) return false; out = std::move(ret) + std::move(subscript) + ")";
ret += std::move(tmp);
}
out = std::move(ret) + ")";
return true; return true;
} }
std::string ToString() const final std::string ToString() const final
{ {
std::string ret; std::string ret;
ToStringHelper(nullptr, ret, false, false); ToStringHelper(nullptr, ret, StringType::PUBLIC);
return AddChecksum(ret); return AddChecksum(ret);
} }
bool ToPrivateString(const SigningProvider& arg, std::string& out) const override final bool ToPrivateString(const SigningProvider& arg, std::string& out) const override final
{ {
bool ret = ToStringHelper(&arg, out, true, false); bool ret = ToStringHelper(&arg, out, StringType::PRIVATE);
out = AddChecksum(out); out = AddChecksum(out);
return ret; return ret;
} }
bool ToNormalizedString(const SigningProvider& arg, std::string& out, bool priv) const override final bool ToNormalizedString(const SigningProvider& arg, std::string& out, const DescriptorCache* cache) const override final
{ {
bool ret = ToStringHelper(&arg, out, priv, true); bool ret = ToStringHelper(&arg, out, StringType::NORMALIZED, cache);
out = AddChecksum(out); out = AddChecksum(out);
return ret; return ret;
} }
@ -577,17 +607,20 @@ public:
std::vector<std::pair<CPubKey, KeyOriginInfo>> entries; std::vector<std::pair<CPubKey, KeyOriginInfo>> entries;
entries.reserve(m_pubkey_args.size()); entries.reserve(m_pubkey_args.size());
// Construct temporary data in `entries` and `subscripts`, to avoid producing output in case of failure. // Construct temporary data in `entries`, `subscripts`, and `subprovider` to avoid producing output in case of failure.
for (const auto& p : m_pubkey_args) { for (const auto& p : m_pubkey_args) {
entries.emplace_back(); entries.emplace_back();
if (!p->GetPubKey(pos, arg, entries.back().first, entries.back().second, read_cache, write_cache)) return false; if (!p->GetPubKey(pos, arg, entries.back().first, entries.back().second, read_cache, write_cache)) return false;
} }
std::vector<CScript> subscripts; std::vector<CScript> subscripts;
if (m_subdescriptor_arg) { FlatSigningProvider subprovider;
FlatSigningProvider subprovider; for (const auto& subarg : m_subdescriptor_args) {
if (!m_subdescriptor_arg->ExpandHelper(pos, arg, read_cache, subscripts, subprovider, write_cache)) return false; std::vector<CScript> outscripts;
out = Merge(out, subprovider); if (!subarg->ExpandHelper(pos, arg, read_cache, outscripts, subprovider, write_cache)) return false;
assert(outscripts.size() == 1);
subscripts.emplace_back(std::move(outscripts[0]));
} }
out = Merge(std::move(out), std::move(subprovider));
std::vector<CPubKey> pubkeys; std::vector<CPubKey> pubkeys;
pubkeys.reserve(entries.size()); pubkeys.reserve(entries.size());
@ -595,17 +628,8 @@ public:
pubkeys.push_back(entry.first); pubkeys.push_back(entry.first);
out.origins.emplace(entry.first.GetID(), std::make_pair<CPubKey, KeyOriginInfo>(CPubKey(entry.first), std::move(entry.second))); out.origins.emplace(entry.first.GetID(), std::make_pair<CPubKey, KeyOriginInfo>(CPubKey(entry.first), std::move(entry.second)));
} }
if (m_subdescriptor_arg) {
for (const auto& subscript : subscripts) { output_scripts = MakeScripts(pubkeys, Span{subscripts}, out);
out.scripts.emplace(CScriptID(subscript), subscript);
std::vector<CScript> addscripts = MakeScripts(pubkeys, &subscript, out);
for (auto& addscript : addscripts) {
output_scripts.push_back(std::move(addscript));
}
}
} else {
output_scripts = MakeScripts(pubkeys, nullptr, out);
}
return true; return true;
} }
@ -626,10 +650,8 @@ public:
if (!p->GetPrivKey(pos, provider, key)) continue; if (!p->GetPrivKey(pos, provider, key)) continue;
out.keys.emplace(key.GetPubKey().GetID(), key); out.keys.emplace(key.GetPubKey().GetID(), key);
} }
if (m_subdescriptor_arg) { for (const auto& arg : m_subdescriptor_args) {
FlatSigningProvider subprovider; arg->ExpandPrivate(pos, provider, out);
m_subdescriptor_arg->ExpandPrivate(pos, provider, subprovider);
out = Merge(out, subprovider);
} }
} }
std::optional<OutputType> GetOutputType() const override { return std::nullopt; } std::optional<OutputType> GetOutputType() const override { return std::nullopt; }
@ -641,9 +663,9 @@ class AddressDescriptor final : public DescriptorImpl
const CTxDestination m_destination; const CTxDestination m_destination;
protected: protected:
std::string ToStringExtra() const override { return EncodeDestination(m_destination); } std::string ToStringExtra() const override { return EncodeDestination(m_destination); }
std::vector<CScript> MakeScripts(const std::vector<CPubKey>&, const CScript*, FlatSigningProvider&) const override { return Vector(GetScriptForDestination(m_destination)); } std::vector<CScript> MakeScripts(const std::vector<CPubKey>&, Span<const CScript>, FlatSigningProvider&) const override { return Vector(GetScriptForDestination(m_destination)); }
public: public:
AddressDescriptor(CTxDestination destination) : DescriptorImpl({}, {}, "addr"), m_destination(std::move(destination)) {} AddressDescriptor(CTxDestination destination) : DescriptorImpl({}, "addr"), m_destination(std::move(destination)) {}
bool IsSolvable() const final { return false; } bool IsSolvable() const final { return false; }
std::optional<OutputType> GetOutputType() const override std::optional<OutputType> GetOutputType() const override
@ -664,9 +686,9 @@ class RawDescriptor final : public DescriptorImpl
const CScript m_script; const CScript m_script;
protected: protected:
std::string ToStringExtra() const override { return HexStr(m_script); } std::string ToStringExtra() const override { return HexStr(m_script); }
std::vector<CScript> MakeScripts(const std::vector<CPubKey>&, const CScript*, FlatSigningProvider&) const override { return Vector(m_script); } std::vector<CScript> MakeScripts(const std::vector<CPubKey>&, Span<const CScript>, FlatSigningProvider&) const override { return Vector(m_script); }
public: public:
RawDescriptor(CScript script) : DescriptorImpl({}, {}, "raw"), m_script(std::move(script)) {} RawDescriptor(CScript script) : DescriptorImpl({}, "raw"), m_script(std::move(script)) {}
bool IsSolvable() const final { return false; } bool IsSolvable() const final { return false; }
std::optional<OutputType> GetOutputType() const override std::optional<OutputType> GetOutputType() const override
@ -687,9 +709,9 @@ public:
class PKDescriptor final : public DescriptorImpl class PKDescriptor final : public DescriptorImpl
{ {
protected: protected:
std::vector<CScript> MakeScripts(const std::vector<CPubKey>& keys, const CScript*, FlatSigningProvider&) const override { return Vector(GetScriptForRawPubKey(keys[0])); } std::vector<CScript> MakeScripts(const std::vector<CPubKey>& keys, Span<const CScript>, FlatSigningProvider&) const override { return Vector(GetScriptForRawPubKey(keys[0])); }
public: public:
PKDescriptor(std::unique_ptr<PubkeyProvider> prov) : DescriptorImpl(Vector(std::move(prov)), {}, "pk") {} PKDescriptor(std::unique_ptr<PubkeyProvider> prov) : DescriptorImpl(Vector(std::move(prov)), "pk") {}
std::optional<OutputType> GetOutputType() const override { return OutputType::LEGACY; } std::optional<OutputType> GetOutputType() const override { return OutputType::LEGACY; }
bool IsSingleType() const final { return true; } bool IsSingleType() const final { return true; }
}; };
@ -698,14 +720,14 @@ public:
class PKHDescriptor final : public DescriptorImpl class PKHDescriptor final : public DescriptorImpl
{ {
protected: protected:
std::vector<CScript> MakeScripts(const std::vector<CPubKey>& keys, const CScript*, FlatSigningProvider& out) const override std::vector<CScript> MakeScripts(const std::vector<CPubKey>& keys, Span<const CScript>, FlatSigningProvider& out) const override
{ {
CKeyID id = keys[0].GetID(); CKeyID id = keys[0].GetID();
out.pubkeys.emplace(id, keys[0]); out.pubkeys.emplace(id, keys[0]);
return Vector(GetScriptForDestination(PKHash(id))); return Vector(GetScriptForDestination(PKHash(id)));
} }
public: public:
PKHDescriptor(std::unique_ptr<PubkeyProvider> prov) : DescriptorImpl(Vector(std::move(prov)), {}, "pkh") {} PKHDescriptor(std::unique_ptr<PubkeyProvider> prov) : DescriptorImpl(Vector(std::move(prov)), "pkh") {}
std::optional<OutputType> GetOutputType() const override { return OutputType::LEGACY; } std::optional<OutputType> GetOutputType() const override { return OutputType::LEGACY; }
bool IsSingleType() const final { return true; } bool IsSingleType() const final { return true; }
}; };
@ -717,7 +739,7 @@ class MultisigDescriptor final : public DescriptorImpl
const bool m_sorted; const bool m_sorted;
protected: protected:
std::string ToStringExtra() const override { return strprintf("%i", m_threshold); } std::string ToStringExtra() const override { return strprintf("%i", m_threshold); }
std::vector<CScript> MakeScripts(const std::vector<CPubKey>& keys, const CScript*, FlatSigningProvider&) const override { std::vector<CScript> MakeScripts(const std::vector<CPubKey>& keys, Span<const CScript>, FlatSigningProvider&) const override {
if (m_sorted) { if (m_sorted) {
std::vector<CPubKey> sorted_keys(keys); std::vector<CPubKey> sorted_keys(keys);
std::sort(sorted_keys.begin(), sorted_keys.end()); std::sort(sorted_keys.begin(), sorted_keys.end());
@ -726,7 +748,7 @@ protected:
return Vector(GetScriptForMultisig(m_threshold, keys)); return Vector(GetScriptForMultisig(m_threshold, keys));
} }
public: public:
MultisigDescriptor(int threshold, std::vector<std::unique_ptr<PubkeyProvider>> providers, bool sorted = false) : DescriptorImpl(std::move(providers), {}, sorted ? "sortedmulti" : "multi"), m_threshold(threshold), m_sorted(sorted) {} MultisigDescriptor(int threshold, std::vector<std::unique_ptr<PubkeyProvider>> providers, bool sorted = false) : DescriptorImpl(std::move(providers), sorted ? "sortedmulti" : "multi"), m_threshold(threshold), m_sorted(sorted) {}
bool IsSingleType() const final { return true; } bool IsSingleType() const final { return true; }
}; };
@ -734,13 +756,18 @@ public:
class SHDescriptor final : public DescriptorImpl class SHDescriptor final : public DescriptorImpl
{ {
protected: protected:
std::vector<CScript> MakeScripts(const std::vector<CPubKey>&, const CScript* script, FlatSigningProvider&) const override { return Vector(GetScriptForDestination(ScriptHash(*script))); } std::vector<CScript> MakeScripts(const std::vector<CPubKey>&, Span<const CScript> scripts, FlatSigningProvider& out) const override
{
auto ret = Vector(GetScriptForDestination(ScriptHash(scripts[0])));
if (ret.size()) out.scripts.emplace(CScriptID(scripts[0]), scripts[0]);
return ret;
}
public: public:
SHDescriptor(std::unique_ptr<DescriptorImpl> desc) : DescriptorImpl({}, std::move(desc), "sh") {} SHDescriptor(std::unique_ptr<DescriptorImpl> desc) : DescriptorImpl({}, std::move(desc), "sh") {}
std::optional<OutputType> GetOutputType() const override std::optional<OutputType> GetOutputType() const override
{ {
assert(m_subdescriptor_arg); assert(m_subdescriptor_args.size() == 1);
return OutputType::LEGACY; return OutputType::LEGACY;
} }
bool IsSingleType() const final { return true; } bool IsSingleType() const final { return true; }
@ -750,7 +777,7 @@ public:
class ComboDescriptor final : public DescriptorImpl class ComboDescriptor final : public DescriptorImpl
{ {
protected: protected:
std::vector<CScript> MakeScripts(const std::vector<CPubKey>& keys, const CScript*, FlatSigningProvider& out) const override std::vector<CScript> MakeScripts(const std::vector<CPubKey>& keys, Span<const CScript> scripts, FlatSigningProvider& out) const override
{ {
std::vector<CScript> ret; std::vector<CScript> ret;
CKeyID id = keys[0].GetID(); CKeyID id = keys[0].GetID();
@ -766,7 +793,7 @@ protected:
} }
public: public:
ComboDescriptor(std::unique_ptr<PubkeyProvider> prov) : DescriptorImpl(Vector(std::move(prov)), {}, "combo") {} ComboDescriptor(std::unique_ptr<PubkeyProvider> prov) : DescriptorImpl(Vector(std::move(prov)), "combo") {}
std::optional<OutputType> GetOutputType() const override { return OutputType::LEGACY; } std::optional<OutputType> GetOutputType() const override { return OutputType::LEGACY; }
bool IsSingleType() const final { return false; } bool IsSingleType() const final { return false; }
}; };
@ -776,8 +803,8 @@ public:
//////////////////////////////////////////////////////////////////////////// ////////////////////////////////////////////////////////////////////////////
enum class ParseScriptContext { enum class ParseScriptContext {
TOP, TOP, //!< Top-level context (script goes directly in scriptPubKey)
P2SH P2SH, //!< Inside sh() (script becomes P2SH redeemScript)
}; };
/** Parse a key path, being passed a split list of elements (the first element is ignored). */ /** Parse a key path, being passed a split list of elements (the first element is ignored). */
@ -804,10 +831,11 @@ enum class ParseScriptContext {
} }
/** Parse a public key that excludes origin information. */ /** Parse a public key that excludes origin information. */
std::unique_ptr<PubkeyProvider> ParsePubkeyInner(uint32_t key_exp_index, const Span<const char>& sp, bool permit_uncompressed, FlatSigningProvider& out, std::string& error) std::unique_ptr<PubkeyProvider> ParsePubkeyInner(uint32_t key_exp_index, const Span<const char>& sp, ParseScriptContext ctx, FlatSigningProvider& out, std::string& error)
{ {
using namespace spanparsing; using namespace spanparsing;
bool permit_uncompressed = ctx == ParseScriptContext::TOP || ctx == ParseScriptContext::P2SH;
auto split = Split(sp, '/'); auto split = Split(sp, '/');
std::string str(split[0].begin(), split[0].end()); std::string str(split[0].begin(), split[0].end());
if (str.size() == 0) { if (str.size() == 0) {
@ -865,7 +893,7 @@ std::unique_ptr<PubkeyProvider> ParsePubkeyInner(uint32_t key_exp_index, const S
} }
/** Parse a public key including origin information (if enabled). */ /** Parse a public key including origin information (if enabled). */
std::unique_ptr<PubkeyProvider> ParsePubkey(uint32_t key_exp_index, const Span<const char>& sp, bool permit_uncompressed, FlatSigningProvider& out, std::string& error) std::unique_ptr<PubkeyProvider> ParsePubkey(uint32_t key_exp_index, const Span<const char>& sp, ParseScriptContext ctx, FlatSigningProvider& out, std::string& error)
{ {
using namespace spanparsing; using namespace spanparsing;
@ -874,7 +902,7 @@ std::unique_ptr<PubkeyProvider> ParsePubkey(uint32_t key_exp_index, const Span<c
error = "Multiple ']' characters found for a single pubkey"; error = "Multiple ']' characters found for a single pubkey";
return nullptr; return nullptr;
} }
if (origin_split.size() == 1) return ParsePubkeyInner(key_exp_index, origin_split[0], permit_uncompressed, out, error); if (origin_split.size() == 1) return ParsePubkeyInner(key_exp_index, origin_split[0], ctx, out, error);
if (origin_split[0].empty() || origin_split[0][0] != '[') { if (origin_split[0].empty() || origin_split[0][0] != '[') {
error = strprintf("Key origin start '[ character expected but not found, got '%c' instead", error = strprintf("Key origin start '[ character expected but not found, got '%c' instead",
origin_split[0].empty() ? /** empty, implies split char */ ']' : origin_split[0][0]); origin_split[0].empty() ? /** empty, implies split char */ ']' : origin_split[0][0]);
@ -896,34 +924,37 @@ std::unique_ptr<PubkeyProvider> ParsePubkey(uint32_t key_exp_index, const Span<c
assert(fpr_bytes.size() == 4); assert(fpr_bytes.size() == 4);
std::copy(fpr_bytes.begin(), fpr_bytes.end(), info.fingerprint); std::copy(fpr_bytes.begin(), fpr_bytes.end(), info.fingerprint);
if (!ParseKeyPath(slash_split, info.path, error)) return nullptr; if (!ParseKeyPath(slash_split, info.path, error)) return nullptr;
auto provider = ParsePubkeyInner(key_exp_index, origin_split[1], permit_uncompressed, out, error); auto provider = ParsePubkeyInner(key_exp_index, origin_split[1], ctx, out, error);
if (!provider) return nullptr; if (!provider) return nullptr;
return std::make_unique<OriginPubkeyProvider>(key_exp_index, std::move(info), std::move(provider)); return std::make_unique<OriginPubkeyProvider>(key_exp_index, std::move(info), std::move(provider));
} }
/** Parse a script in a particular context. */ /** Parse a script in a particular context. */
std::unique_ptr<DescriptorImpl> ParseScript(uint32_t key_exp_index, Span<const char>& sp, ParseScriptContext ctx, FlatSigningProvider& out, std::string& error) std::unique_ptr<DescriptorImpl> ParseScript(uint32_t& key_exp_index, Span<const char>& sp, ParseScriptContext ctx, FlatSigningProvider& out, std::string& error)
{ {
using namespace spanparsing; using namespace spanparsing;
auto expr = Expr(sp); auto expr = Expr(sp);
bool sorted_multi = false; bool sorted_multi = false;
if (Func("pk", expr)) { if (Func("pk", expr)) {
auto pubkey = ParsePubkey(key_exp_index, expr, true, out, error); auto pubkey = ParsePubkey(key_exp_index, expr, ctx, out, error);
if (!pubkey) return nullptr; if (!pubkey) return nullptr;
++key_exp_index;
return std::make_unique<PKDescriptor>(std::move(pubkey)); return std::make_unique<PKDescriptor>(std::move(pubkey));
} }
if (Func("pkh", expr)) { if (Func("pkh", expr)) {
auto pubkey = ParsePubkey(key_exp_index, expr, true, out, error); auto pubkey = ParsePubkey(key_exp_index, expr, ctx, out, error);
if (!pubkey) return nullptr; if (!pubkey) return nullptr;
++key_exp_index;
return std::make_unique<PKHDescriptor>(std::move(pubkey)); return std::make_unique<PKHDescriptor>(std::move(pubkey));
} }
if (ctx == ParseScriptContext::TOP && Func("combo", expr)) { if (ctx == ParseScriptContext::TOP && Func("combo", expr)) {
auto pubkey = ParsePubkey(key_exp_index, expr, true, out, error); auto pubkey = ParsePubkey(key_exp_index, expr, ctx, out, error);
if (!pubkey) return nullptr; if (!pubkey) return nullptr;
++key_exp_index;
return std::make_unique<ComboDescriptor>(std::move(pubkey)); return std::make_unique<ComboDescriptor>(std::move(pubkey));
} else if (ctx != ParseScriptContext::TOP && Func("combo", expr)) { } else if (Func("combo", expr)) {
error = "Cannot have combo in non-top level"; error = "Can only have combo() at top level";
return nullptr; return nullptr;
} }
if ((sorted_multi = Func("sortedmulti", expr)) || Func("multi", expr)) { if ((sorted_multi = Func("sortedmulti", expr)) || Func("multi", expr)) {
@ -941,7 +972,7 @@ std::unique_ptr<DescriptorImpl> ParseScript(uint32_t key_exp_index, Span<const c
return nullptr; return nullptr;
} }
auto arg = Expr(expr); auto arg = Expr(expr);
auto pk = ParsePubkey(key_exp_index, arg, true, out, error); auto pk = ParsePubkey(key_exp_index, arg, ctx, out, error);
if (!pk) return nullptr; if (!pk) return nullptr;
script_size += pk->GetSize() + 1; script_size += pk->GetSize() + 1;
providers.emplace_back(std::move(pk)); providers.emplace_back(std::move(pk));
@ -975,8 +1006,8 @@ std::unique_ptr<DescriptorImpl> ParseScript(uint32_t key_exp_index, Span<const c
auto desc = ParseScript(key_exp_index, expr, ParseScriptContext::P2SH, out, error); auto desc = ParseScript(key_exp_index, expr, ParseScriptContext::P2SH, out, error);
if (!desc || expr.size()) return nullptr; if (!desc || expr.size()) return nullptr;
return std::make_unique<SHDescriptor>(std::move(desc)); return std::make_unique<SHDescriptor>(std::move(desc));
} else if (ctx != ParseScriptContext::TOP && Func("sh", expr)) { } else if (Func("sh", expr)) {
error = "Cannot have sh in non-top level"; error = "Can only have sh() at top level";
return nullptr; return nullptr;
} }
if (ctx == ParseScriptContext::TOP && Func("addr", expr)) { if (ctx == ParseScriptContext::TOP && Func("addr", expr)) {
@ -986,6 +1017,9 @@ std::unique_ptr<DescriptorImpl> ParseScript(uint32_t key_exp_index, Span<const c
return nullptr; return nullptr;
} }
return std::make_unique<AddressDescriptor>(std::move(dest)); return std::make_unique<AddressDescriptor>(std::move(dest));
} else if (Func("addr", expr)) {
error = "Can only have addr() at top level";
return nullptr;
} }
if (ctx == ParseScriptContext::TOP && Func("raw", expr)) { if (ctx == ParseScriptContext::TOP && Func("raw", expr)) {
std::string str(expr.begin(), expr.end()); std::string str(expr.begin(), expr.end());
@ -995,6 +1029,9 @@ std::unique_ptr<DescriptorImpl> ParseScript(uint32_t key_exp_index, Span<const c
} }
auto bytes = ParseHex(str); auto bytes = ParseHex(str);
return std::make_unique<RawDescriptor>(CScript(bytes.begin(), bytes.end())); return std::make_unique<RawDescriptor>(CScript(bytes.begin(), bytes.end()));
} else if (Func("raw", expr)) {
error = "Can only have raw() at top level";
return nullptr;
} }
if (ctx == ParseScriptContext::P2SH) { if (ctx == ParseScriptContext::P2SH) {
error = "A function is needed within P2SH"; error = "A function is needed within P2SH";
@ -1104,7 +1141,8 @@ std::unique_ptr<Descriptor> Parse(const std::string& descriptor, FlatSigningProv
{ {
Span<const char> sp{descriptor}; Span<const char> sp{descriptor};
if (!CheckChecksum(sp, require_checksum, error)) return nullptr; if (!CheckChecksum(sp, require_checksum, error)) return nullptr;
auto ret = ParseScript(0, sp, ParseScriptContext::TOP, out, error); uint32_t key_exp_index = 0;
auto ret = ParseScript(key_exp_index, sp, ParseScriptContext::TOP, out, error);
if (sp.size() == 0 && ret) return std::unique_ptr<Descriptor>(std::move(ret)); if (sp.size() == 0 && ret) return std::unique_ptr<Descriptor>(std::move(ret));
return nullptr; return nullptr;
} }
@ -1134,6 +1172,11 @@ void DescriptorCache::CacheDerivedExtPubKey(uint32_t key_exp_pos, uint32_t der_i
xpubs[der_index] = xpub; xpubs[der_index] = xpub;
} }
void DescriptorCache::CacheLastHardenedExtPubKey(uint32_t key_exp_pos, const CExtPubKey& xpub)
{
m_last_hardened_xpubs[key_exp_pos] = xpub;
}
bool DescriptorCache::GetCachedParentExtPubKey(uint32_t key_exp_pos, CExtPubKey& xpub) const bool DescriptorCache::GetCachedParentExtPubKey(uint32_t key_exp_pos, CExtPubKey& xpub) const
{ {
const auto& it = m_parent_xpubs.find(key_exp_pos); const auto& it = m_parent_xpubs.find(key_exp_pos);
@ -1152,6 +1195,55 @@ bool DescriptorCache::GetCachedDerivedExtPubKey(uint32_t key_exp_pos, uint32_t d
return true; return true;
} }
bool DescriptorCache::GetCachedLastHardenedExtPubKey(uint32_t key_exp_pos, CExtPubKey& xpub) const
{
const auto& it = m_last_hardened_xpubs.find(key_exp_pos);
if (it == m_last_hardened_xpubs.end()) return false;
xpub = it->second;
return true;
}
DescriptorCache DescriptorCache::MergeAndDiff(const DescriptorCache& other)
{
DescriptorCache diff;
for (const auto& parent_xpub_pair : other.GetCachedParentExtPubKeys()) {
CExtPubKey xpub;
if (GetCachedParentExtPubKey(parent_xpub_pair.first, xpub)) {
if (xpub != parent_xpub_pair.second) {
throw std::runtime_error(std::string(__func__) + ": New cached parent xpub does not match already cached parent xpub");
}
continue;
}
CacheParentExtPubKey(parent_xpub_pair.first, parent_xpub_pair.second);
diff.CacheParentExtPubKey(parent_xpub_pair.first, parent_xpub_pair.second);
}
for (const auto& derived_xpub_map_pair : other.GetCachedDerivedExtPubKeys()) {
for (const auto& derived_xpub_pair : derived_xpub_map_pair.second) {
CExtPubKey xpub;
if (GetCachedDerivedExtPubKey(derived_xpub_map_pair.first, derived_xpub_pair.first, xpub)) {
if (xpub != derived_xpub_pair.second) {
throw std::runtime_error(std::string(__func__) + ": New cached derived xpub does not match already cached derived xpub");
}
continue;
}
CacheDerivedExtPubKey(derived_xpub_map_pair.first, derived_xpub_pair.first, derived_xpub_pair.second);
diff.CacheDerivedExtPubKey(derived_xpub_map_pair.first, derived_xpub_pair.first, derived_xpub_pair.second);
}
}
for (const auto& lh_xpub_pair : other.GetCachedLastHardenedExtPubKeys()) {
CExtPubKey xpub;
if (GetCachedLastHardenedExtPubKey(lh_xpub_pair.first, xpub)) {
if (xpub != lh_xpub_pair.second) {
throw std::runtime_error(std::string(__func__) + ": New cached last hardened xpub does not match already cached last hardened xpub");
}
continue;
}
CacheLastHardenedExtPubKey(lh_xpub_pair.first, lh_xpub_pair.second);
diff.CacheLastHardenedExtPubKey(lh_xpub_pair.first, lh_xpub_pair.second);
}
return diff;
}
const ExtPubKeyMap DescriptorCache::GetCachedParentExtPubKeys() const const ExtPubKeyMap DescriptorCache::GetCachedParentExtPubKeys() const
{ {
return m_parent_xpubs; return m_parent_xpubs;
@ -1161,3 +1253,8 @@ const std::unordered_map<uint32_t, ExtPubKeyMap> DescriptorCache::GetCachedDeriv
{ {
return m_derived_xpubs; return m_derived_xpubs;
} }
const ExtPubKeyMap DescriptorCache::GetCachedLastHardenedExtPubKeys() const
{
return m_last_hardened_xpubs;
}

23
src/script/descriptor.h
View File

@ -22,6 +22,8 @@ private:
std::unordered_map<uint32_t, ExtPubKeyMap> m_derived_xpubs; std::unordered_map<uint32_t, ExtPubKeyMap> m_derived_xpubs;
/** Map key expression index -> parent xpub */ /** Map key expression index -> parent xpub */
ExtPubKeyMap m_parent_xpubs; ExtPubKeyMap m_parent_xpubs;
/** Map key expression index -> last hardened xpub */
ExtPubKeyMap m_last_hardened_xpubs;
public: public:
/** Cache a parent xpub /** Cache a parent xpub
@ -50,11 +52,30 @@ public:
* @param[in] xpub The CExtPubKey to get from cache * @param[in] xpub The CExtPubKey to get from cache
*/ */
bool GetCachedDerivedExtPubKey(uint32_t key_exp_pos, uint32_t der_index, CExtPubKey& xpub) const; bool GetCachedDerivedExtPubKey(uint32_t key_exp_pos, uint32_t der_index, CExtPubKey& xpub) const;
/** Cache a last hardened xpub
*
* @param[in] key_exp_pos Position of the key expression within the descriptor
* @param[in] xpub The CExtPubKey to cache
*/
void CacheLastHardenedExtPubKey(uint32_t key_exp_pos, const CExtPubKey& xpub);
/** Retrieve a cached last hardened xpub
*
* @param[in] key_exp_pos Position of the key expression within the descriptor
* @param[in] xpub The CExtPubKey to get from cache
*/
bool GetCachedLastHardenedExtPubKey(uint32_t key_exp_pos, CExtPubKey& xpub) const;
/** Retrieve all cached parent xpubs */ /** Retrieve all cached parent xpubs */
const ExtPubKeyMap GetCachedParentExtPubKeys() const; const ExtPubKeyMap GetCachedParentExtPubKeys() const;
/** Retrieve all cached derived xpubs */ /** Retrieve all cached derived xpubs */
const std::unordered_map<uint32_t, ExtPubKeyMap> GetCachedDerivedExtPubKeys() const; const std::unordered_map<uint32_t, ExtPubKeyMap> GetCachedDerivedExtPubKeys() const;
/** Retrieve all cached last hardened xpubs */
const ExtPubKeyMap GetCachedLastHardenedExtPubKeys() const;
/** Combine another DescriptorCache into this one.
* Returns a cache containing the items from the other cache unknown to current cache
*/
DescriptorCache MergeAndDiff(const DescriptorCache& other);
}; };
/** \brief Interface for parsed descriptor objects. /** \brief Interface for parsed descriptor objects.
@ -94,7 +115,7 @@ struct Descriptor {
virtual bool ToPrivateString(const SigningProvider& provider, std::string& out) const = 0; virtual bool ToPrivateString(const SigningProvider& provider, std::string& out) const = 0;
/** Convert the descriptor to a normalized string. Normalized descriptors have the xpub at the last hardened step. This fails if the provided provider does not have the private keys to derive that xpub. */ /** Convert the descriptor to a normalized string. Normalized descriptors have the xpub at the last hardened step. This fails if the provided provider does not have the private keys to derive that xpub. */
virtual bool ToNormalizedString(const SigningProvider& provider, std::string& out, bool priv) const = 0; virtual bool ToNormalizedString(const SigningProvider& provider, std::string& out, const DescriptorCache* cache = nullptr) const = 0;
/** Expand a descriptor at a specified position. /** Expand a descriptor at a specified position.
* *

15
src/test/descriptor_tests.cpp
View File

@ -24,7 +24,7 @@ void CheckUnparsable(const std::string& prv, const std::string& pub, const std::
auto parse_pub = Parse(pub, keys_pub, error); auto parse_pub = Parse(pub, keys_pub, error);
BOOST_CHECK_MESSAGE(!parse_priv, prv); BOOST_CHECK_MESSAGE(!parse_priv, prv);
BOOST_CHECK_MESSAGE(!parse_pub, pub); BOOST_CHECK_MESSAGE(!parse_pub, pub);
BOOST_CHECK(error == expected_error); BOOST_CHECK_EQUAL(error, expected_error);
} }
constexpr int DEFAULT = 0; constexpr int DEFAULT = 0;
@ -115,14 +115,10 @@ void DoCheck(const std::string& prv, const std::string& pub, const std::string&
// Check that private can produce the normalized descriptors // Check that private can produce the normalized descriptors
std::string norm1; std::string norm1;
BOOST_CHECK(parse_priv->ToNormalizedString(keys_priv, norm1, false)); BOOST_CHECK(parse_priv->ToNormalizedString(keys_priv, norm1));
BOOST_CHECK(EqualDescriptor(norm1, norm_pub)); BOOST_CHECK(EqualDescriptor(norm1, norm_pub));
BOOST_CHECK(parse_pub->ToNormalizedString(keys_priv, norm1, false)); BOOST_CHECK(parse_pub->ToNormalizedString(keys_priv, norm1));
BOOST_CHECK(EqualDescriptor(norm1, norm_pub)); BOOST_CHECK(EqualDescriptor(norm1, norm_pub));
BOOST_CHECK(parse_priv->ToNormalizedString(keys_priv, norm1, true));
BOOST_CHECK(EqualDescriptor(norm1, norm_prv));
BOOST_CHECK(parse_pub->ToNormalizedString(keys_priv, norm1, true));
BOOST_CHECK(EqualDescriptor(norm1, norm_prv));
// Check whether IsRange on both returns the expected result // Check whether IsRange on both returns the expected result
BOOST_CHECK_EQUAL(parse_pub->IsRange(), (flags & RANGE) != 0); BOOST_CHECK_EQUAL(parse_pub->IsRange(), (flags & RANGE) != 0);
@ -335,9 +331,8 @@ BOOST_AUTO_TEST_CASE(descriptor_test)
// Check for invalid nesting of structures // Check for invalid nesting of structures
CheckUnparsable("sh(XJvEUEcFWCHCyruc8ZX5exPZaGe4UR7gC5FHrhwPnQGDs1uWCsT2)", "sh(03a34b99f22c790c4e36b2b3c2c35a36db06226e41c692fc82b8b56ac1c540c5bd)", "A function is needed within P2SH"); // P2SH needs a script, not a key CheckUnparsable("sh(XJvEUEcFWCHCyruc8ZX5exPZaGe4UR7gC5FHrhwPnQGDs1uWCsT2)", "sh(03a34b99f22c790c4e36b2b3c2c35a36db06226e41c692fc82b8b56ac1c540c5bd)", "A function is needed within P2SH"); // P2SH needs a script, not a key
CheckUnparsable("sh(sh(pk(XJvEUEcFWCHCyruc8ZX5exPZaGe4UR7gC5FHrhwPnQGDs1uWCsT2)))", "sh(sh(pk(03a34b99f22c790c4e36b2b3c2c35a36db06226e41c692fc82b8b56ac1c540c5bd)))", "Cannot have sh in non-top level"); // Cannot embed P2SH inside P2SH CheckUnparsable("sh(sh(pk(XJvEUEcFWCHCyruc8ZX5exPZaGe4UR7gC5FHrhwPnQGDs1uWCsT2)))", "sh(sh(pk(03a34b99f22c790c4e36b2b3c2c35a36db06226e41c692fc82b8b56ac1c540c5bd)))", "Can only have sh() at top level"); // Cannot embed P2SH inside P2SH
CheckUnparsable("sh(combo(XJvEUEcFWCHCyruc8ZX5exPZaGe4UR7gC5FHrhwPnQGDs1uWCsT2))", "sh(combo(03a34b99f22c790c4e36b2b3c2c35a36db06226e41c692fc82b8b56ac1c540c5bd))", "Cannot have combo in non-top level"); // Old must be top level CheckUnparsable("sh(combo(XJvEUEcFWCHCyruc8ZX5exPZaGe4UR7gC5FHrhwPnQGDs1uWCsT2))", "sh(combo(03a34b99f22c790c4e36b2b3c2c35a36db06226e41c692fc82b8b56ac1c540c5bd))", "Can only have combo() at top level"); // Old must be top level
// Checksums // Checksums
Check("sh(multi(2,[00000000/111'/222]xprvA1RpRA33e1JQ7ifknakTFpgNXPmW2YvmhqLQYMmrj4xJXXWYpDPS3xz7iAxn8L39njGVyuoseXzU6rcxFLJ8HFsTjSyQbLYnMpCqE2VbFWc,xprv9uPDJpEQgRQfDcW7BkF7eTya6RPxXeJCqCJGHuCJ4GiRVLzkTXBAJMu2qaMWPrS7AANYqdq6vcBcBUdJCVVFceUvJFjaPdGZ2y9WACViL4L/0))#ggrsrxfy", "sh(multi(2,[00000000/111'/222]xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL,xpub68NZiKmJWnxxS6aaHmn81bvJeTESw724CRDs6HbuccFQN9Ku14VQrADWgqbhhTHBaohPX4CjNLf9fq9MYo6oDaPPLPxSb7gwQN3ih19Zm4Y/0))#tjg09x5t", "sh(multi(2,[00000000/111'/222]xprvA1RpRA33e1JQ7ifknakTFpgNXPmW2YvmhqLQYMmrj4xJXXWYpDPS3xz7iAxn8L39njGVyuoseXzU6rcxFLJ8HFsTjSyQbLYnMpCqE2VbFWc,xprv9uPDJpEQgRQfDcW7BkF7eTya6RPxXeJCqCJGHuCJ4GiRVLzkTXBAJMu2qaMWPrS7AANYqdq6vcBcBUdJCVVFceUvJFjaPdGZ2y9WACViL4L/0))#ggrsrxfy", "sh(multi(2,[00000000/111'/222]xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL,xpub68NZiKmJWnxxS6aaHmn81bvJeTESw724CRDs6HbuccFQN9Ku14VQrADWgqbhhTHBaohPX4CjNLf9fq9MYo6oDaPPLPxSb7gwQN3ih19Zm4Y/0))#tjg09x5t", DEFAULT, {{"a91445a9a622a8b0a1269944be477640eedc447bbd8487"}}, OutputType::LEGACY, {{0x8000006FUL,222},{0}}); Check("sh(multi(2,[00000000/111'/222]xprvA1RpRA33e1JQ7ifknakTFpgNXPmW2YvmhqLQYMmrj4xJXXWYpDPS3xz7iAxn8L39njGVyuoseXzU6rcxFLJ8HFsTjSyQbLYnMpCqE2VbFWc,xprv9uPDJpEQgRQfDcW7BkF7eTya6RPxXeJCqCJGHuCJ4GiRVLzkTXBAJMu2qaMWPrS7AANYqdq6vcBcBUdJCVVFceUvJFjaPdGZ2y9WACViL4L/0))#ggrsrxfy", "sh(multi(2,[00000000/111'/222]xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL,xpub68NZiKmJWnxxS6aaHmn81bvJeTESw724CRDs6HbuccFQN9Ku14VQrADWgqbhhTHBaohPX4CjNLf9fq9MYo6oDaPPLPxSb7gwQN3ih19Zm4Y/0))#tjg09x5t", "sh(multi(2,[00000000/111'/222]xprvA1RpRA33e1JQ7ifknakTFpgNXPmW2YvmhqLQYMmrj4xJXXWYpDPS3xz7iAxn8L39njGVyuoseXzU6rcxFLJ8HFsTjSyQbLYnMpCqE2VbFWc,xprv9uPDJpEQgRQfDcW7BkF7eTya6RPxXeJCqCJGHuCJ4GiRVLzkTXBAJMu2qaMWPrS7AANYqdq6vcBcBUdJCVVFceUvJFjaPdGZ2y9WACViL4L/0))#ggrsrxfy", "sh(multi(2,[00000000/111'/222]xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL,xpub68NZiKmJWnxxS6aaHmn81bvJeTESw724CRDs6HbuccFQN9Ku14VQrADWgqbhhTHBaohPX4CjNLf9fq9MYo6oDaPPLPxSb7gwQN3ih19Zm4Y/0))#tjg09x5t", DEFAULT, {{"a91445a9a622a8b0a1269944be477640eedc447bbd8487"}}, OutputType::LEGACY, {{0x8000006FUL,222},{0}});
Check("sh(multi(2,[00000000/111'/222]xprvA1RpRA33e1JQ7ifknakTFpgNXPmW2YvmhqLQYMmrj4xJXXWYpDPS3xz7iAxn8L39njGVyuoseXzU6rcxFLJ8HFsTjSyQbLYnMpCqE2VbFWc,xprv9uPDJpEQgRQfDcW7BkF7eTya6RPxXeJCqCJGHuCJ4GiRVLzkTXBAJMu2qaMWPrS7AANYqdq6vcBcBUdJCVVFceUvJFjaPdGZ2y9WACViL4L/0))", "sh(multi(2,[00000000/111'/222]xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL,xpub68NZiKmJWnxxS6aaHmn81bvJeTESw724CRDs6HbuccFQN9Ku14VQrADWgqbhhTHBaohPX4CjNLf9fq9MYo6oDaPPLPxSb7gwQN3ih19Zm4Y/0))", "sh(multi(2,[00000000/111'/222]xprvA1RpRA33e1JQ7ifknakTFpgNXPmW2YvmhqLQYMmrj4xJXXWYpDPS3xz7iAxn8L39njGVyuoseXzU6rcxFLJ8HFsTjSyQbLYnMpCqE2VbFWc,xprv9uPDJpEQgRQfDcW7BkF7eTya6RPxXeJCqCJGHuCJ4GiRVLzkTXBAJMu2qaMWPrS7AANYqdq6vcBcBUdJCVVFceUvJFjaPdGZ2y9WACViL4L/0))", "sh(multi(2,[00000000/111'/222]xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL,xpub68NZiKmJWnxxS6aaHmn81bvJeTESw724CRDs6HbuccFQN9Ku14VQrADWgqbhhTHBaohPX4CjNLf9fq9MYo6oDaPPLPxSb7gwQN3ih19Zm4Y/0))", DEFAULT, {{"a91445a9a622a8b0a1269944be477640eedc447bbd8487"}}, OutputType::LEGACY, {{0x8000006FUL,222},{0}}); Check("sh(multi(2,[00000000/111'/222]xprvA1RpRA33e1JQ7ifknakTFpgNXPmW2YvmhqLQYMmrj4xJXXWYpDPS3xz7iAxn8L39njGVyuoseXzU6rcxFLJ8HFsTjSyQbLYnMpCqE2VbFWc,xprv9uPDJpEQgRQfDcW7BkF7eTya6RPxXeJCqCJGHuCJ4GiRVLzkTXBAJMu2qaMWPrS7AANYqdq6vcBcBUdJCVVFceUvJFjaPdGZ2y9WACViL4L/0))", "sh(multi(2,[00000000/111'/222]xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL,xpub68NZiKmJWnxxS6aaHmn81bvJeTESw724CRDs6HbuccFQN9Ku14VQrADWgqbhhTHBaohPX4CjNLf9fq9MYo6oDaPPLPxSb7gwQN3ih19Zm4Y/0))", "sh(multi(2,[00000000/111'/222]xprvA1RpRA33e1JQ7ifknakTFpgNXPmW2YvmhqLQYMmrj4xJXXWYpDPS3xz7iAxn8L39njGVyuoseXzU6rcxFLJ8HFsTjSyQbLYnMpCqE2VbFWc,xprv9uPDJpEQgRQfDcW7BkF7eTya6RPxXeJCqCJGHuCJ4GiRVLzkTXBAJMu2qaMWPrS7AANYqdq6vcBcBUdJCVVFceUvJFjaPdGZ2y9WACViL4L/0))", "sh(multi(2,[00000000/111'/222]xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL,xpub68NZiKmJWnxxS6aaHmn81bvJeTESw724CRDs6HbuccFQN9Ku14VQrADWgqbhhTHBaohPX4CjNLf9fq9MYo6oDaPPLPxSb7gwQN3ih19Zm4Y/0))", DEFAULT, {{"a91445a9a622a8b0a1269944be477640eedc447bbd8487"}}, OutputType::LEGACY, {{0x8000006FUL,222},{0}});

6
src/wallet/rpcdump.cpp
View File

@ -1766,7 +1766,7 @@ static UniValue ProcessDescriptorImport(CWallet * const pwallet, const UniValue&
if (!w_desc.descriptor->GetOutputType()) { if (!w_desc.descriptor->GetOutputType()) {
warnings.push_back("Unknown output type, cannot set descriptor to active."); warnings.push_back("Unknown output type, cannot set descriptor to active.");
} else { } else {
pwallet->SetActiveScriptPubKeyMan(spk_manager->GetID(), internal); pwallet->AddActiveScriptPubKeyMan(spk_manager->GetID(), internal);
} }
} }
@ -1972,8 +1972,6 @@ RPCHelpMan listdescriptors()
throw JSONRPCError(RPC_WALLET_ERROR, "listdescriptors is not available for non-descriptor wallets"); throw JSONRPCError(RPC_WALLET_ERROR, "listdescriptors is not available for non-descriptor wallets");
} }
EnsureWalletIsUnlocked(wallet.get());
LOCK(wallet->cs_wallet); LOCK(wallet->cs_wallet);
UniValue descriptors(UniValue::VARR); UniValue descriptors(UniValue::VARR);
@ -1987,7 +1985,7 @@ RPCHelpMan listdescriptors()
LOCK(desc_spk_man->cs_desc_man); LOCK(desc_spk_man->cs_desc_man);
const auto& wallet_descriptor = desc_spk_man->GetWalletDescriptor(); const auto& wallet_descriptor = desc_spk_man->GetWalletDescriptor();
std::string descriptor; std::string descriptor;
if (!desc_spk_man->GetDescriptorString(descriptor, false)) { if (!desc_spk_man->GetDescriptorString(descriptor)) {
throw JSONRPCError(RPC_WALLET_ERROR, "Can't get normalized descriptor string."); throw JSONRPCError(RPC_WALLET_ERROR, "Can't get normalized descriptor string.");
} }
spk.pushKV("desc", descriptor); spk.pushKV("desc", descriptor);

120
src/wallet/rpcwallet.cpp
View File

@ -83,14 +83,12 @@ static bool ParseIncludeWatchonly(const UniValue& include_watchonly, const CWall
/** Checks if a CKey is in the given CWallet compressed or otherwise*/ /** Checks if a CKey is in the given CWallet compressed or otherwise*/
/*
bool HaveKey(const SigningProvider& wallet, const CKey& key) bool HaveKey(const SigningProvider& wallet, const CKey& key)
{ {
CKey key2; CKey key2;
key2.Set(key.begin(), key.end(), !key.IsCompressed()); key2.Set(key.begin(), key.end(), !key.IsCompressed());
return wallet.HaveKey(key.GetPubKey().GetID()) || wallet.HaveKey(key2.GetPubKey().GetID()); return wallet.HaveKey(key.GetPubKey().GetID()) || wallet.HaveKey(key2.GetPubKey().GetID());
} }
*/
bool GetWalletNameFromJSONRPCRequest(const JSONRPCRequest& request, std::string& wallet_name) bool GetWalletNameFromJSONRPCRequest(const JSONRPCRequest& request, std::string& wallet_name)
{ {
@ -2976,7 +2974,7 @@ static RPCHelpMan createwallet()
{ {
{"wallet_name", RPCArg::Type::STR, RPCArg::Optional::NO, "The name for the new wallet. If this is a path, the wallet will be created at the path location."}, {"wallet_name", RPCArg::Type::STR, RPCArg::Optional::NO, "The name for the new wallet. If this is a path, the wallet will be created at the path location."},
{"disable_private_keys", RPCArg::Type::BOOL, /* default */ "false", "Disable the possibility of private keys (only watchonlys are possible in this mode)."}, {"disable_private_keys", RPCArg::Type::BOOL, /* default */ "false", "Disable the possibility of private keys (only watchonlys are possible in this mode)."},
{"blank", RPCArg::Type::BOOL, /* default */ "false", "Create a blank wallet. A blank wallet has no keys or HD seed. One can be set using upgradetohd."}, {"blank", RPCArg::Type::BOOL, /* default */ "false", "Create a blank wallet. A blank wallet has no keys or HD seed. One can be set using upgradetohd (by mnemonic) or sethdseed (WIF private key)."},
{"passphrase", RPCArg::Type::STR, RPCArg::Optional::OMITTED, "Encrypt the wallet with this passphrase."}, {"passphrase", RPCArg::Type::STR, RPCArg::Optional::OMITTED, "Encrypt the wallet with this passphrase."},
{"avoid_reuse", RPCArg::Type::BOOL, /* default */ "false", "Keep track of coin reuse, and treat dirty and clean coins differently with privacy considerations in mind."}, {"avoid_reuse", RPCArg::Type::BOOL, /* default */ "false", "Keep track of coin reuse, and treat dirty and clean coins differently with privacy considerations in mind."},
{"descriptors", RPCArg::Type::BOOL, /* default */ "false", "Create a native descriptor wallet. The wallet will use descriptors internally to handle address creation"}, {"descriptors", RPCArg::Type::BOOL, /* default */ "false", "Create a native descriptor wallet. The wallet will use descriptors internally to handle address creation"},
@ -3517,7 +3515,7 @@ static RPCHelpMan fundrawtransaction()
CAmount fee; CAmount fee;
int change_position; int change_position;
CCoinControl coin_control; CCoinControl coin_control;
// Automatically select (additional) coins. Can be overriden by options.add_inputs. // Automatically select (additional) coins. Can be overridden by options.add_inputs.
coin_control.m_add_inputs = true; coin_control.m_add_inputs = true;
FundTransaction(pwallet, tx, fee, change_position, request.params[1], coin_control); FundTransaction(pwallet, tx, fee, change_position, request.params[1], coin_control);
@ -3938,7 +3936,7 @@ RPCHelpMan getaddressinfo()
DescriptorScriptPubKeyMan* desc_spk_man = dynamic_cast<DescriptorScriptPubKeyMan*>(pwallet->GetScriptPubKeyMan(scriptPubKey)); DescriptorScriptPubKeyMan* desc_spk_man = dynamic_cast<DescriptorScriptPubKeyMan*>(pwallet->GetScriptPubKeyMan(scriptPubKey));
if (desc_spk_man) { if (desc_spk_man) {
std::string desc_str; std::string desc_str;
if (desc_spk_man->GetDescriptorString(desc_str, false)) { if (desc_spk_man->GetDescriptorString(desc_str)) {
ret.pushKV("parent_desc", desc_str); ret.pushKV("parent_desc", desc_str);
} }
} }
@ -4272,6 +4270,85 @@ static RPCHelpMan send()
}; };
} }
static RPCHelpMan sethdseed()
{
return RPCHelpMan{"sethdseed",
"\nSet or generate a new HD wallet seed. Non-HD wallets will not be upgraded to being a HD wallet. Wallets that are already\n"
"HD can not be updated to a new HD seed.\n"
"\nNote that you will need to MAKE A NEW BACKUP of your wallet after setting the HD wallet seed." +
HELP_REQUIRING_PASSPHRASE,
{
{"newkeypool", RPCArg::Type::BOOL, /* default */ "true", "Whether to flush old unused addresses, including change addresses, from the keypool and regenerate it.\n"
"If true, the next address from getnewaddress and change address from getrawchangeaddress will be from this new seed.\n"
"If false, addresses from the existing keypool will be used until it has been depleted."},
{"seed", RPCArg::Type::STR, /* default */ "random seed", "The WIF private key to use as the new HD seed.\n"
"The seed value can be retrieved using the dumpwallet command. It is the private key marked hdseed=1"},
},
RPCResult{RPCResult::Type::NONE, "", ""},
RPCExamples{
HelpExampleCli("sethdseed", "")
+ HelpExampleCli("sethdseed", "false")
+ HelpExampleCli("sethdseed", "true \"wifkey\"")
+ HelpExampleRpc("sethdseed", "true, \"wifkey\"")
},
[&](const RPCHelpMan& self, const JSONRPCRequest& request) -> UniValue
{
// TODO: add mnemonic feature to sethdseed or remove it in favour of upgradetohd
std::shared_ptr<CWallet> const wallet = GetWalletForJSONRPCRequest(request);
if (!wallet) return NullUniValue;
CWallet* const pwallet = wallet.get();
LegacyScriptPubKeyMan& spk_man = EnsureLegacyScriptPubKeyMan(*pwallet, true);
if (pwallet->IsWalletFlagSet(WALLET_FLAG_DISABLE_PRIVATE_KEYS)) {
throw JSONRPCError(RPC_WALLET_ERROR, "Cannot set a HD seed to a wallet with private keys disabled");
}
LOCK2(pwallet->cs_wallet, spk_man.cs_KeyStore);
// Do not do anything to non-HD wallets
if (!pwallet->CanSupportFeature(FEATURE_HD)) {
throw JSONRPCError(RPC_WALLET_ERROR, "Cannot set a HD seed on a non-HD wallet. Use the upgradewallet RPC in order to upgrade a non-HD wallet to HD");
}
if (pwallet->IsHDEnabled()) {
throw JSONRPCError(RPC_WALLET_ERROR, "Cannot set a HD seed. The wallet already has a seed");
}
EnsureWalletIsUnlocked(pwallet);
bool flush_key_pool = true;
if (!request.params[0].isNull()) {
flush_key_pool = request.params[0].get_bool();
}
if (request.params[1].isNull()) {
spk_man.GenerateNewHDChain("", "");
} else {
CKey key = DecodeSecret(request.params[1].get_str());
if (!key.IsValid()) {
throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid private key");
}
if (HaveKey(spk_man, key)) {
throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Already have this key (either as an HD seed or as a loose private key)");
}
CHDChain newHdChain;
if (!newHdChain.SetSeed(SecureVector(key.begin(), key.end()), true)) {
throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid private key: SetSeed failed");
}
if (!spk_man.AddHDChainSingle(newHdChain)) {
throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Invalid private key: AddHDChainSingle failed");
}
// add default account
newHdChain.AddAccount();
}
if (flush_key_pool) spk_man.NewKeyPool();
return NullUniValue;
},
};
}
RPCHelpMan walletprocesspsbt() RPCHelpMan walletprocesspsbt()
{ {
return RPCHelpMan{"walletprocesspsbt", return RPCHelpMan{"walletprocesspsbt",
@ -4432,7 +4509,7 @@ static RPCHelpMan walletcreatefundedpsbt()
CMutableTransaction rawTx = ConstructTransaction(request.params[0], request.params[1], request.params[2]); CMutableTransaction rawTx = ConstructTransaction(request.params[0], request.params[1], request.params[2]);
CCoinControl coin_control; CCoinControl coin_control;
// Automatically select coins, unless at least one is manually selected. Can // Automatically select coins, unless at least one is manually selected. Can
// be overriden by options.add_inputs. // be overridden by options.add_inputs.
coin_control.m_add_inputs = rawTx.vin.size() == 0; coin_control.m_add_inputs = rawTx.vin.size() == 0;
FundTransaction(pwallet, rawTx, fee, change_position, request.params[3], coin_control); FundTransaction(pwallet, rawTx, fee, change_position, request.params[3], coin_control);
@ -4463,14 +4540,18 @@ static RPCHelpMan walletcreatefundedpsbt()
static RPCHelpMan upgradewallet() static RPCHelpMan upgradewallet()
{ {
return RPCHelpMan{"upgradewallet", return RPCHelpMan{"upgradewallet",
"\nUpgrade the wallet. Upgrades to the latest version if no version number is specified\n" "\nUpgrade the wallet. Upgrades to the latest version if no version number is specified.\n"
"New keys may be generated and a new wallet backup will need to be made.", "New keys may be generated and a new wallet backup will need to be made.",
{ {
{"version", RPCArg::Type::NUM, /* default */ strprintf("%d", FEATURE_LATEST), "The version number to upgrade to. Default is the latest wallet version"} {"version", RPCArg::Type::NUM, /* default */ strprintf("%d", FEATURE_LATEST), "The version number to upgrade to. Default is the latest wallet version."}
}, },
RPCResult{ RPCResult{
RPCResult::Type::OBJ, "", "", RPCResult::Type::OBJ, "", "",
{ {
{RPCResult::Type::STR, "wallet_name", "Name of wallet this operation was performed on"},
{RPCResult::Type::NUM, "previous_version", "Version of wallet before this operation"},
{RPCResult::Type::NUM, "current_version", "Version of wallet after this operation"},
{RPCResult::Type::STR, "result", /* optional */ true, "Description of result, if no error"},
{RPCResult::Type::STR, "error", /* optional */ true, "Error message (if there is one)"} {RPCResult::Type::STR, "error", /* optional */ true, "Error message (if there is one)"}
}, },
}, },
@ -4493,11 +4574,27 @@ static RPCHelpMan upgradewallet()
version = request.params[0].get_int(); version = request.params[0].get_int();
} }
bilingual_str error; bilingual_str error;
if (!pwallet->UpgradeWallet(version, error)) { const int previous_version{pwallet->GetVersion()};
throw JSONRPCError(RPC_WALLET_ERROR, error.original); const bool wallet_upgraded{pwallet->UpgradeWallet(version, error)};
const int current_version{pwallet->GetVersion()};
std::string result;
if (wallet_upgraded) {
if (previous_version == current_version) {
result = "Already at latest version. Wallet version unchanged.";
} else {
result = strprintf("Wallet upgraded successfully from version %i to version %i.", previous_version, current_version);
}
} }
UniValue obj(UniValue::VOBJ); UniValue obj(UniValue::VOBJ);
if (!error.empty()) { obj.pushKV("wallet_name", pwallet->GetName());
obj.pushKV("previous_version", previous_version);
obj.pushKV("current_version", current_version);
if (!result.empty()) {
obj.pushKV("result", result);
} else {
CHECK_NONFATAL(!error.empty());
obj.pushKV("error", error.original); obj.pushKV("error", error.original);
} }
return obj; return obj;
@ -4576,6 +4673,7 @@ static const CRPCCommand commands[] =
{ "wallet", "send", &send, {"outputs","conf_target","estimate_mode","options"} }, { "wallet", "send", &send, {"outputs","conf_target","estimate_mode","options"} },
{ "wallet", "sendmany", &sendmany, {"dummy","amounts","minconf","addlocked","comment","subtractfeefrom","use_is","use_cj","conf_target","estimate_mode"} }, { "wallet", "sendmany", &sendmany, {"dummy","amounts","minconf","addlocked","comment","subtractfeefrom","use_is","use_cj","conf_target","estimate_mode"} },
{ "wallet", "sendtoaddress", &sendtoaddress, {"address","amount","comment","comment_to","subtractfeefromamount","use_is","use_cj","conf_target","estimate_mode", "avoid_reuse"} }, { "wallet", "sendtoaddress", &sendtoaddress, {"address","amount","comment","comment_to","subtractfeefromamount","use_is","use_cj","conf_target","estimate_mode", "avoid_reuse"} },
{ "wallet", "sethdseed", &sethdseed, {"newkeypool","seed"} },
{ "wallet", "setcoinjoinrounds", &setcoinjoinrounds, {"rounds"} }, { "wallet", "setcoinjoinrounds", &setcoinjoinrounds, {"rounds"} },
{ "wallet", "setcoinjoinamount", &setcoinjoinamount, {"amount"} }, { "wallet", "setcoinjoinamount", &setcoinjoinamount, {"amount"} },
{ "wallet", "setlabel", &setlabel, {"address","label"} }, { "wallet", "setlabel", &setlabel, {"address","label"} },

146
src/wallet/scriptpubkeyman.cpp
View File

@ -218,14 +218,14 @@ bool LegacyScriptPubKeyMan::CheckDecryptionKey(const CKeyingMaterial& master_key
if (keyFail) { if (keyFail) {
return false; return false;
} }
if (!keyPass && !accept_no_keys && (hdChain.IsNull() || !hdChain.IsNull() && !hdChain.IsCrypted())) { if (!keyPass && !accept_no_keys && (m_hd_chain.IsNull() || !m_hd_chain.IsNull() && !m_hd_chain.IsCrypted())) {
return false; return false;
} }
if(!hdChain.IsNull() && !hdChain.IsCrypted()) { if(!m_hd_chain.IsNull() && !m_hd_chain.IsCrypted()) {
// try to decrypt seed and make sure it matches // try to decrypt seed and make sure it matches
CHDChain hdChainTmp; CHDChain hdChainTmp;
if (!DecryptHDChain(master_key, hdChainTmp) || (hdChain.GetID() != hdChainTmp.GetSeedHash())) { if (!DecryptHDChain(master_key, hdChainTmp) || (m_hd_chain.GetID() != hdChainTmp.GetSeedHash())) {
return false; return false;
} }
} }
@ -267,8 +267,8 @@ bool LegacyScriptPubKeyMan::Encrypt(const CKeyingMaterial& master_key, WalletBat
} }
if (!hdChainCurrent.IsNull()) { if (!hdChainCurrent.IsNull()) {
assert(EncryptHDChain(master_key, hdChain)); assert(EncryptHDChain(master_key, m_hd_chain));
assert(SetHDChain(hdChain)); assert(LoadHDChain(m_hd_chain));
CHDChain hdChainCrypted; CHDChain hdChainCrypted;
assert(GetHDChain(hdChainCrypted)); assert(GetHDChain(hdChainCrypted));
@ -277,7 +277,7 @@ bool LegacyScriptPubKeyMan::Encrypt(const CKeyingMaterial& master_key, WalletBat
assert(hdChainCurrent.GetID() == hdChainCrypted.GetID()); assert(hdChainCurrent.GetID() == hdChainCrypted.GetID());
assert(hdChainCurrent.GetSeedHash() != hdChainCrypted.GetSeedHash()); assert(hdChainCurrent.GetSeedHash() != hdChainCrypted.GetSeedHash());
assert(SetHDChain(*encrypted_batch, hdChainCrypted, false)); assert(AddHDChain(*encrypted_batch, hdChainCrypted));
} }
encrypted_batch = nullptr; encrypted_batch = nullptr;
@ -396,7 +396,7 @@ void LegacyScriptPubKeyMan::GenerateNewCryptedHDChain(const SecureString& secure
CHDChain hdChainPrev = hdChainTmp; CHDChain hdChainPrev = hdChainTmp;
bool res = EncryptHDChain(vMasterKey, hdChainTmp); bool res = EncryptHDChain(vMasterKey, hdChainTmp);
assert(res); assert(res);
res = SetHDChain(hdChainTmp); res = LoadHDChain(hdChainTmp);
assert(res); assert(res);
CHDChain hdChainCrypted; CHDChain hdChainCrypted;
@ -407,8 +407,8 @@ void LegacyScriptPubKeyMan::GenerateNewCryptedHDChain(const SecureString& secure
assert(hdChainPrev.GetID() == hdChainCrypted.GetID()); assert(hdChainPrev.GetID() == hdChainCrypted.GetID());
assert(hdChainPrev.GetSeedHash() != hdChainCrypted.GetSeedHash()); assert(hdChainPrev.GetSeedHash() != hdChainCrypted.GetSeedHash());
if (!SetHDChainSingle(hdChainCrypted, false)) { if (!AddHDChainSingle(hdChainCrypted)) {
throw std::runtime_error(std::string(__func__) + ": SetHDChainSingle failed"); throw std::runtime_error(std::string(__func__) + ": AddHDChainSingle failed");
} }
} }
@ -426,8 +426,8 @@ void LegacyScriptPubKeyMan::GenerateNewHDChain(const SecureString& secureMnemoni
// add default account // add default account
newHdChain.AddAccount(); newHdChain.AddAccount();
if (!SetHDChainSingle(newHdChain, false)) { if (!AddHDChainSingle(newHdChain)) {
throw std::runtime_error(std::string(__func__) + ": SetHDChainSingle failed"); throw std::runtime_error(std::string(__func__) + ": AddHDChainSingle failed");
} }
if (!NewKeyPool()) { if (!NewKeyPool()) {
@ -435,14 +435,24 @@ void LegacyScriptPubKeyMan::GenerateNewHDChain(const SecureString& secureMnemoni
} }
} }
bool LegacyScriptPubKeyMan::SetHDChain(WalletBatch &batch, const CHDChain& chain, bool memonly) bool LegacyScriptPubKeyMan::LoadHDChain(const CHDChain& chain)
{ {
LOCK(cs_KeyStore); LOCK(cs_KeyStore);
if (!SetHDChain(chain)) if (m_storage.HasEncryptionKeys() != chain.IsCrypted()) return false;
m_hd_chain = chain;
return true;
}
bool LegacyScriptPubKeyMan::AddHDChain(WalletBatch &batch, const CHDChain& chain)
{
LOCK(cs_KeyStore);
if (!LoadHDChain(chain))
return false; return false;
if (!memonly) { {
if (chain.IsCrypted() && encrypted_batch) { if (chain.IsCrypted() && encrypted_batch) {
if (!encrypted_batch->WriteHDChain(chain)) if (!encrypted_batch->WriteHDChain(chain))
throw std::runtime_error(std::string(__func__) + ": WriteHDChain failed for encrypted batch"); throw std::runtime_error(std::string(__func__) + ": WriteHDChain failed for encrypted batch");
@ -458,10 +468,10 @@ bool LegacyScriptPubKeyMan::SetHDChain(WalletBatch &batch, const CHDChain& chain
return true; return true;
} }
bool LegacyScriptPubKeyMan::SetHDChainSingle(const CHDChain& chain, bool memonly) bool LegacyScriptPubKeyMan::AddHDChainSingle(const CHDChain& chain)
{ {
WalletBatch batch(m_storage.GetDatabase()); WalletBatch batch(m_storage.GetDatabase());
return SetHDChain(batch, chain, memonly); return AddHDChain(batch, chain);
} }
bool LegacyScriptPubKeyMan::GetDecryptedHDChain(CHDChain& hdChainRet) bool LegacyScriptPubKeyMan::GetDecryptedHDChain(CHDChain& hdChainRet)
@ -539,40 +549,40 @@ bool LegacyScriptPubKeyMan::DecryptHDChain(const CKeyingMaterial& vMasterKeyIn,
if (!m_storage.HasEncryptionKeys()) if (!m_storage.HasEncryptionKeys())
return true; return true;
if (hdChain.IsNull()) if (m_hd_chain.IsNull())
return false; return false;
if (!hdChain.IsCrypted()) if (!m_hd_chain.IsCrypted())
return false; return false;
SecureVector vchSecureSeed; SecureVector vchSecureSeed;
SecureVector vchSecureCryptedSeed = hdChain.GetSeed(); SecureVector vchSecureCryptedSeed = m_hd_chain.GetSeed();
std::vector<unsigned char> vchCryptedSeed(vchSecureCryptedSeed.begin(), vchSecureCryptedSeed.end()); std::vector<unsigned char> vchCryptedSeed(vchSecureCryptedSeed.begin(), vchSecureCryptedSeed.end());
if (!DecryptSecret(vMasterKeyIn, vchCryptedSeed, hdChain.GetID(), vchSecureSeed)) if (!DecryptSecret(vMasterKeyIn, vchCryptedSeed, m_hd_chain.GetID(), vchSecureSeed))
return false; return false;
hdChainRet = hdChain; hdChainRet = m_hd_chain;
if (!hdChainRet.SetSeed(vchSecureSeed, false)) if (!hdChainRet.SetSeed(vchSecureSeed, false))
return false; return false;
// hash of decrypted seed must match chain id // hash of decrypted seed must match chain id
if (hdChainRet.GetSeedHash() != hdChain.GetID()) if (hdChainRet.GetSeedHash() != m_hd_chain.GetID())
return false; return false;
SecureVector vchSecureCryptedMnemonic; SecureVector vchSecureCryptedMnemonic;
SecureVector vchSecureCryptedMnemonicPassphrase; SecureVector vchSecureCryptedMnemonicPassphrase;
// it's ok to have no mnemonic if wallet was initialized via hdseed // it's ok to have no mnemonic if wallet was initialized via hdseed
if (hdChain.GetMnemonic(vchSecureCryptedMnemonic, vchSecureCryptedMnemonicPassphrase)) { if (m_hd_chain.GetMnemonic(vchSecureCryptedMnemonic, vchSecureCryptedMnemonicPassphrase)) {
SecureVector vchSecureMnemonic; SecureVector vchSecureMnemonic;
SecureVector vchSecureMnemonicPassphrase; SecureVector vchSecureMnemonicPassphrase;
std::vector<unsigned char> vchCryptedMnemonic(vchSecureCryptedMnemonic.begin(), vchSecureCryptedMnemonic.end()); std::vector<unsigned char> vchCryptedMnemonic(vchSecureCryptedMnemonic.begin(), vchSecureCryptedMnemonic.end());
std::vector<unsigned char> vchCryptedMnemonicPassphrase(vchSecureCryptedMnemonicPassphrase.begin(), vchSecureCryptedMnemonicPassphrase.end()); std::vector<unsigned char> vchCryptedMnemonicPassphrase(vchSecureCryptedMnemonicPassphrase.begin(), vchSecureCryptedMnemonicPassphrase.end());
if (!vchCryptedMnemonic.empty() && !DecryptSecret(vMasterKeyIn, vchCryptedMnemonic, hdChain.GetID(), vchSecureMnemonic)) if (!vchCryptedMnemonic.empty() && !DecryptSecret(vMasterKeyIn, vchCryptedMnemonic, m_hd_chain.GetID(), vchSecureMnemonic))
return false; return false;
if (!vchCryptedMnemonicPassphrase.empty() && !DecryptSecret(vMasterKeyIn, vchCryptedMnemonicPassphrase, hdChain.GetID(), vchSecureMnemonicPassphrase)) if (!vchCryptedMnemonicPassphrase.empty() && !DecryptSecret(vMasterKeyIn, vchCryptedMnemonicPassphrase, m_hd_chain.GetID(), vchSecureMnemonicPassphrase))
return false; return false;
if (!hdChainRet.SetMnemonic(vchSecureMnemonic, vchSecureMnemonicPassphrase, false)) if (!hdChainRet.SetMnemonic(vchSecureMnemonic, vchSecureMnemonicPassphrase, false))
@ -1090,16 +1100,6 @@ bool LegacyScriptPubKeyMan::AddWatchOnly(const CScript& dest, int64_t nCreateTim
return AddWatchOnly(dest); return AddWatchOnly(dest);
} }
bool LegacyScriptPubKeyMan::SetHDChain(const CHDChain& chain)
{
LOCK(cs_KeyStore);
if (m_storage.HasEncryptionKeys() != chain.IsCrypted()) return false;
hdChain = chain;
return true;
}
bool LegacyScriptPubKeyMan::HaveHDKey(const CKeyID &address, CHDChain& hdChainCurrent) const bool LegacyScriptPubKeyMan::HaveHDKey(const CKeyID &address, CHDChain& hdChainCurrent) const
{ {
LOCK(cs_KeyStore); LOCK(cs_KeyStore);
@ -1322,8 +1322,8 @@ void LegacyScriptPubKeyMan::DeriveNewChildKey(WalletBatch &batch, CKeyMetadata&
if (!hdChainCurrent.SetAccount(nAccountIndex, acc)) if (!hdChainCurrent.SetAccount(nAccountIndex, acc))
throw std::runtime_error(std::string(__func__) + ": SetAccount failed"); throw std::runtime_error(std::string(__func__) + ": SetAccount failed");
if (!SetHDChain(batch, hdChainCurrent, false)) { if (!AddHDChain(batch, hdChainCurrent)) {
throw std::runtime_error(std::string(__func__) + ": SetHDChain failed"); throw std::runtime_error(std::string(__func__) + ": AddHDChain failed");
} }
if (!AddHDPubKey(batch, childKey.Neuter(), fInternal)) if (!AddHDPubKey(batch, childKey.Neuter(), fInternal))
@ -1758,8 +1758,8 @@ std::set<CKeyID> LegacyScriptPubKeyMan::GetKeys() const
bool LegacyScriptPubKeyMan::GetHDChain(CHDChain& hdChainRet) const bool LegacyScriptPubKeyMan::GetHDChain(CHDChain& hdChainRet) const
{ {
LOCK(cs_KeyStore); LOCK(cs_KeyStore);
hdChainRet = hdChain; hdChainRet = m_hd_chain;
return !hdChain.IsNull(); return !m_hd_chain.IsNull();
} }
void LegacyScriptPubKeyMan::SetInternal(bool internal) {} void LegacyScriptPubKeyMan::SetInternal(bool internal) {}
@ -1950,34 +1950,10 @@ bool DescriptorScriptPubKeyMan::TopUp(unsigned int size)
} }
m_map_pubkeys[pubkey] = i; m_map_pubkeys[pubkey] = i;
} }
// Write the cache // Merge and write the cache
for (const auto& parent_xpub_pair : temp_cache.GetCachedParentExtPubKeys()) { DescriptorCache new_items = m_wallet_descriptor.cache.MergeAndDiff(temp_cache);
CExtPubKey xpub; if (!batch.WriteDescriptorCacheItems(id, new_items)) {
if (m_wallet_descriptor.cache.GetCachedParentExtPubKey(parent_xpub_pair.first, xpub)) { throw std::runtime_error(std::string(__func__) + ": writing cache items failed");
if (xpub != parent_xpub_pair.second) {
throw std::runtime_error(std::string(__func__) + ": New cached parent xpub does not match already cached parent xpub");
}
continue;
}
if (!batch.WriteDescriptorParentCache(parent_xpub_pair.second, id, parent_xpub_pair.first)) {
throw std::runtime_error(std::string(__func__) + ": writing cache item failed");
}
m_wallet_descriptor.cache.CacheParentExtPubKey(parent_xpub_pair.first, parent_xpub_pair.second);
}
for (const auto& derived_xpub_map_pair : temp_cache.GetCachedDerivedExtPubKeys()) {
for (const auto& derived_xpub_pair : derived_xpub_map_pair.second) {
CExtPubKey xpub;
if (m_wallet_descriptor.cache.GetCachedDerivedExtPubKey(derived_xpub_map_pair.first, derived_xpub_pair.first, xpub)) {
if (xpub != derived_xpub_pair.second) {
throw std::runtime_error(std::string(__func__) + ": New cached derived xpub does not match already cached derived xpub");
}
continue;
}
if (!batch.WriteDescriptorDerivedCache(derived_xpub_pair.second, id, derived_xpub_map_pair.first, derived_xpub_pair.first)) {
throw std::runtime_error(std::string(__func__) + ": writing cache item failed");
}
m_wallet_descriptor.cache.CacheDerivedExtPubKey(derived_xpub_map_pair.first, derived_xpub_pair.first, derived_xpub_pair.second);
}
} }
m_max_cached_index++; m_max_cached_index++;
} }
@ -2402,15 +2378,41 @@ const std::vector<CScript> DescriptorScriptPubKeyMan::GetScriptPubKeys() const
return script_pub_keys; return script_pub_keys;
} }
bool DescriptorScriptPubKeyMan::GetDescriptorString(std::string& out, bool priv) const bool DescriptorScriptPubKeyMan::GetDescriptorString(std::string& out) const
{ {
LOCK(cs_desc_man); LOCK(cs_desc_man);
if (m_storage.IsLocked()) {
return false;
}
FlatSigningProvider provider; FlatSigningProvider provider;
provider.keys = GetKeys(); provider.keys = GetKeys();
return m_wallet_descriptor.descriptor->ToNormalizedString(provider, out, priv); return m_wallet_descriptor.descriptor->ToNormalizedString(provider, out, &m_wallet_descriptor.cache);
}
void DescriptorScriptPubKeyMan::UpgradeDescriptorCache()
{
LOCK(cs_desc_man);
if (m_storage.IsLocked() || m_storage.IsWalletFlagSet(WALLET_FLAG_LAST_HARDENED_XPUB_CACHED)) {
return;
}
// Skip if we have the last hardened xpub cache
if (m_wallet_descriptor.cache.GetCachedLastHardenedExtPubKeys().size() > 0) {
return;
}
// Expand the descriptor
FlatSigningProvider provider;
provider.keys = GetKeys();
FlatSigningProvider out_keys;
std::vector<CScript> scripts_temp;
DescriptorCache temp_cache;
if (!m_wallet_descriptor.descriptor->Expand(0, provider, scripts_temp, out_keys, &temp_cache)){
throw std::runtime_error("Unable to expand descriptor");
}
// Cache the last hardened xpubs
DescriptorCache diff = m_wallet_descriptor.cache.MergeAndDiff(temp_cache);
if (!WalletBatch(m_storage.GetDatabase()).WriteDescriptorCacheItems(GetID(), diff)) {
throw std::runtime_error(std::string(__func__) + ": writing cache items failed");
}
} }

18
src/wallet/scriptpubkeyman.h
View File

@ -37,7 +37,7 @@ public:
virtual bool IsWalletFlagSet(uint64_t) const = 0; virtual bool IsWalletFlagSet(uint64_t) const = 0;
virtual void UnsetBlankWalletFlag(WalletBatch&) = 0; virtual void UnsetBlankWalletFlag(WalletBatch&) = 0;
virtual bool CanSupportFeature(enum WalletFeature) const = 0; virtual bool CanSupportFeature(enum WalletFeature) const = 0;
virtual void SetMinVersion(enum WalletFeature, WalletBatch* = nullptr, bool = false) = 0; virtual void SetMinVersion(enum WalletFeature, WalletBatch* = nullptr) = 0;
virtual const CKeyingMaterial& GetEncryptionKey() const = 0; virtual const CKeyingMaterial& GetEncryptionKey() const = 0;
virtual bool HasEncryptionKeys() const = 0; virtual bool HasEncryptionKeys() const = 0;
virtual bool IsLocked(bool fForMixing = false) const = 0; virtual bool IsLocked(bool fForMixing = false) const = 0;
@ -278,15 +278,11 @@ private:
/** Add a KeyOriginInfo to the wallet */ /** Add a KeyOriginInfo to the wallet */
bool AddKeyOriginWithDB(WalletBatch& batch, const CPubKey& pubkey, const KeyOriginInfo& info); bool AddKeyOriginWithDB(WalletBatch& batch, const CPubKey& pubkey, const KeyOriginInfo& info);
/* Set the HD chain model (chain child index counters) */
bool SetHDChain(WalletBatch &batch, const CHDChain& chain, bool memonly);
bool EncryptHDChain(const CKeyingMaterial& vMasterKeyIn, CHDChain& chain); bool EncryptHDChain(const CKeyingMaterial& vMasterKeyIn, CHDChain& chain);
bool DecryptHDChain(const CKeyingMaterial& vMasterKeyIn, CHDChain& hdChainRet) const; bool DecryptHDChain(const CKeyingMaterial& vMasterKeyIn, CHDChain& hdChainRet) const;
bool SetHDChain(const CHDChain& chain);
/* the HD chain data model (external chain counters) */ /* the HD chain data model (external chain counters) */
CHDChain hdChain GUARDED_BY(cs_KeyStore); CHDChain m_hd_chain GUARDED_BY(cs_KeyStore);
/* HD derive new child key (on internal or external chain) */ /* HD derive new child key (on internal or external chain) */
void DeriveNewChildKey(WalletBatch& batch, CKeyMetadata& metadata, CKey& secretRet, uint32_t nAccountIndex, bool fInternal /*= false*/) EXCLUSIVE_LOCKS_REQUIRED(cs_KeyStore); void DeriveNewChildKey(WalletBatch& batch, CKeyMetadata& metadata, CKey& secretRet, uint32_t nAccountIndex, bool fInternal /*= false*/) EXCLUSIVE_LOCKS_REQUIRED(cs_KeyStore);
@ -398,11 +394,15 @@ public:
//! Generate a new key //! Generate a new key
CPubKey GenerateNewKey(WalletBatch& batch, uint32_t nAccountIndex, bool fInternal /*= false*/) EXCLUSIVE_LOCKS_REQUIRED(cs_KeyStore); CPubKey GenerateNewKey(WalletBatch& batch, uint32_t nAccountIndex, bool fInternal /*= false*/) EXCLUSIVE_LOCKS_REQUIRED(cs_KeyStore);
/* Set the HD chain model (chain child index counters) and writes it to the database */
bool AddHDChain(WalletBatch &batch, const CHDChain& chain);
//! Load a HD chain model (used by LoadWallet)
bool LoadHDChain(const CHDChain& chain);
/** /**
* Set the HD chain model (chain child index counters) using temporary wallet db object * Set the HD chain model (chain child index counters) using temporary wallet db object
* which causes db flush every time these methods are used * which causes db flush every time these methods are used
*/ */
bool SetHDChainSingle(const CHDChain& chain, bool memonly); bool AddHDChainSingle(const CHDChain& chain);
//! Adds a watch-only address to the store, without saving it to disk (used by LoadWallet) //! Adds a watch-only address to the store, without saving it to disk (used by LoadWallet)
bool LoadWatchOnly(const CScript &dest); bool LoadWatchOnly(const CScript &dest);
@ -606,7 +606,9 @@ public:
const WalletDescriptor GetWalletDescriptor() const EXCLUSIVE_LOCKS_REQUIRED(cs_desc_man); const WalletDescriptor GetWalletDescriptor() const EXCLUSIVE_LOCKS_REQUIRED(cs_desc_man);
const std::vector<CScript> GetScriptPubKeys() const; const std::vector<CScript> GetScriptPubKeys() const;
bool GetDescriptorString(std::string& out, bool priv) const; bool GetDescriptorString(std::string& out) const;
void UpgradeDescriptorCache();
}; };
#endif // BITCOIN_WALLET_SCRIPTPUBKEYMAN_H #endif // BITCOIN_WALLET_SCRIPTPUBKEYMAN_H

90
src/wallet/wallet.cpp
View File

@ -389,6 +389,19 @@ void CWallet::UpgradeKeyMetadata()
SetWalletFlag(WALLET_FLAG_KEY_ORIGIN_METADATA); SetWalletFlag(WALLET_FLAG_KEY_ORIGIN_METADATA);
} }
void CWallet::UpgradeDescriptorCache()
{
if (!IsWalletFlagSet(WALLET_FLAG_DESCRIPTORS) || IsLocked() || IsWalletFlagSet(WALLET_FLAG_LAST_HARDENED_XPUB_CACHED)) {
return;
}
for (ScriptPubKeyMan* spkm : GetAllScriptPubKeyMans()) {
DescriptorScriptPubKeyMan* desc_spkm = dynamic_cast<DescriptorScriptPubKeyMan*>(spkm);
desc_spkm->UpgradeDescriptorCache();
}
SetWalletFlag(WALLET_FLAG_LAST_HARDENED_XPUB_CACHED);
}
bool CWallet::ChangeWalletPassphrase(const SecureString& strOldWalletPassphrase, const SecureString& strNewWalletPassphrase) bool CWallet::ChangeWalletPassphrase(const SecureString& strOldWalletPassphrase, const SecureString& strNewWalletPassphrase)
{ {
bool fWasLocked = IsLocked(true); bool fWasLocked = IsLocked(true);
@ -442,21 +455,13 @@ void CWallet::chainStateFlushed(const CBlockLocator& loc)
batch.WriteBestBlock(loc); batch.WriteBestBlock(loc);
} }
void CWallet::SetMinVersion(enum WalletFeature nVersion, WalletBatch* batch_in, bool fExplicit) void CWallet::SetMinVersion(enum WalletFeature nVersion, WalletBatch* batch_in)
{ {
LOCK(cs_wallet); LOCK(cs_wallet);
if (nWalletVersion >= nVersion) if (nWalletVersion >= nVersion)
return; return;
// when doing an explicit upgrade, if we pass the max version permitted, upgrade all the way
if (fExplicit && nVersion > nWalletMaxVersion)
nVersion = FEATURE_LATEST;
nWalletVersion = nVersion; nWalletVersion = nVersion;
if (nVersion > nWalletMaxVersion)
nWalletMaxVersion = nVersion;
{ {
WalletBatch* batch = batch_in ? batch_in : new WalletBatch(GetDatabase()); WalletBatch* batch = batch_in ? batch_in : new WalletBatch(GetDatabase());
if (nWalletVersion > 40000) if (nWalletVersion > 40000)
@ -466,18 +471,6 @@ void CWallet::SetMinVersion(enum WalletFeature nVersion, WalletBatch* batch_in,
} }
} }
bool CWallet::SetMaxVersion(int nVersion)
{
LOCK(cs_wallet);
// cannot downgrade below current version
if (nWalletVersion > nVersion)
return false;
nWalletMaxVersion = nVersion;
return true;
}
std::set<uint256> CWallet::GetConflicts(const uint256& txid) const std::set<uint256> CWallet::GetConflicts(const uint256& txid) const
{ {
std::set<uint256> result; std::set<uint256> result;
@ -665,7 +658,7 @@ bool CWallet::EncryptWallet(const SecureString& strWalletPassphrase)
// Encryption was introduced in version 0.4.0 // Encryption was introduced in version 0.4.0
SetMinVersion(FEATURE_WALLETCRYPT, encrypted_batch, true); SetMinVersion(FEATURE_WALLETCRYPT, encrypted_batch);
if (!encrypted_batch->TxnCommit()) { if (!encrypted_batch->TxnCommit()) {
delete encrypted_batch; delete encrypted_batch;
@ -1684,19 +1677,28 @@ bool CWallet::IsWalletFlagSet(uint64_t flag) const
return (m_wallet_flags & flag); return (m_wallet_flags & flag);
} }
bool CWallet::SetWalletFlags(uint64_t overwriteFlags, bool memonly) bool CWallet::LoadWalletFlags(uint64_t flags)
{ {
LOCK(cs_wallet); LOCK(cs_wallet);
m_wallet_flags = overwriteFlags; if (((flags & KNOWN_WALLET_FLAGS) >> 32) ^ (flags >> 32)) {
if (((overwriteFlags & KNOWN_WALLET_FLAGS) >> 32) ^ (overwriteFlags >> 32)) {
// contains unknown non-tolerable wallet flags // contains unknown non-tolerable wallet flags
return false; return false;
} }
if (!memonly && !WalletBatch(GetDatabase()).WriteWalletFlags(m_wallet_flags)) { m_wallet_flags = flags;
return true;
}
bool CWallet::AddWalletFlags(uint64_t flags)
{
LOCK(cs_wallet);
// We should never be writing unknown non-tolerable wallet flags
assert(((flags & KNOWN_WALLET_FLAGS) >> 32) == (flags >> 32));
if (!WalletBatch(GetDatabase()).WriteWalletFlags(flags)) {
throw std::runtime_error(std::string(__func__) + ": writing wallet flags failed"); throw std::runtime_error(std::string(__func__) + ": writing wallet flags failed");
} }
return true; return LoadWalletFlags(flags);
} }
int64_t CWalletTx::GetTxTime() const int64_t CWalletTx::GetTxTime() const
@ -4574,8 +4576,9 @@ std::shared_ptr<CWallet> CWallet::Create(interfaces::Chain& chain, interfaces::C
if (fFirstRun) if (fFirstRun)
{ {
walletInstance->SetMaxVersion(FEATURE_LATEST); walletInstance->SetMinVersion(FEATURE_LATEST);
walletInstance->SetWalletFlags(wallet_creation_flags, false);
walletInstance->AddWalletFlags(wallet_creation_flags);
// Only create LegacyScriptPubKeyMan when not descriptor wallet // Only create LegacyScriptPubKeyMan when not descriptor wallet
if (!walletInstance->IsWalletFlagSet(WALLET_FLAG_DESCRIPTORS)) { if (!walletInstance->IsWalletFlagSet(WALLET_FLAG_DESCRIPTORS)) {
@ -4600,8 +4603,8 @@ std::shared_ptr<CWallet> CWallet::Create(interfaces::Chain& chain, interfaces::C
} }
LOCK(walletInstance->cs_wallet); LOCK(walletInstance->cs_wallet);
if (auto spk_man = walletInstance->GetLegacyScriptPubKeyMan()) { if (auto spk_man = walletInstance->GetLegacyScriptPubKeyMan()) {
if (!spk_man->SetHDChainSingle(newHdChain, false)) { if (!spk_man->AddHDChainSingle(newHdChain)) {
error = strprintf(_("%s failed"), "SetHDChainSingle"); error = strprintf(_("%s failed"), "AddHDChainSingle");
return nullptr; return nullptr;
} }
} }
@ -4887,6 +4890,7 @@ std::shared_ptr<CWallet> CWallet::Create(interfaces::Chain& chain, interfaces::C
bool CWallet::UpgradeWallet(int version, bilingual_str& error) bool CWallet::UpgradeWallet(int version, bilingual_str& error)
{ {
int prev_version = GetVersion();
int nMaxVersion = version; int nMaxVersion = version;
auto nMinVersion = DEFAULT_USE_HD_WALLET ? FEATURE_LATEST : FEATURE_COMPRPUBKEY; auto nMinVersion = DEFAULT_USE_HD_WALLET ? FEATURE_LATEST : FEATURE_COMPRPUBKEY;
if (nMaxVersion == 0) { if (nMaxVersion == 0) {
@ -4898,17 +4902,18 @@ bool CWallet::UpgradeWallet(int version, bilingual_str& error)
} }
if (nMaxVersion < GetVersion()) { if (nMaxVersion < GetVersion()) {
error = Untranslated("Cannot downgrade wallet"); error = strprintf(_("Cannot downgrade wallet from version %i to version %i. Wallet version unchanged."), prev_version, version);
return false; return false;
} }
// TODO: consider discourage users to skip passphrase for HD wallets for v21 // TODO: consider discourage users to skip passphrase for HD wallets for v21
if (false && nMaxVersion >= FEATURE_HD && !IsHDEnabled()) { if (false && nMaxVersion >= FEATURE_HD && !IsHDEnabled()) {
error = Untranslated("You should use upgradetohd RPC to upgrade non-HD wallet to HD"); error = Untranslated("You should use upgradetohd RPC to upgrade non-HD wallet to HD");
error = strprintf(_("Cannot upgrade a non HD wallet from version %i to version %i which is non-HD wallet. Use upgradetohd RPC"), prev_version, version);
return false; return false;
} }
SetMaxVersion(nMaxVersion); SetMinVersion(GetClosestWalletFeature(version));
return true; return true;
} }
@ -5649,12 +5654,21 @@ void CWallet::SetupDescriptorScriptPubKeyMans()
spk_manager->SetupDescriptorGeneration(master_key); spk_manager->SetupDescriptorGeneration(master_key);
uint256 id = spk_manager->GetID(); uint256 id = spk_manager->GetID();
m_spk_managers[id] = std::move(spk_manager); m_spk_managers[id] = std::move(spk_manager);
SetActiveScriptPubKeyMan(id, internal); AddActiveScriptPubKeyMan(id, internal);
} }
} }
} }
void CWallet::SetActiveScriptPubKeyMan(uint256 id, bool internal, bool memonly) void CWallet::AddActiveScriptPubKeyMan(uint256 id, bool internal)
{
WalletBatch batch(GetDatabase());
if (!batch.WriteActiveScriptPubKeyMan(id, internal)) {
throw std::runtime_error(std::string(__func__) + ": writing active ScriptPubKeyMan id failed");
}
LoadActiveScriptPubKeyMan(id, internal);
}
void CWallet::LoadActiveScriptPubKeyMan(uint256 id, bool internal)
{ {
WalletLogPrintf("Setting spkMan to active: id = %s, type = %d, internal = %d\n", id.ToString(), static_cast<int>(OutputType::LEGACY), static_cast<int>(internal)); WalletLogPrintf("Setting spkMan to active: id = %s, type = %d, internal = %d\n", id.ToString(), static_cast<int>(OutputType::LEGACY), static_cast<int>(internal));
auto& spk_mans = internal ? m_internal_spk_managers : m_external_spk_managers; auto& spk_mans = internal ? m_internal_spk_managers : m_external_spk_managers;
@ -5662,12 +5676,6 @@ void CWallet::SetActiveScriptPubKeyMan(uint256 id, bool internal, bool memonly)
spk_man->SetInternal(internal); spk_man->SetInternal(internal);
spk_mans = spk_man; spk_mans = spk_man;
if (!memonly) {
WalletBatch batch(GetDatabase());
if (!batch.WriteActiveScriptPubKeyMan(id, internal)) {
throw std::runtime_error(std::string(__func__) + ": writing active ScriptPubKeyMan id failed");
}
}
NotifyCanGetAddressesChanged(); NotifyCanGetAddressesChanged();
} }

36
src/wallet/wallet.h
View File

@ -128,6 +128,7 @@ static constexpr uint64_t KNOWN_WALLET_FLAGS =
WALLET_FLAG_AVOID_REUSE WALLET_FLAG_AVOID_REUSE
| WALLET_FLAG_BLANK_WALLET | WALLET_FLAG_BLANK_WALLET
| WALLET_FLAG_KEY_ORIGIN_METADATA | WALLET_FLAG_KEY_ORIGIN_METADATA
| WALLET_FLAG_LAST_HARDENED_XPUB_CACHED
| WALLET_FLAG_DISABLE_PRIVATE_KEYS | WALLET_FLAG_DISABLE_PRIVATE_KEYS
| WALLET_FLAG_DESCRIPTORS; | WALLET_FLAG_DESCRIPTORS;
@ -138,6 +139,7 @@ static const std::map<std::string,WalletFlags> WALLET_FLAG_MAP{
{"avoid_reuse", WALLET_FLAG_AVOID_REUSE}, {"avoid_reuse", WALLET_FLAG_AVOID_REUSE},
{"blank", WALLET_FLAG_BLANK_WALLET}, {"blank", WALLET_FLAG_BLANK_WALLET},
{"key_origin_metadata", WALLET_FLAG_KEY_ORIGIN_METADATA}, {"key_origin_metadata", WALLET_FLAG_KEY_ORIGIN_METADATA},
{"last_hardened_xpub_cached", WALLET_FLAG_LAST_HARDENED_XPUB_CACHED},
{"disable_private_keys", WALLET_FLAG_DISABLE_PRIVATE_KEYS}, {"disable_private_keys", WALLET_FLAG_DISABLE_PRIVATE_KEYS},
{"descriptor_wallet", WALLET_FLAG_DESCRIPTORS}, {"descriptor_wallet", WALLET_FLAG_DESCRIPTORS},
}; };
@ -706,9 +708,6 @@ private:
//! the current wallet version: clients below this version are not able to load the wallet //! the current wallet version: clients below this version are not able to load the wallet
int nWalletVersion GUARDED_BY(cs_wallet){FEATURE_BASE}; int nWalletVersion GUARDED_BY(cs_wallet){FEATURE_BASE};
//! the maximum wallet format version: memory-only variable that specifies to what version this wallet may be upgraded
int nWalletMaxVersion GUARDED_BY(cs_wallet) = FEATURE_BASE;
int64_t nNextResend = 0; int64_t nNextResend = 0;
bool fBroadcastTransactions = false; bool fBroadcastTransactions = false;
// Local time that the tip block was received. Used to schedule wallet rebroadcasts. // Local time that the tip block was received. Used to schedule wallet rebroadcasts.
@ -911,8 +910,8 @@ public:
const CWalletTx* GetWalletTx(const uint256& hash) const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); const CWalletTx* GetWalletTx(const uint256& hash) const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
bool IsTrusted(const CWalletTx& wtx, std::set<uint256>& trusted_parents) const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); bool IsTrusted(const CWalletTx& wtx, std::set<uint256>& trusted_parents) const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
//! check whether we are allowed to upgrade (or already support) to the named feature //! check whether we support the named feature
bool CanSupportFeature(enum WalletFeature wf) const override EXCLUSIVE_LOCKS_REQUIRED(cs_wallet) { AssertLockHeld(cs_wallet); return nWalletMaxVersion >= wf; } bool CanSupportFeature(enum WalletFeature wf) const override EXCLUSIVE_LOCKS_REQUIRED(cs_wallet) { AssertLockHeld(cs_wallet); return IsFeatureSupported(nWalletVersion, wf); }
/** /**
* populate vCoins with vector of available COutputs. * populate vCoins with vector of available COutputs.
@ -981,7 +980,10 @@ public:
//! Upgrade stored CKeyMetadata objects to store key origin info as KeyOriginInfo //! Upgrade stored CKeyMetadata objects to store key origin info as KeyOriginInfo
void UpgradeKeyMetadata() EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); void UpgradeKeyMetadata() EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
bool LoadMinVersion(int nVersion) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet) { AssertLockHeld(cs_wallet); nWalletVersion = nVersion; nWalletMaxVersion = std::max(nWalletMaxVersion, nVersion); return true; } //! Upgrade DescriptorCaches
void UpgradeDescriptorCache() EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
bool LoadMinVersion(int nVersion) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet) { AssertLockHeld(cs_wallet); nWalletVersion = nVersion; return true; }
//! Adds a destination data tuple to the store, without saving it to disk //! Adds a destination data tuple to the store, without saving it to disk
void LoadDestData(const CTxDestination& dest, const std::string& key, const std::string& value) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); void LoadDestData(const CTxDestination& dest, const std::string& key, const std::string& value) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
@ -1198,11 +1200,8 @@ public:
unsigned int GetKeyPoolSize() const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); unsigned int GetKeyPoolSize() const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
//! signify that a particular wallet feature is now used. this may change nWalletVersion and nWalletMaxVersion if those are lower //! signify that a particular wallet feature is now used.
void SetMinVersion(enum WalletFeature, WalletBatch* batch_in = nullptr, bool fExplicit = false) override; void SetMinVersion(enum WalletFeature, WalletBatch* batch_in = nullptr) override;
//! change which version we're allowed to upgrade to (note that this does not immediately imply upgrading to that format)
bool SetMaxVersion(int nVersion);
//! get the current wallet format (the oldest client version guaranteed to understand this wallet) //! get the current wallet format (the oldest client version guaranteed to understand this wallet)
int GetVersion() const { LOCK(cs_wallet); return nWalletVersion; } int GetVersion() const { LOCK(cs_wallet); return nWalletVersion; }
@ -1334,7 +1333,9 @@ public:
/** overwrite all flags by the given uint64_t /** overwrite all flags by the given uint64_t
returns false if unknown, non-tolerable flags are present */ returns false if unknown, non-tolerable flags are present */
bool SetWalletFlags(uint64_t overwriteFlags, bool memOnly); bool AddWalletFlags(uint64_t flags);
/** Loads the flags into the wallet. (used by LoadWallet) */
bool LoadWalletFlags(uint64_t flags);
/** Determine if we are a legacy wallet */ /** Determine if we are a legacy wallet */
bool IsLegacy() const; bool IsLegacy() const;
@ -1415,12 +1416,15 @@ public:
//! Instantiate a descriptor ScriptPubKeyMan from the WalletDescriptor and load it //! Instantiate a descriptor ScriptPubKeyMan from the WalletDescriptor and load it
void LoadDescriptorScriptPubKeyMan(uint256 id, WalletDescriptor& desc); void LoadDescriptorScriptPubKeyMan(uint256 id, WalletDescriptor& desc);
//! Sets the active ScriptPubKeyMan for the specified type and internal //! Adds the active ScriptPubKeyMan for the specified type and internal. Writes it to the wallet file
//! @param[in] id The unique id for the ScriptPubKeyMan //! @param[in] id The unique id for the ScriptPubKeyMan
//! @param[in] type The OutputType this ScriptPubKeyMan provides addresses for
//! @param[in] internal Whether this ScriptPubKeyMan provides change addresses //! @param[in] internal Whether this ScriptPubKeyMan provides change addresses
//! @param[in] memonly Whether to record this update to the database. Set to true for wallet loading, normally false when actually updating the wallet. void AddActiveScriptPubKeyMan(uint256 id, bool internal);
void SetActiveScriptPubKeyMan(uint256 id, bool internal, bool memonly = false);
//! Loads an active ScriptPubKeyMan for the specified type and internal. (used by LoadWallet)
//! @param[in] id The unique id for the ScriptPubKeyMan
//! @param[in] internal Whether this ScriptPubKeyMan provides change addresses
void LoadActiveScriptPubKeyMan(uint256 id, bool internal);
//! Create new DescriptorScriptPubKeyMans and add them to the wallet //! Create new DescriptorScriptPubKeyMans and add them to the wallet
void SetupDescriptorScriptPubKeyMans(); void SetupDescriptorScriptPubKeyMans();

75
src/wallet/walletdb.cpp
View File

@ -60,6 +60,7 @@ const std::string TX{"tx"};
const std::string VERSION{"version"}; const std::string VERSION{"version"};
const std::string WALLETDESCRIPTOR{"walletdescriptor"}; const std::string WALLETDESCRIPTOR{"walletdescriptor"};
const std::string WALLETDESCRIPTORCACHE{"walletdescriptorcache"}; const std::string WALLETDESCRIPTORCACHE{"walletdescriptorcache"};
const std::string WALLETDESCRIPTORLHCACHE{"walletdescriptorlhcache"};
const std::string WALLETDESCRIPTORCKEY{"walletdescriptorckey"}; const std::string WALLETDESCRIPTORCKEY{"walletdescriptorckey"};
const std::string WALLETDESCRIPTORKEY{"walletdescriptorkey"}; const std::string WALLETDESCRIPTORKEY{"walletdescriptorkey"};
const std::string WATCHMETA{"watchmeta"}; const std::string WATCHMETA{"watchmeta"};
@ -272,6 +273,35 @@ bool WalletBatch::WriteDescriptorParentCache(const CExtPubKey& xpub, const uint2
return WriteIC(std::make_pair(std::make_pair(DBKeys::WALLETDESCRIPTORCACHE, desc_id), key_exp_index), ser_xpub); return WriteIC(std::make_pair(std::make_pair(DBKeys::WALLETDESCRIPTORCACHE, desc_id), key_exp_index), ser_xpub);
} }
bool WalletBatch::WriteDescriptorLastHardenedCache(const CExtPubKey& xpub, const uint256& desc_id, uint32_t key_exp_index)
{
std::vector<unsigned char> ser_xpub(BIP32_EXTKEY_SIZE);
xpub.Encode(ser_xpub.data());
return WriteIC(std::make_pair(std::make_pair(DBKeys::WALLETDESCRIPTORLHCACHE, desc_id), key_exp_index), ser_xpub);
}
bool WalletBatch::WriteDescriptorCacheItems(const uint256& desc_id, const DescriptorCache& cache)
{
for (const auto& parent_xpub_pair : cache.GetCachedParentExtPubKeys()) {
if (!WriteDescriptorParentCache(parent_xpub_pair.second, desc_id, parent_xpub_pair.first)) {
return false;
}
}
for (const auto& derived_xpub_map_pair : cache.GetCachedDerivedExtPubKeys()) {
for (const auto& derived_xpub_pair : derived_xpub_map_pair.second) {
if (!WriteDescriptorDerivedCache(derived_xpub_pair.second, desc_id, derived_xpub_map_pair.first, derived_xpub_pair.first)) {
return false;
}
}
}
for (const auto& lh_xpub_pair : cache.GetCachedLastHardenedExtPubKeys()) {
if (!WriteDescriptorLastHardenedCache(lh_xpub_pair.second, desc_id, lh_xpub_pair.first)) {
return false;
}
}
return true;
}
class CWalletScanState { class CWalletScanState {
public: public:
unsigned int nKeys{0}; unsigned int nKeys{0};
@ -516,7 +546,7 @@ ReadKeyValue(CWallet* pwallet, CDataStream& ssKey, CDataStream& ssValue,
CHDChain chain; CHDChain chain;
ssValue >> chain; ssValue >> chain;
assert ((strType == DBKeys::CRYPTED_HDCHAIN) == chain.IsCrypted()); assert ((strType == DBKeys::CRYPTED_HDCHAIN) == chain.IsCrypted());
if (!pwallet->GetOrCreateLegacyScriptPubKeyMan()->SetHDChainSingle(chain, true)) if (!pwallet->GetOrCreateLegacyScriptPubKeyMan()->LoadHDChain(chain))
{ {
strErr = "Error reading wallet database: SetHDChain failed"; strErr = "Error reading wallet database: SetHDChain failed";
return false; return false;
@ -554,13 +584,6 @@ ReadKeyValue(CWallet* pwallet, CDataStream& ssKey, CDataStream& ssValue,
strErr = "Invalid governance object: LoadGovernanceObject"; strErr = "Invalid governance object: LoadGovernanceObject";
return false; return false;
} }
} else if (strType == DBKeys::FLAGS) {
uint64_t flags;
ssValue >> flags;
if (!pwallet->SetWalletFlags(flags, true)) {
strErr = "Error reading wallet database: Unknown non-tolerable wallet flags found";
return false;
}
} else if (strType == DBKeys::OLD_KEY) { } else if (strType == DBKeys::OLD_KEY) {
strErr = "Found unsupported 'wkey' record, try loading with version 0.17"; strErr = "Found unsupported 'wkey' record, try loading with version 0.17";
return false; return false;
@ -610,6 +633,17 @@ ReadKeyValue(CWallet* pwallet, CDataStream& ssKey, CDataStream& ssValue,
} else { } else {
wss.m_descriptor_caches[desc_id].CacheDerivedExtPubKey(key_exp_index, der_index, xpub); wss.m_descriptor_caches[desc_id].CacheDerivedExtPubKey(key_exp_index, der_index, xpub);
} }
} else if (strType == DBKeys::WALLETDESCRIPTORLHCACHE) {
uint256 desc_id;
uint32_t key_exp_index;
ssKey >> desc_id;
ssKey >> key_exp_index;
std::vector<unsigned char> ser_xpub(BIP32_EXTKEY_SIZE);
ssValue >> ser_xpub;
CExtPubKey xpub;
xpub.Decode(ser_xpub.data());
wss.m_descriptor_caches[desc_id].CacheLastHardenedExtPubKey(key_exp_index, xpub);
} else if (strType == DBKeys::WALLETDESCRIPTORKEY) { } else if (strType == DBKeys::WALLETDESCRIPTORKEY) {
uint256 desc_id; uint256 desc_id;
CPubKey pubkey; CPubKey pubkey;
@ -665,7 +699,8 @@ ReadKeyValue(CWallet* pwallet, CDataStream& ssKey, CDataStream& ssValue,
} else if (strType != DBKeys::BESTBLOCK && strType != DBKeys::BESTBLOCK_NOMERKLE && } else if (strType != DBKeys::BESTBLOCK && strType != DBKeys::BESTBLOCK_NOMERKLE &&
strType != DBKeys::MINVERSION && strType != DBKeys::ACENTRY && strType != DBKeys::MINVERSION && strType != DBKeys::ACENTRY &&
strType != DBKeys::VERSION && strType != DBKeys::SETTINGS && strType != DBKeys::VERSION && strType != DBKeys::SETTINGS &&
strType != DBKeys::PRIVATESEND_SALT && strType != DBKeys::COINJOIN_SALT) { strType != DBKeys::PRIVATESEND_SALT && strType != DBKeys::COINJOIN_SALT &&
strType != DBKeys::FLAGS) {
wss.m_unknown_records++; wss.m_unknown_records++;
} }
} catch (const std::exception& e) { } catch (const std::exception& e) {
@ -711,6 +746,16 @@ DBErrors WalletBatch::LoadWallet(CWallet* pwallet)
pwallet->LoadMinVersion(nMinVersion); pwallet->LoadMinVersion(nMinVersion);
} }
// Load wallet flags, so they are known when processing other records.
// The FLAGS key is absent during wallet creation.
uint64_t flags;
if (m_batch->Read(DBKeys::FLAGS, flags)) {
if (!pwallet->LoadWalletFlags(flags)) {
pwallet->WalletLogPrintf("Error reading wallet database: Unknown non-tolerable wallet flags found\n");
return DBErrors::CORRUPT;
}
}
// Get cursor // Get cursor
if (!m_batch->StartCursor()) if (!m_batch->StartCursor())
{ {
@ -768,10 +813,10 @@ DBErrors WalletBatch::LoadWallet(CWallet* pwallet)
// Set the active ScriptPubKeyMans // Set the active ScriptPubKeyMans
for (auto spk_man : wss.m_active_external_spks) { for (auto spk_man : wss.m_active_external_spks) {
pwallet->SetActiveScriptPubKeyMan(spk_man.second, /* internal */ false, /* memonly */ true); pwallet->LoadActiveScriptPubKeyMan(spk_man.second, /* internal */ false);
} }
for (auto spk_man : wss.m_active_internal_spks) { for (auto spk_man : wss.m_active_internal_spks) {
pwallet->SetActiveScriptPubKeyMan(spk_man.second, /* internal */ true, /* memonly */ true); pwallet->LoadActiveScriptPubKeyMan(spk_man.second, /* internal */ true);
} }
// Set the descriptor caches // Set the descriptor caches
@ -840,6 +885,14 @@ DBErrors WalletBatch::LoadWallet(CWallet* pwallet)
result = DBErrors::CORRUPT; result = DBErrors::CORRUPT;
} }
// Upgrade all of the descriptor caches to cache the last hardened xpub
// This operation is not atomic, but if it fails, only new entries are added so it is backwards compatible
try {
pwallet->UpgradeDescriptorCache();
} catch (...) {
result = DBErrors::CORRUPT;
}
return result; return result;
} }

2
src/wallet/walletdb.h
View File

@ -217,6 +217,8 @@ public:
bool WriteDescriptor(const uint256& desc_id, const WalletDescriptor& descriptor); bool WriteDescriptor(const uint256& desc_id, const WalletDescriptor& descriptor);
bool WriteDescriptorDerivedCache(const CExtPubKey& xpub, const uint256& desc_id, uint32_t key_exp_index, uint32_t der_index); bool WriteDescriptorDerivedCache(const CExtPubKey& xpub, const uint256& desc_id, uint32_t key_exp_index, uint32_t der_index);
bool WriteDescriptorParentCache(const CExtPubKey& xpub, const uint256& desc_id, uint32_t key_exp_index); bool WriteDescriptorParentCache(const CExtPubKey& xpub, const uint256& desc_id, uint32_t key_exp_index);
bool WriteDescriptorLastHardenedCache(const CExtPubKey& xpub, const uint256& desc_id, uint32_t key_exp_index);
bool WriteDescriptorCacheItems(const uint256& desc_id, const DescriptorCache& cache);
/// Write destination data key,value tuple to database /// Write destination data key,value tuple to database
bool WriteDestData(const std::string &address, const std::string &key, const std::string &value); bool WriteDestData(const std::string &address, const std::string &key, const std::string &value);

14
src/wallet/walletutil.cpp
View File

@ -28,3 +28,17 @@ fs::path GetWalletDir()
return path; return path;
} }
bool IsFeatureSupported(int wallet_version, int feature_version)
{
return wallet_version >= feature_version;
}
WalletFeature GetClosestWalletFeature(int version)
{
const std::array<WalletFeature, 5> wallet_features{{FEATURE_LATEST, FEATURE_HD, FEATURE_COMPRPUBKEY, FEATURE_WALLETCRYPT, FEATURE_BASE}};
for (const WalletFeature& wf : wallet_features) {
if (version >= wf) return wf;
}
return static_cast<WalletFeature>(0);
}

5
src/wallet/walletutil.h
View File

@ -23,6 +23,8 @@ enum WalletFeature
FEATURE_LATEST = FEATURE_HD FEATURE_LATEST = FEATURE_HD
}; };
bool IsFeatureSupported(int wallet_version, int feature_version);
WalletFeature GetClosestWalletFeature(int version);
enum WalletFlags : uint64_t { enum WalletFlags : uint64_t {
// wallet flags in the upper section (> 1 << 31) will lead to not opening the wallet if flag is unknown // wallet flags in the upper section (> 1 << 31) will lead to not opening the wallet if flag is unknown
@ -35,6 +37,9 @@ enum WalletFlags : uint64_t {
// Indicates that the metadata has already been upgraded to contain key origins // Indicates that the metadata has already been upgraded to contain key origins
WALLET_FLAG_KEY_ORIGIN_METADATA = (1ULL << 1), WALLET_FLAG_KEY_ORIGIN_METADATA = (1ULL << 1),
// Indicates that the descriptor cache has been upgraded to cache last hardened xpubs
WALLET_FLAG_LAST_HARDENED_XPUB_CACHED = (1ULL << 2),
// will enforce the rule that the wallet can't contain any private keys (only watch-only/pubkeys) // will enforce the rule that the wallet can't contain any private keys (only watch-only/pubkeys)
WALLET_FLAG_DISABLE_PRIVATE_KEYS = (1ULL << 32), WALLET_FLAG_DISABLE_PRIVATE_KEYS = (1ULL << 32),

0
test/functional/data/wallets/high_minversion/.walletlock
View File

8
test/functional/data/wallets/high_minversion/GENERATE.md
View File

@ -1,8 +0,0 @@
The wallet has been created by starting Bitcoin Core with the options
`-regtest -datadir=/tmp -nowallet -walletdir=$(pwd)/test/functional/data/wallets/`.
In the source code, `WalletFeature::FEATURE_LATEST` has been modified to be large, so that the minversion is too high
for a current build of the wallet.
The wallet has then been created with the RPC `createwallet high_minversion true true`, so that a blank wallet with
private keys disabled is created.

0
test/functional/data/wallets/high_minversion/db.log
View File

BIN
test/functional/data/wallets/high_minversion/wallet.dat
View File

Binary file not shown.

152
test/functional/test_framework/bdb.py Normal file
View File

@ -0,0 +1,152 @@
#!/usr/bin/env python3
# Copyright (c) 2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""
Utilities for working directly with the wallet's BDB database file
This is specific to the configuration of BDB used in this project:
- pagesize: 4096 bytes
- Outer database contains single subdatabase named 'main'
- btree
- btree leaf pages
Each key-value pair is two entries in a btree leaf. The first is the key, the one that follows
is the value. And so on. Note that the entry data is itself not in the correct order. Instead
entry offsets are stored in the correct order and those offsets are needed to then retrieve
the data itself.
Page format can be found in BDB source code dbinc/db_page.h
This only implements the deserialization of btree metadata pages and normal btree pages. Overflow
pages are not implemented but may be needed in the future if dealing with wallets with large
transactions.
`db_dump -da wallet.dat` is useful to see the data in a wallet.dat BDB file
"""
import binascii
import struct
# Important constants
PAGESIZE = 4096
OUTER_META_PAGE = 0
INNER_META_PAGE = 2
# Page type values
BTREE_INTERNAL = 3
BTREE_LEAF = 5
BTREE_META = 9
# Some magic numbers for sanity checking
BTREE_MAGIC = 0x053162
DB_VERSION = 9
# Deserializes a leaf page into a dict.
# Btree internal pages have the same header, for those, return None.
# For the btree leaf pages, deserialize them and put all the data into a dict
def dump_leaf_page(data):
page_info = {}
page_header = data[0:26]
_, pgno, prev_pgno, next_pgno, entries, hf_offset, level, pg_type = struct.unpack('QIIIHHBB', page_header)
page_info['pgno'] = pgno
page_info['prev_pgno'] = prev_pgno
page_info['next_pgno'] = next_pgno
page_info['entries'] = entries
page_info['hf_offset'] = hf_offset
page_info['level'] = level
page_info['pg_type'] = pg_type
page_info['entry_offsets'] = struct.unpack('{}H'.format(entries), data[26:26 + entries * 2])
page_info['entries'] = []
if pg_type == BTREE_INTERNAL:
# Skip internal pages. These are the internal nodes of the btree and don't contain anything relevant to us
return None
assert pg_type == BTREE_LEAF, 'A non-btree leaf page has been encountered while dumping leaves'
for i in range(0, entries):
offset = page_info['entry_offsets'][i]
entry = {'offset': offset}
page_data_header = data[offset:offset + 3]
e_len, pg_type = struct.unpack('HB', page_data_header)
entry['len'] = e_len
entry['pg_type'] = pg_type
entry['data'] = data[offset + 3:offset + 3 + e_len]
page_info['entries'].append(entry)
return page_info
# Deserializes a btree metadata page into a dict.
# Does a simple sanity check on the magic value, type, and version
def dump_meta_page(page):
# metadata page
# general metadata
metadata = {}
meta_page = page[0:72]
_, pgno, magic, version, pagesize, encrypt_alg, pg_type, metaflags, _, free, last_pgno, nparts, key_count, record_count, flags, uid = struct.unpack('QIIIIBBBBIIIIII20s', meta_page)
metadata['pgno'] = pgno
metadata['magic'] = magic
metadata['version'] = version
metadata['pagesize'] = pagesize
metadata['encrypt_alg'] = encrypt_alg
metadata['pg_type'] = pg_type
metadata['metaflags'] = metaflags
metadata['free'] = free
metadata['last_pgno'] = last_pgno
metadata['nparts'] = nparts
metadata['key_count'] = key_count
metadata['record_count'] = record_count
metadata['flags'] = flags
metadata['uid'] = binascii.hexlify(uid)
assert magic == BTREE_MAGIC, 'bdb magic does not match bdb btree magic'
assert pg_type == BTREE_META, 'Metadata page is not a btree metadata page'
assert version == DB_VERSION, 'Database too new'
# btree metadata
btree_meta_page = page[72:512]
_, minkey, re_len, re_pad, root, _, crypto_magic, _, iv, chksum = struct.unpack('IIIII368sI12s16s20s', btree_meta_page)
metadata['minkey'] = minkey
metadata['re_len'] = re_len
metadata['re_pad'] = re_pad
metadata['root'] = root
metadata['crypto_magic'] = crypto_magic
metadata['iv'] = binascii.hexlify(iv)
metadata['chksum'] = binascii.hexlify(chksum)
return metadata
# Given the dict from dump_leaf_page, get the key-value pairs and put them into a dict
def extract_kv_pairs(page_data):
out = {}
last_key = None
for i, entry in enumerate(page_data['entries']):
# By virtue of these all being pairs, even number entries are keys, and odd are values
if i % 2 == 0:
out[entry['data']] = b''
last_key = entry['data']
else:
out[last_key] = entry['data']
return out
# Extract the key-value pairs of the BDB file given in filename
def dump_bdb_kv(filename):
# Read in the BDB file and start deserializing it
pages = []
with open(filename, 'rb') as f:
data = f.read(PAGESIZE)
while len(data) > 0:
pages.append(data)
data = f.read(PAGESIZE)
# Sanity check the meta pages
dump_meta_page(pages[OUTER_META_PAGE])
dump_meta_page(pages[INNER_META_PAGE])
# Fetch the kv pairs from the leaf pages
kv = {}
for i in range(3, len(pages)):
info = dump_leaf_page(pages[i])
if info is not None:
info_kv = extract_kv_pairs(info)
kv = {**kv, **info_kv}
return kv

9
test/functional/test_framework/util.py
View File

@ -9,6 +9,7 @@ from base64 import b64encode
from binascii import unhexlify from binascii import unhexlify
from decimal import Decimal, ROUND_DOWN from decimal import Decimal, ROUND_DOWN
from subprocess import CalledProcessError from subprocess import CalledProcessError
import hashlib
import inspect import inspect
import json import json
import logging import logging
@ -268,6 +269,14 @@ def wait_until_helper(predicate, *, attempts=float('inf'), timeout=float('inf'),
else: else:
return False return False
def sha256sum_file(filename):
h = hashlib.sha256()
with open(filename, 'rb') as f:
d = f.read(4096)
while len(d) > 0:
h.update(d)
d = f.read(4096)
return h.digest()
# RPC/P2P connection constants and functions # RPC/P2P connection constants and functions
############################################ ############################################

144
test/functional/wallet_hd.py
View File

@ -11,6 +11,7 @@ from test_framework.blocktools import COINBASE_MATURITY
from test_framework.test_framework import BitcoinTestFramework from test_framework.test_framework import BitcoinTestFramework
from test_framework.util import ( from test_framework.util import (
assert_equal, assert_equal,
assert_raises_rpc_error,
) )
class WalletHDTest(BitcoinTestFramework): class WalletHDTest(BitcoinTestFramework):
@ -137,5 +138,148 @@ class WalletHDTest(BitcoinTestFramework):
assert_equal(keypath[0:13], "m/44'/1'/0'/1") assert_equal(keypath[0:13], "m/44'/1'/0'/1")
if not self.options.descriptors:
# NOTE: sethdseed can't replace existing seed in Dash Core
# though bitcoin lets to do it. Therefore this functional test
# are not the same with bitcoin's
# Generate a new HD seed on node 1 and make sure it is set
self.nodes[1].createwallet(wallet_name='wallet_new_seed', blank=True)
wallet_new_seed = self.nodes[1].get_wallet_rpc('wallet_new_seed')
assert 'hdchainid' not in wallet_new_seed.getwalletinfo()
wallet_new_seed.sethdseed()
new_masterkeyid = wallet_new_seed.getwalletinfo()['hdchainid']
addr = wallet_new_seed.getnewaddress()
# Make sure the new address is the first from the keypool
assert_equal(wallet_new_seed.getaddressinfo(addr)['hdkeypath'], "m/44'/1'/0'/0/1")
wallet_new_seed.keypoolrefill(1) # Fill keypool with 1 key
# Set a new HD seed on node 1 without flushing the keypool
new_seed = self.nodes[0].dumpprivkey(self.nodes[0].getnewaddress())
assert_raises_rpc_error(-4, "Cannot set a HD seed. The wallet already has a seed", wallet_new_seed.sethdseed, False, new_seed)
self.nodes[1].createwallet(wallet_name='wallet_imported_seed', blank=True)
wallet_imported_seed = self.nodes[1].get_wallet_rpc('wallet_imported_seed')
wallet_imported_seed.sethdseed(False, new_seed)
new_masterkeyid = wallet_imported_seed.getwalletinfo()['hdchainid']
addr = wallet_imported_seed.getnewaddress()
assert_equal(new_masterkeyid, wallet_imported_seed.getaddressinfo(addr)['hdchainid'])
# Make sure the new address continues previous keypool
assert_equal(wallet_imported_seed.getaddressinfo(addr)['hdkeypath'], "m/44'/1'/0'/0/0")
# Check that the next address is from the new seed
wallet_imported_seed.keypoolrefill(1)
next_addr = wallet_imported_seed.getnewaddress()
assert_equal(new_masterkeyid, wallet_imported_seed.getaddressinfo(next_addr)['hdchainid'])
# Make sure the new address is not from previous keypool
assert_equal(wallet_imported_seed.getaddressinfo(next_addr)['hdkeypath'], "m/44'/1'/0'/0/1")
assert next_addr != addr
self.nodes[1].createwallet(wallet_name='wallet_no_seed', blank=True)
wallet_no_seed = self.nodes[1].get_wallet_rpc('wallet_no_seed')
wallet_no_seed.importprivkey(non_hd_key)
# Sethdseed parameter validity
assert_raises_rpc_error(-1, 'sethdseed', self.nodes[0].sethdseed, False, new_seed, 0)
assert_raises_rpc_error(-5, "Invalid private key", wallet_no_seed.sethdseed, False, "not_wif")
assert_raises_rpc_error(-1, "JSON value is not a boolean as expected", wallet_no_seed.sethdseed, "Not_bool")
assert_raises_rpc_error(-1, "JSON value is not a string as expected", wallet_no_seed.sethdseed, False, True)
assert_raises_rpc_error(-5, "Already have this key", wallet_no_seed.sethdseed, False, non_hd_key)
self.log.info('Test sethdseed restoring with keys outside of the initial keypool')
self.nodes[0].generate(10)
# Restart node 1 with keypool of 3 and a different wallet
self.nodes[1].createwallet(wallet_name='origin', blank=True)
self.restart_node(1, extra_args=['-keypool=3', '-wallet=origin'])
self.connect_nodes(0, 1)
# sethdseed restoring and seeing txs to addresses out of the keypool
origin_rpc = self.nodes[1].get_wallet_rpc('origin')
seed = self.nodes[0].dumpprivkey(self.nodes[0].getnewaddress())
origin_rpc.sethdseed(True, seed)
self.nodes[1].createwallet(wallet_name='restore', blank=True)
restore_rpc = self.nodes[1].get_wallet_rpc('restore')
restore_rpc.sethdseed(True, seed) # Set to be the same seed as origin_rpc
self.nodes[1].createwallet(wallet_name='restore2', blank=True)
restore2_rpc = self.nodes[1].get_wallet_rpc('restore2')
restore2_rpc.sethdseed(True, seed) # Set to be the same seed as origin_rpc
# Check persistence of inactive seed by reloading restore. restore2 is still loaded to test the case where the wallet is not reloaded
restore_rpc.unloadwallet()
self.nodes[1].loadwallet('restore')
restore_rpc = self.nodes[1].get_wallet_rpc('restore')
# Empty origin keypool and get an address that is beyond the initial keypool
origin_rpc.getnewaddress()
origin_rpc.getnewaddress()
last_addr = origin_rpc.getnewaddress() # Last address of initial keypool
addr = origin_rpc.getnewaddress() # First address beyond initial keypool
# Check that the restored seed has last_addr but does not have addr
info = restore_rpc.getaddressinfo(last_addr)
assert_equal(info['ismine'], True)
info = restore_rpc.getaddressinfo(addr)
assert_equal(info['ismine'], False)
info = restore2_rpc.getaddressinfo(last_addr)
assert_equal(info['ismine'], True)
info = restore2_rpc.getaddressinfo(addr)
assert_equal(info['ismine'], False)
# Check that the origin seed has addr
info = origin_rpc.getaddressinfo(addr)
assert_equal(info['ismine'], True)
# Send a transaction to addr, which is out of the initial keypool.
# The wallet that has set a new seed (restore_rpc) should not detect this transaction.
txid = self.nodes[0].sendtoaddress(addr, 1)
origin_rpc.sendrawtransaction(self.nodes[0].gettransaction(txid)['hex'])
self.nodes[0].generate(1)
self.sync_blocks()
origin_rpc.gettransaction(txid)
assert_raises_rpc_error(-5, 'Invalid or non-wallet transaction id', restore_rpc.gettransaction, txid)
out_of_kp_txid = txid
# Send a transaction to last_addr, which is in the initial keypool.
# The wallet that has set a new seed (restore_rpc) should detect this transaction and generate 3 new keys from the initial seed.
# The previous transaction (out_of_kp_txid) should still not be detected as a rescan is required.
txid = self.nodes[0].sendtoaddress(last_addr, 1)
origin_rpc.sendrawtransaction(self.nodes[0].gettransaction(txid)['hex'])
self.nodes[0].generate(1)
self.sync_blocks()
origin_rpc.gettransaction(txid)
restore_rpc.gettransaction(txid)
assert_raises_rpc_error(-5, 'Invalid or non-wallet transaction id', restore_rpc.gettransaction, out_of_kp_txid)
restore2_rpc.gettransaction(txid)
assert_raises_rpc_error(-5, 'Invalid or non-wallet transaction id', restore2_rpc.gettransaction, out_of_kp_txid)
# After rescanning, restore_rpc should now see out_of_kp_txid and generate an additional key.
# addr should now be part of restore_rpc and be ismine
restore_rpc.rescanblockchain()
restore_rpc.gettransaction(out_of_kp_txid)
info = restore_rpc.getaddressinfo(addr)
assert_equal(info['ismine'], True)
restore2_rpc.rescanblockchain()
restore2_rpc.gettransaction(out_of_kp_txid)
info = restore2_rpc.getaddressinfo(addr)
assert_equal(info['ismine'], True)
# Check again that 3 keys were derived.
# Empty keypool and get an address that is beyond the initial keypool
origin_rpc.getnewaddress()
origin_rpc.getnewaddress()
last_addr = origin_rpc.getnewaddress()
addr = origin_rpc.getnewaddress()
# Check that the restored seed has last_addr but does not have addr
info = restore_rpc.getaddressinfo(last_addr)
assert_equal(info['ismine'], True)
info = restore_rpc.getaddressinfo(addr)
assert_equal(info['ismine'], False)
info = restore2_rpc.getaddressinfo(last_addr)
assert_equal(info['ismine'], True)
info = restore2_rpc.getaddressinfo(addr)
assert_equal(info['ismine'], False)
if __name__ == '__main__': if __name__ == '__main__':
WalletHDTest().main () WalletHDTest().main ()

4
test/functional/wallet_listdescriptors.py
View File

@ -73,6 +73,10 @@ class ListDescriptorsTest(BitcoinTestFramework):
} }
assert_equal(expected, wallet.listdescriptors()) assert_equal(expected, wallet.listdescriptors())
self.log.info("Test listdescriptors with encrypted wallet")
wallet.encryptwallet("pass")
assert_equal(expected, wallet.listdescriptors())
self.log.info('Test non-active non-range combo descriptor') self.log.info('Test non-active non-range combo descriptor')
node.createwallet(wallet_name='w4', blank=True, descriptors=True) node.createwallet(wallet_name='w4', blank=True, descriptors=True)
wallet = node.get_wallet_rpc('w4') wallet = node.get_wallet_rpc('w4')

158
test/functional/wallet_upgradewallet.py
View File

@ -10,25 +10,29 @@ Only v0.15.2 and v0.16.3 are required by this test. The others are used in featu
import os import os
import shutil import shutil
import struct
from test_framework.bdb import dump_bdb_kv
from test_framework.blocktools import COINBASE_MATURITY from test_framework.blocktools import COINBASE_MATURITY
from test_framework.test_framework import BitcoinTestFramework from test_framework.test_framework import BitcoinTestFramework
from test_framework.util import ( from test_framework.util import (
assert_equal, assert_equal,
assert_greater_than, assert_greater_than,
assert_greater_than_or_equal,
assert_is_hex_string, assert_is_hex_string,
sha256sum_file,
) )
UPGRADED_KEYMETA_VERSION = 12
class UpgradeWalletTest(BitcoinTestFramework): class UpgradeWalletTest(BitcoinTestFramework):
def set_test_params(self): def set_test_params(self):
self.setup_clean_chain = True self.setup_clean_chain = True
self.num_nodes = 3 self.num_nodes = 3
self.extra_args = [ self.extra_args = [
[], # current wallet version ["-keypool=2"], # current wallet version
["-usehd=1"], # v18.2.2 wallet ["-usehd=1", "-keypool=2"], # v18.2.2 wallet
["-usehd=0"] # v0.16.1.1 wallet ["-usehd=0", "-keypool=2"], # v0.16.1.1 wallet
] ]
self.wallet_names = [self.default_wallet_name, None, None] self.wallet_names = [self.default_wallet_name, None, None]
@ -65,6 +69,32 @@ class UpgradeWalletTest(BitcoinTestFramework):
v18_2_node.submitblock(b) v18_2_node.submitblock(b)
assert_equal(v18_2_node.getblockcount(), to_height) assert_equal(v18_2_node.getblockcount(), to_height)
def test_upgradewallet(self, wallet, previous_version, requested_version=None, expected_version=None):
unchanged = expected_version == previous_version
new_version = previous_version if unchanged else expected_version if expected_version else requested_version
assert_equal(wallet.getwalletinfo()["walletversion"], previous_version)
assert_equal(wallet.upgradewallet(requested_version),
{
"wallet_name": "",
"previous_version": previous_version,
"current_version": new_version,
"result": "Already at latest version. Wallet version unchanged." if unchanged else "Wallet upgraded successfully from version {} to version {}.".format(previous_version, new_version),
}
)
assert_equal(wallet.getwalletinfo()["walletversion"], new_version)
def test_upgradewallet_error(self, wallet, previous_version, requested_version, msg):
assert_equal(wallet.getwalletinfo()["walletversion"], previous_version)
assert_equal(wallet.upgradewallet(requested_version),
{
"wallet_name": "",
"previous_version": previous_version,
"current_version": previous_version,
"error": msg,
}
)
assert_equal(wallet.getwalletinfo()["walletversion"], previous_version)
def run_test(self): def run_test(self):
self.nodes[0].generatetoaddress(COINBASE_MATURITY + 1, self.nodes[0].getnewaddress()) self.nodes[0].generatetoaddress(COINBASE_MATURITY + 1, self.nodes[0].getnewaddress())
self.dumb_sync_blocks() self.dumb_sync_blocks()
@ -84,12 +114,12 @@ class UpgradeWalletTest(BitcoinTestFramework):
self.log.info("Test upgradewallet RPC...") self.log.info("Test upgradewallet RPC...")
# Prepare for copying of the older wallet # Prepare for copying of the older wallet
node_master_wallet_dir = os.path.join(node_master.datadir, "regtest/wallets") node_master_wallet_dir = os.path.join(node_master.datadir, "regtest/wallets", self.default_wallet_name)
node_master_wallet = os.path.join(node_master_wallet_dir, self.default_wallet_name, self.wallet_data_filename)
v18_2_wallet = os.path.join(v18_2_node.datadir, "regtest/wallets/wallet.dat") v18_2_wallet = os.path.join(v18_2_node.datadir, "regtest/wallets/wallet.dat")
v16_1_wallet = os.path.join(v16_1_node.datadir, "regtest/wallets/wallet.dat") v16_1_wallet = os.path.join(v16_1_node.datadir, "regtest/wallets/wallet.dat")
self.stop_nodes() self.stop_nodes()
# Copy the 0.16.3 wallet to the last Dash Core version and open it:
shutil.rmtree(node_master_wallet_dir) shutil.rmtree(node_master_wallet_dir)
os.mkdir(node_master_wallet_dir) os.mkdir(node_master_wallet_dir)
shutil.copy( shutil.copy(
@ -99,39 +129,45 @@ class UpgradeWalletTest(BitcoinTestFramework):
self.restart_node(0, ['-nowallet']) self.restart_node(0, ['-nowallet'])
node_master.loadwallet('') node_master.loadwallet('')
wallet = node_master.get_wallet_rpc('') def copy_v16():
old_version = wallet.getwalletinfo()["walletversion"] node_master.get_wallet_rpc(self.default_wallet_name).unloadwallet()
# Copy the 0.16.3 wallet to the last Dash Core version and open it:
shutil.rmtree(node_master_wallet_dir)
os.mkdir(node_master_wallet_dir)
shutil.copy(
v18_2_wallet,
node_master_wallet_dir
)
node_master.loadwallet(self.default_wallet_name)
# calling upgradewallet without version arguments def copy_non_hd():
# should return nothing if successful node_master.get_wallet_rpc(self.default_wallet_name).unloadwallet()
assert_equal(wallet.upgradewallet(), {}) # Copy the 19.3.0 wallet to the last Dash Core version and open it:
new_version = wallet.getwalletinfo()["walletversion"] shutil.rmtree(node_master_wallet_dir)
# upgraded wallet version should be greater than older one os.mkdir(node_master_wallet_dir)
assert_greater_than_or_equal(new_version, old_version) shutil.copy(
v16_1_wallet,
node_master_wallet_dir
)
node_master.loadwallet(self.default_wallet_name)
self.restart_node(0)
copy_v16()
wallet = node_master.get_wallet_rpc(self.default_wallet_name)
self.test_upgradewallet(wallet, previous_version=120200, expected_version=120200)
# wallet should still contain the same balance # wallet should still contain the same balance
assert_equal(wallet.getbalance(), v18_2_balance) assert_equal(wallet.getbalance(), v18_2_balance)
self.stop_node(0) copy_non_hd()
# Copy the 19.3.0 wallet to the last Dash Core version and open it: wallet = node_master.get_wallet_rpc(self.default_wallet_name)
shutil.rmtree(node_master_wallet_dir)
os.mkdir(node_master_wallet_dir)
shutil.copy(
v16_1_wallet,
node_master_wallet_dir
)
self.restart_node(0, ['-nowallet'])
node_master.loadwallet('')
wallet = node_master.get_wallet_rpc('')
# should have no master key hash before conversion # should have no master key hash before conversion
assert_equal('hdseedid' in wallet.getwalletinfo(), False) assert_equal('hdchainid' in wallet.getwalletinfo(), False)
# calling upgradewallet with explicit version number # calling upgradewallet with explicit version number
# should return nothing if successful # should return nothing if successful
assert_equal(wallet.upgradewallet(169900), {}) self.log.info("Test upgradewallet to HD will have version 120200 but no HD seed actually appeared")
new_version = wallet.getwalletinfo()["walletversion"] self.test_upgradewallet(wallet, previous_version=61000, expected_version=120200, requested_version=169900)
# upgraded wallet would not have 120200 version until HD seed actually appeared
assert_greater_than(120200, new_version)
# after conversion master key hash should not be present yet # after conversion master key hash should not be present yet
assert 'hdchainid' not in wallet.getwalletinfo() assert 'hdchainid' not in wallet.getwalletinfo()
assert_equal(wallet.upgradetohd(), True) assert_equal(wallet.upgradetohd(), True)
@ -139,5 +175,65 @@ class UpgradeWalletTest(BitcoinTestFramework):
assert_equal(new_version, 120200) assert_equal(new_version, 120200)
assert_is_hex_string(wallet.getwalletinfo()['hdchainid']) assert_is_hex_string(wallet.getwalletinfo()['hdchainid'])
self.log.info("Intermediary versions don't effect anything")
copy_non_hd()
# Wallet starts with 61000 (legacy "latest")
assert_equal(61000, wallet.getwalletinfo()['walletversion'])
wallet.unloadwallet()
before_checksum = sha256sum_file(node_master_wallet)
node_master.loadwallet('')
# Test an "upgrade" from 61000 to 120199 has no effect, as the next version is 120200
self.test_upgradewallet(wallet, previous_version=61000, requested_version=120199, expected_version=61000)
wallet.unloadwallet()
assert_equal(before_checksum, sha256sum_file(node_master_wallet))
node_master.loadwallet('')
self.log.info('Wallets cannot be downgraded')
copy_non_hd()
self.test_upgradewallet_error(wallet, previous_version=61000, requested_version=40000,
msg="Cannot downgrade wallet from version 61000 to version 40000. Wallet version unchanged.")
wallet.unloadwallet()
assert_equal(before_checksum, sha256sum_file(node_master_wallet))
node_master.loadwallet('')
self.log.info('Can upgrade to HD')
# Inspect the old wallet and make sure there is no hdchain
orig_kvs = dump_bdb_kv(node_master_wallet)
assert b'\x07hdchain' not in orig_kvs
# Upgrade to HD
self.test_upgradewallet(wallet, previous_version=61000, requested_version=120200)
# Check that there is now a hd chain and it is version 1, no internal chain counter
new_kvs = dump_bdb_kv(node_master_wallet)
wallet.upgradetohd()
new_kvs = dump_bdb_kv(node_master_wallet)
assert b'\x07hdchain' in new_kvs
hd_chain = new_kvs[b'\x07hdchain']
# hd_chain:
# obj.nVersion int
# obj.id uint256
# obj.fCrypted bool
# obj.vchSeed SecureVector
# obj.vchMnemonic SecureVector
# obj.vchMnemonicPassphrase SecureVector
# obj.mapAccounts map<> accounts
assert_greater_than(220, len(hd_chain))
assert_greater_than(len(hd_chain), 180)
hd_chain_version, seed_id, is_crypted = struct.unpack('<i32s?', hd_chain[:37])
assert_equal(1, hd_chain_version)
seed_id = bytearray(seed_id)
seed_id.reverse()
# Next key should be HD
info = wallet.getaddressinfo(wallet.getnewaddress())
assert_equal(seed_id.hex(), info['hdchainid'])
assert_equal("m/44'/1'/0'/0/0", info['hdkeypath'])
prev_seed_id = info['hdchainid']
# Change key should be the same keypool
info = wallet.getaddressinfo(wallet.getrawchangeaddress())
assert_equal(prev_seed_id, info['hdchainid'])
assert_equal("m/44'/1'/0'/1/0", info['hdkeypath'])
assert_equal(120200, wallet.getwalletinfo()['walletversion'])
if __name__ == '__main__': if __name__ == '__main__':
UpgradeWalletTest().main() UpgradeWalletTest().main()