Commit Graph

28 Commits

Author SHA1 Message Date
laanwj
e4f8b7097d
Merge bitcoin/bitcoin#24852: util: optimize HexStr
5e61532e72c1021fda9c7b213bd9cf397cb3a802 util: optimizes HexStr (Martin Leitner-Ankerl)
4e2b99f72a90b956f3050095abed4949aff9b516 bench: Adds a benchmark for HexStr (Martin Leitner-Ankerl)
67c8411c37b483caa2fe3f7f4f40b68ed2a9bcf7 test: Adds a test for HexStr that checks all 256 bytes (Martin Leitner-Ankerl)

Pull request description:

  In my benchmark, this rewrite improves runtime 27% (g++) to 46% (clang++) for the benchmark `HexStrBench`:

  g++ 11.2.0
  |             ns/byte |              byte/s |    err% |        ins/byte |        cyc/byte |    IPC |       bra/byte |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |                0.94 |    1,061,381,310.36 |    0.7% |           12.00 |            3.01 |  3.990 |           1.00 |    0.0% |      0.01 | `HexStrBench` master
  |                0.68 |    1,465,366,544.25 |    1.7% |            6.00 |            2.16 |  2.778 |           1.00 |    0.0% |      0.01 | `HexStrBench` branch

  clang++ 13.0.1
  |             ns/byte |              byte/s |    err% |        ins/byte |        cyc/byte |    IPC |       bra/byte |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |                0.80 |    1,244,713,415.92 |    0.9% |           10.00 |            2.56 |  3.913 |           0.50 |    0.0% |      0.01 | `HexStrBench` master
  |                0.43 |    2,324,188,940.72 |    0.2% |            4.00 |            1.37 |  2.914 |           0.25 |    0.0% |      0.01 | `HexStrBench` branch

  Note that the idea for this change comes from denis2342 in #23364. This is a rewrite so no unaligned accesses occur.

  Also, the lookup table is now calculated at compile time, which hopefully makes the code a bit easier to review.

ACKs for top commit:
  laanwj:
    Code review ACK 5e61532e72c1021fda9c7b213bd9cf397cb3a802
  aureleoules:
    tACK 5e61532e72c1021fda9c7b213bd9cf397cb3a802.
  theStack:
    ACK 5e61532e72c1021fda9c7b213bd9cf397cb3a802 🚤

Tree-SHA512: 40b53d5908332473ef24918d3a80ad1292b60566c02585fa548eb4c3189754971be5a70325f4968fce6d714df898b52d9357aba14d4753a8c70e6ffd273a2319
2024-10-15 09:25:09 -05:00
Kittywhiskers Van Gogh
eb9e20890f
merge bitcoin#23156: Remove unused ParsePrechecks and ParseDouble 2024-10-08 15:59:16 +00:00
Kittywhiskers Van Gogh
3d769c7a64
merge bitcoin#23249: ParseByteUnits - Parse a string with suffix unit 2024-10-05 17:10:03 +00:00
Kittywhiskers Van Gogh
cf4522f845
partial bitcoin#23595: Add ParseHex<std::byte>() helper
excludes:
- facd1fb911abfc595a3484ee53397eff515d4c40
2024-02-28 13:37:36 -06:00
Kittywhiskers Van Gogh
95b5850434
partial bitcoin#25001: Modernize util/strencodings and util/string: string_view and optional
includes:
- c1d165a8c2678c31aced5e1d46231d9996b0774a
- 40062997f223d88d4f92aaae4622a31476686163
- 963bc9b576f0a62caffede2ce32830aef3473995
- d648b5120b2fefa9e599898bd26f05ecf4428fac
- a4377a0843636eae0aaf698510fc6518582545db
2024-02-28 13:37:35 -06:00
Kittywhiskers Van Gogh
bb8bd9a94d merge bitcoin#23451: Add std::byte helpers 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
8c742e9d15 merge bitcoin#23756: Fix implicit integer sign changes in strencodings 2023-09-04 20:50:27 -05:00
Konstantin Akimov
4aa197dbdb Merge #18673: scripted-diff: Sort test includes
fa4632c41714dfaa699bacc6a947d72668a4deef test: Move boost/stdlib includes last (MarcoFalke)
fa488f131fd4f5bab0d01376c5a5013306f1abcd scripted-diff: Bump copyright headers (MarcoFalke)
fac5c373006a9e4bcbb56843bb85f1aca4d87599 scripted-diff: Sort test includes (MarcoFalke)

Pull request description:

  When writing tests, often includes need to be added or removed. Currently the list of includes is not sorted, so developers that write tests and have `clang-format` installed will either have an unrelated change (sorting) included in their commit or they will have to manually undo the sort.

  This pull preempts both issues by just sorting all includes in one commit.

  Please be aware that this is **NOT** a change to policy to enforce clang-format or any other developer guideline or process. Developers are free to use whatever tool they want, see also #18651.

  Edit: Also includes a commit to bump the copyright headers, so that the touched files don't need to be touched again for that.

ACKs for top commit:
  practicalswift:
    ACK fa4632c41714dfaa699bacc6a947d72668a4deef
  jonatack:
    ACK fa4632c41714dfaa, light review and sanity checks with gcc build and clang fuzz build

Tree-SHA512: 130a8d073a379ba556b1e64104d37c46b671425c0aef0ed725fd60156a95e8dc83fb6f0b5330b2f8152cf5daaf3983b4aca5e75812598f2626c39fd12b88b180
2023-08-29 22:00:59 -05:00
fanquake
15f6a9c26b Merge bitcoin/bitcoin#24297: Fix unintended unsigned integer overflow in strencodings
fac9fe5d051264fcd16e8e36d30f28c05c999837 Fix unintended unsigned integer overflow in strencodings (MarcoFalke)

Pull request description:

  This fixes two issues for strings that start with a colon and only have one colon:

  * `fMultiColon` is incorrectly set to `true`
  * There is an unsigned integer overflow `colon - 1` (`0 - 1`)

  Neither issue matters, as the result is discarded. Though, it makes sense to still fix the issue for clarity and to avoid sanitizer issues in the function.

ACKs for top commit:
  laanwj:
    Code review ACK fac9fe5d051264fcd16e8e36d30f28c05c999837
  shaavan:
    Code Review ACK fac9fe5d051264fcd16e8e36d30f28c05c999837

Tree-SHA512: e71c21a0b617abf241e561ce6b90b963e2d5e2f77bd9547ce47209a1a94b454384391f86ef5d35fedd4f4df19add3896bb3d61fed396ebba8e864e3eeb75ed59
2023-04-18 23:24:06 +03:00
Kittywhiskers Van Gogh
cae9d1791c merge bitcoin#20452: Replace use of locale dependent atoi(…) with locale-independent std::from_chars(…) 2023-01-02 14:19:33 -06:00
Kittywhiskers Van Gogh
bb0a52907d merge bitcoin#19750: remove unused c-string variant of atoi64() 2023-01-02 14:19:33 -06:00
Kittywhiskers Van Gogh
c76b22ec72 merge bitcoin#18449: Remove unused itostr 2023-01-02 14:19:33 -06:00
Kittywhiskers Van Gogh
8133469746 merge bitcoin#20457: Make Parse{Int,UInt}{32,64} use locale independent std::from_chars(…) (C++17) instead of locale dependent strto{l,ll,ul,ull} 2022-11-01 00:28:53 -05:00
Kittywhiskers Van Gogh
55e1853d65 merge bitcoin#21328: pass uint16 CService::port as uint16 2022-11-01 00:28:53 -05:00
fanquake
7f0bdbda11
Merge bitcoin/bitcoin#24298: fuzz: Avoid unsigned integer overflow in FormatParagraph
fa2f7d005932bff9b7d27744ae517b9e7910df8d fuzz: Avoid unsigned integer overflow in FormatParagraph (MarcoFalke)

Pull request description:

  `FormatParagraph` is only ever called with compile time constant arguments, so I don't see the need for fuzzing it.

  Though, keep it for now, but avoid the unsigned integer overflow with this patch.

ACKs for top commit:
  laanwj:
    Code review ACK fa2f7d005932bff9b7d27744ae517b9e7910df8d

Tree-SHA512: 01fc64a9ef73c183921ca1b0cd8db9514c0a242e3acf215a3393f383ae129e01625ebb16eaf9cb86370eda62d0145c3dcf8f62e40edf5958abc1f777c5687280
2022-09-03 10:43:16 -05:00
MeshCollider
ed0e2dd075
Merge #14021: Import key origin data through descriptors in importmulti (+ changes from PRs partially merged earlier: 15368, 15749)
cb3511b9d Add release notes for importing key origin info change (Andrew Chow)
4c75a69f3 Test importing descriptors with key origin information (Andrew Chow)
02d6586d7 Import KeyOriginData when importing descriptors (Andrew Chow)
3d235dff5 Implement a function to add KeyOriginInfo to a wallet (Andrew Chow)
eab63bc26 Store key origin info in key metadata (Andrew Chow)
345bff601 Remove hdmasterkeyid (Andrew Chow)
bac8c676a Add a method to CWallet to write just CKeyMetadata (Andrew Chow)
e7652d3f6 Add WriteHDKeypath function and move *HDKeypath to util/bip32.{h,cpp} (Andrew Chow)
c45415f73 Refactor keymetadata writing to a separate method (Andrew Chow)

Pull request description:

  This PR allows for key origin data as defined by the descriptors document to be imported to the wallet when importing a descriptor using `importmulti`. This allows the `walletprocesspsbt` to include the BIP 32 derivation paths for keys that it is watching that are from a different HD wallet.

  In order to make this easier to use, a new field `hdmasterkeyfingerprint` has been added to `getaddressinfo`. Additionally I have removed `hdmasterkeyid` as was planned. I think that this API change is fine since it was going to be removed in 0.18 anyways. `CKeyMetadata` has also been extended to store key origin info to facilitate this.

Tree-SHA512: 9c7794f3c793da57e23c5abbdc3d58779ee9dea3d53168bb86c0643a4ad5a11a446264961e2f772f35eea645048cb60954ed58050002caee4e43cd9f51215097
2022-02-18 03:42:05 +03:00
Kittywhiskers Van Gogh
e13d93ec78 merge bitcoin#16566: refactor upper/lowercase functions 2021-11-01 21:41:35 +05:30
MarcoFalke
67f1de4312 Merge #17753: util: Don't allow Base32/64-decoding or ParseMoney(…) on strings with embedded NUL characters. Add tests.
137c80d579502e329964d7d1028a9507d4667774 tests: Add tests for decoding/parsing of base32, base64 and money strings containing NUL characters (practicalswift)
a6fc26da55dea3b76bd89fbbca24ded170238674 util: Don't allow DecodeBase32(...) of strings with embedded NUL characters (practicalswift)
93cc18b0f6fa5fa8144079a4f51904d8b3087e94 util: Don't allow DecodeBase64(...) of strings with embedded NUL characters (practicalswift)
ccc53e43c5464058171d6291da861a88184b230e util: Don't allow ParseMoney(...) of strings with embedded NUL characters (practicalswift)

Pull request description:

  Don't allow Base32/64-decoding or `ParseMoney(…)` on strings with embedded `NUL` characters. Add tests.

  Added tests before:

  ```
  $ src/test/test_bitcoin
  Running 385 test cases...
  test/base32_tests.cpp(31): error: in "base32_tests/base32_testvectors":
      check failure == true has failed [false != true]
  test/base64_tests.cpp(31): error: in "base64_tests/base64_testvectors":
      check failure == true has failed [false != true]
  test/util_tests.cpp(1074): error: in "util_tests/util_ParseMoney":
      check !ParseMoney(std::string("\0-1", 3), ret) has failed
  test/util_tests.cpp(1076): error: in "util_tests/util_ParseMoney":
      check !ParseMoney(std::string("1\0", 2), ret) has failed

  *** 4 failures are detected in the test module "Bitcoin Core Test Suite"
  ```

  Added tests after:

  ```
  $ src/test/test_bitcoin
  Running 385 test cases...

  *** No errors detected
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 137c80d579502e329964d7d1028a9507d4667774

Tree-SHA512: 9486a0d32b4cf686bf5a47a0778338ac571fa39c66ad6d6d6cede58ec798e87bb50a2f9b7fd79ecd1fef1ba284e4073c1b430110967073ff87bdbbde7cada447
2021-09-17 15:56:38 -04:00
MarcoFalke
6bb5dfe2d9 Merge #14585: refactor: remove usage of locale dependent std::isspace
15db77f4dd Don't rely on locale dependent functions in base_blob<BITS>::SetHex(...) (uint256), DecodeBase58(...), ParseMoney(...) and ParseHex(...) (practicalswift)

Pull request description:

  Don't rely on locale dependent function `std::isspace` in `base_blob<BITS>::SetHex(...)` (uint256), `DecodeBase58(...)`, `ParseMoney(...)` and `ParseHex(...)`.

  Rationale:

  ```
  $ uname -s
  Darwin
  $ cat poc.cpp
  #include <iostream>
  #include <locale>

  int main(void) {
      setlocale(LC_ALL, "");
      std::cout << std::isspace(133) << ' ' << std::isspace(154) << ' ' << std::isspace(160);
      std::cout << '\n';
  }
  $ clang++ -o poc poc.cpp
  $ ./poc
  1 0 1
  $ LC_ALL=en_US ./poc
  1 0 1
  $ LC_ALL=C ./poc
  0 0 0
  $ LC_ALL=ru_RU.KOI8-R ./poc # an "interesting" locale
  0 1 0
  ```

Tree-SHA512: 4eafb267342b8a777da6cca07c353afd1f90f3fc1d91e01f526f1b384a2b97c1da25b7bd7dfc300655182a4eaec6a4bea855a45723ab53c750a734b60e1e3c9f
2021-09-09 10:53:54 -07:00
MarcoFalke
b39ec37988 Merge #15139: util: Remove [U](BEGIN|END) macros
332b3dd7c1 util: Make ToLower and ToUpper take a char (Wladimir J. van der Laan)
edb5bb3500 util: remove unused [U](BEGIN|END) macros (Wladimir J. van der Laan)
7fa238c701 Replace use of BEGIN and END macros on uint256 (Wladimir J. van der Laan)

Pull request description:

  Two cleanups in `util/strencodings.h`:

  - Remove `[U](BEGIN|END)` macros — The only use of these was in the Merkle tree code with `uint256` which has its own `begin` and `end` methods which are better.
  - Make ToLower and ToUpper take a char — Unfortunately, `std::string` elements are (bare) chars. As these are the most likely type to be passed to these functions, make them use char instead of unsigned char. This avoids some casts.

Tree-SHA512: 96c8292e1b588d3d7fde95c2e98ad4e7eb75e7baab40a8e8e8209d4e8e7a1bd3b6846601d20976be34a9daabefc50cbc23f3b04200af17d0dfc857c4ec42aca7
2021-08-24 19:14:49 -04:00
Wladimir J. van der Laan
ff5a94748d
Merge #13249: Make objects in range declarations immutable by default. Avoid unnecessary copying of objects in range declarations.
f34c8c466a0e514edac2e8683127b4176ad5d321 Make objects in range declarations immutable by default. Avoid unnecessary copying of objects in range declarations. (practicalswift)

Pull request description:

  Make objects in range declarations immutable by default.

  Rationale:
  * Immutable objects are easier to reason about.
  * Prevents accidental or hard-to-notice change of value.

Tree-SHA512: cad69d35f0cf8a938b848e65dd537c621d96fe3369be306b65ef0cd1baf6cc0a9f28bc230e1e383d810c555a6743d08cb6b2b0bd51856d4611f537a12e5abb8b
2021-07-19 17:11:18 -05:00
UdjinM6
88da87d202
Merge pull request #4262 from PastaPastaPasta/backport-triv-pr17
Backport triv pr17
2021-07-16 20:30:46 +03:00
Kittywhiskers Van Gogh
4619630551 merge #13671: Remove boost/algorithm/string/case_conv.hpp 2021-07-16 20:19:42 +05:30
W. J. van der Laan
0d8f4fae3b Merge bitcoin/bitcoin#21173: util: faster HexStr => 13% faster blockToJSON
74bf850ac47735f2ef4306059d3e664d40cac85e faster HexStr => 13% faster blockToJSON (Martin Ankerl)

Pull request description:

  `std::string`'s push_back is rather slow because it needs to check & update the string size. For
  `HexStr` the output string size is already easily know, so we can initially create the string with
  the correct size and then just assign the data.

  `HexStr` is heavily usd in `blockToJSON`, so this change is a noticeable benefit. Benchmark on an i7-8700 @3.2GHz:

  * 71,315,461.00 ns/op master
  * 62,842,490.00 ns/op this commit

  So this little change makes `blockToJSON` about ~13% faster.

ACKs for top commit:
  laanwj:
    Code review ACK 74bf850ac47735f2ef4306059d3e664d40cac85e
  theStack:
    re-ACK 74bf850ac47735f2ef4306059d3e664d40cac85e

Tree-SHA512: fc99105123edc11f4e40ed77aea80cf7f32e49c53369aa364b38395dcb48575e15040b0489ed30d0fe857c032a04e225c33e9d95cdfa109a3cb5a6ec9a972415
2021-07-15 16:01:43 -05:00
Kittywhiskers Van Gogh
8b891c2b10 Merge #13723: PSBT key path cleanups 2021-07-13 22:00:18 +05:30
Wladimir J. van der Laan
87979a56bf Merge #14618: rpc: Make HTTP RPC debug logging more informative
ab8c6f24d28ea1d1e6258cf316b4b97a0baf2377 Add SAFE_CHARS[SAFE_CHARS_URI]: Chars allowed in URIs (RFC 3986) (practicalswift)
991248649b76a5a071e1360a700f3e2ecf3e1e1f rpc: Make HTTP RPC debug logging more informative (practicalswift)

Pull request description:

  * Make HTTP RPC debug logging more informative
  * Avoid excessively large log messages (which could theoretically fill up the disk) when running with debug option `-debug=http`

Tree-SHA512: 9068862fb7d34db1e12e6b9dde78b669b86c65b4fed3ea8c9eb6c35310d77fd12b16644728fd7e9fbf25059d25114bded9e061eb3de649d8847486ec42041ce9
2021-07-03 13:44:50 -05:00
Kittywhiskers Van Gogh
f2f1843f0c merge bitcoin#20499: Remove obsolete NODISCARD ifdef forest. Use [[nodiscard]] (C++17)
------------- BEGIN SCRIPT ---------------
gsed -i "s/NODISCARD/[[nodiscard]]/g" $(git grep -l "NODISCARD" ":(exclude)src/bench/nanobench.h" ":(exclude)src/attributes.h")
------------- END   SCRIPT ---------------
2021-06-27 12:03:14 +05:30
Kittywhiskers Van Gogh
ef4dfa8524 merge bitcoin#14555: Move util files to directory
(script modified to account for Dash backports, doesn't account for rebasing)

------------- BEGIN SCRIPT ---------------
mkdir -p src/util
git mv src/util.h src/util/system.h
git mv src/util.cpp src/util/system.cpp
git mv src/utilmemory.h src/util/memory.h
git mv src/utilmoneystr.h src/util/moneystr.h
git mv src/utilmoneystr.cpp src/util/moneystr.cpp
git mv src/utilstrencodings.h src/util/strencodings.h
git mv src/utilstrencodings.cpp src/util/strencodings.cpp
git mv src/utiltime.h src/util/time.h
git mv src/utiltime.cpp src/util/time.cpp
git mv src/utilasmap.h src/util/asmap.h
git mv src/utilasmap.cpp src/util/asmap.cpp
git mv src/utilstring.h src/util/string.h
git mv src/utilstring.cpp src/util/string.cpp

gsed -i 's/<util\.h>/<util\/system\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilmemory\.h>/<util\/memory\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilmoneystr\.h>/<util\/moneystr\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilstrencodings\.h>/<util\/strencodings\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utiltime\.h>/<util\/time\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilasmap\.h>/<util\/asmap\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
gsed -i 's/<utilstring\.h>/<util\/string\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')

gsed -i 's/BITCOIN_UTIL_H/BITCOIN_UTIL_SYSTEM_H/g' src/util/system.h
gsed -i 's/BITCOIN_UTILMEMORY_H/BITCOIN_UTIL_MEMORY_H/g' src/util/memory.h
gsed -i 's/BITCOIN_UTILMONEYSTR_H/BITCOIN_UTIL_MONEYSTR_H/g' src/util/moneystr.h
gsed -i 's/BITCOIN_UTILSTRENCODINGS_H/BITCOIN_UTIL_STRENCODINGS_H/g' src/util/strencodings.h
gsed -i 's/BITCOIN_UTILTIME_H/BITCOIN_UTIL_TIME_H/g' src/util/time.h
gsed -i 's/BITCOIN_UTILASMAP_H/BITCOIN_UTIL_ASMAP_H/g' src/util/asmap.h
gsed -i 's/BITCOIN_UTILSTRING_H/BITCOIN_UTIL_STRING_H/g' src/util/string.h

gsed -i 's/ util\.\(h\|cpp\)/ util\/system\.\1/g' src/Makefile.am
gsed -i 's/utilmemory\.\(h\|cpp\)/util\/memory\.\1/g' src/Makefile.am
gsed -i 's/utilmoneystr\.\(h\|cpp\)/util\/moneystr\.\1/g' src/Makefile.am
gsed -i 's/utilstrencodings\.\(h\|cpp\)/util\/strencodings\.\1/g' src/Makefile.am
gsed -i 's/utiltime\.\(h\|cpp\)/util\/time\.\1/g' src/Makefile.am
gsed -i 's/utilasmap\.\(h\|cpp\)/util\/asmap\.\1/g' src/Makefile.am
gsed -i 's/utilstring\.\(h\|cpp\)/util\/string\.\1/g' src/Makefile.am

gsed -i 's/-> util ->/-> util\/system ->/' test/lint/lint-circular-dependencies.sh
gsed -i 's/src\/util\.cpp/src\/util\/system\.cpp/g' test/lint/lint-format-strings.py test/lint/lint-locale-dependence.sh
gsed -i 's/src\/utilmoneystr\.cpp/src\/util\/moneystr\.cpp/g' test/lint/lint-locale-dependence.sh
gsed -i 's/src\/utilstrencodings\.\(h\|cpp\)/src\/util\/strencodings\.\1/g' test/lint/lint-locale-dependence.sh
------------- END   SCRIPT ---------------
2021-06-27 12:03:13 +05:30