Commit Graph

19726 Commits

Author SHA1 Message Date
fanquake
ab19552f31 Merge #18535: build: remove -Qunused-arguments workaround for clang + ccache
a029805f57fa9a4ab9867c0d1e865675d57537c7 build: remove -Qunused-arguments workaround for clang + ccache (fanquake)

Pull request description:

  This was added in 386efb7695 to address spammy Clang warnings when building with ccache.

  The issue was addressed in [ccache 3.2](https://bugzilla.samba.org/show_bug.cgi?id=8118), and from a look at most major distros, it's only Debian Jessie that has a version of ccache older than that ([3.1](https://packages.debian.org/jessie/ccache)).

  Therefore I think it's acceptable to drop this workaround, and re-enable warnings for unused driver arguments (when compiling using Clang and ccache).

ACKs for top commit:
  hebasto:
    ACK a029805f57fa9a4ab9867c0d1e865675d57537c7.
  vasild:
    utACK a029805f57fa9a4ab9867c0d1e865675d57537c7

Tree-SHA512: f887b9bd12f9c1c8d209943b86e8dafe33cfd1572912f2cafabe08ffe403973e48f0f7289280a8c6db9263c57aad43fbd4bb72f42db762eb090f3b1ef0538f43
2021-07-15 13:26:07 -05:00
Wladimir J. van der Laan
b566c464ed Merge #18443: lockedpool: avoid sensitive data in core files (FreeBSD)
f85203097f78d9daa1d35c4097a80beab31da2a4 lockedpool: avoid sensitive data in core files (FreeBSD) (Vasil Dimov)

Pull request description:

  This is a followup to
  23991ee53 / https://github.com/bitcoin/bitcoin/pull/15600
  to also use madvise(2) on FreeBSD to avoid sensitive data allocated
  with secure_allocator ending up in core files in addition to preventing
  it from going to the swap.

ACKs for top commit:
  sipa:
    ACK f85203097f78d9daa1d35c4097a80beab31da2a4 if someone verifies this works as intended on *BSD.
  laanwj:
    ACK f85203097f78d9daa1d35c4097a80beab31da2a4
  practicalswift:
    Code-review ACK f85203097f78d9daa1d35c4097a80beab31da2a4 assuming a reviewer with FreeBSD access verifies that the PR goal is achieved :)

Tree-SHA512: 2e6d4ab6a9fbe18732c8ba530eacc17f58128c97140758b80c905b5b838922a2bcaa5f9abc45ab69d5a1a2baa0cba322f006048b60a877228e089c7e64dadd2a
2021-07-15 13:26:07 -05:00
MarcoFalke
d54f195f3d Merge #18785: Prevent valgrind false positive in rest_blockhash_by_height
fcb72616253ed22e364bc312992d77efc1c4a3c1 Prevent valgrind false positive in rest_blockhash_by_height (Russell Yanofsky)

Pull request description:

  A bad interaction between valgrind and clang 6.0.0-1ubuntu2 with -O2 optimizations makes valgrind misleadingly imply C++ code is reading an uninitialized blockheight value in `rest_blockhash_by_height` just because that's what clang optimized code is doing. The C++ code looks like:

  ```c++
  int32_t blockheight;
  if (!ParseInt32(height_str, &blockheight) || blockheight < 0) {
  ```

  while the optimized code looks like:

  ```
  0x00000000000f97ab <+123>:   callq  0x4f8860 <ParseInt32(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int*)>
  0x00000000000f97b0 <+128>:   mov    0xc(%rsp),%ebx
  0x00000000000f97b4 <+132>:   test   %ebx,%ebx
  0x00000000000f97b6 <+134>:   js     0xf98aa <rest_blockhash_by_height(util::Ref const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+378>
  0x00000000000f97bc <+140>:   xor    $0x1,%al
  0x00000000000f97be <+142>:   jne    0xf98aa <rest_blockhash_by_height(util::Ref const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+378>
  ```

  During the rest_interface.py test:

  eef90c14ed/test/functional/interface_rest.py (L266)

  when `height_str` is empty, `ParseInt32` returns false and `blockheight` value is never assigned. The optimized code reads the uninitialized `blockheight` value in `0xc(%rsp)` before the checking the `ParseInt32` return value in `%al`, which is harmless, but triggers the following error from valgrind:

  ```
  ==30660== Thread 13 b-httpworker.2:
  ==30660== Conditional jump or move depends on uninitialised value(s)
  ==30660==    at 0x2017B6: rest_blockhash_by_height(util::Ref const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (rest.cpp:614)
  ==30660==    by 0x2041B9: operator() (rest.cpp:670)
  ==30660==    by 0x2041B9: std::_Function_handler<bool (HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), StartREST(util::Ref const&)::$_1>::_M_invoke(std::_Any_data const&, HTTPRequest*&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (std_function.h:301)
  ==30660==    by 0x3EC994: operator() (std_function.h:706)
  ==30660==    by 0x3EC994: HTTPWorkItem::operator()() (httpserver.cpp:55)
  ==30660==    by 0x3ED16D: WorkQueue<HTTPClosure>::Run() (httpserver.cpp:114)
  ==30660==    by 0x3E9168: HTTPWorkQueueRun(WorkQueue<HTTPClosure>*, int) (httpserver.cpp:342)
  ==30660==    by 0x3EDAAA: __invoke_impl<void, void (*)(WorkQueue<HTTPClosure> *, int), WorkQueue<HTTPClosure> *, int> (invoke.h:60)
  ==30660==    by 0x3EDAAA: __invoke<void (*)(WorkQueue<HTTPClosure> *, int), WorkQueue<HTTPClosure> *, int> (invoke.h:95)
  ==30660==    by 0x3EDAAA: _M_invoke<0, 1, 2> (thread:234)
  ==30660==    by 0x3EDAAA: operator() (thread:243)
  ==30660==    by 0x3EDAAA: std:🧵:_State_impl<std:🧵:_Invoker<std::tuple<void (*)(WorkQueue<HTTPClosure>*, int), WorkQueue<HTTPClosure>*, int> > >::_M_run() (thread:186)
  ==30660==    by 0x64256DE: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
  ==30660==    by 0x54876DA: start_thread (pthread_create.c:463)
  ==30660==    by 0x6DC888E: clone (clone.S:95)
  ==30660==  Uninitialised value was created by a stack allocation
  ==30660==    at 0x20173A: rest_blockhash_by_height(util::Ref const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (rest.cpp:608)
  ==30660==
  {
     <insert_a_suppression_name_here>
     Memcheck:Cond
     fun:_ZL24rest_blockhash_by_heightRKN4util3RefEP11HTTPRequestRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
     fun:operator()
     fun:_ZNSt17_Function_handlerIFbP11HTTPRequestRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEZ9StartRESTRKN4util3RefEE3$_1E9_M_invokeERKSt9_Any_dataOS1_S9_
     fun:operator()
     fun:_ZN12HTTPWorkItemclEv
     fun:_ZN9WorkQueueI11HTTPClosureE3RunEv
     fun:_ZL16HTTPWorkQueueRunP9WorkQueueI11HTTPClosureEi
     fun:__invoke_impl<void, void (*)(WorkQueue<HTTPClosure> *, int), WorkQueue<HTTPClosure> *, int>
     fun:__invoke<void (*)(WorkQueue<HTTPClosure> *, int), WorkQueue<HTTPClosure> *, int>
     fun:_M_invoke<0, 1, 2>
     fun:operator()
     fun:_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJPFvP9WorkQueueI11HTTPClosureEiES6_iEEEEE6_M_runEv
     obj:/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
     fun:start_thread
     fun:clone
  }
  ```

  This is a known bad interaction between clang and valgrind. The clang optimized code is correct but valgrind has no way of knowing that accessing the uninitialized value isn't a problem. Issue has been reported previously:

  - https://bugs.llvm.org/show_bug.cgi?id=32604#c4
  - https://github.com/Z3Prover/z3/issues/972

  This commit just sets blockheight to -1 as a workaround.

  This change was originally made in 41d5d651594c6c939add7a58b7e30c97dccdf24a from #18740 to fix the travis error there (https://travis-ci.org/github/bitcoin/bitcoin/jobs/678453061#L7157) but MarcoFalke suggested https://github.com/bitcoin/bitcoin/pull/18740#discussion_r414772851 moving to a new PR, since apparently the error's been seen on travis previously

ACKs for top commit:
  MarcoFalke:
    ACK fcb72616253ed22e364bc312992d77efc1c4a3c1
  practicalswift:
    ACK fcb72616253ed22e364bc312992d77efc1c4a3c1

Tree-SHA512: ec8abf45bd3d6c6e0e7e404d0b2a749efd43910619b84b0b5fe7dab22881598d1011a0f3ff2e146bf46320b63eb152bf63c62c06f1ab84c35dd640abc468f18f
2021-07-13 21:19:48 -05:00
Wladimir J. van der Laan
51fb668c3a Merge #18558: build: Fix boost detection for arch armv7l
da0842dcd44f8c9c9b167917fac0949b4978c3b0 build: Update ax_boost_mase.m4 to the latest serial (Hennadii Stepanov)

Pull request description:

  Picked from the upstream 90814f1895

  Fix #17010.

  This PR is [alternative](https://github.com/bitcoin/bitcoin/issues/17010#issuecomment-610651736) to #18501.

ACKs for top commit:
  laanwj:
    ACK da0842dcd44f8c9c9b167917fac0949b4978c3b0

Tree-SHA512: 5e43e12c524e4ea6b967c9be02c81a75948eac6cf55b819e3339222a2e3414731581d40af3524ad865abae7c5247c190448ebf2aa5e0d9a338edb501cc23ba38
2021-07-13 21:19:48 -05:00
MarcoFalke
c542cd3a98 Merge #18383: refactor: Check for overflow when calculating sum of tx outputs
f65c9ad40f2f5cdc581bdaf72e7dc68e9d7f7a80 Check for overflow when calculating sum of outputs (Elichai Turkel)

Pull request description:

  This was reported by practicalswift here #18046
  The exact order of the if, is important, we first do `!MoneyRange(tx_out.nValue)` to make sure the amount is non-negative. and then `std::numeric_limits<CAmount>::max() - tx_out.nValue < nValueOut` checks that the addition cannot overflow (if we won't check that the amount is positive this check can also overflow! (by doing something like `max - -max`))
  and only then we make sure that the some is also valid `!MoneyRange(nValueOut + tx_out.nValue)`
  if any of these conditions fail we throw.

  the overflowing logic:
  ```
  a + b > max // we want to fail if a+b is more than the maximum -> will overflow
  b > max - a
  max - a < b
  ```

  Closes: #18046

ACKs for top commit:
  MarcoFalke:
    ACK f65c9ad40f2f5cdc581bdaf72e7dc68e9d7f7a80, checked that clang with O2 produces identical binaries 💕
  practicalswift:
    ACK f65c9ad40f2f5cdc581bdaf72e7dc68e9d7f7a80
  instagibbs:
    utACK f65c9ad40f
  vasild:
    ACK f65c9ad40f2f5cdc581bdaf72e7dc68e9d7f7a80 modulo `s/assert.h/cassert/`

Tree-SHA512: 512d6cf4762f24c41cf9a38da486b17b19c634fa3f4efbdebfe6608779e96fc3014d5d2d29adb8001e113152c0217bbd5b3900ac4edc7b8abe77f82f36209e33
2021-07-13 21:19:48 -05:00
fanquake
4f632952bf Merge #18433: serialization: prevent int overflow for big Coin::nHeight
e980214bc4fd49530e8d50fe0a6657b8583bc6b5 serialization: prevent int overflow for big Coin::nHeight (pierrenn)

Pull request description:

  This is an attempt to fix fuzzer issues 1,2,8 reported by practicalswift here : https://github.com/bitcoin/bitcoin/issues/18046

  The fuzzer harness doesn't prevent deserialization of unrealistic high values for `Coin::nHeight`.  In the [provided examples](https://github.com/bitcoin/bitcoin/issues/18046), we have :
  - `blockundo_deserialize` : the varint `0x8DD88DD700` is deserialized as `3944983552` in `Coin::nHeight` (`TxInOutFormatter::Unser`)
  - `coins_deserialize` : the varint `0x8DD5D5EC40` is deserialized as `3939874496` similarly
  - `txundo_deserialize`: the varint `0x8DCD828F01` is deserialized as `3921725441` in `Coin::nHeight` (`Coin::Unserialize`)

  Since `Coin::nHeight` is 31 bit long, multiplying a large value by 2 triggers the fuzzer.

  AFAIK those values are unrealistic (~70k years for the smallest..). I've looked a bit a reducing the range of values the fuzzer can deserialize, but this seems to be too much code change for not much.

  Hence this PR chooses to static cast `nHeight` when re-serializing; it seems to be the less intrusive/safest way to prevent the fuzzer output.

  Another more "upstream" approach would be to limit `Coin::nHeight` values to something more realistic, e.g. `0xFFFFFFF` (~5k years) :

  de3a30bab2/src/undo.h (L39) and de3a30bab2/src/coins.h (L71)

  Thanks !

  NB: i was also not sure about the component/area to prefix the PR/commit with.. ?

ACKs for top commit:
  practicalswift:
    ACK e980214bc4fd49530e8d50fe0a6657b8583bc6b5 -- patch looks correct
  promag:
    ACK e980214bc4fd49530e8d50fe0a6657b8583bc6b5.
  sipa:
    utACK e980214bc4fd49530e8d50fe0a6657b8583bc6b5
  MarcoFalke:
    re-ACK e980214bc4fd49530e8d50fe0a6657b8583bc6b5 🎑
  ryanofsky:
    Code review ACK e980214bc4fd49530e8d50fe0a6657b8583bc6b5. Just removed ternary ? 1 : 0 and replaced / 2 with >> 1 since last review

Tree-SHA512: 905fc9e5e52a6857abee4a1c863751767835965804bb8c39474f27a120f65399ff4ba7a49ef1da0ba565379f8c12095bd384b6c492cf06776f01b2db68d522b8
2021-07-13 21:19:48 -05:00
MarcoFalke
286acf3926 Merge #18412: script: fix SCRIPT_ERR_SIG_PUSHONLY error string
41ff4992e57f8626019c0b2ab3d024db71e4c20f script: fix SCRIPT_ERR_SIG_PUSHONLY error string (Sebastian Falbesoner)

Pull request description:

  Fixes #18411, changing the error message from `"Only non-push operators allowed in signatures"` to `"Only push operators allowed in signatures"`.

ACKs for top commit:
  laanwj:
    ACK 41ff4992e57f8626019c0b2ab3d024db71e4c20f

Tree-SHA512: 3b75d83e2198d638d599ef6a4a8da986f0158600fe3f89f55b3759554588157acf2b0cba3f6a907164617264e7aee727d6d460b510c8b37ca7728aa79e11ad80
2021-07-13 21:19:48 -05:00
Wladimir J. van der Laan
8e8048e1c3 Merge #15600: lockedpool: When possible, use madvise to avoid including sensitive information in core dumps
d831831822885717e9841f1ff67c19add566fa45 lockedpool: When possible, use madvise to avoid including sensitive information in core dumps (Luke Dashjr)

Pull request description:

  If we're mlocking something, it's because it's sensitive information. Therefore, don't include it in core dump files, ~~and unmap it from forked processes~~.

  The return value is not checked because the madvise calls might fail on older kernels as a rule (unsure).

ACKs for top commit:
  practicalswift:
    Code review ACK d831831822885717e9841f1ff67c19add566fa45 -- patch looks correct
  laanwj:
    ACK d831831822885717e9841f1ff67c19add566fa45
  jonatack:
    ACK d831831822885717e9841f1ff67c19add566fa45
  vasild:
    ACK d831831822885717e9841f1ff67c19add566fa45

Tree-SHA512: 9a6c1fef126a4bbee0698bfed5a01233460fbcc86380d984e80dfbdfbed3744fef74527a8e3439ea226167992cff9d3ffa8f2d4dbd5ae96ebe0c12f3eee0eb9e
2021-07-13 21:19:48 -05:00
MarcoFalke
9156e07334 Merge #18350: test: Fix mining to an invalid target + ensure that a new block has the correct hash internally
7a6627ae87b637bf32c03122865402bd71adf0d1 Fix mining to an invalid target + ensure that a new block has the correct hash internally in Python tests (Samer Afach)

Pull request description:

  Test with block 47 in the `feature_block.py` creates a block with a hash higher than the target, which is supposed to fail. Now two issues exist there, and both have low probability of showing up:

  1. The creation is done with `while (hash < target)`, which is wrong, because hash = target is a valid mined value based on the code in the function `CheckProofOfWork()` that validates the mining target:
  ```
      if (UintToArith256(hash) > bnTarget)
          return false;
  ```
  2. As we know the hash stored in CBlock class in Python is stateful, unlike how it's in C++, where calling `CBlock::GetHash()` will actively calculate the hash and not cache it anywhere. With this, blocks that come out of the method `next_block` can have incorrect hash value when `solve=False`. This is because the `next_block` is mostly used with `solve=True`, and solving does call the function `rehash()` which calculates the hash of the block, but with `solve=False`, nothing calls that method. And since the work to be done in regtests is very low, the probably of this problem showing up is very low, but it practically happens (well, with much higher probability compared to issue No. 1 above).

  This PR fixes both these issues.

Top commit has no ACKs.

Tree-SHA512: f3b54d18f5073d6f1c26eab89bfec78620dda4ac1e4dde4f1d69543f1b85a7989d64c907e091db63f3f062408f5ed1e111018b842819ba1a5f8348c7b01ade96
2021-07-13 21:19:47 -05:00
fanquake
3b20481aec Merge #18357: Fix missing header in sync.h
686c5456f2fcf7e301907223d16a85f7eb378c6c Fix missing header in sync.h (João Barbosa)

Pull request description:

  `std::string` is referenced in `sync.h` but the relevant header is not explicitly included as required by current guideline. Furthermore on osx 10.14.6 with clang-900.0.31 the following error occurs:
  ```
  In file included from threadinterrupt.cpp:6:
  In file included from ./threadinterrupt.h:8:
  ./sync.h:206:21: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
          std::string lockname;
  ```

ACKs for top commit:
  practicalswift:
    ACK 686c5456f2fcf7e301907223d16a85f7eb378c6c
  laanwj:
    ACK 686c5456f2fcf7e301907223d16a85f7eb378c6c

Tree-SHA512: 7c1acdfa5b0dd148d1114e14c9450d5907006e63e1a04e82ed8a1e29757925476e6f8ef6024b0c6d1bb596623115209ad580d5035be1e4785337bd01b738c9f2
2021-07-13 21:19:47 -05:00
Wladimir J. van der Laan
2f4413304f Merge #18290: build: Set minimum Automake version to 1.13
ddc7e42d600a0cb3e763cda0dc04a1f2f34e9440 build: Set minimum Automake version to 1.13 (Hennadii Stepanov)

Pull request description:

  This PR suggests to set the required minimum Automake version to `1.13` explicitly for the following reasons:
  - it guarantees that [CVE-2012-3386](https://lists.gnu.org/archive/html/automake/2012-07/msg00023.html) has been fixed
  - `AC_CONFIG_MACRO_DIR` macro support, which we already use; from the [release notes](https://lists.gnu.org/archive/html/automake/2012-12/msg00038.html):
  > Improvements to aclocal and related rebuilds rules:
  >    - Autoconf-provided macros AC_CONFIG_MACRO_DIR and AC_CONFIG_MACRO_DIRS are now traced by aclocal, and can be used to declare the local m4 include directories.  Formerly, one had to specify it with an explicit '-I' option to the 'aclocal' invocation.
  - `AM_SILENT_RULES` macro support (since version `1.11`)

  Automake `1.13` requires Autoconf `2.65` or greater. We already have `2.69` since #17769.

  ---

  For reference, Automake `1.13` was released in [December of 2012](https://lists.gnu.org/archive/html/automake/2012-12/msg00038.html).

  CentOS 7 uses Automake [`1.13.4`](https://centos.pkgs.org/7/centos-x86_64/automake-1.13.4-3.el7.noarch.rpm.html)

  See the Automake docs for more info:
  - [`AM_INIT_AUTOMAKE`](https://www.gnu.org/software/automake/manual/automake.html#Public-Macros)
  - [List of Automake options](https://www.gnu.org/software/automake/manual/automake.html#List-of-Automake-options)

ACKs for top commit:
  laanwj:
    so also ACK ddc7e42d600a0cb3e763cda0dc04a1f2f34e9440
  fanquake:
    ACK ddc7e42d600a0cb3e763cda0dc04a1f2f34e9440 - I think adding a minimum required version here is fine. I'd be surprised if someone who is currently building Bitcoin Core was unable to after this change.

Tree-SHA512: a1f97864bc3a513450c03d041498f28e823e6f8cd9710d81df081435d72bd4b6cd2f3deb997dbf902f950215a859e48a2ee7ca1f8ebf4271778dd951ab78abf4
2021-07-13 21:19:43 -05:00
Wladimir J. van der Laan
2f6f313942 Merge #18285: test: Check that wait_until returns if time point is in the past
fab7d14ea5a4305317d66f35beb3225a07823d42 test: Check that wait_until returns if time point is in the past (MarcoFalke)

Pull request description:

  Add an explicit regression test for the condvar bug (#18227), so that this doesn't happen again

ACKs for top commit:
  laanwj:
    ACK fab7d14ea5a4305317d66f35beb3225a07823d42

Tree-SHA512: 6ec0d0b3945cae87a001e367af34cca1953a8082b4a0d9f8a20d30acd1f36363e98035d4eb173ff786cf6692d352d41f960633415c46394af042eb44e3b5ad71
2021-07-13 21:19:43 -05:00
Wladimir J. van der Laan
eaab6cd68d Merge #18168: httpserver: use own HTTP status codes
aff2748f8aee72f03b5fb6f6f97f0d0f66391755 httpserver: use own HTTP status codes (Filip Gospodinov)

Pull request description:

  Before, macros defined in `<event2/http.h>` have been used for some HTTP status codes. `<event2/http.h>` is included implicitly and the usage of its status code macros is inconsistent with the majority HTTP response implementations in this file.

  Now, the `HTTPStatusCode` enum from `<rpc/protocol.h>` is consistently used for all HTTP response implementations.

ACKs for top commit:
  practicalswift:
    ACK aff2748f8aee72f03b5fb6f6f97f0d0f66391755 -- patch looks correct
  laanwj:
    ACK aff2748f8aee72f03b5fb6f6f97f0d0f66391755

Tree-SHA512: 6a7043488b88dcd584215d16b5f16f7bd668fe5553d31298d1beef134d2b0648aef81533014e34d1cd600baa36ee4e853f195352f4a00c866bd5ab1ff688bd50
2021-07-13 21:19:43 -05:00
MarcoFalke
d74366e54b Merge #18183: test: Set catch_system_errors=no on boost unit tests
fac52dafa013047b051ca7163cc30ac69ad35531 test: Set catch_system_errors=no on boost unit tests (MarcoFalke)

Pull request description:

  Closes #16700

  Can be tested by adding an `assert(0)` and then running either `make check` or `./src/test/test_bitcoin -t bla_tests --catch_system_errors=no/yes`

ACKs for top commit:
  practicalswift:
    ACK fac52dafa013047b051ca7163cc30ac69ad35531
  Empact:
    Tested ACK fac52dafa0

Tree-SHA512: ec00636951b2c1137aaf43610739d78d16f823f7da76a726d47f93b8b089766fb66b21504b3c5413bcf8b6b5c3db0ad74027d677db24a44487d6d79a6bdee2e0
2021-07-13 21:19:43 -05:00
MarcoFalke
6a0627d073 Merge #18172: test: Transaction expiry from mempool
d6d2602a32251c1017da88b47c801b7283c66ce3 add: test that transactions expire from mempool (0xb10c)

Pull request description:

  This adds the functional test `mempool_expiry.py` covering mempool transaction expiry. Both the default `DEFAULT_MEMPOOL_EXPIRY` of 336 hours (two weeks, set in #9312) and the user definable mempool expiry via the `-mempoolexpiry=<n>` command line option are tested. The test checks that descendants of expired transactions are removed as well.

  *Notes for reviewers*
  - `LimitMempoolSize()` (which is the only caller of `CTxMemPool::Expire()`) is only called when a transaction is added to the mempool. In order to test expiry of a transaction-that-should-expire, the mocktime is set and a random transaction is broadcast to trigger `LimitMempoolSize()`. The transaction-that-should-expire is then checked for expiry. LMK if there is another way, but I don't think there is.

ACKs for top commit:
  MarcoFalke:
    ACK d6d2602a32251c1017da88b47c801b7283c66ce3
  theStack:
    ACK d6d2602a32
  promag:
    Code review ACK d6d2602a32251c1017da88b47c801b7283c66ce3.

Tree-SHA512: eb68cd9e2d870872b8e8e1522fed8954fb99cc9e4edda4b28bb2a4e41cddbc53fe6f7d9c090f1e0e98ab49beb24bf37ff3787a9e9801a95e8ae9ca9eb34fe6f0
2021-07-13 21:19:43 -05:00
UdjinM6
3004c3498b
Merge pull request #4245 from PastaPastaPasta/backport-triv-pr9
backport: 'trivial' pr9
2021-07-14 02:52:00 +03:00
PastaPastaPasta
e98241da5d
Merge pull request #4186 from kittywhiskers/psbt
merge #13269, #13425,  #13557,  #13721,  #13666, #13723: BIP 174 PSBT Serializations and RPCs
2021-07-13 13:54:34 -05:00
PastaPastaPasta
4f5d4be443
dashification 2021-07-13 13:14:09 -05:00
Wladimir J. van der Laan
9d1bbabbc8
Merge #17302: cli: Add "headers" and "verificationprogress" to -getinfo
31879345ee9a222b12014c8b2359e1737ff0d8c4 cli: Add "headers" and "verificationprogress" to -getinfo (Wladimir J. van der Laan)

Pull request description:

  These values are useful to know the current progress of initial sync, or of catching up, which is arguably the use of a quick `-getinfo` command.

ACKs for top commit:
  MarcoFalke:
    unsigned ACK 31879345ee9a222b12014c8b2359e1737ff0d8c4
  jonasschnelli:
    utACK 31879345ee9a222b12014c8b2359e1737ff0d8c4
  jonatack:
    Tested ACK 31879345ee9a222b12014c8b2359e1737ff0d8c4 on Debian 4.19.37-5+deb10u2 (2019-08-08) x86_64 GNU/Linux

Tree-SHA512: 185180ab426b4db5d99eb208ee88d1606f585361875ba3a92b6c28a74fe181d72ed710c8859b969ba49b1ca7d2385695932b79ff621c7a2a7cedd0df717a99ed
2021-07-13 13:14:09 -05:00
fanquake
9b8250c7c6
Merge #16986: doc: Doxygen-friendly CuckooCache comments
7aad3b68e7e1680870ca70d945eee88f790d6454 doc: Doxygen-friendly CuckooCache comments (Jon Layton)

Pull request description:

  Similar theme to #16947.

  - `invalid`, `contains` now appear in Doxygen docs
  - `setup` refers to correct argument name `b`
  - Argument references in `code blocks `
  - Lists markdown conformant, uniform line endings

  Tested with `make docs`

ACKs for top commit:
  laanwj:
    ACK 7aad3b68e7e1680870ca70d945eee88f790d6454
  practicalswift:
    ACK 7aad3b68e7e1680870ca70d945eee88f790d6454

Tree-SHA512: 70b38c10e534bad9c6ffcd88cc7a4797644afba5956d47a6c7cc655fcd5857a91f315d6da60e28ce9678d420ed4a51e22267eb8b89e26002b99cad63373dd349
2021-07-13 13:14:08 -05:00
Wladimir J. van der Laan
6a980d92d9
Merge #17226: gui: Fix payAmount tooltip in SendCoinsEntry
0fc81a1e8724547a226e5fae5c32fc32d8dfb733 gui: Fix payAmount tooltip in SendCoinsEntry (João Barbosa)

Pull request description:

  Before the tooltip shows in wrong places:

  ![Screenshot 2019-10-23 at 11 33 49](https://user-images.githubusercontent.com/3534524/67384904-f6b6a380-f589-11e9-832c-ec1643014b96.png)
  ![Screenshot 2019-10-23 at 11 33 23](https://user-images.githubusercontent.com/3534524/67384905-f74f3a00-f589-11e9-9944-a52fee097e02.png)

  Now only shows in the amount field:

  ![Screenshot 2019-10-23 at 11 35 30](https://user-images.githubusercontent.com/3534524/67384919-ff0ede80-f589-11e9-8ce4-c122e11fe885.png)

ACKs for top commit:
  laanwj:
    ACK 0fc81a1e8724547a226e5fae5c32fc32d8dfb733

Tree-SHA512: 0857e568c21d380a68c81e9be3212b1745d7d3199a1d5fdef9afc8feed0272f215726fa98bbf8a3fb332389c5454f2316bc1581f1a2ccd76cef46a0e3ac6f99f
2021-07-13 13:14:08 -05:00
MarcoFalke
432f329f83
Merge #17206: test: Add testcase to simulate bitcoin schema in leveldb
4896bacc00549c14f3284f5a2b61fb848ac31be0 Add testcase to simulate bitcoin schema in leveldb (MapleLaker)

Pull request description:

  Resurrecting #14125 with updates based on comments of closed PR

ACKs for top commit:
  laanwj:
    ACK 4896bacc00549c14f3284f5a2b61fb848ac31be0
  dongcarl:
    ACK 4896bacc00549c14f3284f5a2b61fb848ac31be0

Tree-SHA512: 3290ea7e1e998901d5ee8921d1d76cec399cae30ac1911a45b86826afed47cee1acf92bd6438f1fa11ed785a3b17abdcb1c169bc0419945eda9fe4c089d0b6eb
2021-07-13 13:14:08 -05:00
Wladimir J. van der Laan
44e79f0740
Merge #17180: gui: Improved tooltip for send amount field
088a730fe6f893d43b55ec52e6b5080fd445dd9e static tooltip (JeremyCrookshank)

Pull request description:

  I noticed that on Bitcoin sends the tooltip wasn't very clear for new users and I hope my PR is more concise. If it needs changing more will happily change too 👍
  ![IMG_20191017_192739](https://user-images.githubusercontent.com/46864828/67036925-75d45380-f114-11e9-88bf-bab58161f80a.jpg)

ACKs for top commit:
  laanwj:
    ACK 088a730fe6f893d43b55ec52e6b5080fd445dd9e

Tree-SHA512: 2b1103ac934d8f68d22333af3c0f5d4228b665b1e507378d4ae5b83cc2b6d6aeb46a3d68298cca93feb839db5caa560322c8df5261dc2f7db5abeed9f0dd9c69
2021-07-13 13:14:07 -05:00
fanquake
5857a5d5b4
Merge #17157: doc: Added instructions for how to add an upsteam to forked repo
f09ba060cacd42e4cb9a242c1d731deb1f6623c6 doc: Added instructions for how to add an upsteam to forked repo (dannmat)

Pull request description:

  As a first time git developer, I struggled to understand whether to create a new fork for each pull request or not.

  After asking the IRC chat, I have added this to the documentation to further help new developers using git.

ACKs for top commit:
  fanquake:
    ACK f09ba060cacd42e4cb9a242c1d731deb1f6623c6 - For such a simple change, I think we've bike-shed this enough already. The `bitcoin/bitcoin` repo isn't really where anyone should be learning how to use `git` etc, but I think linking out here is ok.

Tree-SHA512: e0e9d655d0725e0128673afedb81dc5ba9387968fcbb681de7e50155a2cfa1a7f39fad040b596f4de9ad6727a1a8a90fd3d36eaa5242bc12186c3b82abd23fb2
2021-07-13 13:14:07 -05:00
fanquake
062e61dc4a
Merge #17186: gui: Add placeholder text to the sign message field
7005d6ab8f4393545ec27f528d8c03fc4516ddab gui: Add placeholder text to the sign message field (Danny-Scott)

Pull request description:

  When using the sign message functionality I noticed the "message" field had no label or placeholder text to highlight what it's for.

  I've added the placeholder text to match the tool tip to help it be more user friendly.

ACKs for top commit:
  hebasto:
    Re-ACK 7005d6ab8f4393545ec27f528d8c03fc4516ddab
  fanquake:
    ACK 7005d6ab8f4393545ec27f528d8c03fc4516ddab

Tree-SHA512: 17fe51c134f6373d8d5f9ca98b15bd936da4e61aa5258ceb5d318575d49b43cbfde6f4c3f720eb5928206902e6ba52811ba08737a03c95224e45dabc947d9d11
2021-07-13 13:14:07 -05:00
fanquake
b485210d68
Merge #17162: chain: Remove CBlockIndex::SetNull helper
fa0467326ff004a0a20c6af9c8f6adcc74a0c8af chain: Set all CBlockIndex members to null, remove SetNull helper (MarcoFalke)

Pull request description:

  The first commit removes the `SetNull` helper and inlines the member initialization (C++11). See https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#c-data-structures for rationale.

  <strike>The second commit adds the `cs_main` lock annotation to `RaiseValidity`. See also #17161.</strike>

ACKs for top commit:
  promag:
    Code review ACK fa0467326ff004a0a20c6af9c8f6adcc74a0c8af.
  practicalswift:
    ACK fa0467326ff004a0a20c6af9c8f6adcc74a0c8af -- diff still looks correct :)
  laanwj:
    ACK fa0467326ff004a0a20c6af9c8f6adcc74a0c8af, this makes it easy to see that all fields are initialized.

Tree-SHA512: 1b2b9fb0951c03c75b9cce322b89d4ecc9a364ae78b94d91b0b4669437824394dfada820ab6f74dfac3193f602899abfdc244ae2d9351ad293f555488f03470e
2021-07-13 13:14:06 -05:00
Wladimir J. van der Laan
bd18cd8e07
Merge #17131: rpc: fix -rpcclienttimeout 0 option
b3b26e149c34fee9c7ae8548c6e547ec6254b441 rpc: fix -rpcclienttimeout 0 option (Fabian Jahr)

Pull request description:

  fixes #17117

  I understood the bug as the help string being wrong, rather than that this feature is missing and should be added. Let me know if it should be the other way around.

  It is notable that if 0 is given as an argument, the fallback that is being used is the libevent default of 50 seconds, rather than `DEFAULT_HTTP_CLIENT_TIMEOUT` (900 seconds). This is not intuitive for the user. I could handle this in this PR but I am unsure which would be the better solution then: Actually adding the feature as described in the help string or falling back to `DEFAULT_HTTP_CLIENT_TIMEOUT`? Happy to hear opinions.

ACKs for top commit:
  MarcoFalke:
    unsigned ACK b3b26e149c34fee9c7ae8548c6e547ec6254b441

Tree-SHA512: 65e526a652c0adcdb4f895e8d78d60c7caa5904c9915b165a3ae95725c87d13af1f916359f80302452a2fcac1a80f4c58cd805ec8c28720fa4b91b3c8baa4155
2021-07-13 13:14:06 -05:00
fanquake
4c5e542c46
Merge #17125: gui: Add toolTip and placeholderText to sign message fields
610d9384de7f4de861d94c9b1af4fddc8aa57ad9 gui: Added label & tooltip for Verify Message labels (dannmat)

Pull request description:

  When using the Verify Message functionality, I found the input boxes to be rather confusing as they had no guidance for their purpose.

  I have added tooltips and labels to aid users when verifying messages in future

ACKs for top commit:
  promag:
    Code review ACK 610d9384de7f4de861d94c9b1af4fddc8aa57ad9. Nit, commit and title are a little weird. Suggestion: "gui: Add toolTip and placeholderText to sign message fields"
  MarcoFalke:
    ACK 610d9384de7f4de861d94c9b1af4fddc8aa57ad9 (looks good, didn't compile or tested the changes)
  fanquake:
    ACK - 610d9384de7f4de861d94c9b1af4fddc8aa57ad9

Tree-SHA512: d6a1bc872ad270dce440e96a163ce72cdd4708913d87a0fea749fc8cf2d8163b791cbb96a82030e0cb7d239920ceb0e3f05e0eec113f45a1a8e1309fbd92b4b0
2021-07-13 13:14:06 -05:00
Wladimir J. van der Laan
a5d5bc5ecf
Merge #17033: Disable _FORTIFY_SOURCE when enable-debug
44f7a8d7a774f82417106c452d793e6f091bc23e Disable _FORTIFY_SOURCE when enable-debug (Andrew Chow)

Pull request description:

  The `_FORTIFY_SOURCE` macro is enabled by default when hardening is enabled, but it requires optimization in order to be used. Since we disable all optimization with `--enable-debug`, this macro doesn't actually do anything and instead just causes a lot of warnings to be printed. This PR explicitly disables `_FORTIFY_SOURCE` so that these useless warnings aren't printed.

ACKs for top commit:
  laanwj:
    Thanks. ACK 44f7a8d7a774f82417106c452d793e6f091bc23e

Tree-SHA512: e9302aef794dfd9ca9d0d032179ecc51d3212a9a0204454419f410011343b27c32e6be05f385051b5b594c607b91b8e0e588f644584d6684429a649a413077d9
2021-07-13 13:14:05 -05:00
Wladimir J. van der Laan
e4457ce2ef
Merge #17085: init: Change fallback locale to C.UTF-8
facb9a1315f97489a20eb0e969fdb14b5128ed2f init: Change fallback locale to C.UTF-8 (Wladimir J. van der Laan)

Pull request description:

  Much of our code assumes file system UTF-8 support, and this is a more realistic guess for modern systems anyway than the default character set (which would be ASCII only). So change the assumed fallback locale (if no locale is defined by the user or OS) to `C.UTF-8`.

  related: https://github.com/bitcoin/bitcoin/issues/14948#issuecomment-488385462

ACKs for top commit:
  MarcoFalke:
    ACK facb9a1315f97489a20eb0e969fdb14b5128ed2f

Tree-SHA512: 5075f9fe6791572d76ec38c58cd56f04ed8086c06a7d7f446d062dffc313c62466ba81f1a7d6b8c7e95791fcff82e4f76871c3534478fbfe5beb456dd8eea340
2021-07-13 13:14:05 -05:00
fanquake
c79a03ec93
Merge #17087: build: Add variable printing target to Makefiles
181989f6c9427fc266dbdcc84cb60ac03e67cdb2 build: Add variable printing target to Makefiles (Carl Dong)

Pull request description:

  ```
  I kept finding myself needing these to debug our build system, since
  they are innocuous and are very helpful they probably belong in the
  codebase.

  Source: John Graham-Cumming
  https://www.cmcrossroads.com/article/printing-value-makefile-variable
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 181989f6c9427fc266dbdcc84cb60ac03e67cdb2
  fanquake:
    ACK 181989f6c9427fc266dbdcc84cb60ac03e67cdb2 - concise amount of useful code. Tested on macOS. Did not visit the link.

Tree-SHA512: 2139621e68a499c7347663ca9dc04e166ea6280e05986c27858df0156016ef2f9461262464d70c601419384f43a4ae3bcc67dfc0a05dbeef64f08386ab429cd8
2021-07-13 13:14:05 -05:00
Wladimir J. van der Laan
22111c88f6
Merge #17066: build: Remove workaround for ancient libtool
30fc1a3f5470cb347eb4aed281242f120510466f build: Remove workaround for ancient libtool (Hennadii Stepanov)
6ca01b9a104ebadbe7e180cb2b9d390f7c09b4ab build: Ensure a minimal version of libtool (Hennadii Stepanov)

Pull request description:

  Since libtool 1.5.2, on Linux libtool no longer sets RPATH for any directories in the dynamic linker search path, so there is no longer an issue.

  This commit reverts a98356fee8.

  Refs:
  - https://wiki.debian.org/RpathIssue
  - [Debian jessie has libtool 2.4.2](https://packages.debian.org/jessie/libtool)

ACKs for top commit:
  laanwj:
    ACK 30fc1a3f5470cb347eb4aed281242f120510466f

Tree-SHA512: fab56265d4d2c96216a353cc076c6f510e15748d8134f97bae2f67b6d8c0b6a1a9f362d2ab23b19ccc3a8bba8eac3bb1668fc3e42037590f63a7ab4819c9ee15
2021-07-13 13:14:04 -05:00
Wladimir J. van der Laan
3d8cb7b430
Merge #17030: test: Fix Python Docstring to include all Args.
8acd58927a614e006641384f61f8e7fca1cc68fc Fix Python Docstring to include all Args. (John Bampton)

Pull request description:

  Found a Python function that had incorrect and missing arguments in its Docstring.

ACKs for top commit:
  laanwj:
    ACK 8acd58927a614e006641384f61f8e7fca1cc68fc

Tree-SHA512: 936f275f29a700d630bb479b5283e47b66f2df76d8b8c053f594e6aedf783cc98a29c924c3a46613f112dfc884acb50f21a0b18f96d939e887b12b921ef2e10f
2021-07-13 13:14:04 -05:00
fanquake
d96e363a3f
Merge #17031: gui: Prevent processing duplicate payment requests
3f89e1eb237efcbd6415ca2cd0acddb6596153d7 Prevent processing duplicate payment requests (João Barbosa)

Pull request description:

  Considering the following from Qt [src/plugins/platforms/cocoa/qcocoaapplicationdelegate.mm#L267](13e0a36626/src/plugins/platforms/cocoa/qcocoaapplicationdelegate.mm (L267))

  ```cpp
  - (void)application:(NSApplication *)sender openFiles:(NSArray *)filenames
  {
      Q_UNUSED(filenames);
      Q_UNUSED(sender);

      for (NSString *fileName in filenames) {
          QString qtFileName = QString::fromNSString(fileName);
          if (inLaunch) {
              // We need to be careful because Cocoa will be nice enough to take
              // command line arguments and send them to us as events. Given the history
              // of Qt Applications, this will result in behavior people don't want, as
              // they might be doing the opening themselves with the command line parsing.
              if (qApp->arguments().contains(qtFileName))
                  continue;
          }
          QWindowSystemInterface::handleFileOpenEvent(qtFileName);
      }
  ```

  And that a2714a5c69f0b0506689af04c3e785f71ee0915d was merged, now Qt isn't able to filter out the above notifications, and then a [QFileOpenEvent](https://doc.qt.io/qt-5/qfileopenevent.html) event is delivered to `PaymentServer::eventFilter`, which in turn (re)adds the payment request.

  This change fixes #17025, but makes sense regardless of the issue.

ACKs for top commit:
  laanwj:
    Nah, this seems fine, utACK 3f89e1eb237efcbd6415ca2cd0acddb6596153d7
  Sjors:
    ACK 3f89e1e on macOS 10.14.6
  achow101:
    Code review ACK 3f89e1eb237efcbd6415ca2cd0acddb6596153d7

Tree-SHA512: dd1e0c73fd84953418173ca71f6f5a67ad74a5dc7e3b1d54915ef0545f513df6a24f27242a77bb094e2833a478e2f3bf30ecd50251f3c55b65e780097cb8ab4d
2021-07-13 13:14:04 -05:00
MarcoFalke
f46f16772d
Merge #16964: gui: Change sendcoins dialogue Yes to Send
a649cc6a17b8d8d602c6b67037b0c926960f9cdb Change sendcoins dialogue Yes to Send (Gregory Sanders)

Pull request description:

  It's more self-explanatory, matches "cancel" better, and makes future extensions such as https://github.com/bitcoin/bitcoin/pull/16944 more directly understandable to the user.

ACKs for top commit:
  Sjors:
    Trivial code review ACK a649cc6. I also used Send in #16966 (`ui - make send a wizard`)
  laanwj:
    ACK a649cc6a17b8d8d602c6b67037b0c926960f9cdb
  jonatack:
    Code review ACK a649cc6a17b8d8d602c6b67037b0c926960f9cdb

Tree-SHA512: fe4993bc7ac653d28f3d399ade046bcfd405511aec06ff041bb5aef47e0736faf3e3112a6db660cd761af56392dc6b97f2c2341ed3eff4490079c5eb8a0d465a
2021-07-13 13:14:03 -05:00
Wladimir J. van der Laan
ecf642bb8a
Merge #16952: gui: make sure to update the UI when deleting a transaction
addaf8af8268d918973883a304025d40af5a33c1 make sure to update the UI when deleting a transaction (Jonas Schnelli)

Pull request description:

  `CWallet::ZapSelectTx` removes transactions from the internal model, but leaves the UI in the dark.
  Adding a `NotifyTransactionChanged()` should avoid having invalid transactions in the GUI.

  Fixes #16950

ACKs for top commit:
  fanquake:
    ACK addaf8af8268d918973883a304025d40af5a33c1 - tested that this fixes #16950
  Sjors:
    tACK addaf8a: tested with an unpruned wallet by calling `removeprunedfunds` on an RBF-replaced transaction. It neatly disappears from the UI.
  kristapsk:
    ACK addaf8af8268d918973883a304025d40af5a33c1 (tested both with and without this change)

Tree-SHA512: 65e8c690847f7499e82c9fef67b60d9aaa63c853732fe7fa7281da33054fcdcd9d24f5b86de71b0827728c25bac8efb7db445863f990304ebfee6fc450620c47
2021-07-13 13:14:03 -05:00
Wladimir J. van der Laan
732b4bcc63
Merge #16931: test: add unittests for CheckProofOfWork
0cc7dd74e0af735dddf7e786f4ed136c382a4ad5 test: add unittests for CheckProofOfWork (soroosh-sdi)

Pull request description:

  following situations are covered:
  - negative target
  - overflow target
  - target easier then powLimit
  - invalid hash (hash > target)

  Signed-off-by: soroosh-sdi <soroosh.sardari@gmail.com>

ACKs for top commit:
  promag:
    ACK 0cc7dd74e0af735dddf7e786f4ed136c382a4ad5, just read the code.
  laanwj:
    ACK 0cc7dd74e0af735dddf7e786f4ed136c382a4ad5

Tree-SHA512: 9f9ee952ebb211202939450aa3d61b3c2fae992dcfcab085e877507d78e02ea39a51ccacfc4852a0555f3cba07504ee132abd5cbfed75489553bee45c760bc7e
2021-07-13 13:14:03 -05:00
Wladimir J. van der Laan
d181c0126e
Merge #16971: qt: Change default size of intro frame
8cf9898b53d74bd474cc5188e07e671e24a3791f qt: Change default size of intro frame (Emil Engler)

Pull request description:

  Because of the new pruning feature in the intro frame, the size of the intro frame is too small.
  Like you see, some text is not visible completely.

  ### Before
  ![Before](https://i.imgur.com/ppZ3Gf9.png)
  ### After
  ![After](https://i.imgur.com/wcElqLA.png)

  Update: I changed it so it adjusts the size dynamically

ACKs for top commit:
  fanquake:
    ACK 8cf9898b53d74bd474cc5188e07e671e24a3791f - Before and after macOS screens below. Given that most users will only ever see this screen once, I think Qts best effort to dynamically size it is fine.
  jonasschnelli:
    utACK 8cf9898b53d74bd474cc5188e07e671e24a3791f
  Sjors:
    Tested ACK 8cf9898 on macOS. English already fit, so to reproduce the issue, launch in German with `-resetguisettings -lang=de`.
  laanwj:
    ACK 8cf9898b53d74bd474cc5188e07e671e24a3791f

Tree-SHA512: 568b0ae0d5feeda603c0ccf67b5bb3857becea8f22fb98695e1901e662cb1e76377589e39ec743258154d7f6c4a5e544bb003fcc73597400dd427db047392638
2021-07-13 13:14:02 -05:00
Wladimir J. van der Laan
68810e8e57
Merge #16962: doc: Put PR template in comments
203a67d21f566634165531a7a75c3f8c9f9c9d6a doc: Put PR template in comments (Wladimir J. van der Laan)

Pull request description:

  This prevents the common annoyance of the text being included into PRs
  accidentally.

ACKs for top commit:
  sdaftuar:
    utACK 203a67d21f566634165531a7a75c3f8c9f9c9d6a
  fanquake:
    ACK 203a67d21f566634165531a7a75c3f8c9f9c9d6a - I make an effort to remove it whenever I see it in a PR.

Tree-SHA512: 3514d285488b7930d7f3d7f8823198d7325d8b7de57a6d8f13e559c0c23b30d58916b15782cbbdc347a375b418e9d0f7a5b99b34d26f3b957d7d5a03a3d83dfd
2021-07-13 13:14:02 -05:00
Wladimir J. van der Laan
bca013ddd6
Merge #16957: 9% less memory: make SaltedOutpointHasher noexcept
67d99900b0d770038c9c5708553143137b124a6c make SaltedOutpointHasher noexcept (Martin Ankerl)

Pull request description:

  If the hash is not `noexcept`, `unorderd_map` has to assume that it can throw an exception. Thus when rehashing care needs to be taken. libstdc++ solves this by simply caching the hash value, which increases memory of each node by 8 bytes. Adding `noexcept` prevents this caching. In my experiments with `-reindex-chainstate -stopatheight=594000`, memory usage (maximum resident set size) has decreased by 9.4% while runtime has increased by 1.6% due to additional hashing. Additionally, memusage::DynamicUsage() is now more accurate and does not underestimate.

  |                                       | runtime h:mm:ss | max RSS kbyte |
  |---------------------------------------|-----------------|--------------|
  | master                                |         4:13:59 |      7696728 |
  | 2019-09-SaltedOutpointHasher-noexcept |         4:18:11 |      6971412 |
  | change                                |          +1.65% |       -9,42% |

  Comparison of progress masters vs. 2019-09-SaltedOutpointHasher-noexcept
  ![out](https://user-images.githubusercontent.com/14386/65541887-69424e00-df0e-11e9-8644-b3a068ed8c3f.png)

ACKs for top commit:
  jamesob:
    Tested ACK 67d99900b0

Tree-SHA512: 9c44e3cca993b5a564dd61ebd2926b9c4a238609ea4d283514c018236f977d935e35a384dd4696486fd3d78781dd2ba190bb72596e20a5e931042fa465872a0b
2021-07-13 13:03:30 -05:00
Wladimir J. van der Laan
a597c1bb8e
Merge #16577: util: CBufferedFile fixes and unit test
efd2474d17098c754367b844ec646ebececc7c74 util: CBufferedFile fixes (Larry Ruane)

Pull request description:

  The `CBufferedFile` object guarantees its user is able to "rewind" the data stream (that's being read from a file) up to a certain number of bytes, as specified by the user in the constructor. This guarantee is not honored due to a bug in the `SetPos` method.

  Such rewinding is done in `LoadExternalBlockFile()` (currently the only user of this object), which deserializes a series of `CBlock` objects. If that function encounters something unexpected in the data stream, which is coming from a `blocks/blk00???.dat` file, it "rewinds" to an earlier position in the stream to try to get in sync again. The `CBufferedFile` object does not actually rewind its file offset; it simply repositions its internal offset, `nReadPos`, to an earlier position within the object's private buffer; this is why there's a limit to how far the user may rewind.

  If `LoadExternalBlockFile()` needs to rewind (call `blkdat.SetPos()`), the stream may not be positioned as it should be, causing errors in deserialization. This need to rewind is probably rare, which is likely why this bug hasn't been noticed already. But if this object is used elsewhere in the future, this could be a serious problem, especially as, due to the nature of the bug, the `SetPos()` _sometimes_ works.

  This PR adds a unit test for `CBufferedFile` that fails due to this bug. (Until now it has had no unit tests.) The unit test provides good documentation and examples for developers trying to understand `LoadExternalBlockFile()` and for future users of this object.

  This PR also adds code to throw an exception from the constructor if the rewind argument is not less than the buffer size (since that doesn't make any sense).

  Finally, I discovered that the object is too restrictive in one respect: When the deserialization methods call this object's `read` method, a check ensures that the number of bytes being requested is less than the size of the buffer (adjusting for the rewind size), else it throws an exception. This restriction is unnecessary; the object being deserialized can be larger than the buffer because multiple reads from disk can satisfy the request.

ACKs for top commit:
  laanwj:
    ACK ~after squash.~ efd2474d17098c754367b844ec646ebececc7c74
  mzumsande:
    I had intended to follow up earlier on my last comment, ACK efd2474d17098c754367b844ec646ebececc7c74. I reviewed the code, ran tests and did a successful reindex on testnet with this branch.

Tree-SHA512: 695529e0af38bae2af4e0cc2895dda56a71b9059c3de04d32e09c0165a50f6aacee499f2042156ab5eaa6f0349bab6bcca4ef9f6f9ded4e60d4483beab7e4554
2021-07-13 13:03:30 -05:00
UdjinM6
90e990e1ac
Merge pull request #4244 from PastaPastaPasta/backport-triv-pr8
backport: 'trivial' pr8
2021-07-13 20:53:57 +03:00
UdjinM6
6aea99094b
Merge pull request #4243 from Munkybooty/backports-0.18-pr7.1
Merge bitcoin#13877: utils: Make fs::path::string() always return utf-8 string on Windows
2021-07-13 20:53:30 +03:00
PastaPastaPasta
48487122ad
Merge pull request #4239 from Munkybooty/backports-0.18-pr7
Backports 0.18 pr7
2021-07-13 11:51:50 -05:00
Kittywhiskers Van Gogh
1a887a0067 tests: adapt PSBT RPC tests for Dash testnet 2021-07-13 22:00:18 +05:30
UdjinM6
fa5956a64d tests: Swap some valid/invalid test cases
Can't spend from segwit but at the same time we do not care about errors in segwit keys (they are unknown to us).
2021-07-13 22:00:18 +05:30
Kittywhiskers Van Gogh
622c7034da trivial: replace references of bitcoin with dash 2021-07-13 22:00:18 +05:30
Kittywhiskers Van Gogh
1e15a6116d core: remove all leftover references to segwit/rbf 2021-07-13 22:00:18 +05:30
Kittywhiskers Van Gogh
8b891c2b10 Merge #13723: PSBT key path cleanups 2021-07-13 22:00:18 +05:30
Kittywhiskers Van Gogh
472cfcbf7f Merge #13666: Always create signatures with Low R values 2021-07-13 22:00:18 +05:30