Merge #6347: backport: trivial 2024 10 23 pr3

c75a0d4c57 Merge bitcoin/bitcoin#29177: build: Fix check whether `-latomic` needed (fanquake)
f670118cce Merge bitcoin/bitcoin#28851: build: Patch Qt to handle minimum macOS version properly (fanquake)
685ee8a46f Merge bitcoin/bitcoin#28884: doc: remove x86_64 build assumption from depends doc (fanquake)
47f6126504 Merge bitcoin/bitcoin#28881: doc: remove mention of missing bdb being a configure error (fanquake)
a9021db4ec Merge bitcoin/bitcoin#28777: doc: update docs for `CHECK_ATOMIC` macro (fanquake)
d5e15dfc5a Merge bitcoin/bitcoin#26839: Add support for RNDR/RNDRRS for AArch64 on Linux (Andrew Chow)
5aedcbfb43 Merge bitcoin/bitcoin#28778: depends: drop -O1 workaround from arm64 apple Qt build (fanquake)
95a8d8cfdc Merge bitcoin/bitcoin#21161: Fee estimation: extend bucket ranges consistently (glozow)
f4ea48e623 Merge bitcoin/bitcoin#28693: build: Include `config/bitcoin-config.h` explicitly in `util/trace.h` (fanquake)
f160e0dbb2 Merge bitcoin/bitcoin#28691: refactor: Remove CBlockFileInfo::SetNull (fanquake)
0278163aa3 Merge bitcoin/bitcoin#28697: fuzz: Increase merge -rss_limit_mb (fanquake)
90a1fb0e8d Merge bitcoin/bitcoin#28650: fuzz: Merge with -set_cover_merge=1 (fanquake)
f007abd19d Merge bitcoin/bitcoin#28459: build: add `-mbranch-protection=bti` (aarch64) to hardening flags (fanquake)
af8d12445a Merge bitcoin/bitcoin#28624: docs: fix typo (fanquake)
c740264da8 Merge bitcoin/bitcoin#28532: qt: enable` -ltcg` for windows under LTO (fanquake)
ccd3920d40 Merge bitcoin/bitcoin#28556: doc: fix link to developer-notes.md file in multiprocess.md (fanquake)

Pull request description:

  ## Issue being fixed or feature implemented
  Batch of trivial backports

  ## What was done?
  See commits

  ## How Has This Been Tested?
  built locally; large combined merge passed tests locally

  ## Breaking Changes
  Should be none

  ## Checklist:
    _Go over all the following points, and put an `x` in all the boxes that apply._
  - [ ] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  UdjinM6:
    utACK c75a0d4c57

Tree-SHA512: 035dc3fa9812c7f381946ae4798b8e729a58b38a090d94502a8d992e9cfaab3307173c602d7b782c637a79c5c41b62570dc73bb4bb367e4505a039964926181b

build-aux/m4/l_atomic.m4

@@ -4,8 +4,10 @@ dnl permitted in any medium without royalty provided the copyright notice
 dnl and this notice are preserved. This file is offered as-is, without any
 dnl warranty.
 
-# Some versions of gcc/libstdc++ require linking with -latomic if
-# using the C++ atomic library.
+# Clang, when building for 32-bit,
+# and linking against libstdc++, requires linking with
+# -latomic if using the C++ atomic library.
+# Can be tested with: clang++ test.cpp -m32
 #
 # Sourced from http://bugs.debian.org/797228
 
@@ -22,6 +24,8 @@ m4_define([_CHECK_ATOMIC_testbody], [[
 
     std::atomic<std::chrono::seconds> t{0s};
     t.store(2s);
+    auto t1 = t.load();
+    t.compare_exchange_strong(t1, 3s);
 
     std::atomic<int64_t> a{};
 

configure.ac

@@ -981,6 +981,11 @@ if test x$use_hardening != xno; then
       ;;
   esac
 
+  case $host in
+    *aarch64*)
+      AX_CHECK_COMPILE_FLAG([-mbranch-protection=bti], [HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -mbranch-protection=bti"])
+    ;;
+  esac
 
   dnl When enable_debug is yes, all optimizations are disabled.
   dnl However, FORTIFY_SOURCE requires that there is some level of optimization, otherwise it does nothing and just creates a compiler warning.

contrib/devtools/README.md

@@ -83,7 +83,7 @@ A small script to automatically create manpages in ../../doc/man by running the
 This requires help2man which can be found at: https://www.gnu.org/software/help2man/
 
 With in-tree builds this tool can be run from any directory within the
-repostitory. To use this tool with out-of-tree builds set `BUILDDIR`. For
+repository. To use this tool with out-of-tree builds set `BUILDDIR`. For
 example:
 
 ```bash

depends/description.md

@@ -6,8 +6,7 @@ There are several features that make it different from most similar systems:
 In theory, binaries for any target OS/architecture can be created, from a
 builder running any OS/architecture. In practice, build-side tools must be
 specified when the defaults don't fit, and packages must be amended to work
-on new hosts. For now, a build architecture of x86_64 is assumed, either on
-Linux or macOS.
+on new hosts.
 
 ### No reliance on timestamps
 

depends/packages/qt.mk

@@ -23,6 +23,8 @@ $(package)_patches += guix_cross_lib_path.patch
 $(package)_patches += fast_fixed_dtoa_no_optimize.patch
 $(package)_patches += fix-macos-linker.patch
 $(package)_patches += memory_resource.patch
+$(package)_patches += windows_lto.patch
+$(package)_patches += fix-minimum-macos.patch
 
 $(package)_qttranslations_file_name=qttranslations-$($(package)_suffix)
 $(package)_qttranslations_sha256_hash=a31785948c640b7c66d9fe2db4993728ca07f64e41c560b3625ad191b276ff20
@@ -132,9 +134,6 @@ $(package)_config_opts_darwin += -no-feature-corewlan
 $(package)_config_opts_darwin += -no-freetype
 $(package)_config_opts_darwin += QMAKE_MACOSX_DEPLOYMENT_TARGET=$(OSX_MIN_VERSION)
 
-# Optimizing using > -O1 causes non-determinism when building across arches.
-$(package)_config_opts_aarch64_darwin += "QMAKE_CFLAGS_OPTIMIZE_FULL = -O1"
-
 ifneq ($(build_os),darwin)
 $(package)_config_opts_darwin += -xplatform macx-clang-linux
 $(package)_config_opts_darwin += -device-option MAC_SDK_PATH=$(OSX_SDK)
@@ -184,6 +183,9 @@ $(package)_config_opts_mingw32 += "QMAKE_LFLAGS = '$($(package)_ldflags)'"
 $(package)_config_opts_mingw32 += "QMAKE_LIB = '$($(package)_ar) rc'"
 $(package)_config_opts_mingw32 += -device-option CROSS_COMPILE="$(host)-"
 $(package)_config_opts_mingw32 += -pch
+ifneq ($(LTO),)
+$(package)_config_opts_mingw32 += -ltcg
+endif
 
 $(package)_config_opts_android = -xplatform android-clang
 $(package)_config_opts_android += -android-sdk $(ANDROID_SDK)
@@ -247,6 +249,7 @@ endef
 define $(package)_preprocess_cmds
   cp $($(package)_patch_dir)/qt.pro qt.pro && \
   cp $($(package)_patch_dir)/qttools_src.pro qttools/src/src.pro && \
+  patch -p1 -i $($(package)_patch_dir)/fix-minimum-macos.patch && \
   patch -p1 -i $($(package)_patch_dir)/fix-macos-linker.patch && \
   patch -p1 -i $($(package)_patch_dir)/dont_hardcode_pwd.patch && \
   patch -p1 -i $($(package)_patch_dir)/fix_qt_pkgconfig.patch && \
@@ -260,6 +263,7 @@ define $(package)_preprocess_cmds
   patch -p1 -i $($(package)_patch_dir)/qtbase-moc-ignore-gcc-macro.patch && \
   patch -p1 -i $($(package)_patch_dir)/fast_fixed_dtoa_no_optimize.patch && \
   patch -p1 -i $($(package)_patch_dir)/guix_cross_lib_path.patch && \
+  patch -p1 -i $($(package)_patch_dir)/windows_lto.patch && \
   mkdir -p qtbase/mkspecs/macx-clang-linux &&\
   cp -f qtbase/mkspecs/macx-clang/qplatformdefs.h qtbase/mkspecs/macx-clang-linux/ &&\
   cp -f $($(package)_patch_dir)/mac-qmake.conf qtbase/mkspecs/macx-clang-linux/qmake.conf && \

depends/patches/qt/fix-minimum-macos.patch

@@ -0,0 +1,18 @@
+Ensure that Qt handles the minimum macOS version properly
+
+This patch can be dropped for LLVM Clang 17+, after commit
+https://github.com/llvm/llvm-project/commit/c8e2dd8c6f490b68e41fe663b44535a8a21dfeab
+
+
+--- a/qtbase/src/corelib/global/qsystemdetection.h
++++ b/qtbase/src/corelib/global/qsystemdetection.h
+@@ -220,6 +220,9 @@
+ #  include <Availability.h>
+ #  include <AvailabilityMacros.h>
+ #
++#  undef __MAC_OS_X_VERSION_MIN_REQUIRED
++#  define __MAC_OS_X_VERSION_MIN_REQUIRED MAC_OS_X_VERSION_MIN_REQUIRED
++#
+ #  ifdef Q_OS_MACOS
+ #    if !defined(__MAC_OS_X_VERSION_MIN_REQUIRED) || __MAC_OS_X_VERSION_MIN_REQUIRED < __MAC_10_6
+ #       undef __MAC_OS_X_VERSION_MIN_REQUIRED

depends/patches/qt/windows_lto.patch

@@ -0,0 +1,31 @@
+Qt (for Windows) fails to build under LTO, due to multiple definition issues, i.e
+
+multiple definition of `QAccessibleLineEdit::~QAccessibleLineEdit()';
+
+Possibly related to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94156.
+
+diff --git a/qtbase/src/widgets/accessible/simplewidgets.cpp b/qtbase/src/widgets/accessible/simplewidgets.cpp
+index 107fd729fe..0e61878f39 100644
+--- a/qtbase/src/widgets/accessible/simplewidgets.cpp
++++ b/qtbase/src/widgets/accessible/simplewidgets.cpp
+@@ -109,6 +109,8 @@ QString qt_accHotKey(const QString &text);
+   \ingroup accessibility
+ */
+ 
++QAccessibleLineEdit::~QAccessibleLineEdit(){};
++
+ /*!
+   Creates a QAccessibleButton object for \a w.
+ */
+diff --git a/qtbase/src/widgets/accessible/simplewidgets_p.h b/qtbase/src/widgets/accessible/simplewidgets_p.h
+index 73572e3059..658da86143 100644
+--- a/qtbase/src/widgets/accessible/simplewidgets_p.h
++++ b/qtbase/src/widgets/accessible/simplewidgets_p.h
+@@ -155,6 +155,7 @@ class QAccessibleLineEdit : public QAccessibleWidget, public QAccessibleTextInte
+ public:
+     explicit QAccessibleLineEdit(QWidget *o, const QString &name = QString());
+ 
++    ~QAccessibleLineEdit();
+     QString text(QAccessible::Text t) const override;
+     void setText(QAccessible::Text t, const QString &text) override;
+     QAccessible::State state() const override;

doc/build-osx.md

@@ -231,7 +231,6 @@ There are many ways to configure Dash Core, here are a few common examples:
 ##### Wallet (BDB + SQlite) Support, No GUI:
 
 If `berkeley-db@4` is installed, then legacy wallet support will be built.
-If `berkeley-db@4` is not installed, then this will throw an error.
 If `sqlite` is installed, then descriptor wallet support will also be built.
 Additionally, this explicitly disables the GUI.
 

doc/multiprocess.md

@@ -38,7 +38,7 @@ Alternately, you can install [Cap'n Proto](https://capnproto.org/) and [libmulti
 
 Cross process Node, Wallet, and Chain interfaces are defined in
 [`src/interfaces/`](../src/interfaces/). These are C++ classes which follow
-[conventions](developer-notes.md#internal-interface-guidelines), like passing
+[conventions](../developer-notes.md#internal-interface-guidelines), like passing
 serializable arguments so they can be called from different processes, and
 making methods pure virtual so they can have proxy implementations that forward
 calls between processes.

src/chain.h

@@ -40,13 +40,13 @@ extern RecursiveMutex cs_main;
 class CBlockFileInfo
 {
 public:
-    unsigned int nBlocks;      //!< number of blocks stored in file
-    unsigned int nSize;        //!< number of used bytes of block file
-    unsigned int nUndoSize;    //!< number of used bytes in the undo file
-    unsigned int nHeightFirst; //!< lowest height of block in file
-    unsigned int nHeightLast;  //!< highest height of block in file
-    uint64_t nTimeFirst;       //!< earliest time of block in file
-    uint64_t nTimeLast;        //!< latest time of block in file
+    unsigned int nBlocks{};      //!< number of blocks stored in file
+    unsigned int nSize{};        //!< number of used bytes of block file
+    unsigned int nUndoSize{};    //!< number of used bytes in the undo file
+    unsigned int nHeightFirst{}; //!< lowest height of block in file
+    unsigned int nHeightLast{};  //!< highest height of block in file
+    uint64_t nTimeFirst{};       //!< earliest time of block in file
+    uint64_t nTimeLast{};        //!< latest time of block in file
 
     SERIALIZE_METHODS(CBlockFileInfo, obj)
     {
@@ -59,21 +59,7 @@ public:
         READWRITE(VARINT(obj.nTimeLast));
     }
 
-    void SetNull()
-    {
-        nBlocks = 0;
-        nSize = 0;
-        nUndoSize = 0;
-        nHeightFirst = 0;
-        nHeightLast = 0;
-        nTimeFirst = 0;
-        nTimeLast = 0;
-    }
-
-    CBlockFileInfo()
-    {
-        SetNull();
-    }
+    CBlockFileInfo() {}
 
     std::string ToString() const;
 

src/node/blockstorage.cpp

@@ -164,7 +164,7 @@ void BlockManager::PruneOneBlockFile(const int fileNumber)
         }
     }
 
-    m_blockfile_info[fileNumber].SetNull();
+    m_blockfile_info.at(fileNumber) = CBlockFileInfo{};
     m_dirty_fileinfo.insert(fileNumber);
 }
 

src/node/blockstorage.h

@@ -21,7 +21,6 @@ class CActiveMasternodeManager;
 class ArgsManager;
 class BlockValidationState;
 class CBlock;
-class CBlockFileInfo;
 class CBlockUndo;
 class CChain;
 class CChainParams;

src/policy/fees.cpp

@@ -243,6 +243,11 @@ double TxConfirmStats::EstimateMedianVal(int confTarget, double sufficientTxVal,
     unsigned int curFarBucket = maxbucketindex;
     unsigned int bestFarBucket = maxbucketindex;
 
+    // We'll always group buckets into sets that meet sufficientTxVal --
+    // this ensures that we're using consistent groups between different
+    // confirmation targets.
+    double partialNum = 0;
+
     bool foundAnswer = false;
     unsigned int bins = unconfTxs.size();
     bool newBucketRange = true;
@@ -258,6 +263,7 @@ double TxConfirmStats::EstimateMedianVal(int confTarget, double sufficientTxVal,
         }
         curFarBucket = bucket;
         nConf += confAvg[periodTarget - 1][bucket];
+        partialNum += txCtAvg[bucket];
         totalNum += txCtAvg[bucket];
         failNum += failAvg[periodTarget - 1][bucket];
         for (unsigned int confct = confTarget; confct < GetMaxConfirms(); confct++)
@@ -267,7 +273,14 @@ double TxConfirmStats::EstimateMedianVal(int confTarget, double sufficientTxVal,
         // we can test for success
         // (Only count the confirmed data points, so that each confirmation count
         // will be looking at the same amount of data and same bucket breaks)
-        if (totalNum >= sufficientTxVal / (1 - decay)) {
+
+        if (partialNum < sufficientTxVal / (1 - decay)) {
+            // the buckets we've added in this round aren't sufficient
+            // so keep adding
+            continue;
+        } else {
+            partialNum = 0; // reset for the next range we'll add
+
             double curPct = nConf / (totalNum + failNum + extraNum);
 
             // Check to see if we are no longer getting confirmed at the success rate

src/random.cpp

@@ -41,6 +41,9 @@
 #ifdef HAVE_SYSCTL_ARND
 #include <sys/sysctl.h>
 #endif
+#if defined(HAVE_STRONG_GETAUXVAL) && defined(__aarch64__)
+#include <sys/auxv.h>
+#endif
 
 [[noreturn]] static void RandFailure()
 {
@@ -176,6 +179,62 @@ static uint64_t GetRdSeed() noexcept
 #endif
 }
 
+#elif defined(__aarch64__) && defined(HWCAP2_RNG)
+
+static bool g_rndr_supported = false;
+
+static void InitHardwareRand()
+{
+    if (getauxval(AT_HWCAP2) & HWCAP2_RNG) {
+        g_rndr_supported = true;
+    }
+}
+
+static void ReportHardwareRand()
+{
+    // This must be done in a separate function, as InitHardwareRand() may be indirectly called
+    // from global constructors, before logging is initialized.
+    if (g_rndr_supported) {
+        LogPrintf("Using RNDR and RNDRRS as additional entropy sources\n");
+    }
+}
+
+/** Read 64 bits of entropy using rndr.
+ *
+ * Must only be called when RNDR is supported.
+ */
+static uint64_t GetRNDR() noexcept
+{
+    uint8_t ok;
+    uint64_t r1;
+    do {
+        // https://developer.arm.com/documentation/ddi0601/2022-12/AArch64-Registers/RNDR--Random-Number
+        __asm__ volatile("mrs %0, s3_3_c2_c4_0; cset %w1, ne;"
+                         : "=r"(r1), "=r"(ok)::"cc");
+        if (ok) break;
+        __asm__ volatile("yield");
+    } while (true);
+    return r1;
+}
+
+/** Read 64 bits of entropy using rndrrs.
+ *
+ * Must only be called when RNDRRS is supported.
+ */
+static uint64_t GetRNDRRS() noexcept
+{
+    uint8_t ok;
+    uint64_t r1;
+    do {
+        // https://developer.arm.com/documentation/ddi0601/2022-12/AArch64-Registers/RNDRRS--Reseeded-Random-Number
+        __asm__ volatile("mrs %0, s3_3_c2_c4_1; cset %w1, ne;"
+                         : "=r"(r1), "=r"(ok)::"cc");
+        if (ok) break;
+        __asm__ volatile("yield");
+    } while (true);
+    return r1;
+}
+
 #else
 /* Access to other hardware random number generators could be added here later,
  * assuming it is sufficiently fast (in the order of a few hundred CPU cycles).
@@ -194,6 +253,12 @@ static void SeedHardwareFast(CSHA512& hasher) noexcept {
         hasher.Write((const unsigned char*)&out, sizeof(out));
         return;
     }
+#elif defined(__aarch64__) && defined(HWCAP2_RNG)
+    if (g_rndr_supported) {
+        uint64_t out = GetRNDR();
+        hasher.Write((const unsigned char*)&out, sizeof(out));
+        return;
+    }
 #endif
 }
 
@@ -219,6 +284,14 @@ static void SeedHardwareSlow(CSHA512& hasher) noexcept {
         }
         return;
     }
+#elif defined(__aarch64__) && defined(HWCAP2_RNG)
+    if (g_rndr_supported) {
+        for (int i = 0; i < 4; ++i) {
+            uint64_t out = GetRNDRRS();
+            hasher.Write((const unsigned char*)&out, sizeof(out));
+        }
+        return;
+    }
 #endif
 }
 

src/util/trace.h

@@ -5,6 +5,10 @@
 #ifndef BITCOIN_UTIL_TRACE_H
 #define BITCOIN_UTIL_TRACE_H
 
+#if defined(HAVE_CONFIG_H)
+#include <config/bitcoin-config.h>
+#endif
+
 #ifdef ENABLE_TRACING
 
 #include <sys/sdt.h>

test/fuzz/test_runner.py

@@ -231,7 +231,12 @@ def merge_inputs(*, fuzz_pool, corpus, test_list, src_dir, build_dir, merge_dir)
     for t in test_list:
         args = [
             os.path.join(build_dir, 'src', 'test', 'fuzz', 'fuzz'),
-            '-merge=1',
+            '-rss_limit_mb=8000',
+            '-set_cover_merge=1',
+            # set_cover_merge is used instead of -merge=1 to reduce the overall
+            # size of the qa-assets git repository a bit, but more importantly,
+            # to cut the runtime to iterate over all fuzz inputs [0].
+            # [0] https://github.com/bitcoin-core/qa-assets/issues/130#issuecomment-1761760866
             '-shuffle=0',
             '-prefer_small=1',
             '-use_value_profile=1',  # Also done by oss-fuzz https://github.com/google/oss-fuzz/issues/1406#issuecomment-387790487