Commit Graph

28034 Commits

Author SHA1 Message Date
pasta
7530f3d245
Merge #6490: ci: container improvements
Some checks failed
CI / Build Image (push) Failing after 2m9s
CI / Build Dependencies (arm-linux, arm-linux-gnueabihf) (push) Has been skipped
CI / Build Dependencies (linux64, x86_64-pc-linux-gnu) (push) Has been skipped
CI / Build (arm-linux, arm-linux, arm-linux-gnueabihf) (push) Has been skipped
CI / Build (linux64, linux64, x86_64-pc-linux-gnu) (push) Has been skipped
CI / Build (linux64_cxx20, linux64, x86_64-pc-linux-gnu) (push) Has been skipped
CI / Build (linux64_fuzz, linux64, x86_64-pc-linux-gnu) (push) Has been skipped
CI / Build (linux64_nowallet, linux64, x86_64-pc-linux-gnu) (push) Has been skipped
CI / Build (linux64_sqlite, linux64, x86_64-pc-linux-gnu) (push) Has been skipped
CI / Build (linux64_tsan, linux64, x86_64-pc-linux-gnu) (push) Has been skipped
CI / Build (linux64_ubsan, linux64, x86_64-pc-linux-gnu) (push) Has been skipped
Label Merge Conflicts / main (push) Failing after 2m49s
Check Merge Fast-Forward Only / check_merge (push) Successful in 1m51s
Guix Build / build-image (push) Failing after 6m6s
Guix Build / build (aarch64-linux-gnu) (push) Has been skipped
Guix Build / build (arm-linux-gnueabihf) (push) Has been skipped
Guix Build / build (arm64-apple-darwin) (push) Has been skipped
Guix Build / build (x86_64-apple-darwin) (push) Has been skipped
Guix Build / build (powerpc64-linux-gnu) (push) Has been skipped
Guix Build / build (x86_64-linux-gnu) (push) Has been skipped
Guix Build / build (riscv64-linux-gnu) (push) Has been skipped
Guix Build / build (x86_64-w64-mingw32) (push) Has been skipped
04ce1fea52 ci: deduplicate macOS SDK setup logic (Kittywhiskers Van Gogh)
8dd0db7de9 ci: fix "LC_ALL: cannot change locale (en_US.UTF-8)" in Guix container (Kittywhiskers Van Gogh)
187fe17650 ci: don't stage packages in `/tmp`, reduce layers for `cppcheck` build (Kittywhiskers Van Gogh)
eef863554a ci: install `i386` packages only if host is `amd64`, merge layers (Kittywhiskers Van Gogh)
e7702292d1 ci: purge package manager cache after each interaction (Kittywhiskers Van Gogh)
b7099eed47 ci: remove redundant `version` attribute, avoid `lldb` personality error (Kittywhiskers Van Gogh)
64cdc42130 ci: add LLVM library path to `LD_LIBRARY_PATH` and GDB allowlist (Kittywhiskers Van Gogh)
440fd3fe21 ci: drop distro LLVM packages, move Clang install up, set defaults (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * This pull request pulls container-specific changes from [dash#6387](https://github.com/dashpay/dash/pull/6387), [dash#6400](https://github.com/dashpay/dash/pull/6400) and [dash#6421](https://github.com/dashpay/dash/pull/6421)

  * The `HOST` check before running `setup_sdk.sh` isn't a part of the script itself as the script is written to be independent of external variables set. The caller is expected to know the conditions needed to run `setup_sdk.sh` as the script is _relatively_ agnostic to its environment.

  * The `version` attribute in the [`develop`](a8e2316d6f/contrib/containers/develop/docker-compose.yml) and [`guix`](a8e2316d6f/contrib/containers/guix/docker-compose.yml) container's `docker-compose.yml` has been dropped as the attribute has been deprecated in the compose spec ([source](65ef9f4a5d/spec.md (version-top-level-element-obsolete))).

  * Using `LD_LIBRARY_PATH` to point to LLVM's libraries are acceptable and will not interfere with executing binaries built using the distro's packaged compiler as it will eventually search default paths and find the libraries shipped with the distro ([source](https://man7.org/linux/man-pages/man8/ld.so.8.html)).

  * Currently, running LLDB will result in a "personality set failed: Operation not permitted" error ([source](https://discourse.llvm.org/t/running-lldb-in-a-container/76801)). This is caused by its attempt at disabling ASLR for debugging.

    To work around this error, the container will now operate under relaxed restrictions (`seccomp=unconfined`). As disabling ASLR is valuable when debugging and the container is meant for developers (i.e. it isn't used for CI), we have opted to relax restrictions instead of skipping ASLR disablement.

  * As of `develop` (a8e2316d6f), packages built by the container are stored in `/tmp`, which is inadvisable as it is the same directory used to store functional test runs and it's not too difficult to delete `/tmp`'s contents to save space in a long running [`develop`](a8e2316d6f/contrib/containers/develop/docker-compose.yml) container and then realize that both `shellcheck` and `cppcheck` are stored there and now you have to ditch the container you're working in and restart it.
    * To remedy this, packages are now built and stored in `/opt` in accordance with the FHS ([source](https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s13.html)). `/usr/local` was a contender but it's pre-populated, meanwhile `ls /opt` would give you a very quick picture of what's built for the container.

    * `/tmp` will not be entirely empty because [pypa/pip#10753](https://github.com/pypa/pip/issues/10753) results in residual `.pem` files leaking into `/tmp` and `pyenv` stores its build log there and keeping it around has some debug value.

  ## Breaking Changes

  None expected.

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests **(note: N/A)**
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  UdjinM6:
    ACK 04ce1fea52
  PastaPastaPasta:
    utACK 04ce1fea52

Tree-SHA512: 5442ae06cb73b9bc4eec908803548195ae8fd9150422789e5f98578ad01a303b5361f9ba42fe8faee27ac91e38328b7771e4ba42b296dfa70ecbbfc7d10436b6
2024-12-16 22:29:09 -06:00
pasta
5bf0409eba
Merge #6300: backport: Merge bitcoin#23642, 22794, 23316, 24365, gui#517, 24219, 23253, 24449, 22543
3931608858 Merge bitcoin/bitcoin#22543: test: Use MiniWallet in mempool_limit.py (merge-script)
f147373a32 Merge bitcoin/bitcoin#24449: fuzz: FuzzedFileProvider::write should not return negative value (MarcoFalke)
2a2a2693d0 Merge bitcoin/bitcoin#23253: bitcoin-tx: Reject non-integral and out of range int strings (W. J. van der Laan)
11eeae2ab9 Merge bitcoin/bitcoin#24219: Fix implicit-integer-sign-change in bloom (MarcoFalke)
f16265dd50 Merge bitcoin-core/gui#517: refactor, qt: Use std::chrono for parameters of QTimer methods (Hennadii Stepanov)
b212ca0515 Merge bitcoin/bitcoin#24365: wallet: Don't generate keys for wallets with private keys disabled during upgradewallet (laanwj)
66e77f7879 Merge bitcoin/bitcoin#23316: test: make the node param explicit in init_wallet() (MarcoFalke)
995cae46af Merge bitcoin/bitcoin#22794: test: Verify if wallet is compiled in rpc_invalid_address_message.py test (MarcoFalke)
61a0140362 Merge bitcoin/bitcoin#23642: refactor: Call type-solver earlier in decodescript (MarcoFalke)

Pull request description:

  Bitcoin Backports

ACKs for top commit:
  UdjinM6:
    utACK 3931608858
  PastaPastaPasta:
    utACK 3931608858

Tree-SHA512: 38f384776002e8014b2510aeaf1f4655fea0531011eb326eb2ab546d9e7193ad9e5c4b570d9831f88bb696e06ded04259a21ddb750d7ffedfedebdbb9a951379
2024-12-16 21:01:12 -06:00
pasta
a9cfd39390
Merge #6491: ci: merge bitcoin#27314, #28954, fix multiprocess builds
27d9763b1b fix: add `linux64_multiprocess` `BUILD_TARGET` to matrix, mend C(XX) (Kittywhiskers Van Gogh)
26cc5a1c90 ci: use underscore to separate variant name from target triple (Kittywhiskers Van Gogh)
d0131a5259 trivial: sort `BUILD_TARGET` on GitHub and in `matrix.sh` alphabetically (Kittywhiskers Van Gogh)
4f1b5c165b merge bitcoin#28954: Reduce use of bash -c (Kittywhiskers Van Gogh)
a49162ffae merge bitcoin#27314: Fix handling of `CXX=clang++` when building `qt` package (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * [bitcoin#27314](https://github.com/bitcoin/bitcoin/pull/27314) has been backported in this PR as [bitcoin#25838](https://github.com/bitcoin/bitcoin/pull/25838) (backported in [dash#6384](https://github.com/dashpay/dash/pull/6384)) broke Clang depends builds.

  * [bitcoin#28954](https://github.com/bitcoin/bitcoin/pull/28954) has been backported to fix a problem associated with multiprocess runs ([build](https://gitlab.com/dashpay/dash/-/jobs/8396677312#L2921)).

  * Support for multiprocess builds were enabled _proper_ in [dash#6143](https://github.com/dashpay/dash/pull/6143) but unfortunately, the configuration params for multiprocess builds were not processed by CI as the build variant was not added to `matrix.sh` ([source](6a51ab271d/ci/dash/matrix.sh)). This is evident by comparing two variants with Boost::Process enablement (`--with-boost-process`), `linux64_fuzz` ([source](6a51ab271d/ci/test/00_setup_env_native_fuzz.sh (L19))) and `linux64_multiprocess` ([source](6a51ab271d/ci/test/00_setup_env_native_multiprocess.sh (L13))).

    Looking at a `develop` (6a51ab271d) build, the fuzz build has it enabled ([source](https://gitlab.com/dashpay/dash/-/jobs/8394892905#L737)) while the multiprocess build doesn't ([source](https://gitlab.com/dashpay/dash/-/jobs/8394892909#L1524)) despite both scripts having the enablement argument.

  ## Breaking Changes

  None expected.

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests **(note: N/A)**
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK 27d9763b1b
  UdjinM6:
    utACK 27d9763b1b

Tree-SHA512: 3e2fb72d4211875a162d3aecb994c5bd43b2f6d9fea0804d7e00a38a034672730f9351ceb9256ace38e32f7ef81527c8a034a870e5099a277dfd06f9fa54b480
2024-12-16 20:42:35 -06:00
pasta
0968a0023b
Merge #6492: test: add functional tests for coinjoinsalt RPC
16c2e13fb4 test: add functional tests for `coinjoinsalt` RPC (Kittywhiskers Van Gogh)
a1b256b06f test: extend CoinJoin RPC tests to include more cases, add logging (Kittywhiskers Van Gogh)
c6dd3dd567 test: rename test functions to reflect RPC used, simplify them (Kittywhiskers Van Gogh)
ff29c62103 test: run CoinJoin RPC tests using blank wallet (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * Current suite of tests do not check if restoring salt results in restoring CoinJoin balance. This is because functional tests currently do not test CoinJoin mixing (and thus, routines for the same are not currently present).

  ## Breaking Changes

  None expected.

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  knst:
    utACK 16c2e13fb4
  UdjinM6:
    utACK 16c2e13fb4

Tree-SHA512: 0ce4e67f2caf0619cae42e8158cd39fba24c0a86050e061511ea23c4c0bf34a0eede72917516b6039d7ac15f85730ab36ba9ec1c42d0eb271f6cb4341389bcec
2024-12-16 20:38:11 -06:00
Kittywhiskers Van Gogh
04ce1fea52
ci: deduplicate macOS SDK setup logic
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2024-12-16 12:13:27 +00:00
Kittywhiskers Van Gogh
16c2e13fb4
test: add functional tests for coinjoinsalt RPC 2024-12-16 08:33:23 +00:00
Kittywhiskers Van Gogh
a1b256b06f
test: extend CoinJoin RPC tests to include more cases, add logging 2024-12-16 08:33:22 +00:00
Kittywhiskers Van Gogh
c6dd3dd567
test: rename test functions to reflect RPC used, simplify them 2024-12-16 08:31:27 +00:00
Kittywhiskers Van Gogh
ff29c62103
test: run CoinJoin RPC tests using blank wallet 2024-12-16 08:31:27 +00:00
pasta
3d5dc160d8
Merge #6462: backport: merge bitcoin#23092, #23005, #24704, #24391, #25338, #25422, #25493, #26826, #26814, #26832, #27418, partial bitcoin#27254 (compat backports)
Some checks failed
Guix Build / build-image (push) Failing after 6m2s
Guix Build / build (aarch64-linux-gnu) (push) Has been skipped
Guix Build / build (arm-linux-gnueabihf) (push) Has been skipped
Guix Build / build (arm64-apple-darwin) (push) Has been skipped
Guix Build / build (powerpc64-linux-gnu) (push) Has been skipped
Guix Build / build (riscv64-linux-gnu) (push) Has been skipped
Guix Build / build (x86_64-apple-darwin) (push) Has been skipped
Guix Build / build (x86_64-linux-gnu) (push) Has been skipped
Guix Build / build (x86_64-w64-mingw32) (push) Has been skipped
Check Merge Fast-Forward Only / check_merge (push) Successful in 1m49s
Label Merge Conflicts / main (push) Failing after 40s
CI / Build Image (push) Failing after 15m29s
CI / Build Dependencies (arm-linux, arm-linux-gnueabihf) (push) Has been cancelled
CI / Build Dependencies (linux64, x86_64-pc-linux-gnu) (push) Has been cancelled
CI / Build (arm-linux, arm-linux, arm-linux-gnueabihf) (push) Has been cancelled
CI / Build (linux64, linux64, x86_64-pc-linux-gnu) (push) Has been cancelled
CI / Build (linux64_cxx20, linux64, x86_64-pc-linux-gnu) (push) Has been cancelled
CI / Build (linux64_fuzz, linux64, x86_64-pc-linux-gnu) (push) Has been cancelled
CI / Build (linux64_nowallet, linux64, x86_64-pc-linux-gnu) (push) Has been cancelled
CI / Build (linux64_sqlite, linux64, x86_64-pc-linux-gnu) (push) Has been cancelled
CI / Build (linux64_tsan, linux64, x86_64-pc-linux-gnu) (push) Has been cancelled
CI / Build (linux64_ubsan, linux64, x86_64-pc-linux-gnu) (push) Has been cancelled
2ab561d281 partial bitcoin#27254: Extract util/fs from util/system (Kittywhiskers Van Gogh)
280988bee3 merge bitcoin#27418: Remove windows workaround in authproxy (Kittywhiskers Van Gogh)
a497df0339 merge bitcoin#26832: move (win) S_* defines into bdb (Kittywhiskers Van Gogh)
db676a7e5f merge bitcoin#26814: remove windows-only compat.h usage in random (Kittywhiskers Van Gogh)
94e6637c33 merge bitcoin#26826: remove windows-only compat.h usage in randomenv (Kittywhiskers Van Gogh)
bbb0cceb7a merge bitcoin#25493: document code in compat.h (Kittywhiskers Van Gogh)
3f143096c8 merge bitcoin#25422: globally define NOMINMAX when building with mingw-w64 (Kittywhiskers Van Gogh)
000495df8e merge bitcoin#25338: Avoid incompatibility with CMake AUTOUIC feature (Kittywhiskers Van Gogh)
714ea55dbe build: migrate stacktrace-related flags to `DEBUG_`{`C`,`CXX`}`FLAGS` (Kittywhiskers Van Gogh)
11323c3851 merge bitcoin#24391: stop overriding user autoconf flags (Kittywhiskers Van Gogh)
548121d366 merge bitcoin#24704: remove strnlen back-compat code (Kittywhiskers Van Gogh)
b65038ec94 merge bitcoin#23005: Delay wallet client construction (Kittywhiskers Van Gogh)
f3b065cd9c merge bitcoin#23092: Remove Windows workaround in authproxy (WinError 10053) (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * This PR originally included [bitcoin#27378](https://github.com/bitcoin/bitcoin/pull/27378) but including it caused multiple tests to fail ([build](https://gitlab.com/dashpay/dash/-/jobs/8577987174#L2224)), even after adjusting `timeout`. It has been omitted instead.

    <details>

    <summary>Test failures:</summary>

    ```
    feature_abortnode.py                               |   Failed  | 201 s
    feature_dip3_v19.py                                |   Failed  | 237 s
    feature_llmq_rotation.py                           |   Failed  | 54 s
    p2p_invalid_messages.py                            |   Failed  | 5 s
    rpc_bind.py --nonloopback                          |   Failed  | 63 s
    wallet_multiwallet.py --descriptors                |   Failed  | 79 s
    wallet_multiwallet.py --legacy-wallet              |   Failed  | 89 s

    ALL                                                |   Failed  | 8540 s (accumulated)
    ```

    </details>

    <details>

    <summary>feature_abortnode.py failure:</summary>

    ```
    dash@107cfae2b2b5:/src/dash$ ./test/functional/feature_abortnode.py
    2024-12-09T18:31:01.776000Z TestFramework (INFO): PRNG seed is: 6843309100291447752
    2024-12-09T18:31:01.776000Z TestFramework (INFO): Initializing test directory /tmp/dash_func_test_fcijb99n
    2024-12-09T18:31:02.802000Z TestFramework (INFO): Waiting for crash
    2024-12-09T18:34:23.026000Z TestFramework.utils (ERROR): wait_until() failed. Predicate: ''''
        def is_node_stopped(self):
            """Checks whether the node has stopped.

            Returns True if the node has stopped. False otherwise.
            This method is responsible for freeing resources (self.process)."""
            if not self.running:
                return True
            return_code = self.process.poll()
            if return_code is None:
                return False

            # process has stopped. Assert that it didn't return an error code.
            assert return_code == 0, self._node_msg(
                "Node returned non-zero exit code (%d) when stopping" % return_code)
            self.running = False
            self.process = None
            self.rpc_connected = False
            self.rpc = None
            self.log.debug("Node stopped")
            return True
    '''
    2024-12-09T18:34:23.027000Z TestFramework (ERROR): Assertion failed
    Traceback (most recent call last):
      File "/src/dash/test/functional/test_framework/test_framework.py", line 161, in main
        self.run_test()
      File "/src/dash/./test/functional/feature_abortnode.py", line 44, in run_test
        self.nodes[0].wait_until_stopped(timeout=200)
      File "/src/dash/test/functional/test_framework/test_node.py", line 414, in wait_until_stopped
        wait_until_helper(self.is_node_stopped, timeout=timeout, timeout_factor=self.timeout_factor)
      File "/src/dash/test/functional/test_framework/util.py", line 287, in wait_until_helper
        raise AssertionError("Predicate {} not true after {} seconds".format(predicate_source, timeout))
    AssertionError: Predicate ''''
        def is_node_stopped(self):
            """Checks whether the node has stopped.

            Returns True if the node has stopped. False otherwise.
            This method is responsible for freeing resources (self.process)."""
            if not self.running:
                return True
            return_code = self.process.poll()
            if return_code is None:
                return False

            # process has stopped. Assert that it didn't return an error code.
            assert return_code == 0, self._node_msg(
                "Node returned non-zero exit code (%d) when stopping" % return_code)
            self.running = False
            self.process = None
            self.rpc_connected = False
            self.rpc = None
            self.log.debug("Node stopped")
            return True
    ''' not true after 200.0 seconds
    2024-12-09T18:34:23.530000Z TestFramework (INFO): Stopping nodes
    2024-12-09T18:34:23.530000Z TestFramework (ERROR): Unexpected exception caught during shutdown
    Traceback (most recent call last):
      File "/src/dash/test/functional/test_framework/test_framework.py", line 341, in shutdown
        self.stop_nodes()
      File "/src/dash/test/functional/test_framework/test_framework.py", line 673, in stop_nodes
        node.stop_node(expected_stderr=expected_stderr, wait=wait, wait_until_stopped=False)
      File "/src/dash/test/functional/test_framework/test_node.py", line 368, in stop_node
        self.stop(wait=wait)
      File "/src/dash/test/functional/test_framework/coverage.py", line 49, in __call__
        return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
      File "/src/dash/test/functional/test_framework/authproxy.py", line 124, in __call__
        response, status = self._request('POST', self.__url.path, postdata.encode('utf-8'))
      File "/src/dash/test/functional/test_framework/authproxy.py", line 103, in _request
        return self._get_response()
      File "/src/dash/test/functional/test_framework/authproxy.py", line 162, in _get_response
        raise JSONRPCException(
    test_framework.authproxy.JSONRPCException: non-JSON HTTP response with '503 Service Unavailable' from server (-342)
    2024-12-09T18:34:23.531000Z TestFramework (WARNING): Not cleaning up dir /tmp/dash_func_test_fcijb99n
    2024-12-09T18:34:23.531000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/dash_func_test_fcijb99n/test_framework.log
    2024-12-09T18:34:23.531000Z TestFramework (ERROR):
    2024-12-09T18:34:23.531000Z TestFramework (ERROR): Hint: Call /src/dash/test/functional/combine_logs.py '/tmp/dash_func_test_fcijb99n' to consolidate all logs
    2024-12-09T18:34:23.531000Z TestFramework (ERROR):
    2024-12-09T18:34:23.531000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
    2024-12-09T18:34:23.531000Z TestFramework (ERROR): https://github.com/dashpay/dash/issues
    2024-12-09T18:34:23.531000Z TestFramework (ERROR):
    [node 1] Cleaning up leftover process
    [node 0] Cleaning up leftover process
    ```

    </details>

  * [bitcoin#27254](https://github.com/bitcoin/bitcoin/pull/27254) has been partially backported to include a missing header to allow it to build.

    <details>

    <summary>Compile error:</summary>

    ```
    In file included from ./util/system.h:19,
                   from util/system.cpp:7:
    ./compat/assumptions.h:49:22: error: 'size_t' was not declared in this scope; did you mean 'std::size_t'?
       49 | static_assert(sizeof(size_t) == 4 || sizeof(size_t) == 8, "size_t assumed to be 32-bit or 64-bit");
          |                      ^~~~~~
          |                      std::size_t
    In file included from /usr/include/c++/11/limits:42,
                   from ./compat/assumptions.h:11,
                   from ./util/system.h:19,
                   from util/system.cpp:7:
    /usr/include/x86_64-linux-gnu/c++/11/bits/c++config.h:280:33: note: 'std::size_t' declared here
      280 |   typedef __SIZE_TYPE__         size_t;
          |                                 ^~~~~~
    ```

    </details>

  * ~~GitLab job failures as of 2339ae6459cad77d94795c0e92ba3f3fe31c485a, for `linux64-build` ([source](https://gitlab.com/dashpay/dash/-/jobs/8590291130#L4621)), `linux64_cxx20-build` ([source](https://gitlab.com/dashpay/dash/-/jobs/8590291134#L3497)) and `linux64_sqlite-build` ([source](https://gitlab.com/dashpay/dash/-/jobs/8590291137#L3489)) are not related to failures in compilation or unit tests but are to do with reaching the artifact limit.~~ Resolved with [dash#6487](https://github.com/dashpay/dash/pull/6487), thanks Udjin!

  ## Breaking Changes

  None expected.

  ## Checklist

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  UdjinM6:
    utACK 2ab561d281
  PastaPastaPasta:
    utACK 2ab561d281

Tree-SHA512: 6d05305f17fd607ed93b4a2e00de195a96bde1a34bedf04f6dd84892dc71028ae71590fa5063026ec7ff728d6e411af410c363b4a88947ce6e5999d4431bb29b
2024-12-15 12:30:54 -06:00
pasta
032fc21198
Merge #6479: feat: enable coverage linter for functional tests
2e509b96c4 fix: add a workaround for RPC getmerkleblocks, debug, coinjoinsalt, voteraw (Konstantin Akimov)
f0decc8790 feat: add unit test for ClearDiscouraged (Konstantin Akimov)
865b24ea00 feat: hide cleardiscouraged RPC so far as it no intent to use by regular users (Konstantin Akimov)
1f5fa7e7cf feat: enable linter coverage for functional tests (Konstantin Akimov)
59ddac5656 feat: hide deprecated RPC from help and add TODOes to remove them (Konstantin Akimov)
05732aceaf feat: implement functional tests for RPC getblockheaders (Konstantin Akimov)

Pull request description:

  ## Issue being fixed or feature implemented
  https://github.com/dashpay/dash-issues/issues/63

  ## What was done?
  Add functional tests for `getblockheaders`
  Hide RPC `cleardiscouraged` (as it is used only for functional tests) and RPC `getpoolinfo` (deprecated long time ago)
  Add a workaround to ignore these RPCs `getmerkleblocks`, `voteraw`, `debug`, `coinjoinsalt` at the moment
  Enables linter for coverage

  ## How Has This Been Tested?
  Run locally with `test/functional/test_runner.py -j20 --previous-releases --coverage --extended`
  Enabled in CI

  ## Breaking Changes
  N/A if hidding `cleardiscouraged` is not a breaking change.

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  UdjinM6:
    LGTM, utACK 2e509b96c4
  PastaPastaPasta:
    utACK 2e509b96c4

Tree-SHA512: bb31465d9a71ef824533d9310393d89293c87c7407ec3e37697f6d36dc6c010381a6e0408f9598354e610d51ef662485d8a653cc0e198842e2198ac1c30c90f1
2024-12-15 12:12:40 -06:00
pasta
05ca6cffff
Merge #6489: ci: allow overriding MAKEJOBS
feb6a1ad3f ci: allow overriding MAKEJOBS (UdjinM6)

Pull request description:

  ## Issue being fixed or feature implemented
  That's how these variable are ment to behave
  >echo "Fallback to default values in env (if not yet set)"

  https://github.com/dashpay/dash/blame/develop/ci/test/00_setup_env.sh#L26

  That's also how it's done in bitcoin https://github.com/bitcoin/bitcoin/blob/master/ci/test/00_setup_env.sh#L38.

  But we broke it in c52992aaa4 and I'm not sure why 🤷‍♂️

  ## What was done?

  ## How Has This Been Tested?

  ## Breaking Changes

  ## Checklist:
  - [ ] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK feb6a1ad3f
  kwvg:
    utACK feb6a1ad3f

Tree-SHA512: 617e761cf5c52d8e4329bf8aa1680c72a7816ae3248ed415d2c3232e5aa0941f18e21f0d6d363848238f53730097aa22f51a6e1cd62bd0b31d3da853f9f79e28
2024-12-15 11:57:04 -06:00
Kittywhiskers Van Gogh
27d9763b1b
fix: add linux64_multiprocess BUILD_TARGET to matrix, mend C(XX) 2024-12-15 13:12:30 +00:00
Kittywhiskers Van Gogh
26cc5a1c90
ci: use underscore to separate variant name from target triple 2024-12-15 13:12:25 +00:00
Kittywhiskers Van Gogh
d0131a5259
trivial: sort BUILD_TARGET on GitHub and in matrix.sh alphabetically 2024-12-15 13:12:04 +00:00
Kittywhiskers Van Gogh
4f1b5c165b
merge bitcoin#28954: Reduce use of bash -c 2024-12-15 13:08:32 +00:00
Kittywhiskers Van Gogh
a49162ffae
merge bitcoin#27314: Fix handling of CXX=clang++ when building qt package 2024-12-15 13:08:29 +00:00
Kittywhiskers Van Gogh
8dd0db7de9
ci: fix "LC_ALL: cannot change locale (en_US.UTF-8)" in Guix container 2024-12-15 11:02:55 +00:00
Kittywhiskers Van Gogh
187fe17650
ci: don't stage packages in /tmp, reduce layers for cppcheck build 2024-12-15 11:02:55 +00:00
Kittywhiskers Van Gogh
eef863554a
ci: install i386 packages only if host is amd64, merge layers 2024-12-15 11:02:55 +00:00
Kittywhiskers Van Gogh
e7702292d1
ci: purge package manager cache after each interaction 2024-12-15 11:02:55 +00:00
Kittywhiskers Van Gogh
b7099eed47
ci: remove redundant version attribute, avoid lldb personality error 2024-12-15 11:02:54 +00:00
Kittywhiskers Van Gogh
64cdc42130
ci: add LLVM library path to LD_LIBRARY_PATH and GDB allowlist 2024-12-15 10:59:47 +00:00
Kittywhiskers Van Gogh
440fd3fe21
ci: drop distro LLVM packages, move Clang install up, set defaults
Also simplify the download and execution of `llvm.sh`
2024-12-15 10:59:46 +00:00
Kittywhiskers Van Gogh
2ab561d281
partial bitcoin#27254: Extract util/fs from util/system
includes:
- b202b3dd6393b415fa68e18dc49c9431dc6b58b2
2024-12-15 09:08:55 +00:00
Kittywhiskers Van Gogh
280988bee3
merge bitcoin#27418: Remove windows workaround in authproxy 2024-12-15 09:08:55 +00:00
Kittywhiskers Van Gogh
a497df0339
merge bitcoin#26832: move (win) S_* defines into bdb 2024-12-15 09:08:55 +00:00
Kittywhiskers Van Gogh
db676a7e5f
merge bitcoin#26814: remove windows-only compat.h usage in random 2024-12-15 09:08:55 +00:00
Kittywhiskers Van Gogh
94e6637c33
merge bitcoin#26826: remove windows-only compat.h usage in randomenv 2024-12-15 09:08:55 +00:00
Kittywhiskers Van Gogh
bbb0cceb7a
merge bitcoin#25493: document code in compat.h 2024-12-15 09:08:55 +00:00
Kittywhiskers Van Gogh
3f143096c8
merge bitcoin#25422: globally define NOMINMAX when building with mingw-w64 2024-12-15 09:08:22 +00:00
Kittywhiskers Van Gogh
000495df8e
merge bitcoin#25338: Avoid incompatibility with CMake AUTOUIC feature 2024-12-15 09:08:22 +00:00
Kittywhiskers Van Gogh
714ea55dbe
build: migrate stacktrace-related flags to DEBUG_{C,CXX}FLAGS
`DEBUG_CFLAGS` doesn't exist so we need to create it
2024-12-15 09:08:22 +00:00
Kittywhiskers Van Gogh
11323c3851
merge bitcoin#24391: stop overriding user autoconf flags 2024-12-15 09:08:22 +00:00
Kittywhiskers Van Gogh
548121d366
merge bitcoin#24704: remove strnlen back-compat code 2024-12-15 09:08:22 +00:00
Kittywhiskers Van Gogh
b65038ec94
merge bitcoin#23005: Delay wallet client construction 2024-12-15 09:08:21 +00:00
Kittywhiskers Van Gogh
f3b065cd9c
merge bitcoin#23092: Remove Windows workaround in authproxy (WinError 10053) 2024-12-15 09:08:21 +00:00
pasta
e2095bd68a
Merge #6488: backport: merge bitcoin#27452, #29347, #29356, #29353, #29452, #29483, #30545, #31383 (BIP324 backports: part 5)
c6f23a718c merge bitcoin#31383: Add missing node.setmocktime(self.mocktime) to p2p_ibd_stalling.py (Kittywhiskers Van Gogh)
9072a10754 merge bitcoin#30545: fix intermittent failures in feature_proxy.py (Kittywhiskers Van Gogh)
7e2d435e35 merge bitcoin#29483: add --v1transport option, add --v2transport to a CI task (Kittywhiskers Van Gogh)
7e59a965f8 merge bitcoin#29452: document that BIP324 on by default (Kittywhiskers Van Gogh)
0f3b5e081e merge bitcoin#29353: adhere to typical VERSION message protocol flow (Kittywhiskers Van Gogh)
dfdddfd2ff rpc: enable `v2transport` for `masternode connect` when capable (Kittywhiskers Van Gogh)
3c1636174b merge bitcoin#29356: make v2transport arg in addconnection mandatory and few cleanups (Kittywhiskers Van Gogh)
c53cd93aee merge bitcoin#29347: enable v2transport by default (Kittywhiskers Van Gogh)
7dcf561306 merge bitcoin#27452: cover addrv2 anchors by adding TorV3 to CAddress in messages.py (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * When backporting [bitcoin#27452](https://github.com/bitcoin/bitcoin/pull/27452) in `feature_anchors.py`, `P2P_SERVICES` (`NODE_NETWORK | NODE_HEADERS_COMPRESSED`) has been replaced with `NODE_NETWORK` as the former evaluates to a value greater than `256` (specifically `2049`), which causes test failure. The replacement value is acceptable as `NODE_NETWORK` is the desired service flag expected by Dash Core ([source](779e4295ad/src/protocol.cpp (L249-L254))).

    <details>

    <summary>Test failure:</summary>

    ```
    dash@89afd55ae77e:/src/dash$ ./test/functional/feature_anchors.py
    2024-12-14T12:31:22.244000Z TestFramework (INFO): PRNG seed is: 8365703189892653614
    2024-12-14T12:31:22.244000Z TestFramework (INFO): Initializing test directory /tmp/dash_func_test_j0ya02yu
    2024-12-14T12:31:22.776000Z TestFramework (INFO): When node starts, check if anchors.dat doesn't exist
    2024-12-14T12:31:22.776000Z TestFramework (INFO): Add 2 block-relay-only connections to node
    2024-12-14T12:31:24.781000Z TestFramework (INFO): Add 5 inbound connections to node
    2024-12-14T12:31:29.843000Z TestFramework (INFO): Check node connections
    2024-12-14T12:31:30.848000Z TestFramework (INFO): Check the addresses in anchors.dat
    2024-12-14T12:31:30.848000Z TestFramework (INFO): Perturb anchors.dat to test it doesn't throw an error during initialization
    2024-12-14T12:31:31.356000Z TestFramework (INFO): When node starts, check if anchors.dat doesn't exist anymore
    2024-12-14T12:31:31.357000Z TestFramework (INFO): Ensure addrv2 support
    2024-12-14T12:31:32.364000Z TestFramework (INFO): Add 256-bit-address block-relay-only connections to node
    2024-12-14T12:31:33.368000Z TestFramework (INFO): Check for addrv2 addresses in anchors.dat
    2024-12-14T12:31:33.369000Z TestFramework (ERROR): Unexpected exception caught during testing
    Traceback (most recent call last):
      File "/src/dash/test/functional/test_framework/test_framework.py", line 161, in main
        self.run_test()
      File "/src/dash/./test/functional/feature_anchors.py", line 135, in run_test
        new_data[services_index] = P2P_SERVICES
    ValueError: byte must be in range(0, 256)
    2024-12-14T12:31:33.870000Z TestFramework (INFO): Stopping nodes
    2024-12-14T12:31:33.871000Z TestFramework (WARNING): Not cleaning up dir /tmp/dash_func_test_j0ya02yu
    2024-12-14T12:31:33.871000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/dash_func_test_j0ya02yu/test_framework.log
    ```

    </details>

  ## Breaking Changes

  None expected.

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  UdjinM6:
    utACK c6f23a718c
  PastaPastaPasta:
    utACK c6f23a718c81b7c574ba3fecbe52f342eda35a46;

Tree-SHA512: ca134432d000d521827a20c75c03913421fe52a31fda1cbf632e0b207c31728406feb090469d592d8e2fd8d64298faa2752ff703de79f737a62c276c6a231097
2024-12-14 19:41:06 -06:00
UdjinM6
feb6a1ad3f
ci: allow overriding MAKEJOBS 2024-12-15 00:16:21 +03:00
pasta
99ca07f607
Merge #6487: ci: exclude more files from artifacts, better names for downloaded files
a191c09cdc ci: better names for artifacts (UdjinM6)
3984a49a43 ci: exclude `*.a` and `*.o` files from artifacts (UdjinM6)

Pull request description:

  ## Issue being fixed or feature implemented
  We include too many files in artifacts on `build` ci step, some of which (`*.a` and `*.o`) can be pretty heavy. This was ok-ish for some time but artifacts size is getting closer to the limit and even starts to cause issues, see #6462.

  ## What was done?
  Exclude `*.a` and `*.o` files from artifacts. Also, change artifacts name to make it easier to distinguish them when you get a few of them from the same pipeline - `build-arm-linux.zip`, `build-linux64.zip` etc. instead of `binaries.zip`, `binaries (1).zip` etc., same for `depends`.

  A simpler alternative to #6483

  As a result the size of tsan artifacts for example is down from 508MB in 6462 to 154MB in this PR.

  ## How Has This Been Tested?

  ## Breaking Changes

  ## Checklist:
  - [ ] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK a191c09cdc
  kwvg:
    utACK a191c09cdc

Tree-SHA512: eb3029c95a11546958d34ad284f4eb600b6113a2cabb885f7381afd16696f51da7eaf1677a1cdfec53311838c6845bb1c2c7c98f0ae112294b33f6ab6c66c500
2024-12-14 14:13:06 -06:00
pasta
4a63d64340
Merge #6486: ci: do not check --ff-only on master branch
dcc1ff37bc ci: do not check `--ff-only` on master branch (UdjinM6)

Pull request description:

  ## Issue being fixed or feature implemented
  https://github.com/dashpay/dash/actions/runs/12303817802/job/34340079798

  It makes no sense to check it anyway.

  ## What was done?

  ## How Has This Been Tested?

  ## Breaking Changes

  ## Checklist:
  - [ ] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK dcc1ff37bc

Tree-SHA512: bcff6ff7bcb70f2d48e9df7db2a76b65b386f0cc7f213e44dc13a44416e1cab9449c1344166ac58e8e577dcfbdc22b0b4ebb93d610b0790623188c1680804b69
2024-12-14 13:08:18 -06:00
pasta
f656c51979
Merge #6481: test: actually use masternode with basic bls pubkey in mnauth test
ec00c372c7 test: fix off-by-one in dynamically_add_masternode (UdjinM6)
6519856344 test: don't add legacy bls mn on start (UdjinM6)
3db20e3ed4 test: actually use masternode with basic bls pubkey in mnauth test (UdjinM6)

Pull request description:

  ## Issue being fixed or feature implemented
  `rpc_manuth` is failing in ~50% cases locally because we still use legacy pubkeys (not in 100% cases because sometimes they look like basic ones). In CI it fails too but we retry failed tests a few times so it's less noticeable. Example of "unlucky" tests: https://gitlab.com/dashpay/dash/-/jobs/8613271300#L1867.

  #6467 follow-up

  ## What was done?
  Add another masternode after v19 activaition to actually use basic bls pubkey

  ## How Has This Been Tested?
  run tests

  ## Breaking Changes
  n/a

  ## Checklist:
  - [ ] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  knst:
    utACK ec00c372c7
  PastaPastaPasta:
    utACK ec00c372c7

Tree-SHA512: 850a02ea1bd943762cdb0be706f3703742944c294ee9603b1f9ab95a6b10fb827bc9376e03333177d956b2c13df7384cfe0eb2ffef4d05ff3ec239caa8318d24
2024-12-14 12:11:42 -06:00
pasta
c35cf1c278
Merge #6485: revert: Revert "feat: re-bury DIP0024 with new height when quorums actually appeared"
2c296de6dd revert: Revert "feat: re-bury DIP0024 with new height when quorums actually appeared" (UdjinM6)

Pull request description:

  ## Issue being fixed or feature implemented
  `-assumevalid=0` sync is broken on develop, v22 and master aren't affected

  mainnet:
  ```
  [   loadblk] [util/system.h:57] [error] ERROR: ConnectBlock(DASH): ProcessSpecialTxsInBlock for block 00000000000000195f69a348a14fddc7c8ff61036348e95b818ca42c6c724bbb failed with bad-qc-dup
  [   loadblk] [validation.cpp:1365] [InvalidChainFound] InvalidChainFound: invalid block=00000000000000195f69a348a14fddc7c8ff61036348e95b818ca42c6c724bbb  height=1737834  log2_work=78.911419  date=2022-09-13T22:36:20Z
  [   loadblk] [validation.cpp:1370] [InvalidChainFound] InvalidChainFound:  current best=0000000000000002e91bac9f95f96885a89de2c46846d197933c8c5f03cb806f  height=1737833  log2_work=78.911418  date=2022-09-13T22:35:44Z
  [   loadblk] [util/system.h:57] [error] ERROR: ConnectTip: ConnectBlock 00000000000000195f69a348a14fddc7c8ff61036348e95b818ca42c6c724bbb failed, bad-qc-dup
  ```

  testnet:
  ```
  [   loadblk] [util/system.h:57] [error] ERROR: ConnectBlock(DASH): ProcessSpecialTxsInBlock for block 00000050ec7aa1b187c3a9df0d80a2ed2821d5836b596b962bcbc4c261b4bcd5 failed with bad-qc-dup
  [   loadblk] [validation.cpp:1357] [InvalidChainFound] InvalidChainFound: invalid block=00000050ec7aa1b187c3a9df0d80a2ed2821d5836b596b962bcbc4c261b4bcd5  height=769866  log2_work=57.312136  date=2022-07-28T03:32:01Z
  [   loadblk] [validation.cpp:1362] [InvalidChainFound] InvalidChainFound:  current best=0000001acc25de06893220ed663f35ef8b8e82ec951963c4092c1e9ee103af6f  height=769865  log2_work=57.312136  date=2022-07-28T03:28:21Z
  [   loadblk] [util/system.h:57] [error] ERROR: ConnectTip: ConnectBlock 00000050ec7aa1b187c3a9df0d80a2ed2821d5836b596b962bcbc4c261b4bcd5 failed, bad-qc-dup
  ```

  ## What was done?
  This reverts commit 632c4c4bcd.

  ## How Has This Been Tested?
  Hint: sync to pre-dip24 block with `--stopatheight=1737790` and no `-assumevalid` to speed things up, then start from that block with `-assumevalid=0`

  ## Breaking Changes

  ## Checklist:
  - [ ] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK 2c296de6dd

Tree-SHA512: 02d62d34b3914af61c3b415cdffcfca8d7e18b192bffadb0ceb06d6aca8b828491a68cc2a23b9e6b369efb6f2e1c8e84bbd6332b3c74f27f3d9469efaed72817
2024-12-14 12:05:55 -06:00
Kittywhiskers Van Gogh
c6f23a718c
merge bitcoin#31383: Add missing node.setmocktime(self.mocktime) to p2p_ibd_stalling.py 2024-12-14 12:00:52 +00:00
Kittywhiskers Van Gogh
9072a10754
merge bitcoin#30545: fix intermittent failures in feature_proxy.py 2024-12-14 12:00:51 +00:00
Kittywhiskers Van Gogh
7e2d435e35
merge bitcoin#29483: add --v1transport option, add --v2transport to a CI task 2024-12-14 12:00:51 +00:00
Kittywhiskers Van Gogh
7e59a965f8
merge bitcoin#29452: document that BIP324 on by default 2024-12-14 12:00:51 +00:00
Kittywhiskers Van Gogh
0f3b5e081e
merge bitcoin#29353: adhere to typical VERSION message protocol flow 2024-12-14 12:00:51 +00:00
Kittywhiskers Van Gogh
dfdddfd2ff
rpc: enable v2transport for masternode connect when capable 2024-12-14 12:00:51 +00:00
Kittywhiskers Van Gogh
3c1636174b
merge bitcoin#29356: make v2transport arg in addconnection mandatory and few cleanups 2024-12-14 12:00:51 +00:00