04ce1fea52 ci: deduplicate macOS SDK setup logic (Kittywhiskers Van Gogh)
8dd0db7de9 ci: fix "LC_ALL: cannot change locale (en_US.UTF-8)" in Guix container (Kittywhiskers Van Gogh)
187fe17650 ci: don't stage packages in `/tmp`, reduce layers for `cppcheck` build (Kittywhiskers Van Gogh)
eef863554a ci: install `i386` packages only if host is `amd64`, merge layers (Kittywhiskers Van Gogh)
e7702292d1 ci: purge package manager cache after each interaction (Kittywhiskers Van Gogh)
b7099eed47 ci: remove redundant `version` attribute, avoid `lldb` personality error (Kittywhiskers Van Gogh)
64cdc42130 ci: add LLVM library path to `LD_LIBRARY_PATH` and GDB allowlist (Kittywhiskers Van Gogh)
440fd3fe21 ci: drop distro LLVM packages, move Clang install up, set defaults (Kittywhiskers Van Gogh)
Pull request description:
## Additional Information
* This pull request pulls container-specific changes from [dash#6387](https://github.com/dashpay/dash/pull/6387), [dash#6400](https://github.com/dashpay/dash/pull/6400) and [dash#6421](https://github.com/dashpay/dash/pull/6421)
* The `HOST` check before running `setup_sdk.sh` isn't a part of the script itself as the script is written to be independent of external variables set. The caller is expected to know the conditions needed to run `setup_sdk.sh` as the script is _relatively_ agnostic to its environment.
* The `version` attribute in the [`develop`](a8e2316d6f/contrib/containers/develop/docker-compose.yml) and [`guix`](a8e2316d6f/contrib/containers/guix/docker-compose.yml) container's `docker-compose.yml` has been dropped as the attribute has been deprecated in the compose spec ([source](65ef9f4a5d/spec.md (version-top-level-element-obsolete))).
* Using `LD_LIBRARY_PATH` to point to LLVM's libraries are acceptable and will not interfere with executing binaries built using the distro's packaged compiler as it will eventually search default paths and find the libraries shipped with the distro ([source](https://man7.org/linux/man-pages/man8/ld.so.8.html)).
* Currently, running LLDB will result in a "personality set failed: Operation not permitted" error ([source](https://discourse.llvm.org/t/running-lldb-in-a-container/76801)). This is caused by its attempt at disabling ASLR for debugging.
To work around this error, the container will now operate under relaxed restrictions (`seccomp=unconfined`). As disabling ASLR is valuable when debugging and the container is meant for developers (i.e. it isn't used for CI), we have opted to relax restrictions instead of skipping ASLR disablement.
* As of `develop` (a8e2316d6f), packages built by the container are stored in `/tmp`, which is inadvisable as it is the same directory used to store functional test runs and it's not too difficult to delete `/tmp`'s contents to save space in a long running [`develop`](a8e2316d6f/contrib/containers/develop/docker-compose.yml) container and then realize that both `shellcheck` and `cppcheck` are stored there and now you have to ditch the container you're working in and restart it.
* To remedy this, packages are now built and stored in `/opt` in accordance with the FHS ([source](https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s13.html)). `/usr/local` was a contender but it's pre-populated, meanwhile `ls /opt` would give you a very quick picture of what's built for the container.
* `/tmp` will not be entirely empty because [pypa/pip#10753](https://github.com/pypa/pip/issues/10753) results in residual `.pem` files leaking into `/tmp` and `pyenv` stores its build log there and keeping it around has some debug value.
## Breaking Changes
None expected.
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e tests **(note: N/A)**
- [x] I have made corresponding changes to the documentation **(note: N/A)**
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
ACK 04ce1fea52
PastaPastaPasta:
utACK 04ce1fea52
Tree-SHA512: 5442ae06cb73b9bc4eec908803548195ae8fd9150422789e5f98578ad01a303b5361f9ba42fe8faee27ac91e38328b7771e4ba42b296dfa70ecbbfc7d10436b6
They run multiprocess build for ALL PRs in travis since that commit
```diff
- if: type != pull_request OR commit_message =~ /depends:|multiprocess:/ # Skip on non-depends, non-multiprocess PRs
```
There's no actual multiprocess code yet, that's just part related to build system.
To test build add an option --enable-multiprocess to configure
So far as there's no real multiprocessing yet, let's save CPU money - last commit in this PR disables gitlab's CI temporary
e2bab2aa162ae38b2bf8195b577c982402fbee9d multiprocess: add multiprocess travis configuration (Russell Yanofsky)
603fd6a2e708c04ef6c9880f89d0a4cbaa6fc7c5 depends: add MULTIPROCESS depends option (Russell Yanofsky)
5d1377b52bfcd4edf8553aaf332bfeb92fc554cc build: multiprocess autotools changes (Russell Yanofsky)
Pull request description:
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
---
This PR consists of build changes only. It adds an `--enable-multiprocess` autoconf option (off by default and marked experimental), that builds new `bitcoin-node` and `bitcoin-gui` binaries. These currently function the same as existing `bitcoind` and `bitcoin-qt` binaries, but are extended in #10102 with IPC features to execute node, wallet, and gui functions in separate processes.
In addition to adding the `--enable-multiprocess` config flag, it also adds a depends package and autoconf rules to build with the [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library, and it adds new travis configuration to exercise the build code and run functional tests with the new binaries.
The changes in this PR were originally part of #10102 but were moved into #16367 to be able to develop and review the multiprocess build changes independently of the code changes. #16367 was briefly merged and then reverted in #18588. Only change since #16367 has been dropping the `native_boost.mk` depends package which was pointed out to be no longer necessary in https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-596484337 and https://github.com/bitcoin/bitcoin/pull/18588#pullrequestreview-391765649
ACKs for top commit:
practicalswift:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d
Sjors:
tACK e2bab2aa162ae38b2bf8195b577c982402fbee9d on macOS 10.15.4
hebasto:
ACK e2bab2aa162ae38b2bf8195b577c982402fbee9d, tested on Linux Mint 19.3 (x86_64):
Tree-SHA512: b5a76eab5abf63d9d8b6d628cbdff4cc1888eef15cafa0a5d56369e2f9d02595fed623f4b74b2cf2830c42c05a774f0943e700f9c768a82d9d348cad199e135c
## Issue being fixed or feature implemented
Make CI builds faster.
We have this
https://github.com/dashpay/dash/blob/develop/ci/dash/build_src.sh#L18
but by that time `CCACHE_SIZE` is set to 100M via
https://github.com/dashpay/dash/blob/develop/ci/test/00_setup_env.sh#L51
already.
## What was done?
set `CCACHE_SIZE` variable in `.gitlab-ci.yml`, drop confusing and
useless lines from `ci/dash/build_src.sh` (`CCACHE_SIZE` and
`CCACHE_COMPRESS` defaults are handled by `ci/test/00_setup_env.sh`)
## How Has This Been Tested?
I set `CCACHE_SIZE` to 400M yesterday via Gitlab UI to test it.
results:
before, trivial doc change, 100M:
https://gitlab.com/dashpay/dash/-/jobs/4931233566#L59 (~17 minutes)
after, trivial code change, 400M:
https://gitlab.com/dashpay/dash/-/jobs/4935764148#L80 (~12 minutes)
Removed it from Gitlab UI now but it should still say `Set cache size
limit to 400.0 MB` in logs for this PR.
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
fa082d0a57afedca9122fac4aecd6a3070f06b04 travis: Remove valgrind (MarcoFalke)
Pull request description:
When the valgrind run was added, it took 2 hours. Travis kindly raised the timeout limit to the maximum possible of 3 hours.
Today, a full build of Bitcoin Core with all tests takes more than three hours. Thus, it is impossible to run all tests on travis.
Moreover, the feedback loop for developers that create a pull request takes at least 2 hours, but in some cases (when the travis queue is full) until the next day. This is unacceptable.
Fix both issues by removing the build from travis.
Please note that the `ci/test/` configurations are *not* removed. They will stay in the repo and can be executed anywhere (just not on travis).
ACKs for top commit:
jamesob:
ACK fa082d0a57
jnewbery:
utACK fa082d0a57afedca9122fac4aecd6a3070f06b04
Tree-SHA512: 9acaa0e2d3926014fadb7dd2e86c4e01df382e9399f6ae99f989fa609da66a77bdd1b75d6ff42d2686f38f730b8564e6dc722aa597a473290c9d30c2abe7ef0f
## Issue being fixed or feature implemented
We should not be running jobs if their ancestors failed. This was broken
with the introduction of `FAST_MODE`/`.skip-in-fast-mode-template` I
think (via #3635).
## What was done?
Adjust ci rules, make them explicit.
## How Has This Been Tested?
Compare `test` stages:
https://gitlab.com/dashpay/dash/-/pipelines/888510672 (note: some of
`test` stage jobs were started, even though all related jobs from
`build` stage have failed)
vs
https://gitlab.com/UdjinM6/dash/-/pipelines/888923382 (note: none of
`test` stage jobs were started)
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
## Issue being fixed or feature implemented
CI should not be running that many jobs in "FAST_MODE"
https://github.com/dashpay/dash/blob/develop/.gitlab-ci.yml#L5
## What was done?
assign `.skip-in-fast-mode-template` template to more jobs
## How Has This Been Tested?
ran CI in my repo
before: https://gitlab.com/UdjinM6/dash/-/pipelines/832175740
after: https://gitlab.com/UdjinM6/dash/-/pipelines/832176325
## Breaking Changes
n/a
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
**For repository code-owners and collaborators only**
- [ ] I have assigned this pull request to a milestone
Make fMixing atomic as it has concurrent access
Add tsan suppression for zmq namespace
Suppress deadlock false positive in ConnectTip
Switch ubsan target to linux32
Add new test job for linux64_cxx17 target without any sanitizers
Increase rpc time out for block reward reallocation test
Fix heap use after free in CConnman::GetExtraOutboundCount()
Different builds for linux32 and linux64 tsan and ubsan
Increase timeout for llmq_signing functional test
fa193dc8e6f3b96fa2dba2f1c1668f7720fed320 doc: Remove win32 from the release process (MarcoFalke)
faf666f8148eeb305a9c4f78459aff2c7268016b Remove Windows 32 bit build (MarcoFalke)
Pull request description:
The Windows 32 bit build has been removed from https://bitcoincore.org/en/download/, so unless there are complaints, we don't need to build it even
ACKs for commit fa193d:
fanquake:
utACK fa193dc8e6
Tree-SHA512: d6f2976a2e0c407698f720b00ac23ec4056626de4eff8621f4c5581120af0460afd1bdef72329cc0e7d92afca48d94ae5fce6777cb36bfabb60b8034ff08fd88
