2d4db48606
73f83036298e4932772704b2d5359c78d81bd7cc ci: Add pip cache (Hennadii Stepanov) 6d480667b24dc630e7c3fe7ff032cbe1a958ab77 ci: Print ccache statistics summary (Hennadii Stepanov) a6eea715ff549febe6d4bd52f17acb50abd99120 ci: Drop unused WINEDEBUG variable (Hennadii Stepanov) Pull request description: The Travis [pip cache](https://docs.travis-ci.com/user/caching/#pip-cache) is free and saves a dozen of seconds :) Here are some excerpts from the Travis logs with `ccache` statistics (I found useful): 2) [Arm64](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507749) ``` ccache version 3.6 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats updated Fri Apr 10 18:21:06 2020 stats zeroed Fri Apr 10 18:03:04 2020 cache hit (direct) 156 cache hit (preprocessed) 198 cache miss 143 cache hit rate 71.23 % called for link 8 cleanups performed 9 files in cache 1255 cache size 80.6 MB max cache size 100.0 MB ``` 3) [s390x](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507750) ``` ccache version 3.4.1 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats zero time Fri Apr 10 17:15:35 2020 cache hit (direct) 115 cache hit (preprocessed) 163 cache miss 94 cache hit rate 74.73 % called for link 6 cleanups performed 0 files in cache 804 cache size 304.8 MB max cache size 5.0 GB ``` 4) [Win64](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507751) ``` ccache version 3.4.1 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats zero time Fri Apr 10 17:38:07 2020 cache hit (direct) 147 cache hit (preprocessed) 199 cache miss 138 cache hit rate 71.49 % called for link 7 cleanups performed 7 files in cache 1242 cache size 87.0 MB max cache size 100.0 MB ``` 5) [CentOS 7](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507752) ``` ccache version 3.7.7 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats updated Fri Apr 10 17:45:59 2020 stats zeroed Fri Apr 10 17:34:27 2020 cache hit (direct) 152 cache hit (preprocessed) 200 cache miss 144 cache hit rate 70.97 % called for link 8 cleanups performed 4 files in cache 1352 cache size 86.1 MB max cache size 100.0 MB ``` 6) [bionic](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507753) ``` ccache version 3.4.1 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats zero time Fri Apr 10 17:21:10 2020 cache hit (direct) 136 cache hit (preprocessed) 35 cache miss 330 cache hit rate 34.13 % called for link 8 cleanups performed 18 files in cache 1302 cache size 83.8 MB max cache size 100.0 MB ``` 7) [xenial](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507754) ``` ccache version 3.2.4 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf cache hit (direct) 161 cache hit (preprocessed) 174 cache miss 95 called for link 7 files in cache 3411 cache size 1.1 GB max cache size 5.0 GB ``` 10) [focal](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507757) ``` ccache version 3.7.7 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats updated Fri Apr 10 17:35:57 2020 stats zeroed Fri Apr 10 17:21:32 2020 cache hit (direct) 78 cache hit (preprocessed) 125 cache miss 107 cache hit rate 65.48 % called for link 120 cleanups performed 0 files in cache 6218 cache size 1.8 GB max cache size 5.0 GB ``` 14) [macOS 10.12](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507761) ``` ccache version 3.4.1 cache directory /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats zero time Fri Apr 10 17:47:19 2020 cache hit (direct) 28 cache hit (preprocessed) 1 cache miss 469 cache hit rate 5.82 % called for link 8 cleanups performed 17 files in cache 1946 cache size 82.6 MB max cache size 100.0 MB ``` 15) [macOS 10.14 native](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507762) ``` ccache version 3.7.8 cache directory /Users/travis/build/bitcoin/bitcoin/ci/scratch/.ccache primary config /Users/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf secondary config (readonly) /usr/local/Cellar/ccache/3.7.8/etc/ccache.conf stats updated Fri Apr 10 17:38:44 2020 stats zeroed Fri Apr 10 17:22:36 2020 cache hit (direct) 213 cache hit (preprocessed) 293 cache miss 144 cache hit rate 77.85 % called for link 11 called for preprocessing 56 compile failed 30 preprocessor error 56 bad compiler arguments 14 autoconf compile/link 77 no input file 66 cleanups performed 0 files in cache 1564 cache size 284.6 MB max cache size 5.0 GB ``` Top commit has no ACKs. Tree-SHA512: f9f2c6d72680bafc7a982efc04dc37ab9bba77ffd2a4cbbc56bb0e86c5c3501d57aa2f0ebeac0113e9d8daac1ee4eac1eddf74c1172c267b37ee2d9324366c8a |
||
---|---|---|
.. | ||
dash | ||
lint | ||
retry | ||
test | ||
README.md | ||
test_run_all.sh |
ci scripts
This directory contains scripts for each build step in each build stage.
Currently three stages lint
, extended_lint
and test
are defined. Each stage has its own lifecycle, similar to the
Travis CI lifecycle. Every script in here is named
and numbered according to which stage and lifecycle step it belongs to.
Running a stage locally
Be aware that the tests will be built and run in-place, so please run at your own risk. If the repository is not a fresh git clone, you might have to clean files from previous builds or test runs first.
The ci needs to perform various sysadmin tasks such as installing packages or writing to the user's home directory. While most of the actions are done inside a docker container, this is not possible for all. Thus, cache directories, such as the depends cache or ccache, are mounted as read-write into the docker container. While it should be fine to run the ci system locally on you development box, the ci scripts can generally be assumed to have received less review and testing compared to other parts of the codebase. If you want to keep the work tree clean, you might want to run the ci system in a virtual machine with a Linux operating system of your choice.
To allow for a wide range of tested environments, but also ensure reproducibility to some extent, the test stage
requires docker
to be installed. To install all requirements on Ubuntu, run
sudo apt install docker.io bash
To run the default test stage,
./ci/test_run_all.sh
To run the test stage with a specific configuration,
FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh