fa4bcaf82a travis: Compile once on xenial (MarcoFalke)
Pull request description:
Currently we only build on bionic (since that is also the current gitian environment). However, building on the current and previous Ubuntu LTS should be supported with only system packages and without depends.
Tree-SHA512: bf5725cfb1be09220510d53010c7b7deb20051a9995e39fe5e83505c63db09ac877a41b896c97b253052fefea58ca0a9b6d9c5962a7ac4b258782c476d6ee7c0
3387bb0829 travis: avoid timeout without saving caches, also enable all qt (Chun Kuan Lee)
Pull request description:
- If depends build take more than 20 mins, skip Bitcoin Core build to store depends caches and mark it fail. Then restart the job for Bitcoin Core build.
- Enable Qt build for Windows and 32-bit Linux
- Enable wallet for depends x86-64 Linux
- Disable gui tests for Windows since they are not supported
This would be helpful for upgrading Qt (#12971) and protobuf (#13513)
Tree-SHA512: e943cbd848d90f9f70e29c94ed717f96ad2c2d27b433bafea762015756a2d2794fc28976c54aee087bf0f3726ac2c9140920272445a902038719b956e2160cf9
414326952c use export LC_ALL=C.UTF-8 (Julian Fleischer)
728c82d029 make script exit if a command fails (Julian Fleischer)
506890b24d move remaining travis build steps into individual files (Julian Fleischer)
272306ea57 number .travis/ script according to build lifecycle and add README to explain (Julian Fleischer)
519e2739cf move lint stage up to resemble travis build ui (Julian Fleischer)
86d34f0e65 abort script in END_FOLD on non-zero exit code (Julian Fleischer)
4f2f88c7b0 move script sections info individual files and comply with shellcheck (Julian Fleischer)
Pull request description:
This PR is extracted from https://github.com/bitcoin/bitcoin/pull/13816 to make that one easier to review. It follows on https://github.com/bitcoin/bitcoin/pull/13849 and https://github.com/bitcoin/bitcoin/pull/13851
In here the shell script parts from `travis.yml` are extracted into `.travis/before_install.sh`, `.travis/install.sh`, `.travis/before_script.sh`, `.travis/script.sh`, and `.travis/lint.sh`.
This has the benefit that `test/lint/lint-shell.sh` will also shellcheck these parts. Also it makes the individual script parts more readable.
Tree-SHA512: c497e1687ceb1c1d795de177d3fc35af908bc8e3f781a871afabdecf031e581d4db229290627249e35ef7c09952bc34884e4734ea91d40f57b4a9efb85bba2e3
e5b2cd8e7564b9fc2ed4f63fe49efb0af60b4460 Use python instead of slow shell script on verify-commits (Chun Kuan Lee)
Pull request description:
The cron job that runs every day would fail because of git checkout a single commit, not a branch.
#12708 introduce a method to check whether merges are clean.
However, there are four merges are not clean.
So, I add a list of merges that are dirty and ignore them.
Also, I modify the current shell script to python, it makes the script speed up a lot.
The python code `tree_sha512sum` was copied from `github-merge.py`
I've re-designed this. Now we verify all the things by default.
- Add `--disable-tree-check` option, not to check SHA-512 tree
- Add `--clean-merge NUMBER` option, only verify commits after <NUMBER> days ago
Travis running time:
|option|time|
|-|-|
|verify-commits.py|[25m47.02s(1547.02s)](https://travis-ci.org/ken2812221/bitcoin/jobs/373321423)|
|verify-commits.py --disable-tree-check|[19m10.08s(1150.08s)](https://travis-ci.org/ken2812221/bitcoin/jobs/373321423)|
|verify-commits.py --clean-merge 30|[9m18.18s(558.18s)](https://travis-ci.org/ken2812221/bitcoin/jobs/373321423)|
|verify-commits.py --disable-tree-check --clean-merge 30|[1m16.51s(76.51s)](https://travis-ci.org/ken2812221/bitcoin/jobs/373321423)|
Since the cron job always fail, I've created a respository to verify this daily.
[](https://travis-ci.org/ken2812221/bitcoin-verify-commits)
Tree-SHA512: 476bcf707d92ed3d431ca5642e013036df1506120d3dd2aa718f74240063ce856abd78f4c948336c2a6230dfe5c60c6f2d52d19bdb52d647a1c5f838eaa02e3b
For some reason Travis doesn't seem to like having the same name on different stages and ignores the cache that was built earlier. Making job names stage specific fixes cache issues.
Currently Travis displays env vars which make it hard to identify individual jobs by simply looking at the list because all env vars start with the same strings.
fa3c910bfeab00703c947c5200a64c21225b50ef test: Move linters to test/lint, add readme (MarcoFalke)
Pull request description:
This moves the checks and linters from `devtools` to a subfolder in `test`. (Motivated by my opinion that the dev tools are mostly for generating code and updating the repo whereas the linters are read-only checks.)
Also, adds a readme to clarify that checks and linters are only meant to prevent bugs and user facing issues, not merely stylistic preference or inconsistencies. (This is motivated by the diversity in developers and work flows as well as existing code styles. It would be too disruptive to change all existing code to a single style or too burdensome to force all developers to adhere to a single style. Also note that our style guide is changing, so locking in at the wrong style "too early" would only waste resources.)
Tree-SHA512: 9b10e89f2aeaf0c8a9ae248aa891d74e0abf0569f8e5dfd266446efa8bfaf19f0ea0980abf0b0b22f0d8416ee90d7435d21a9f9285b66df43f370b7979173406
* Merge #11796: [tests] Functional test naming convention
5fecd84 [tests] Remove redundant import in blocktools.py test (Anthony Towns)
9b20bb4 [tests] Check tests conform to naming convention (Anthony Towns)
7250b4e [tests] README.md nit fixes (Anthony Towns)
82b2712 [tests] move witness util functions to blocktools.py (John Newbery)
1e10854 [tests] [docs] update README for new test naming scheme (John Newbery)
Pull request description:
Splitting #11774 into two parts -- this part updates the README with the proposed naming convention, and adds some checks to test_runner.py that the number of tests violating the naming convention doesn't increase too much. Idea is this part of the change should not introduce merge conflicts or require much rebasing, so reviews of the complicated bits won't become invalidated too often; while the second part will just be file renames, which will require regular rebasing and will introduce merge conflicts with pending PRs, but can be merged later, and should also be much easier to review, since it will only include relatively trivial changes.
Tree-SHA512: b96557d41714addbbfe2aed62fb5a48639eaeb1eb3aba30ac1b3a86bb3cb8d796c6247f9c414c4695c4bf54c0ec9968ac88e2f88fb62483bc1a2f89368f7fc80
* update violation count
Signed-off-by: pasta <pasta@dashboost.org>
* Merge #11774: [tests] Rename functional tests
6f881cc880 [tests] Remove EXPECTED_VIOLATION_COUNT (Anthony Towns)
3150b3fea7 [tests] Rename misc functional tests. (Anthony Towns)
81b79f2c39 [tests] Rename rpc_* functional tests. (Anthony Towns)
61b8f7f273 [tests] Rename p2p_* functional tests. (Anthony Towns)
90600bc7db [tests] Rename wallet_* functional tests. (Anthony Towns)
ca6523d0c8 [tests] Rename feature_* functional tests. (Anthony Towns)
Pull request description:
This PR changes the functional tests to have a consistent naming scheme:
tests for individual RPC methods are named rpc_...
tests for interfaces (REST, ZMQ, RPC features) are named interface_...
tests that explicitly test the p2p interface are named p2p_...
tests for wallet features are named wallet_...
tests for mining features are named mining_...
tests for mempool behaviour are named mempool_...
tests for full features that aren't wallet/mining/mempool are named feature_...
Rationale: it's sometimes difficult for new contributors to know what's already covered by existing tests and where new tests should be added. Naming in a consistent fashion makes it easier to see what's already covered at a glance.
Tree-SHA512: 4246790552d42bbd95f6d5bdf67702b81b3b2c583ce7eaf1fe6d8e254721279b47315973c6e9ae82dad6e4c747f12188160764bf2624c0f8f3b4d39330ec8b16
* rename tests and edit associated strings to align test-suite with test name standards
Signed-off-by: pasta <pasta@dashboost.org>
* fix grammar in test/functional/test_runner.py
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
* ci: Fix excluded test names
* rename feature_privatesend.py to rpc_privatesend.py
Signed-off-by: pasta <pasta@dashboost.org>
Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: dustinface <35775977+xdustinface@users.noreply.github.com>
Co-authored-by: xdustinface <xdustinfacex@gmail.com>
* Remove unused jenkins stuff
* Install all dependencies in builder image
Instead of only target specific dependencies.
* Use docker builder image for builds
* Optimize apt installations
* Move building of dependencies into separate stage
The build-depends-xxx jobs will create artifacts (depends/$HOST) which are
then pulled in by the build jobs with the help of "needs"
* Remove use of caches from develop branch
* Use gitlab specific extends instead of YAML anchors
* Move before_script of build_template into base_template
* Add hack for parallel installation of i686 and arm cross compilation
* Install python3-setuptools in builder image
* Remove unnecessary change-dir
* Use variables to pass BUILD_TARGET instead of relying on the job name
* Move integration tests into separate stage
* Don't use --quiet for integration tests on Gitlab
* Add .gitlab-ci.yml
* Use | instead of > for multiline commands
This honor new-lines and makes ; unnecessary
* Use ubuntu:bionic as base image
* Move cache initialization before apt-get installs
* Cache apt packages
* Move installation of wget and unzip up as we need it for the cache
* Prevent apt from deleting caches
* Collect test logs into artifact
* Make combine_logs.py always look for the template in the correct dir
* Move final cache stuff into after_script
* Reintroduce PYTHON_DEBUG=1, but only for .travis.yml
* Install jinja2 in Travis builder image
* Enable ChainLocks after quorums have been created
Creating 4 quorums causes a lot of blocks to be created and signed by
ChainLocks, which then causes timeouts later.
* Increase timeout in wallet-dump.py test
The first dumpwallet is quite slow sometimes, which then makes the
later called dumpwallet throw a wallet locked exception.
3d1c311 Revert "travis: filter out pyenv" (Cory Fields)
a86e81b travis: move back to the minimal image (Cory Fields)
Pull request description:
The most recent update replaced the minimal image with a large one for the
'generic' image. Switching back to 'minimal' should reduce dependencies and
maybe speed us up some.
It should also eliminiate the need for aa2e0f09e.
Tree-SHA512: 0e5f3e97e8d97add07ea228bc5ce1e51e8e069950dbb2871a7eece297995f20b671afdf1c68211ce404cba3ba393d61dfef30ed54d46d6805fde9388f6b4455e
* Retry downloading of depends sources 3 times
* Manually invoke apt-get update and install instead of using addon
This allows us to use travis_retry on these operations.
* Add exception for depends/Makefile in .gitignore
* Remove --retry from curl call
This avoids future confusion as we don't rely on curl anymore to properly
retry downloads. curls does only retry on transient errors, while Travis
might cause all kinds of errors which circumvent the retry logic of curl.
This allows daily testing of extended tests. The --exclude also contains
"dbcrash", which we haven't backported yet, but I decided to leave it in
when I copied the code from Bitcoin, so that we later don't forget to add
it.
* Manually pull builder base image and let travis retry it on failure
* Split base package installation in Dockerfile.builder into multiple RUN lines
This allows better local cache usage on failure and retry.
* Use travis_retry for docker build
* Merge #9744: Remove unused module from rpc-tests
a432aa0 Remove unused module from rpc-tests (Takashi Mitsuta)
* Merge #9696: [trivial] Fix recently introduced typos in comments
0c9b9b7 [trivial] Fix recently introduced typos in comments (practicalswift)
* Merge #9657: Improve rpc-tests.py
a6a3e58 Various review markups for rpc-tests.py improvements (John Newbery)
3de3ccd Refactor rpc-tests.py (John Newbery)
afd38e7 Improve rpc-tests.py arguments (John Newbery)
91bffff Use argparse in rpc_tests.py (John Newbery)
1581ecb Use configparser in rpc-tests.py (John Newbery)
* Merge #9724: Qt/Intro: Add explanation of IBD process
f6d18f5 Qt/Intro: Explain a bit more what will happen first time (Luke Dashjr)
50c5657 Qt/Intro: Storage shouldn't grow significantly with pruning enabled (Luke Dashjr)
9adb694 Qt/Intro: Move sizeWarningLabel text into C++ code (Luke Dashjr)
* Merge #9794: Minor update to qrencode package builder
1bfe6b4 Use package name variable inside $(package)_file_name variable (Mitchell Cash)
* Merge #9726: netbase: Do not print an error on connection timeouts through proxy
3ddfe29 netbase: Do not print an error on connection timeouts through proxy (Wladimir J. van der Laan)
13f6085 netbase: Make InterruptibleRecv return an error code instead of bool (Wladimir J. van der Laan)
* Merge #9727: Remove fallbacks for boost_filesystem < v3
056aba2 Remove fallbacks for boost_filesystem < v3 (Wladimir J. van der Laan)
* Merge #9485: ZMQ example using python3 and asyncio
b471daf Adddress nits, use asyncio signal handling, create_task (Bob McElrath)
4bb7d1b Add python version checks and 3.4 example (Bob McElrath)
5406d51 Rewrite to not use Polling wrapper for asyncio, link to python2.7 example (Bob McElrath)
5ea5368 ZMQ example using python3 and asyncio (Bob McElrath)
* Merge #9807: RPC doc fix-ups.
851f6a3 [qa][doc] Correct rpc test options in readme (fanquake)
41e7219 [trivial] Add tests_config.ini to .gitignore (fanquake)
* Dashify
Co-Authored-By: PastaPastaPasta <pasta@dashboost.org>
* Change file permissions
* update travis.yml -parallel -> --jobs
* GCC-7 and glibc-2.27 compat code
* Statically link libstdc++ for GCC based builds
Makes sure binaries which are built on a newer build host still work
on older distros.
* Use python3 when installing MacOS native tools
* Move actual build logic out of Travis and upgrade to gcc-7
Travis will now simply call a few scripts which do the actual work.
These scripts will first create a "builder image" which contains the
necessary environment for the actual build. Then scripts are called
inside this builder image to do the build.
This should make us more independant from Travis and also allows us
to do local CI testing.
The build matrix is also moved out of .travis.yml and instead moved
into ci/matrix.sh. This script is sourced with only "BUILD_TARGET" being
set so that it internally can figure out which other environment
variables need to be set.
This commit also upgrades the used GCC version to 7. This is due to the
use of ubuntu:bionic as base image for the builder image.
* Add Jenkinsfiles for regular CI and nightly gitian builds
* Automatically download OSX SDK in gitian-build.sh
* Remove bogus "export MAKEJOBS=-j5"
* Forward cache/src dirs into builder container
Fixes caching issues on Travis.
* fix
* Fail build immediately when building depends took too long
* Automatically build and push docker image to dashpay/dashd-develop
* Remove DOCKER_REPO env var from .travis.yml and add new-line to .gitignore
This should not have been there and results in errournous pushes to
"dashpay/dashd/dashpay/dashd-develop".
Also add missing new-line in .gitignore
* Fix Travis build failures happening due to large writes to stdout
Travis issue: https://github.com/travis-ci/travis-ci/issues/4704
Build failure happened since the introduction of secrets in .travis.yml
* Fix BUILD_DIR
Due to changes in develop the build dir has changed to be just "build".
* Remove bogus upload example from .travis.yml
* Switch to pushing on "develop" branch
Used "pr_docker" before for testing purposes.
* Only push docker image when TRAVIS_PULL_REQUEST = false
* Revert "Adjust tests to avoid Travis timeouts (#1745)"
This reverts commit 32f21698e8.
* Revert "Show test progress for tests running in wine to avoid Travis timeout (#1740)"
This reverts commit 31bc9d4ee8.
* Use travis_wait when calling "wine test_dash.exe"
* Allow compilation with `--disable-wallet`
* fix remaining references
* Drop wallet references/include in CActiveMasternode and fix other files affected by this change
* Wrap privatesend-client.h include with ifdef/endif and fix other files affected by this change
* Re-enable Travis build with no wallet
reverts 267e57877b
- add some stdout spam to heavy tests (versionbits_tests, coins_tests and PrevectorTests)
- replace `--show_progress` with `--log_level=test_suite` to print these messages and avoid timeout
589827975 scripted-diff: various renames for per-utxo consistency (Pieter Wuille)
a5e02bc7f Increase travis unit test timeout (Pieter Wuille)
73de2c1ff Rename CCoinsCacheEntry::coins to coin (Pieter Wuille)
119e552f7 Merge CCoinsViewCache's GetOutputFor and AccessCoin (Pieter Wuille)
580b02309 [MOVEONLY] Move old CCoins class to txdb.cpp (Pieter Wuille)
8b25d2c0c Upgrade from per-tx database to per-txout (Pieter Wuille)
b2af357f3 Reduce reserved memory space for flushing (Pieter Wuille)
41aa5b79a Pack Coin more tightly (Pieter Wuille)
97072d668 Remove unused CCoins methods (Pieter Wuille)
ce23efaa5 Extend coins_tests (Pieter Wuille)
508307968 Switch CCoinsView and chainstate db from per-txid to per-txout (Pieter Wuille)
4ec0d9e79 Refactor GetUTXOStats in preparation for per-COutPoint iteration (Pieter Wuille)
13870b56f Replace CCoins-based CTxMemPool::pruneSpent with isSpent (Pieter Wuille)
05293f3cb Remove ModifyCoins/ModifyNewCoins (Pieter Wuille)
961e48397 Switch tests from ModifyCoins to AddCoin/SpendCoin (Pieter Wuille)
8b3868c1b Switch CScriptCheck to use Coin instead of CCoins (Pieter Wuille)
c87b957a3 Only pass things committed to by tx's witness hash to CScriptCheck (Matt Corallo)
f68cdfe92 Switch from per-tx to per-txout CCoinsViewCache methods in some places (Pieter Wuille)
000391132 Introduce new per-txout CCoinsViewCache functions (Pieter Wuille)
bd83111a0 Optimization: Coin&& to ApplyTxInUndo (Pieter Wuille)
cb2c7fdac Replace CTxInUndo with Coin (Pieter Wuille)
422634e2f Introduce Coin, a single unspent output (Pieter Wuille)
7d991b55d Store/allow tx metadata in all undo records (Pieter Wuille)
c3aa0c119 Report on-disk size in gettxoutsetinfo (Pieter Wuille)
d34242430 Remove/ignore tx version in utxo and undo (Pieter Wuille)
7e0032290 Add specialization of SipHash for 256 + 32 bit data (Pieter Wuille)
e484652fc Introduce CHashVerifier to hash read data (Pieter Wuille)
f54580e7e error() in disconnect for disk corruption, not inconsistency (Pieter Wuille)
e66dbde6d Add SizeEstimate to CDBBatch (Pieter Wuille)
Tree-SHA512: ce1fb1e40c77d38915cd02189fab7a8b125c7f44d425c85579d872c3bede3a437760997907c99d7b3017ced1c2de54b2ac7223d99d83a6658fe5ef61edef1de3
