708586c77e
5f9c0b6360215636cfa62a70d3a70f1feb3977ab wallet: Remove -upgradewallet from dummywallet (MarcoFalke) a314271f08215feba53ead27096ac7fda34acb3c test: Remove unused wallet.dat (MarcoFalke) bf7635963c03203e7189ddaa56c6b086a0108cbf tests: Test specific upgradewallet scenarios and that upgrades work (Andrew Chow) 4b418a9decc3e855ee4b0bbf9e61121c8e9904e5 test: Add test_framework/bdb.py module for inspecting bdb files (Andrew Chow) 092fc434854f881330771a93a1280ac67b1d3549 tests: Add a sha256sum_file function to util (Andrew Chow) 0bd995aa19be65b0dd23df1df571c71428c2bc32 wallet: upgrade the CHDChain version number when upgrading to split hd (Andrew Chow) 8e32e1c41c995e832e643f605d35a7aa112837e6 wallet: remove nWalletMaxVersion (Andrew Chow) bd7398cc6258c258e9f4411c50630ec4a552341b wallet: have ScriptPubKeyMan::Upgrade check against the new version (Andrew Chow) 5f720544f34dedf75b063b962845fa8eca604514 wallet: Add GetClosestWalletFeature function (Andrew Chow) 842ae3842df489f1b8d68e67a234788966218184 wallet: Add utility method for CanSupportFeature (Andrew Chow) Pull request description: This PR cleans up the wallet upgrade mechanism a bit, fixes some probably bugs, and adds more test cases. The `nWalletMaxVersion` member variable has been removed as it made `CanSupportFeature` unintuitive and was causing a couple of bugs. The reason this was introduced originally was to allow a wallet upgrade to only occur when the new feature is first used. While this makes sense for the old `-upgradewallet` option, for an RPC, this does not quite make sense. It's more intuitive for an upgrade to occur if possible if the `upgradewallet` RPC is used as that's an explicit request to upgrade a particular wallet to a newer version. `nWalletMaxVersion` was only relevant for upgrades to `FEATURE_WALLETCRYPT` and `FEATURE_COMPRPUBKEY` both of which are incredibly old features. So for such wallets, the behavior of `upgradewallet` will be that the feature is enabled immediately without the wallet needing to be encrypted at that time (note that `FEATURE_WALLETCRYPT` indicates support for encryption, not that the wallet is encrypted) or for a new key to be generated. `CanSupportFeature` would previously indicate whether we could upgrade to `nWalletMaxVersion` not just whether the current wallet version supported a feature. While this property was being used to determine whether we should upgrade to HD and HD chain split, it was also causing a few bugs. Determining whether we should upgrade to HD or HD chain split is resolved by passing into `ScriptPubKeyMan::Upgrade` the version we are upgrading to and checking against that. By removing `nWalletMaxVersion` we also fix a bug where you could upgrade to HD chain split without the pre-split keypool. `nWalletMaxVersion` was also the version that was being reported by `getwalletinfo` which meant that the version reported was not always consistent across restarts as it depended on whether `upgradewallet` was used. Additionally to make the wallet versions consistent with actually supported versions, instead of just setting the wallet version to whatever is given to `upgradewallet`, we normalize the version number to the closest supported version number. For example, if given 150000, we would store and report 139900. Another bug where CHDChain was not being upgraded to the version supporting HD chain split is also fixed by this PR. Lastly several more tests have been added. Some refactoring to the test was made to make these tests easier. These tests check specific upgrading scenarios, such as from non-HD (version 60000) to HD to pre-split keypool. Although not specifically related to `upgradewallet`, `UpgradeKeyMetadata` is now being tested too. Part of the new tests is checking that the wallet files are identical before and after failed upgrades. To facilitate this, a utility function `sha256sum_file` has been added. Another part of the tests is to examine the wallet file itself to ensure that the records in the wallet.dat file have been correctly modified. So a new `bdb.py` module has been added to deserialize the BDB db of the wallet.dat file. This format isn't explicitly documented anywhere, but the code and comments in BDB's source code in file `dbinc/db_page.h` describe it. This module just dumps all of the fields into a dict. ACKs for top commit: MarcoFalke: approach ACK 5f9c0b6360 laanwj: Code review ACK 5f9c0b6360215636cfa62a70d3a70f1feb3977ab jonatack: ACK 5f9c0b6360215636cfa62a70d3a70f1feb3977ab, approach seems fine, code review, only skimmed the test changes but they look well done, rebased on current master, debug built and verified the `wallet_upgradewallet.py` test runs green both before and after running `test/get_previous_releases.py -b v0.19.1 v0.18.1 v0.17.2 v0.16.3 v0.15.2` Tree-SHA512: 7c4ebf420850d596a586cb6dd7f2ef39c6477847d12d105fcd362abb07f2a8aa4f7afc5bfd36cbc8b8c72fcdd1de8d2d3f16ad8e8ba736b6f4f31f133fe5feba |
||
---|---|---|
.. | ||
functional | ||
fuzz | ||
lint | ||
sanitizer_suppressions | ||
util | ||
config.ini.in | ||
get_previous_releases.py | ||
README.md |
This directory contains integration tests that test dashd and its utilities in their entirety. It does not contain unit tests, which can be found in /src/test, /src/wallet/test, etc.
This directory contains the following sets of tests:
- functional which test the functionality of dashd and dash-qt by interacting with them through the RPC and P2P interfaces.
- util which tests the dash utilities, currently only dash-tx.
- lint which perform various static analysis checks.
The util tests are run as part of make check
target. The functional
tests and lint scripts can be run as explained in the sections below.
Running tests locally
Before tests can be run locally, Dash Core must be built. See the building instructions for help.
Functional tests
Dependencies and prerequisites
Many Dash specific tests require dash_hash. To install it:
- Clone the repo
git clone https://github.com/dashpay/dash_hash
- Install dash_hash
cd dash_hash && pip3 install -r requirements.txt .
The ZMQ functional test requires a python ZMQ library. To install it:
- on Unix, run
sudo apt-get install python3-zmq
- on mac OS, run
pip3 install pyzmq
On Windows the PYTHONUTF8
environment variable must be set to 1:
set PYTHONUTF8=1
Running the tests
Individual tests can be run by directly calling the test script, e.g.:
test/functional/wallet_hd.py
or can be run through the test_runner harness, eg:
test/functional/test_runner.py wallet_hd.py
You can run any combination (incl. duplicates) of tests by calling:
test/functional/test_runner.py <testname1> <testname2> <testname3> ...
Wildcard test names can be passed, if the paths are coherent and the test runner
is called from a bash
shell or similar that does the globbing. For example,
to run all the wallet tests:
test/functional/test_runner.py test/functional/wallet*
functional/test_runner.py functional/wallet* (called from the test/ directory)
test_runner.py wallet* (called from the test/functional/ directory)
but not
test/functional/test_runner.py wallet*
Combinations of wildcards can be passed:
test/functional/test_runner.py ./test/functional/tool* test/functional/mempool*
test_runner.py tool* mempool*
Run the regression test suite with:
test/functional/test_runner.py
Run all possible tests with
test/functional/test_runner.py --extended
In order to run backwards compatibility tests, download the previous node binaries:
test/get_previous_releases.py -b v19.3.0 v18.2.2 v0.17.0.3 v0.16.1.1 v0.15.0.0
By default, up to 4 tests will be run in parallel by test_runner. To specify
how many jobs to run, append --jobs=n
The individual tests and the test_runner harness have many command-line
options. Run test/functional/test_runner.py -h
to see them all.
Speed up test runs with a ramdisk
If you have available RAM on your system you can create a ramdisk to use as the cache
and tmp
directories for the functional tests in order to speed them up.
Speed-up amount varies on each system (and according to your ram speed and other variables), but a 2-3x speed-up is not uncommon.
To create a 4GB ramdisk on Linux at /mnt/tmp/
:
sudo mkdir -p /mnt/tmp
sudo mount -t tmpfs -o size=4g tmpfs /mnt/tmp/
Configure the size of the ramdisk using the size=
option.
The size of the ramdisk needed is relative to the number of concurrent jobs the test suite runs.
For example running the test suite with --jobs=100
might need a 16GB ramdisk, but running with --jobs=4
will only need a 4GB ramdisk.
To use, run the test suite specifying the ramdisk as the cachedir
and tmpdir
:
test/functional/test_runner.py --cachedir=/mnt/tmp/cache --tmpdir=/mnt/tmp
Once finished with the tests and the disk, and to free the ram, simply unmount the disk:
sudo umount /mnt/tmp
Troubleshooting and debugging test failures
Resource contention
The P2P and RPC ports used by the dashd nodes-under-test are chosen to make conflicts with other processes unlikely. However, if there is another dashd process running on the system (perhaps from a previous test which hasn't successfully killed all its dashd nodes), then there may be a port conflict which will cause the test to fail. It is recommended that you run the tests on a system where no other dashd processes are running.
On linux, the test framework will warn if there is another dashd process running when the tests are started.
If there are zombie dashd processes after test failure, you can kill them by running the following commands. Note that these commands will kill all dashd processes running on the system, so should not be used if any non-test dashd processes are being run.
killall dashd
or
pkill -9 dashd
Data directory cache
A pre-mined blockchain with 200 blocks is generated the first time a functional test is run and is stored in test/cache. This speeds up test startup times since new blockchains don't need to be generated for each test. However, the cache may get into a bad state, in which case tests will fail. If this happens, remove the cache directory (and make sure dashd processes are stopped as above):
rm -rf test/cache
killall dashd
Test logging
The tests contain logging at five different levels (DEBUG, INFO, WARNING, ERROR
and CRITICAL). From within your functional tests you can log to these different
levels using the logger included in the test_framework, e.g.
self.log.debug(object)
. By default:
- when run through the test_runner harness, all logs are written to
test_framework.log
and no logs are output to the console. - when run directly, all logs are written to
test_framework.log
and INFO level and above are output to the console. - when run by our CI (Continuous Integration), no logs are output to the console. However, if a test
fails, the
test_framework.log
and dashddebug.log
s will all be dumped to the console to help troubleshooting.
These log files can be located under the test data directory (which is always printed in the first line of test output):
<test data directory>/test_framework.log
<test data directory>/node<node number>/regtest/debug.log
.
The node number identifies the relevant test node, starting from node0
, which
corresponds to its position in the nodes list of the specific test,
e.g. self.nodes[0]
.
To change the level of logs output to the console, use the -l
command line
argument.
test_framework.log
and dashd debug.log
s can be combined into a single
aggregate log by running the combine_logs.py
script. The output can be plain
text, colorized text or html. For example:
test/functional/combine_logs.py -c <test data directory> | less -r
will pipe the colorized logs from the test into less.
Use --tracerpc
to trace out all the RPC calls and responses to the console. For
some tests (eg any that use submitblock
to submit a full block over RPC),
this can result in a lot of screen output.
By default, the test data directory will be deleted after a successful run.
Use --nocleanup
to leave the test data directory intact. The test data
directory is never deleted after a failed test.
Attaching a debugger
A python debugger can be attached to tests at any point. Just add the line:
import pdb; pdb.set_trace()
anywhere in the test. You will then be able to inspect variables, as well as call methods that interact with the dashd nodes-under-test.
If further introspection of the dashd instances themselves becomes
necessary, this can be accomplished by first setting a pdb breakpoint
at an appropriate location, running the test to that point, then using
gdb
(or lldb
on macOS) to attach to the process and debug.
For instance, to attach to self.node[1]
during a run you can get
the pid of the node within pdb
.
(pdb) self.node[1].process.pid
Alternatively, you can find the pid by inspecting the temp folder for the specific test you are running. The path to that folder is printed at the beginning of every test run:
2017-06-27 14:13:56.686000 TestFramework (INFO): Initializing test directory /tmp/user/1000/testo9vsdjo3
Use the path to find the pid file in the temp folder:
cat /tmp/user/1000/testo9vsdjo3/node1/regtest/dashd.pid
Then you can use the pid to start gdb
:
gdb /home/example/dashd <pid>
Note: gdb attach step may require ptrace_scope to be modified, or sudo
preceding the gdb
.
See this link for considerations: https://www.kernel.org/doc/Documentation/security/Yama.txt
Often while debugging rpc calls from functional tests, the test might reach timeout before
process can return a response. Use --timeout-factor 0
to disable all rpc timeouts for that partcular
functional test. Ex: test/functional/wallet_hd.py --timeout-factor 0
.
Profiling
An easy way to profile node performance during functional tests is provided
for Linux platforms using perf
.
Perf will sample the running node and will generate profile data in the node's
datadir. The profile data can then be presented using perf report
or a graphical
tool like hotspot.
To generate a profile during test suite runs, use the --perf
flag.
To see render the output to text, run
perf report -i /path/to/datadir/send-big-msgs.perf.data.xxxx --stdio | c++filt | less
For ways to generate more granular profiles, see the README in test/functional.
Util tests
Util tests can be run locally by running test/util/bitcoin-util-test.py
.
Use the -v
option for verbose output.
Lint tests
Dependencies
Lint test | Dependency | Version used by CI | Installation |
---|---|---|---|
lint-python.sh |
flake8 | 3.8.3 | pip3 install flake8==3.8.3 |
lint-python.sh |
mypy | 0.781 | pip3 install mypy==0.781 |
lint-shell.sh |
ShellCheck | 0.7.2 | details... |
lint-shell.sh |
yq | default | pip3 install yq |
lint-spelling.sh |
codespell | 2.0.0 | pip3 install codespell==2.0.0 |
Please be aware that on Linux distributions all dependencies are usually available as packages, but could be outdated.
Running the tests
Individual tests can be run by directly calling the test script, e.g.:
test/lint/lint-filenames.sh
You can run all the shell-based lint tests by running:
test/lint/lint-all.sh
Writing functional tests
You are encouraged to write functional tests for new or existing features. Further information about the functional test framework and individual tests is found in test/functional.