e12e970df6fcae08ff8008812cdeef600d6b2db8 docs: match usage text to script and location (Peter Bushnell)
Pull request description:
Update the usage text in the README to match the usage text in the Python script.
02b26ba1c1/contrib/testgen/gen_key_io_test_vectors.py (L9)02b26ba1c1/contrib/testgen/gen_key_io_test_vectors.py (L10)
Also to match the file names in the actual destination.
02b26ba1c1/src/test/data/key_io_valid.json02b26ba1c1/src/test/data/key_io_invalid.json
Following the README usage text generates new files when the user is likely to have wanted to update the existing files.
ACKs for top commit:
fanquake:
ACK e12e970df6fcae08ff8008812cdeef600d6b2db8 - this looks correct.
Tree-SHA512: b7ab61e19a54597a8fbd1844b9cfaef78879e53b882eefe4e0140fa115674df7f061e468835186963b89c963244a17d922f2ad0829b10f62b84f02019ee33edb
47b49a05eafddcaef373f70436d794e9f9f7495c contrib: Fix SyntaxWarning in Python base58 implementation (Alex Willmer)
Pull request description:
In Python integers should be compared for equality (`i == j`), not identity (`i is j`). Recent versions of CPython 3.x emit a SyntaxWarning when they encounter this incorrect usage, e.g.
```
$ python3 base58.py
base58.py:110: SyntaxWarning: "is" with a literal. Did you mean "=="?
assert get_bcaddress_version('15VjRaDX9zpbA8LVnbrCAFzrVzN7ixHNsC') is 0
Tests passed
```
ACKs for top commit:
MarcoFalke:
ACK 47b49a05eafddcaef373f70436d794e9f9f7495c
Tree-SHA512: 9f8962025dcdfa062c0515c68a1864f5bbeb86bd0510c0ec0e413a5edb6afbfd5f41b4c0255784e53db8eaf39c68b7cfa7cc8a33a2e5214aae463fda374f8719
c966ff14c77870378847b9e6063b9671739ddc1f gitian: fixed SC2001 regex (willyk)
Pull request description:
Currently the gitian-win-signer.yml produces OUTFILE names without `-unsigned` stripped out
This is due to regex having an`%` in front of it
```
$ INFILE="bitcoin-0.19.0-win64-setup-unsigned.exe"
$ echo "${INFILE/%-unsigned}"
bitcoin-0.19.0-win64-setup-unsigned.exe
$ echo "${INFILE/-unsigned}"
bitcoin-0.19.0-win64-setup.exe
```
Fixes#17361
ACKs for top commit:
laanwj:
ACK c966ff14c77870378847b9e6063b9671739ddc1f
hebasto:
ACK c966ff14c77870378847b9e6063b9671739ddc1f
Tree-SHA512: 954547f9dfa4cab4def5f284d4837c21f0e6fed7454a04e83e6b1b7d3fd3f9661ea657047f0d8162f6591909d32ef2f72e801b2f3a44cbb1131ac344cb913a69
3284e6c09a84e9557ec72723ad636053d3ef7122 scripts: search for next position of magic bytes rather than fail (Tim Akinbo)
Pull request description:
When using the `linearize-data.py` contrib script to export block data, there are edge cases where the script fails with an `Invalid magic: 00000000` error. This error occurs due to the presence of padding bytes that occasionally appears between consecutive blocks in the block data file.
There's an ongoing conversation about this in #14986. sipa also admitted that it is a bug in #5028. Fortunately, this is not an issue in bitcoin core as it handles this type of situation gracefully and so no fix in bitcoin core is required.
This PR is an improvement on how the script handles these "invalid magic bytes". Rather than failing, this patch allows the script to search for the next occurrence of the magic bytes and then starts reading the block from there.
ACKs for top commit:
laanwj:
ACK 3284e6c09a84e9557ec72723ad636053d3ef7122
Tree-SHA512: 18067ae0b4b62e822dfc558a86439ad6acaf939b98479e38e8e4248536574643b26eb48e96ec7139375c88b42cbe7705a64deb13a3c239e16025a6aad3d69bfa
bd6a24307564a56d54d98331d5e1680d27ddc09e script: Add Keyserver to verify-commits README (Emil Engler)
Pull request description:
When I use the option with the default keyserver on `gpg (GnuPG) 2.2.12` from the Debian repositories only the keys from meshcollider and fanquake are actually found. Using the ubuntu keyserver works without any problems and all keys are getting found.
As this keyserver is also suggested on [https://bitcoincore.org/en/download/](), it would be good to have a common keyserver.
ACKs for top commit:
laanwj:
ACK bd6a24307564a56d54d98331d5e1680d27ddc09e, better to be explicit here
Tree-SHA512: d4127c42490390bed4e0074ebb1f8a6cc983f96bbea0ec1da011a93bed325a7f465ed9936e4bc2ef7c4b2e4501a2868d00b67ceb7bf85f2a902db9c6173a6c61
eafd2593674b6890418a50eadf69425071b824e4 build: Add NX workaround for RV64 (Wladimir J. van der Laan)
f6e42256fe8900ed90f9644906b77f97e1600952 build: Allow export of environ symbols (Wladimir J. van der Laan)
Pull request description:
This export was introduced in #17270 which added
```
//! Necessary on some platforms
extern char** environ;
```
This should (finally) make the gitian build pass again (fix issue #17525.).
Built on top of #17538 which should be merged first.
Top commit has no ACKs.
Tree-SHA512: 5c2054d52d0957aec3dc945b76d8e219187d22dc03889e7a88fb76049bf8e4a3e9f4da00dd1e9dd0351211f8e70d1a1b8ad7244f0348dab698e9d14b9d0c0bd4
0e01e4522e devtools: Fetch and display ACKs at sign-off time in github-merge (Wladimir J. van der Laan)
Pull request description:
- Fetch the ACKs only at sign-off time. This makes sure that any last-minute ACKs are included (fixes#16200)
- Show a list of ACKs that will be included and their author before signing off, and warn if there are none
There's a slight change to the merge commit format—before it was
```
ACKs for commit 88884c:
(list of ACKs, could be empty)
```
now it is
```
ACKs for top commit:
jnewbery:
ACK 5ebc6b0eb
... (list of ACKs cannot be empty)
```
or
```
Top commit has no ACKs.
```
I don't think there's a reason to have the abbreviated commit ID there, after all the full commit id is already in the beginning of the merge commit message, and at least the abbreviated one is in every single ACK message.
ACKs for commit 0e01e4:
fanquake:
ACK 0e01e4522e0f9e2e994f80fc61f40d61a4a70d79
Tree-SHA512: 8576de016137d71cfc101747e9bb6779c13e0953cf2babee7afc9972bf2bd46f6912be4982b54fa5abf4d91e98e8fdae6b4ca3eef7d6892b7a5f04a7017b6882
942ff2054b41fe3f78f1b3d88cfd032bc95fd62f contrib: gh-merge: Use pagination to fetch all review comments (nkostoulas)
Pull request description:
Use GitHub API pagination to do multiple requests if required.
Tested with some PRs that have a large number of comments.
For issue #15816
ACKs for commit 942ff2:
MarcoFalke:
utACK 942ff2054b41fe3f78f1b3d88cfd032bc95fd62f
laanwj:
utACK 942ff2054b41fe3f78f1b3d88cfd032bc95fd62f
Tree-SHA512: cf0dcdc40212a7c5cfcd1afe873e068399bc6499fd2e32207c5516595c93e5cb478178b04185ce21c3de0312621e05fda268411e59864d51046fb7a049989f42
8b8d8eeae9 Remove travis_wait from lint script (Graham Krizek)
Pull request description:
Using the `travis_wait` command in conjunction with `set -o errexit` causes problems. The `travis_wait` command will correctly log the command's output if successful, but if the command fails the process exits before the `travis_wait` command can dump the logs. This will hide important debugging information like error messages and stack traces. We ran into this in #15196 and it was very hard to debug because output was being suppressed.
`travis_wait` was being used because the `contrib/verify-commits/verify-commits.py` script can sometimes run for a long time without producing any output. If a script runs for 10 minutes without logging anything, the CI run times out. The `travis_wait` command will extend this timeout by logging a message for you, while sending stderr and stdout to a file.
This PR removes the `travis_wait` command from our CI system and adds additional logging to the `verify-commits.py` script so it doesn't make Travis timeout.
ACKs for commit 8b8d8e:
MarcoFalke:
utACK 8b8d8eeae9e8feff6d78420ee172c820ccef9db1
Tree-SHA512: 175a8dd3f4d4e03ab272ddba94fa8bb06875c9027c3f3f81577feda4bc8918b5f0e003a19027f04f8cf2d0b56c68633716a6ab23f95b910121a8d1132428767d
fa1c073154c6a39dca878f5c9a37abee8af0fd30 contrib: gh-merge: Include review comments in merge commit (MarcoFalke)
Pull request description:
This includes all up-to-date ACKs in the merge commit for reference
Tree-SHA512: 32c9352d884f9ecf94940f50f2921fc9fc026083c120f54d0651a41814872e852aee8d0c4ad5bcd03292329f05d76fcb7bac11741e1dd3bf417211a186005afb
bad1716c6d30fdf4be6d5050a04e1211f920bbd6 init: Modify docs and add release note for 12255 (Carl Dong)
b0c7b54d0c2e116d61e686b1adfdea6a1f7f02fe init: Use systemd automatic directory creation (Carl Dong)
Pull request description:
- `-datadir` option specified.
- Ask systemd to create and set the right mode for PID directory, configuration directory, and data directory.
- Tell systemd our group so it will set the right owner for aforementioned directories.
More information: https://www.freedesktop.org/software/systemd/man/systemd.exec.html
Tree-SHA512: a6fad1efa2be433c1fdd863df3ff232736ed709a9e281f51a003b40987d8c213dc64a52bc13a19c85bf85680e78f0be112ecaf32ac274b1ff93bac84a1208845
* Merge #14060: ZMQ: add options to configure outbound message high water mark, aka SNDHWM
a4edb168b635b6f5c36324e44961cd42cf9bbbaa ZMQ: add options to configure outbound message high water mark, aka SNDHWM (mruddy)
Pull request description:
ZMQ: add options to configure outbound message high water mark, aka SNDHWM
This is my attempt at https://github.com/bitcoin/bitcoin/pull/13315
Tree-SHA512: a4cc3bcf179776899261a97c8c4f31f35d1d8950fd71a09a79c5c064879b38e600b26824c89c4091d941502ed5b0255390882f7d44baf9e6dc49d685a86e8edb
* High watermark settings for Dash-specific messages
Signed-off-by: Dzutte <dzutte.tomsk@gmail.com>
Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
851380ce17 remove deprecated mentions of signrawtransaction from fundraw help (Gregory Sanders)
Pull request description:
RPC call has been removed as of 0.17.99.
Tree-SHA512: a6a12a0e4572acd9b532c1719be85ed6f29d1c1a28f9ce691398528b8dde4fb4a3222b8f68632fcb1a8eddfe2d31e96d5efd5bc51c041af8e7cb99b61ca3a167
166266a372fed2b1f73053084a923fab7edc77f4 script: Make LXC container size suitable for gitian builds (Hennadii Stepanov)
Pull request description:
This change prevents "No space left on device" error.
See:
- https://github.com/bitcoin/bitcoin/pull/21036#issuecomment-774771873
- https://github.com/bitcoin/bitcoin/pull/21036#issuecomment-775031315
ACKs for top commit:
jonasschnelli:
ACK 166266a372fed2b1f73053084a923fab7edc77f4 - I had to add this manually to my nighly build base image
Tree-SHA512: 47d84c3a65f0a17013b2cb970c34bfa4e600e83066be302ff10280aefefa0a7c6cb6c21a191b3e8e6fcd1c292d1c434cc4769e04626c4536050aced29b34d573
624091b7b97ed371900ca7c68f3e2929954e3464 Fix docker args conditional (setpill)
Pull request description:
The conditional that checks if docker needs to be installed has the side effect of triggering the default `lxc` branch in case docker comes preinstalled. This is clearly not intentional.
ACKs for top commit:
laanwj:
Code review ACK 624091b7b97ed371900ca7c68f3e2929954e3464
theStack:
Code review ACK 624091b7b97ed371900ca7c68f3e2929954e3464
Tree-SHA512: e37e2c35aaed813762223e5963e5416d5865b3fb53efb2aac86daaa03b95ccf07db9c3a779446029d055ab89491147c4d900117273e22caed201b21bdf287c58
97dce72261 contrib: add curl as a required program in gitian-build.py (fanquake)
Pull request description:
Fixes: #16109
Adds `curl` to the list of base programs required by the `gitian-build.py` script.
ACKs for commit 97dce7:
hebasto:
tACK 97dce7226194d5d8a06787afef7df08cebb66bd6 on Debian Buster RC1.
Tree-SHA512: 68847a527aa6b5d883bffd6a6fe6bbbe4b96ceddb30f55ed5ffbfa690a10c2e9c1bc7ba4520319531ab3baa7a7f64c3c8ce89a791f7c746abe73a84c2942b94d
0f22a0cf2f Fix gitian-build.py --verify option (Hennadii Stepanov)
4c56a798c0 Set/unset USE_LXC, USE_VBOX, USE_DOCKER explicitly (Hennadii Stepanov)
cbbd98863b Fix Docker related issues for gitian-build.py (Hennadii Stepanov)
Pull request description:
1. The Docker does not depend on `apt-cacher-ng` package. Ref: #14002.
2. Do not try to install the Docker if `docker.service` is detected on the system (e.g., the Docker was installed manually). Fix https://github.com/bitcoin/bitcoin/pull/13623#issuecomment-405684241 by **Sjors**.
3. Prevent the setting of more than one environment variable for the `gitian-builder` (an alternative to #13999). E.g., USE_LXC being set shadows USE_DOCKER; for details see [`gitian-builder/libexec/make-clean-vm`](93a62c7d7d/libexec/make-clean-vm (L7)):
```sh
VMSW=KVM
if [ -n "$USE_LXC" ]; then
VMSW=LXC
elif [ -n "$USE_VBOX" ]; then
VMSW=VBOX
elif [ -n "$USE_DOCKER" ]; then
VMSW=DOCKER
fi
```
4. The [`gitian-builder/bin/gverify`](https://github.com/devrandom/gitian-builder/blob/master/bin/gverify) script returns the exit code 1 if a signature verification ends with 'BAD SIGNATURE' or 'MISMATCH' by design. This PR allows to see the verification results for all signatures without a premature fail of the `gitian-build.py` script. Ref: #14014.
ACKs for commit 0f22a0:
Tree-SHA512: 55f8a5cffa20d0c745f51a687f3199cea015fa616e56a0aee4c25b5ca0985036c61e8cf1922515338d8c6a85f873674ebe7a9a56a5069d65a187e383150f1a83
fa193dc8e6f3b96fa2dba2f1c1668f7720fed320 doc: Remove win32 from the release process (MarcoFalke)
faf666f8148eeb305a9c4f78459aff2c7268016b Remove Windows 32 bit build (MarcoFalke)
Pull request description:
The Windows 32 bit build has been removed from https://bitcoincore.org/en/download/, so unless there are complaints, we don't need to build it even
ACKs for commit fa193d:
fanquake:
utACK fa193dc8e6
Tree-SHA512: d6f2976a2e0c407698f720b00ac23ec4056626de4eff8621f4c5581120af0460afd1bdef72329cc0e7d92afca48d94ae5fce6777cb36bfabb60b8034ff08fd88
beda0dae953aa125e08302713d2c7b214eddbe5a Upgrade gitian image before signing (Hennadii Stepanov)
Pull request description:
The package upgrade in the Ubuntu repositories in the period between the building and the signing causes (particularly, using LXC) an error:
```
Creating package manifest
Could not download some packages, please run gbuild --upgrade
```
For example, the [`busybox-initramfs`](https://packages.ubuntu.com/bionic/busybox-initramfs) package was [upgraded](http://changelogs.ubuntu.com/changelogs/pool/main/b/busybox/busybox_1.27.2-2ubuntu3.2/changelog) from `1:1.27.2-2ubuntu3.1` to `1:1.27.2-2ubuntu3.2` on 2019-03-06.
This PR forces gitian image upgrade for the `--sign` command.
Ref:
[devrandom/gitian-builder/target-bin/grab-packages.sh](04ab7c1218/target-bin/grab-packages.sh)
```
#!/bin/sh
# Get an installed package manifest
set -e
cd /var/cache/apt/archives
# make sure all packages with installed versions are downloaded
# (except for held packages, which may not be available for download)
dpkg-query -W -f '${Status}\t${Package}=${Version}\n' | grep -v ^hold | cut -f2- | xargs -n 50 apt-get install -q --reinstall -y -d > /tmp/download.log
grep "cannot be downloaded" /tmp/download.log && { echo Could not download some packages, please run gbuild --upgrade 1>&2 ; exit 1 ; }
sha256sum *.deb | sort --key 2
```
ACKs for commit beda0d:
laanwj:
utACK beda0dae953aa125e08302713d2c7b214eddbe5a
fanquake:
utACK beda0da
Tree-SHA512: e2e3b3e3719e098d266ceec39bd69b950344a4eb2f43ae6ad3e696add70f743b363cc83676e339f7caa207d6478029869a8af01fe1f6d5690d2857003f7d8ce8
5c04814b2d Move non-linux source tarball to bitcoin-binaries (Hennadii Stepanov)
Pull request description:
Currently, if a user makes a non-linux (`--os=w`, `--os=m` or `--os=wm`) gitian building with the `gitian-build.py` script, source tarballs are not moved to the `bitcoin-binaries/${VERSION}` directory.
This PR fixes this bug.
~~In addition, the `src` subdirectory in the `gitian-builder/build/out` directory is no longer used as unnecessary.~~
ACKs for commit 5c0481:
fanquake:
utACK 5c04814
ken2812221:
utACK 5c04814b2de179fd03ca18c19049f035a2454f81
Tree-SHA512: 8648b6cbf502c012b12642783870e37aea385bd5f4cba5cb577fee924c09685e9a117676be502e4d4783c7a8ab31a2bd495970eec42a42d78e86ac5d39323091
d813266db1f23f49465aa2aca3c3c80a95cf63d9 [gitian] use versioned unsigned tarballs instead of generically named ones (Andrew Chow)
Pull request description:
Instead of re-naming the tarballs used for the code signing step to the generically named tarball that is used, keep the versioned naming. Only copy them to the correct filename when they are needed at build time.
This makes it easier to handle situations when multiple different releases are being built simultaneously as the version that the code signatures are applied to will actually be the correct version and not require a rebuild to get the correct tarball.
Tree-SHA512: 434f721485521c6f7487038705a5d2e48ac2eb79cfad5f92a93b7ea9c91e1fb33702ce7aeed4d840c61ec43e808af5cc15a553b6e16e2ed1d7ac697485019204
f1bd219a5b318e4bea361e1247a233e4f251f517 contrib: Allow use of github API authentication in github-merge (Wladimir J. van der Laan)
a4c5bbfcd3a12f310b26cccc78ded32dd3f32ebb contrib: Add support for http[s] URLs in github-merge (Wladimir J. van der Laan)
059a3cffdfa596aa3adaace3f57fa86fdd3f80fc contrib: Detailed reporting for http errors in github-merge (Wladimir J. van der Laan)
Pull request description:
Three commits I had locally for `github-merge.py`:
- *Detailed reporting for http errors in github-merge*: Print detailed error, this makes it easier to diagnose github API issues.
- *Add support for http[s] URLs in github-merge*: Sometimes it can be useful to use github-merge with read-only access (say, for reviewing and testing from untrusted VMs).
- *Allow use of github API authentication in github-merge*: The API request limit for unauthenticated requests is quite low. I started running into rate limiting errors. The limit for authenticated requests is much higher. This patch adds an optional configuration setting `user.ghtoken` that, when set, is used to authenticate requests to the API.
Tree-SHA512: ca8ae1874a787263e49d915d7cf31c0c0f50aba229c9440265bf1fda69f7e00641d1492512b93d76c17ff1766859283d640d37770acb120898736ad97efbd5c2
ae5594d51b489826f2f2315c92e8df5ee0267efc [Trivial] Update license year range to 2019 (Emil Engler)
Pull request description:
Bitcoin Core is going to become 10 so the license year range also needs an update.
2019 is coming very soon so it is the time to update the year range like every year.
Same as #12063
Tree-SHA512: 64d8b5ad12c4a9a1afdeaac7c5b5d874cda66eb9b7040b58dc253b359d0090dadab033d2ead65aad331d9dc5c56bcf8066b6d09fd85de5bfc7de7309db16d155
688f665a5e526fda0fb797bf617412fe9cbe64fd Scripts and tools & Docs: Used #!/usr/bin/env bash instead of obsolete #!/bin/bash, added linting for .sh files shebang and updated the Developer Notes. (vim88)
Pull request description:
As it was discussed in [#13510](https://github.com/bitcoin/bitcoin/pull/13510), it is better to use `#!/usr/bin/env bash` instead of `#!/bin/bash`.
Tree-SHA512: 25f71eb9a6a0cdc91568b5c6863205c5fe095f77a69e633503a2ac7805bd9013af8538e538c0c666ce96a28e3f43ce7a8df5f08d4ff007723bb588d85674f2da
* build: detect the presence of libgmp before generating Makefile
* depends: add arximboldi/immer@v0.6.2 as a package and add detection
* depends: remove immer from source tree, build using package only
* Drop immer refs from tools
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
4de11a3682 Remove Python 2 import workarounds (practicalswift)
Pull request description:
Remove Python 2 import workarounds.
As noted by @jnewbery in https://github.com/bitcoin/bitcoin/pull/14903#discussion_r241396925:
> This exception handling is a vestige from when github-merge.py supported Python 2 and Python 3. We only support Python 3 now so we should be able to remove it entirely and just import from urllib.request.
Tree-SHA512: e0d21e6299dd62fb669ad95cbd3d19f7c803195fd336621aac72fd10ddc7431d90443831072a2e1eb2fc880d1d88eb7c3e2ead3da59f545f6db07d349af98fb3
fa4da3c058 [doc] conf: Remove deprecated options from docs, Other cleanup (MarcoFalke)
Pull request description:
Some dumb fixes, like removing the mention that free transactions are still a thing or that rpcuser/pass should be used (as opposed to rpcauth or rpc cookie).
Combined with other fixes because I don't want to create 3 pull requests:
* conf: Remove deprecated options from docs
* Remove only mention of MIT/X11
* Link to developer notes in README.md
Tree-SHA512: 9e45dc6c63037e7618cf3c871d7d9e65b66f1a952f91a6e623d97d90171e29bc40299a06029c4dc21a0f579e68021e3663186bd3a65e3ab333aff711f7dcb2bf
a786c3b30639a63ded5b3b81c393d56336d34dce Slight tweak to the verify-commits script directions (Douglas Roark)
Pull request description:
Clarify that GnuPG may be used on both Linux and macOS to obtain the keys required to verify the commits.
Tree-SHA512: cec556370f03e00bbd6f585d26b360ca236cf55cb5c0996f6d950d8a98f77c92cc02f1719c8f9b9dc9eac6900eb341a13b50a012752832f39095b7e84046f2cd
fdf82ba18 Update all subprocess.check_output functions in CI scripts to be Python 3.4 compatible (Graham Krizek)
Pull request description:
CI is failing the `lint` stage on every Cron run (regular PR/Push runs still pass). The failure was introduced in 74ce326 and has been broken since. The Python version running in CI was downgraded to 3.4 from 3.6. There were a couple files that were using the `encoding` argument in the `subprocess.check_output` function. This was introduced in Python 3.6 and therefore broke the scripts that were using it. The `universal_newlines` argument was used as well, but in order to use it we must be able to set encoding because of issues on some BSD systems.
To get CI to pass, I removed all `universal_newline` and `encoding` args to the `check_ouput` function. Then I decoded all `check_output` return values. This should keep the same behavior but be Python 3.4 compatible.
Tree-SHA512: f5e5885e98cf4777be9cc254446a873eedb03bdccbd8e06772a964db95e9fcf46736aa9cdcab1d8f123ea9f4947ed6020679898d8b2f47ffb1d94c21a4b08209
