b6718e373e tests: Use MakeUnique to construct objects owned by unique_ptrs (practicalswift)
Pull request description:
A subset of #14211 ("Use MakeUnique to construct objects owned by unique_ptrs") as suggested by @MarcoFalke in https://github.com/bitcoin/bitcoin/pull/14211#issuecomment-423324019.
Use `MakeUnique` to construct objects owned by `unique_ptr`s.
Rationale:
* `MakeUnique` ensures exception safety in complex expressions.
* `MakeUnique` gives a more concise statement of the construction.
Tree-SHA512: 1228ae6ce7beb178d79142c4e936b728178ccaa8aa35c6d8feeb33d1a667abfdd010c59996a9d833594611e913877ce5794e75953d11d9b1fdbac04aa491d9cf
65bc38d1c1f666e2c2d773111921b115d4249563 [doc] add notes on release notes (John Newbery)
Pull request description:
Explains when and how release notes should be written.
Tree-SHA512: 94085d5a30499f41e6d1821b9f157aea40b3cff61a8ba606fed1b239e794ffe6769f985f53400715d712d12aadaa8db8cfca08dd1700a1fe17df86e0e554eac2
a94e470921 A few textual improvements (Martin Erlandsson)
Pull request description:
Found a few places where the reading flow was interrupted by minor grammar and punctuation issues.
Tree-SHA512: 50640288449eb035f463bce563d259efbe97c14517d92c916e1bc52b7c54961ba2fc6fca1272470153803397f20ae7570bd090c16850ebd0180ebcf6bb2415d1
5b76c314d6 doc: Add separate productivity notes document (Carl Dong)
Pull request description:
Many developers have their own tools and tricks to be more productive
during their cycles, so let's document the best ones so that everyone
can benefit from them.
Tree-SHA512: b4989e7a815e972a9a646f448fb6c08bd896b4bce77fd7fb22a71a7602971d4cbe34f88183f503f5b851d002784d9e91b87df5348c661eeb9cefa69c52e0de2b
42ff30ec6 [Docs] add short documentation for /rest/blockhashbyheight (Jonas Schnelli)
579d418f7 [QA] add rest tests for /rest/blockhashbyheight/<HEIGHT>.<FORMAT> (Jonas Schnelli)
eb9ef04c4 REST: add "blockhashbyheight" call, fetch blockhash by height (Jonas Schnelli)
Pull request description:
Completes the REST interface for trivial block exploring by adding a call that allows to fetch the blockhash in the main chain by a given height.
Tree-SHA512: 94be9e56718f857279b11cc16dfa8d04f3b5a762e87ae54281b4d87247c71c844895f4944d5a47f09056bf851f4c4761ac4fbdbaaee957265d14de5c1c73e8d2
7cf994d5cfd53dcff76ebd0e0007e3477a7570e8 qa: Improve tests of /rest/headers and /rest/block (João Barbosa)
0825b86b280c684c32c60bac9e862298c7279f27 doc: /rest/block responds with 404 if block does not exist (João Barbosa)
be625f7c5562afed517ff51d2d85268ba5ce6017 doc: Explain empty result of /rest/headers (João Barbosa)
Pull request description:
Follow up of #15107.
Tree-SHA512: a7fdeed05216e3eda9604664db529237c2d0ddf422cfac139d6345a22b6e00bfe870d4e3f177423db7d4efb295ac2dc0ca2eb20c9c27c0719b89fd5428860d03
a62e667296 docs: Add more Doxygen information to Developer Notes (Jon Layton)
Pull request description:
Update information about Doxygen in `doc/developer-notes.md`.
Alternatively, this could have its own file (like `doc/web-documentation.md`), since there are installation steps included.
For example, I had to run:
```
brew install doxygen graphviz
```
on MacOS, otherwise failures occurred.
This information could also be linked to the `doc/release-process.md`.
Tree-SHA512: 5d77ee83e1b96fde036482b502f676a90a56f3f667753545a7cfba5c2e3b825644bb4cf0f8a84b7f9ba92fa5f2e1cd6ef1e27a94277f43d012355df741f7dd2f
989c8990bb765eef45c8ee471f084ca81a0bead4 Rename “OS X” to the newer “macOS” convention (Giulio Lombardo)
Pull request description:
Since 2016, with [macOS 10.12 Sierra](https://en.wikipedia.org/wiki/MacOS_Sierra), Mac OS X has been renamed in macOS. It would be a nice if Bitcoin's macOS build instructions follow this naming convention to avoid misunderstandings.
Tree-SHA512: 51b7d54bfc39a1a9d0773c64780817c7beca7094aded80481086287474dfa272bf0a1dfa6ef6e3cae91548aa127f65fa730003dddcb97147cdc8c249146aea22
ee5e896 Organise Linux build instructions to be categorised by distro (Alex Vear)
4c85517 Add NetBSD build instruction links (Alex Vear)
Pull request description:
* Added references to the newly created [`doc/build-netbsd.md`] (#12294) instructions in the [`doc/README.md`] and the [`doc/build-unix.md`] files.
* Organise [`doc/build-unix.md`] dependency build instructions by Linux distribution. This will help discoverability of dependency build instructions for specific distros. Future instructions will also be able to be added easier.
I am not quite sure about the FreeBSD instructions being in the [`doc/build-unix.md`], while both the OpenBSD and NetBSD instructions are contained within separate files ([`doc/build-openbsd.md`] and [`doc/build-netbsd.md`] respectively).
Feedback is greatly appreciated. 😄
[`doc/build-netbsd.md`]:https://github.com/bitcoin/bitcoin/blob/master/doc/build-netbsd.md
[`doc/build-unix.md`]:https://github.com/bitcoin/bitcoin/blob/master/doc/build-unix.md
[`doc/build-openbsd.md`]:https://github.com/bitcoin/bitcoin/blob/master/doc/build-openbsd.md
[`doc/README.md`]:https://github.com/bitcoin/bitcoin/blob/master/doc/README.md
Tree-SHA512: ebe2604d1802795851bbfce2335f159b53ea696bc9afb309be7825c697b992cc3963270fe945ca3e449b18522046e227fde3fae1b9c01bd49c3a7a513b5bd40c
be5ca825a38bc71c3a79ef35335e9c2e597ad225 doc: update NetBSD build instructions for 8.0 (fanquake)
Pull request description:
Updates the NetBSD build documentation for 8.0.
Use Python37 and add pkg-config.
Switches to using our `contrib/install_db4.sh` script for installing bdb.
Tree-SHA512: c0ac1a89349a752d9d4d87e2d134fd402e9beaac84e471ec9a0b507ebc5e762e973c8d2821db3824dea82841e38c39b0bb0a0d97f7e58318f2b15e93e81bf654
9b2704777c [doc] Include txindex changes in the release notes. (Jim Posen)
ed77dd6b30 [test] Simple unit test for TxIndex. (Jim Posen)
6d772a3d44 [rpc] Public interfaces to GetTransaction block until synced. (Jim Posen)
a03f804f2a [index] Move disk IO logic from GetTransaction to TxIndex::FindTx. (Jim Posen)
e0a3b80033 [validation] Replace tx index code in validation code with TxIndex. (Jim Posen)
8181db88f6 [init] Initialize and start TxIndex in init code. (Jim Posen)
f90c3a62f5 [index] TxIndex method to wait until caught up. (Jim Posen)
70d510d93c [index] Allow TxIndex sync thread to be interrupted. (Jim Posen)
94b4f8bbb9 [index] TxIndex initial sync thread. (Jim Posen)
34d68bf3a3 [index] Create new TxIndex class. (Jim Posen)
c88bcec93f [db] Migration for txindex data to new, separate database. (Jim Posen)
0cb8303241 [db] Create separate database for txindex. (Jim Posen)
Pull request description:
I'm re-opening #11857 as a new pull request because the last one stopped loading for people
-------------------------------
This refactors the tx index code to be in it's own class and get built concurrently with validation code. The main benefit is decoupling and moving the txindex into a separate DB. The primary motivation is to lay the groundwork for other indexers that might be desired (such as the [compact filters](https://github.com/bitcoin/bips/pull/636)). The basic idea is that the TxIndex spins up its own thread, which first syncs the txindex to the current block index, then once in sync the BlockConnected ValidationInterface hook writes new blocks.
### DB changes
At the suggestion of some other developers, the txindex has been split out into a separate database. A data migration runs at startup on any nodes with a legacy txindex. Currently the migration blocks node initialization until complete.
### Open questions
- Should the migration of txindex data from the old DB to the new DB block in init or should it happen in a background thread? The downside to backgrounding it is that `getrawtransaction` would return an error message saying the txindex is syncing while the migration is running.
### Impact
In a sample size n=1 test where I synced nodes from scratch, the average time [Index writing](https://github.com/bitcoin/bitcoin/blob/master/src/validation.cpp#L1903) was 3.36ms in master and 1.72ms in this branch. The average time between `UpdateTip` log lines for sequential blocks between 400,000 and IBD end on mainnet was 0.297204s in master and 0.286134s in this branch. Most likely this is just variance in IBD times, but I can try with some more trials if people want.
Tree-SHA512: 451fd7d95df89dfafceaa723cdf0f7b137615b531cf5c5035cfb54e9ccc2026cec5ac85edbcf71b7f4e2f102e36e9202b8b3a667e1504a9e1a9976ab1f0079c4
* build: Add cmake as depends package (cmake.mk)
The bls-signatures library requires cmake 3.14
* depends: Update chia_bls to version 1.0.0 of dashpay/bls-signatures
* depends: Rename package chia_bls to bls-dash
* depends: Disable blspy/tests/benchmarks build for chia_bls
Note: Building with tests would require the following:
depends: Fix macOS build for versions < 10.12
Seems like older versions of macOS pretend to support c++17 std libs but do not have/have issues with uncaught_exceptions. "Catch", the testing framework used in the bls lib wants those by default but setting `DCATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS` disables them which should be just fine to do in all cases here.
---
depends/packages/chia_bls.mk | 2 ++
1 file changed, 2 insertions(+)
diff --git a/depends/packages/chia_bls.mk b/depends/packages/chia_bls.mk
index e79a85844f..25749b74ab 100644
--- a/depends/packages/chia_bls.mk
+++ b/depends/packages/chia_bls.mk
@@ -28,6 +28,8 @@ define $(package)_set_vars
$(package)_config_opts_darwin+= -DCMAKE_AR="$(host_prefix)/native/bin/$($(package)_ar)"
$(package)_config_opts_darwin+= -DCMAKE_RANLIB="$(host_prefix)/native/bin/$($(package)_ranlib)"
endif
+
+ $(package)_cppflags+= -DCATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS
endef
define $(package)_config_cmds
--
* depends: Drop obsolete variable
* bls: Integrate the upgraded version into the codebase
* depends: Pre-fetch relic to fix gitian
Can fetch apt packages only when building with lxc
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
a9e898a4ad547ad344671db2b942b99925f0c732 docs: Link to python style guidelines from developer notes (Mason Simon)
Pull request description:
Initially I moved the python style guidelines from the functional test README, but some of the python rules are test-specific, and most of the developer notes doc is C++ centric, so just dropping a link seemed better.
Tree-SHA512: 9d4d5cc45526319a118595d90fcfad2c9aced22007aa096d8af04ba1b963312822804f4c15b0b227d66af49565034437691b7760e7ff6d1e3f8b10b898906362
3828a79711 scripted-diff: prefer MAC_OSX over __APPLE__ (fanquake)
fa6e841e89 gui: remove macOS ProgressBar workaround (fanquake)
68c272527f gui: remove SubstituteFonts (fanquake)
6c6dbd8af5 doc: mention that macOS 10.10 is now required (fanquake)
84b0cfa8b6 release: bump minimum required macOS to 10.10 (fanquake)
26b15df99d depends: set OSX_MIN_VERSION to 10.10 (fanquake)
Pull request description:
Closes#13362
d99abfddb0c8f2111340a6127e77cc686e0043d8
This workaround should no longer be required, as it should have only been in use when compiled with the 10.7 SDK, which we haven't been building with for a while now.
5bc5ae30982a0f0f6a9804b05d99434af770c724
The bugreport linked with this code is for an unrelated? issue, however from what I can tell the correct QTBUG is this one https://bugreports.qt.io/browse/QTBUG-20880. Reading though the discussion there, it seems that the way progress bars are animated changed in macOS 10.10.
Qt was patched [here (5.5+)](https://codereview.qt-project.org/#/c/112379/):
> Disable progress bar animations on 10.10 Yosemite and higher - the native style does not animate them any more. Keep the indeterminate progress bar animation.
Given all of that, I don't think this is worth keeping around, as it would seem to only be useful in the case that a macOS user is compiling with a Qt < 5.5. That should be pretty unlikely, as we don't support downloaded Qt binaries, and brew currently provides [5.11.1](571b46213c/Formula/qt.rb).
Tree-SHA512: 4278cb30cc9bcb313e166129ecf032c808995f8b51a3123637c47860a0010ac88f86f82ec44792153b6b1e5cca595f25013b2eaeae80194647b9ce4f7eaf32c1
1874058 Make base58 python contrib code work with python3 (Evan Klitzke)
bc6fdf2 Change all python files to use Python3 (John Newbery)
Pull request description:
Following discussion here: https://github.com/bitcoin/bitcoin/pull/11843#issuecomment-351033742
It's easier for maintainers if all python tools/scripts support only a single version of Python. There are only a few scripts that aren't explicitly python3 at this point, so this PR changes those remaining scripts to explicitly require python3.
Tree-SHA512: 5d38eef6e0fc7d8515e23a1f4c75e8b4160fd0fe23cba52a1f41689b114e54a9e503e0724829e8b41982ef98f2d113df80d9e238213b74f09ceaed0344a19e24
f381299 Move CKeyStore::cs_KeyStore to CBasicKeyStore (João Barbosa)
25eb9f5 Inline CKeyStore::AddKey(const CKey &) in CBasicKeyStore (João Barbosa)
Pull request description:
Made these simplifications while reviewing #12714. This aims to make `CKeyStore` a *pure* interface:
- no variable members - the mutex is moved to `CBasicKeyStore` which is where it is used;
- no method implementations - `AddKey(const CKey &)` is moved to `CBasicKeyStore` which is where it is needed.
Tree-SHA512: 84e44f4390c59600e5cefa599b5464e1771c31dd4abc678ef50db8e06ffac778d692860a352918444f8bcd66430634637b6277a818a658721ffc4f381c1c6a90
6feb46c Add --with-sanitizers option to configure (Evan Klitzke)
Pull request description:
This adds configure options for `-fsanitize=address`, `-fsanitize=thread`, and `-fsanitize=undefined` which are all disabled by default. These flags are useful for developers who wish to do additional safety checking. Note that some of these are mutually incompatible, and these may have a large performance overhead.
There's some kind of strange logic required to properly check for the availability of these flags in a way that works on both GCC and Clang, hopefully the comments make it clear what's going on.
Tree-SHA512: 2d6fe402799110e59ee452dddf37f7ca2d26a7fecec50be25c8a134e4a20beb31f1e8f438dffd443641562418075896d1eeb450623425b272d80e05e3027a587
0bd2ec5 Improve formatting of developer notes (Evan Klitzke)
Pull request description:
The developer notes file has gotten pretty large and unwieldly. This reorganizes some content, most notably by adding a TOC to the page. Compare how the page looks in [my branch](https://github.com/eklitzke/bitcoin/blob/developer-notes/doc/developer-notes.md) vs [master](https://github.com/eklitzke/bitcoin/blob/master/doc/developer-notes.md).
I know there's long-term interest in moving a lot of this content to the bitcoin-core/docs repo on GitHub, but this makes things better now.
The TOC format here is a semi-standard extension to Markdown files that you may have seen on GitHub pages in other projects. The `<!-- markdown-toc -->` comments used by these tools to know where the TOC starts/ends. The following tools all understand this format:
* [Sphinx](http://www.sphinx-doc.org/en/master/)
* [Pandoc](https://pandoc.org/)
* [markdown-toc.el](https://github.com/ardumont/markdown-toc) (what I used)
* various plugins for [vim](https://github.com/mzlogin/vim-markdown-toc) and other editors
I used this TOC extension at a previous job, and my observation is that it's fine if people just edit the TOC manually. It's just text and it's not a huge deal if it gets a little out of sync. It can also be regenerated at any time by anyone with any of these tools.
Tree-SHA512: 298d1605ea5e8bfc0f75e70570c23ebd6891e4ffcdedd24fefadc23edd6e4b96509d8d102209868468a1b3ddbe2c3b8462698cdda8b9421348b5bc6f7b8d0cb8
ccedbaf Increase LevelDB max_open_files unless on 32-bit Unix. (Evan Klitzke)
Pull request description:
Currently we set `max_open_files = 64` on all architectures due to concerns about file descriptor exhaustion. This is extremely expensive due to how LevelDB is designed.
When a LevelDB file handle is opened, a bloom filter and block index are decoded, and some CRCs are checked. Bloom filters and block indexes in open table handles can be checked purely in memory. This means that when doing a key lookup, if a given table file may contain a given key, all of the lookup operations can happen completely in RAM until the block itself is fetched. In the common case fetching the block is one disk seek, because the block index stores its physical offset. This is the ideal case, and what we want to happen as often as possible.
If a table file handle is not open in the table cache, then in addition to the regular system calls to open the file, the block index and bloom filter need to be decoded before they can be checked. This is expensive and is something we want to avoid.
The current setting of 64 file handles means that on a synced node, only about 4% of key lookups can be satisifed by table file handles that are actually open and in memory.
The original concerns about file descriptor exhaustion are unwarranted on most systems because:
* On 64-bit POSIX hosts LevelDB will open up to 1000 file descriptors using `mmap()`, and it does not retain an open file descriptor for such files.
* On Windows non-socket files do not interfere with the main network `select()` loop, so the same fd exhaustion issues do not apply there.
This change keeps the default `max_open_files` value (which is 1000) on all systems except 32-bit POSIX hosts (which do not use `mmap()`). Open file handles use about 20 KB of memory (for the block index), so the extra file handles do not cause much memory overhead. At most 1000 will be open, and a fully synced node right now has about 1500 such files.
Profile of `loadblk` thread before changes: https://monad.io/maxopenfiles-master.svg
Profile of `loadblk` thread after changes: https://monad.io/maxopenfiles-increase.svg
Tree-SHA512: de54f77d57e9f8999eaf8d12592aab5b02f5877be8fa727a1f42cf02da2693ce25846445eb19eb138ce4e5045d1c65e14054df72faf3ff32c7655c9cfadd27a9
