Commit Graph

106 Commits

Author SHA1 Message Date
Wladimir J. van der Laan
86de06e86a Merge #15243: [doc] add notes on release notes
65bc38d1c1f666e2c2d773111921b115d4249563 [doc] add notes on release notes (John Newbery)

Pull request description:

  Explains when and how release notes should be written.

Tree-SHA512: 94085d5a30499f41e6d1821b9f157aea40b3cff61a8ba606fed1b239e794ffe6769f985f53400715d712d12aadaa8db8cfca08dd1700a1fe17df86e0e554eac2
2021-08-24 19:14:50 -04:00
MarcoFalke
33384816b5 Merge #14519: tests: add utility to easily profile node performance with perf
13782b8ba8 docs: add perf section to developer docs (James O'Beirne)
58180b5fd4 tests: add utility to easily profile node performance with perf (James O'Beirne)

Pull request description:

  Adds a context manager to easily (and selectively) profile node performance during functional test execution using `perf`.

  While writing some tests, I encountered some odd bitcoind slowness. I wrote up a utility (`TestNode.profile_with_perf`) that generates performance diagnostics for a node by running `perf` during the execution of a particular region of test code.

  `perf` usage is detailed in the excellent (and sadly unmerged) https://github.com/bitcoin/bitcoin/pull/12649; all due props to @eklitzke.

  ### Example

  ```python
  with node.profile_with_perf("large-msgs"):
      for i in range(200):
          node.p2p.send_message(some_large_msg)
      node.p2p.sync_with_ping()
  ```

  This generates a perf data file in the test node's datadir (`/tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data`).

  Running `perf report` generates nice output about where the node spent most of its time while running that part of the test:

  ```bash
  $ perf report -i /tmp/testtxmpod0y/node0/node-0-TestName-large-msgs.perf.data --stdio \
    | c++filt \
    | less

  # To display the perf.data header info, please use --header/--header-only options.
  #
  #
  # Total Lost Samples: 0
  #
  # Samples: 135  of event 'cycles:pp'
  # Event count (approx.): 1458205679493582
  #
  # Children      Self  Command          Shared Object        Symbol
  # ........  ........  ...............  ...................  ........................................................................................................................................................................................................................................................................
  #
      70.14%     0.00%  bitcoin-net      bitcoind             [.] CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)
                  |
                  ---CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

      70.14%     0.00%  bitcoin-net      bitcoind             [.] CNetMessage::readData(char const*, unsigned int)
                  |
                  ---CNetMessage::readData(char const*, unsigned int)
                     CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

      35.52%     0.00%  bitcoin-net      bitcoind             [.] std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
                  |
                  ---std::vector<char, zero_after_free_allocator<char> >::_M_fill_insert(__gnu_cxx::__normal_iterator<char*, std::vector<char, zero_after_free_allocator<char> > >, unsigned long, char const&)
                     CNetMessage::readData(char const*, unsigned int)
                     CNode::ReceiveMsgBytes(char const*, unsigned int, bool&)

  ...
  ```

Tree-SHA512: 9ac4ceaa88818d5eca00994e8e3c8ad42ae019550d6583972a0a4f7b0c4f61032e3d0c476b4ae58756bc5eb8f8015a19a7fc26c095bd588f31d49a37ed0c6b3e
2021-08-18 22:24:36 -03:00
Wladimir J. van der Laan
debac13960 Partial Merge #14831: Scripts and tools: Use #!/usr/bin/env bash instead of #!/bin/bash.
688f665a5e526fda0fb797bf617412fe9cbe64fd Scripts and tools & Docs: Used #!/usr/bin/env bash instead of obsolete #!/bin/bash, added linting for .sh files shebang and updated the Developer Notes. (vim88)

Pull request description:

  As it was discussed in [#13510](https://github.com/bitcoin/bitcoin/pull/13510), it is better to use `#!/usr/bin/env bash` instead of `#!/bin/bash`.

Tree-SHA512: 25f71eb9a6a0cdc91568b5c6863205c5fe095f77a69e633503a2ac7805bd9013af8538e538c0c666ce96a28e3f43ce7a8df5f08d4ff007723bb588d85674f2da
2021-08-15 11:08:08 -04:00
MarcoFalke
fc40144859 Merge #14395: Fix typos and cleanup
4a9f064ea1 Fix typos (Dimitris Apostolou)

Pull request description:

Tree-SHA512: 011133fb524c3c3cfd272aad0fe9103192b1cc00ac8b14d5b224368fffdb3893e8a19fe281e59ea1aba81bd5e5ab4b7b2de5c3f269e4a7aee66959727cd82cec

# Conflicts:
#	doc/init.md
2021-07-22 16:02:34 -04:00
UdjinM6
95f89eec56
Fix more spelling errors 2021-07-19 12:52:03 -05:00
MarcoFalke
289c8772a8 Merge bitcoin/bitcoin#22204: doc: remove obsolete okSafeMode RPC guideline from developer notes
6780a095d8526a46c3c2b20a14831687e9fc2462 doc: remove obsolete `okSafeMode` RPC guideline from developer notes (Sebastian Falbesoner)

Pull request description:

  Since the flag has been removed from the RPC command table in commit ec6902d0ea (PR #11179), this guideline is not relevant anymore and can be removed.

ACKs for top commit:
  MarcoFalke:
    ACK 6780a095d8526a46c3c2b20a14831687e9fc2462. Also, safe mode was removed completely in commit 2ae705d841

Tree-SHA512: 2a6b002ae302e979ce403171b79a892e32f5083792c3b0b8204edb5eb08c6b24ab77bbeeae0e3bb6d6564a6f1678cfce00eb7b5b82063b7741f89a96b0c0aef3
2021-07-15 19:39:10 -05:00
MarcoFalke
5a7e219d81 Merge #20986: docs: update developer notes to discourage very long lines
aa929abf8dc022e900755234c857541faeea8239 [docs] Update developer notes to discourage very long lines (John Newbery)

Pull request description:

  Mandatory rules on line lengths are bad - there will always be cases where a longer line is more readable than the alternative.

  However, very long lines for no good reason _do_ hurt readability. For example, this declaration in validation.h is 274 chars:

  ```c++
      bool ConnectTip(BlockValidationState& state, const CChainParams& chainparams, CBlockIndex* pindexNew, const std::shared_ptr<const CBlock>& pblock, ConnectTrace& connectTrace, DisconnectedBlockTransactions& disconnectpool) EXCLUSIVE_LOCKS_REQUIRED(cs_main, m_mempool.cs);
  ```

  That won't fit on one line without wrapping on my 27" monitor with a comfortable font size. Much easier to read is something like:

  ```c++
      bool ConnectTip(BlockValidationState& state, const CChainParams& chainparams,
                      CBlockIndex* pindexNew, const std::shared_ptr<const CBlock>& pblock,
                      ConnectTrace& connectTrace, DisconnectedBlockTransactions& disconnectpool)
          EXCLUSIVE_LOCKS_REQUIRED(cs_main, m_mempool.cs);
  ```

  Therefore, _discourage_ (don't forbid) line lengths greater than 100 characters in our developer style guide.

  100 chars is somewhat arbitrary. The old standard was 80, but that seems very limiting with modern displays.

ACKs for top commit:
  fanquake:
    ACK aa929abf8dc022e900755234c857541faeea8239 - this is basically just something to point too when a PR has unreasonably long lines for no particularly reason.
  practicalswift:
    ACK aa929abf8dc022e900755234c857541faeea8239
  amitiuttarwar:
    ACK aa929abf8dc022e900755234c857541faeea8239
  theStack:
    ACK aa929abf8dc022e900755234c857541faeea8239
  glozow:
    ACK aa929abf8d

Tree-SHA512: 17f1b11f811137497ede8851ede93fa612dc622922b5ad7ac8f065ea026d9a718db5b92325754b74d24012b4d45c4e2cd5cd439a6a8d34bbabf5da927d783970
2021-07-15 19:39:10 -05:00
Kittywhiskers Van Gogh
da33c9619c
partial merge #17398: Update leveldb to 1.22+ (#4230)
* Update to leveldb upstream using subtree merge

* Import crc32c using subtree merge as as 'src/crc32c'

* build: Update build system for new leveldb

Upstream leveldb switched build systems, which means we need to define
a few different values.

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* doc: Add crc32c subtree to developer notes

* test: Add crc32c to subtree check linter

* test: Add crc32c exception to various linters and generation scripts

* build: Add LCOV exception for crc32c

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* build: CRC32C build system integration

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-07-15 15:42:55 -05:00
fanquake
f74aea23f1
Merge #18577: doc: Correct scripted-diff example link
478c11dde326e2ff0480c14f76f9f6b52a7bdfd0 Correct scripted-diff example link (Yahia Chiheb)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK 478c11dde326e2ff0480c14f76f9f6b52a7bdfd0

Tree-SHA512: 3bc741a79db9bd7abb17ef11f697b768565ec01303a5823ee6a7d8dfa6e888a99a15e9eda69f97a912abc3fd56a54f698f9a580596511bc9bcf62a6870b273f6
2021-07-14 18:44:45 -05:00
Wladimir J. van der Laan
41aba0a27a Merge #17411: doc: Add some better examples for scripted diff
adbe15504713ddba6e9c024c59d977675d49e350 doc: Add some better examples for scripted diff (Wladimir J. van der Laan)

Pull request description:

  The current example isn't too great, for example it uses `find` instead of `git ls-files`. Add a subsection with suggestions and examples.

  Feel free to propose some other great examples to add.

ACKs for top commit:
  hebasto:
    re-ACK adbe15504713ddba6e9c024c59d977675d49e350

Tree-SHA512: 38f03716a122a1791c93abc052ea7572a3d2108b3d0d93dc95d3c4a7eb190c6b639d1cc66e4f74d378c4b11d6951dbd901d0973792f8f13cbeb9d9dcf4f8e037
2021-07-13 20:43:16 -05:00
UdjinM6
3004c3498b
Merge pull request #4245 from PastaPastaPasta/backport-triv-pr9
backport: 'trivial' pr9
2021-07-14 02:52:00 +03:00
fanquake
6b46bddf64 Merge #16914: doc: Update homebrew instruction for doxygen
14c6a2de1a4cf8cc17116d418242709ba2519b9e [doc] update brew instruction for doxygen (Sjors Provoost)

Pull request description:

  I noticed while testing #16912 that `brew install doxygen --with-graphviz` no long works. Instead you need to use `brew install graphviz doxygen`.

ACKs for top commit:
  fanquake:
    ACK 14c6a2de1a4cf8cc17116d418242709ba2519b9e - tested a `make docs` on macOS with and without `graphviz` (`dot`) available.

Tree-SHA512: 2682568e558c16e9e0a657421c449b74cc14a89771844c1c88623fb75b07b89afb63c45a919eb7b9c3dba9bdfaef21489b5f7ea45a08d8d5da18614657c19e47
2021-07-12 20:54:22 -05:00
fanquake
ee46c9545e Merge #16461: doc: Tidy up shadowing section
9452802480bd154e23771230bbdfebde1dbaa941 doc: Tidy up shadowing section (João Barbosa)

Pull request description:

  Removes the example because it violates the code format.

ACKs for top commit:
  MarcoFalke:
    unsigned ACK 9452802480bd154e23771230bbdfebde1dbaa941
  ryanofsky:
    ACK 9452802480bd154e23771230bbdfebde1dbaa941
  fanquake:
    ACK 9452802480bd154e23771230bbdfebde1dbaa941 - Thanks for following up.

Tree-SHA512: 1fc31355d368225713298da7803e39e99014fbfcd229f2d3b56c082de95ab2965e51c80b172a5abce4646c53f845fa62a6d94d5df714e7835cac07a8ec7d5da7
2021-07-12 20:52:56 -05:00
fanquake
d37c1fc650 Merge #16256: doc: remove orphaned header in developer notes
5a88ea7c67448748a63ac7963c70a047a5daca79 doc: remove orphaned header in developer notes (Jon Atack)

Pull request description:

  The "Git and GitHub tips" section was moved from doc/developer-notes.md to doc/productivity.md in 5b76c31, but the header link to that long-gone section in the developer notes remains and needs to go.

  So long, Git and GitHub tips, we barely knew ya.

ACKs for commit 5a88ea:
  fanquake:
    ACK 5a88ea7c67448748a63ac7963c70a047a5daca79

Tree-SHA512: d2a0bea27cd9209adec2127fae5e336c44771aa46af7c544fa2c80a3df4868adafbccc30ef370369404b882fa009f7198cbf41777afd30eccf0c21a7eb1d0ad1
2021-07-12 20:16:37 -05:00
UdjinM6
08558d67ee
Merge pull request #4235 from PastaPastaPasta/backport-triv-pr7
backport: 'trivial' pr7
2021-07-10 21:46:25 +03:00
MarcoFalke
3ba32a1116 Merge #15887: docs: Align code example style with clang-format
201393f932 Align code example with clang-format (Hennadii Stepanov)

Pull request description:

  With this PR running [clang-format-diff.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/clang-format-diff.py) on the code example will not fire a format adjustment.

ACKs for commit 201393:
  MarcoFalke:
    trivial ACK 201393f93268f6775d1b5d54119a7210628b333d

Tree-SHA512: 825c5e8cfba1bc140c2dfc38b82c5eec268b82b528af4301f25dfacc1f4f0788e268e72e8512f7ce001be665a8b07964a0af832fd9a1c6bd1c27d252f93619bc
2021-07-10 12:10:51 -05:00
UdjinM6
637a8a28fe
Merge pull request #4226 from PastaPastaPasta/backport-triv-pr6
backport: 'trivial' pr6
2021-07-08 23:25:09 +03:00
MarcoFalke
752797b6bf
Merge #14882: [doc] developer-notes.md: point out that UniValue deviates from upstream
a67d71311d [doc] developer-notes.md: point out that UniValue deviates from upstream (Sjors Provoost)

Pull request description:

  While debugging an issue I was somewhat surprised to [learn](https://github.com/bitcoin/bitcoin/pull/14164#issuecomment-419752678) that we've moved `src/univalue` from https://github.com/jgarzik/univalue to https://github.com/bitcoin-core/univalue, that these repos are both maintained and they're different.

  The first mention of using the bitcoin-core repo is from late 2015 in #7157. I didn't check when the last common ancestor commit is.

  I couldn't find documentation as to why (these things just happen in open source of course), but at minimum we should make this more clear.

  There's also the following line in `config.ac` that I'm not sure what to do with:
  ```
  AC_INIT([univalue], [1.0.3],
          [http://github.com/jgarzik/univalue/])
  ```

Tree-SHA512: e58105677b5ebe0005772282da4a805fee7dfccacfb1b2686a874517bf46072d1481181f8a8865d25526f6ed9e5fcd55d8d49906bf27cd0f5aefe4f258aa4d63
2021-07-08 11:57:11 -05:00
Wladimir J. van der Laan
a54a48b70d Merge #14625: Make clear function argument case in dev notes
9605bbd315eb14690427560fd9a274fe837f59f5 Make clear function argument case in dev notes (Carl Dong)

Pull request description:

  Rationale:

  For new developers, they might be confused if they see that function arguments are sometimes `camelCase`'d in the codebase. This makes it clear that they _should_ be `snake_case`'d (maybe because no one's gotten to fixing them yet).

Tree-SHA512: 9db16d1fedf9761121844a0865ae3fefea94b5dbdfb36cb18f99cbc73e117f7d798a019f28a1c8bca19772502de2f9ed063f03bd911ffc4d248ec7386cd87d97
2021-07-03 13:44:50 -05:00
MarcoFalke
9f42adb761 Merge #14731: doc: Improve scripted-diff developer docs
07e286d940 Improve scripted-diff developer docs (Carl Dong)

Pull request description:

  Instead of verifying all scripted-diffs, provide an example that only verifies all scripted-diffs in commits since `origin/master`.

Tree-SHA512: 2faecfd1df04046c77ca2b7426675265f7ed656b2b7e2714ae61c0a38b3ea239f35d2be6d615c5191a21f3a453e37f6a4622bd4558173b768cb73183b56c4d62
2021-07-01 17:37:50 -05:00
MarcoFalke
f4956266df Merge #14213: Docs: Fix reference to lint-locale-dependence.sh
14b29a77ac Fix reference to lint-locale-dependence.sh (Hennadii Stepanov)

Pull request description:

  The wrong reference sneaked through #13041 and #13281.

Tree-SHA512: 39cc74297d00141206ce68b84575288a20e39e20ef717fa8b8c09076b5fb46d8281320b144a596094365f2a0704e5dd6bf960e35980ae4730546a72957403d69
2021-07-01 16:54:18 -05:00
UdjinM6
49ebff2702
Merge pull request #4217 from PastaPastaPasta/backports-0.18-triv-pr2
backport: 'trivial' pr2
2021-07-01 23:26:07 +03:00
MarcoFalke
afe42bd422 Merge #14287: tests: Use MakeUnique to construct objects owned by unique_ptrs
b6718e373e tests: Use MakeUnique to construct objects owned by unique_ptrs (practicalswift)

Pull request description:

  A subset of #14211 ("Use MakeUnique to construct objects owned by unique_ptrs") as suggested by @MarcoFalke in https://github.com/bitcoin/bitcoin/pull/14211#issuecomment-423324019.

  Use `MakeUnique` to construct objects owned by `unique_ptr`s.

  Rationale:
  * `MakeUnique` ensures exception safety in complex expressions.
  * `MakeUnique` gives a more concise statement of the construction.

Tree-SHA512: 1228ae6ce7beb178d79142c4e936b728178ccaa8aa35c6d8feeb33d1a667abfdd010c59996a9d833594611e913877ce5794e75953d11d9b1fdbac04aa491d9cf
2021-06-28 21:05:29 -04:00
Wladimir J. van der Laan
90fc614500
Merge #15243: [doc] add notes on release notes
65bc38d1c1f666e2c2d773111921b115d4249563 [doc] add notes on release notes (John Newbery)

Pull request description:

  Explains when and how release notes should be written.

Tree-SHA512: 94085d5a30499f41e6d1821b9f157aea40b3cff61a8ba606fed1b239e794ffe6769f985f53400715d712d12aadaa8db8cfca08dd1700a1fe17df86e0e554eac2
2021-06-28 19:05:24 -05:00
MarcoFalke
447ff166fb
Merge #15348: doc: Add separate productivity notes document
5b76c314d6 doc: Add separate productivity notes document (Carl Dong)

Pull request description:

  Many developers have their own tools and tricks to be more productive
  during their cycles, so let's document the best ones so that everyone
  can benefit from them.

Tree-SHA512: b4989e7a815e972a9a646f448fb6c08bd896b4bce77fd7fb22a71a7602971d4cbe34f88183f503f5b851d002784d9e91b87df5348c661eeb9cefa69c52e0de2b
2021-06-28 18:13:20 -05:00
PastaPastaPasta
988a1693fc
Dashification 2021-06-28 17:39:58 -05:00
MarcoFalke
342741ba33
Merge #14832: docs: Add more Doxygen information to Developer Notes
a62e667296 docs: Add more Doxygen information to Developer Notes (Jon Layton)

Pull request description:

  Update information about Doxygen in `doc/developer-notes.md`.

  Alternatively, this could have its own file (like `doc/web-documentation.md`), since there are installation steps included.

  For example, I had to run:
  ```
  brew install doxygen graphviz
  ```
  on MacOS, otherwise failures occurred.

  This information could also be linked to the `doc/release-process.md`.

Tree-SHA512: 5d77ee83e1b96fde036482b502f676a90a56f3f667753545a7cfba5c2e3b825644bb4cf0f8a84b7f9ba92fa5f2e1cd6ef1e27a94277f43d012355df741f7dd2f
2021-06-28 13:40:51 -05:00
Wladimir J. van der Laan
a9ba11c343 Merge #13717: docs: Link to python style guidelines from developer notes
a9e898a4ad547ad344671db2b942b99925f0c732 docs: Link to python style guidelines from developer notes (Mason Simon)

Pull request description:

  Initially I moved the python style guidelines from the functional test README, but some of the python rules are test-specific, and most of the developer notes doc is C++ centric, so just dropping a link seemed better.

Tree-SHA512: 9d4d5cc45526319a118595d90fcfad2c9aced22007aa096d8af04ba1b963312822804f4c15b0b227d66af49565034437691b7760e7ff6d1e3f8b10b898906362
2021-02-04 14:58:13 -06:00
Wladimir J. van der Laan
c902e6075d
Merge #12762: Make CKeyStore an interface
f381299 Move CKeyStore::cs_KeyStore to CBasicKeyStore (João Barbosa)
25eb9f5 Inline CKeyStore::AddKey(const CKey &) in CBasicKeyStore (João Barbosa)

Pull request description:

  Made these simplifications while reviewing #12714. This aims to make `CKeyStore` a *pure* interface:
   - no variable members - the mutex is moved to `CBasicKeyStore` which is where it is used;
   - no method implementations - `AddKey(const CKey &)` is moved to `CBasicKeyStore` which is where it is needed.

Tree-SHA512: 84e44f4390c59600e5cefa599b5464e1771c31dd4abc678ef50db8e06ffac778d692860a352918444f8bcd66430634637b6277a818a658721ffc4f381c1c6a90
2020-10-26 20:35:28 -04:00
PastaPastaPasta
6d3fe3f6b8
Dashify
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-10-16 02:31:42 +00:00
Wladimir J. van der Laan
c08855de88
Merge #12692: Add configure options for various -fsanitize flags
6feb46c Add --with-sanitizers option to configure (Evan Klitzke)

Pull request description:

  This adds configure options for `-fsanitize=address`, `-fsanitize=thread`, and `-fsanitize=undefined` which are all disabled by default. These flags are useful for developers who wish to do additional safety checking. Note that some of these are mutually incompatible, and these may have a large performance overhead.

  There's some kind of strange logic required to properly check for the availability of these flags in a way that works on both GCC and Clang, hopefully the comments make it clear what's going on.

Tree-SHA512: 2d6fe402799110e59ee452dddf37f7ca2d26a7fecec50be25c8a134e4a20beb31f1e8f438dffd443641562418075896d1eeb450623425b272d80e05e3027a587
2020-10-14 13:27:34 -04:00
Wladimir J. van der Laan
a461438fd1
Merge #12759: [Docs] Improve formatting of developer notes
0bd2ec5 Improve formatting of developer notes (Evan Klitzke)

Pull request description:

  The developer notes file has gotten pretty large and unwieldly. This reorganizes some content, most notably by adding a TOC to the page. Compare how the page looks in [my branch](https://github.com/eklitzke/bitcoin/blob/developer-notes/doc/developer-notes.md) vs [master](https://github.com/eklitzke/bitcoin/blob/master/doc/developer-notes.md).

  I know there's long-term interest in moving a lot of this content to the bitcoin-core/docs repo on GitHub, but this makes things better now.

  The TOC format here is a semi-standard extension to Markdown files that you may have seen on GitHub pages in other projects. The `<!-- markdown-toc -->` comments used by these tools to know where the TOC starts/ends. The following tools all understand this format:

   * [Sphinx](http://www.sphinx-doc.org/en/master/)
   * [Pandoc](https://pandoc.org/)
   * [markdown-toc.el](https://github.com/ardumont/markdown-toc) (what I used)
   * various plugins for [vim](https://github.com/mzlogin/vim-markdown-toc) and other editors

  I used this TOC extension at a previous job, and my observation is that it's fine if people just edit the TOC manually. It's just text and it's not a huge deal if it gets a little out of sync. It can also be regenerated at any time by anyone with any of these tools.

Tree-SHA512: 298d1605ea5e8bfc0f75e70570c23ebd6891e4ffcdedd24fefadc23edd6e4b96509d8d102209868468a1b3ddbe2c3b8462698cdda8b9421348b5bc6f7b8d0cb8
2020-10-14 13:27:33 -04:00
Wladimir J. van der Laan
89ced17da0
Merge #12495: Increase LevelDB max_open_files
ccedbaf Increase LevelDB max_open_files unless on 32-bit Unix. (Evan Klitzke)

Pull request description:

  Currently we set `max_open_files = 64` on all architectures due to concerns about file descriptor exhaustion. This is extremely expensive due to how LevelDB is designed.

  When a LevelDB file handle is opened, a bloom filter and block index are decoded, and some CRCs are checked. Bloom filters and block indexes in open table handles can be checked purely in memory. This means that when doing a key lookup, if a given table file may contain a given key, all of the lookup operations can happen completely in RAM until the block itself is fetched. In the common case fetching the block is one disk seek, because the block index stores its physical offset. This is the ideal case, and what we want to happen as often as possible.

  If a table file handle is not open in the table cache, then in addition to the regular system calls to open the file, the block index and bloom filter need to be decoded before they can be checked. This is expensive and is something we want to avoid.

  The current setting of 64 file handles means that on a synced node, only about 4% of key lookups can be satisifed by table file handles that are actually open and in memory.

  The original concerns about file descriptor exhaustion are unwarranted on most systems because:
   * On 64-bit POSIX hosts LevelDB will open up to 1000 file descriptors using `mmap()`, and it does not retain an open file descriptor for such files.
   * On Windows non-socket files do not interfere with the main network `select()` loop, so the same fd exhaustion issues do not apply there.

  This change keeps the default `max_open_files` value (which is 1000) on all systems except 32-bit POSIX hosts (which do not use `mmap()`). Open file handles use about 20 KB of memory (for the block index), so the extra file handles do not cause much memory overhead. At most 1000 will be open, and a fully synced node right now has about 1500 such files.

  Profile of `loadblk` thread before changes: https://monad.io/maxopenfiles-master.svg
  Profile of `loadblk` thread after changes: https://monad.io/maxopenfiles-increase.svg

Tree-SHA512: de54f77d57e9f8999eaf8d12592aab5b02f5877be8fa727a1f42cf02da2693ce25846445eb19eb138ce4e5045d1c65e14054df72faf3ff32c7655c9cfadd27a9
2020-10-14 13:27:33 -04:00
xdustinface
cee8c151fa scripted-diff: Merge #12906: Avoid interface keyword to fix windows gitian build
17780d6f35 scripted-diff: Avoid `interface` keyword to fix windows gitian build (Russell Yanofsky)

Pull request description:

  Rename `interface` to `interfaces`

  Build failure reported by ken2812221 in https://github.com/bitcoin/bitcoin/pull/10244#issuecomment-379434756

Tree-SHA512: e02c97c728540f344202c13b036f9f63af23bd25e25ed7a5cfe9e2c2f201a12ff232cc94a93fbe37ef6fb6bf9e036fe62210ba798ecd30de191d09338754a8d0

-BEGIN VERIFY SCRIPT-
git mv src/interface src/interfaces
ren() { git grep -l "$1" | xargs sed -i "s,$1,$2,g"; }
ren interface/            interfaces/
ren interface::           interfaces::
ren BITCOIN_INTERFACE_    BITCOIN_INTERFACES_
ren "namespace interface" "namespace interfaces"
-END VERIFY SCRIPT-
2020-10-14 12:10:12 +01:00
Russell Yanofsky
ce41506a3e Add developer notes about blocking GUI code 2020-10-05 15:13:34 +02:00
PastaPastaPasta
e8e34b836a
docs: Adjust developer-notes.md to use signed ints normally (#3663)
* Adjust developer-notes.md to use signed ints normally

Signed-off-by: pasta <pasta@dashboost.org>

* Adjust wording in dev-notes

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* remove trailing whitespace

Signed-off-by: pasta <pasta@dashboost.org>

* Remove trailing whitespace

* maybe fix linter

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-08-30 17:22:51 +03:00
Wladimir J. van der Laan
7be2b2456a Merge #13281: test: Move linters to test/lint, add readme
fa3c910bfeab00703c947c5200a64c21225b50ef test: Move linters to test/lint, add readme (MarcoFalke)

Pull request description:

  This moves the checks and linters from `devtools` to a subfolder in `test`. (Motivated by my opinion that the dev tools are mostly for generating code and updating the repo whereas the linters are read-only checks.)

  Also, adds a readme to clarify that checks and linters are only meant to prevent bugs and user facing issues, not merely stylistic preference or inconsistencies. (This is motivated by the diversity in developers and work flows as well as existing code styles. It would be too disruptive to change all existing code to a single style or too burdensome to force all developers to adhere to a single style. Also note that our style guide is changing, so locking in at the wrong style "too early" would only waste resources.)

Tree-SHA512: 9b10e89f2aeaf0c8a9ae248aa891d74e0abf0569f8e5dfd266446efa8bfaf19f0ea0980abf0b0b22f0d8416ee90d7435d21a9f9285b66df43f370b7979173406
2020-07-29 10:59:04 -05:00
MarcoFalke
7b091fb99f
Merge #12933: doc: Refine header include policy
fad0fc3c9a Refine travis check for duplicate includes (MarcoFalke)

Pull request description:

  Since there is no harm in having "duplicate" includes and it makes it obvious what are the dependencies of each file, without having to do static analysis or jumping between files, I'd suggest to revert the travis check for duplicate includes.

  Generally, I think that enforcing minor style preferences should not be done via travis. The cost of maintaining and the burden on other developers is too high. C.f discussion in https://github.com/bitcoin/bitcoin/pull/10973#discussion_r180142594

Tree-SHA512: 97ab0e769d457ccfb873fff6c99613f8b944cd7ef95bfdccb0e1bbe8f5df1f16548c658fa03af42516f806546e75646d338a061e7b057619490235d311ca21f1
2020-07-21 11:54:50 -05:00
Wladimir J. van der Laan
5d007d8b3f
Merge #12896: docs: Fix conflicting statements about initialization in developer notes
b119e78 docs: Fix conflicting statements about initialization in developer notes (practicalswift)

Pull request description:

  Fix conflicting statements about initialization in developer notes.

  Context: https://github.com/bitcoin/bitcoin/pull/12785#issuecomment-378941151

Tree-SHA512: 601b18cbeb963f99a4180e652d6c1b78210df89743fd3565c0bce95fd2dcc9784b6af212795a43d3a40a5858b1a03e0d2c7982295c92d6ea710db0e6ee69f0b4
2020-07-21 11:54:50 -05:00
UdjinM6
6e592cc691
Merge pull request #3618 from PastaPastaPasta/backports-0.17-pr16
Backports 0.17 pr16
2020-07-20 18:03:11 +03:00
MarcoFalke
db747ea384 Merge #12757: Clarify include guard naming convention
3bcc0059b8 Add lint-include-guards.sh which checks include guard consistency (practicalswift)
8fd6af89a0 Fix missing or inconsistent include guards (practicalswift)
8af65d96f4 Document include guard convention (practicalswift)

Pull request description:

  * **Documentation**: Document include guard convention
  * **Fix**: Fix missing or inconsistent include guards
  * **Regression test**: Add `lint-include-guards.sh` which checks include guard consistency

Tree-SHA512: 8171878f60fd08ccbea943a11e835195750592abb9d7ab74eaa4265ae7fac523b1da9d31ca13d6ab73dd596e49986bfb7593c696e5f39567c93e610165bc2acc
Signed-off-by: pasta <pasta@dashboost.org>

# Conflicts:
#	src/bech32.h
#	src/consensus/merkle.h
#	src/key_io.h
#	src/policy/fees.h
#	src/rpc/server.h
#	src/script/bitcoinconsensus.h
#	src/wallet/coinselection.h
2020-07-19 00:42:54 -05:00
Wladimir J. van der Laan
52784085cb
Merge #12800: doc: Add note about our preference for scoped enumerations ("enum class")
0fee2b4 doc: Add note about our preference for scoped enumerations ("enum class") (practicalswift)

Pull request description:

  Add note about our preference for scoped enumerations (`enum class`).

  Context: #10742

Tree-SHA512: 0ab3465c2b734240cb38a05c2f6e75f1af54207a0f1a2e8115e7b367fd37e8966a2fc0240c6d4c2c66b6677b5f367eda4f4b783bbaa419777336c17f04adff06
2020-07-17 15:42:51 -05:00
Wladimir J. van der Laan
ea7575375d
Merge #12700: Document RPC method aliasing
4c317d89e Document RPC method aliasing (Russell Yanofsky)

Pull request description:

  Suggested by @Sjors in https://github.com/bitcoin/bitcoin/pull/11536#issuecomment-372820660

Tree-SHA512: 7bf16238e41b6c6c078e9103d8eac2ac76739a2c16b4f964be49bfde1f20f31a1fb30badf1faaa6ddc301a74f0d785d19567069b50de78c502144479143cb38c
2020-07-17 15:42:51 -05:00
MarcoFalke
f77b58e89f Merge #13312: docs: Add a note about the source code filename naming convention
e56771365b Do not use uppercase characters in source code filenames (practicalswift)
419a1983ca docs: Add a note about the source code filename naming convention (practicalswift)

Pull request description:

  Add a note about the source code filename naming convention.

Tree-SHA512: 8d329bd9e19bcd26e74b0862fb0bc2369b46095dbd3e69d34859908632763abd7c3d00ccc44ee059772ad4bae4460c2bcc1c0e22fd9d8876d57e5fcd346cea4b
2020-07-07 11:17:45 -05:00
Wladimir J. van der Laan
6e5b3f3b9e Merge #13230: Simplify include analysis by enforcing the developer guide's include syntax
16e3cd380af570fb2f656e0344bab88829a4bcda Clarify include recommendation (practicalswift)
6d10f43738d58bf623975e3124fd5735aac7d3e1 Enforce the use of bracket syntax includes ("#include <foo.h>") (practicalswift)
906bee8e5f474f8718d02e6f1938f20dcfe3d2cc Use bracket syntax includes ("#include <foo.h>") (practicalswift)

Pull request description:

  When analysing includes in the project it is often assumed that the preferred bracket include syntax (`#include <foo.h>`) mentioned in `developer-docs.md` is used consistently. @sipa:s excellent circular dependencies script [`circular-dependencies.py`](50c69b7801/contrib/devtools/circular-dependencies.py) (#13228) is an example of a script making this reasonable assumption.

  This PR enables automatic Travis checking of the include syntax making sure that the bracket syntax includes (`#include <foo.h>`) is used consistently.

Tree-SHA512: a414921aabe8e487ebed42f3f1cbd02fecd1add385065c1f2244cd602c31889e61fea5a801507ec501ef9bd309b05d3c999f915cec1c2b44f085bb0d2835c182
Signed-off-by: pasta <pasta@dashboost.org>
2020-07-01 20:12:03 -05:00
Wladimir J. van der Laan
28b02f42e1 Merge #13041: build: Add linter checking for accidental introduction of locale dependence
698cfd081144845f6246171b8a2a0cfa8daaecdb docs: Mention lint-locale-dependence.sh in developer-notes.md (practicalswift)
0a4ea2f4589961868ab4d25e0277485c31938e20 build: Add linter for checking accidental locale dependence (practicalswift)

Pull request description:

  This linter will check for code accidentally introducing locale dependencies.

  Unnecessary locale dependence can cause bugs that are very tricky to isolate and fix. We should avoid using locale dependent functions if possible.

  Context: https://github.com/bitcoin/bitcoin/pull/12881#issuecomment-378564722

  Example output:

  ```
  $ contrib/devtools/lint-locale-dependence.sh
  The locale dependent function tolower(...) appears to be used:
  src/init.cpp:    if (s[0] == '0' && std::tolower(s[1]) == 'x') {

  Unnecessary locale dependence can cause bugs that are very
  tricky to isolate and fix. Please avoid using locale dependent
  functions if possible.

  Advice not applicable in this specific case? Add an exception
  by updating the ignore list in contrib/devtools/lint-locale-dependence.sh
  ```

  **Note to reviewers:** What is the most appropriate `LOCALE_DEPENDENT_FUNCTIONS` function list? What should be added or removed?

Tree-SHA512: 14e448828804bb02bf59070647e38b52fce120c700c903a4a8472769a2cee5dd529bd3fc182386993cb8720482cf4250b63a0a477db61b941ae4babe5c65025f
2020-07-01 20:12:03 -05:00
MarcoFalke
e5b434d411 Merge #12982: Fix inconsistent namespace formatting guidelines
cd0e1e91dd Fix inconsistent namespace formatting guidelines (Russell Yanofsky)

Pull request description:

  Suggested formatting for namespaces in the developer guide is currently inconsistent. This commit updates the developer guide and clang-format configuration to consistently put "namespace" and opening/closing braces on the same line. Example:

  ```c++
  namespace boost {
  namespace signals2 {
  class connection;
  } // namespace signals2
  } // namespace boost
  ```

  Currently the [Source code organization](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#source-code-organization) section has an example like the one above, but the [Coding style](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#coding-style) section example and description put a newline between the opening "namespace foo" and brace (but oddly no newline between closing namespace and brace).

  Avoiding newlines before namespace opening braces makes nested declarations less verbose and also avoids asymmetry with closing braces. It's also a common style used in our own and other codebases:

  * https://google.github.io/styleguide/cppguide.html#Namespaces
  * https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Coding_Style#Classes
  * https://llvm.org/docs/CodingStandards.html#namespace-indentation

Tree-SHA512: 507259478e1a7f6f96db386dd04eb25aa04294f533503fdd82368cf809c3ceaa20204b2cb6ae65322eb27446e5934c1aa1ccb6240ead12aa06b314af76f68139
2020-06-27 10:43:32 -05:00
Jonas Schnelli
a0dac26647 Merge #12029: Build: Add a makefile target for Doxygen documentation
a777244e4 Build: Add a makefile target for Doxygen documentation (Andrea Comand)

Pull request description:

  You can now build the doxygen documentation with `make docs` and clean it with `make clean-docs`.

  Fixes: #11949

Tree-SHA512: f2361ec7f771227367dd04bba1a444b44e59f13901463a678a5f2f579a10a56d67db2e29552e754e312a1c472a31593b6af189cbaac5cd351a428c57baf5ace7
2020-06-17 14:29:55 -05:00
Wladimir J. van der Laan
78ef3fdd3a
Merge #12373: Build: Add build support for profiling.
cfaac2a60 Add build support for 'gprof' profiling. (murrayn)

Pull request description:

  Support for profiling build: `./configure --enable-profiling`

Tree-SHA512: ea983cfce385f1893bb4ab7f94ac141b7d620951dc430da3bbc92ae1357fb05521eac689216e66dc87040171a8a57e76dd7ad98036e12a2896cfe5ab544347f0
2020-06-14 11:41:08 -05:00
Wladimir J. van der Laan
9dd01a17a3
Merge #12260: [Trivial] link mentioned scripted-diff-commit (developer-doc)
7eb665fc8 [Trivial] link mentioned scripted-diff-commit (Felix Wolfsteller)

Pull request description:

  Make it easier for people who do not operate on a cloned repository to access the example mentioned.

Tree-SHA512: 1c06e551c68cad03e6bd541bf0e0076cdf0b48ef9b8b4e4a61435367c3435e2e4ccb934112e8dc29d3d70217d8834924704aaf839e25d1133312df86848ca1a1
2020-06-14 11:41:06 -05:00