Commit Graph

26838 Commits

Author SHA1 Message Date
Kittywhiskers Van Gogh
2d99be0aea
partial bitcoin#25331: Add HashWriter without ser-type and ser-version
includes:
- faa5425629d35708326b255570c51139aef0c8c4
2024-09-11 16:18:46 +00:00
pasta
96685be685
Merge #5167: refactor(stats): modernize statsd::StatsdClient, make global unique_ptr
cc998abec1 fmt: apply `clang-format-diff.py` suggestions (Kittywhiskers Van Gogh)
0401c581eb stats: `const`-ify variables and arguments (Kittywhiskers Van Gogh)
9f96723774 stats: stop using error codes, switch over to `bool` (Kittywhiskers Van Gogh)
1a81979c1e stats: initialize socket after we have a valid socket address (Kittywhiskers Van Gogh)
dbbfc8d766 stats: use `Socks` wrapper, use `CService` to generate our `sockaddr` (Kittywhiskers Van Gogh)
2def905044 stats: move init logic into constructor (Kittywhiskers Van Gogh)
4bc727cd6c stats: clean up randomization code, move `FastRandomContext` inward (Kittywhiskers Van Gogh)
840241eefd stats: cleanup error logging, improve code sanity (Kittywhiskers Van Gogh)
85890ddb13 docs: add copyright notice to source file, update notice in header (Kittywhiskers Van Gogh)
a9d1b1494d stats: move `_StatsdClientData` variables into `StatsdClient` (Kittywhiskers Van Gogh)
30c30c1397 stats: fetch all arguments needed when constructing `g_stats_client` (Kittywhiskers Van Gogh)
5133d88415 stats: s/statsClient/g_stats_client/g (Kittywhiskers Van Gogh)
f81951dd00 stats: make `statsClient` a `std::unique_ptr`, denote as global variable (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  Support for transmitting stats to a Statsd server has been courtesy of Statoshi ([repo](https://github.com/jlopp/statoshi)), implemented Dec, 2020 by [dash#2515](https://github.com/dashpay/dash/pull/2515) but since then, it hasn't gotten much attention aside from benefiting from codebase-wide changes and the occasional compiler appeasement. This pull request aims to give our statistics code some TLC.

  Changes include:

  * Limiting initialization to solely during construction and moving the responsibility of fetching arguments outside of `statsd::StatsdClient`.
  * Using the RAII `Socks` wrapper as early as possible (we still need to construct a raw socket ourselves but this is done in the initializer and control is moved to the wrapper and everywhere else, the wrapper is used)
  * Utilizing existing networking code to generate the socket address
    * This lets us trivially allow IPv6 connections as the responsibility to construct it safely is moved to `CService`.
  * Using `std::string` and our string manipulation capabilities (replacing `snprintf` with `strprintf`), replacing platform-specific types (replacing `short` with `uint16_t`).

  ## Breaking Changes

  None observed.

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests **(note: N/A)**
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  PastaPastaPasta:
    utACK [cc998ab](cc998abec1)
  UdjinM6:
    utACK cc998abec1

Tree-SHA512: 433c92160d6ac7ebb8582ada3cbb65ead7913618266b773619a528c90dfe0e286aafa46dc3b0bca62f246938e5948a732080e2cddba942d3627f007ca6efcc1f
2024-09-11 11:17:52 -05:00
pasta
f2645df7d3
Merge #6262: chore: add stats as a pull request header type and scope
4faf35f749 chore: add `stats` as a pull request header scope (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  Would allow tagging https://github.com/dashpay/dash/pull/5167, https://github.com/dashpay/dash/pull/6267 and https://github.com/dashpay/dash/pull/6237 as `feat(stats)`.

  ## Breaking Changes

  None.

  ## Checklist:

  - [x] I have performed a self-review of my own code **(note: N/A)**
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests **(note: N/A)**
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ **(note: N/A)**

ACKs for top commit:
  PastaPastaPasta:
    utACK 4faf35f749
  UdjinM6:
    utACK 4faf35f749
  thephez:
    utACK 4faf35f749

Tree-SHA512: 25cc28792351852b9e920d980b8814d93274bc53d22ce63fb1a5bf32821b10902d22b384a408e1c4a7b97239e53e235c45ac008d0f82e1afe5d6071b392acb47
2024-09-11 11:11:11 -05:00
Kittywhiskers Van Gogh
4faf35f749
chore: add stats as a pull request header scope 2024-09-11 15:08:28 +00:00
Kittywhiskers Van Gogh
cc998abec1
fmt: apply clang-format-diff.py suggestions 2024-09-11 15:02:10 +00:00
pasta
6019a8708f
Merge #6260: backport: merge bitcoin#20018, #22141, #22221, #22653, #23054, #23398, #23175, #22362, #23769, #23936, #24238, #24331 (auxiliary backports: part 15)
6c7335e002 merge bitcoin#24331: Revert back `MoveFileExW` call for MinGW-w64 (Kittywhiskers Van Gogh)
15e794bdd8 merge bitcoin#24238: use arc4random on OpenBSD (Kittywhiskers Van Gogh)
e039aecbdc merge bitcoin#23936: Add and use EnsureArgsman helper (Kittywhiskers Van Gogh)
b4bfacfd24 merge bitcoin#23769: Disallow copies of CChain (Kittywhiskers Van Gogh)
5b66688160 merge bitcoin#22362: Drop only invalid entries when reading banlist.json (Kittywhiskers Van Gogh)
109c963f6a merge bitcoin#23175: Add CJDNS network to -addrinfo and -netinfo (Kittywhiskers Van Gogh)
d57c96ea37 merge bitcoin#23398: add return message to savemempool RPC (Kittywhiskers Van Gogh)
22e59fb464 merge bitcoin#23054: Use C++11 member initializer in CTxMemPoolEntry (Kittywhiskers Van Gogh)
d158063b6d merge bitcoin#22653: Rename JoinErrors and re-use it (Kittywhiskers Van Gogh)
e24324d266 merge bitcoin#22221: Pass block reference instead of pointer to PeerManagerImpl::BlockRequested (Kittywhiskers Van Gogh)
68657efc03 merge bitcoin#22141: net processing: Remove hash and fValidatedHeaders from QueuedBlock (Kittywhiskers Van Gogh)
c0e6792e27 merge bitcoin#20018: ProcessAddrFetch(-seednode) is unnecessary if -connect is specified (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * When backporting [bitcoin#23054](https://github.com/bitcoin/bitcoin/pull/23054), `sigOpCount` and `nSigOpCountWithAncestors` were switched from `unsigned int` to `int64_t`. This change was done to prevent integer narrowing when accepting the `int64_t` taken from the constructor.

    This isn't a problem upstream as the same changes were as part of [bitcoin#8149](https://github.com/bitcoin/bitcoin/pull/8149/files#diff-8a2230436880b65a205db9299ab2e4e4adb1d4069146791b5db566f3fb752adaL90-L107), which was omitted but the type changes remain valid as sigop count cannot be a negative number.

  ## Breaking Changes

  None observed.

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK 6c7335e002
  UdjinM6:
    utACK 6c7335e002

Tree-SHA512: 29cae42dd82235305d3558562bae346170e742ce0b65897e396b331294b39cad0dd831fa9a09b34780a67844e55292e5b4e784cf544a894cc3f8897afe617ca1
2024-09-11 09:53:26 -05:00
Kittywhiskers Van Gogh
0401c581eb
stats: const-ify variables and arguments
We cannot convert `DEFAULT_STATSD*` to `std::string_view`s as they're
being used as default arguments and `GetArgs` expects `std::string`s
2024-09-11 14:37:58 +00:00
Kittywhiskers Van Gogh
9f96723774
stats: stop using error codes, switch over to bool 2024-09-11 14:34:53 +00:00
Kittywhiskers Van Gogh
1a81979c1e
stats: initialize socket after we have a valid socket address
As creating the socket is now the last step, we don't need
`m_init` anymore. We can just see if a socket is successfully
constructed and take that as our validity indicator.

We'll also move it out of the inner `send` function so we can bail out
before we bother with all the string processing and manipulation.
2024-09-11 14:34:20 +00:00
Kittywhiskers Van Gogh
dbbfc8d766
stats: use Socks wrapper, use CService to generate our sockaddr 2024-09-11 14:34:20 +00:00
Kittywhiskers Van Gogh
2def905044
stats: move init logic into constructor
Also, add log messages to inform us if we're skipping initialization or
it has successfully been initialized.
2024-09-11 14:34:19 +00:00
Kittywhiskers Van Gogh
4bc727cd6c
stats: clean up randomization code, move FastRandomContext inward
We can skip the computationally expensive dice-roll if our sample rate
is zero as that means we should never send it. Also get rid of the
`FastRandomContext::operator()` that isn't used anywhere else in the
codebase.
2024-09-11 14:34:19 +00:00
Kittywhiskers Van Gogh
840241eefd
stats: cleanup error logging, improve code sanity
- Use `uint16_t` instead of `short`, `int64_t` instead of `size_t`
- Get rid of the `errmsg` buffer and use `LogPrintf` to report errors
- Use `strprintf` instead of `snprintf`
- Rephrase networking error logs to allow inclusion of error strings
2024-09-11 14:33:55 +00:00
pasta
4d312b5456
Merge #6252: test: Improve tests robustness
0c5179462e test: rework feature_governance_cl.py (UdjinM6)
11ac0819da feat: bump_mocktime also bumps schedulers now (UdjinM6)
1937f503fe feat: regtest-only: do not auto-reset mnsync after 1h (UdjinM6)

Pull request description:

  ## Issue being fixed or feature implemented
  1. The original idea behind forced reset was to help desktop clients which go into sleep/hibernation mode from time to time to sync with no issues once they are online again. For regtest however it doesn't do anything good and only causes issues.
  2. We rely on schedulers a lot, bumping them should let nodes behave more like on a real network.
  3. Forcing mnsync to skip governance votes doesn't always work as we'd expect cause we don't control connection creation. To make it more deterministic create a proposal that should get into one superblock only. This way both the proposal and the trigger will expire/be deleted after another superblock meaning we can be 100% sure the isolated node never gets any of them and can only sync thanks to chainlock.

  ## What was done?

  ## How Has This Been Tested?
  run tests

  ## Breaking Changes
  n/a

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  knst:
    utACK 0c5179462e

Tree-SHA512: 9a40eeaba880f3f27f86736e92afa19a4ecb2a2d157bc42b65dd9da0d4109c9cd1d83a5abdf1ec16be2f8a8b31821fb700a7f0d2265c094fd4fdff7f18bc6ec7
2024-09-11 08:19:02 -05:00
pasta
48c7f98b1a
doc: drop trailing whitespace 2024-09-11 08:15:08 -05:00
UdjinM6
697743d77b
test: add missing import 2024-09-11 08:11:08 -05:00
pasta
cfe99fd289
docs: add release notes for 6239 2024-09-10 13:01:24 -05:00
UdjinM6
a6bbaacfaa
fix: GetHeadersLimit is used for getheaders(2) and headers(2), refactor it to accept compressed instead of msg_type 2024-09-10 12:57:55 -05:00
PastaPastaPasta
b224f3f6ca
bump p2p_version in tests
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2024-09-10 12:56:59 -05:00
UdjinM6
0c5179462e
test: rework feature_governance_cl.py 2024-09-10 18:36:40 +03:00
UdjinM6
11ac0819da
feat: bump_mocktime also bumps schedulers now 2024-09-10 18:36:40 +03:00
MarcoFalke
3c51f3ab1e
Merge bitcoin/bitcoin#23064: fuzz: Fix memory leak in system fuzz target
aaaa37abbab69fe1fdc2d332bbaf6ecce0c5cc00 fuzz: Fix memory leak in system fuzz target (MarcoFalke)

Pull request description:

  Might fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=36906

  To test:

  ```
  $ FUZZ=system valgrind --tool=massif ./src/test/fuzz/fuzz ../btc_qa_assets/fuzz_seed_corpus/system/
  ^C
  $ massif-visualizer ./massif.out.952024
  ```

ACKs for top commit:
  practicalswift:
    cr ACK aaaa37abbab69fe1fdc2d332bbaf6ecce0c5cc00

Tree-SHA512: 6aa47ea12ec76133ae326ed41c31414d7a418abe1f28b05f698e9bb5439bbe26b814c4827999b15b77b47608dbc71c9b35789d0b84e25f56928381d06d6460e5
2024-09-10 21:06:38 +05:30
UdjinM6
b423f42aae
refactor: sort imports 2024-09-10 10:25:06 -05:00
UdjinM6
f6c68ba71b
refactor: simplify _compute_requested_block_headers 2024-09-10 10:25:06 -05:00
pasta
07876b2c4a
use MAX_HEADERS_UNCOMPRESSED_RESULT not MAX_HEADERS_UNCOMPRESSED_RESULTS ; use MAX_HEADERS_UNCOMPRESSED_RESULT in RPC to avoid breaking changes 2024-09-10 10:25:05 -05:00
pasta
b137280df4
change to _COMPRESSED or _UNCOMPRESSED 2024-09-10 10:25:05 -05:00
UdjinM6
303bc7af99
fix: increase it for headers2 only 2024-09-10 10:25:03 -05:00
Kittywhiskers Van Gogh
e23410ffdd
trivial: rename MAX_HEADERS_RESULTS_NEW to MAX_HEADERS_RESULTS
We can reduce the diff by keeping the name `MAX_HEADERS_RESULTS` for
`MAX_HEADERS_RESULTS_NEW` as `MAX_HEADERS_RESULTS_OLD` is only
referenced once (in `GetHeadersLimit()`)
2024-09-10 10:23:54 -05:00
Kittywhiskers Van Gogh
bcf0320691
trivial: move the headers limit determination to GetHeadersLimit() 2024-09-10 10:23:53 -05:00
pasta
993c7c0f90
feat: increase the number of block headers able to be downloaded at once to 8000 in protocol version 70234 2024-09-10 10:23:36 -05:00
pasta
1464e69058
Merge #6148: feat: broadcast dsq messages using the inventory system
57fb0874ef feat: broadcast dsq messages using the inventory system (pasta)

Pull request description:

  ## Issue being fixed or feature implemented
  DSQ messages are 142 bytes.

  Previously, assuming a relatively highly connected masternode hosting 100 connection, each round of coinjoin will result in 14.2KB (100*142) of inbound and outbound traffic each.

  ## What was done?
  Now, using the inventory system, a message will first use 36 bytes per peer (sending and receiving), plus the size of a `getdata` message and the actual message itself. As a result, bandwidth usage for 1 round of mixing would be closer to 36 * 100 + 142 (dsq) + 36 (getdata) = ~3.8KB, a reduction of around ~73%

  ## How Has This Been Tested?
  Has not been; @UdjinM6 especially please review well :)

  ## Breaking Changes
  Does introduce a new protocol version, but in a backwards compatible way. I don't think this would need to be delayed to v22 for any reason.

  ## Checklist:
    _Go over all the following points, and put an `x` in all the boxes that apply._
  - [x] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

Top commit has no ACKs.

Tree-SHA512: 3dc39a339cba29d8cf207cec76ecace5ad0e11d1892ca0f65f9253a2b1d90313da21c6c178c2476756c5566ece0fab777006cd609b7984df906a9206c25d921d
2024-09-10 10:20:59 -05:00
pasta
57fb0874ef
feat: broadcast dsq messages using the inventory system
DSQ messages are 142 bytes.

Previously, assuming a relatively highly connected masternode hosting 100 connection, each round of coinjoin will result in 14.2KB (100*142) of inbound and outbound traffic each.

Now, using the inventory system, a message will first use 36 bytes per peer (sending and receiving), plus the size of a `getdata` message and the actual message itself. As a result, bandwidth usage for 1 round of mixing would be closer to 36 * 100 + 142 (dsq) + 36 (getdata) = ~3.8KB, a reduction of around ~73%
2024-09-10 09:23:22 -05:00
UdjinM6
1937f503fe
feat: regtest-only: do not auto-reset mnsync after 1h 2024-09-10 17:05:11 +03:00
pasta
6d426515a5
Merge #6225: feat: bury v20 fork - fire up test chains by first block - 4/n
23812555b1 fix: possible deadlock during calculation of signals for historical blocks during re-index (Konstantin Akimov)
1087489fd4 feat: bury v20 deployment (Konstantin Akimov)
64cedb30bd feat: actually test something EHF unit tests (Konstantin Akimov)
762a808b8c chore: drop irrelevant bip9 code from feature_llmq_rotation.py (Konstantin Akimov)
7735631aad fix: remove v20 from test feature_llmq_evo as far as mn_rr used (Konstantin Akimov)
ca83b26815 fix: crash in CreditPool: it meant to check that DIP0003 is activated (Konstantin Akimov)

Pull request description:

  ## Issue being fixed or feature implemented
  V20 is activated on mainnet: time to bury it!

  https://github.com/dashpay/dash/issues/6186

  ## What was done?
  Hard-fork v20 is buried and it requires to implement multiple fixes, simplifications, refactoring:
   - some tests for EHF moved from functional tests to unit tests
   - fixed crash in Credit Pool if DIP3 is not activated yet
   - added a requirement for v20 activation for `CMNHFManager::GetSignalsStage`
   - removed useless code from functional test feature_llmq_rotation
   - renamed variables "v20" to "mn_rr" in feature_llmq_evo.py so far as actually used fork is mn_rr

  ## How Has This Been Tested?
  Some unit and functional tests to succeed.

  Done reindex  (just in case):

      src/qt/dash-qt -reindex  -assumevalid=0
      src/qt/dash-qt -reindex  -assumevalid=0 -testnet

  ## Breaking Changes
  N/A

  ## Checklist:
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone

ACKs for top commit:
  UdjinM6:
    ACK 23812555b1
  PastaPastaPasta:
    utACK 23812555b1

Tree-SHA512: eec35745baa695f3f286d39b6a61fa0a9f34820b13d1dd4cfbd1efe707850283892c39bf7fe49c49c812e0c02465d64df11480b3f12aa7f21b59a71eeae7260e
2024-09-10 08:50:23 -05:00
Kittywhiskers Van Gogh
6c7335e002
merge bitcoin#24331: Revert back MoveFileExW call for MinGW-w64 2024-09-08 16:29:03 +00:00
Kittywhiskers Van Gogh
15e794bdd8
merge bitcoin#24238: use arc4random on OpenBSD 2024-09-08 16:24:37 +00:00
Kittywhiskers Van Gogh
e039aecbdc
merge bitcoin#23936: Add and use EnsureArgsman helper 2024-09-08 16:24:37 +00:00
Kittywhiskers Van Gogh
b4bfacfd24
merge bitcoin#23769: Disallow copies of CChain 2024-09-08 16:24:36 +00:00
Kittywhiskers Van Gogh
5b66688160
merge bitcoin#22362: Drop only invalid entries when reading banlist.json 2024-09-08 16:24:36 +00:00
Kittywhiskers Van Gogh
109c963f6a
merge bitcoin#23175: Add CJDNS network to -addrinfo and -netinfo 2024-09-08 16:24:36 +00:00
Kittywhiskers Van Gogh
d57c96ea37
merge bitcoin#23398: add return message to savemempool RPC 2024-09-08 16:24:36 +00:00
Kittywhiskers Van Gogh
22e59fb464
merge bitcoin#23054: Use C++11 member initializer in CTxMemPoolEntry 2024-09-08 16:24:36 +00:00
Kittywhiskers Van Gogh
d158063b6d
merge bitcoin#22653: Rename JoinErrors and re-use it 2024-09-08 15:35:22 +00:00
Kittywhiskers Van Gogh
e24324d266
merge bitcoin#22221: Pass block reference instead of pointer to PeerManagerImpl::BlockRequested 2024-09-08 15:35:22 +00:00
Kittywhiskers Van Gogh
68657efc03
merge bitcoin#22141: net processing: Remove hash and fValidatedHeaders from QueuedBlock 2024-09-08 15:35:21 +00:00
Kittywhiskers Van Gogh
c0e6792e27
merge bitcoin#20018: ProcessAddrFetch(-seednode) is unnecessary if -connect is specified 2024-09-08 15:35:20 +00:00
pasta
0c243ab4f5
Merge #6253: ci: fix, in github actions CI actually check out the PR
fb92a8ef7b ci: fix, in github actions CI, we don't actually check out the PR, but the base (pasta)

Pull request description:

  ## Issue being fixed or feature implemented
  Actually check out the PR for GitHub actions CI. I knew this was an issue for a lil bit since we implemented it. Time to fix it.

  Note: Github Actions based CI is experimental, and we still primarily use GitLab. Functionals tests don't even run on GitHub Actions CI yet.

  ## What was done?
  Check out the actual PR commit

  ## How Has This Been Tested?

  ## Breaking Changes
  None

  ## Checklist:
  - [ ] I have performed a self-review of my own code
  - [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have added or updated relevant unit/integration/functional/e2e tests
  - [ ] I have made corresponding changes to the documentation
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  UdjinM6:
    utACK fb92a8ef7b

Tree-SHA512: 31b3737328ac66217c66c99efce6a3ccfe9e8b9a2453a8b226c12c394e2433828d20165a9dfa4c9463ae4fa5015a7d07f451dde98e937a230e04087383230a02
2024-09-06 14:20:21 -05:00
Kittywhiskers Van Gogh
85890ddb13
docs: add copyright notice to source file, update notice in header 2024-09-06 19:08:19 +00:00
Kittywhiskers Van Gogh
a9d1b1494d
stats: move _StatsdClientData variables into StatsdClient
There doesn't seem to be a benefit to keeping these values in a separate
struct.
2024-09-06 19:08:19 +00:00
Kittywhiskers Van Gogh
30c30c1397
stats: fetch all arguments needed when constructing g_stats_client
There's little sense in passing a ref to `ArgsManager` just to set a few
values because we'll be `const`-ing them in an upcoming commit.
Arguments supplied are expected to last the lifetime of the program's
instance and there's little reason to keep re-fetching those values.
2024-09-06 19:08:19 +00:00