Commit Graph

148 Commits

Author SHA1 Message Date
Kittywhiskers Van Gogh
2b26a87874
merge bitcoin#28012: Allow FastRandomContext::randbytes for std::byte, Allow std::byte serialization 2024-02-28 13:37:36 -06:00
Kittywhiskers Van Gogh
4eeafa267c
partial bitcoin#27927: Allow std::byte and char Span serialization
includes:
- fa257bc8312b91c2d281f48ca2500d9cba353cc5
2024-02-28 13:37:36 -06:00
Kittywhiskers Van Gogh
e933d78a88
merge bitcoin#23438: Use spans of std::byte in serialize
continuation of de54b878 from dash#5574
2024-02-28 13:37:34 -06:00
Kittywhiskers Van Gogh
2c32a09f4e
merge bitcoin#21969: Switch serialize to uint8_t 2024-02-28 13:37:33 -06:00
Kittywhiskers Van Gogh
d0b4e560a6
merge bitcoin#21966: Remove double serialization; use software encoder for fee estimation
continuation of f946c68f8 from dash#4197
2024-02-28 13:37:33 -06:00
Kittywhiskers Van Gogh
d9a8ce2749
trivial: move GetSerializeSize away from Stream (Un)serialize functions
done to reduce the number of conflicts associated with the unexpected
forward decl, moved it just above WriteAutoBitSet and added comment to
clarify reason.
2024-02-28 13:37:30 -06:00
Konstantin Akimov
4aa197dbdb Merge #18673: scripted-diff: Sort test includes
fa4632c41714dfaa699bacc6a947d72668a4deef test: Move boost/stdlib includes last (MarcoFalke)
fa488f131fd4f5bab0d01376c5a5013306f1abcd scripted-diff: Bump copyright headers (MarcoFalke)
fac5c373006a9e4bcbb56843bb85f1aca4d87599 scripted-diff: Sort test includes (MarcoFalke)

Pull request description:

  When writing tests, often includes need to be added or removed. Currently the list of includes is not sorted, so developers that write tests and have `clang-format` installed will either have an unrelated change (sorting) included in their commit or they will have to manually undo the sort.

  This pull preempts both issues by just sorting all includes in one commit.

  Please be aware that this is **NOT** a change to policy to enforce clang-format or any other developer guideline or process. Developers are free to use whatever tool they want, see also #18651.

  Edit: Also includes a commit to bump the copyright headers, so that the touched files don't need to be touched again for that.

ACKs for top commit:
  practicalswift:
    ACK fa4632c41714dfaa699bacc6a947d72668a4deef
  jonatack:
    ACK fa4632c41714dfaa, light review and sanity checks with gcc build and clang fuzz build

Tree-SHA512: 130a8d073a379ba556b1e64104d37c46b671425c0aef0ed725fd60156a95e8dc83fb6f0b5330b2f8152cf5daaf3983b4aca5e75812598f2626c39fd12b88b180
2023-08-29 22:00:59 -05:00
fanquake
497ad8d4b5 Merge #18087: Get rid of VARINT default argument
0e0fa27acb74b4f0075afcf59a0dff51a21baddb Get rid of VARINT default argument (Pieter Wuille)

Pull request description:

  This removes the need for the non-strandard use of variadic macros.

ACKs for top commit:
  ryanofsky:
    Code review ACK 0e0fa27acb74b4f0075afcf59a0dff51a21baddb. Only change since last review reverting outdated documentation change from earlier version of pr
  jonatack:
    ACK 0e0fa27 code review, built/ran tests/bitcoind
  practicalswift:
    ACK 0e0fa27acb74b4f0075afcf59a0dff51a21baddb -- diff looks correct
  MarcoFalke:
    ACK 0e0fa27acb74b4f0075afcf59a0dff51a21baddb 📯

Tree-SHA512: 6e335e4b586d62112b7260a12481cd949d1b3bbdb83edf8db690348f0a01852e68504336ff3e072e5131a7c8cb404ef11a2f786f842b8d08bbf6ea0e688777b1
2023-05-24 12:43:57 -05:00
Kittywhiskers Van Gogh
f5f41e5348 merge bitcoin#19314: Use uint16_t instead of unsigned short 2022-10-20 16:08:45 -05:00
Wladimir J. van der Laan
bbaab7ca5c Merge #18167: Fix a violation of C++ standard rules where unions are used for type-punning
0653939ac130eddffe40c53ac418bea305d3bf82 Add static_asserts to ser_X_to_Y() methods (Samer Afach)
be94096dfb0c4862e2314cbae4120d7360b08ef2 Fix a violation of C++ standard rules that unions cannot be switched. (Samer Afach)

Pull request description:

  Type punning in C++ is not like C. As per the C++ standard, one cannot use unions to convert the bit type. A discussion about this can be found [here](https://stackoverflow.com/questions/25664848/unions-and-type-punning). In C++, a union is supposed to only hold one type at a time. It's intended to be used only as `std::variant`. Switching types is undefined behavior.

  In fact, C++20 has a special casting function, called [`bit_cast`](https://en.cppreference.com/w/cpp/numeric/bit_cast) that solved this problem.

  Why has it been working so far? Because some compilers tolerate using unions and switching types, like gcc. More information [here](https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#Type-punning).

  One important thing to mention is that performance is generally not affected by that memcpy. Compilers are smart enough to convert that to a memory cast when possible. But we have to do it the right way, otherwise, it's jut undefined behavior that depends on the compiler.

ACKs for top commit:
  practicalswift:
    ACK 0653939ac130eddffe40c53ac418bea305d3bf82
  elichai:
    ACK 0653939ac130eddffe40c53ac418bea305d3bf82
  laanwj:
    Code review ACK 0653939ac130eddffe40c53ac418bea305d3bf82
  kristapsk:
    ACK 0653939ac130eddffe40c53ac418bea305d3bf82

Tree-SHA512: f6e89de39fc964750429139bab6b5a1346f7060334b7afa020e315bdad8f8c195bce2b8a9e343f06e7fff175e2dfb1cdabfcb6fe405bea0febe4962f0cc62557
2022-10-20 11:47:02 -04:00
Wladimir J. van der Laan
d413109da5 Merge #13558: Drop unused GetType() from CSizeComputer
893628be0166b4096b6e52f516e0f65bb63a75a2 Drop minor GetSerializeSize template (Ben Woosley)
da74db0940720407fafaf3582bbaf9c81a4d3b4d Drop unused GetType() from CSizeComputer (Ben Woosley)

Pull request description:

  Based on conversation in #13462, it seems the serialization `GetType` has very narrow use/effect. In every case except for `CAddress`, which specifically relates to a network peer's address, not a wallet address etc., the serialized representation of an object is irrespective of its destination / type.

  This removes the unused `GetType` method from `CSizeComputer` as a step to further narrowing that use.

Tree-SHA512: e72b8e9e5160396691e05aeaee3aba5a57935a75bd5005cfcc7fb51c936f3d1728a397f999da5c36696506dd815fafa5c738f3894df8864f25f91f639eba9c3d
2022-05-30 01:11:03 -04:00
Wladimir J. van der Laan
089cabbd2d Merge #13719: Avoid creating a temporary vector for size-prefixed elements
84547fa6d408bdda1685f6d5972232bb19d97a7d Avoid creating a temporary vector for size-prefixed elements (Pieter Wuille)

Pull request description:

  This is a simple improvement to the PSBT serialization code, avoiding the need for temporary vectors everywhere.

Tree-SHA512: 9f7243b7169ec8ba00ffad31af03c016ab84e4f76ebac810167f91f5e8008f3827ad59fbcee0cb2bd2334fc26466eb222404af24e7fb6ec040fd78229ebe0fd1
2022-04-26 13:27:11 -04:00
PastaPastaPasta
fe0ebb3c04
refactor: fix numerous compilation warnings (#4682)
* style: use clang-tidy style named parameters

* refactor: make IsTimeOutOfBounds testable by having current time be a parameter

* style: use x-> not (*x).

* refactor: make SelectCoinsGroupedByAddresses return a vector, remove out param

previous semantics was return false if the vecTally vector was empty. Now we just let the caller check if it is empty or not

* refactor: fix some sign-compare warnings

* refactor: consistently pre-declare stuff as struct / class inline with underlying type

* refactor: don't return const bool

* refactor: use ref to string

* refactor: use = default for CompactTallyItem

* refactor: adjust "initialization" ordering

* refactor: adjust how we handle negatives in GetProjectedMNPayees, use std::min

* refactor: don't bind a reference to a temporary value

* refactor: use a ref

* refactor: ensure attempt in SelectMemberForRecovery is non-negative.

* refactor: remove unused this capture

* refactor: fix numerous sign-compare warnings

* refactor: more consistently use size_t, use empty()
2022-02-11 19:15:26 +03:00
PastaPastaPasta
72e1d79fb8
refactor: misc spork refactoring (#4620)
* Use string_view instead of string for SERIALIZATION_VERSION_STRING. Also introduces string_view serialization. Also use string_view where (trivially) possible for params

* separate GetSporks and sporks net msg processing into its own method

* refactor: spork, use c++17 if-init

* refactor: spork, remove unused params, make param pt

* refactor: spork, use simplier construction

* refactor: spork, make strError part of if-init, also w/o redundant initialization

* refactor: spork, use structured binding

* refactor: spork, use for instead of while

* clang-format: spork.cpp

* resolve cppcheck linter warning about unused variable
2021-12-17 20:22:11 +03:00
MarcoFalke
3c5dcb036a Merge #16659: refactoring: Remove unused includes
084e17cebd424b8e8ced674bc810eef4e6ee5d3b Remove unused includes (practicalswift)

Pull request description:

  As requested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/16273#issuecomment-521332089:

  This PR removes unused includes.

  Please note that in contrast to #16273 I'm limiting the scope to the trivial cases of pure removals (i.e. no includes added) to make reviewing easier.

  I'm seeking "Concept ACK":s for this obviously non-urgent minor cleanup.

  Rationale:
  * Avoids unnecessary re-compiles in case of header changes.
  * Makes reasoning about code dependencies easier.
  * Reduces compile-time memory usage.
  * Reduces compilation time.
  * Warm fuzzy feeling of being lean :-)

ACKs for top commit:
  ryanofsky:
    Code review ACK 084e17cebd424b8e8ced674bc810eef4e6ee5d3b. PR only removes include lines and it still compiles. In the worst case someone might have to explicitly add an include later for something now included implicitly. But maybe some effort was taken to avoid this, and it wouldn't be a tragedy anyway.

Tree-SHA512: 89de56edc6ceea4696e9579bccff10c80080821685b9fb4e8c5ef593b6e43cf662f358788701bb09f84867693f66b2e4db035b92b522a0a775f50b7ecffd6a6d
2021-12-12 16:44:54 -05:00
PastaPastaPasta
a37a63ffd6
masternode-meta.*: add thread annotations, atomic usage, remove unneeded locks (#4466)
* masternode-meta.*: add thread annotations, atomic usage, remove unneeded locks

* masternode-meta.cpp lock cs for metaInfos

* Implement (un)serialization for atomic

* partial revert

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-10-01 00:00:52 +03:00
Kittywhiskers Van Gogh
680067ce7a
merge #19954: Complete the BIP155 implementation and upgrade to TORv3 2021-05-29 23:24:52 +03:00
UdjinM6
d60a6d1729
Convert bitset serializationis to use formatters 2021-05-28 12:31:13 +03:00
Wladimir J. van der Laan
9029448233
partial merge #19032: Serialization improvements: final step
71f016c6eb42e1ac2c905e04ba4d20c2009e533f Remove old serialization primitives (Pieter Wuille)
92beff15d3ae2646c00bd78146d7592a7097ce9c Convert LimitedString to formatter (Pieter Wuille)
ef17c03e074b6c3f185afa4eff572ba687c2a171 Convert wallet to new serialization (Pieter Wuille)
65c589e45e8b8914698a0fd25cd5aafdda30869c Convert Qt to new serialization (Pieter Wuille)

Pull request description:

  This is the final step 🥳 of the serialization improvements extracted from #10785.

  It converts the LimitedString wrapper to a new-style formatter, and updates the wallet and Qt code to use the new serialization framework. Finally all remaining old primitives are removed.

ACKs for top commit:
  jonatack:
    ACK 71f016c6eb42e1ac2 reviewed diff, builds/tests/re-fuzzed.
  laanwj:
    Code review ACK 71f016c6eb42e1ac2c905e04ba4d20c2009e533f

Tree-SHA512: d952194bc73259f6510bd4ab1348a1febbbf9862af30f905991812fb0e1f23f15948cdb3fc662be54d648e8f6d95b11060055d2e7a8c2cb5bf008224870b1ea1
2021-05-28 11:39:25 +03:00
MarcoFalke
6a36c4e4b1
partial merge #18317: Serialization improvements step 6 (all except wallet/gui)
f9ee0f37c28f604bc82dab502ce229c66ef5b3b9 Add comments to CustomUintFormatter (Pieter Wuille)
4eb5643e3538863c9d2ff261f49a9a1b248de243 Convert everything except wallet/qt to new serialization (Pieter Wuille)
2b1f85e8c52c8bc5a17eae4c809eaf61d724af98 Convert blockencodings_tests to new serialization (Pieter Wuille)
73747afbbeb013669faf4c4d2c0903cec4526fb0 Convert merkleblock to new serialization (Pieter Wuille)
d06fedd1bc26bf5bf2b203d4445aeaebccca780e Add SER_READ and SER_WRITE for read/write-dependent statements (Russell Yanofsky)
6f9a1e5ad0a270d3b5a715f3e3ea0911193bf244 Extend CustomUintFormatter to support enums (Russell Yanofsky)
769ee5fa0011ae658770586442715452a656559d Merge BigEndian functionality into CustomUintFormatter (Pieter Wuille)

Pull request description:

  The next step of changes from #10785.

  This:
  * Adds support for enum serialization to `CustomUintFormatter`, used in `CAddress` for service flags.
  * Merges `BigEndian` into `CustomUintFormatter`, used in `CNetAddr` for port numbers.
  * Converts everything (except wallet and gui) to use the new serialization framework.

ACKs for top commit:
  MarcoFalke:
    re-ACK f9ee0f37c2, only change is new documentation commit for CustomUintFormatter 📂
  ryanofsky:
    Code review ACK f9ee0f37c28f604bc82dab502ce229c66ef5b3b9. Just new commit adding comment since last review
  jonatack:
    Code review re-ACK f9ee0f37c28f604bc82dab502ce229c6 only change since last review is an additional commit adding Doxygen documentation for `CustomUintFormatter`.

Tree-SHA512: e7a0a36afae592d5a4ff8c81ae04d858ac409388e361f2bc197d9a78abca45134218497ab2dfd6d031e0cce0ca586cf857077b7c6ce17fccf67e2d367c1b6cd4
2021-05-28 11:39:24 +03:00
Kittywhiskers Van Gogh
6a25151bbd Merge #16730: Support serialization of std::vector<bool> 2021-05-18 22:31:54 +05:30
Kittywhiskers Van Gogh
97caca3033
Merge #18112: Convert blockencodings.h to new serialization framework
353f376277

Partial #18112: Add CustomUintFormatter

e574fff53e

Partial #18112: Add DifferenceFormatter

10633398f2

Partial #18112: Make VectorFormatter support stateful formatters

56dd9f04c7

Partial #18112: Convert CCompactSize to proper formatter

3ca574cef0
2021-04-18 17:10:25 -04:00
Kittywhiskers Van Gogh
bfe5971359
Merge #17957: Convert compression.h to new serialization framework
4de934b9b5

Partial #17957: Add FORMATTER_METHODS, similar to SERIALIZE_METHODS, but for formatters

ca34c5cba5
2021-04-18 17:10:25 -04:00
Kittywhiskers Van Gogh
0802bdfa37
Merge #18021: Convert undo.h to new serialization framework
3c94b0039d

Partial #18021: Make std::vector and prevector reuse the VectorFormatter logic

3cd8ab9d11

Partial #18021: Add custom vector-element formatter

abf8624356

Partial #18021: Add a constant for the maximum vector allocation (5 Mbyte)

37d800bea016d5cba5635db036f53a486614ed30
2021-04-18 17:08:20 -04:00
Kittywhiskers Van Gogh
46cb6e23dc
Merge #17896: Convert chain to new serialization
9b66083788

Partial #17896: Convert VARINT to the formatter/Using approach

2f1b2f4ed0

Partial #17896: Add a generic approach for (de)serialization of objects

ca62563df3
2021-04-18 17:03:22 -04:00
Kittywhiskers Van Gogh
e483a689e3
Merge #17850: Convert addrdb/addrman to new serialization
9250a087d2

Partial #17850: Introduce new serialization macros without casts

ca33451535
2021-04-18 17:02:40 -04:00
Wladimir J. van der Laan
6ca09aaae3
Merge #9753: Add static_assert to prevent VARINT(<signed value>)
499d95e27 Add static_assert to prevent VARINT(<signed value>) (Russell Yanofsky)

Pull request description:

  Using VARINT with signed types is dangerous because negative values will appear to serialize correctly, but then deserialize as positive values mod 128.

  This commit changes the VARINT macro to trigger a compile error by default if called with an signed value, and it updates existing broken uses of VARINT to pass a special flag that lets them keep working with no changes in behavior.

  There is some discussion about this issue here: https://github.com/bitcoin/bitcoin/pull/9693#issuecomment-278701473. I think another good change along these lines would be to make `GetSizeOfVarInt` and `WriteVarInt` throw exceptions if they are passed numbers less than 0 to serialize. But unlike this change, that would be a change in runtime behavior, and need more consideration.

Tree-SHA512: 082c65598cfac6dc1da042bdb47dbc9d5d789fc849fe52921cc238578588f4e5ff976c8b4b2ce42cb75290eb14f3b42ea76e26202c223c5b2aa63ef45c2ea3cc
2020-12-17 15:20:31 +03:00
Wladimir J. van der Laan
f8ac5a555b
Merge #12916: Introduce BigEndian wrapper and use it for netaddress ports
ece88fd Introduce BigEndian wrapper and use it for netaddress ports (Pieter Wuille)

Pull request description:

  This is another small improvement taken from #10785.

  Instead of manually converting from/to BE format in the `CService` serializer, provide a generic way in serialize.h to serialize BE data (only 16 bits for now).

Tree-SHA512: bd67cf7eed465dad08551fb62f659e755e0691e4597a9f59d285d2b79975b50e5710d35a34a185b5ad232e1deda9a4946615f9132b1ed7d96ed8087f73ace66b
2020-12-17 12:02:44 +03:00
Wladimir J. van der Laan
4bb1132438
Merge #12886: Introduce Span type and use it instead of FLATDATA
9272d70 Support serializing Span<unsigned char> and use that instead of FLATDATA (Pieter Wuille)
833bc08 Add Slice: a (pointer, size) array view that acts like a container (Pieter Wuille)

Pull request description:

  Introduce a new data type `Span`, which is an encapsulated pointer + size (like C++20's `std::span` or LevelDB's `Slice`), and represents a view to a sequence of objects laid out continuously in memory.

  The immediate use case is replacing the remaining `FLATDATA` invocations. Instead of those, we support serializing/deserializing unsigned char `Span`s (treating them as arrays).

  A longer term goal for `Span`s is making the script execution operate on them rather than on `CScript` itself. This will allow separate storage mechanisms for scripts.

Tree-SHA512: 7b0da3c802e5df367f223275004d16b04262804c007b7c73fda927176f0a9c3b2ef3225fa842cb73500b0df73175ec1419f1f5239de2402e21dd9ae8e5d05233
2020-12-17 11:21:20 +03:00
Pieter Wuille
ec71097801
Merge #12683: Fix more constness violations in serialization code
172f5fa738 Support deserializing into temporaries (Pieter Wuille)
2761bca997 Merge READWRITEMANY into READWRITE (Pieter Wuille)

Pull request description:

  This is another fragment of improvements from #10785.

  The current serialization code does not support serializing/deserializing from/to temporaries (like `s >> CFlatData(script)`). As a result, there are many invocations of the `REF` macro which in addition to changing the reference type also changes the constness. This is unnecessary in C++11 as we can use rvalue references now instead.

  The first commit is an extra simplification we can make that removes the duplication of code between `READWRITE` and `READWRITEMANY` (and related functions).

Tree-SHA512: babfa9cb268cc3bc39917e4f0a90e4651c33d85032161e16547a07f3b257b7ca7940e0cbfd69f09439d26fafbb1a6cf6359101043407e2c7aeececf7f20b6eed
2020-12-17 05:07:07 +03:00
Wladimir J. van der Laan
ca268bd678
Merge #12740: Add native support for serializing char arrays without FLATDATA
a7c45bc Add native support for serializing char arrays without FLATDATA (Pieter Wuille)

Pull request description:

  Support is added to serialize arrays of type `char` or `unsigned char` directly, without any wrappers. All invocations of the `FLATDATA` wrappers that are obsoleted by this are removed.

  This includes a patch by @ryanofsky to make `char` casting type safe.

  The serialization of `CSubNet` is changed to serialize a `bool` directly rather than though `FLATDATA`. This makes the serialization independent of the size of the bool type (and will use 1 byte everywhere).

  This is a small change taken from #10785.

Tree-SHA512: a41f61ca5fdc2fadb2d0e1702351a58a23841d551f505292a9542602cdb19f90d8944b8df14b872810a56bd201648fa4c0e958f3e9427fe829886284e85b9bfd
Signed-off-by: pasta <pasta@dashboost.org>

# Conflicts:
#	src/test/serialize_tests.cpp
2020-10-22 11:36:39 -04:00
pasta
c49e506176 scripted-diff: Merge #13753: Remove trailing whitespaces
-BEGIN VERIFY SCRIPT-

sed --in-place'' --regexp-extended 's/[[:space:]]+$//g' $(git grep -I --files-with-matches --extended-regexp '[[:space:]]+$' -- src test  ':!*.svg' ':!src/crypto/sha256_sse4*' ':!src/leveldb' ':!src/qt/locale' ':!src/secp256k1' ':!src/univalue')

-END VERIFY SCRIPT-

Signed-off-by: pasta <pasta@dashboost.org>
2020-07-28 20:23:12 -05:00
Wladimir J. van der Laan
06a1e20d8a Merge #13580: build: Detect if char equals int8_t
49d1f4cdd Detect if char equals int8_t (Chun Kuan Lee)

Pull request description:

  Probably fixes #13576. I'm not able to test this. @stacepellegrino, can you test this?

Tree-SHA512: b750e00e11e6b6f6341fec668ec2254cc101c8ebdd4878f320d6cb3b07cf326761146e4ceff0b6405b7e503ff64c093a8274bd524a097e2c49382dc296972c4f
2020-07-08 18:32:05 -05:00
Wladimir J. van der Laan
96e2c5a25f
Merge #12731: Support serialization as another type without casting
818dc74 Support serialization as another type without casting (Pieter Wuille)

Pull request description:

  This adds a `READWRITEAS(type, obj)` macro which serializes `obj` as if it were converted to `const type&` when `const`, and to `type&` when non-`const`. No actual cast is involved, so this only works when this conversion can be done automatically.

  This makes it usable in serialization code that uses a single implementation for both serialization and deserializing, which doesn't know the constness of the object involved.

  This is a redo of #12712, using a slightly different interface.

Tree-SHA512: 262f0257284ff99b5ffaec9b997c194e221522ba35c3ac8eaa9bb344449d7ea0a314de254dc77449fa7aaa600f8cd9a24da65aade8c1ec6aa80c6e9a7bba5ca7
2020-06-11 23:20:48 -05:00
MarcoFalke
a98db86ada Merge #10493: Use range-based for loops (C++11) when looping over map elements
680bc2cbb Use range-based for loops (C++11) when looping over map elements (practicalswift)

Pull request description:

  Before this commit:

  ```c++
  for (std::map<T1, T2>::iterator x = y.begin(); x != y.end(); ++x) {
      T1 z = (*x).first;
      …
  }
  ```

  After this commit:

  ```c++
  for (auto& x : y) {
      T1 z = x.first;
      …
  }
  ```

Tree-SHA512: 954b136b7f5e6df09f39248a6b530fd9baa9ab59d7c2c7eb369fd4afbb591b7a52c92ee25f87f1745f47b41d6828b7abfd395b43daf84a55b4e6a3d45015e3a0
2020-04-01 12:43:18 -05:00
PastaPastaPasta
8a1ec935a0
Backport 11651 (#3358)
* scripted-diff: Replace #include "" with #include <> (ryanofsky)

-BEGIN VERIFY SCRIPT-
for f in \
  src/*.cpp \
  src/*.h \
  src/bench/*.cpp \
  src/bench/*.h \
  src/compat/*.cpp \
  src/compat/*.h \
  src/consensus/*.cpp \
  src/consensus/*.h \
  src/crypto/*.cpp \
  src/crypto/*.h \
  src/crypto/ctaes/*.h \
  src/policy/*.cpp \
  src/policy/*.h \
  src/primitives/*.cpp \
  src/primitives/*.h \
  src/qt/*.cpp \
  src/qt/*.h \
  src/qt/test/*.cpp \
  src/qt/test/*.h \
  src/rpc/*.cpp \
  src/rpc/*.h \
  src/script/*.cpp \
  src/script/*.h \
  src/support/*.cpp \
  src/support/*.h \
  src/support/allocators/*.h \
  src/test/*.cpp \
  src/test/*.h \
  src/wallet/*.cpp \
  src/wallet/*.h \
  src/wallet/test/*.cpp \
  src/wallet/test/*.h \
  src/zmq/*.cpp \
  src/zmq/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>

* scripted-diff: Replace #include "" with #include <> (Dash Specific)

-BEGIN VERIFY SCRIPT-
for f in \
  src/bls/*.cpp \
  src/bls/*.h \
  src/evo/*.cpp \
  src/evo/*.h \
  src/governance/*.cpp \
  src/governance/*.h \
  src/llmq/*.cpp \
  src/llmq/*.h \
  src/masternode/*.cpp \
  src/masternode/*.h \
  src/privatesend/*.cpp \
  src/privatesend/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-

Signed-off-by: Pasta <pasta@dashboost.org>

* build: Remove -I for everything but project root

Remove -I from build system for everything but the project root,
and built-in dependencies.

Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/Makefile.test.include

* qt: refactor: Use absolute include paths in .ui files

* qt: refactor: Changes to make include paths absolute

This makes all include paths in the GUI absolute.

Many changes are involved as every single source file in
src/qt/ assumes to be able to use relative includes.

Signed-off-by: Pasta <pasta@dashboost.org>

# Conflicts:
#	src/qt/dash.cpp
#	src/qt/optionsmodel.cpp
#	src/qt/test/rpcnestedtests.cpp

* test: refactor: Use absolute include paths for test data files

* Recommend #include<> syntax in developer notes

* refactor: Include obj/build.h instead of build.h

* END BACKPORT #11651 Remove trailing whitespace causing travis failure

* fix backport 11651

Signed-off-by: Pasta <pasta@dashboost.org>

* More of 11651

* fix blockchain.cpp

Signed-off-by: pasta <pasta@dashboost.org>

* Add missing "qt/" in includes

* Add missing "test/" in includes

* Fix trailing whitespaces

Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: MeshCollider <dobsonsa68@gmail.com>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-03-20 01:46:56 +03:00
Wladimir J. van der Laan
fdf3f25a0a
Merge #10969: Declare single-argument (non-converting) constructors "explicit"
64fb0ac Declare single-argument (non-converting) constructors "explicit" (practicalswift)

Pull request description:

  Declare single-argument (non-converting) constructors `explicit`.

  In order to avoid unintended implicit conversions.

  For a more thorough discussion, see ["C.46: By default, declare single-argument constructors explicit"](http://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#c46-by-default-declare-single-argument-constructors-explicit) in the C++ Core Guidelines (Stroustrup & Sutter).

Tree-SHA512: e0c6922e56b11fa402621a38656d8b1122d16dd8f160e78626385373cf184ac7f26cb4c1851eca47e9b0dbd5e924e39a85c3cbdcb627a05ee3a655ecf5f7a0f1
2020-01-10 10:33:57 -06:00
Wladimir J. van der Laan
2be67c7605 Merge #12324: speed up Unserialize_impl for prevector
86b47fa741408b061ab0bda784b8678bfd7dfa88 speed up Unserialize_impl for prevector (Akio Nakamura)

Pull request description:

  The unserializer for prevector uses `resize()` for reserve the area, but it's prefer to use `reserve()` because `resize()` have overhead to call its constructor many times.

  However, `reserve()` does not change the value of `_size` (a private member of prevector).

  This PR make the logic of read from stream to callback function, and prevector handles initilizing new values with that call-back and ajust the value of `_size`.

  The changes are as follows:
  1. prevector.h
  Add a public member function named 'append'.
  This function has 2 params, number of elemenst to append and call-back function that initilizing new appended values.

  2. serialize.h
  In the following two function:
  - `Unserialize_impl(Stream& is, prevector<N, T>& v, const unsigned char&)`
  - `Unserialize_impl(Stream& is, prevector<N, T>& v, const V&)`
  Make a callback function from each original logic of reading values from stream, and call prevector's `append()`.

  3. test/prevector_tests.cpp
  Add a test for `append()`.

  ## A benchmark result is following:
  [Machine]
  MacBook Pro (macOS 10.13.3/i7 2.2GHz/mem 16GB/SSD)

  [result]
  DeserializeAndCheckBlockTest  => 22% faster
  DeserializeBlockTest => 29% faster

  [before PR]
      # Benchmark, evals, iterations, total, min, max, median
      DeserializeAndCheckBlockTest, 60, 160, 94.4901, 0.0094644, 0.0104715, 0.0098339
      DeserializeBlockTest, 60, 130, 65.0964, 0.00800362, 0.00895134, 0.00824187

  [After PR]
      # Benchmark, evals, iterations, total, min, max, median
      DeserializeAndCheckBlockTest, 60, 160, 77.1597, 0.00767013, 0.00858959, 0.00805757
      DeserializeBlockTest, 60, 130, 49.9443, 0.00613926, 0.00691187, 0.00635527

ACKs for top commit:
  laanwj:
    utACK 86b47fa741408b061ab0bda784b8678bfd7dfa88

Tree-SHA512: 62ea121ccd45a306fefc67485a1b03a853435af762607dae2426a87b15a3033d802c8556e1923727ddd1023a1837d0e5f6720c2c77b38196907e750e15fbb902
2019-10-02 15:25:27 +02:00
Pieter Wuille
526036ead8
Merge #9804: Fixes subscript 0 (&var[0]) where should use (var.data()) instead.
30ac7688e Fix subscript[0] potential bugs in key.cpp (Jeremy Rubin)
4b1c0f2e2 Remove unnecessary branches in utilstrencodings string constructors. (Jeremy Rubin)
e19db7b5a Fix subscript[0] in utilstrencodings.cpp (Jeremy Rubin)
bc2e7fd98 Fix subscript[0] in streams.h (Jeremy Rubin)
4cac0d1e0 Fix subscript[0] in validation.cpp (Jeremy Rubin)
ac658e55f Fix subscript[0] in torcontrol (Jeremy Rubin)
b6856ebed Fix subscript[0] in netaddress.cpp (Jeremy Rubin)
361d95265 Fix subscript[0] in base58.cpp (Jeremy Rubin)
6896dbf16 Cleanup (safe, it was checked) subscript[0] in MurmurHash3 (and cleanup MurmurHash3 to be more clear). (Jeremy Rubin)
96f2119e6 Fix subscript[0] in compressor.cpp (Jeremy Rubin)
500710bd2 Fix 2 subscript[0] bugs in pubkey.cpp, and eliminate one extra size check (Jeremy Rubin)
e0451e3e2 Fix subscript[0] bug in net.cpp if GetGroup returns a 0-sized vector (Jeremy Rubin)

Tree-SHA512: 5b9103652cf8c615bd8f4f32b3573d291d6b67c39e0308ce00100bc6625f346e8e016b4c999f4f34f5c37ae059490a83c3b513deb21f838af785227d06e02362
2019-07-17 19:01:41 -05:00
UdjinM6
09d66c7764
Fix compiler warning (#2956)
```
./serialize.h:762:49: warning: static_assert with no message is a C++17 extension [-Wc++17-extensions]
    static_assert(is_serializable_enum<T>::value);
                                                ^
                                                , ""
```
2019-05-30 11:00:31 +03:00
Alexander Block
186f7d2861 Implement support for explicit enum serialization
This allows to serialize enum values, but only if a specialized class of
is_serializable_enum is found which is derived from std::true_type.
2019-05-29 08:06:11 +02:00
Pieter Wuille
632956a806
Merge #9693: Prevent integer overflow in ReadVarInt.
45f0961 Prevent integer overflow in ReadVarInt. (Gregory Maxwell)

Tree-SHA512: 385ea0efb6b59d44c45a49227e5f6fff236b4775544cbeb236312a3fd87fd75c226ac56f7aa1bca66b853639da75a579610074f7582f92cf2ebd4a74bc40f6f0
2019-05-19 20:43:01 -05:00
Alexander Block
241f76f9bf Collection of minor performance optimizations (#2855)
* Merge #13176: Improve CRollingBloomFilter performance: replace modulus with FastMod

9aac9f90d5e56752cc6cbfac48063ad29a01143c replace modulus with FastMod (Martin Ankerl)

Pull request description:

  Not sure if this is optimization is necessary, but anyway I have some spare time so here it is. This replaces the slow modulo operation with a much faster 64bit multiplication & shift. This works when the hash is uniformly distributed between 0 and 2^32-1. This speeds up the benchmark by a factor of about 1.3:

  ```
  RollingBloom, 5, 1500000, 3.73733, 4.97569e-07, 4.99002e-07, 4.98372e-07 # before
  RollingBloom, 5, 1500000, 2.86842, 3.81630e-07, 3.83730e-07, 3.82473e-07 # FastMod
  ```

  Be aware that this changes the internal data of the filter, so this should probably
  not be used for CBloomFilter because of interoperability problems.

Tree-SHA512: 04104f3fb09f56c9d14458a6aad919aeb0a5af944e8ee6a31f00e93c753e22004648c1cd65bf36752b6addec528d19fb665c27b955ce1666a85a928e17afa47a

* Use unordered_map in CSporkManager

In one of my profiling sessions with many InstantSend transactions
happening, calls into CSporkManager added up to about 1% of total CPU time.
This is easily avoidable by using unordered maps.

* Use std::unordered_map instead of std::map in limitedmap

* Use unordered_set for CNode::setAskFor

* Add serialization support for unordered maps and sets

* Use unordered_map for mapArgs and mapMultiArgs

* Let limitedmap prune in batches and use unordered_multimap

Due to the batched pruning, there is no need to maintain an ordered map
of values anymore. Only when nPruneAfterSize, there is a need to create
a temporary ordered vector of values to figure out what can be removed.

* Instead of using a multimap for mapAskFor, use a vector which we sort on demand

CNode::AskFor will now push entries into an initially unordered vector
instead of an ordered multimap. Only when we later want to use vecAskFor in
SendMessages, we sort the vector.

The vector will actually be mostly sorted in most cases as insertion order
usually mimics the desired ordering. Only the last few entries might need
some shuffling around. Doing the sort on-demand should be less wasteful
then trying to maintain correct order all the time.

* Fix compilation of tests

* Fix limitedmap tests

* Rename limitedmap to unordered_limitedmap to ensure backports conflict

This ensures that future backports that depends on limitedmap's ordering
conflict so that we are made aware of needed action.

* Fix compilation error on Travis
2019-04-11 15:42:14 +03:00
Alexander Block
b18f8cb77c
Implement MNAUTH and allow unlimited inbound MN connections (#2790)
* Sort evo/* source files in Makefile.am

* Keep track of proRegTxHash in CConnman::masternodeQuorumNodes map

We will later need the proRegTxHash

* Fix serialization of std::tuple with const rvalue elements

Having serialization and deserialization in the same specialized template
results in compilation failures due to the "if(for_read)" branch.

* Implement MNAUTH message

This allows masternodes to authenticate themself.

* Protect fresh incoming connections for a second from eviction

Give fresh connections some time to do the VERSION/VERACK handshake and
an optional MNAUTH when it's a masternode. When an MNAUTH happened, the
incoming connection is then forever protected against eviction.

If a timeout of 1 second occurs or the first message after VERACK is not
MNAUTH, the node is not protected anymore and becomes eligable for
eviction.

* Avoid connecting to masternodes if an incoming connection is from the same one

Now that incoming connections from MNs authenticate them self, we can avoid
connecting to the same MNs through intra-quorum connections.

* Apply review suggestions
2019-03-22 11:52:37 +01:00
Alexander Block
a69a5cf4a0 Use smaller (3 out of 5) quorums for regtest/Travis (#2774)
* Require only 3 out of 5 signatures for old InstantSend in regtest mode

* Use LLMQs of size 5 with threshold of 3 for regtest

* Fix wrong check for out-of-range bits in CFixedBitSet

* Reduce number of masternodes in masternode/LLMQ tests

* Add missing \n to LogPrintf call

* Use correct indexes for isolated/receiver/sender nodes

The way it was before resulted in nodes 1-3 being unused and 6-8 being used
for these 3 special nodes even though these are masternodes.

* Avoid stopping/starting isolated node in p2p-instantsend.py

It's enough to disable networking for this node.
2019-03-15 11:48:24 +03:00
Alexander Block
49de417265 Implement CFixedVarIntsBitSet and CAutoBitSet 2019-01-17 18:51:48 +01:00
Alexander Block
d1910eaff4 Refactor remains of CMasternode/-Man into CMasternodeMeta/-Man (#2606)
* Split up remaining logic from CMasternodeMan into CMasternodeMetaMan and CMasternodeUtils

Also get rid of CMastermode and store remaining meta info
in CMasternodeMetaInfo

* Also allow non-const T in Serialize/Unserialize for shared_ptr

* Rename CActiveDeterministicMasternodeManager to CActiveMasternodeManager

* Fix nowallet compile in masternode-utils.cpp
2019-01-03 23:08:34 +03:00
Alexander Block
016681cd38 Add support for serialization of bitsets and tuples (#2293)
* Add serialization support for dynamic and fixed size bitsets

* Support serialization of std::tuple
2018-09-14 18:53:49 +03:00
Wladimir J. van der Laan
0982d6d3c8 Merge #9305: Refactor: Removed begin/end_ptr functions.
8c1dbc5 Refactor: Removed begin/end_ptr functions. (Karl-Johan Alm)
2018-01-17 17:31:12 +01:00
Wladimir J. van der Laan
f12610c030 Merge #9125: Make CBlock a vector of shared_ptr of CTransactions
b4e4ba4 Introduce convenience type CTransactionRef (Pieter Wuille)
1662b43 Make CBlock::vtx a vector of shared_ptr<CTransaction> (Pieter Wuille)
da60506 Add deserializing constructors to CTransaction and CMutableTransaction (Pieter Wuille)
0e85204 Add serialization for unique_ptr and shared_ptr (Pieter Wuille)
2018-01-15 06:14:16 +01:00