Dash - Reinventing Cryptocurrency
Go to file
pasta 96c4442253
Merge #5901: backport: merge bitcoin#21817, #21966, #21824, #21969, #23653, #23438, #24190, #24253, #24231, #26258, #28012, partial bitcoin#25001, #25296, #23595, #27927 (serialization updates)
2b26a87874 merge bitcoin#28012: Allow FastRandomContext::randbytes for std::byte, Allow std::byte serialization (Kittywhiskers Van Gogh)
4eeafa267c partial bitcoin#27927: Allow std::byte and char Span serialization (Kittywhiskers Van Gogh)
cb2fa8360a test: place the std::ostream operator<< definition in namespace std (Kittywhiskers Van Gogh)
cf4522f845 partial bitcoin#23595: Add ParseHex<std::byte>() helper (Kittywhiskers Van Gogh)
e4091aa477 partial bitcoin#25296: Add DataStream without ser-type and ser-version (Kittywhiskers Van Gogh)
eab031a9b0 merge bitcoin#26258: Remove unused CDataStream::rdbuf method (Kittywhiskers Van Gogh)
95b5850434 partial bitcoin#25001: Modernize util/strencodings and util/string: string_view and optional (Kittywhiskers Van Gogh)
5fe72bbb8e merge bitcoin#24231: Fix read-past-the-end and integer overflows (Kittywhiskers Van Gogh)
24af37256f merge bitcoin#24253: Remove broken and unused CDataStream methods (Kittywhiskers Van Gogh)
baf8dd65cd merge bitcoin#24190: Fix sanitizer suppresions in streams_tests (Kittywhiskers Van Gogh)
e933d78a88 merge bitcoin#23438: Use spans of std::byte in serialize (Kittywhiskers Van Gogh)
d3b282208b merge bitcoin#23653: Generalize/simplify VectorReader into SpanReader (Kittywhiskers Van Gogh)
2c32a09f4e merge bitcoin#21969: Switch serialize to uint8_t (Kittywhiskers Van Gogh)
0a08dbf3f4 merge bitcoin#21824: Replace deprecated char with uint8_t in serialization (Kittywhiskers Van Gogh)
d0b4e560a6 merge bitcoin#21966: Remove double serialization; use software encoder for fee estimation (Kittywhiskers Van Gogh)
1d6aafea47 merge bitcoin#21817: Replace &foo[0] with foo.data() (Kittywhiskers Van Gogh)
d9a8ce2749 trivial: move GetSerializeSize away from Stream (Un)serialize functions (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * Dependency for https://github.com/dashpay/dash/pull/5902

  * [bitcoin#24231](https://github.com/bitcoin/bitcoin/pull/24231) is merged after [bitcoin#24253](https://github.com/bitcoin/bitcoin/pull/24253) due to a MinGW bug ([comment](https://github.com/bitcoin/bitcoin/pull/24231#issuecomment-1031179638))

  * [bitcoin#25001](https://github.com/bitcoin/bitcoin/pull/25001) is listed as unmerged despite being committed upstream as 132d5f8c2f

  * [bitcoin#25296](https://github.com/bitcoin/bitcoin/pull/25296) is listed as unmerged despite being committed upstream as 79e007d1d6

  * [bitcoin#21966](https://github.com/bitcoin/bitcoin/pull/21966) was partially backported in [dash#4197](https://github.com/dashpay/dash/pull/4197) as f946c68f83, including only 2be4cd94f4c7d92a4287971233a20d68db81c9c9.
    The excluded commits have been backported, marking the pull request as fully merged.

  * [bitcoin#23438](https://github.com/bitcoin/bitcoin/pull/23438) was partially backported in [dash#5574](https://github.com/dashpay/dash/pull/5574) as de54b8784c, including only fa65bbf.
     The excluded commits have been backported, marking the pull request as fully merged.

  * [bitcoin#27927](https://github.com/bitcoin/bitcoin/pull/27927) opened a fresh can of hell thanks to being (possibly?) the first pull request to include `std::byte` `BOOST_CHECK`'s to the unit test suite. For reasons still unbeknownst to me, it refused to compile, despite being perfectly happy when checked-out as a commit directly and built as-is from upstream.

    The compile error was like this (edited for brevity):
    ```
      CXX      test/test_dash-serialize_tests.o
    In file included from test/serialize_tests.cpp:13:
    In file included from /src/dash/depends/x86_64-pc-linux-gnu/include/boost/test/unit_test.hpp:18:
    In file included from /src/dash/depends/x86_64-pc-linux-gnu/include/boost/test/test_tools.hpp:46:
    In file included from /src/dash/depends/x86_64-pc-linux-gnu/include/boost/test/tools/old/impl.hpp:24:
    /src/dash/depends/x86_64-pc-linux-gnu/include/boost/test/tools/detail/print_helper.hpp:53:13: error: static assertion failed due to requirement 'boost::has_left_shift<std::basic_ostream<char, std::char_traits<char>>, std::byte, boost::binary_op_detail::dont_care>::value': Type has to implement operator<< to be printable
                BOOST_STATIC_ASSERT_MSG( (boost::has_left_shift<std::ostream,T>::value),
                ^                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    [...]
    <scratch space>:206:1: note: expanded from here
    BOOST_PP_REPEAT_1
    ^
    test/serialize_tests.cpp:347:9: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::check_frwd<boost::test_tools::tt_detail::equal_impl_frwd, std::byte, std::byte>' requested here
            BOOST_CHECK_EQUAL(out.at(0), std::byte{'a'});
            ^
    [...]
    In file included from test/serialize_tests.cpp:13:
    In file included from /src/dash/depends/x86_64-pc-linux-gnu/include/boost/test/unit_test.hpp:18:
    In file included from /src/dash/depends/x86_64-pc-linux-gnu/include/boost/test/test_tools.hpp:46:
    In file included from /src/dash/depends/x86_64-pc-linux-gnu/include/boost/test/tools/old/impl.hpp:24:
    /src/dash/depends/x86_64-pc-linux-gnu/include/boost/test/tools/detail/print_helper.hpp:55:18: error: invalid operands to binary expression ('std::ostream' (aka 'basic_ostream<char>') and 'const std::byte')
                ostr << t;
                ~~~~ ^  ~
    /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/cstddef:130:5: note: candidate function template not viable: no known conversion from 'std::ostream' (aka 'basic_ostream<char>') to 'byte' for 1st argument
        operator<<(byte __b, _IntegerType __shift) noexcept
        ^
    [...]
    5 warnings and 2 errors generated.
    make[2]: *** [Makefile:17842: test/test_dash-serialize_tests.o] Error 1
    make[2]: *** Waiting for unfinished jobs....
    make[2]: Leaving directory '/src/dash/src'
    make[1]: *** [Makefile:18525: all-recursive] Error 1
    make[1]: Leaving directory '/src/dash/src'
    make: *** [Makefile:802: all-recursive] Error 1
    ```
    * No such error was present on Bitcoin.

      It is true, that no `std::ostream& operator<<(std::ostream& a, const std::byte& b)` is present by default but attempting to `grep` for any specializations didn't show anything _relevant_ that Dash didn't have. Searching on GitHub didn't help either.
    * Then, assuming that perhaps Boost's assertion logic may have changed, upgraded the version of Boost to match the pull request at the time, Boost 1.81. That also did not do anything (and actually, caused a trailing slashes unit test to fail but doesn't cause any problem in Bitcoin because they got rid of their `boost::filesystem` usage by then).
    * If that isn't it, then let's try building Bitcoin with Dash's depends. It built successfully, ran successfully. The problem isn't in the dependency, it's in the codebase.
    * Since it seemed to be `std::byte` related, pull requests that are related to `std::byte` serialization were backported and `std::byte` serialization related changes needed for [dash#5902](https://github.com/dashpay/dash/pull/5902) were cherry-picked. That's why this pull request came to be. But it didn't help this particular issue (though it did smooth out the cherry-picks).
    * Running out of ideas, `gdb` is used to step through `serialize_tests`'s `class_methods` and understand why `BOOST_CHECK_EQUAL(out.at(0), std::byte{'a'});` is valid in Bitcoin but not Dash by finding the elusive `operator<<`. This is where things go from bad to worse.

      Turns out, when you build with `clang`, `gdb` loses the ability to do breakpoints by file. So, an attempt is made to use `lldb` (which btw, is called `lldb-16`, running `lldb` with yield an error if you're using the develop container) and it refuses to work, erroring `personality set failed: Operation not permitted`.

      Turns out, the [`docker-compose.yml`](37f43e4e56/contrib/containers/develop/docker-compose.yml) needs the following additions:

      ```
        cap_add:
          - SYS_PTRACE
        security_opt:
          - "seccomp:unconfined"
      ```

      After making these changes, `lldb` works and then we resume trying to find `operator<<`. After too many hours and nimbly alternating between `next` and `step`, tried making a return to `gdb` (compiling with `gcc` this time with the appropriate `CXXFLAGS`) hoping for different results and a while later, realized that it cannot step through Boost's headers (it doesn't recognize the filenames) and then recompile it with `clang` and return to `lldb`.

      This was a wild goose chase.

    * After a lot of futile efforts to find the operator by stepping through `BOOST_CHECK_EQUAL`, a basic example addressing the static assertion (that a left shift operator must exist of `<type>` (here `std::byte`) for `std::ostream`) was added in

      ```c++
      std::ostream& ostr = std::cout;
      ostr << std::byte{'a'};
      ```

      ...and it compiled in both codebases.

      So the left shift that Boost is asserting doesn't exist does exist but it isn't being detected for some reason. Upon hovering the `<<`, VSCode highlighted the source of the definition as [`setup_common.h`](96ac317c27/src/test/util/setup_common.h (L27-L32)) thanks to the comment above it.

      Diffing between Bitcoin and Dash revealed the secret, the `operator<<` definition was placed under namespace `std` by [bitcoin#23497](https://github.com/bitcoin/bitcoin/pull/23497) in f7086fd8ff (see [change](f7086fd8ff/src/test/util/setup_common.h (L29-L35))).

      That change has now been made in a separate commit.

  ## Breaking Changes

  Changes in serialization APIs will make backports predating [bitcoin#23438](https://github.com/bitcoin/bitcoin/pull/23438) annoying but will _not_ change how data is stored on disk.

  ## Checklist:
    _Go over all the following points, and put an `x` in all the boxes that apply._
  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests **(note: N/A)**
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

Top commit has no ACKs.

Tree-SHA512: 1d7c67f5fe282f78e24cb720828e5f1f48b6926006b903c28399938532cc5c470c175b00c8b80e9662c4467c1201e09ae6d1cd9b95e8b20ace5e4410c72c472e
2024-02-28 13:43:52 -06:00
.github Merge bitcoin/bitcoin#24509: doc: Add guix prefix for changes to reproducible builds 2024-02-07 10:14:43 -06:00
.tx fix: follow-up #5393 - should be used [dash.dash_ents] (#5472) 2023-07-01 14:16:50 +03:00
build-aux/m4 Merge #21064: refactor: use std::shared_mutex & remove Boost Thread 2024-01-16 09:29:52 -06:00
ci non-scripted-diff: bump copyright year to 2023 2024-02-24 11:05:37 -06:00
contrib Merge bitcoin/bitcoin#21818: doc: fixup -coinstatsindex help, update bitcoin.conf and files.md 2024-02-27 12:01:02 -06:00
depends Merge bitcoin/bitcoin#25697: depends: expat 2.4.8 & fix building with -flto 2024-02-28 13:16:40 -06:00
doc Merge bitcoin/bitcoin#21818: doc: fixup -coinstatsindex help, update bitcoin.conf and files.md 2024-02-27 12:01:02 -06:00
share feat: Set client version for non-release binaries and version in guix based on git tags (#5653) 2024-01-11 21:43:42 -06:00
src merge bitcoin#28012: Allow FastRandomContext::randbytes for std::byte, Allow std::byte serialization 2024-02-28 13:37:36 -06:00
test merge bitcoin#24190: Fix sanitizer suppresions in streams_tests 2024-02-28 13:37:34 -06:00
.cirrus.yml Merge bitcoin/bitcoin#24574: test: Actually print TSan tracebacks 2024-02-22 20:58:43 -06:00
.dockerignore build: add dash minimal development environment container 2021-12-21 12:43:37 +05:30
.editorconfig Merge #21123: code style: Add EditorConfig file 2021-07-16 10:04:09 -05:00
.fuzzbuzz.yml Merge #21064: refactor: use std::shared_mutex & remove Boost Thread 2024-01-16 09:29:52 -06:00
.gitattributes
.gitignore merge bitcoin#21336: Make .gitignore ignore src/test/fuzz/fuzz.exe 2024-02-06 08:39:51 -06:00
.gitlab-ci.yml build: disable multiprocess build in gitlab CI 2024-01-16 09:34:28 -06:00
.python-version partial bitcoin#27483: Bump python minimum version to 3.8 2023-05-11 09:18:48 -05:00
.style.yapf
.travis.yml Merge #20339: ci: Run more ci configs on cirrus 2024-02-05 10:20:31 -06:00
autogen.sh Merge #17829: scripted-diff: Bump copyright of files changed in 2019 2023-12-06 11:40:14 -06:00
CMakeLists.txt chore: Added missing sources files in CMake (#5503) 2023-07-25 12:23:56 -05:00
configure.ac Merge bitcoin/bitcoin#25425: build: Fix ::_wsystem check 2024-02-27 10:02:43 -06:00
CONTRIBUTING.md Merge bitcoin/bitcoin#25165: doc: Explain squashing with merge commits 2024-02-22 20:58:44 -06:00
COPYING docs: update license year range to 2024 (#5890) 2024-02-22 20:56:43 -06:00
INSTALL.md
libdashconsensus.pc.in revert dash#1432: Rename consensus source library and API 2022-08-09 14:16:28 +05:30
Makefile.am Merge #20549: Support make src/bitcoin-node and src/bitcoin-gui 2024-01-16 09:34:27 -06:00
README.md chore: drop version from README.md which is not really useful (#5811) 2024-01-10 12:12:41 -06:00
SECURITY.md Merge bitcoin/bitcoin#23466: doc: Suggest keys.openpgp.org as keyserver in SECURITY.md 2022-04-03 18:46:47 -05:00

Dash Core staging tree

CI master develop
Gitlab Build Status Build Status

https://www.dash.org

For an immediately usable, binary version of the Dash Core software, see https://www.dash.org/downloads/.

Further information about Dash Core is available in the doc folder.

What is Dash?

Dash is an experimental digital currency that enables instant, private payments to anyone, anywhere in the world. Dash uses peer-to-peer technology to operate with no central authority: managing transactions and issuing money are carried out collectively by the network. Dash Core is the name of the open source software which enables the use of this currency.

For more information read the original Dash whitepaper.

License

Dash Core is released under the terms of the MIT license. See COPYING for more information or see https://opensource.org/licenses/MIT.

Development Process

The master branch is meant to be stable. Development is normally done in separate branches. Tags are created to indicate new official, stable release versions of Dash Core.

The develop branch is regularly built (see doc/build-*.md for instructions) and tested, but is not guaranteed to be completely stable.

The contribution workflow is described in CONTRIBUTING.md and useful hints for developers can be found in doc/developer-notes.md.

Testing

Testing and code review is the bottleneck for development; we get more pull requests than we can review and test on short notice. Please be patient and help out by testing other people's pull requests, and remember this is a security-critical project where any mistake might cost people lots of money.

Automated Testing

Developers are strongly encouraged to write unit tests for new code, and to submit new unit tests for old code. Unit tests can be compiled and run (assuming they weren't disabled in configure) with: make check. Further details on running and extending unit tests can be found in /src/test/README.md.

There are also regression and integration tests, written in Python. These tests can be run (if the test dependencies are installed) with: test/functional/test_runner.py

The Travis CI system makes sure that every pull request is built for Windows, Linux, and macOS, and that unit/sanity tests are run automatically.

Manual Quality Assurance (QA) Testing

Changes should be tested by somebody other than the developer who wrote the code. This is especially important for large or high-risk changes. It is useful to add a test plan to the pull request description if testing the changes is not straightforward.

Translations

Changes to translations as well as new translations can be submitted to Dash Core's Transifex page.

Translations are periodically pulled from Transifex and merged into the git repository. See the translation process for details on how this works.

Important: We do not accept translation changes as GitHub pull requests because the next pull from Transifex would automatically overwrite them again.