Commit Graph

24077 Commits

Author SHA1 Message Date
PastaPastaPasta
2958aacf5d
Merge pull request #5577 from kwvg/bitcore_idx
refactor: cleanup {address,spent}index, remove platform-specific types, split out timestamp index
2023-09-26 20:56:48 -05:00
Kittywhiskers Van Gogh
93ddd3f7e8 refactor: consolidate P2PK{H} types to P2PK_OR_P2PKH 2023-09-25 22:57:42 +05:30
Kittywhiskers Van Gogh
f011c31b1a refactor: make AddressType a strong enum, remove uint8_t for address_type
Co-authored-by: Konstantin Akimov <knstqq@gmail.com>
2023-09-25 22:57:42 +05:30
Kittywhiskers Van Gogh
c65a10f349 refactor: use structure binding in for loop in rpc/misc
Co-authored-by: Konstantin Akimov <knstqq@gmail.com>
2023-09-25 22:57:41 +05:30
Kittywhiskers Van Gogh
109f57c83e refactor: use range-based loops instead of iterators
Co-authored-by: Konstantin Akimov <knstqq@gmail.com>
2023-09-25 22:57:41 +05:30
Kittywhiskers Van Gogh
b08a6343a7 refactor: add missing headers to {address,timestamp}index 2023-09-25 22:57:41 +05:30
Kittywhiskers Van Gogh
f47021a536 refactor: remove more code duplication from (dis)connect operations 2023-09-25 20:28:56 +05:30
Kittywhiskers Van Gogh
b29ef2eb4a refactor: deduplicate scriptPubKey trimming, index operations 2023-09-25 20:28:56 +05:30
Kittywhiskers Van Gogh
51203a83dc refactor: use static_cast for (de)serializing bool 2023-09-25 20:28:55 +05:30
Kittywhiskers Van Gogh
0cc321cd8c refactor: move CAddressIndex* out of spentindex 2023-09-25 20:25:34 +05:30
Kittywhiskers Van Gogh
cc2efac4b4 refactor: move timestamp index out of spentindex 2023-09-25 20:21:07 +05:30
Kittywhiskers Van Gogh
ce3b5acae4 docs: add BitPay copyright to file headers 2023-09-25 20:21:07 +05:30
Kittywhiskers Van Gogh
fc2d193d07 refactor: remove size_t usage in spentindex structs
the size of size_t is platform-dependent, (de)serialization is done
assuming the the value is 32 bits. changed to uint32_t as index value
cannot be less than zero.
2023-09-25 20:21:07 +05:30
Kittywhiskers Van Gogh
6a5e74eff4 refactor: use uint8_t for m_address_type
m_address_type is (de)serialized using ser_writedata8, making the maximum
numbers of bits read or written to, 8. AddressType does not have values
below 0, therefore the data type is changed to better reflect the way
it is stored.
2023-09-25 20:21:07 +05:30
Kittywhiskers Van Gogh
5c34da0675 refactor: cleanup spentindex structs, use fixed-length integers 2023-09-25 20:21:07 +05:30
Kittywhiskers Van Gogh
98e61857fe refactor: add separators between members, functions and serialization 2023-09-25 20:20:32 +05:30
Kittywhiskers Van Gogh
53977827ff refactor: cleanup addressindex structs, use fixed-length integers 2023-09-25 20:20:32 +05:30
Kittywhiskers Van Gogh
12fd3571e3 refactor: use boolean for transaction state, rename variable
there are two problems with "int spending;"
- the integer, which implies that there are gradient states when
  there is none, only boolean (vin is spent, vout is UTXO)
- the name, "spending" implies the existence of a middle state,
  there is none

a reason why int may have been used is due to needing it in the
comparison struct CMempoolAddressDeltaKeyCompare, though, using
an int isn't necessary as when used with a comparison operator,
a bool is implicitly converted to an int.

see, https://en.cppreference.com/w/cpp/language/implicit_conversion
(Integral promotion)
2023-09-25 20:19:57 +05:30
Kittywhiskers Van Gogh
5950f7ce35 refactor: cleanup map comparison struct definitions 2023-09-25 20:19:30 +05:30
Kittywhiskers Van Gogh
5faf29d318 refactor: use scoped weak enum instead of bare values for address type
strong enums (enum class) cannot be converted implicitly to another
type, requiring you to either use a static_cast or use to_underlying,
which is a part of C++23, which this codebase doesn't support.

the idea of scoping a weak enum into a namespace is courtesy of
https://stackoverflow.com/a/46294875/13845753
2023-09-24 21:59:07 +05:30
PastaPastaPasta
7bff85e031
Merge pull request #5574 from kwvg/auxports11
backport: merge bitcoin#19326, #20464, #23451, #20852, #20355, #22050, #23115, #23409, #18468, #23413, partial bitcoin#21560, #23438 (auxiliary backports: part 11)
2023-09-24 09:51:49 -05:00
Kittywhiskers Van Gogh
117083f39e merge bitcoin#23413: Replace MakeSpan helper with Span deduction guide 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
b89963abf2 merge bitcoin#22570: Ignore banlist.dat 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
c3b4b6746a merge bitcoin#20966: save the banlist in a JSON format on disk 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
0e2ca7d3a5 merge bitcoin#20516: Well-defined CAddress disk serialization, and addrv2 anchors.dat 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
3feccd7925 partial bitcoin#17428: Try to preserve outbound block-relay-only connections during restart
excludes:
- 0a85e5a7bc8dc6587963e2e37ac1b087a1fc97fe
2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
bda40398a5 merge bitcoin#20839: Avoid extraneous copy of input data, using Span<> 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
71f8c55d54 merge bitcoin#19020: Use C++11 member initialization in protocol 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
6b68cf82e1 merge bitcoin#18468: Span improvements
includes:
- 2676aeadfa0e43dcaaccc4720623cdfe0beed528

completion of partial merge done in dash#4036 as ec2eec0c
2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
984f58d4df merge bitcoin#23409: Take Span in SetSeed 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
de54b8784c partial bitcoin#23438: Use spans of std::byte in serialize
includes:
- fa65bbf217b725ada35107b4ad646d250228355c
2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
8c2d480f1a merge bitcoin#23115: use Span instead of std::vector for insert and contains 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
5d7367e366 merge bitcoin#22050: remove tor v2 support 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
546f7557d8 merge bitcoin#19951: CNetAddr scoped ipv6 test coverage, rename scopeId to m_scope_id 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
aa76506bc9 partial bitcoin#21560: Add Tor v3 hardcoded seeds
excludes:
- 2a257de113fd31539b68c28c47ef94f257b6e427
- 9b29d5df7fc555eaea42029f334f2995c6ccde3d
2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
4041d20716 merge bitcoin#20355: Check for addrv1 compatibility before using addrv1 serializer/deserializer on CSubNet 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
4b29a74904 merge bitcoin#20852: allow CSubNet of non-IP networks 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
bb8bd9a94d merge bitcoin#23451: Add std::byte helpers 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
dba0dc9501 merge bitcoin#20464: Treat CDataStream bytes as uint8_t 2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
65ff12c303 merge bitcoin#19326: Simplify hash.h interface using Spans
includes:
- 2a2182c387f607cd8284f33890bd285a81077b7f
- 77c507358bda9bd6c496f33e0f4418c0603bb08d

completion of partial merge done in dash#4164 as 56f1b2d (blocker
bitcoin#17938 was merged in dash#5246 as 00802bb)
2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
7ddf2ff59b refactor: remove unused Hash function variants
The three object variant was removed in dash#3276 through the backport
of bitcoin#11385, leaving behind the four, five and six variant options.
They aren't used anywhere in the codebase and aren't present in Bitcoin
Core currently, and so, have been removed.
2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
35f1c94672 refactor: wrap P2PK scriptPubKey range into Span
Co-authored-by: Konstantin Akimov <knstqq@gmail.com>
2023-09-24 09:50:50 -05:00
Kittywhiskers Van Gogh
ee313525ad
refactor: decouple db hooks from CFlatDB-based C*Manager objects, migrate to *Store structs (#5555)
## Motivation

As highlighted in https://github.com/dashpay/dash-issues/issues/52,
decoupling of `CFlatDB`-interacting components from managers of objects
like `CGovernanceManager` and `CSporkManager` is a key task for
achieving deglobalization of Dash-specific components.

The design of `CFlatDB` as a flat database agent relies on hooking into
the object's state its meant to load and store, using its
(de)serialization routines and other miscellaneous functions (notably,
without defining an interface) to achieve those ends. This approach was
taken predominantly for components that want a single-file cache.

Because of the method it uses to hook into the object (templates and the
use of temporary objects), it explicitly prevented passing arguments
into the object constructor, an explicit requirement for storing
references to other components during construction. This, in turn,
created an explicit dependency on those same components being available
in the global context, which would block the backport of bitcoin#21866,
a requirement for future backports meant to achieve parity in
`assumeutxo` support.

The design of these objects made no separation between persistent (i.e.
cached) and ephemeral (i.e. generated/fetched during initialization or
state transitions) data and the design of `CFlatDB` attempts to "clean"
the database by breaching this separation and attempting to access this
ephemeral data.

This might be acceptable if it is contained within the manager itself,
like `CSporkManager`'s `CheckAndRemove()` but is utterly unacceptable
when it relies on other managers (that, as a reminder, are only
accessible through the global state because of restrictions caused by
existing design), like `CGovernanceManager`'s `UpdateCachesAndClean()`.

This pull request aims to separate the `CFlatDB`-interacting portions of
these managers into a struct, with `CFlatDB` interacting only with this
struct, while the manager inherits the struct and manages
load/store/update of the database through the `CFlatDB` instance
initialized within its scope, though the instance only has knowledge of
what is exposed through the limited parent struct.

## Additional information

* As regards to existing behaviour, `CFlatDB` is written entirely as a
header as it relies on templates to specialize itself for the object it
hooks into. Attempting to split the logic and function definitions into
separate files will require you to explicitly define template
specializations, which is tedious.

* `m_db` is defined as a pointer as you cannot instantiate a
forward-declared template (see [this Stack Overflow
answer](https://stackoverflow.com/a/12797282) for more information),
which is done when defined as a member in the object scope.

* The conditional cache flush predicating on RPC _not_ being in the
warm-up state has been replaced with unconditional flushing of the
database on object destruction (@UdjinM6, is this acceptable?)

## TODOs

This is a list of things that aren't within the scope of this pull
request but should be addressed in subsequent pull requests

* [ ] Definition of an interface that `CFlatDB` stores are expected to
implement
* [ ] Lock annotations for all potential uses of members protected by
the `cs` mutex in each manager object and store
* [ ] Additional comments documenting what each function and member does
* [ ] Deglobalization of affected managers

---------

Co-authored-by: Kittywhiskers Van Gogh <63189531+kittywhiskers@users.noreply.github.com>
2023-09-24 09:50:21 -05:00
Konstantin Akimov
633cc3260f
feat: new rpc `gettxchainlocks' to get transaction statuses by batch (#5578)
## Issue being fixed or feature implemented
Requested by @QuantumExplorer for platform needs

## What was done?
New rpc `gettransactionsarelocked` that returns list of txes.
it does less heavy calculations and transfer less data by gRPC.


## How Has This Been Tested?
```
$ src/dash-cli gettransactionsarelocked  '["e469de7994b9c1da8efd262fee8843efd7bdcab80c700dc1059c98b28f7c5c1b", "0d9fdf00c9568ff9103742b64e6b8287794633072f8824fa2c475f59e71dbace","0d3f48eebead54d640a7fc5692ddfcba619d8b49347d9a7c04586057c02dec9f"]'

[
  {
    "height": 907801,
    "chainlock": true
  },
  {
    "height": 101,
    "chainlock": true
  },
  {
    "height": -1,
    "chainlock": false
  }
]
```
Limiter tested by this call:
```
src/dash-cli gettransactionsarelocked  '["", "","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","","",""]'  | wc
```

## Breaking Changes
N/A

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone

---------

Co-authored-by: pasta <pasta@dashboost.org>
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2023-09-20 09:07:24 -05:00
PastaPastaPasta
400d171d04
feat(rpc): add fundingthreshold to output of getgovernanceinfo (#5581)
## Issue being fixed or feature implemented
DashCentral and DMT are both providing incorrect funding thresholds;
output this from core to communicate this more clearly

## What was done?
Added RPC output

## How Has This Been Tested?
Running on main net

## Breaking Changes
none

## Checklist:
_Go over all the following points, and put an `x` in all the boxes that
apply._
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-09-19 21:30:13 +03:00
Pander
5e04b9f1d4
docs: improved Docker documentation (#5543)
## Issue being fixed or feature implemented
Make Dash on Docker Hub easier to find, a search on `dash` there does
not provide the result.

## What was done?
Improved Docker documentation in `contrib/containers/README.md`

## How Has This Been Tested?
n/a

## Breaking Changes
n/a

## Checklist:
_Go over all the following points, and put an `x` in all the boxes that
apply._
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [x] I have made corresponding changes to the documentation
- [ ] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-09-19 21:26:43 +03:00
Odysseas Gabrielides
19aa3ab31a
feat: Remove outdated quorum data from evodb (#5576)
## Issue being fixed or feature implemented
Grabbed this from #5480. 

## What was done?
Cleans quorum data from evoDB for old quorums.

## How Has This Been Tested?


## Breaking Changes


## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2023-09-19 09:00:30 -05:00
PastaPastaPasta
135be62d44
Merge pull request #5537 from knst/bc-bp-v20-missing-8v2
backport: bitcoin#12134, #18426, #18534, #18828, #18864, #19153, #19201, #19205, #19560, #19813, #19859, partial #20354, #20955, #22442, #22790
2023-09-19 08:54:33 -05:00
UdjinM6
affcacaa10 feat: cache downloaded releases 2023-09-19 08:54:12 -05:00
merge-script
90285f5327 Merge bitcoin/bitcoin#22790: test: add aarch64-apple-darwin platform entry to get_previous_releases
f6e4db27ceb67157dc13d13f34351cb87fec2be5 test: add aarch64-apple-darwin platform entry to get_previous_releases (Zero-1729)

Pull request description:

  Over the course of reviewing a PR, I had to edit `test/get_previous_releases.py` (after I ran `git clean -xdff`) to run the backwards compatibility tests (e.g. `wallet_upgradewallet`, `feature_backwards_compatibility`, etc.), as currently on master, running the script as indicated in [`test/README.md`](https://github.com/bitcoin/bitcoin/blob/master/test/README.md), for example, on an M1 machine results in the following error, as the `aarch64-apple-darwin*` platform entry is presently not recognised:

  > Output from an M1 machine running macOS v11.5.2

  ```sh
  $ test/get_previous_releases.py -b v0.20.1 v0.19.1 v0.18.1 v0.17.2 v0.16.3 v0.15.2
  Releases directory: releases
  Not sure which binary to download for aarch64-apple-darwin20.6.0
  ```

  As a quick fix, this PR adds the missing `aarch64-apple-darwin*` platform entry. Running the script now results in fetching the old binaries, as expected:

  ```sh
  $ test/get_previous_releases.py -b v0.20.1 v0.19.1 v0.18.1 v0.17.2 v0.16.3 v0.15.2

  Releases directory: releases
  Fetching: https://bitcoincore.org/bin/bitcoin-core-0.20.1/bitcoin-0.20.1-osx64.tar.gz
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
    0 20.9M    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
  100 20.9M  100 20.9M    0     0   136k      0  0:02:37  0:02:37 --:--:-- 95607
  Checksum matched

  …

  Checksum matched
  ```

  After this patch, the backwards compatibility tests also run successfully, as expected.

  **Note**: I am open to other possible solutions.

  ---

  Steps to reproduce:

  > Ensure you take out the binaries in `releases` if they already exist.

  Try running `test/get_previous_releases.py -b v0.15.2` or similar to fetch the old release binaries.

Top commit has no ACKs.

Tree-SHA512: a238d909b70a61be622234bc49b05d2e91a8acfc5ea348d29f2c8a927fb793cb97365e558571e3f46d6a5650c4f3c6e28fa126c6e56b38e1eb98f7c3e3594d0f
2023-09-19 08:54:12 -05:00