sikkienl/dash
1
0
Fork 0
mirror of https://github.com/dashpay/dash.git synced 2024-12-26 04:22:55 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity
20,448 Commits 24 Branches 554 Tags 573 MiB
6ac3b30107
Commit Graph

47 Commits

Author SHA1 Message Date
Wladimir J. van der Laan
bba59cabc3
Merge #16802: scripts: In linearize, search for next position of magic bytes rather than fail 
3284e6c09a84e9557ec72723ad636053d3ef7122 scripts: search for next position of magic bytes rather than fail (Tim Akinbo)

Pull request description:

  When using the `linearize-data.py` contrib script to export block data, there are edge cases where the script fails with an `Invalid magic: 00000000` error. This error occurs due to the presence of padding bytes that occasionally appears between consecutive blocks in the block data file.

  There's an ongoing conversation about this in #14986. sipa also admitted that it is a bug in #5028. Fortunately, this is not an issue in bitcoin core as it handles this type of situation gracefully and so no fix in bitcoin core is required.

  This PR is an improvement on how the script handles these "invalid magic bytes". Rather than failing, this patch allows the script to search for the next occurrence of the magic bytes and then starts reading the block from there.

ACKs for top commit:
  laanwj:
    ACK 3284e6c09a84e9557ec72723ad636053d3ef7122

Tree-SHA512: 18067ae0b4b62e822dfc558a86439ad6acaf939b98479e38e8e4248536574643b26eb48e96ec7139375c88b42cbe7705a64deb13a3c239e16025a6aad3d69bfa
 2021-09-15 13:54:38 -04:00
Kittywhiskers Van Gogh
0b13db2ac5 merge #14954: Require python 3.5 2021-08-31 11:16:12 +05:30
MarcoFalke
c4fee672a4 Merge #15351: Update linearize-hashes.py 
7fdb92e53b Update linearize-hashes.py (OverlordQ)

Pull request description:

  Fix class case issue.

Tree-SHA512: 42d26e38b75b6b419ae4a9ca5c110d4ced0f7c5db997a64c8ab5dfc25dc228008349b6423c20ef4e396a773ff31f1f3f0092331c5e89748216e253e4d8337e9a
 2021-08-18 22:24:36 -03:00
MarcoFalke
a309982e82 Merge #14947: scripts: Remove Python 2 import workarounds 
4de11a3682 Remove Python 2 import workarounds (practicalswift)

Pull request description:

  Remove Python 2 import workarounds.

  As noted by @jnewbery in https://github.com/bitcoin/bitcoin/pull/14903#discussion_r241396925:

  > This exception handling is a vestige from when github-merge.py supported Python 2 and Python 3. We only support Python 3 now so we should be able to remove it entirely and just import from urllib.request.

Tree-SHA512: e0d21e6299dd62fb669ad95cbd3d19f7c803195fd336621aac72fd10ddc7431d90443831072a2e1eb2fc880d1d88eb7c3e2ead3da59f545f6db07d349af98fb3
 2021-08-03 18:39:42 -03:00
PastaPastaPasta
f13d26dbfe
Dashification 
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
 2021-07-14 18:46:13 -05:00
fanquake
ecbaf20937
Merge #17393: doc: Added regtest config for linearize script 
582e66b6e75d58033987a7b0474226cfdd724ce0 doc: Added regtest config for linearize script (Gr0kchain)

Pull request description:

  Updated the example-linearize.cfg file to include support for the regtest chain network config which is used by the ./linearize-data.py

  Problem:
  Without the regtest magic, genesis hash and path config, the `linearize-data.py` script cannot generate a bootstrap.dat file.

  Example:

  ./linearize-data.py ./linearize.cfg
  Read 102 hashes
  Genesis block not found in hashlist

  Solution:

  Added netmagic, genesis and input example parameters to file.

  Resolution

  1. Starting bitcoind in regtest mode
  2. bitcoin-cli generatetoaddress 101 $(bitcoin-cli getnewaddress)
  3. ./linearize-hashes.py ./linearize.cfg > ./hashlist.txt
  4. ./linearize-data.py ./linearize.cfg

  ```
  $ ./linearize-data.py ./linearize.cfg
  Read 102 hashes
  Input file /Users/gr0kchain/.bitcoin/regtest/blocks/blk00000.dat
  Output file /Users/gr0kchain/Downloads/bootstrap.dat
  Done (102 blocks written)
  ```

ACKs for top commit:
  fanquake:
    ACK 582e66b6e75d58033987a7b0474226cfdd724ce0

Tree-SHA512: 699e92e740e68e2e5190ba37538efbbe3e4d4e725ebd6af704a0cf5517683b691754f7ea097bf840845d2b53b793c63258d406e9bd37922db810cf58bed053c3
 2021-07-14 18:44:42 -05:00
fanquake
573d3a0e02
Merge #17336: scripts: search for first block file for linearize-data with some block files pruned 
317fb96de9c6257972f1213b4ef2c3fe87dde99f Add search for first blk file with pruned node (Rjected)

Pull request description:

  <!--
  *** Please remove the following help text before submitting: ***

  Pull requests without a rationale and clear improvement may be closed
  immediately.
  -->

  <!--
  Please provide clear motivation for your patch and explain how it improves
  Bitcoin Core user experience or Bitcoin Core developer experience
  significantly:

  * Any test improvements or new tests that improve coverage are always welcome.
  * All other changes should have accompanying unit tests (see `src/test/`) or
    functional tests (see `test/`). Contributors should note which tests cover
    modified code. If no tests exist for a region of modified code, new tests
    should accompany the change.
  * Bug fixes are most welcome when they come with steps to reproduce or an
    explanation of the potential issue as well as reasoning for the way the bug
    was fixed.
  * Features are welcome, but might be rejected due to design or scope issues.
    If a feature is based on a lot of dependencies, contributors should first
    consider building the system outside of Bitcoin Core, if possible.
  * Refactoring changes are only accepted if they are required for a feature or
    bug fix or otherwise improve developer experience significantly. For example,
    most "code style" refactoring changes require a thorough explanation why they
    are useful, what downsides they have and why they *significantly* improve
    developer experience or avoid serious programming bugs. Note that code style
    is often a subjective matter. Unless they are explicitly mentioned to be
    preferred in the [developer notes](/doc/developer-notes.md), stylistic code
    changes are usually rejected.
  -->
  When bitcoind is running in pruned mode, producing a hashlist with `./linearize-hashes.py linearize.cfg > hashlist.txt` and then executing `linearize-data.py linearize.cfg` will produce:
  ```
  Read 313001 hashes
  Input file /home/dan/.bitcoin/blocks/blk00000.dat
  Premature end of block data
  ```
  This happens because `linearize-data` starts by attempting to process `blk00000.dat` regardless of whether or not `blk00000.dat` actually exists - this may not be the case if working with a pruned node.
  This PR adds a function which finds the first block file that does exist, and calls that function when the `BlockDataCopier` is initialized.

  This is a refactor of #16431.

  <!--
  Bitcoin Core has a thorough review process and even the most trivial change
  needs to pass a lot of eyes and requires non-zero or even substantial time
  effort to review. There is a huge lack of active reviewers on the project, so
  patches often sit for a long time.
  -->

ACKs for top commit:
  darosior:
    ACK 317fb96de9c6257972f1213b4ef2c3fe87dde99f
  laanwj:
    Code review ACK 317fb96de9c6257972f1213b4ef2c3fe87dde99f
  theStack:
    Code review ACK 317fb96de9

Tree-SHA512: fc8014282df6cfe7b267e64db8ce7d82b86b758c302fbfea4a3c39b62d93512f5c2e31a0de4e9c5ec18fc0268c917f011257d37b45afaef6033eec90e4aa585f
 2021-07-14 18:43:56 -05:00
MarcoFalke
17acd6b472
Merge #12987: tests/tools: Enable additional Python flake8 rules for automatic linting via Travis 
643aad17fa Enable additional flake8 rules (practicalswift)
f020aca297 Minor Python cleanups to make flake8 pass with the new rules enabled (practicalswift)

Pull request description:

  Enabled rules:

  ```
  * E242: tab after ','
  * E266: too many leading '#' for block comment
  * E401: multiple imports on one line
  * E402: module level import not at top of file
  * E701: multiple statements on one line (colon)
  * E901: SyntaxError: invalid syntax
  * E902: TokenError: EOF in multi-line string
  * F821: undefined name 'Foo'
  * W293: blank line contains whitespace
  * W606: 'async' and 'await' are reserved keywords starting with Python 3.7
  ```

  Note to reviewers:
  * In general we don't allow whitespace cleanups to existing code, but in order to allow for enabling Travis checking for these rules a few smaller whitespace cleanups had to made as part of this PR.
  * Use [this `?w=1` link](https://github.com/bitcoin/bitcoin/pull/12987/files?w=1) to show a diff without whitespace changes.

  Before this commit:

  ```
  $ flake8 -qq --statistics --ignore=B,C,E,F,I,N,W --select=E112,E113,E115,E116,E125,E131,E133,E223,E224,E242,E266,E271,E272,E273,E274,E275,E304,E306,E401,E402,E502,E701,E702,E703,E714,E721,E741,E742,E743,F401,E901,E902,F402,F404,F406,F407,F601,F602,F621,F622,F631,F701,F702,F703,F704,F705,F706,F707,F811,F812,F821,F822,F823,F831,F841,W292,W293,W504,W601,W602,W603,W604,W605,W606 .
  5     E266 too many leading '#' for block comment
  4     E401 multiple imports on one line
  6     E402 module level import not at top of file
  5     E701 multiple statements on one line (colon)
  1     F812 list comprehension redefines 'n' from line 159
  4     F821 undefined name 'ConnectionRefusedError'
  28    W293 blank line contains whitespace
  ```

  After this commit:

  ```
  $ flake8 -qq --statistics --ignore=B,C,E,F,I,N,W --select=E112,E113,E115,E116,E125,E131,E133,E223,E224,E242,E266,E271,E272,E273,E274,E275,E304,E306,E401,E402,E502,E701,E702,E703,E714,E721,E741,E742,E743,F401,E901,E902,F402,F404,F406,F407,F601,F602,F621,F622,F631,F701,F702,F703,F704,F705,F706,F707,F811,F812,F821,F822,F823,F831,F841,W292,W293,W504,W601,W602,W603,W604,W605,W606 .
  $
  ```

Tree-SHA512: fc7d5e752298a50d4248afc620ee2c173135b4ca008e48e02913ac968e5a24a5fd5396926047ec62f1d580d537434ccae01f249bb2f3338fa59dc630bf97ca7a
Signed-off-by: pasta <pasta@dashboost.org>
 2020-07-22 12:12:46 -05:00
PastaPastaPasta
95f83945b9
remove extra space at end line 
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
 2020-07-08 19:16:32 +00:00
pasta
e6a1c6b5bb scripted-diff: change all tabs to spaces in linearize-hashes.py and linearize-data.py 
It appears bitcoin has already done this previously(seemingly forever), however, we for some reason have tabs. This was causing the linter to fail because a backport touched lines which resulted in linter yelling at me for adding lines beginning with tabs. To fix this, replacing all tabs with spaces in these two files.

-BEGIN VERIFY SCRIPT-
sed -i 's/\t/    /g' contrib/linearize/linearize-*.py
-END VERIFY SCRIPT-

Signed-off-by: pasta <pasta@dashboost.org>
 2020-07-07 16:03:07 -05:00
Wladimir J. van der Laan
d1200755f1 Merge #13448: Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python 
c8176b3cc7556d7bcec39a55ae4d6ba16453baaa Add linter: Make sure we explicitly open all text files using UTF-8 or ASCII encoding in Python (practicalswift)
634bd970013eca90f4b4c1f9044eec8c97ba62c2 Explicitly specify encoding when opening text files in Python code (practicalswift)

Pull request description:

  Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python.

  As requested by @laanwj in #13440.

Tree-SHA512: 1651c00fe220ceb273324abd6703aee504029b96c7ef0e3029145901762c733c9b9d24927da281394fd4681a5bff774336c04eed01fafea997bb32192c334c06
Signed-off-by: pasta <pasta@dashboost.org>

# Conflicts:
#	contrib/devtools/circular-dependencies.py
#	contrib/linearize/linearize-data.py
#	contrib/linearize/linearize-hashes.py
#	contrib/seeds/generate-seeds.py
#	contrib/verify-commits/verify-commits.py
#	test/functional/multiwallet.py
#	test/functional/notifications.py
#	test/functional/test_runner.py
#	test/util/rpcauth-test.py
 2020-07-07 11:50:56 -05:00
Pasta
b40dc8f1ec fix lint-python.sh after 11835 and 12295 
Signed-off-by: Pasta <pasta@dashboost.org>
 2020-06-17 14:29:55 -05:00
UdjinM6
469d6aa737
Some Dashification (#3513) 
* Trivial Dashification

* Tweak getnetworkinfo and dumpwallet help text

We don't have RBF and Segwit

* CopyrightHolders should also check for missing "Dash Core" copyright
 2020-06-11 11:39:04 +03:00
MarcoFalke
561ec27683 Merge #10781: Python cleanups 
78214588d Use for-loop instead of list comprehension (practicalswift)
823979436 Use the variable name _ for unused return values (practicalswift)
2e6080bbf Remove unused variables and/or function calls (practicalswift)
9b94054b7 Avoid reference to undefined name: stderr does not exist, sys.stderr does (practicalswift)
51cb6b822 Use print(...) instead of undefined printf(...) (practicalswift)
25cd520fc Use sys.exit(...) instead of exit(...): exit(...) should not be used in programs (practicalswift)

Pull request description:

  Python cleanups:
  * Avoid reference to undefined name: `stderr` does not exist, `sys.stderr` does
  * Use `print(...)` instead of undefined `printf(...)`
  * Avoid redefinition of variable (`tx`) in list comprehension
  * Remove unused variables and/or function calls
  * Use `sys.exit(...)` instead of `exit(...)`: [`exit(...)` should not be used in programs](https://github.com/bitcoin/bitcoin/pull/10753#discussion_r125935027)

Tree-SHA512: 1238dfbc1d20f7edadea5e5406a589f293065638f6234809f0d5b6ba746dffe3d276bc5884c7af388a6c798c61a8759faaccf57f381225644754c0f61914eb4b
 2020-01-02 22:01:29 -06:00
MarcoFalke
c9b20f027f
Merge #10956: Fix typos 
9d5e98ff8 Fix typos. (practicalswift)

Pull request description:

  Fix some typos not covered by #10705.

Tree-SHA512: f06e9541f6ae13ef5d6731399b61795997b21a8816abeb1749c93e99a5c47354e6cbd4a3d145f4dc6ef8a13db179799a3121ecbb7288abf3e8d81cdf81500d37
 2020-01-01 21:31:39 -06:00
Wladimir J. van der Laan
13cdff4989
Merge #10104: linearize script: Option to use RPC cookie 
bd41d98 Datadir option in linearize scripts (Andrew Chow)

Tree-SHA512: 0d11866b574986c087ec962a8a9fc0b6dfee8175ae20ef827f8b4a143f657c5bffc9f9696e9dabf29b68002003a5b6a7d8ac473231b5c9c81c3a4fa0318f5bd0
 2019-05-19 20:43:00 -05:00
Wladimir J. van der Laan
3dcbfc744a Merge #9580: Fix various minor linearization script issues 
b9d95bd Fix various minor linearization script issues (Douglas Roark)
 2018-01-23 09:24:27 +01:00
MarcoFalke
660c027cd5 Merge #9510: [trivial] Fix typos in comments 
cc16d99 [trivial] Fix typos in comments (practicalswift)
 2018-01-21 12:48:34 +01:00
MarcoFalke
be63fb7953 Merge #9508: Remove unused Python imports 
95bab82 Remove unused Python imports (practicalswift)
 2018-01-21 12:48:33 +01:00
Alexander Block
e5daae2913 Dashify "linearize" README.md and switch the tool to mainnet 2018-01-18 19:39:43 +01:00
Wladimir J. van der Laan
882e2cdb9f Merge #9373: Linearize script update (hash byte reversal and Python 3 support) 
3c8f63b Make linearize scripts Python 3-compatible. (Doug)
d5aa198 Allow linearization scripts to support hash byte reversal (Doug)
 2018-01-18 07:33:44 +01:00
MarcoFalke
bc659c3e6e Merge #8270: Tests: Use portable #! in python scripts (/usr/bin/env) 
7b01ce2 Favour python over python2 as per PR #7723 (Matthew King)
873e81f Use portable #! in python scripts (/usr/bin/env) (Matthew King)
 2018-01-08 18:04:55 +01:00
UdjinM6
dd5bd97561 change default datadir to DashCore(Win,Mac)/.dashcore(Linux) as discussed in #898 2016-07-06 23:26:21 +03:00
UdjinM6
a5ac60b868 Merge remote-tracking branch 'bitcoin/0.12' into HEAD 
+ merge fixes
+ keepass on evhttp
 2016-02-06 16:48:04 +03:00
Holger Schinzel
c38e5233c1 fixing reference 
Closes #603
 2015-09-12 16:25:31 +02:00
UdjinM6
a31df6e4ae fix linearization tool to make it work with Dash 2015-09-12 16:24:04 +02:00
Paul Georgiou
e3c42973b2 Update Linearize tool to support Windows paths 2015-07-10 19:21:21 +03:00
Jeff Garzik
7f68c7dc05 contrib/linearize: Support linearization of testnet blocks 2015-05-20 15:27:51 -07:00
UdjinM6
64eebc3316 0.12 dirty merge'n'fix (bitcoin 0.10) 2015-04-04 19:24:37 +03:00
sandakersmann
f914f1a746
Added "Core" to copyright headers 
Github-Pull: #5494
Rebased-From: 15de949bb9277e442302bdd8dee299a8d6deee60
 2014-12-19 19:55:32 +01:00
Michael Ford
78253fcbad Remove references to X11 licence 2014-12-16 15:56:50 +08:00
Micha
ebdd6bf30f
Remove genesis block hash from error 2014-12-08 13:55:12 +02:00
Micha
8d174d6b5d
Make several small string fixes in linearize-data.py 
- Add a space after the fixed string prepended to file names when input or
  output file changes
- Clarify the error message when the genesis block is not found in the
  hash list (...why do we have this at all?)
 2014-12-07 17:55:53 +02:00
vertoe
aedda3b379 Change port to 9999 and rpcport to 9998 2014-11-27 18:09:11 +01:00
Wladimir J. van der Laan
97a34c28d5
Merge pull request #5051 
aedc74d contrib: make linearize-data.py cope with out-of-order blocks (Wladimir J. van der Laan)
 2014-10-08 09:47:41 +02:00
Wladimir J. van der Laan
aedc74dfa6 contrib: make linearize-data.py cope with out-of-order blocks 
Make it possible to read blocks in any order. This will be required
after headers-first (#4468), so should be merged before that.

- Read block header. For expected blocks, continue, else skip.
- For in-order blocks: copy block contents directly. Write prior
  out-of-order blocks if this connects a consecutive span.
- For out-of-order blocks, store extents of block data for later
  retrieval. Cache out-of-order blocks in memory up to 100MB
  (configurable).
 2014-10-06 18:30:12 +02:00
Wladimir J. van der Laan
772ab0ea98 contrib: use batched JSON-RPC in linarize-hashes 
Batch up to 10000 requests for a ~30x speedup.
 2014-10-06 13:27:13 +02:00
Suhas Daftuar
07d5287f42 Catch error when done reading files 2014-09-11 09:22:39 -04:00
Jeff Garzik
399cdbc700
contrib/linearize: Add feature to set file's timestamp based on block header time. 2014-09-04 15:13:37 +02:00
Jeff Garzik
8f5a423344 contrib/linearize: split block files based on year-month, not just year 2014-08-24 01:40:40 -04:00
Jeff Garzik
75400a2a41 contrib/linearize: Guarantee that output is generated in-order 
This was typically ensured implicitly by virtue of normal bitcoind
operation.  Adding an explicit check provides a stronger guarantee, and
it is cheap to add.
 2014-08-24 01:18:24 -04:00
Jeff Garzik
b4a72a75b4 contrib/linearize: split output files based on new-timestamp-year or max-file-size 2014-08-23 22:59:16 -04:00
Jeff Garzik
476eb7eb53 Update linearize scripts. 
Break into two steps:
* Generate hash list
* Build data file(s) from local bitcoind blocks/ directory.

This supports building one large bootstrap.dat, or multiple
smaller blocks/blkNNNNN.dat files.
 2014-08-23 21:46:12 -04:00
Subo1978
3078b582cc linearize.py: Harmonize rpcpass to rpcpassword 
linearize.py: harmonize rpcpass to rpcpassword according settings in
bitcoin.conf
 2014-02-10 15:52:31 +01:00
Micha
6873cfb6c0
Update linearize.py defaults to block 279,000 2014-01-18 21:53:07 +02:00
Micha
7a29fb5940
Tweak linearize.py to give more flexibility 
Add the ability to start at a non-zero height, and allow for appending to
an existing file.
 2014-01-12 13:48:39 +02:00
super3
27e2d8bfb5 Finished /Contrib Index. Standardized READMEs. 
File and Link Fix.
 2013-10-15 23:37:30 -04:00