dash/contrib/linearize
MarcoFalke 17acd6b472
Merge #12987: tests/tools: Enable additional Python flake8 rules for automatic linting via Travis
643aad17fa Enable additional flake8 rules (practicalswift)
f020aca297 Minor Python cleanups to make flake8 pass with the new rules enabled (practicalswift)

Pull request description:

  Enabled rules:

  ```
  * E242: tab after ','
  * E266: too many leading '#' for block comment
  * E401: multiple imports on one line
  * E402: module level import not at top of file
  * E701: multiple statements on one line (colon)
  * E901: SyntaxError: invalid syntax
  * E902: TokenError: EOF in multi-line string
  * F821: undefined name 'Foo'
  * W293: blank line contains whitespace
  * W606: 'async' and 'await' are reserved keywords starting with Python 3.7
  ```

  Note to reviewers:
  * In general we don't allow whitespace cleanups to existing code, but in order to allow for enabling Travis checking for these rules a few smaller whitespace cleanups had to made as part of this PR.
  * Use [this `?w=1` link](https://github.com/bitcoin/bitcoin/pull/12987/files?w=1) to show a diff without whitespace changes.

  Before this commit:

  ```
  $ flake8 -qq --statistics --ignore=B,C,E,F,I,N,W --select=E112,E113,E115,E116,E125,E131,E133,E223,E224,E242,E266,E271,E272,E273,E274,E275,E304,E306,E401,E402,E502,E701,E702,E703,E714,E721,E741,E742,E743,F401,E901,E902,F402,F404,F406,F407,F601,F602,F621,F622,F631,F701,F702,F703,F704,F705,F706,F707,F811,F812,F821,F822,F823,F831,F841,W292,W293,W504,W601,W602,W603,W604,W605,W606 .
  5     E266 too many leading '#' for block comment
  4     E401 multiple imports on one line
  6     E402 module level import not at top of file
  5     E701 multiple statements on one line (colon)
  1     F812 list comprehension redefines 'n' from line 159
  4     F821 undefined name 'ConnectionRefusedError'
  28    W293 blank line contains whitespace
  ```

  After this commit:

  ```
  $ flake8 -qq --statistics --ignore=B,C,E,F,I,N,W --select=E112,E113,E115,E116,E125,E131,E133,E223,E224,E242,E266,E271,E272,E273,E274,E275,E304,E306,E401,E402,E502,E701,E702,E703,E714,E721,E741,E742,E743,F401,E901,E902,F402,F404,F406,F407,F601,F602,F621,F622,F631,F701,F702,F703,F704,F705,F706,F707,F811,F812,F821,F822,F823,F831,F841,W292,W293,W504,W601,W602,W603,W604,W605,W606 .
  $
  ```

Tree-SHA512: fc7d5e752298a50d4248afc620ee2c173135b4ca008e48e02913ac968e5a24a5fd5396926047ec62f1d580d537434ccae01f249bb2f3338fa59dc630bf97ca7a
Signed-off-by: pasta <pasta@dashboost.org>
2020-07-22 12:12:46 -05:00
..
example-linearize-testnet.cfg Some Dashification (#3513) 2020-06-11 11:39:04 +03:00
example-linearize.cfg Some Dashification (#3513) 2020-06-11 11:39:04 +03:00
linearize-data.py Merge #12987: tests/tools: Enable additional Python flake8 rules for automatic linting via Travis 2020-07-22 12:12:46 -05:00
linearize-hashes.py Merge #12987: tests/tools: Enable additional Python flake8 rules for automatic linting via Travis 2020-07-22 12:12:46 -05:00
README.md Merge #10956: Fix typos 2020-01-01 21:31:39 -06:00

Linearize

Construct a linear, no-fork, best version of the Dash blockchain. The scripts run using Python 3 but are compatible with Python 2.

Step 0: Install dash_hash

https://github.com/dashpay/dash_hash

Step 1: Download hash list

$ ./linearize-hashes.py linearize.cfg > hashlist.txt

Required configuration file settings for linearize-hashes:

  • RPC: datadir (Required if rpcuser and rpcpassword are not specified)
  • RPC: rpcuser, rpcpassword (Required if datadir is not specified)

Optional config file setting for linearize-hashes:

  • RPC: host (Default: 127.0.0.1)
  • RPC: port (Default: 9998)
  • Blockchain: min_height, max_height
  • rev_hash_bytes: If true, the written block hash list will be byte-reversed. (In other words, the hash returned by getblockhash will have its bytes reversed.) False by default. Intended for generation of standalone hash lists but safe to use with linearize-data.py, which will output the same data no matter which byte format is chosen.

The linearize-hashes script requires a connection, local or remote, to a JSON-RPC server. Running dashd or dash-qt -server will be sufficient.

Step 2: Copy local block data

$ ./linearize-data.py linearize.cfg

Required configuration file settings:

  • output_file: The file that will contain the final blockchain. or
  • output: Output directory for linearized blocks/blkNNNNN.dat output.

Optional config file setting for linearize-data:

  • debug_output: Some printouts may not always be desired. If true, such output will be printed.
  • file_timestamp: Set each file's last-accessed and last-modified times, respectively, to the current time and to the timestamp of the most recent block written to the script's blockchain.
  • genesis: The hash of the genesis block in the blockchain. (default is '00000ffd590b1485b3caadc19b22e6379c733355108f107a430458cdf3407ab6', mainnet)
  • input: dashd blocks/ directory containing blkNNNNN.dat
  • hashlist: text file containing list of block hashes created by linearize-hashes.py.
  • max_out_sz: Maximum size for files created by the output_file option. (Default: 1000*1000*1000 bytes)
  • netmagic: Network magic number. (default is 'bf0c6bbd', mainnet)
  • out_of_order_cache_sz: If out-of-order blocks are being read, the block can be written to a cache so that the blockchain doesn't have to be sought again. This option specifies the cache size. (Default: 100*1000*1000 bytes)
  • rev_hash_bytes: If true, the block hash list written by linearize-hashes.py will be byte-reversed when read by linearize-data.py. See the linearize-hashes entry for more information.
  • split_timestamp: Split blockchain files when a new month is first seen, in addition to reaching a maximum file size (max_out_sz).