2017-02-16 15:05:46 +01:00
|
|
|
# Block and Transaction Broadcasting with ZeroMQ
|
2014-11-18 18:06:32 +01:00
|
|
|
|
2020-09-11 07:12:43 +02:00
|
|
|
[ZeroMQ](https://zeromq.org/) is a lightweight wrapper around TCP
|
2015-09-29 19:48:45 +02:00
|
|
|
connections, inter-process communication, and shared-memory,
|
2015-10-17 12:10:45 +02:00
|
|
|
providing various message-oriented semantics such as publish/subscribe,
|
2014-11-18 18:06:32 +01:00
|
|
|
request/reply, and push/pull.
|
|
|
|
|
2016-03-04 08:25:16 +01:00
|
|
|
The Dash Core daemon can be configured to act as a trusted "border
|
|
|
|
router", implementing the dash wire protocol and relay, making
|
2014-11-18 18:06:32 +01:00
|
|
|
consensus decisions, maintaining the local blockchain database,
|
|
|
|
broadcasting locally generated transactions into the network, and
|
|
|
|
providing a queryable RPC interface to interact on a polled basis for
|
|
|
|
requesting blockchain related data. However, there exists only a
|
|
|
|
limited service to notify external software of events like the arrival
|
|
|
|
of new blocks or transactions.
|
|
|
|
|
2015-09-29 19:48:45 +02:00
|
|
|
The ZeroMQ facility implements a notification interface through a set
|
|
|
|
of specific notifiers. Currently there are notifiers that publish
|
2014-11-18 18:06:32 +01:00
|
|
|
blocks and transactions. This read-only facility requires only the
|
2015-09-29 19:48:45 +02:00
|
|
|
connection of a corresponding ZeroMQ subscriber port in receiving
|
2014-11-18 18:06:32 +01:00
|
|
|
software; it is not authenticated nor is there any two-way protocol
|
|
|
|
involvement. Therefore, subscribers should validate the received data
|
|
|
|
since it may be out of date, incomplete or even invalid.
|
|
|
|
|
2015-09-29 19:48:45 +02:00
|
|
|
ZeroMQ sockets are self-connecting and self-healing; that is,
|
|
|
|
connections made between two endpoints will be automatically restored
|
|
|
|
after an outage, and either end may be freely started or stopped in
|
|
|
|
any order.
|
2014-11-18 18:06:32 +01:00
|
|
|
|
|
|
|
Because ZeroMQ is message oriented, subscribers receive transactions
|
|
|
|
and blocks all-at-once and do not need to implement any sort of
|
|
|
|
buffering or reassembly.
|
|
|
|
|
|
|
|
## Prerequisites
|
|
|
|
|
Merge #13578: [depends, zmq, doc] upgrade zeromq to 4.2.5 and avoid deprecated zeromq api functions (#4397)
f1bd03eb013b96ff040a8f835e4137fbd2a38cda [depends, zmq, doc] upgrade zeromq to 4.2.5 and avoid deprecated zeromq api functions (mruddy)
Pull request description:
Upgrade the ZeroMQ dependency from version 4.2.3 to the latest stable version 4.2.5.
This PR Follows the lead of https://github.com/bitcoin/bitcoin/pull/11986.
I upgraded both patch files to correspond to the version `4.2.5` libzmq files.
I assume doing so is still necessary and correct.
Without updating the patch line numbers, things appear to work, but you get extra log messages while building `depends` because things don't exactly match, e.g.:
```
/bitcoin/depends> make zeromq
Extracting zeromq...
/bitcoin/depends/sources/zeromq-4.2.5.tar.gz: OK
Preprocessing zeromq...
patching file src/windows.hpp
Hunk #1 succeeded at 58 (offset 3 lines).
patching file src/thread.cpp
Hunk #1 succeeded at 307 with fuzz 2 (offset 87 lines).
Hunk #2 succeeded at 323 with fuzz 2 (offset 90 lines).
```
Updating the patches seemed cleaner, so I did it. Note that libzmq had some whitespace changes, so that's why the updated patches do too.
More info: https://github.com/zeromq/libzmq/releases/tag/v4.2.5
tags: libzmq, zmq, 0mq
Tree-SHA512: 78659dd276b5311e40634b1bbebb802ddd6b69662ba3c84995ef1e3795c49a78b1635112c7fd72a405ea36e2cc3bdeb84e6d00d4e491a349bba1dafff50e2fa5
Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
2021-09-08 18:25:20 +02:00
|
|
|
The ZeroMQ feature in Dash Core requires the ZeroMQ API >= 4.0.0
|
|
|
|
[libzmq](https://github.com/zeromq/libzmq/releases).
|
|
|
|
For version information, see [dependencies.md](dependencies.md).
|
|
|
|
Typically, it is packaged by distributions as something like
|
2015-09-29 19:48:45 +02:00
|
|
|
*libzmq3-dev*. The C++ wrapper for ZeroMQ is *not* needed.
|
2014-11-18 18:06:32 +01:00
|
|
|
|
2020-09-11 07:12:43 +02:00
|
|
|
In order to run the example Python client scripts in the `contrib/zmq/`
|
|
|
|
directory, one must also install [PyZMQ](https://github.com/zeromq/pyzmq)
|
|
|
|
(generally with `pip install pyzmq`), though this is not necessary for daemon
|
2015-09-29 19:48:45 +02:00
|
|
|
operation.
|
2014-11-18 18:06:32 +01:00
|
|
|
|
|
|
|
## Enabling
|
|
|
|
|
2015-10-06 05:09:04 +02:00
|
|
|
By default, the ZeroMQ feature is automatically compiled in if the
|
|
|
|
necessary prerequisites are found. To disable, use --disable-zmq
|
2020-06-11 10:39:04 +02:00
|
|
|
during the *configure* step of building dashd:
|
2014-11-18 18:06:32 +01:00
|
|
|
|
2015-10-06 05:09:04 +02:00
|
|
|
$ ./configure --disable-zmq (other options)
|
2014-11-18 18:06:32 +01:00
|
|
|
|
2015-10-06 05:09:04 +02:00
|
|
|
To actually enable operation, one must set the appropriate options on
|
2017-02-16 15:05:46 +01:00
|
|
|
the command line or in the configuration file.
|
2014-11-18 18:06:32 +01:00
|
|
|
|
|
|
|
## Usage
|
|
|
|
|
|
|
|
Currently, the following notifications are supported:
|
|
|
|
|
2019-05-08 11:12:54 +02:00
|
|
|
-zmqpubhashblock=address
|
|
|
|
-zmqpubhashchainlock=address
|
2014-11-18 18:06:32 +01:00
|
|
|
-zmqpubhashtx=address
|
2016-07-15 08:38:33 +02:00
|
|
|
-zmqpubhashtxlock=address
|
2019-05-08 11:12:54 +02:00
|
|
|
-zmqpubhashgovernancevote=address
|
|
|
|
-zmqpubhashgovernanceobject=address
|
|
|
|
-zmqpubhashinstantsenddoublespend=address
|
2020-11-17 20:41:30 +01:00
|
|
|
-zmqpubhashrecoveredsig=address
|
2014-11-18 18:06:32 +01:00
|
|
|
-zmqpubrawblock=address
|
2019-05-08 11:12:54 +02:00
|
|
|
-zmqpubrawchainlock=address
|
2019-05-23 11:13:58 +02:00
|
|
|
-zmqpubrawchainlocksig=address
|
2014-11-18 18:06:32 +01:00
|
|
|
-zmqpubrawtx=address
|
2016-07-15 08:38:33 +02:00
|
|
|
-zmqpubrawtxlock=address
|
2019-05-23 11:13:58 +02:00
|
|
|
-zmqpubrawtxlocksig=address
|
2018-07-12 11:06:30 +02:00
|
|
|
-zmqpubrawgovernancevote=address
|
2019-05-08 11:12:54 +02:00
|
|
|
-zmqpubrawgovernanceobject=address
|
2018-09-12 13:12:44 +02:00
|
|
|
-zmqpubrawinstantsenddoublespend=address
|
2020-11-17 20:41:30 +01:00
|
|
|
-zmqpubrawrecoveredsig=address
|
2014-11-18 18:06:32 +01:00
|
|
|
|
2015-09-29 19:48:45 +02:00
|
|
|
The socket type is PUB and the address must be a valid ZeroMQ socket
|
|
|
|
address. The same address can be used in more than one notification.
|
2014-11-18 18:06:32 +01:00
|
|
|
|
|
|
|
For instance:
|
|
|
|
|
2016-03-06 16:26:01 +01:00
|
|
|
$ dashd -zmqpubhashtx=tcp://127.0.0.1:28332 \
|
|
|
|
-zmqpubrawtx=ipc:///tmp/dashd.tx.raw
|
2014-11-18 18:06:32 +01:00
|
|
|
|
|
|
|
Each PUB notification has a topic and body, where the header
|
2015-09-29 19:48:45 +02:00
|
|
|
corresponds to the notification type. For instance, for the
|
|
|
|
notification `-zmqpubhashtx` the topic is `hashtx` (no null
|
2017-08-22 08:23:09 +02:00
|
|
|
terminator) and the body is the transaction hash (32
|
2015-09-29 19:48:45 +02:00
|
|
|
bytes).
|
2014-11-18 18:06:32 +01:00
|
|
|
|
2016-03-04 08:25:16 +01:00
|
|
|
These options can also be provided in dash.conf.
|
2014-11-18 18:06:32 +01:00
|
|
|
|
|
|
|
ZeroMQ endpoint specifiers for TCP (and others) are documented in the
|
2015-10-12 22:39:47 +02:00
|
|
|
[ZeroMQ API](http://api.zeromq.org/4-0:_start).
|
2014-11-18 18:06:32 +01:00
|
|
|
|
|
|
|
Client side, then, the ZeroMQ subscriber socket must have the
|
2015-09-29 19:48:45 +02:00
|
|
|
ZMQ_SUBSCRIBE option set to one or either of these prefixes (for
|
|
|
|
instance, just `hash`); without doing so will result in no messages
|
2020-05-12 10:22:06 +02:00
|
|
|
arriving. Please see [`contrib/zmq/zmq_sub.py`](/contrib/zmq/zmq_sub.py) for a working example.
|
2014-11-18 18:06:32 +01:00
|
|
|
|
|
|
|
## Remarks
|
|
|
|
|
2016-03-06 16:26:01 +01:00
|
|
|
From the perspective of dashd, the ZeroMQ socket is write-only; PUB
|
2014-11-18 18:06:32 +01:00
|
|
|
sockets don't even have a read function. Thus, there is no state
|
2016-03-06 16:26:01 +01:00
|
|
|
introduced into dashd directly. Furthermore, no information is
|
2014-11-18 18:06:32 +01:00
|
|
|
broadcast that wasn't already received from the public P2P network.
|
|
|
|
|
|
|
|
No authentication or authorization is done on connecting clients; it
|
|
|
|
is assumed that the ZeroMQ port is exposed only to trusted entities,
|
|
|
|
using other means such as firewalling.
|
|
|
|
|
2015-09-29 19:48:45 +02:00
|
|
|
Note that when the block chain tip changes, a reorganisation may occur
|
|
|
|
and just the tip will be notified. It is up to the subscriber to
|
|
|
|
retrieve the chain from the last known block to the new tip.
|
2016-10-17 22:09:21 +02:00
|
|
|
|
|
|
|
There are several possibilities that ZMQ notification can get lost
|
2018-03-03 15:24:28 +01:00
|
|
|
during transmission depending on the communication type you are
|
2016-10-17 22:09:21 +02:00
|
|
|
using. Dashd appends an up-counting sequence number to each
|
|
|
|
notification which allows listeners to detect lost notifications.
|