c276df775914e4e42993c76e172ef159e3b830d4 zmq: enable tcp keepalive (mruddy)
Pull request description:
This addresses https://github.com/bitcoin/bitcoin/issues/12754.
These changes enable node operators to address the silent dropping (by network middle boxes) of long-lived low-activity ZMQ TCP connections via further operating system level TCP keepalive configuration. For example, ZMQ sockets that publish block hashes can be affected in this way due to the length of time it sometimes takes between finding blocks (e.g.- sometimes more than an hour).
Prior to this patch, operating system level TCP keepalive configurations would not take effect since the SO_KEEPALIVE option was not enabled on the underlying socket.
There are additional ZMQ socket options related to TCP keepalive that can be set. However, I decided not to implement those options in this changeset because doing so would require adding additional bitcoin node configuration options, and would not yield a better outcome. I preferred a small, easily reviewable patch that doesn't add a bunch of new config options, with the tradeoff that the fine tuning would have to be done via well-documented operating system specific configurations.
I tested this patch by running a node with:
`./src/qt/bitcoin-qt -regtest -txindex -datadir=/tmp/node -zmqpubhashblock=tcp://127.0.0.1:28332 &`
and connecting to it with:
`python3 ./contrib/zmq/zmq_sub.py`
Without these changes, `ss -panto | grep 28332 | grep ESTAB | grep bitcoin` will report no keepalive timer information. With these changes, the output from the prior command will show keepalive timer information consistent with the configuration at the time of connection establishment, e.g.-: `timer:(keepalive,119min,0)`.
I also tested with a non-TCP transport and did not witness any adverse effects:
`./src/qt/bitcoin-qt -regtest -txindex -datadir=/tmp/node -zmqpubhashblock=ipc:///tmp/bitcoin.block &`
ACKs for top commit:
adamjonas:
Just to summarize for those looking to review - as of c276df775914e4e42993c76e172ef159e3b830d4 there are 3 tACKs (n-thumann, Haaroon, and dlogemann), 1 "looks good to me" (laanwj) with no NACKs or any show-stopping concerns raised.
jonasschnelli:
utACK c276df775914e4e42993c76e172ef159e3b830d4
Tree-SHA512: b884c2c9814e97e666546a7188c48f9de9541499a11a934bd48dd16169a900c900fa519feb3b1cb7e9915fc7539aac2829c7806b5937b4e1409b4805f3ef6cd1
* Merge #14060: ZMQ: add options to configure outbound message high water mark, aka SNDHWM
a4edb168b635b6f5c36324e44961cd42cf9bbbaa ZMQ: add options to configure outbound message high water mark, aka SNDHWM (mruddy)
Pull request description:
ZMQ: add options to configure outbound message high water mark, aka SNDHWM
This is my attempt at https://github.com/bitcoin/bitcoin/pull/13315
Tree-SHA512: a4cc3bcf179776899261a97c8c4f31f35d1d8950fd71a09a79c5c064879b38e600b26824c89c4091d941502ed5b0255390882f7d44baf9e6dc49d685a86e8edb
* High watermark settings for Dash-specific messages
Signed-off-by: Dzutte <dzutte.tomsk@gmail.com>
Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
f1bd03eb013b96ff040a8f835e4137fbd2a38cda [depends, zmq, doc] upgrade zeromq to 4.2.5 and avoid deprecated zeromq api functions (mruddy)
Pull request description:
Upgrade the ZeroMQ dependency from version 4.2.3 to the latest stable version 4.2.5.
This PR Follows the lead of https://github.com/bitcoin/bitcoin/pull/11986.
I upgraded both patch files to correspond to the version `4.2.5` libzmq files.
I assume doing so is still necessary and correct.
Without updating the patch line numbers, things appear to work, but you get extra log messages while building `depends` because things don't exactly match, e.g.:
```
/bitcoin/depends> make zeromq
Extracting zeromq...
/bitcoin/depends/sources/zeromq-4.2.5.tar.gz: OK
Preprocessing zeromq...
patching file src/windows.hpp
Hunk #1 succeeded at 58 (offset 3 lines).
patching file src/thread.cpp
Hunk #1 succeeded at 307 with fuzz 2 (offset 87 lines).
Hunk #2 succeeded at 323 with fuzz 2 (offset 90 lines).
```
Updating the patches seemed cleaner, so I did it. Note that libzmq had some whitespace changes, so that's why the updated patches do too.
More info: https://github.com/zeromq/libzmq/releases/tag/v4.2.5
tags: libzmq, zmq, 0mq
Tree-SHA512: 78659dd276b5311e40634b1bbebb802ddd6b69662ba3c84995ef1e3795c49a78b1635112c7fd72a405ea36e2cc3bdeb84e6d00d4e491a349bba1dafff50e2fa5
Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com>
d97fac422eaaefe13e1ed376e617882a100872ae Add a link from ZMQ doc to ZMQ example in contrib/ (Damian Mee)
Pull request description:
No code changes :). Only a small convenience improvement in zmq doc.
ACKs for top commit:
fanquake:
ACK d97fac422eaaefe13e1ed376e617882a100872ae
Tree-SHA512: f05a8a7a77c0a698637fd24ffc94d0d617743b434f46695a56576a53331ede254aeece416baf3f8275ae4dfad85ae6e14d1920aa32af53150847420a176d90fb
* Trivial Dashification
* Tweak getnetworkinfo and dumpwallet help text
We don't have RBF and Segwit
* CopyrightHolders should also check for missing "Dash Core" copyright
06a3aec Docs: Hash in ZMQ hash is raw bytes, not hex (Karel Bílek)
Pull request description:
Transaction hex cannot be in hexadecimal, that would be 64 bytes. In reality it's just raw bytes.
Tree-SHA512: 0f884c73a370be1fe39a1354e2b7ef02d4ff27348dbb8d35a77bbd309f417024959589b28a560b2c9e2713074b90534771fe8c29ff8c0ee1999966997a4827a0
* Implement IS Double spend notifications in zmq
* copy/paste error
* typo
* Send both conflicting and conflicts against as ZMQ notifications
* CTransaction based not hash based
* @UdjinM6 requested changes
* fix whitespace
* added zmq stuff for governance objects and votes
it seems that zmq is currently not in a working state, need to figure that out.
Need to:
plug in the new methods added
possibly plug in the old methods, as it doesn't look like they are.
* formatting fix. Will probably need to revert for this PR
* continue linking new zmq messages in
* added comment, might need to revert
* fixes error of it not knowing about the classes
* Actually link in, all new govobjects and govvotes should be broadcast over zmq now.
* fix compile error, forgot to change params when copying
* fix compile error
* add imports to the header files in zmqconfig.h
* fixing linking(i think) error
* Revert "added comment, might need to revert"
This reverts commit 2918ea40fe.
* Revert "formatting fix. Will probably need to revert for this PR"
This reverts commit ca10558866.
* fix tabs etc
* EOL added
* optimization of hash.begin() @nmarley thoughts?
* remove formatting changes
* iterator i -> it and removal of notifier
* typo in df879f57
* use auto for the iterators
* introduce hash prefix
* implement changes in zmq_sub.py, update the doc, and change argument name to fix typo
* deref iterators before calling methods
* continued e8a4c505
* missed one... continued e8a4c505
* killing some tabs
* fix spacing for setting or comparing iterators
* change order of new variables to match current setup
* re-add elif's I didn't realize got removed
* Revert "fix spacing for setting or comparing iterators"
This reverts commit 8ce2068148.
* Revert "use auto for the iterators"
This reverts commit cb16cf0760.
* Revert "missed one... continued e8a4c505"
This reverts commit 2087cf894f.
* Revert "continued e8a4c505"
This reverts commit a78c8ad2c9.
* Revert "deref iterators before calling methods"
This reverts commit e8a4c505d1.
* Revert "iterator i -> it and removal of notifier"
This reverts commit 29574248b1.
* Revert "fix whitespace"
This reverts commit 612be48d96.
* Revert "typo in df879f5"
* Revert "Optimization of hash.begin()"
* fixes problem with revert
* Udjin complain's a lot ;)
* help text, init.cpp
* add signals in validationinterface.cpp
* Change location of vote notification call.
* remain consistent
* remove unneeded include due to change of notification location
* implement raw notifications
* Zmq sequence (#1)
* Fixes ZMQ startup with bad arguments.
pr 7621
* [ZMQ] append a message sequence number to every ZMQ notification
- pr 7762
- contrib/zmq/zmq_sub.py to python 3 compatible
* typo in MSG_RAWTXLOCK
MMSG_RAWTXLOCK to MSG_RAWTXLOCK
* s/Bitcoind/dashd/
15a6a16 Implement transaction lock zmq notifications
3bc86a6 always push zmqpubhashtxlock even if tx is not from/to our wallet
568315b fix typo
b6d41d2 fix data size for notification name
339be11 implement zmqpubrawtxlock
70f44f9 update zmq_sub.py with hashtxlock and rawtxlock
Continues Johnathan Corgan's work.
Publishing multipart messages
Bugfix: Add missing zmq header includes
Bugfix: Adjust build system to link ZeroMQ code for Qt binaries
