dash/src/scheduler.h

130 lines
4.6 KiB
C
Raw Normal View History

// Copyright (c) 2015 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#ifndef BITCOIN_SCHEDULER_H
#define BITCOIN_SCHEDULER_H
partial Merge #18234: refactor: Replace boost::mutex,condition_var,chrono with std equivalents in scheduler This backport does not include changes that depend on bitcoin pr 18037 70a6b529f306ff72ea1badf25e970a92b2b17ab3 lint-cppcheck: Remove -DHAVE_WORKING_BOOST_SLEEP_FOR (Anthony Towns) 294937b39de5924e772f8ed90d35c53290c8acab scheduler_tests: re-enable mockforward test (Anthony Towns) cea19f685915be8affb2203184a549576194413f Drop unused reverselock.h (Anthony Towns) d0ebd93270758ea97ea956b8821e17a2d001ea94 scheduler: switch from boost to std (Anthony Towns) b9c426012770d166e6ebfab27689be44e6e89aa5 sync.h: add REVERSE_LOCK (Anthony Towns) 306f71b4eb4a0fd8e64f47dc008bc235b80b13d9 scheduler: don't rely on boost interrupt on shutdown (Anthony Towns) Pull request description: Replacing boost functionality with C++11 stuff. Motivated by #18227, but should stand alone. Changing from `boost::condition_var` to `std::condition_var` means `threadGroup.interrupt_all` isn't enough to interrupt `serviceQueue` anymore, so that means calling `stop()` before `join_all()` is needed. And the existing reverselock.h code doesn't work with sync.h's DebugLock code (because the reversed lock won't be removed from `g_lockstack` which then leads to incorrect potential deadlock warnings), so I've replaced that with a dedicated class and macro that's aware of our debug lock behaviour. Fixes #16027, Fixes #14200, Fixes #18227 ACKs for top commit: laanwj: ACK 70a6b529f306ff72ea1badf25e970a92b2b17ab3 Tree-SHA512: d1da13adeabcf9186d114e2dad9a4fdbe2e440f7afbccde0c13dfbaf464efcd850b69d3371c5bf8b179d7ceb9d81f4af3cc22960b90834e41eaaf6d52ef7d331 # Conflicts: # src/reverselock.h # src/rpc/misc.cpp # src/scheduler.cpp # src/scheduler.h # src/sync.cpp # src/sync.h # src/test/reverselock_tests.cpp # src/test/scheduler_tests.cpp # src/test/test_dash.cpp # test/lint/extended-lint-cppcheck.sh
2020-03-06 20:47:49 +01:00
#include <condition_variable>
#include <functional>
#include <list>
#include <map>
Backport 11651 (#3358) * scripted-diff: Replace #include "" with #include <> (ryanofsky) -BEGIN VERIFY SCRIPT- for f in \ src/*.cpp \ src/*.h \ src/bench/*.cpp \ src/bench/*.h \ src/compat/*.cpp \ src/compat/*.h \ src/consensus/*.cpp \ src/consensus/*.h \ src/crypto/*.cpp \ src/crypto/*.h \ src/crypto/ctaes/*.h \ src/policy/*.cpp \ src/policy/*.h \ src/primitives/*.cpp \ src/primitives/*.h \ src/qt/*.cpp \ src/qt/*.h \ src/qt/test/*.cpp \ src/qt/test/*.h \ src/rpc/*.cpp \ src/rpc/*.h \ src/script/*.cpp \ src/script/*.h \ src/support/*.cpp \ src/support/*.h \ src/support/allocators/*.h \ src/test/*.cpp \ src/test/*.h \ src/wallet/*.cpp \ src/wallet/*.h \ src/wallet/test/*.cpp \ src/wallet/test/*.h \ src/zmq/*.cpp \ src/zmq/*.h do base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f done -END VERIFY SCRIPT- Signed-off-by: Pasta <pasta@dashboost.org> * scripted-diff: Replace #include "" with #include <> (Dash Specific) -BEGIN VERIFY SCRIPT- for f in \ src/bls/*.cpp \ src/bls/*.h \ src/evo/*.cpp \ src/evo/*.h \ src/governance/*.cpp \ src/governance/*.h \ src/llmq/*.cpp \ src/llmq/*.h \ src/masternode/*.cpp \ src/masternode/*.h \ src/privatesend/*.cpp \ src/privatesend/*.h do base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f done -END VERIFY SCRIPT- Signed-off-by: Pasta <pasta@dashboost.org> * build: Remove -I for everything but project root Remove -I from build system for everything but the project root, and built-in dependencies. Signed-off-by: Pasta <pasta@dashboost.org> # Conflicts: # src/Makefile.test.include * qt: refactor: Use absolute include paths in .ui files * qt: refactor: Changes to make include paths absolute This makes all include paths in the GUI absolute. Many changes are involved as every single source file in src/qt/ assumes to be able to use relative includes. Signed-off-by: Pasta <pasta@dashboost.org> # Conflicts: # src/qt/dash.cpp # src/qt/optionsmodel.cpp # src/qt/test/rpcnestedtests.cpp * test: refactor: Use absolute include paths for test data files * Recommend #include<> syntax in developer notes * refactor: Include obj/build.h instead of build.h * END BACKPORT #11651 Remove trailing whitespace causing travis failure * fix backport 11651 Signed-off-by: Pasta <pasta@dashboost.org> * More of 11651 * fix blockchain.cpp Signed-off-by: pasta <pasta@dashboost.org> * Add missing "qt/" in includes * Add missing "test/" in includes * Fix trailing whitespaces Co-authored-by: Wladimir J. van der Laan <laanwj@gmail.com> Co-authored-by: Russell Yanofsky <russ@yanofsky.org> Co-authored-by: MeshCollider <dobsonsa68@gmail.com> Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2020-03-19 23:46:56 +01:00
#include <sync.h>
//
// Simple class for background tasks that should be run
// periodically or once "after a while"
//
// Usage:
//
// CScheduler* s = new CScheduler();
// s->scheduleFromNow(doSomething, 11); // Assuming a: void doSomething() { }
// s->scheduleFromNow(std::bind(Class::func, this, argument), 3);
// std::thread* t = new std::thread([&] { s->serviceQueue(); });
//
partial Merge #18234: refactor: Replace boost::mutex,condition_var,chrono with std equivalents in scheduler This backport does not include changes that depend on bitcoin pr 18037 70a6b529f306ff72ea1badf25e970a92b2b17ab3 lint-cppcheck: Remove -DHAVE_WORKING_BOOST_SLEEP_FOR (Anthony Towns) 294937b39de5924e772f8ed90d35c53290c8acab scheduler_tests: re-enable mockforward test (Anthony Towns) cea19f685915be8affb2203184a549576194413f Drop unused reverselock.h (Anthony Towns) d0ebd93270758ea97ea956b8821e17a2d001ea94 scheduler: switch from boost to std (Anthony Towns) b9c426012770d166e6ebfab27689be44e6e89aa5 sync.h: add REVERSE_LOCK (Anthony Towns) 306f71b4eb4a0fd8e64f47dc008bc235b80b13d9 scheduler: don't rely on boost interrupt on shutdown (Anthony Towns) Pull request description: Replacing boost functionality with C++11 stuff. Motivated by #18227, but should stand alone. Changing from `boost::condition_var` to `std::condition_var` means `threadGroup.interrupt_all` isn't enough to interrupt `serviceQueue` anymore, so that means calling `stop()` before `join_all()` is needed. And the existing reverselock.h code doesn't work with sync.h's DebugLock code (because the reversed lock won't be removed from `g_lockstack` which then leads to incorrect potential deadlock warnings), so I've replaced that with a dedicated class and macro that's aware of our debug lock behaviour. Fixes #16027, Fixes #14200, Fixes #18227 ACKs for top commit: laanwj: ACK 70a6b529f306ff72ea1badf25e970a92b2b17ab3 Tree-SHA512: d1da13adeabcf9186d114e2dad9a4fdbe2e440f7afbccde0c13dfbaf464efcd850b69d3371c5bf8b179d7ceb9d81f4af3cc22960b90834e41eaaf6d52ef7d331 # Conflicts: # src/reverselock.h # src/rpc/misc.cpp # src/scheduler.cpp # src/scheduler.h # src/sync.cpp # src/sync.h # src/test/reverselock_tests.cpp # src/test/scheduler_tests.cpp # src/test/test_dash.cpp # test/lint/extended-lint-cppcheck.sh
2020-03-06 20:47:49 +01:00
// ... then at program shutdown, make sure to call stop() to clean up the thread(s) running serviceQueue:
// s->stop();
// t->join();
// delete t;
// delete s; // Must be done after thread is interrupted/joined.
//
class CScheduler
{
public:
CScheduler();
~CScheduler();
typedef std::function<void()> Function;
// Call func at/after time t
partial Merge #18234: refactor: Replace boost::mutex,condition_var,chrono with std equivalents in scheduler This backport does not include changes that depend on bitcoin pr 18037 70a6b529f306ff72ea1badf25e970a92b2b17ab3 lint-cppcheck: Remove -DHAVE_WORKING_BOOST_SLEEP_FOR (Anthony Towns) 294937b39de5924e772f8ed90d35c53290c8acab scheduler_tests: re-enable mockforward test (Anthony Towns) cea19f685915be8affb2203184a549576194413f Drop unused reverselock.h (Anthony Towns) d0ebd93270758ea97ea956b8821e17a2d001ea94 scheduler: switch from boost to std (Anthony Towns) b9c426012770d166e6ebfab27689be44e6e89aa5 sync.h: add REVERSE_LOCK (Anthony Towns) 306f71b4eb4a0fd8e64f47dc008bc235b80b13d9 scheduler: don't rely on boost interrupt on shutdown (Anthony Towns) Pull request description: Replacing boost functionality with C++11 stuff. Motivated by #18227, but should stand alone. Changing from `boost::condition_var` to `std::condition_var` means `threadGroup.interrupt_all` isn't enough to interrupt `serviceQueue` anymore, so that means calling `stop()` before `join_all()` is needed. And the existing reverselock.h code doesn't work with sync.h's DebugLock code (because the reversed lock won't be removed from `g_lockstack` which then leads to incorrect potential deadlock warnings), so I've replaced that with a dedicated class and macro that's aware of our debug lock behaviour. Fixes #16027, Fixes #14200, Fixes #18227 ACKs for top commit: laanwj: ACK 70a6b529f306ff72ea1badf25e970a92b2b17ab3 Tree-SHA512: d1da13adeabcf9186d114e2dad9a4fdbe2e440f7afbccde0c13dfbaf464efcd850b69d3371c5bf8b179d7ceb9d81f4af3cc22960b90834e41eaaf6d52ef7d331 # Conflicts: # src/reverselock.h # src/rpc/misc.cpp # src/scheduler.cpp # src/scheduler.h # src/sync.cpp # src/sync.h # src/test/reverselock_tests.cpp # src/test/scheduler_tests.cpp # src/test/test_dash.cpp # test/lint/extended-lint-cppcheck.sh
2020-03-06 20:47:49 +01:00
void schedule(Function f, std::chrono::system_clock::time_point t);
// Convenience method: call f once deltaMilliSeconds from now
void scheduleFromNow(Function f, int64_t deltaMilliSeconds);
// Another convenience method: call f approximately
// every deltaMilliSeconds forever, starting deltaMilliSeconds from now.
// To be more precise: every time f is finished, it
// is rescheduled to run deltaMilliSeconds later. If you
// need more accurate scheduling, don't use this method.
void scheduleEvery(Function f, int64_t deltaMilliSeconds);
/**
* Mock the scheduler to fast forward in time.
* Iterates through items on taskQueue and reschedules them
* to be delta_seconds sooner.
*/
void MockForward(std::chrono::seconds delta_seconds);
// To keep things as simple as possible, there is no unschedule.
// Services the queue 'forever'. Should be run in a thread,
// and interrupted using boost::interrupt_thread
void serviceQueue();
// Tell any threads running serviceQueue to stop as soon as they're
// done servicing whatever task they're currently servicing (drain=false)
// or when there is no work left to be done (drain=true)
void stop(bool drain=false);
// Returns number of tasks waiting to be serviced,
// and first and last task times
partial Merge #18234: refactor: Replace boost::mutex,condition_var,chrono with std equivalents in scheduler This backport does not include changes that depend on bitcoin pr 18037 70a6b529f306ff72ea1badf25e970a92b2b17ab3 lint-cppcheck: Remove -DHAVE_WORKING_BOOST_SLEEP_FOR (Anthony Towns) 294937b39de5924e772f8ed90d35c53290c8acab scheduler_tests: re-enable mockforward test (Anthony Towns) cea19f685915be8affb2203184a549576194413f Drop unused reverselock.h (Anthony Towns) d0ebd93270758ea97ea956b8821e17a2d001ea94 scheduler: switch from boost to std (Anthony Towns) b9c426012770d166e6ebfab27689be44e6e89aa5 sync.h: add REVERSE_LOCK (Anthony Towns) 306f71b4eb4a0fd8e64f47dc008bc235b80b13d9 scheduler: don't rely on boost interrupt on shutdown (Anthony Towns) Pull request description: Replacing boost functionality with C++11 stuff. Motivated by #18227, but should stand alone. Changing from `boost::condition_var` to `std::condition_var` means `threadGroup.interrupt_all` isn't enough to interrupt `serviceQueue` anymore, so that means calling `stop()` before `join_all()` is needed. And the existing reverselock.h code doesn't work with sync.h's DebugLock code (because the reversed lock won't be removed from `g_lockstack` which then leads to incorrect potential deadlock warnings), so I've replaced that with a dedicated class and macro that's aware of our debug lock behaviour. Fixes #16027, Fixes #14200, Fixes #18227 ACKs for top commit: laanwj: ACK 70a6b529f306ff72ea1badf25e970a92b2b17ab3 Tree-SHA512: d1da13adeabcf9186d114e2dad9a4fdbe2e440f7afbccde0c13dfbaf464efcd850b69d3371c5bf8b179d7ceb9d81f4af3cc22960b90834e41eaaf6d52ef7d331 # Conflicts: # src/reverselock.h # src/rpc/misc.cpp # src/scheduler.cpp # src/scheduler.h # src/sync.cpp # src/sync.h # src/test/reverselock_tests.cpp # src/test/scheduler_tests.cpp # src/test/test_dash.cpp # test/lint/extended-lint-cppcheck.sh
2020-03-06 20:47:49 +01:00
size_t getQueueInfo(std::chrono::system_clock::time_point &first,
std::chrono::system_clock::time_point &last) const;
// Returns true if there are threads actively running in serviceQueue()
bool AreThreadsServicingQueue() const;
private:
partial Merge #18234: refactor: Replace boost::mutex,condition_var,chrono with std equivalents in scheduler This backport does not include changes that depend on bitcoin pr 18037 70a6b529f306ff72ea1badf25e970a92b2b17ab3 lint-cppcheck: Remove -DHAVE_WORKING_BOOST_SLEEP_FOR (Anthony Towns) 294937b39de5924e772f8ed90d35c53290c8acab scheduler_tests: re-enable mockforward test (Anthony Towns) cea19f685915be8affb2203184a549576194413f Drop unused reverselock.h (Anthony Towns) d0ebd93270758ea97ea956b8821e17a2d001ea94 scheduler: switch from boost to std (Anthony Towns) b9c426012770d166e6ebfab27689be44e6e89aa5 sync.h: add REVERSE_LOCK (Anthony Towns) 306f71b4eb4a0fd8e64f47dc008bc235b80b13d9 scheduler: don't rely on boost interrupt on shutdown (Anthony Towns) Pull request description: Replacing boost functionality with C++11 stuff. Motivated by #18227, but should stand alone. Changing from `boost::condition_var` to `std::condition_var` means `threadGroup.interrupt_all` isn't enough to interrupt `serviceQueue` anymore, so that means calling `stop()` before `join_all()` is needed. And the existing reverselock.h code doesn't work with sync.h's DebugLock code (because the reversed lock won't be removed from `g_lockstack` which then leads to incorrect potential deadlock warnings), so I've replaced that with a dedicated class and macro that's aware of our debug lock behaviour. Fixes #16027, Fixes #14200, Fixes #18227 ACKs for top commit: laanwj: ACK 70a6b529f306ff72ea1badf25e970a92b2b17ab3 Tree-SHA512: d1da13adeabcf9186d114e2dad9a4fdbe2e440f7afbccde0c13dfbaf464efcd850b69d3371c5bf8b179d7ceb9d81f4af3cc22960b90834e41eaaf6d52ef7d331 # Conflicts: # src/reverselock.h # src/rpc/misc.cpp # src/scheduler.cpp # src/scheduler.h # src/sync.cpp # src/sync.h # src/test/reverselock_tests.cpp # src/test/scheduler_tests.cpp # src/test/test_dash.cpp # test/lint/extended-lint-cppcheck.sh
2020-03-06 20:47:49 +01:00
mutable Mutex newTaskMutex;
std::condition_variable newTaskScheduled;
std::multimap<std::chrono::system_clock::time_point, Function> taskQueue GUARDED_BY(newTaskMutex);
int nThreadsServicingQueue GUARDED_BY(newTaskMutex);
bool stopRequested GUARDED_BY(newTaskMutex);
bool stopWhenEmpty GUARDED_BY(newTaskMutex);
bool shouldStop() const EXCLUSIVE_LOCKS_REQUIRED(newTaskMutex) { return stopRequested || (stopWhenEmpty && taskQueue.empty()); }
};
/**
* Class used by CScheduler clients which may schedule multiple jobs
Merge #13247: Add tests to SingleThreadedSchedulerClient() and document the memory model cbeaa91dbb Update ValidationInterface() documentation to explicitly specify threading and memory model (Jesse Cohen) b296b425a7 Update documentation for SingleThreadedSchedulerClient() to specify the memory model (Jesse Cohen) 9994d01d8b Add Unit Test for SingleThreadedSchedulerClient (Jesse Cohen) Pull request description: As discussed in #13023 I've split this test out into a separate pr This test (and documentation update) makes explicit the guarantee (previously undefined, but implied by the 'SingleThreaded' in `SingleThreadedSchedulerClient()`) - that callbacks pushed to the `SingleThreadedSchedulerClient()` obey the single threaded model for memory and execution - specifically, the callbacks are executed fully and in order, and even in cases where a subsequent callback is executed by a different thread, sequential consistency of memory for all threads executing these callbacks is maintained. Maintaining memory consistency should make the api more developer friendly - especially for users of the validationinterface. To the extent that there are performance implications from this decision, these are not currently present in practice because all use of this scheduler happens on a single thread currently, furthermore the lock should guarantee consistency across callback executions even when callbacks are executed by multiple threads (as the test does). Tree-SHA512: 5d95a7682c402e5ad76b05bc9dfbca99ca64105f62ab9e78f6fc0f6ea8c5277aa399fbb94298e35cc677b0c2181ff17259584bb7ae230e38aa68b85ecbc22856
2018-08-01 02:50:18 +02:00
* which are required to be run serially. Jobs may not be run on the
* same thread, but no two jobs will be executed
* at the same time and memory will be release-acquire consistent
* (the scheduler will internally do an acquire before invoking a callback
* as well as a release at the end). In practice this means that a callback
* B() will be able to observe all of the effects of callback A() which executed
* before it.
*/
class SingleThreadedSchedulerClient {
private:
CScheduler *m_pscheduler;
CCriticalSection m_cs_callbacks_pending;
std::list<std::function<void ()>> m_callbacks_pending GUARDED_BY(m_cs_callbacks_pending);
bool m_are_callbacks_running GUARDED_BY(m_cs_callbacks_pending) = false;
void MaybeScheduleProcessQueue();
void ProcessQueue();
public:
explicit SingleThreadedSchedulerClient(CScheduler *pschedulerIn) : m_pscheduler(pschedulerIn) {}
Merge #13247: Add tests to SingleThreadedSchedulerClient() and document the memory model cbeaa91dbb Update ValidationInterface() documentation to explicitly specify threading and memory model (Jesse Cohen) b296b425a7 Update documentation for SingleThreadedSchedulerClient() to specify the memory model (Jesse Cohen) 9994d01d8b Add Unit Test for SingleThreadedSchedulerClient (Jesse Cohen) Pull request description: As discussed in #13023 I've split this test out into a separate pr This test (and documentation update) makes explicit the guarantee (previously undefined, but implied by the 'SingleThreaded' in `SingleThreadedSchedulerClient()`) - that callbacks pushed to the `SingleThreadedSchedulerClient()` obey the single threaded model for memory and execution - specifically, the callbacks are executed fully and in order, and even in cases where a subsequent callback is executed by a different thread, sequential consistency of memory for all threads executing these callbacks is maintained. Maintaining memory consistency should make the api more developer friendly - especially for users of the validationinterface. To the extent that there are performance implications from this decision, these are not currently present in practice because all use of this scheduler happens on a single thread currently, furthermore the lock should guarantee consistency across callback executions even when callbacks are executed by multiple threads (as the test does). Tree-SHA512: 5d95a7682c402e5ad76b05bc9dfbca99ca64105f62ab9e78f6fc0f6ea8c5277aa399fbb94298e35cc677b0c2181ff17259584bb7ae230e38aa68b85ecbc22856
2018-08-01 02:50:18 +02:00
/**
* Add a callback to be executed. Callbacks are executed serially
* and memory is release-acquire consistent between callback executions.
* Practically, this means that callbacks can behave as if they are executed
Merge #13247: Add tests to SingleThreadedSchedulerClient() and document the memory model cbeaa91dbb Update ValidationInterface() documentation to explicitly specify threading and memory model (Jesse Cohen) b296b425a7 Update documentation for SingleThreadedSchedulerClient() to specify the memory model (Jesse Cohen) 9994d01d8b Add Unit Test for SingleThreadedSchedulerClient (Jesse Cohen) Pull request description: As discussed in #13023 I've split this test out into a separate pr This test (and documentation update) makes explicit the guarantee (previously undefined, but implied by the 'SingleThreaded' in `SingleThreadedSchedulerClient()`) - that callbacks pushed to the `SingleThreadedSchedulerClient()` obey the single threaded model for memory and execution - specifically, the callbacks are executed fully and in order, and even in cases where a subsequent callback is executed by a different thread, sequential consistency of memory for all threads executing these callbacks is maintained. Maintaining memory consistency should make the api more developer friendly - especially for users of the validationinterface. To the extent that there are performance implications from this decision, these are not currently present in practice because all use of this scheduler happens on a single thread currently, furthermore the lock should guarantee consistency across callback executions even when callbacks are executed by multiple threads (as the test does). Tree-SHA512: 5d95a7682c402e5ad76b05bc9dfbca99ca64105f62ab9e78f6fc0f6ea8c5277aa399fbb94298e35cc677b0c2181ff17259584bb7ae230e38aa68b85ecbc22856
2018-08-01 02:50:18 +02:00
* in order by a single thread.
*/
void AddToProcessQueue(std::function<void ()> func);
// Processes all remaining queue members on the calling thread, blocking until queue is empty
// Must be called after the CScheduler has no remaining processing threads!
void EmptyQueue();
size_t CallbacksPending();
};
#endif