mining: add getMemoryLoad() and track template non-mempool memory footprint#33922
mining: add getMemoryLoad() and track template non-mempool memory footprint#33922Sjors wants to merge 19 commits into
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33922. ReviewsSee the guideline and AI policy for information on the review process.
If your review is incorrectly listed, please copy-paste ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
|
I haven't benchmarked this yet on mainnet, so I'm not sure if checking every (unique) transaction for mempool presence is unacceptably expensive. If people prefer, I could also add a way for the |
21ad8c1 to
f22413f
Compare
|
🚧 At least one of the CI tasks failed. HintsTry to run the tests locally, according to the documentation. However, a CI failure may still
Leave a comment here, if you need help tracking down a confusing failure. |
|
|
||
| TxTemplateMap& tx_refs{*Assert(m_tx_template_refs)}; | ||
| // Don't track the dummy coinbase, because it can be modified in-place | ||
| // by submitSolution() |
There was a problem hiding this comment.
b9306b79b8f5667a2679236af8792bb1c36db817: in addition, we might be wiping the dummy coinbase from the template later: Sjors#106
f22413f to
3b77529
Compare
There was a problem hiding this comment.
Concept ACK
I think it would be better if we have internal memory management for the mining interface IPC, since we hold on to the block templates.
I would suggest the following approach:
- Add memory budget for the mining interface.
- Introduce a tracking list of recently built block templates and total memory usage.
- Add templates to the list and increment the memory usage after every
createnewblockorwaitnextreturn. - Whenever the memory budget is exhausted, we should release templates in FIFO order.
I think since we create a new template after a time interval elapses even if fees increase and that interval is usually enough for the client to receive and distribute the template to miners, this mechanism should be safe as the miners have long switch to most recent template when the budget elapsed because of the time interval being used in between returns of waitnext.
Mining interface clients should also handle their own memory internally.
Currently, I don’t see much use for the exposed getMemoryLoad method. In my opinion, we should not rely on the IPC client to manage our memory.
It seems counter intuitive, but from a memory management perspective IPC clients are treated no different than our own code. And if we started FIFO deleting templates that are used by our own code, we'd crash. So I think FIFO deletion should be a last resort (not implemented here). There's another reason why we should give clients an opportunity to gracefully release templates in whatever order they prefer. Maybe there's 100 downstream ASIC's, one of which is very slow at loading templates, so it's only given a new template when the tip changes, not when there's a fee change. In that scenario you have a specific template that the client wants to "defend" at all cost. In practice I'm hoping none of this matters and we can pick and recommend defaults that make it unlikely to get close to a memory limit, other than during some weird token launch. |
IMHO I think we should separate that, and treat clients differently from our own code, because they are different codebases and separate applications with their own memory.
I see your point but I don’t think that’s a realistic scenario, and I think we shouldn’t design software to be one-size-fits-all.
Delegating template eviction responsibility to the client can put us in a situation where they handle it poorly and cause us to OOM (but I guess your argument is that we rather take that chance than being in a situation where we make miners potentially lose on rewards). |
Note that it's already the clients responsibility, that's inherent to how multiprocess works. In the scenario where they handle it poorly, we can use FIFO deletion. All
We currently don't track whether any given
Afaik that means revalidating the block from scratch, removing one advantage the |
|
I tracked the non-mempool transaction memory footprint for half a day on mainnet, using fairly aggressive template update criteria (minimum fee delta 1 sat and no more than once per second). So far the footprint is minuscule, but of course this depends on the mempool weather: 
The memory spike after each new block is because |
3b77529 to
24592b7
Compare
|
I restructured the implementation and commits a bit. The It's also less code churn because I don't have to touch the It also made it easier to move This in turn let me split out a separate commit that introduces the actual I added some comments to point out that we don't hold a |
24592b7 to
03dcfae
Compare
|
One caveat is that Expanded the PR description. |
03dcfae to
ac1e97a
Compare
|
🚧 At least one of the CI tasks failed. HintsTry to run the tests locally, according to the documentation. However, a CI failure may still
Leave a comment here, if you need help tracking down a confusing failure. |
Add BlockTemplateManager, a wrapper around BlockAssembler::CreateNewBlock(), and store it in NodeContext. Wire it into node init and test setup so it is reset before its mempool/chainman dependencies, and update tests and fuzz setups that rebuild chainman or mempool. Add a unit test that verifies a block template can be created through the manager.
Pass the parsed mining args to BlockTemplateManager at construction and expose them via BlockCreateArgs(), so the manager owns the init-time block create options instead of NodeContext.
Move SubmitBlockStateCatcher and SubmitBlock from miner.cpp into BlockTemplateManager as a member function. This groups block submission with block creation in the same class. The function uses m_chainman directly instead of taking it as a parameter.
Move the mining tip lookup and block-template waiting helpers (GetTip, WaitTipChanged, WaitAndCreateNewBlock, InterruptWait, CooldownIfHeadersAhead) into BlockTemplateManager so the manager owns the template waiting flow. The manager now takes KernelNotifications at construction.
Route the Mining interface's createNewBlock() through BlockTemplateManager::CreateNewTemplate() instead of constructing a BlockAssembler directly. Merging the init-time defaults into unset options now happens inside CreateNewTemplate(), so every caller gets them applied.
Add EnsureBlockTemplateManager() and use it in getblocktemplate for tip lookup and longpoll waiting, while using ChainstateManager directly for test-chain and IBD checks.
The transactions, fees and sigops costs, and the coinbase outputs are only read, so reference them from the template instead of copying them out of it.
The block copy exists solely to apply header adjustments (time, nonce, version bits) that must not mutate the cached template. Now that the transactions are read directly from the template, copy only the header instead of the entire block.
generateblock is no longer safe for fuzzing: it previously threw at EnsureMining (node.mining is never set in the fuzz setup), but with EnsureBlockTemplateManager it now executes, mining and submitting a real block. Submission mutates the chain state shared across fuzz iterations and writes the block to disk.
The waitforblock, waitforblockheight and waitfornewblock RPCs are no longer safe for fuzzing: they previously threw at EnsureMining (node.mining is never set in the fuzz setup), but with EnsureBlockTemplateManager they now really wait. The tip never changes during fuzzing and nothing interrupts the wait, so a call without a timeout blocks forever.
|
Rebased to track the latest #35675. |
IPC clients can hold on to block templates indefinately, which has the same impact as when the node holds a shared pointer to the CBlockTemplate. Because each template in turn tracks CTransactionRefs, transactions that are removed from the mempool will not have their memory cleared. This commit adds bookkeeping to the block template constructor and destructor that will let us track the resulting memory footprint. Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Calculate the non-mempool memory footprint for template transaction references. Add bench logging to collect data on whether caching or simplified heuristics are needed, such as not checking for mempool presence. Check the calculation in the block template manager fuzz test.
Allow IPC clients to inspect the amount of memory consumed by non-mempool transactions in blocks. Returns a MemoryLoad struct which can later be expanded to e.g. include a limit. Expand the interface_ipc.py test to demonstrate the behavior and to illustrate how clients can call destroy() to reduce memory pressure.
|
🚧 At least one of the CI tasks failed. HintsTry to run the tests locally, according to the documentation. However, a CI failure may still
Leave a comment here, if you need help tracking down a confusing failure. |
9 participants
Implements a way to track the memory footprint of all non-mempool transactions that are still being referenced by block templates, see discussion in #33899. It does not impose a limit.
IPC clients can query this footprint (total, across all clients) using the
getMemoryLoad()IPC method. Its client-side usage is demonstrated here:Additionally, the functional test in
interface_ipc.pyis expanded to demonstrate how template memory management works: templates are not released until the client drops references to them, or calls the template destroy method, or disconnects. The destroy method is called automatically by clients using libmultiprocess, as sv2-tp does. In the Python tests it also happens when references are destroyed or go out of scope.Based on:
Commits:
TxTemplateMaptoBlockTemplateManagerto track how many templates contain any given transaction. This map is updated by theBlockTemplateconstructor and destructor.