Add heuristic fee engine and related services by markettes · Pull Request #533 · Elenpay/NodeGuard

markettes · 2026-07-08T10:42:24Z

Introduce foundational components for a heuristic routing engine, including new methods for LightningClientService, migrations for routing engine tables, and repositories for channel fee and routing states. Implement helper classes for block height and channel ownership, along with tests to ensure functionality. This update enhances dynamic fee management and automated rebalancing capabilities.

…e and LightningService

…tables

…ting states; add corresponding interfaces and tests

…ding tests

…on logic

…ng configuration

…fee management and automated rebalancing

Copilot

Pull request overview

Introduces the Phase 1 “foundation + signal” components for a heuristic routing engine, adding persisted routing/fee state models, a scheduled job to compute per-channel signals from forwarding history + live channel data, and UI/config surfaces to stage nodes for future Phase 2/3 actuation.

Changes:

Add ChannelRoutingState / ChannelFeeState entities + repositories and EF migration to persist routing-engine state.
Add TargetRatioReevaluationJob plus PeerCategorizationService (hysteresis categorization + EWMA smoothing) to compute targets and peer flow categories.
Extend services/UI/config: GetInfo/block height retrieval, routing-engine constants, and node settings UI for dynamic fees/rebalancing.

Reviewed changes

Copilot reviewed 31 out of 32 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/NodeGuard.Tests/Services/PeerCategorizationServiceTests.cs	Unit tests for categorization hysteresis + target/EMA smoothing math.
test/NodeGuard.Tests/Helpers/ChannelOwnershipHelperTests.cs	Tests ownership dedup logic for managed↔managed channels.
test/NodeGuard.Tests/Helpers/BlockHeightHelperTests.cs	Tests SCID funding-height/age parsing and alias/zero handling.
test/NodeGuard.Tests/Data/Repositories/RebalanceRepositoryRoutingEngineTests.cs	Tests new rebalance in-flight count and budget-consumption calculations.
test/NodeGuard.Tests/Data/Repositories/ChannelRoutingStateRepositoryTests.cs	Tests routing-state upsert semantics and filtering by node pubkey.
test/NodeGuard.Tests/Data/Repositories/ChannelFlowAnalyticsRepositoryTests.cs	Tests settled-only, windowed per-channel forwarding aggregations.
src/Services/PeerCategorizationService.cs	Adds categorization decision record + core categorization/target smoothing logic.
src/Services/LightningService.cs	Adds `GetBlockHeight` to expose chain tip via LND GetInfo.
src/Services/LightningClientService.cs	Adds `GetInfo` RPC wrapper with error handling/logging.
src/Program.cs	Registers new repositories/service and schedules the new Quartz job/trigger.
src/Pages/Nodes.razor	Adds routing-engine configuration fields (dynamic fees/rebalance budgets/dry-run) to node UI.
src/Migrations/ApplicationDbContextModelSnapshot.cs	Updates EF snapshot with routing-engine entities and new columns/defaults.
src/Migrations/20260707155251_AddRoutingEngineFoundation.Designer.cs	Migration designer for routing-engine foundation schema changes.
src/Migrations/20260707155251_AddRoutingEngineFoundation.cs	Adds new node/channel columns and creates `ChannelRoutingStates` / `ChannelFeeStates` tables.
src/Jobs/TargetRatioReevaluationJob.cs	Implements Phase 1 job to compute/store routing signals and target ratios per channel.
src/Helpers/Constants.cs	Adds routing-engine constants and env var parsing.
src/Helpers/ChannelOwnershipHelper.cs	Adds helper to decide managed-node channel “ownership” for dedup.
src/Helpers/BlockHeightHelper.cs	Adds helper to parse funding height and channel age from SCID.
src/Data/Repositories/RebalanceRepository.cs	Adds in-flight count and consumed-fees calculations for routing-engine budgeting.
src/Data/Repositories/Interfaces/IRebalanceRepository.cs	Declares new rebalance query methods for Phase 3 budgeting/caps.
src/Data/Repositories/Interfaces/IChannelRoutingStateRepository.cs	New repository interface for routing-state persistence/upsert.
src/Data/Repositories/Interfaces/IChannelFlowAnalyticsRepository.cs	New interface for settled-only forwarding flow aggregations.
src/Data/Repositories/Interfaces/IChannelFeeStateRepository.cs	New interface for fee-state persistence/upsert (Phase 2).
src/Data/Repositories/ChannelRoutingStateRepository.cs	Implements routing-state get/upsert logic keyed by ChannelId.
src/Data/Repositories/ChannelFlowAnalyticsRepository.cs	Implements settled-only forwarding aggregations over `ForwardingHtlcEvents`.
src/Data/Repositories/ChannelFeeStateRepository.cs	Implements fee-state get/upsert logic keyed by ChannelId.
src/Data/Models/Rebalance.cs	Adds `ReservedFeeSats` for in-flight budget accounting.
src/Data/Models/Node.cs	Adds routing-engine node-level feature flags and budget/cap settings.
src/Data/Models/ChannelRoutingState.cs	Adds routing-state entity + `PeerFlowCategory` enum.
src/Data/Models/ChannelFeeState.cs	Adds fee-state entity for future fee engine (baseline/last-applied/circuit breaker).
src/Data/Models/Channel.cs	Adds per-channel opt-out flag `IsDynamicFeeEnabled` (default true).
src/Data/ApplicationDbContext.cs	Wires up new DbSets, 1:1 mappings, and default values for new flags.

Files not reviewed (1)

src/Migrations/20260707155251_AddRoutingEngineFoundation.Designer.cs: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…ent variable

Jossec101 · 2026-07-13T08:29:39Z

+    /// Fee reserved for this in-flight rebalance so its spend counts against the node budget
+    /// before <see cref="FeePaidSats"/> settles (Phase 3 in-flight budget accounting).
+    /// </summary>
+    public long? ReservedFeeSats { get; set; }


I dont fully get this field, if the rebalance is in pending already you know the amount it's counted into the budget?

Jossec101 · 2026-07-13T08:36:25Z

+/// Per-channel fee-engine state (1:1 with <see cref="Channel"/>). Declared in the Phase 1
+/// migration so Phase 2 needs no schema change; rows are not created or populated until the


I dont like this "phase 1" comments coming from the LLM, They should be todo or pure comments outside the summary as this can get obsolete if not changed

yes, will remove all this

Jossec101 · 2026-07-13T08:37:36Z

+
+    /// <summary>
+    /// Circuit-breaker counter: incremented on each consecutive failure to apply a fee update,
+    /// reset to 0 on success.
+    /// </summary>
+    public int ConsecutiveFailures { get; set; } = 0;


not really, just a feature, will remove it

Jossec101 · 2026-07-13T09:14:54Z

+    /// <summary>
+    /// Proportional gain mapping net-flow to target drift: target_goal = 0.5 + clamp(K·netFlowRatio, ±maxDrift).
+    /// Default 0.70. Env: ROUTING_ENGINE_TARGET_K.
+    /// </summary>
+    public static double ROUTING_ENGINE_TARGET_K = 0.70;


Can you simplify what's this field about?

Jossec101 · 2026-07-13T09:16:04Z

+
+    /// <summary>
+    /// Maximum drift of target_goal away from 0.5 (reached at |netFlowRatio| = 0.5). Default 0.35,
+    /// giving a target_goal range of [0.15, 0.85]. Env: ROUTING_ENGINE_TARGET_MAX_DRIFT.
+    /// </summary>
+    public static double ROUTING_ENGINE_TARGET_MAX_DRIFT = 0.35;
+
+    /// <summary>
+    /// EWMA smoothing factor folding target_goal into the stored TargetLocalRatio. Default 0.10
+    /// (~10 cycles to converge). Env: ROUTING_ENGINE_TARGET_ALPHA.
+    /// </summary>
+    public static double ROUTING_ENGINE_TARGET_ALPHA = 0.10;
+
+    /// <summary>
+    /// EWMA smoothing factor for EmaLocalRatio (~24h effective window at 30-min sampling).
+    /// Default 0.04. Env: ROUTING_ENGINE_FEE_EMA_ALPHA.
+    /// </summary>
+    public static double ROUTING_ENGINE_FEE_EMA_ALPHA = 0.04;
+
+    /// <summary>
+    /// Consecutive divergent cycles required before a category flip commits (anti-flap hysteresis).
+    /// Default 3. Env: ROUTING_ENGINE_CATEGORY_FLIP_HYSTERESIS_CYCLES.
+    /// </summary>
+    public static int ROUTING_ENGINE_CATEGORY_FLIP_HYSTERESIS_CYCLES = 3;


Same, clearer summaries help and examples also

I have added clarifying formulas to the alpha factors for the target and simpler descriptions

Jossec101 · 2026-07-13T09:19:29Z

+                if (channel.Status == Channel.ChannelStatus.Open && channel.ChanId != 0)
+                {


Also check if the channel is active/enabled? Check Availability on channels

added here:

NodeGuard/src/Jobs/TargetRatioReevaluationJob.cs

Lines 154 to 158 in a706fec

// Act only on channels that are active.

if (!lndChannel.Active)

{

continue;

}

Jossec101 · 2026-07-13T10:28:30Z

+                        <Field>
+                            <FieldLabel>Restore Baseline on Disable</FieldLabel>
+                            <FieldHelp>When dynamic fees are turned off, restore each channel's captured fee baseline in one final write. When off, last-set fees are frozen in place.</FieldHelp>
+                            <Check TValue="bool" Checked="@_selectedNodeForLiquidity.RestoreFeeBaselineOnDisable"
+                                   CheckedChanged="@((value) => _selectedNodeForLiquidity.RestoreFeeBaselineOnDisable = value)">
+                                Restore fee baseline on disable
+                            </Check>
+                        </Field>


I dont think this has any benefit tbh

Okay, I thought this was a good idea. If a node is disabled, you will have to change fees for every single channel, and this solves it.

I mean what's the issue leaving old fees as they are?

we would leave them mid strategy instead in a market avg fee. No strong in this opinion though

… repository and remove unused repository interfaces

…lated documentation

…model and related migrations

…Program.cs

…parameters

…rations

…utingEngineTests

…tionJob

markettes added 7 commits July 7, 2026 13:47

feat: add GetInfo and GetBlockHeight methods to LightningClientServic…

0fa6aa5

…e and LightningService

feat(migrations): add routing engine foundation with new columns and …

c60b85a

…tables

feat: implement repositories for channel fee, flow analytics, and rou…

42b04f4

…ting states; add corresponding interfaces and tests

feat: add BlockHeightHelper and ChannelOwnershipHelper with correspon…

7cf82ce

…ding tests

feat: add PeerCategorizationService with tests for category computati…

1a4ffeb

…on logic

feat: add TargetRatioReevaluationJob for routing engine with scheduli…

cd75994

…ng configuration

feat: add heuristic routing engine configuration options for dynamic …

62a35c8

…fee management and automated rebalancing

markettes requested a review from Copilot July 8, 2026 10:43

Copilot started reviewing on behalf of markettes July 8, 2026 10:44 View session

Merge branch 'main' into feat/heuristic-fee-engine-phase1

1e31d41

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Comment thread src/Data/Repositories/RebalanceRepository.cs

Comment thread src/Helpers/Constants.cs Outdated

Comment thread src/Data/Repositories/ChannelFlowAnalyticsRepository.cs Outdated

Comment thread src/Jobs/TargetRatioReevaluationJob.cs

markettes requested a review from Jossec101 July 8, 2026 12:25

feat: update ROUTING_ENGINE_DRY_RUN to handle null values in environm…

def4d49

…ent variable

This was referenced Jul 8, 2026

feat: FeeOptimizerService + positive inbound fees (routing engine Phase 2a) #534

Closed

feat: ChannelFeeOptimizerJob dynamic fee actuator (routing engine Phase 2b) #535

Closed

Jossec101 reviewed Jul 13, 2026

View reviewed changes

Comment thread src/Data/Repositories/ChannelFlowAnalyticsRepository.cs Outdated

Jossec101 reviewed Jul 13, 2026

View reviewed changes

markettes added 11 commits July 13, 2026 17:39

chore: rename test file

789bb6f

feat: migrate channel flow analytics methods to forwarding HTLC event…

75539ca

… repository and remove unused repository interfaces

feat: update RoutingEngineDryRun default value to false and adjust re…

cb3ef19

…lated documentation

fix: change default value of IsDynamicFeeEnabled to false in Channel …

bf9f5dc

…model and related migrations

feat: remove PeerCategorizationService from service registrations in …

a38cc5e

…Program.cs

fix: remove every mention to phases or development steps

b9d125d

docs: enhance comments in Constants.cs for clarity on routing engine …

b5e6f0c

…parameters

refactor: remove circuit breaker feature

e5e4ac2

refactor: remove ReservedFeeSats from Rebalance model and related mig…

0bab265

…rations

test: clarify comments and update test logic in RebalanceRepositoryRo…

02058b4

…utingEngineTests

fix: ensure only active channels are processed in TargetRatioReevalua…

a706fec

…tionJob

markettes requested a review from Jossec101 July 15, 2026 10:18

		/// Per-channel fee-engine state (1:1 with <see cref="Channel"/>). Declared in the Phase 1
		/// migration so Phase 2 needs no schema change; rows are not created or populated until the

		if (channel.Status == Channel.ChannelStatus.Open && channel.ChanId != 0)
		{

	// Act only on channels that are active.
	if (!lndChannel.Active)
	{
	continue;
	}

Uh oh!

Conversation

markettes commented Jul 8, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants