Skip to content

Phase B3 (part 1): DhtNode over its own ConnectionManager + Layer0 end-to-end tests#77

Merged
ptesavol merged 1 commit into
mainfrom
claude/phase-b3
Jul 11, 2026
Merged

Phase B3 (part 1): DhtNode over its own ConnectionManager + Layer0 end-to-end tests#77
ptesavol merged 1 commit into
mainfrom
claude/phase-b3

Conversation

@ptesavol

Copy link
Copy Markdown
Collaborator

First half of Phase B3 (trackerless-network-completion-plan.md, Milestone B hardening): the DhtNode self-managed connectivity path and the five Layer0 end-to-end tests over real sockets — the C++ side of the milestone's exit criterion. The C++↔TS interop test follows as part 2 (via the granular published @streamr npm packages, per review guidance).

DhtNode: the owned-ConnectionManager branch

Milestone A stubbed this out ("ConnectionManager path arrives in milestone B"). Now, when no transport is given (TS: options.transport === undefined), the node:

  • creates and owns a ConnectionManager over a DefaultConnectorFacade, with itself as the connectors' signalling transport (TS transport: this — WebRTC signalling rides the DHT's routed messages);
  • generates its signed local peer descriptor from the connectivity response (generatePeerDescriptorCallBack → B1's createPeerDescriptor; region from options or 0 — the GeoIP/CDN fallbacks stay deferred to milestone E);
  • honours a given peerDescriptor's websocket (serves on exactly that port) or the websocketHost/websocketPortRange options (TS DhtNode lines 231–243), and plumbs iceServers / webrtc buffer & port options / externalIp through;
  • new DhtNodeOptions: websocketHost, websocketPortRange, nodeId, region, iceServers, webrtcAllowPrivateAddresses, buffer thresholds, webrtcPortRange, externalIp, maxMessageSize.

stop() stops the owned manager but deliberately keeps the object alive: straggler detached tasks are only drained at the communicators' destruction, and a send into a stopped ConnectionManager is a guarded no-op while a send into a destroyed one is a use-after-free (TS keeps the stopped object alive via GC).

Fixed in passing — found by the first end-to-end run

WebsocketServerConnector's constructor initialized host from the moved-from ctor parameter (member init order follows declaration order, so the options member was move-initialized first). The advertised descriptor carried an empty websocket host, producing ws://:port connectivity URLs. I'd spotted this during the B1 work and wrongly assessed it benign — the end-to-end path was its first real consumer. Now initialized from the moved-into member, with a comment.

Tests

New streamr-dht-test-end-to-end target; all five ports run real websocket servers and real WebRTC data channels on 127.0.0.1, no simulator:

Test What it proves
Layer0Test 5 nodes, all with websocket servers, port-range contention, parallel joins
Layer0WebrtcTest serverless nodes: connectivity check + ws-client to the entry point, WebRTC between each other
Layer0MixedConnectionTypesTest mixed server/serverless network, both join orders
Layer0Layer1Test stream DHTs stacked on real-socket layer-0; closest-contact identity asserts
Layer0WebrtcLayer1Test layer-1 over an all-WebRTC layer-0, shared node ids
  • End-to-end 7/7, unit 181/181, lint green (the five e2e files join the documented clangd std-type-unification exclusion list; MockLayer1Layer0Test moved from the own-process lint leg to outright exclusion — DhtNode's BMI grew past the same false-positive cliff).

Known pre-existing flake (documented, not addressed here)

Layer1ScaleTest.SingleLayer1Dht (simulator-based, from milestone A) fails its neighbour-count assert (2 vs 4) in ~40% of isolated runs on a fast dev machine — observed once before these changes too, and this PR's changes are inert to its code path (transport-given + simulator). It passes reliably on CI, where ctest --repeat until-pass:2 also retries. Root-causing it is queued as follow-up work alongside the interop test.

🤖 Generated with Claude Code

…d-to-end tests

Completes the DhtNode branch that milestone A stubbed out: when no
transport is given (TS: options.transport === undefined), the node creates
and owns a ConnectionManager over a DefaultConnectorFacade, with ITSELF as
the connectors' signalling transport, generates its signed local peer
descriptor from the connectivity response (createPeerDescriptor, region
from options or 0 — GeoIP fallbacks stay deferred to milestone E), honours
a given peerDescriptor's websocket (fixed port) or the
websocketHost/websocketPortRange options, and plumbs the
iceServers/webrtc/externalIp options through. stop() stops the owned
manager but deliberately keeps it alive: straggler detached tasks are only
drained at the communicators' destruction, and a send into a STOPPED
ConnectionManager is a guarded no-op while a send into a destroyed one is
a use-after-free.

Fixed in passing (found by the first end-to-end run):
WebsocketServerConnector's constructor initialized `host` from the
moved-from ctor parameter (member init order follows declaration order, so
the options member was move-initialized first) — the advertised descriptor
carried an empty websocket host, producing `ws://:port` connectivity URLs.
Spotted during B1 review and wrongly assessed benign; the end-to-end path
was its first real consumer.

Tests ported (v103.8.0-rc.3), all over REAL sockets on 127.0.0.1 in a new
streamr-dht-test-end-to-end target: end-to-end/Layer0.test.ts,
Layer0Webrtc.test.ts (serverless nodes connect to each other over real
WebRTC), Layer0MixedConnectionTypes.test.ts, Layer0-Layer1.test.ts,
Layer0Webrtc-Layer1.test.ts. 7/7 green; unit 181/181; lint green (the five
e2e files join the documented clangd exclusion list; MockLayer1Layer0Test
moved from the own-process leg to outright exclusion — DhtNode's BMI grew
past the same false-positive cliff).

Known pre-existing flake, NOT addressed here:
Layer1ScaleTest.SingleLayer1Dht fails its neighbour-count assert (2 vs 4)
in ~40% of ISOLATED runs on this machine (observed once before these
changes too); it passes reliably on CI where until-pass:2 also retries.
Root-causing it is follow-up work, as is the C++<->TS interop test (B3
part 2, via the granular @streamr npm packages).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions github-actions Bot added the dht label Jul 11, 2026
@ptesavol ptesavol merged commit 838799e into main Jul 11, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant