feat(indexer): implement DIN Protocol subgraph — platform + task-level contracts#29
Open
robertocarlous wants to merge 18 commits into
Open
Conversation
Audits all events emitted by DINModelRegistry, DinValidatorStake, DinCoordinator, and DinToken against indexer reconstruction requirements. Findings: - ModelRegistrationRequested: missing isOpenSource and fee — cannot filter pending requests by type or surface fee amounts without a storage call - ManifestUpdateRequested: missing requester address — cannot attribute pending updates to their submitter - DinCoordinator.withdraw(): no event emitted — ETH treasury outflows are invisible to the indexer - SlasherContractAdded/Removed: identical signatures on DinCoordinator and DinValidatorStake — resolved at schema level via sourceContract field, no contract change needed - ValidatorJailed path is dead code (jailedUntil is never set); flagged for the P3-4.x slashing spec to ensure a jailing event lands when that mechanism is added Proposed additions (confirmed with Umer, 2026-07-09 — land as one PR after audit review): 1. ModelRegistrationRequested: + bool isOpenSource, uint256 fee 2. ManifestUpdateRequested: + address indexed requester 3. DinCoordinator: new ETHTreasuryWithdrawn(address indexed to, uint256 amount) Includes daemon event schema table (P4-7.1 feed) mapping platform events to dind subscription requirements and coverage gaps.
…ph (P4-IDX1)
Defines all queryable entities for the DIN platform subgraph covering
DINModelRegistry, DinValidatorStake, DinCoordinator, and DinToken.
Entities:
Registry: ModelRegistrationRequest, Model, ManifestUpdateRequest,
ModelKillSwitchEvent, FeeSnapshot, RegistryFeeWithdrawal,
DAOAdminTransfer
Staking: Validator (aggregate), StakeEvent, SlashEvent, UnstakeRequest,
WithdrawalClaim, SlasherRegistration
Coordinator: DINMintEvent, TreasuryWithdrawal, ExchangeRateSnapshot
Token: TokenMintEvent, TokenTransferEvent
Design notes:
- SlasherRegistration carries sourceContract to keep DinCoordinator and
DinValidatorStake event streams distinguishable (identical event signatures)
- ModelRegistrationRequest.isOpenSource / .feePaid and ManifestUpdateRequest
.requester are marked pending the §6.1/6.2 event additions; mapping handlers
will use storage calls as a bridge until those events land
- TreasuryWithdrawal entity is defined but will not be populated until the
ETHTreasuryWithdrawn event addition (audit §6.3) lands
- Task-level contracts (DINTaskCoordinator/DINTaskAuditor) are out of scope;
deferred to P5+ dynamic data sources
Defines the Graph Protocol subgraph manifest covering all four platform
contracts on the local development chain (chainId 1337, network: din-local).
Data sources:
- DINModelRegistry → src/registry.ts (15 event handlers)
- DinValidatorStake → src/staking.ts (8 event handlers)
- DinCoordinator → src/coordinator.ts (5 event handlers + 1 commented-out
pending ETHTreasuryWithdrawn addition)
- DinToken → src/token.ts (2 event handlers)
ABIs copied from dincli/abis/ into subgraph/abis/ — single source of truth
until a build step syncs them automatically.
Contract addresses are placeholders (0x000...001–004); replace with actual
deployed addresses after running the platform contract deployment script.
SlasherContractAdded/Removed duplicate-signature handling: DinValidatorStake
handlers produce the canonical slasher-set records; DinCoordinator handlers
produce audit-trail records with sourceContract set to the coordinator address.
Defines which platform-contract events the dind daemon must subscribe to,
the reaction each event triggers in the daemon job queue, and all current
coverage gaps. Handoff document from P4-IDX1 design to Santiago's P4-7.1
on-chain event listening engine.
Contents:
§1 Subscription priority — immediate (blacklist/kill-switch), high-priority
(slash/approval/manifest), and informational buckets
§2 Full event payload reference for all daemon-consumed events, including
PENDING field annotations for the two confirmed additions (audit §6.1/6.2)
§3 Coverage gaps:
- Task-level GI events (T1AggregationStarted etc.) not in platform
contracts — listed as P5+ task-level indexing requirement with the
exact event schema needed for daemon filtering
- ValidatorJailed dead code path — required event shape documented for
when P3-4.x slashing spec adds the jailing mechanism
- ETHTreasuryWithdrawn missing from DinCoordinator.withdraw()
§4 Subscription mechanics recommendation for Santiago (WebSocket
eth_subscribe, lastProcessedBlock per-contract, replay on restart)
§5 Integration order — dincli → SDK extraction → dind, per Umer 2026-07-09
…ontracts
Five source files covering all 30 event handlers declared in subgraph.yaml.
src/utils.ts
- eventId(): stable txHash-logIndex entity IDs for event-based records
- loadOrCreateValidator(): initialises Validator aggregate on first encounter
- syncValidatorStatus(): mirrors DinValidatorStake._syncValidatorStatus()
to keep the Validator.status field in sync after every mutation
src/registry.ts (DINModelRegistry — 15 handlers)
- Registration request lifecycle: Requested → Approved / Rejected
- Bridge for missing event fields: modelRequests(id) storage call populates
isOpenSource, feePaid, manifestCID, taskCoordinator, taskAuditor until the
audit §6.1/6.2 event additions land
- Manifest update lifecycle: Requested → Updated / Rejected; requester
derived from Model.owner as bridge until §6.2 addition lands
- Kill switch: ModelDisabled / ModelEnabled update Model.disabled and emit
ModelKillSwitchEvent
- Fee snapshots: individual setters read full fee schedule from storage so
every FeeSnapshot carries all four values; FeesUpdated reads from event
params directly (atomic — no storage call needed)
- FeesWithdrawn → RegistryFeeWithdrawal
- DAOAdminUpdated → DAOAdminTransfer
src/staking.ts (DinValidatorStake — 8 handlers)
- ValidatorStaked / Slashed / UnstakeRequested / WithdrawalClaimed update the
Validator aggregate and create the corresponding history entities
- Slash handler mirrors contract's active-stake-first, then pending-withdrawal
deduction logic for accurate aggregate tracking
- Blacklisted / Unblacklisted flip Validator.blacklisted and re-sync status
- SlasherContractAdded/Removed: canonical slasher-set records with
sourceContract = DinValidatorStake address
src/coordinator.ts (DinCoordinator — 5 handlers)
- EthDepositAndDINminted → DINMintEvent
- DinPerEthUpdated → ExchangeRateSnapshot
- ValidatorStakeContractUpdated: intentional no-op (one-time deploy wiring)
- SlasherContractAdded/Removed: audit-trail SlasherRegistration records with
sourceContract = DinCoordinator address (distinct from staking.ts records)
- ETHTreasuryWithdrawn handler stubbed and commented pending audit §6.3
src/token.ts (DinToken — 2 handlers)
- TokensMinted → TokenMintEvent
- Transfer → TokenTransferEvent (covers mints from 0x0 and normal transfers)
Runs graph-node, IPFS (kubo), and Postgres against the local anvil chain
(chainId 1153 1337, network: din-local) for subgraph development and testing.
Services:
graph-node graphprotocol/graph-node:v0.34.1
- ports 8000 (HTTP), 8001 (WS), 8020 (deploy), 8030 (status), 8040 (metrics)
- ethereum env var maps din-local → host.docker.internal:8545
- GRAPH_ETHEREUM_TARGET_TRIGGERS_PER_BLOCK_RANGE=1 for on-demand anvil blocks
ipfs ipfs/kubo:v0.27.0
- port 5001 (API); used by graph-node and graph deploy
postgres postgres:16-alpine
- POSTGRES_INITDB_ARGS sets UTF8/C locale required by graph-node TOAST
Both ipfs and postgres have healthchecks; graph-node depends_on postgres
with condition: service_healthy so it waits for the DB to be ready before
attempting to connect.
.env.example documents the ETHEREUM_RPC and Postgres credential overrides.
Linux users must replace host.docker.internal with 172.17.0.1.
Startup sequence is documented in the docker-compose.yml header comment.
Replace the two for-idx-in-range enumeration loops in list-pending-requests with GraphQL queries to the local subgraph, with a silent fallback to the original RPC loops when the graph-node endpoint is unavailable. Add unit tests covering the GraphQL happy path and all three fallback triggers (connection error, GraphQL error body, HTTP 5xx).
Documents the integration point, GraphQL queries used, fallback behaviour, DIN_SUBGRAPH_URL config, and follow-up RPC loop candidates in lms.py and aggregation.py.
…slash history (P4-IDX2)
…ions from P4-IDX (P4-IDX3)
Member
|
@robertocarlous One correction and a note on what to work on next. Correction — there is no Phase 5. The "deferred to P5+" framing in this PR description and in the earlier issue discussion was a mistake on my part: Next step: until your next weekly task is assigned, continue working on the subgraph — extend it to the task-level contracts ( |
…nd DINTaskAuditor
…gistration entities
…Update, GIStateTransition entities
… source templates
… DINTaskAuditor templates taskCoordinator.ts — four handlers covering aggregator registration, T1/T2 batch creation, and aggregator slashing. GlobalIteration is lazily instantiated on the first event per GI. taskAuditor.ts — eight handlers covering auditor registration, audit batch creation, scoring, eligibility voting/finalization, pass score governance, and auditor slashing. Introduces loadOrCreateLMS() which pre-creates a stub LocalModelSubmission (zero client/modelCID) on first reference so the required relationship field on AuditScoreSubmission, EligibilityVote, and EligibilityResult is never null at query time; stubs are upgraded in-place once the LocalModelSubmitted event lands.
robertocarlous
changed the title
Collaborator
Author
|
@umermjd11 Pls can you take a look at this, thank you |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the full DIN Protocol subgraph covering both platform-level and
task-level contracts, along with GraphQL integration into the CLI.
P4-IDX1 — Design
subgraph/docs/event-coverage-audit.md) — per-contractevent table, gap analysis, and three proposed contract-side additions
subgraph/schema.graphql) — 17 entities covering modelregistration, manifest updates, validators, stake/slash/withdrawal, slasher
registrations, DIN mint events, and token transfers
subgraph/subgraph.yaml) — 4 data sources, 31 handlers,network
din-local(chainId 1337)subgraph/docs/daemon-event-schema.md) — subscriptionpriority table for
dind, coverage gaps, WebSocket recommendation for P4-7.1P4-IDX2 — Implementation
subgraph/src/) — one file per contract:registry.ts(15 handlers),staking.ts,coordinator.ts,token.ts,shared
utils.ts. Storage call bridge used for pending event additionssubgraph/docker-compose.yml) — graph-node v0.34.1subgraph/docs/example-queries.md) — 14 queries acrossvalidator registry, model registry, slash history, and slasher audit trail
P4-IDX3 — CLI Integration
dincli/cli/dindao.pywith GraphQLqueries; silent fallback to RPC on connection error, HTTP error, or GraphQL
error body
tests/test_list_pending_requests.py) — 6 tests, no live noderequired
subgraph/docs/handoff.md) —DIN_SUBGRAPH_URLconfig,remaining RPC loop candidates in
lms.pyandaggregation.pyDeveloper/issues/indexer.md) — resolved all 5 openquestions, confirmed no stale references, added artefacts table
Task-level contracts (DINTaskCoordinator / DINTaskAuditor)
Implemented following the contract event audit and Umer's design confirmation.
Documentation/technical/audits/task-contract-event-audit.md)— existing events, 23-site silent GI state machine, missing submission and
finalization events, and six proposed additions with exact signatures and emit sites
GlobalIteration,aggregator/auditor registrations, T1/T2 batches, audit batches, local model
submissions, scoring, eligibility voting/finalization, slash events, pass score
governance, and GI state transition history
subgraph.yaml— oneDINTaskCoordinator+DINTaskAuditorpair instantiated per model whenModelApprovedfires onDINModelRegistryregistry.tsupdated to callcreateWithContext()with shared context(modelId, paired contract addresses) so handlers on either side resolve the
model back-reference and paired address without extra storage calls
taskCoordinator.ts— 4 handlers: aggregator registration, T1/T2 batchcreation, aggregator slashing
taskAuditor.ts— 8 handlers: auditor registration, audit batch creation,scoring, eligibility voting/finalization, pass score governance, auditor
slashing. Includes
loadOrCreateLMS()stub so required relationship fieldsare never null before
LocalModelSubmittedevent landsPending — blocked on contract event additions
Five handlers are commented out in
subgraph.yamlpending the additionsproposed in the audit (§4). All schema fields and handler stubs are in place.
GIStateChanged(uint GI, uint8 newState)LocalModelSubmitted(uint GI, uint modelIndex, address client, bytes32 cid)T1AggregationSubmitted/T2AggregationSubmittedT1BatchFinalized/T2FinalizedUncommenting the entries in
subgraph.yamland regenerating the ABI is allthat is needed once these events land in the contracts.