start DIN-SDK

[umeradl]

Branch: feat/din-sdk (from develop)
Assignee: @Santiagocetran
Roadmap ref: P4-1.2 (CLI Integration Layer — Shared SDK Extraction) · P3-DOC7 (dind Architecture Document)

Long-running issue (expected to stay open 1–2 months). A detailed task file under Developer/tasks/ will follow — this issue tracks the overall effort, not a weekly task.

Goal

Extract the logic-dependent parts of dincli commands into a shared SDK layer at dincli/sdk/, so the same logic can be driven by both the interactive CLI and the upcoming DIN daemon (dind).

Design

• All logic lives in dincli/sdk/. CLI command modules become thin wrappers: parse arguments → call SDK function → render output.
• SDK functions return structured JSON outputs. This is the contract between the SDK and its two consumers:
  • the CLI reads the JSON and renders terminal UI/UX (tables, colors, human-readable messages);
  • dind reads the same JSON to update its configuration and job queue.
• The CLI is refactored onto the SDK in this same branch — the branch is done when dincli commands run through the SDK layer and all existing tests pass.
• No circular dependencies: SDK must be importable standalone (IPFS client, contract interfaces, wallet/account helpers, manifest loading — see P4-1.2 in Developer/ROADMAP.md for the candidate module list).

Scope

• Inventory dincli command modules and identify which logic moves to dincli/sdk/
• Define the JSON output schema/conventions for SDK functions (status, data, error codes)
• Extract shared modules (IPFS, contracts, wallet/keystore, manifest/config) into the SDK
• Refactor CLI commands to call SDK functions and render their JSON output
• All existing tests pass; add SDK-level tests where extraction changes call paths
• Document the public SDK interface and backward-compatibility guarantees for existing dincli commands

References

• Developer/ROADMAP.md — P4-1.2, P3-DOC7, and the P4 phase overview ("Daemon and Agentic AI Layer")
• Developer/issues/containerization.md — worker/container runner must be callable "from both CLI commands and future dind daemon jobs"
• Developer/issues/validator-operations.md — daemon-mode logging/health expectations
• Documentation/ — DIN workflow and Model workflow docs for the command surfaces being extracted

Branching note

Kept separate from the DIN daemon work (see the "start DIN daemon" issue) so the CLI-on-SDK refactor can merge into develop without waiting for the daemon. dind builds on top of this SDK's JSON interface.

[Santiagocetran]

Started on this. Set up feat/din-sdk and synced it with develop — the doc-reorg (0e45baa) fast-forwarded cleanly.

Inventory (first checkbox) — done locally. Mapped the current dincli/cli/*.py command modules against the logic-vs-presentation boundary. Key findings:

• The codebase is already partly SDK-shaped: dincli/services/ (cid_utils, ipfs, runtime) and dincli/cli/contract_utils.py are effectively shared logic today. DinContext is also a de-facto service layer, although it still mixes dependency access with CLI rendering/exits.
• The main structural work is separating command effects from command presentation: many command paths currently console.print(...) and raise typer.Exit(1) / sys.exit(1) inside logic. For the SDK contract, operations should return JSON-serializable structured outputs and raise domain exceptions; CLI wrappers should handle rendering and process exits.
• Interactive _confirm_or_exit / typer.confirm gates are embedded mid-flow in several pipelines (task.register_request, client.train_lms, model.*, gi.start, plus some system commands). Since dind has no console, these need to become preflight checks, explicit options, or injectable confirmation policies.

Proposed extraction order:

1. Clean seed modules: cid, runtime, ipfs, contracts into dincli/sdk/, establishing package layout and the return/raise pattern.
2. Loaders/context helpers: config, wallet, manifest, state, web3, worker.
3. Shared keystones: build_and_send_tx and the validate_* predicates, with tests kept green because these are touched everywhere.
4. Role operations into sdk/operations/, with thin CLI wrappers rendering the structured results.

Question for @umeradl: P3-DOC7 defines the SDK public interface + backward-compat guarantees and is currently slated around Aug–Sep. Do you want me to propose an initial SDK interface and JSON schema conventions now for review against DOC7, or should I keep the first pass limited to mechanical extraction and wait on DOC7 before treating the public interface as stable?

[umeradl]

Great progress, and thanks for the detailed inventory — the logic-vs-presentation boundary you've mapped (and the note that services/, contract_utils, and DinContext are already de-facto SDK layers) is exactly the right read.

On how much to take on: the current ROADMAP timeline isn't optimized for a team. It was drafted assuming a single developer working the phases sequentially, so slots like the Aug–Sep window for P3-DOC7 reflect that pacing constraint, not a hard dependency. Now that we have a team, the goal is to accelerate rather than pace to the original single-developer schedule.

So please keep pushing on this issue as far as you can carry it. Don't hold the first pass to mechanical extraction only — go ahead and propose an initial SDK public interface and JSON schema conventions now. We'll review them here against DOC7 rather than blocking on DOC7 landing first; in practice your proposal is likely to become the seed for DOC7 itself.

The one thing that takes priority over this: if a weekly task unrelated to this issue gets assigned to you, work that first and then come back here. Absent that, treat this issue as your main line and keep going.

[Santiagocetran]

Posted the SDK interface + JSON output conventions proposal (v0.3) as the candidate P3-DOC7 seed:

📄 Developer/design/sdk-interface-proposal.md (on feat/din-sdk)

Synced feat/din-sdk with develop @ 8c4735d first (picked up the PR #16 wallet follow-ups; unit baseline is green — 70 passed / 1 skipped), and verified every code citation in the proposal against that commit.

Decisions taken (the open questions from my previous comment are now resolved in §9):

• Output contract: SDK functions return typed dataclasses; the JSON envelope {status, data, error, meta} is produced at the consumer boundary. wei/uint256 → decimal strings in JSON, typed int in Python. meta carries schema_version + chain_id + optional correlation_id.
• Errors: DinError hierarchy with stable machine codes; reserved tx/rpc subcodes (tx_reverted, tx_timeout, tx_nonce_conflict, rpc_unreachable, …) for daemon retry; allowlist-based details sanitization so no secrets land in JSON logs.
• No-console / no-prompt by contract: DinSession + a SignerProvider protocol; the SDK never calls getpass. Prompting stays in the CLI adapter; the daemon supplies a non-interactive signer or gets SignerUnavailable.
• Confirmations: plan/apply is the standard for irreversible ops; three-phase prepare → upload → apply where a step produces a CID (e.g. register_request, which uploads the manifest mid-flow today, so a pure plan can't yield the CID). assume_yes is a CLI-compat adapter only.
• Backward-compat: command names/options/output/exit-codes unchanged; enforced by golden CLI tests + an import-boundary test (import dincli.sdk pulls in no typer/rich/dincli.cli.*).

It went through two rounds of code-audit against real call sites — the richer TxReceiptInfo (deploy reads contractAddress; registry uses process_receipt) and the plan-purity vs IPFS-upload interaction were the two substantive fixes.

Ready for your review, @umeradl. Anything you want changed before I freeze this as the contract and start the clean-seed extraction (cid/runtime/ipfs/contracts → dincli/sdk/)?

[Santiagocetran]

Opened a draft PR tracking this work: #31 (fork → feat/din-sdk base, kept as draft). It carries the interface proposal + the dincli/sdk foundation (cid/errors + config/log/web3 + ipfs/contracts) with the import-boundary test green. Explicitly partial — CLI-renders-JSON and the keystones are still ahead, so it doesn't close this issue.

[Santiagocetran]

SDK loader tier complete — pushed to the draft PR #31.

dincli/sdk/ now holds all the mechanically-extractable layers: cid, errors, config, log, web3, ipfs, contracts, and (latest) manifest + runtime. Old import paths keep working via re-export shims, so existing dincli commands are unchanged.

• 117 tests green (1 skipped); import-boundary test enforces that import dincli.sdk pulls in no typer/rich/dincli.cli.* (CLI depends on SDK, never the reverse).
• Presentation was stripped as it moved (return data + raise DinError; console/typer.Exit stay in thin CLI wrappers). Manifest was the one with real coupling — split into a pure download_manifest/get_model_info with the CLI wrapper preserving the exact info output.

What's left on #20 is the keystones — DinSession, tx.send, SignerProvider, and the validate_* predicates. Those encode the public interface, so I've deliberately held them for your review of the interface + JSON conventions proposal (v0.3, in PR #31 / Developer/design/sdk-interface-proposal.md). Once you've weighed in and we freeze the contract, I'll build them straight away.

(Daemon side, for context: the P4-1.1 scaffold is up as draft PR #32; its role-automation layer is gated on the same SDK contract.)