Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ All notable changes to Tabellio are recorded here.

### Added

- Provider-neutral exact-commit validation runner with committed argv manifests and no shell execution.
- Bounded SHA-256 output evidence, detached worktree cleanup, and durable results on `refs/tabellio/validations`.
- Local validation results integrated into durable review readiness.
- Git-native JSON ledger on compare-and-swap refs for durable control-plane state.
- Durable review cycles covering Forgejo feedback, provider-neutral agent findings, triage, checks, fixes, and readiness.
- Entire-checkpoint-bound fix records with commit remapping across git-spice restacks.
Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@ GitHub remains an optional storage, CI, and review surface. Entire is the defaul
| `scripts/providers/forgejo-provider.mjs` | Read-only Forgejo repository and review adapter |
| `scripts/lib/git-json-ledger.mjs` | Versioned, compare-and-swap JSON state on standard Git refs |
| `scripts/lib/review-cycle.mjs` | Durable forge and agent review/fix state machine |
| `scripts/lib/validation-runner.mjs` | Exact-commit, shell-free validation with bounded evidence logs |
| `infra/forgejo/` | Disposable localhost Forgejo integration lab |
| `scripts/lib/` | Git process, repository contract, worktree, and context primitives |
| `scripts/` | Dependency-free capture, writer, and validators |
Expand Down Expand Up @@ -177,6 +178,7 @@ npm run tabellio:stack -- --repo . --repo-id IntelIP/Tabellio --out tabellio-sta
npm run tabellio:stack:check
npm run tabellio:stack:operation:example:check
npm run tabellio:review:example:check
npm run tabellio:validate:example:check
npm run tabellio:ledger -- --repo . --repo-id IntelIP/Tabellio --base main --head HEAD --out tabellio-ledger.json
npm run tabellio:ledger:check
npm run tabellio:forge -- version --base-url http://127.0.0.1:3300 --token-file .tabellio/forgejo/credentials/admin-token
Expand Down Expand Up @@ -209,6 +211,7 @@ The external-action checker fails when an action is marked `attempted: true` wit
- [Agent run lifecycle](docs/agent-run-lifecycle.md)
- [Approved stack operations](docs/stack-operations.md)
- [Durable review and fix loop](docs/review-loop.md)
- [Provider-neutral validation](docs/validation-runner.md)
- [Workflow model](docs/workflow-model.md)
- [Native Git foundation](docs/native-git-foundation.md)
- [Evidence schema](docs/evidence-schema.md)
Expand Down
4 changes: 2 additions & 2 deletions docs/review-loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Review status is deterministic:
| `changes_requested` | Actionable feedback remains open |
| `update_required` | Fix exists locally but is not in the remote PR head |
| `blocked` | A remote check failed or the forge reports the change as non-mergeable |
| `validating` | Checks are pending or running |
| `validating` | No validation result exists yet, or checks are pending/running |
| `ready` | Feedback is handled, fixes are published, and checks are clear |
| `merged` / `closed` | Forge terminal state |

Expand All @@ -41,7 +41,7 @@ node scripts/tabellio-review.mjs sync \
--actor review-sync-agent
```

Sync imports reviews, inline review comments, issue comments, and commit checks. Missing provider items are retained as stale evidence rather than silently deleted.
Sync imports reviews, inline review comments, issue comments, forge commit checks, and the newest provider-neutral Tabellio validation for the PR head. Missing provider items are retained as stale evidence rather than silently deleted. A PR with no validation remains `validating`.

## Import A Codex Review

Expand Down
1 change: 1 addition & 0 deletions docs/tooling-stack.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ Included:
- read-only Forgejo provider for repositories, pull requests, reviews, comments, and commit statuses
- approval-gated git-spice submit, update, sync, restack, and merge operations with one-use receipts
- Git-native review ledger with Forgejo feedback, provider-neutral agent findings, triage, fixes, and readiness state
- provider-neutral exact-commit validation runner with durable results on `refs/tabellio/validations`

Not included yet:

Expand Down
64 changes: 64 additions & 0 deletions docs/validation-runner.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Provider-Neutral Validation

Tabellio validation replaces the GitHub Actions runtime dependency with a small local contract. Any trusted worker that has Git, Node.js, the repository, and its declared tools can run the same manifest. Results live in standard Git objects on `refs/tabellio/validations`.

## Manifest

`tabellio.validation.json` is committed with the code being tested:

```json
{
"schemaVersion": "tabellio-validation/v0.1",
"id": "tabellio-default",
"failFast": true,
"requireEntireCheckpoint": true,
"commands": [{
"id": "repository-check",
"argv": ["npm", "run", "check"],
"cwd": ".",
"timeoutMs": 1200000,
"required": true
}]
}
```

Commands are executable-plus-argument arrays. Tabellio never invokes a shell, interpolates command strings, or reads a newer working-tree manifest while validating an older commit.

## Run

```bash
node scripts/tabellio-validate.mjs run \
--repo . \
--repo-id example/project \
--base main \
--commit HEAD \
--manifest tabellio.validation.json \
--runner-id worker-01
```

The runner:

1. Freezes base and head object IDs and computes the merge base.
2. Reads the manifest from the exact head commit.
3. Requires at least one Entire checkpoint in the change range when configured.
4. Creates a detached temporary worktree at the exact head commit.
5. Runs argv commands with an isolated `HOME`/`TMPDIR`, minimal environment, and per-command timeout.
6. Hashes all output while retaining only the newest 16 KiB of each stream.
7. Removes the worktree even after failure.
8. Writes an integrity-protected result to `refs/tabellio/validations` with compare-and-swap retries.

Read the newest result for a commit:

```bash
node scripts/tabellio-validate.mjs latest --repo . --commit HEAD
```

## CI Worker Shape

The runner is deliberately scheduler-agnostic. A Forgejo webhook worker, systemd service, AWS Batch task, Kubernetes Job, Buildkite agent, Woodpecker pipeline, or local Codex worker can invoke the same CLI. The scheduler supplies capacity; Tabellio supplies the immutable validation contract and result ledger.

Publishing `refs/tabellio/validations` to a remote is a separate approved Git write. Review sync consumes the latest result for the exact PR head and will not reuse a result from an older commit.

## Security Boundary

Detached worktrees and isolated home directories protect host source and common credential paths; they are not a hostile-code sandbox. Validation commands can execute arbitrary repository code and use the network. Run untrusted contributions in a disposable VM, container, or sandbox with scoped credentials and network policy. Never place secrets in manifest arguments or print them: argv and bounded output tails are retained as durable evidence.
49 changes: 49 additions & 0 deletions examples/tabellio-validation/minimal-result.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
{
"schemaVersion": "tabellio-validation-result/v0.1",
"runId": "validation-example-001",
"repository": { "id": "example/tabellio" },
"revision": {
"baseCommit": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"mergeBase": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"headCommit": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
},
"suite": {
"id": "tabellio-default",
"manifestPath": "tabellio.validation.json",
"manifestDigest": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"
},
"runner": { "id": "example-runner", "runtime": "node-v20.0.0" },
"status": "passed",
"checkpoints": ["checkpoint-example-001"],
"commands": [{
"id": "repository-check",
"argv": ["npm", "run", "check"],
"cwd": ".",
"required": true,
"status": "passed",
"exitCode": 0,
"signal": null,
"durationMs": 1000,
"stdout": {
"bytes": 14,
"digest": "0c7f9ceb9b131d26399e998396e479d5d9e46d67c2819faef8d2ea184e63424b",
"tail": "checks passed\n",
"truncated": false
},
"stderr": {
"bytes": 0,
"digest": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"tail": "",
"truncated": false
},
"startedAt": "2026-07-10T12:00:00.000Z",
"completedAt": "2026-07-10T12:00:01.000Z",
"error": null
}],
"startedAt": "2026-07-10T12:00:00.000Z",
"completedAt": "2026-07-10T12:00:01.000Z",
"integrity": {
"algorithm": "sha256",
"digest": "e0bf535d574ca275855e3fee7cccc4512785eaa9f4041975459d898145a4338f"
}
}
5 changes: 4 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
"tabellio-stack": "scripts/tabellio-stack.mjs",
"tabellio-stack-operation": "scripts/tabellio-stack-operation.mjs",
"tabellio-review": "scripts/tabellio-review.mjs",
"tabellio-validate": "scripts/tabellio-validate.mjs",
"tabellio-ledger": "scripts/tabellio-ledger.mjs",
"tabellio-forge": "scripts/tabellio-forge.mjs"
},
Expand Down Expand Up @@ -51,6 +52,8 @@
"tabellio:stack:operation:example:check": "node scripts/check-tabellio-stack-operation.mjs --intent examples/tabellio-stack-operation/minimal-intent.json --approval examples/tabellio-stack-operation/minimal-approval.json --at 2026-07-10T12:02:00.000Z",
"tabellio:review": "node scripts/tabellio-review.mjs",
"tabellio:review:example:check": "node scripts/check-tabellio-review-cycle.mjs --cycle examples/tabellio-review/minimal-cycle.json && node scripts/check-tabellio-agent-review.mjs --review examples/tabellio-review/minimal-agent-review.json",
"tabellio:validate": "node scripts/tabellio-validate.mjs",
"tabellio:validate:example:check": "node scripts/check-tabellio-validation.mjs --manifest tabellio.validation.json && node scripts/check-tabellio-validation.mjs --result examples/tabellio-validation/minimal-result.json",
"tabellio:ledger": "node scripts/tabellio-ledger.mjs",
"tabellio:ledger:check": "node scripts/check-tabellio-ledger.mjs --ledger tabellio-ledger.json",
"tabellio:ledger:example:check": "node scripts/check-tabellio-ledger.mjs --ledger examples/tabellio-ledger/minimal-ledger.json",
Expand All @@ -62,7 +65,7 @@
"tabellio:evidence:write": "node scripts/write-tabellio-evidence-envelope.mjs --out tabellio-pr-evidence.json",
"tabellio:evidence:check": "node scripts/check-tabellio-evidence-envelope.mjs --evidence tabellio-pr-evidence.json",
"tabellio:external-actions:check": "node scripts/check-tabellio-external-actions.mjs --evidence tabellio-pr-evidence.json",
"check": "npm test && npm run tabellio:run:example:check && npm run tabellio:stack:example:check && npm run tabellio:stack:operation:example:check && npm run tabellio:review:example:check && npm run tabellio:ledger:example:check && npm run tabellio:context:example:check && npm run tabellio:evidence:example:check && node scripts/check-tabellio-external-actions.mjs --evidence examples/tabellio-evidence/minimal-evidence.json"
"check": "npm test && npm run tabellio:run:example:check && npm run tabellio:stack:example:check && npm run tabellio:stack:operation:example:check && npm run tabellio:review:example:check && npm run tabellio:validate:example:check && npm run tabellio:ledger:example:check && npm run tabellio:context:example:check && npm run tabellio:evidence:example:check && node scripts/check-tabellio-external-actions.mjs --evidence examples/tabellio-evidence/minimal-evidence.json"
},
"engines": {
"node": ">=20"
Expand Down
31 changes: 31 additions & 0 deletions schemas/validation-manifest.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "urn:tabellio:schema:validation-manifest:v0.1",
"title": "Tabellio Validation Manifest",
"type": "object",
"required": ["schemaVersion", "id", "failFast", "requireEntireCheckpoint", "commands"],
"additionalProperties": false,
"properties": {
"schemaVersion": { "const": "tabellio-validation/v0.1" },
"id": { "type": "string", "minLength": 1 },
"failFast": { "type": "boolean" },
"requireEntireCheckpoint": { "type": "boolean" },
"commands": {
"type": "array",
"minItems": 1,
"maxItems": 50,
"items": {
"type": "object",
"required": ["id", "argv", "cwd", "timeoutMs", "required"],
"additionalProperties": false,
"properties": {
"id": { "type": "string", "minLength": 1 },
"argv": { "type": "array", "minItems": 1, "maxItems": 100, "items": { "type": "string", "minLength": 1 } },
"cwd": { "type": "string", "minLength": 1 },
"timeoutMs": { "type": "integer", "minimum": 100, "maximum": 3600000 },
"required": { "type": "boolean" }
}
}
}
}
}
96 changes: 96 additions & 0 deletions schemas/validation-result.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "urn:tabellio:schema:validation-result:v0.1",
"title": "Tabellio Validation Result",
"type": "object",
"required": ["schemaVersion", "runId", "repository", "revision", "suite", "runner", "status", "checkpoints", "commands", "startedAt", "completedAt", "integrity"],
"additionalProperties": false,
"properties": {
"schemaVersion": { "const": "tabellio-validation-result/v0.1" },
"runId": { "type": "string", "minLength": 1 },
"repository": {
"type": "object",
"required": ["id"],
"additionalProperties": false,
"properties": { "id": { "type": "string", "minLength": 1 } }
},
"revision": {
"type": "object",
"required": ["baseCommit", "mergeBase", "headCommit"],
"additionalProperties": false,
"properties": {
"baseCommit": { "$ref": "#/$defs/oid" },
"mergeBase": { "$ref": "#/$defs/oid" },
"headCommit": { "$ref": "#/$defs/oid" }
}
},
"suite": {
"type": "object",
"required": ["id", "manifestPath", "manifestDigest"],
"additionalProperties": false,
"properties": {
"id": { "type": "string", "minLength": 1 },
"manifestPath": { "type": "string", "minLength": 1 },
"manifestDigest": { "$ref": "#/$defs/sha256" }
}
},
"runner": {
"type": "object",
"required": ["id", "runtime"],
"additionalProperties": false,
"properties": {
"id": { "type": "string", "minLength": 1 },
"runtime": { "type": "string", "minLength": 1 }
}
},
"status": { "enum": ["passed", "failed"] },
"checkpoints": { "type": "array", "uniqueItems": true, "items": { "type": "string", "minLength": 1 } },
"commands": { "type": "array", "minItems": 1, "items": { "$ref": "#/$defs/commandResult" } },
"startedAt": { "type": "string", "format": "date-time" },
"completedAt": { "type": "string", "format": "date-time" },
"integrity": {
"type": "object",
"required": ["algorithm", "digest"],
"additionalProperties": false,
"properties": {
"algorithm": { "const": "sha256" },
"digest": { "$ref": "#/$defs/sha256" }
}
}
},
"$defs": {
"oid": { "type": "string", "pattern": "^(?:[0-9a-f]{40}|[0-9a-f]{64})$" },
"sha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" },
"output": {
"type": "object",
"required": ["bytes", "digest", "tail", "truncated"],
"additionalProperties": false,
"properties": {
"bytes": { "type": "integer", "minimum": 0 },
"digest": { "$ref": "#/$defs/sha256" },
"tail": { "type": "string" },
"truncated": { "type": "boolean" }
}
},
"commandResult": {
"type": "object",
"required": ["id", "argv", "cwd", "required", "status", "exitCode", "signal", "durationMs", "stdout", "stderr", "startedAt", "completedAt", "error"],
"additionalProperties": false,
"properties": {
"id": { "type": "string", "minLength": 1 },
"argv": { "type": "array", "minItems": 1, "items": { "type": "string", "minLength": 1 } },
"cwd": { "type": "string", "minLength": 1 },
"required": { "type": "boolean" },
"status": { "enum": ["passed", "failed", "error", "timed_out", "skipped"] },
"exitCode": { "type": ["integer", "null"] },
"signal": { "type": ["string", "null"] },
"durationMs": { "type": "integer", "minimum": 0 },
"stdout": { "$ref": "#/$defs/output" },
"stderr": { "$ref": "#/$defs/output" },
"startedAt": { "oneOf": [{ "type": "string", "format": "date-time" }, { "type": "null" }] },
"completedAt": { "oneOf": [{ "type": "string", "format": "date-time" }, { "type": "null" }] },
"error": { "type": ["string", "null"] }
}
}
}
}
36 changes: 36 additions & 0 deletions scripts/check-tabellio-validation.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
#!/usr/bin/env node

import { readFile } from "node:fs/promises";
import { resolve } from "node:path";

import { validateValidationManifest, validateValidationResult } from "./lib/validation-runner.mjs";

const type = process.argv[2];
const path = process.argv[3];
const blockers = [];
let value = null;
try {
if (!path || process.argv.length !== 4 || !["--manifest", "--result"].includes(type)) {
throw new Error("Usage: check-tabellio-validation (--manifest|--result) <path>.");
}
value = JSON.parse(await readFile(resolve(path), "utf8"));
if (type === "--manifest") validateValidationManifest(value);
else validateValidationResult(value);
} catch (error) {
blockers.push(error instanceof Error ? error.message : String(error));
}
const result = {
ok: blockers.length === 0,
status: blockers.length === 0 ? "validation_contract_ready" : "blocked",
checkedAt: new Date().toISOString(),
type: type === "--manifest" ? "manifest" : type === "--result" ? "result" : null,
path,
summary: value ? {
schemaVersion: value.schemaVersion,
id: value.id ?? value.runId ?? null,
commandCount: Array.isArray(value.commands) ? value.commands.length : null,
} : null,
blockers,
};
if (!result.ok) process.exitCode = 1;
console.log(JSON.stringify(result, null, 2));
Loading
Loading