Adopt Diataxis + Open Knowledge Format as the docs architecture (small human+agent pages, self-updating remote brain)

[MariusStorhaug]

Context

MSXOrg/docs is meant to be the single home for everything MSX does on GitHub — product docs, coding standards, ways of working, and the instructions our agents run on. Today it is effectively empty (just a README.md), and the in-flight standards issues (#2–#8) already assume a src/docs/Coding-Standards/… tree without an agreed overall architecture.

Two things make this more than a normal docs repo:

1. The audience is both humans and agents. Pages must be small and self-contained so an agent can pull one concept into context without dragging in the whole tree, while a human reads the same file unchanged.
2. We want a thin agent config. Instead of fat AGENTS.md / CLAUDE.md / *.instructions.md files, the agent config should be a thin pointer that relates to the docs. Ways of working and instructions live in the docs and are referenced, not duplicated into every agent.

On top of that we want the docs to be able to self-update (agents curate and correct their own files) and to act as a "remote brain": a fairly low-friction git repo where both humans and agents dump what they learn, and an agent keeps it tidy — replacing the local, per-machine *.md memory files we use today.

This issue proposes an architecture and asks us to commit to it (via an ADR) before we pour content in.

Goals

• One coherent, intentional body of docs (not a pile of pages).
• A clear, predictable place for every page — discoverable the same way it was filed.
• Pages small enough to be context-cheap for agents and skimmable for humans.
• A file format that is human- and agent-native, version-controlled, and lock-in free.
• Thin agent config that references docs instead of embedding them.
• Docs that can be safely self-updated by agents and dumped into by anyone.
• A remote brain that supersedes local *.md memory files.

Proposed approach

1. Architecture — Diátaxis + Procida's principles

Adopt Diátaxis as the information architecture: every page is one of four types, chosen by user need.

Type	Need	Example content here
Tutorial	learning	"Set up your first MSX repo"
How-to guide	a task	ways of working, contribution flow, agent runbooks
Reference	facts	coding standards (#2–#8), APIs, config surfaces
Explanation	understanding	ADRs, rationale, design notes

Layer Procida's twelve principles on top as the quality bar — most importantly: single source of truth (no duplication), every page has one correct place, docs have a life-cycle (create → review → maintain → expurgate), and links are first-class.

2. Format — Open Knowledge Format (OKF)

Store everything as an OKF bundle: plain markdown + YAML frontmatter, one concept per file, file path = identity. OKF gives us for free the exact properties we need:

• index.md at every level → progressive disclosure. Agents (and humans) walk the tree one level at a time instead of loading everything — this is the mechanism for "small pages, no context bloat."
• log.md → chronological change history per area. This is the substrate for self-learning and the remote brain.
• Markdown cross-links → a knowledge graph, not just a tree — matches Procida's "connections are robustness."
• Producer/consumer independence — a human hand-authors, an agent enriches, both read the same file with no translation layer.
• Just files in git — PRs, diffs, blame, and review already work; the repo is the API.

3. Diátaxis × OKF synthesis

They compose cleanly: Diátaxis decides what kind of page it is; OKF decides how it is stored. Carry the Diátaxis type in frontmatter so it is queryable:

---
type: reference            # OKF required field
diataxis: reference        # tutorial | how-to | reference | explanation
title: PowerShell output streams
description: How MSX PowerShell emits objects vs. text.
audience: both             # human | agent | both
lifecycle: stable          # draft | reviewed | stable | deprecated
tags: [powershell, coding-standard]
timestamp: 2026-07-03T00:00:00Z
---

Keep frontmatter minimal (OKF principle: only add fields you will actually query/filter on). Everything else stays in the markdown body.

4. Thin agent config that relates to the docs

The agent config shrinks to a pointer: "read src/docs/index.md, then descend into the relevant index.md for the task." Instructions and ways of working become how-to pages in the bundle, referenced by path — never copied into AGENTS.md / CLAUDE.md. A new instruction = a new/edited doc page, not a config change.

5. Self-updating docs

Because the bundle is just markdown in git, an agent can maintain it the way it maintains code: open a PR, touch many files in one pass, and keep cross-references consistent. (Karpathy's LLM-wiki observation, quoted in the OKF launch: "LLMs don't get bored, don't forget to update a cross-reference, and can touch 15 files in one pass.") Guardrail: agents propose via PR into curated areas; only the low-friction brain area is directly writable (below).

6. The "remote brain" — replacing local memory files

Add a deliberately low-friction, low-ceremony area (e.g. brain/) where humans and agents dump raw notes, and an agent periodically curates them into proper Diátaxis/OKF pages (create → review → expurgate). Each area keeps a log.md. This replaces the per-machine local *.md memory files with one shared, versioned, reviewable brain that both humans and agents read and write.

Does OKF fit? (the "find if this could work" answer)

Yes — strong fit, with one caveat. OKF was framed around data-catalog "concepts" (tables, metrics), but it is explicitly minimally opinionated and format-not-platform: the only hard requirement is a type field. Everything we need — progressive-disclosure index.md, historical log.md, a cross-linked graph, human+agent readability, git-native curation, and extensible frontmatter — is already in the spec. The caveat: OKF says nothing about pedagogy / information architecture, which is exactly the gap Diátaxis fills. So we use OKF as the physical format and Diátaxis as the information architecture — they do not overlap, they complete each other.

Proposed layout (topic-based, Diátaxis-typed)

Recommend organizing by topic (keeps the #2–#8 coding-standards/powershell/ tree intact) and tagging each page's Diátaxis type in frontmatter — per Diátaxis's own "complex hierarchies" guidance — rather than forcing rigid top-level tutorials/ how-to/ reference/ explanation/ folders.

src/docs/
  index.md                     # OKF root — entry point for humans + agents
  coding-standards/
    index.md
    powershell/
      index.md                 # issues #2–#8 land here
      output-streams.md
      error-handling.md
      ...
  ways-of-working/
    index.md                   # how-to guides, contribution flow, agent runbooks
  reference/
    index.md
  explanation/
    index.md                   # ADRs / rationale (incl. the ADR from this issue)
  brain/
    index.md
    log.md                     # the remote-brain dump + curation area

(Alternative: pure Diátaxis quadrant folders at the top level — captured as an open question below.)

Open questions

1. Topic-based + frontmatter type (recommended) vs. pure Diátaxis quadrant folders at the top?
2. Exact writable boundary for agents: PR-only everywhere, or direct-write in brain/ only?
3. Minimal frontmatter schema — which fields do we actually query on? (type, diataxis, lifecycle, audience, tags?)
4. Do we add a CI link-checker / frontmatter lint (and optionally OKF's viz.html graph view) to enforce "connections are robustness"?
5. How does this repo relate to the existing Confluence / ai-platform docs pipeline — mirror, or is this the source of truth?

References

• Twelve principles of documentation — Daniele Procida — quality/management principles (single source of truth, life-cycle, links as robustness): https://vurt.eu/articles/principles-documentation/
• Diátaxis — the four-type information architecture (tutorial / how-to / reference / explanation): https://diataxis.fr/
• Introducing the Open Knowledge Format (Google Cloud blog) — the "living wiki for humans + agents" rationale, index.md / log.md, progressive disclosure: https://cloud.google.com/blog/products/data-analytics/how-the-open-knowledge-format-can-improve-data-sharing
• OKF README + v0.1 spec — the concrete format (frontmatter, reserved filenames, cross-links, conformance): https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/README.md

Acceptance

• An ADR under explanation/ records the decision: Diátaxis architecture + OKF format + remote-brain model, including the frontmatter schema and the agent-writable boundary.
• A short structure / CONTRIBUTING guide (itself a how-to page) explains where a new page goes and how to file and curate a brain note.
• The root src/docs/index.md and per-area index.md skeleton exist so progressive disclosure works from day one.
• Existing issues Add tools vs. controllers and raw-data output to the PowerShell standard #2–Add PowerShell code-sample conventions to the Markdown standard #8 are reconciled to the agreed layout (paths + frontmatter).
• Follow-up issues are cut for: the thin agent-config pointer, the brain-curation workflow, and any CI (link-check / frontmatter lint).

[MariusStorhaug]

Relationship to Cleo (msx-cleo)

A lot of this architecture already exists as a working implementation: Cleo, an agent whose entire self is git-backed markdown. Her public machinery repo documents the six-repo model — see msx-cleo/app › "How this fits — the Cleo ecosystem". Mapping this issue onto it:

This issue	Cleo (already built)
remote brain	memory — tiered summary.md → working/ → archive/ → essences/, curated with a reason per commit, "markdown + ripgrep over embeddings"
progressive disclosure (index.md)	those same load-on-relevance tiers
log.md / self-learning	journal + memory/archive + infra/CHANGELOG.md
thin agent config that relates to docs	identity/PROMPT.md derived from FOUNDING.md
thinking-as-issues	the reflections repo (and this issue)

So the direction is: MSXOrg/docs should not invent a parallel remote-brain — it should generalize Cleo's memory model as the pattern, while Cleo adopts OKF as the shared format so the two interoperate (same search / graph / link-check tooling, no drift). The one real gap is format: Cleo's memory is markdown without formal OKF frontmatter / index.md.

A useful framing that came out of this: three brains, one format — shared craft knowledge here (Diátaxis-shaped, public), private self knowledge in Cleo's memory (episodic, tiered, not Diátaxis), with Cleo consuming this repo as reference. Skills are the bridge (definition = an OKF/Diátaxis concept; runtime = msx-cleo/app).

Full synthesis and open questions tracked privately in msx-cleo/reflections#1.