Skip to content

Add marimo MNIST + W&B Registry example - #621

Open
johndmulhausen wants to merge 22 commits into
mainfrom
jmulhausen/marimo-mnist-registry-example
Open

Add marimo MNIST + W&B Registry example#621
johndmulhausen wants to merge 22 commits into
mainfrom
jmulhausen/marimo-mnist-registry-example

Conversation

@johndmulhausen

@johndmulhausen johndmulhausen commented May 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds the first marimo notebook to this repo at examples/marimo/mnist-registry/. It trains a small PyTorch CNN on MNIST, produces a rich Weights & Biases run, saves the model as an Artifact, links it into a W&B Registry collection, and then consumes the registered model to classify held-out digits. Two small GitHub Actions workflows support the example.

Dependencies are declared in a PEP 723 inline block, so uvx marimo edit mnist_registry.py --sandbox self-installs them. No new top-level deps for the repo.

What the notebook demonstrates

  • Form-driven hyperparameters. A marimo mo.ui.form collects hyperparameters and W&B targets; training runs only when you submit it with Train model (editing a field never kicks off a run).
  • A single "Training" W&B section: loss and accuracy charts, a confusion matrix, a per-digit PR curve, a predictions table (with images), and penultimate-layer embeddings for W&B's 2D projector (one cluster per digit). Final/best accuracy and parameter count are set on the run summary.
  • Model Artifact + cross-org Registry link. run.link_artifact(target_path="wandb-registry-<registry>/<collection>"), with logged.wait() before linking and a try/except that surfaces inline remediation (e.g. view-only-seat guidance) instead of crashing — the run and artifact still succeed.
  • Consume the model. The Evaluation cell downloads the model (registered version first, run artifact as fallback) into a fresh network and classifies ten test digits.
  • Team entity. The run must target a team entity (personal entities were removed for accounts created after May 2024); the notebook surfaces a friendly error if it isn't set.
  • Defensive run lifecycle. A guard finishes any prior run before wandb.init, so re-submitting the form never nests runs.

The notebook is organized into helper functions via marimo's @app.function for readability.

Metrics use W&B's default step x-axis (per-batch Training/loss, per-epoch Training/accuracy); there is no wandb.define_metric/epoch-axis setup.

Supporting CI

  • .github/workflows/marimo_export_md.yml — on push, exports each marimo notebook to a peer Markdown file and commits it, so the notebook renders as a reviewable diff on GitHub.
  • .github/workflows/marimo_molab.yml — posts a PR comment linking each modified marimo notebook to molab to run it in a hosted environment.

Files

  • examples/marimo/mnist-registry/mnist_registry.py — the notebook
  • examples/marimo/mnist-registry/mnist_registry.md — its exported Markdown peer (kept in sync by CI)
  • examples/marimo/mnist-registry/README.md — prereqs, how to run, design notes
  • examples/marimo/mnist-registry/requirements.txt — pip mirror of the PEP 723 block
  • .github/workflows/marimo_export_md.yml, .github/workflows/marimo_molab.yml

Test plan

  • uvx marimo edit examples/marimo/mnist-registry/mnist_registry.py --sandbox opens with all cells rendering; the form is interactive and does not train until Train model is submitted.
  • Submitting with defaults (3 epochs) trains in ~2 min on CPU, reaching >97% test accuracy; the run shows the Training section (loss, accuracy, confusion matrix, PR curve, predictions table, embeddings) and run-summary headline metrics.
  • The mnist-cnn-<run-id> model Artifact is logged and linked to wandb-registry-model/MNIST Classifiers; the Evaluation cell classifies 10 test digits.
  • A view-only seat / missing Registry surfaces inline remediation instead of crashing; unchecking Link artifact to Registry still logs the Artifact.

🤖 Generated with Claude Code

johndmulhausen and others added 2 commits May 13, 2026 12:16
Introduces the first marimo notebook in this repo. Trains a small PyTorch CNN
on MNIST, logs the run with wandb, saves the trained model as an Artifact,
and links it to a W&B Registry collection via the cross-org link_artifact
API.

The notebook is interactive: hyperparameters live in a marimo form and
training is gated by an explicit Train button so slider tweaks do not
trigger runs. PEP 723 inline script metadata makes it self-installing
under `uvx marimo edit --sandbox`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Active voice, present tense, plain English, and consistent brand
capitalization across the README and the notebook's prose cells.
Code identifiers wrapped in backticks where missing; a sentence
fragment in the verify step rewritten as a complete clause.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@socket-security

socket-security Bot commented May 13, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedpypi/​marimo@​0.23.1572100100100100

View full report

@socket-security

socket-security Bot commented May 13, 2026

Copy link
Copy Markdown

All alerts resolved. Learn more about Socket for GitHub.

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

View full report

Mirrors the existing "Get Colabs" PR-comment flow (which serves .ipynb
files via nb_helpers) for marimo .py notebooks, which nb_helpers does not
handle. On each PR that touches a .py file, the job detects marimo notebooks
by their `marimo.App(` marker and upserts a single comment linking each to
molab — marimo's hosted runtime that loads any public notebook on GitHub.

Uses pull_request_target with pull-requests:write so the comment also lands
on fork PRs, and never checks out or runs PR code (reads file content as
text only). Links pin to the head commit SHA because branch names with
slashes make the molab blob/<ref>/<path> split ambiguous.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@johndmulhausen

johndmulhausen commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

▶️ Run the marimo notebook(s) in this PR

molab launches any public marimo notebook on GitHub in a hosted environment — no local setup required.

Notebook molab
examples/marimo/mnist-registry/mnist_registry.py Open in molab

Links track the head of jmulhausen/marimo-mnist-registry-example.

johndmulhausen and others added 2 commits June 12, 2026 13:08
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a masked mo.ui.text(kind="password") field so users can authenticate
by pasting a key instead of running `wandb login` in a shell. The resolved
value is kept out of the run config (never logged) and feeds
wandb.login(key=...) in the gated training cell; blank falls back to ambient
auth (shell login, WANDB_API_KEY, or netrc). Downstream cells read the key
via the returned wandb_api_key.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@johndmulhausen

johndmulhausen commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Triage: Socket license alerts on marimo@0.23.9

Both alerts (sdist + wheel) are license-policy warnings (licenseSpdxDisj) — not vulnerability or malware findings — and the Socket checks themselves pass, so they're non-blocking.

marimo publishes under Apache-2.0, declared via the PEP 639 license_expression field with the legacy license field left null (verified on PyPI). Apache-2.0 is a permissive, OSI-approved license. The alert is an artifact of the license being expressed in the newer metadata format rather than a real licensing risk, so accepting it for this PR is appropriate.

@SocketSecurity ignore pypi/marimo@0.23.9

johndmulhausen and others added 6 commits June 12, 2026 13:26
The link step's "common causes" did not name the most common real failure:
a view-only org seat (`view-only member cannot write to project`), where the
run and artifact succeed but linking is blocked. Lead the remediation with
that case and how to resolve it, and note the write-access requirement up
front in the prerequisites.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GitHub resolves multi-segment (slashed) branch names in blob/<ref>/<path>
and molab fetches from GitHub, so a branch-based link points at the latest
revision and needs no per-commit comment maintenance. Content detection
still pins to the head SHA. Skip the comment write when the body is
unchanged so no-op pushes don't churn it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The view-only link failure now names the required role (Member) and the
three ways an admin can grant it: the Registry Members UI, the Python SDK
(wandb.Api().registry(...) then add_member()/update_member()), and SCIM
(PATCH /scim/Users/{id} with registryRoles), with a link to the configure
registry access docs. README prerequisite updated to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The bare molab GitHub URL renders a static preview. Appending /server opens
the notebook in a hosted runtime so reviewers can actually run it — needed
here since the notebook depends on torch, which the in-browser /wasm mode
cannot run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a push-triggered workflow that runs `marimo export md` on each notebook
under examples/marimo/, writing a peer <notebook>.md, and commits the result
back to the branch so the rendered Markdown always tracks the notebook. marimo
is pinned for byte-deterministic output (no version-only churn), and three
guards prevent commit loops (GITHUB_TOKEN pushes don't re-trigger, *.py-only
trigger vs *.md-only commits, and [skip ci]).

Includes the initial export of mnist_registry.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Collapses 19 cells into 5, breaking only where marimo requires it or where
the user actually provides input:

- intro (markdown)
- setup + form: imports, device detection, and all input widgets in one cell
- train button: its own cell (a widget must be defined separately from the
  cell that reads its value)
- run training: one gated cell that configs, authenticates, inits, builds and
  trains the model, logs, saves, and links the artifact — streaming each
  milestone with mo.output.append so it isn't a black box
- verify + next steps (markdown), rendered once a run exists

run_button.value resets to False after its cascade, so editing the form after
a run re-runs the training cell but mo.stop halts it immediately — no silent
retrain. Regenerates the Markdown peer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@johndmulhausen

Copy link
Copy Markdown
Contributor Author

Socket re-scan 8a2c0e64 — same marimo@0.23.9 license alert, re-affirming

This is the already-triaged licenseSpdxDisj warning, re-posted by a fresh scan on the latest push — not a new finding. marimo is Apache-2.0 (permissive, OSI-approved; verified on PyPI); the alert is about SPDX license-expression formatting, not a vulnerability or malware; and both Socket checks pass (non-blocking). No code change applies here — marimo is the notebook's runtime dependency and requirements.txt is a mirror of its PEP 723 block.

Re-affirming PR-scoped acceptance for this scan:

@SocketSecurity ignore pypi/marimo@0.23.9

Durable fix (so this stops re-nagging on every push): allowlist Apache-2.0 for the org in the Socket dashboard — an admin/security-team action, outside what I can do from a PR.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new examples/marimo/ MNIST notebook demonstrating training a small PyTorch CNN, logging to W&B, and linking a logged model artifact into a W&B Registry collection, plus GitHub Actions automation for exporting marimo notebooks to Markdown and for posting molab “run this notebook” PR links.

Changes:

  • Add a self-contained marimo MNIST → W&B Artifact → W&B Registry notebook (+ exported Markdown peer) under examples/marimo/mnist-registry/.
  • Add docs and a requirements.txt mirror for the notebook’s PEP 723 dependency block.
  • Add GitHub Actions workflows to (a) auto-export marimo notebooks to Markdown on push and (b) comment molab links on PRs.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
examples/marimo/mnist-registry/mnist_registry.py New marimo notebook implementing training, W&B logging, artifact creation, and registry linking.
examples/marimo/mnist-registry/mnist_registry.md Exported Markdown version of the notebook for static rendering.
examples/marimo/mnist-registry/README.md Run instructions, prerequisites, and design notes for the example.
examples/marimo/mnist-registry/requirements.txt Pip requirements mirroring the PEP 723 dependency block.
.github/workflows/marimo_export_md.yml Workflow to export examples/marimo/**/*.py notebooks to peer .md files and commit updates.
.github/workflows/marimo_molab.yml Workflow to post/update PR comments with molab links for modified marimo notebooks.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/marimo_molab.yml Outdated
Comment thread examples/marimo/mnist-registry/mnist_registry.py Outdated
Comment thread examples/marimo/mnist-registry/mnist_registry.py Outdated
Comment thread examples/marimo/mnist-registry/mnist_registry.py Outdated
johndmulhausen and others added 7 commits June 12, 2026 15:32
- Remove the unused `import os` and drop `os` from the setup cell's returns.
- Compute final_acc from the full-precision last-epoch test_acc instead of the
  display-rounded history entry, so artifact `test_accuracy` keeps full precision.
- marimo_molab.yml: use `**/*.py` (canonical glob) instead of `**.py` so the
  path filter reliably matches notebooks in subdirectories.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reorders the notebook to: intro -> setup+form -> training pipeline -> Train
button -> verify. The button still gates the pipeline (so it doesn't run on
open), but now reads as the explicit "run the code above" trigger. The
training cell's pre-run message explains what it does and points to the button
below.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The entity form field read just "W&B entity", which reads as jargon. Label it
"username or team" to match the prerequisites wording. Also tighten the
registry-link remediation: an entity is a team or username (not an org), so it
now reads "set W&B entity to a team in an org where you have Registry write
access."

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Personal-username entities were removed for W&B accounts created after
2024-05-21, so a run's entity must be a team. Relabel the entity field
accordingly, fix the prerequisite, and wrap wandb.init so an
"entity ... not found" failure renders actionable guidance (set the entity to
a team) instead of a raw CommError traceback.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pulls the CNN definition into its own cell so it can be shared, then adds a
cell that downloads the model from W&B (preferring the registered version,
falling back to the run's artifact so it works even when the registry link is
blocked), loads the weights into a fresh network, and classifies 10 held-out
MNIST test digits — rendering each as an image with predicted vs. true label
and an N/10 correct tally. No new dependency (numpy ships with torch; mo.image
renders the arrays).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The consume cell reused `correct` and `pred`, which the training cell already
defines — marimo forbids defining a name in more than one cell, so the cell
refused to run. Rename them to `n_correct` / `prediction` (both cell-local to
the consume step). marimo export does not enforce the single-definition rule,
only the kernel does, so this slipped through earlier validation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
)

* Refactor the mnist notebook for readability and usability

- Move imports and constants into a setup cell
- Wrap the UI elements in a submittable form, removing the
  ui.run_button()
- Refactor the monolithic training and logging cell into multiple helper
  functions, each declared in their own cell as a reusable function
- Separate logic cells from view cells when possible
- Clean up the evaluation presentation, using a table instead of
  markdown+emoji
- Add a mo.outline() so users can see an outline of the notebook at a
  glance

  Core principles:

  * Gate execution once on mo.stop(), then let the graph handle the
    rest. All other cells reference run/config, which don't exist
    until the form is submitted. This lets us remove lots of
    confusing conditions.
  * Separate logic from presentation
  * Push temporary variables into functions to reduce notebook globals.
    marimo notebooks work best when there are as few global variables
    as possible.

* docs: export marimo notebook(s) to Markdown [skip ci]

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@w-b-hivemind

w-b-hivemind Bot commented Jul 16, 2026

Copy link
Copy Markdown

HiveMind Sessions

1 session · 45m · $41

Session Agent Duration Tokens Cost Lines
Improve W&B Dashboard Training Visualizations
8bf6cd13-9203-4c2d-a1c2-35499359d19c
claude 45m 165.8K $41 +146 -81
Total 45m 165.8K $41 +146 -81

View all sessions in HiveMind →

Run claude --resume 8bf6cd13-9203-4c2d-a1c2-35499359d19c to pickup where you left off.

johndmulhausen and others added 3 commits July 16, 2026 16:52
W&B groups panels by the prefix before "/", so train/* and test/* produced
separate sections, and the example images were logged from normalized tensors
(values outside [0,1]) so they rendered black. Log train loss and test
accuracy under a single "Training/" prefix, drop the redundant test/loss, the
bare "epoch" metric, the broken "examples" panel, and wandb.watch (which added
a separate gradients section). Result: one "Training" section with a loss
chart and an accuracy chart. Also restore the verify cell's f-string
interpolation (it had regressed to literal {run.name}) and update its panel
references.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bring back example predictions the right way: a wandb.Table of un-normalized
digit images with true/predicted/confidence, logged once (no per-step slider).
Add a confusion matrix so "accuracy" is concrete — which digits the model
nails and which it confuses. Both log under the Training prefix after training.
Update the intro and verify narration to define accuracy and describe the new
panels.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Expand log_eval_report to also log a per-digit PR curve and penultimate-layer
embeddings (Net.features) for W&B's 2D projector — one cluster per digit,
colored by label. Set run.summary with final/best accuracy and parameter count
so they surface on the run overview. Document how to add the 2D Projection
panel in the verify steps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@willgoldby
willgoldby self-requested a review July 30, 2026 20:31
@thedriftofwords

Copy link
Copy Markdown

Note: I was just having my agent extract Marimo-specific constraints and style choices from this PR, and it flagged something:

I flagged that the README has drifted from the code (still describes wandb.watch gradient histograms and a 16-image prediction gallery that were removed a few commits later). Worth pinging the PR author, since it'll confuse anyone using this as a template.

Worth running a final agent check to ensure the README caught up with all the changes made and that it's internally consistent; you can ping me for a re-review if you end up changing anything.

@ngrayluna

Copy link
Copy Markdown
Contributor

The clean notebooks check is probably not relevant here. It is supposed to clear metadata, cells outputs from a notebook. (But we're converting to a Py script.)

The README had drifted: it still described wandb.watch gradient histograms and
a 16-image prediction gallery, both removed. Update "What you get" to the
actual W&B report (Training loss/accuracy, confusion matrix, PR curve,
predictions table, 2D-projector embeddings, run-overview headline metrics),
describe the in-notebook Evaluation step, add the team-entity prerequisite,
and reword the gating note for the marimo form. An agent audit confirms
README<->notebook consistency.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 30, 2026 20:53
@johndmulhausen

Copy link
Copy Markdown
Contributor Author

@thedriftofwords good catch — fixed the README drift in 9c104aa. Dropped the stale wandb.watch gradient-histogram and 16-image-gallery claims and rewrote What you get to the actual W&B report (Training loss/accuracy, confusion matrix, PR curve, predictions table, 2D-projector embeddings, run-overview headline metrics). Also added the team-entity prerequisite, documented the in-notebook Evaluation step, and reworded the gating note for the marimo form. I ran an independent agent audit of README↔notebook afterward and it reports full consistency. Ready for a re-review whenever you like.

🤖 Addressed by Claude Code

Comment thread examples/marimo/mnist-registry/mnist_registry.py
@johndmulhausen

Copy link
Copy Markdown
Contributor Author

Agreed, @ngrayluna — that check runs nb_helpers.fix_nbs ./colabs/ to strip .ipynb metadata/outputs, and this PR adds only a marimo .py, so it doesn't apply. The failure itself is unrelated dependency rot: nb_helpers imports Param from fastcore.script, which fastcore ≥ 1.8 removed (verified: 1.7.29 still has it). No change needed on this PR — a fastcore<1.8 pin in cleanup.yml would green it repo-wide if the team wants that.

🤖 Addressed by Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (8)

examples/marimo/mnist-registry/mnist_registry.py:582

  • wandb.define_metric("epoch") (and epoch-based charts) only work if each wandb.log call includes an epoch value. Right now loss logs omit epoch, so the x-axis will remain W&B's default step counter.
        if batch_idx % 50 == 0:
            wandb.log({"Training/loss": loss.item()})

examples/marimo/mnist-registry/mnist_registry.py:683

  • Accuracy logs also need to include the current epoch value; otherwise wandb.define_metric(... step_metric="epoch") won’t affect the chart x-axis.
        wandb.log({"Training/accuracy": test_acc})

examples/marimo/mnist-registry/mnist_registry.py:289

  • The notebook claims (in the PR description) to use wandb.define_metric with epoch as the x-axis, but the run initialization never calls wandb.define_metric. Add metric definitions right after wandb.init() so subsequent logs can attach to epoch.

This issue also appears in the following locations of the same file:

  • line 581
  • line 683
        run = wandb.init(
            project=cfg["project"] or None,
            entity=cfg["entity"] or None,
            name=cfg["run_name"] or None,
            config=config,
            job_type="train",
        )

examples/marimo/mnist-registry/mnist_registry.py:4

  • The inline dependency block declares marimo>=0.9, but this notebook was generated with marimo 0.23.11 (__generated_with) and the repo workflow pins 0.23.9 for exports. The minimum marimo version should be aligned to the generated/pinned version so users don’t end up installing an older incompatible marimo release.
#     "marimo>=0.9",

examples/marimo/mnist-registry/requirements.txt:3

  • requirements.txt should stay in sync with the PEP 723 dependency block; marimo>=0.9 is much lower than the marimo version this notebook was generated/exported with. Bump the minimum marimo version to match the notebook’s __generated_with / export pin.
marimo>=0.9

examples/marimo/mnist-registry/mnist_registry.md:3

  • This exported Markdown front matter records marimo-version: 0.23.9, but the source notebook advertises __generated_with = "0.23.11". Align the export version with the notebook’s generated version (by bumping the exporter pin and re-exporting) so the repo stays byte-deterministic and version-consistent.
marimo-version: 0.23.9

.github/workflows/marimo_molab.yml:107

  • This workflow builds a Markdown table row using untrusted PR-controlled values (path, headRef). Unescaped backticks/pipes can break the table or inject unintended Markdown, and unencoded characters (e.g., #, spaces) can produce broken molab URLs. Escape Markdown metacharacters in the rendered path and URL-encode ref/path components when constructing the molab link.
            const rows = notebooks.map((path) => {
              // The `/server` suffix opens the notebook in a hosted runtime;
              // without it molab shows a static, non-runnable preview.
              const url = `https://molab.marimo.io/github/${headOwner}/${headRepo}/blob/${headRef}/${path}/server`;
              return `| \`${path}\` | [![Open in molab](https://marimo.io/molab-shield.svg)](${url}) |`;

.github/workflows/marimo_export_md.yml:31

  • MARIMO_VERSION is pinned to 0.23.9, but the notebook being exported declares __generated_with = "0.23.11". Pinning the exporter to the same version avoids version skew and reduces the chance that a future export changes output formatting unexpectedly.
  MARIMO_VERSION: "0.23.9"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants