A self-hosted, cinematic AI generation studio — an open Higgsfield. Image, video, 3D, and audio — plus cutout, upscale, reframe, edit, and more — from one prompt-first studio. Driven by humans through a web UI and by coding agents through MCP. Runs on a bundled mock engine locally and a real SwarmUI engine on a GPU box — you flip between them with one env var.
MIT licensed · Next.js + FastMCP · bring your own GPU
- A full generation stack — create with Image, Video, 3D (image/text → GLB), Audio (voice / music / SFX), Consistent-subject, and Product-placement; transform with Cutout, Upscale, Reframe, Edit (mask inpaint), Lip sync, Motion transfer, and Video upscale. One prompt-first composer, driven by a data spec — not a wall of one-off screens. Most models are opt-in, each carrying an untested badge until verified on real hardware.
- A studio, not a form — every render is kept in a gallery you can chain from: reuse a prompt, animate a still into video, remove a background, upscale, reframe, score it with audio, or download.
- Presets — a card wall of authored cinematic looks (camera moves, lighting,
portrait, product, VFX, seamless-loop backgrounds) with a fill-in
{subject}slot. - Agent parity via MCP — anything the UI can do, a coding agent can do through a FastMCP (streamable-HTTP) server that is a thin client of the same API.
- A model registry, not hardcoded models — add a model by adding one JSON entry.
- Correct by construction — per-model required injections live in the registry, so a video can never render as a blank teal clip (the LTX-2.3 failure mode) from any surface — UI or agent.
 |
 |
| Presets — click a card, fill the subject, shoot | Gallery — every render, with prompt/seed/settings |
An open studio, not a hosted catalog. Where it exceeds the reference product:
- True self-hosting. Your weights, your box, your data. No per-generation fees, no credits, no rate limits, no vendor holding the render queue. Run it on a rented GPU or your own tower — the studio is the same.
- 3D generation. Image → GLB and text → GLB (TRELLIS.2, MIT-licensed) sit right next to image and video in the same composer and gallery. The reference product ships no 3D at all.
- Agent-native by design (MCP). The whole studio is scriptable by coding
agents over a FastMCP server — full parity with the UI, zero drift, because the
MCP server is a thin client of the same API. A
motion-specMCP tool even hands an agent a structured motion brief to build in GSAP / SVG / Lottie. - An open preset + model registry. Models and looks are data you can PR — one JSON entry adds a model, one JSON file adds a cinematic preset. It's a community catalog you extend, not a closed one you're stuck inside.
cd ui
npm install
npm run devOpen http://localhost:3000. No API keys, no .env, no GPU. The mock engine
serves bundled cinematic sample media with real queue behaviour, so every surface
and every flow (generate → gallery → animate → cutout) works exactly as it will
against real hardware. Requires Node 20+.
cd mcp
python3.11 -m venv .venv
.venv/bin/pip install -r requirements.txt
OPENFIELD_API_URL=http://localhost:3000 .venv/bin/python server.pyThe MCP server comes up at http://127.0.0.1:8000/mcp (streamable-HTTP) — add it
to Claude Design → Settings → Connectors. Tools: generate_image,
generate_video, remove_background, use_preset, list_presets, list_models,
get_job, list_active_jobs. List them without a client:
.venv/bin/python server.py --print-tools.
One driver. The Next.js app (ui/) is the backend: it owns the SwarmUI
driver (mock + real), the model registry, inference resolution, the SQLite gallery,
and the async queue, all behind a REST API. The Python MCP server (mcp/) makes no
engine decisions — it forwards to that API. So there is exactly one place where
model rules, persistence, and the video-safety guarantee live, and no drift between
what a human and an agent can do.
Mock vs real is one env var. SWARM_URL unset → the bundled mock engine;
SWARM_URL=http://127.0.0.1:7801 → a real SwarmUI. Same driver, queue, gallery,
presets, and registry either way.
The model registry (models/registry.json) is the single source of truth. Each
entry declares a model's overridable defaults and its required injections —
forced params and LoRA tags that win over everything. The resolver is generic and
reads the registry, so:
A model that isn't in the registry still works, on safe generic defaults — you're never blocked. Presets never hardcode a model, so they run on whatever stack you have installed.
The Models page (a Model Library) is the easiest way: see every model with
live status (ready / downloading / available), install or download with one
click, browse HuggingFace by type to discover new weights, and add your
own (a persisted overlay in models/user-registry.json that survives updates).
Under the hood it's still the registry — you can also edit models/registry.json
directly (install: true to have server/setup.sh download it). Full
field-by-field guide: docs/adding-models.md. The
shipped stack:
| Model | Capabilities | Notes |
|---|---|---|
flux2-dev-fp8mixedfromscaled |
image | FLUX.2 — cfg 1 enforced; verified on hardware |
ltx-2.3-22b-dev |
video | LTX-2.3 — distilled LoRA + steps 8 enforced; verified on hardware |
supir |
upscale | SUPIR diffusion upscaler (ComfyUI workflow) |
The first two are verified end-to-end on a real GPU; the rest of the stack is
opt-in, each carrying an untested badge until verified on real hardware:
TRELLIS.2 (3D),
Chatterbox / Stable Audio / MMAudio (voice / music / SFX), FLUX.2 Fill
(reframe + edit), PuLID (consistency), FLUX.2 multi-reference (placement),
InfiniteTalk / LatentSync (lip sync), Wan 2.2 Animate / MimicMotion (motion
transfer), SeedVR2 (video upscale), plus best-effort SDXL and Wan 2.2 T2V. Flip
install: true on any entry to add it to the download stack.
From your Mac, once you have SSH to a fresh box:
./server/deploy.sh 'ssh -p <PORT> root@<host>'deploy.sh rsyncs the repo and runs server/setup.sh, which detects its
environment — DATA_DIR (auto-prefers big mounted volumes like AutoDL's
/root/autodl-tmp or RunPod's /workspace), supervision (systemd units / Docker
Compose / tmux), and downloads exactly the models the registry marks install:true.
Fully hands-off — no browser wizard. setup.sh drives SwarmUI's own installer
headlessly (server/swarm-install.py): one ComfyUI backend, bound to localhost.
It then brings up the Studio + MCP and tunnels only those two — SwarmUI and
ComfyUI stay private (never exposed). Model downloads (~72 GB) plus a fast aria2
encoder pre-fetch overlap the rest, so the first generation of each model
isn't stuck on SwarmUI's slow single-stream encoder download. When it finishes it
prints the Studio + MCP URLs and you're done — generation works as soon as a
model + its encoder land. Update everything later with server/update.sh.
Guides: deploy overview — AutoDL ·
RunPod · generic Linux / Docker.
There's also a docker-compose.yml (UI + MCP; point SWARM_URL at your SwarmUI).
OPENFIELD is meant to look like a film studio, not a SaaS admin: warm near-black
chrome, a single amber accent, self-hosted Fraunces / Hanken Grotesk / JetBrains
Mono, a custom icon set, film grain, motion with a job. The generated media is the
only real color on screen. Design tokens live in ui/src/app/globals.css — derive
components from them and keep the discipline (no purple gradients, glassmorphism,
emoji icons, or generic rounded-2xl cards).
openfield/
├── ui/ Next.js studio — driver, registry resolver, gallery, queue, API
├── mcp/ FastMCP (streamable-HTTP) server — thin client of the UI API
├── models/ registry.json — the single source of truth for models
├── presets/ authored looks (plain JSON, {subject} slots)
├── server/ setup.sh (env-detecting bootstrap) · deploy.sh (from your Mac)
│ swarm-install.py (headless SwarmUI install) · update.sh
├── docs/ deploy/ guides · adding-models.md · media/ screenshots
├── Dockerfile · mcp/Dockerfile · docker-compose.yml
├── DECISIONS.md every product/engineering call and why
└── .env.example every environment variable, documented
Presets and the model registry are where community PRs are most welcome — a new
cinematic preset is a JSON file; a new model is a registry entry. See
CONTRIBUTING.md and docs/adding-models.md.
The golden rule: model-specific behaviour goes in models/registry.json, never in
code — the resolver stays generic.
MIT — see LICENSE.