Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

#### 🔌 MCP Server
- **Rebuilt `mcp-server/` on [FastMCP](https://gofastmcp.com) 3.x** (up from the low-level `mcp` SDK), replacing ~15 hand-written tool wrappers with 27 tools generated dynamically from `tryon.cli.registry` -- every model reachable via the `opentryon` CLI (Kimi, FLUX VTO/2, GPT Image, Sora, Veo, Nano Banana, BEN2, etc.) is now automatically exposed as an MCP tool, and new registry entries need zero MCP-server changes to show up
- New `tryon.cli.runner.invoke_model()`: a kwargs-based, non-argv equivalent of the CLI's `run_service()`, shared by both the CLI and the MCP server so they can never drift apart; always returns a structured `{"success": ...}` dict instead of raising
- Two discovery tools, `list_opentryon_tools` and `opentryon_status`, report live per-model configuration/readiness straight from the registry and the loaded `.env`
- Every generated tool supports `dry_run` (preview the resolved adapter call, no API/GPU cost) and `output_dir`, matching the CLI's `--dry-run`/`--output-dir` flags
- `mcp-server/test_server.py`: offline test suite covering tool/registry parity, schema generation (required fields, `choices` -> enum), dry-run calls across all six services, and `alt_method_on_image` switching (veo/sora/luma-video)

#### 🖥️ Unified CLI
- **`opentryon` command-line interface**: Installable console script exposing every adapter through `opentryon <service> --model <model> [params...]` (services: `vton`, `generate`, `edit`, `understand`, `video-generate`, `bg-remove`)
- Data-driven model registry (`tryon/cli/registry.py`) with two-stage argument parsing, `--dry-run`, and automatic `local`-extra detection
Expand Down
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ OpenTryOn is an open-source AI toolkit designed for fashion technology and virtu
- [Installation](#installation)
- [Quick Start](#quick-start)
- [Unified CLI (`opentryon`)](#unified-cli-opentryon)
- [MCP Server](#-mcp-server)
- [Usage](#usage)
- [Datasets Module](#datasets-module)
- [Virtual Try-On with Amazon Nova Canvas](#virtual-try-on-with-amazon-nova-canvas)
Expand Down Expand Up @@ -299,6 +300,18 @@ adapter (all models above except `flux2-turbo`, `llava-next`, `kimi-vl`, and
`pip install opentryon[local]` and, for GPU-accelerated inference, a
CUDA-capable GPU.

## 🔌 MCP Server

Every model above is also available as an [MCP](https://modelcontextprotocol.io) tool, so agents in Claude Desktop, Cursor, or any other MCP client can call them directly. The server is built on [FastMCP](https://gofastmcp.com) and generates one tool per (service, model) straight from the same registry that powers the CLI -- so it's always up to date with every model in this table.

```bash
cd mcp-server
pip install -r requirements.txt
python server.py # stdio transport, ready for Claude Desktop / Cursor
```

See [`mcp-server/README.md`](mcp-server/README.md) for the full tool list, client configuration examples, and architecture notes.

## 📖 Usage

### Datasets Module
Expand Down
204 changes: 204 additions & 0 deletions mcp-server/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,204 @@
# OpenTryOn MCP Server

An [MCP](https://modelcontextprotocol.io) (Model Context Protocol) server that exposes every model in the [`opentryon`](../README.md) toolkit -- virtual try-on, image/video generation & editing, multimodal image & video understanding, and background removal -- as tools an LLM agent (Claude, Cursor, ChatGPT, or any MCP client) can call directly.

Built on [FastMCP](https://gofastmcp.com) 3.x, the actively-maintained, high-level Python framework for MCP servers (FastMCP 1.0 was folded into the official MCP Python SDK in 2024; this server uses the standalone FastMCP 2/3 project, which adds streamable-HTTP transport, better auth, and much less boilerplate).

## Why this is registry-driven

Every tool below is generated **dynamically** at import time from `tryon.cli.registry.SERVICES` -- the exact same data-driven registry that powers the `opentryon` CLI (see [`getting-started/cli.md`](../docs/docs/getting-started/cli.md)). There is no hand-written per-model wrapper function and no JSON schema to maintain by hand.

This means:

- **New models require zero MCP-server changes.** Add a model to `tryon/cli/registry.py` (see [`advanced/new-model-checklist.md`](../docs/docs/advanced/new-model-checklist.md)) and it instantly appears here as a new tool, with an accurate schema, the next time the server starts.
- **The CLI and the MCP server can never drift apart.** Both call the same `tryon.cli.runner.invoke_model()` execution path.
- Tool schemas (required/optional fields, enums for `choices`, per-field descriptions) are generated straight from each model's `Arg` definitions via Python's `inspect` + Pydantic, not duplicated JSON.

## Installation

```bash
cd opentryon
pip install -e . # core (API-backed) models
# or, to also enable local/GPU models (llava-next, kimi-vl, flux2-turbo, ben2):
pip install -e ".[local]"

cd mcp-server
pip install -r requirements.txt
```

Copy `env.template` (repo root) to `.env` and fill in whichever API keys you plan to use -- the server (and every adapter) reads that same `.env` file, so no separate MCP-specific secret handling is needed.

## Running

```bash
# stdio transport (what Claude Desktop / Cursor / most MCP clients expect)
python server.py

# streamable-HTTP transport, for remote/network clients
python server.py --transport http --host 0.0.0.0 --port 8000
```

On startup the server prints a configuration status report to stderr (which API keys are set, which models are ready) -- the same information the `opentryon_status` tool returns at runtime.

### Claude Desktop

Add to `claude_desktop_config.json` (see [`examples/claude_desktop_config.json`](examples/claude_desktop_config.json)):

```json
{
"mcpServers": {
"opentryon": {
"command": "python",
"args": ["/absolute/path/to/opentryon/mcp-server/server.py"]
}
}
}
```

### Cursor

Add to `.cursor/mcp.json` (project) or `~/.cursor/mcp.json` (global) -- see [`examples/cursor_mcp_config.json`](examples/cursor_mcp_config.json):

```json
{
"mcpServers": {
"opentryon": {
"command": "python",
"args": ["/absolute/path/to/opentryon/mcp-server/server.py"]
}
}
}
```

### Programmatically (FastMCP client)

See [`examples/example_usage.py`](examples/example_usage.py) for a full walkthrough. Quick version:

```python
import asyncio
from fastmcp import Client

async def main():
async with Client("server.py") as client:
result = await client.call_tool("vton_flux_vto", {
"person": "model.jpg",
"garment": "garment.jpg",
"dry_run": True, # preview the call, spend no API credits
})
print(result.data)

asyncio.run(main())
```

## Discovery tools

Two meta tools are always available regardless of what's configured:

- **`list_opentryon_tools(service=None)`** -- lists every service/model combination, its MCP tool name, which env var(s) it needs, and whether it's currently configured. Call this first.
- **`opentryon_status()`** -- the same human-readable status report printed on startup.

## Every generated tool accepts

- All of the model's own parameters (mirrors `opentryon <service> --model <model> --help` exactly), with the same required/optional-ness, defaults, and choice/enum constraints as the CLI.
- **`dry_run`** (bool, default `false`) -- preview the resolved adapter call (`ClassName(**init_kwargs).method(**call_kwargs)`) without hitting any API, GPU, or network.
- **`output_dir`** (str, default `"outputs"`) -- where to save any resulting images/video/JSON.

Every tool returns a structured dict: `{"success": true/false, ...}` -- never raises, so an LLM caller always gets a clean result to reason about instead of a stack trace.

## Available tools (27 models across 6 services)

### vton -- Virtual try-on: compose a garment onto a person image

| Tool | Model | Requires |
|---|---|---|
| `vton_flux_vto` | Black Forest Labs FLUX VTO | `BFL_API_KEY` |
| `vton_nova_canvas` | Amazon Nova Canvas | `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` |
| `vton_kling_ai` | Kling AI (Kolors Virtual Try-On) | `KLING_AI_API_KEY` / `KLING_AI_SECRET_KEY` |
| `vton_segmind` | Segmind Try-On Diffusion | `SEGMIND_API_KEY` |

### generate -- Text-to-image generation

| Tool | Model | Requires |
|---|---|---|
| `generate_nano_banana` | Nano Banana (Gemini 2.5 Flash Image) | `GEMINI_API_KEY` |
| `generate_nano_banana_pro` | Nano Banana Pro (Gemini 3 Pro Image Preview) | `GEMINI_API_KEY` |
| `generate_nano_banana_2` | Nano Banana 2 (Gemini 3.1 Flash Image) | `GEMINI_API_KEY` |
| `generate_flux2_pro` | FLUX.2 [pro] | `BFL_API_KEY` |
| `generate_flux2_flex` | FLUX.2 [flex] | `BFL_API_KEY` |
| `generate_flux2_turbo` | FLUX.2-dev Turbo (local, 8-step) | local/GPU |
| `generate_gpt_image` | OpenAI GPT Image | `OPENAI_API_KEY` |
| `generate_luma_image` | Luma Photon | `LUMA_AI_API_KEY` |

### edit -- Image editing (image + instruction -> image)

| Tool | Model | Requires |
|---|---|---|
| `edit_nano_banana` | Nano Banana (Gemini 2.5 Flash Image) | `GEMINI_API_KEY` |
| `edit_nano_banana_pro` | Nano Banana Pro | `GEMINI_API_KEY` |
| `edit_nano_banana_2` | Nano Banana 2 | `GEMINI_API_KEY` |
| `edit_flux2_pro` | FLUX.2 [pro] | `BFL_API_KEY` |
| `edit_flux2_flex` | FLUX.2 [flex] | `BFL_API_KEY` |
| `edit_flux2_turbo` | FLUX.2-dev Turbo (local, image-to-image) | local/GPU |
| `edit_gpt_image` | OpenAI GPT Image | `OPENAI_API_KEY` |

### understand -- Image & video understanding / captioning

| Tool | Model | Requires |
|---|---|---|
| `understand_llava_next` | LLaVA-NeXT (local VLM captioning) | local/GPU |
| `understand_kimi_k2_6` | Kimi K2.6 (Moonshot AI multimodal understanding) | `MOONSHOT_API_KEY` |
| `understand_kimi_k2_7_code` | Kimi K2.7 Code (coding + multimodal understanding) | `MOONSHOT_API_KEY` |
| `understand_kimi_vl` | Kimi-VL (open-weight, local) | local/GPU |

### video-generate -- Text/image-to-video generation

| Tool | Model | Requires |
|---|---|---|
| `video_generate_veo` | Google Veo | `GEMINI_API_KEY` |
| `video_generate_sora` | OpenAI Sora | `OPENAI_API_KEY` |
| `video_generate_luma_video` | Luma Dream Machine | `LUMA_AI_API_KEY` |

### bg-remove -- Background removal

| Tool | Model | Requires |
|---|---|---|
| `bg_remove_ben2` | BEN2 background remover (local) | local/GPU |

Run `list_opentryon_tools()` at any time for the live, authoritative version of this table (including per-model parameter docs) plus real-time configuration status.

## Architecture

```
mcp-server/
├── server.py # FastMCP app; builds one tool per (service, model) from the registry
├── config.py # .env loading + registry-driven readiness/status report
├── pyproject.toml
├── requirements.txt
├── test_server.py # offline dry-run tests (no pytest, no network/API keys needed)
└── examples/
├── claude_desktop_config.json
├── cursor_mcp_config.json
└── example_usage.py
```

`server.py`'s core trick (`_build_tool_fn`): for each `ModelSpec` in the registry, it builds a real Python function whose `inspect.signature()` has one keyword-only parameter per `Arg` (correct type -- `str`/`int`/`float`/`List[str]`/`Literal[...]` for `choices`/`bool` for flags -- and correct required-ness/default), then registers it with `@mcp.tool`. FastMCP inspects that signature to generate the tool's JSON schema, so the schema is always in sync with the registry with no manual duplication. The function body itself is a one-liner that forwards everything to `tryon.cli.runner.invoke_model(service, model, **kwargs)` -- the same execution path `opentryon <service> --model <model>` uses on the CLI.

## Testing

```bash
python test_server.py
```

Checks (all offline, no API keys or GPU required):
- every registry (service, model) has a corresponding MCP tool, and vice versa
- generated schemas have the right required/optional fields and turn `choices` into JSON schema enums
- representative `dry_run` calls resolve to the expected adapter call across services (vton, generate, edit, understand, video-generate, bg-remove)
- `alt_method_on_image` models (veo/sora/luma-video) switch from text-to-video to image-to-video only when an image is supplied
- unknown service/model and missing-local-extra cases return structured errors instead of raising
- the two discovery tools (`list_opentryon_tools`, `opentryon_status`) work and reject invalid input

## Troubleshooting

- **"requires local ML dependencies that aren't installed"** -- run `pip install opentryon[local]` in the same environment the server runs in, and make sure you have a CUDA GPU for the local models that need one.
- **A tool call returns `{"success": false, "error": "..."}`** -- this is by design: adapter/API errors are caught and returned as structured data rather than crashing the MCP connection. Check `error` (and `traceback` for real failures) for details, or call `opentryon_status` to check whether the required API key is set.
- **Tool not appearing** -- restart the server after editing `.env` or the registry; tools are built once at import time.
83 changes: 83 additions & 0 deletions mcp-server/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
"""Environment configuration for the OpenTryOn MCP server.

Loads the parent repo's ``.env`` (same file the ``opentryon`` CLI and the
adapters themselves read via ``python-dotenv``/``os.getenv``) and derives a
readiness report straight from :data:`tryon.cli.registry.SERVICES`, so this
never drifts out of sync with which models actually need which keys.
"""

from __future__ import annotations

import os
from pathlib import Path
from typing import Any, Dict, Optional

from dotenv import load_dotenv

_PARENT_DIR = Path(__file__).resolve().parent.parent
_ENV_PATH = _PARENT_DIR / ".env"
if _ENV_PATH.exists():
load_dotenv(_ENV_PATH)


def is_configured(env_hint: Optional[str]) -> Optional[bool]:
"""Return True/False for whether the env var(s) in ``env_hint`` (e.g.
``"AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY"``) are all set, or None if
the model needs no API key at all (local/open-weight models)."""
if not env_hint:
return None
names = [v.strip() for v in env_hint.split("/")]
return all(os.getenv(name) for name in names)


def status_report() -> Dict[str, Any]:
"""Build a ``{service: {model: {...}}}`` readiness map straight from the
registry."""
from tryon.cli.registry import SERVICES

report: Dict[str, Any] = {}
for service, models in SERVICES.items():
report[service] = {}
for model_id, spec in models.items():
report[service][model_id] = {
"label": spec.label,
"requires_env": spec.env_hint,
"configured": is_configured(spec.env_hint),
"runs_locally": spec.extra == "local",
}
return report


def status_message() -> str:
"""Human-readable configuration summary, printed to stderr on server
startup and returned by the ``opentryon_status`` MCP tool."""
report = status_report()

lines = ["OpenTryOn MCP Server Configuration Status:", ""]
ready = 0
total_api = 0
local_count = 0

for service, models in report.items():
lines.append(f"[{service}]")
for model_id, info in models.items():
if info["runs_locally"]:
local_count += 1
lines.append(f" {model_id:<20} local (no API key needed)")
continue
total_api += 1
if info["configured"]:
ready += 1
mark = "\u2713 configured"
else:
mark = f"\u2717 missing {info['requires_env']}"
lines.append(f" {model_id:<20} {mark}")
lines.append("")

lines.append(f"API-backed models ready: {ready}/{total_api}")
lines.append(f"Local models available (need `pip install opentryon[local]` + GPU): {local_count}")
if ready == 0:
lines.append("")
lines.append("\u26a0\ufe0f No API keys configured yet.")
lines.append(" Copy env.template to .env in the repo root and add your API keys.")
return "\n".join(lines)
11 changes: 11 additions & 0 deletions mcp-server/examples/claude_desktop_config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"mcpServers": {
"opentryon": {
"command": "python",
"args": ["/absolute/path/to/opentryon/mcp-server/server.py"],
"env": {
"PYTHONUNBUFFERED": "1"
}
}
}
}
8 changes: 8 additions & 0 deletions mcp-server/examples/cursor_mcp_config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"mcpServers": {
"opentryon": {
"command": "python",
"args": ["/absolute/path/to/opentryon/mcp-server/server.py"]
}
}
}
Loading
Loading