Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 41 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ from openai import OpenAI

client = OpenAI() # picks up the env vars above
r = client.chat.completions.create(
model="claude-sonnet-4-6",
model="claude-sonnet-5",
messages=[{"role": "user", "content": "Explain TEE attestation in one line."}],
)
print(r.choices[0].message.content)
Expand All @@ -75,7 +75,7 @@ With `og-veil` running, set a custom endpoint — either via the CLI:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:11434/v1
hermes config set OPENAI_API_KEY og-veil # ignored; your Chat login authenticates
hermes config set model.default claude-sonnet-4-6
hermes config set model.default claude-sonnet-5
```

…or by editing `~/.hermes/config.yaml` directly:
Expand All @@ -91,6 +91,43 @@ model:
Now `hermes` runs against attested, end-to-end-encrypted inference with no other
changes. Confirm it's flowing through the enclave with `og-veil status`.

## Models

The same lineup [OpenGradient Chat](https://chat.opengradient.ai) offers, reached
through the same verified TEE path. `og-veil models` prints it (add `--all` for
superseded ones), and `GET /v1/models` serves it to any agent that probes.

| Provider | Models |
|----------|--------|
| Anthropic | `claude-haiku-4-5`, `claude-sonnet-5`, `claude-opus-5`, `claude-fable-5` |
| OpenAI | `gpt-4.1-nano`, `gpt-5-mini`, `gpt-5`, `gpt-5.6-luna`, `gpt-5.6-terra`, `gpt-5.6-sol` |
| Google | `gemini-3.6-flash`, `gemini-3.5-flash-lite`, `gemini-2.5-pro`, `gemini-2.5-flash-lite` |
| xAI | `grok-4.5`, `grok-4.3`, `grok-4-fast`, `grok-4` |
| DeepSeek | `deepseek-v4-flash`, `deepseek-v4-pro` |
| ByteDance | `seed-2.0-lite`, `seed-1.8`, `seed-1.6` |
| Nous Research | `hermes-4-405b`, `hermes-4-70b` |
| Z.ai | `glm-5.2` |
| Image generation | `gpt-image-2`, `gemini-3.1-flash-image`, `seedream-5.0-lite`, `seedance-4.5`, `glm-image` |

Names are **bare** - send `claude-opus-5`, not `anthropic/claude-opus-5`. Earlier
generations (`claude-opus-4-8`, `gpt-5.5`, `grok-4-1-fast`, `o3`, …) stay
routable and stay listed, so an agent already configured for one keeps working.

**Image models are called like any other chat completion** and the images come
back on the response — but note what the guarantee covers: the request is
OHTTP-sealed exactly as a text prompt is, so the privacy story is unchanged,
while the enclave's signature covers the assistant text, *not* the generated
image bytes. Those ride out-of-band and are unsigned (see the SDK's
`response_content_for_hash`). So `X-OpenGradient-Verified: true` on an
image-generation response attests the exchange, not the pixels.

**No video models.** Chat's Video Studio (Veo, Kling, Seedance, Wan) doesn't run
over this path at all — those requests go to chat-api, which holds the
fal/BytePlus/Atlas keys and calls those providers directly, with no enclave, no
OHTTP and no signature anywhere in the chain. Listing them here would advertise
a guarantee that doesn't exist. It needs the gateway to proxy video generation
in-enclave first.

---

## How it works
Expand Down Expand Up @@ -147,7 +184,8 @@ the local OpenAI-compatible server.
| `og-veil stop` | Stop the background server. |
| `og-veil restart` | Stop and start the background server — e.g. after `og-veil update`. |
| `og-veil status` | Login + network config + whether the server is running. |
| `og-veil test ["prompt"]` | Send a one-off prompt to the running server and print the verified reply. Automatically targets the port the server was started on; override with `--host`/`--port`. |
| `og-veil test ["prompt"]` | Send a one-off prompt to the running server and print the verified reply. Automatically targets the port the server was started on; override with `--host`/`--port`. Pick a model with `--model`. |
| `og-veil models [--all]` | List the models you can send, grouped by provider. |
| `og-veil update` | Update og-veil to the latest version. |
| `og-veil login` | Authorize / re-authorize this device. |
| `og-veil setup` | Re-run the setup wizard. |
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "opengradient-veil"
version = "0.2.9"
version = "0.3.0"
description = "OpenGradient Veil — a drop-in, self-verifying private-inference proxy for AI agents. Point your OpenAI SDK at it; it routes prompts through OpenGradient's decentralized network of attestable Nitro TEE gateways and cryptographically verifies every response before a single token reaches your code."
readme = "README.md"
requires-python = ">=3.11"
Expand Down
23 changes: 19 additions & 4 deletions tests/test_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,14 +65,29 @@ def test_env_prints_env_vars():
assert result.exit_code == 0


def test_models_lists_known_models():
def test_models_lists_the_catalog_grouped_by_provider():
result = CliRunner().invoke(cli.main, ["models"])
assert result.exit_code == 0
# Should print at least one model name derived from the SDK's TEE_LLM enum.
assert "Anthropic" in result.output
assert "claude-opus-5" in result.output
assert "hermes-4-405b" in result.output
# Superseded models are held back for --all.
assert "claude-opus-4-8" not in result.output


def test_models_all_adds_superseded_and_sdk_only_models():
result = CliRunner().invoke(cli.main, ["models", "--all"])
assert result.exit_code == 0
assert "claude-opus-4-8" in result.output
# An id only the SDK's enum knows about still gets listed.
from opengradient import TEE_LLM

sample = next(iter(TEE_LLM)).value.split("/", 1)[1]
assert sample in result.output
from veil import models as catalog

curated = {m.id for m in catalog.CATALOG}
sdk_only = sorted({m.value.split("/", 1)[1] for m in TEE_LLM} - curated)
if sdk_only:
assert sdk_only[0] in result.output


def test_test_command_posts_prompt_to_localhost_and_prints_reply():
Expand Down
92 changes: 92 additions & 0 deletions tests/test_models.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
"""The advertised model catalog.

The point of these is that the listing can only ever grow: the catalog mirrors
what the Chat app offers, the SDK's ``TEE_LLM`` enum is folded in on top, and
neither source can silently drop a model an agent is already configured for.
"""

from __future__ import annotations

from opengradient import TEE_LLM

from veil import models


def test_catalog_ids_are_bare_wire_names():
# The gateway is sent `claude-opus-5`, never `anthropic/claude-opus-5` --
# both the SDK and the Chat app strip the provider prefix before the wire.
for model in models.CATALOG:
assert "/" not in model.id, model.id


def test_catalog_ids_are_unique():
ids = [m.id for m in models.CATALOG]
assert len(ids) == len(set(ids))


def test_every_provider_has_a_label():
for model in models.CATALOG:
assert model.provider in models.PROVIDER_LABELS, model.provider


def test_catalog_covers_the_models_the_chat_app_offers():
# The ten the SDK enum lagged on, which is what this catalog exists to fix,
# plus a couple of long-standing ones to catch a wholesale regression.
expected = {
"gpt-5.6-luna",
"gpt-5.6-terra",
"gpt-5.6-sol",
"claude-sonnet-5",
"claude-opus-5",
"claude-fable-5",
"gemini-3.6-flash",
"gemini-3.5-flash-lite",
"grok-4.5",
"grok-4.3",
"hermes-4-405b",
"glm-5.2",
"deepseek-v4-pro",
}
assert expected <= {m.id for m in models.CATALOG}


def test_known_models_is_a_superset_of_the_sdk_enum():
# Folding the enum in is what keeps a model the Chat app has retired (but
# the network still serves) listed, and picks up anything a future SDK adds.
sdk_ids = {m.value.split("/", 1)[1] for m in TEE_LLM}
assert sdk_ids <= set(models.known_model_ids())


def test_known_models_includes_the_whole_catalog_first():
known = models.known_model_ids()
assert known[: len(models.CATALOG)] == [m.id for m in models.CATALOG]
assert len(known) == len(set(known))


def test_default_model_is_in_the_catalog():
assert models.get(models.DEFAULT_MODEL) is not None


def test_owned_by_reports_the_provider_for_curated_models():
assert models.owned_by("claude-opus-5") == "anthropic"
assert models.owned_by("deepseek-v4-pro") == "deepseek"
# An id we don't curate keeps the generic owner rather than guessing.
assert models.owned_by("some-future-model") == "opengradient"


def test_by_provider_hides_superseded_models_by_default():
visible = {m.id for _label, entries in models.by_provider() for m in entries}
assert "claude-opus-5" in visible
assert "claude-opus-4-8" not in visible

everything = {
m.id for _label, entries in models.by_provider(include_legacy=True) for m in entries
}
assert "claude-opus-4-8" in everything


def test_by_provider_separates_chat_from_image_models():
chat = {m.id for _label, entries in models.by_provider() for m in entries}
image = {m.id for _label, entries in models.by_provider(kind=models.IMAGE) for m in entries}
assert "gpt-image-2" in image
assert not chat & image
16 changes: 16 additions & 0 deletions tests/test_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
from opengradient import RelayError, VerificationError, VerifiedChatResponse
from opengradient.client.tee_verify import TeeProof

from veil import models
from veil.server import create_app


Expand Down Expand Up @@ -139,3 +140,18 @@ def test_models_and_health():
health = client.get("/health").get_json()
assert health["status"] == "ok"
assert health["active_tee_id"] == "0xabc"


def test_models_lists_the_advertised_catalog_in_openai_shape():
client = _client(_StubGateway())
data = client.get("/v1/models").get_json()["data"]

assert [m["id"] for m in data] == models.known_model_ids()
assert all(m["object"] == "model" for m in data)
by_id = {m["id"]: m for m in data}
# Bare wire names, and the provider is reported as the owner.
assert "/" not in "".join(by_id)
assert by_id["claude-opus-5"]["owned_by"] == "anthropic"
# Superseded models stay listed: an agent already configured for one must
# keep working after it drops out of the Chat app's UI.
assert "claude-opus-4-8" in by_id
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

60 changes: 50 additions & 10 deletions veil/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
import click

from veil.config import DEFAULT_APP_URL, ServerConfig
from veil.models import DEFAULT_MODEL
from veil.session import AuthError, Session, login, login_manual


Expand Down Expand Up @@ -276,20 +277,54 @@ def env_cmd() -> None:


@main.command()
def models() -> None:
@click.option(
"-a",
"--all",
"show_all",
is_flag=True,
help="Also show superseded models and any extras the SDK knows about.",
)
def models(show_all: bool) -> None:
"""List the models available through OpenGradient Veil.

Derived from the SDK's canonical ``TEE_LLM`` enum — the same source the
local server uses for ``GET /v1/models`` — so it stays in sync with the
network as models are added or retired.
The same lineup the OpenGradient Chat app offers, grouped by provider. Names
printed here are what an agent puts in its ``model`` field.
"""
from opengradient import TEE_LLM
from veil import models as catalog

def _print(entries: list[catalog.Model]) -> None:
width = max(len(m.id) for m in entries)
for model in entries:
suffix = " (superseded)" if model.legacy else ""
click.echo(f" {model.id.ljust(width)} {model.description}{suffix}")

# Each enum value is ``provider/model``; agents send the bare model name.
names = sorted(m.value.split("/", 1)[1] for m in TEE_LLM)
click.echo("Models available through OpenGradient Veil:")
for name in names:
click.echo(f" {name}")
for label, entries in catalog.by_provider(include_legacy=show_all):
click.secho(f"\n {label}", bold=True)
_print(entries)

image = [
model
for _label, entries in catalog.by_provider(include_legacy=show_all, kind=catalog.IMAGE)
for model in entries
]
if image:
click.secho("\n Image generation", bold=True)
click.echo(" Same /v1/chat/completions call. The images come back on the response,")
click.echo(" outside the signed output hash -- the exchange is verified, not the pixels.")
_print(image)

if show_all:
# Older generations the Chat app has retired from its UI. Still routable,
# and still valid on an agent config that already names one.
curated = {m.id for m in catalog.CATALOG}
extra = [m for m in catalog.known_model_ids() if m not in curated]
if extra:
click.secho("\n Also routable", bold=True)
for name in extra:
click.echo(f" {name}")
else:
click.echo("\nRun `og-veil models --all` to include superseded models.")


@main.command(name="login")
Expand All @@ -315,7 +350,12 @@ def login_cmd(app_url: str, no_browser: bool, manual: bool) -> None:

@main.command(name="test")
@click.argument("prompt", nargs=-1)
@click.option("--model", default="gpt-4.1", show_default=True, help="Model to send the prompt to.")
@click.option(
"--model",
default=DEFAULT_MODEL,
show_default=True,
help="Model to send the prompt to (see `og-veil models`).",
)
@click.option(
"--host", default=None, help="Override the host to reach (default: the running server's)."
)
Expand Down
Loading
Loading