Serve with gunicorn (1 worker) and raise the OHTTP request cap to 20 MiB

[adambalogh]

Summary

Two gateway-side fixes for the intermittent failures reported in OpenGradient/chat-app#704 (opaque 502s / CORS-blocked errors on the OHTTP chat path).

1. Serve with gunicorn instead of the werkzeug dev server

start.sh ran python3 -m tee_gateway, which serves production traffic through Flask's development server (application.run()), whose fragile connection handling under concurrent load surfaces to clients as opaque connection resets — a 502 at the relay, CORS-blocked in the browser. Gunicorn was already pinned in pyproject.toml/uv.lock but nothing invoked it (the app factory even has a comment anticipating it).

Behavior-preservation constraints, encoded in the config:

• Exactly 1 worker process — all of this is in-process state that every request must share: the TEE keys (RSA signing + HPKE, generated per process and registered with nitriding — a second worker would sign/decrypt with different keys behind the same endpoint), the one-time /v1/keys provider-key injection and the x402 middleware it installs, and the price-feed/heartbeat background threads. Clients (including the OHTTP relay) always hit the same process, so any request-to-request affinity (e.g. polling flows) is preserved by construction.
• Concurrency via gthread pool (32 threads) — matches the dev server's threaded=True behavior.
• --timeout 0 — no worker watchdog, so long-lived requests (image generation up to 120s, streamed completions) aren't killed mid-flight; duration is already bounded by the app's own httpx timeouts, and the dev server had no watchdog either.
• No --max-requests — recycling the worker would regenerate the TEE keys mid-life.
• No --preload — the price feed thread starts at import time and must live in the worker process.

python3 -m tee_gateway (make test-local) still runs the dev server for local development. The deploy pipeline needs no changes — the entrypoint is baked into the image (CMD /bin/start.sh).

2. Raise _MAX_ENCAPSULATED_REQUEST_BYTES from 16 MiB to 20 MiB

The Image Studio sends reference images inline (base64) and allows up to 10 attachments, so multi-image edit requests could legitimately exceed the old cap sized for a single ~10 MiB image (surfacing as a flaky 413). Both enforcement points use the constant. The chat-app now also downscales references at attach time (OpenGradient/chat-app#710), so this is headroom, not an invitation for bigger payloads.

Testing

• uv run pytest tee_gateway/test — 194 passed, 3 skipped (includes the oversized-body test, which derives its payload from the constant).
• Smoke-tested the exact gunicorn invocation from start.sh locally: /health 200 (live price feed), /v1/ohttp/config 200, and garbage to /v1/ohttp returns the proper {"error": "malformed encapsulated request"}.

Note: both changes alter the enclave image, so PCR measurements move on the next deploy.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Pr2LKM6cnW5wbGo6DeRCZp