Skip to content

camerasearch/Multifront

BranchesTags

Repository files navigation

Multifront

Multifront

Python 3.12+ License Camera Search

Multi-agent orchestration engine for hard reasoning tasks.
Drop in your API keys. Point it at a problem. Let the agents figure it out.

Quick Start · How It Works · Highlights · Tools · Structure · Migration Guide

Multifront is a multi-agent reasoning engine built by Camera Search. It coordinates multiple LLMs (GPT-5, O3, Claude Opus, Gemini 3 Pro) through a tiered kernel orchestrator — escalating from a fast planner pass (T0) through research and synthesis (T1) up to a full multi-agent pipeline with posterior voting (T2) — backed by a rich ecosystem of MCP tool servers for search, browsing, vision, code execution, and more.

Built and evaluated against the GAIA benchmark — real-world tasks that require multi-step reasoning, tool use, and information synthesis.

How It Works

flowchart TB
    subgraph orch ["Tiered Orchestrator"]
        direction LR
        T0["T0: PlannerAgent only\n(2 steps · 8K tokens · 60s)"]
        T1["T1: Planner → Research → Composer\n(10 steps · 50K tokens · 120s)"]
        T2["T2: Full pipeline + Posterior Voting\n(40 steps · 200K tokens · 600s)"]
        T0 -->|"confidence < 85"| T1
        T1 -->|"confidence < 70"| T2
    end

    Scheduler

    subgraph agents ["Kernel Agents"]
        PlannerAgent
        ResearchAgent
        ReaderAgent
        CodeAgent
        ComposerAgent
        VerifierAgent
    end

    subgraph infra ["Shared Infrastructure"]
        EvidenceStore
        ModelRouter
        Budget
        ToolRegistry
    end

    subgraph tools ["MCP Tool Servers"]
        search
        browser
        vision
        audio
        python
        reader
    end

    orch ==>|"runs agents via"| Scheduler
    Scheduler -->|"dispatches"| agents
    agents -->|"emit Decisions"| Scheduler
    Scheduler --> ToolRegistry --> tools
    Scheduler --> Budget
    agents -.->|"gather facts"| EvidenceStore
    agents -.->|"LLM calls"| ModelRouter

    classDef tier fill:#f5f3ff,stroke:#7c3aed,stroke-width:2px,color:#4c1d95
    classDef agent fill:#eff6ff,stroke:#3b82f6,stroke-width:2px,color:#1e40af
    classDef sched fill:#fef2f2,stroke:#ef4444,stroke-width:2.5px,color:#991b1b
    classDef inf fill:#ecfdf5,stroke:#10b981,stroke-width:2px,color:#065f46
    classDef tool fill:#fffbeb,stroke:#f59e0b,stroke-width:2px,color:#92400e

    class T0,T1,T2 tier
    class PlannerAgent,ResearchAgent,ReaderAgent,CodeAgent,ComposerAgent,VerifierAgent agent
    class Scheduler sched
    class EvidenceStore,ModelRouter,Budget,ToolRegistry inf
    class search,browser,vision,audio,python,reader tool

    style orch fill:#faf5ff,stroke:#7c3aed,stroke-width:2px
    style agents fill:#f0f7ff,stroke:#3b82f6,stroke-width:2px
    style infra fill:#f0fdf8,stroke:#10b981,stroke-width:2px
    style tools fill:#fffef5,stroke:#f59e0b,stroke-width:2px
Loading

The Orchestrator runs tiers T0 → T1 → T2, stopping as soon as a tier's answer meets its confidence threshold. Each tier spawns a child RunContext with a budgeted slice of steps, tokens, and time. Agents are stateless — they emit Decision objects (Respond, CallTool, Spawn, Retry, Escalate) which the Scheduler executes. Facts gathered during execution are stored in an append-only EvidenceStore and cited in the final answer. The ModelRouter routes each agent role to the appropriate model (see Model Routing).

Tier Details

Tier Agents Confidence threshold Verification Posterior voting
T0 Planner 85 — —
T1 Planner → Research → Composer 70 VerifierAgent —
T2 Planner → Research → Composer 0 (always accept) VerifierAgent ✓

Highlights

  • Tiered escalation — T0/T1/T2 confidence-based promotion; higher tiers receive the prior tier's answer as context
  • Decision-based agents — stateless agents emit typed Decision objects; the Scheduler handles all side-effects and tool execution
  • EvidenceStore — append-only collection of Fact objects with Citation provenance; final answers are composed from evidence, not re-extracted from chat history
  • Role-based model routing — ModelRouter maps semantic roles (PLANNER, WORKER, VERIFIER, REASONER, SUMMARIZER) to the best available model; fully configurable
  • Budget management — every tier and spawn gets a sliced Budget (steps, tokens, wall-clock time); exhausted budgets fall back gracefully
  • VerifierAgent — 7-dimension answer verification (factual accuracy, entity precision, units, temporal accuracy, arithmetic, source quality, completeness); score < 7 triggers retry
  • Posterior voting — multi-model consensus across diverse providers at T2 for uncertain answers
  • MCP tool ecosystem — 6 tool servers over stdio/SSE: search (Exa+Parallel), browser, vision, audio (Whisper+Deepgram), reader (Firecrawl+Docling), code execution
  • Structured tracing — every model call, tool call, and decision is recorded in a TraceCollector; per-task traces saved to JSON
  • Dashboard — FastAPI web UI for monitoring runs, streaming logs, and browsing results (python -m dashboard)
  • Session checkpointing — agent runs can be checkpointed and resumed mid-task

Model Routing

The ModelRouter assigns a model to each semantic role. Defaults (configurable via CLI flags or register()):

Role Default model Purpose
PLANNER anthropic/claude-opus-4-6 Strong reasoning, strategy, direct answers
WORKER openai/gpt-5 Fast tool use across research and code steps
VERIFIER google/gemini-3-pro-preview Diverse provider for answer verification and voting
REASONER openai/o3 Chain-of-thought, extended thinking
SUMMARIZER anthropic/claude-sonnet-4-6 Fast extraction and final answer formatting

Override at runtime:

python -m eval.gaia_runner \
  --worker-model openai/gpt-5 \
  --planner-model anthropic/claude-opus-4-6 \
  --verifier-model google/gemini-3-pro-preview

MCP Tool Servers

Tool servers live in mcp_servers/mcp_servers/ and are wired into the ToolRegistry via adapter classes. Tools are namespaced as {server}.{tool} (e.g. search.web, reader.read_url). Two servers run as persistent SSE services (started by deploy.sh); the rest are launched inline via stdio.

Server Transport Port What it does
python_server SSE (persistent) 8931 Sandboxed Python execution via E2B
browser_use_mcp_server SSE (persistent) 8930 Full browser automation (Playwright/CDP)
searching_mcp_server stdio (inline) — Web search via Exa + Parallel API (deep fallback)
reading_mcp_server stdio (inline) — Web page and document content extraction (Firecrawl → Jina → Docling → markitdown)
vision_mcp_server stdio (inline) — Image analysis via Gemini
audio_mcp_server stdio (inline) — Audio transcription via Whisper (Deepgram fallback)

Quick Start

Runtime: Python 3.12+ · uv

git clone <repo-url> && cd multifront
uv sync

Set up the MCP tool environment:

cd tool
python3 -m venv .venv-tool --python 3.12
source .venv-tool/bin/activate
pip install --no-deps -r requirements_mac.txt  # macOS
# pip install --no-deps -r requirements.txt    # Linux

Create .env from the template (see .env.example):

Required keys:

Key Provider Used for
OPENAI_API_KEY OpenAI GPT-5 (worker), O3 (reasoner)
ANTHROPIC_API_KEY Anthropic Claude Opus (planner), Claude Sonnet (summarizer)
ANTHROPIC_BASE_URL Anthropic Must be https://api.anthropic.com/v1/
ANTHROPIC_CACHE_CONTROL — Set to 0 (cache_control not supported on OAI-compat path)
GEMINI_API_KEY Google Gemini 3 Pro Preview (verifier), vision tools
PARALLEL_API_KEY Parallel Primary search engine
E2B_API_KEY E2B Sandboxed Python code execution

Optional: EXA_API_KEY, FIRECRAWL_API_KEY, DEEPGRAM_API_KEY, MODEL_PROFILE

Start the persistent tool servers (required before running benchmark):

# Terminal 1 — E2B Python sandbox SSE server (port 8931)
PYTHONPATH=. mcp_servers/.venv-tool/bin/python mcp_servers/mcp_servers/python_server.py

# Terminal 2 — Browser automation SSE server (port 8930)
PYTHONPATH=. mcp_servers/.venv-tool/bin/python mcp_servers/mcp_servers/browser_use_mcp_server.py \
    --transport sse --port 8930

Run the GAIA evaluation:

source .venv/bin/activate

# Full validation set (all 165 tasks)
python -m eval.gaia_runner --output results/

# Single task (for debugging)
python -m eval.gaia_runner --task-id <id>

# Fast mode (T0 + T1 only, skips T2)
python -m eval.gaia_runner --limit 20 --fast

# Level 2/3 tasks, full T2 pipeline
python -m eval.gaia_runner --level 2 --tier2-only

Launch the monitoring dashboard:

python -m dashboard          # http://localhost:8080

Testing

The test suite covers all kernel components, agents, model clients, tools, and the end-to-end pipeline. No API keys required — all external dependencies are mocked.

uv run pytest tests/ -v          # full suite (359 tests)
uv run pytest tests/ -q          # quiet mode
uv run pytest tests/ -k "not e2e"  # skip E2E smoke tests

# Coverage
uv run coverage run -m pytest tests/ && uv run coverage report

GCP Deployment

See DEPLOYMENT.md for the full deployment guide, including:

  • One-time VM setup (./scripts/deploy.sh --setup)
  • Deploying code changes (git push && ./scripts/deploy.sh)
  • Starting the benchmark (./scripts/deploy.sh --benchmark)
  • Monitoring all 4 services (dashboard, python_server, browser_server, gaia)
  • Troubleshooting common issues

Quick deploy:

# Configure once:
cp scripts/deploy.env.example scripts/deploy.env  # fill in GCP_PROJECT etc.

# Every deploy:
git add . && git commit -m "msg" && git push
./scripts/deploy.sh --benchmark

Project Structure

kernel/                    Core execution engine
  orchestrator.py            T0/T1/T2 tiered escalation chain
  scheduler.py               Decision dispatcher and tool executor
  context.py                 RunContext, AgentState, EvidenceStore
  decisions.py               Decision types (Respond, CallTool, Spawn, Retry, Escalate)
  budgets.py                 Budget management (steps, tokens, wall-clock)
  tracing.py                 Structured trace collection

agents/                    Kernel agent implementations (stateless)
  planner.py                 PlannerAgent — strategy + direct answer (T0)
  research.py                ResearchAgent — iterative search + evidence gathering
  reader.py                  ReaderAgent — PDF, HTML, spreadsheet extraction
  code.py                    CodeAgent — Python computation in E2B sandbox
  composer.py                ComposerAgent — final answer formatting from evidence
  verifier.py                VerifierAgent — 7-dimension verification + posterior voting
  prompts/                   Per-agent system prompt files (*.md)

models/                    LLM client abstraction
  router.py                  ModelRouter — role-based model selection
  openai.py                  OpenAIClient (GPT-5, O3)
  anthropic.py               AnthropicClient (Claude Opus, Sonnet)
  google.py                  GoogleClient (Gemini Pro)

tools/                     Tool adapter system
  registry.py                ToolRegistry — unified namespace + LRU cache
  adapters/mcp.py            MCPAdapter — stdio/SSE MCP server wrapper
  adapters/browser.py        BrowserAdapter — connects to browser SSE server (port 8930)
  adapters/local.py          LocalAdapter — simple local functions

eval/                      Evaluation framework
  gaia_runner.py             GAIA benchmark runner (primary entry point)
  scorers.py                 Scoring utilities

dashboard/                 FastAPI monitoring dashboard
  app.py                     FastAPI application
  api/                       Run management, results, and log streaming APIs
  templates/index.html       Dashboard UI

tool/                      MCP tool servers + browser automation
  mcp_servers/               Individual MCP server modules (search, browser, vision, …)
  browser/                   Browser automation tooling (Playwright, action memory)

tests/                     Test suite (359 tests, zero API keys needed)
  conftest.py                MockModelClient, MockToolAdapter, shared fixtures
  test_*.py                  Unit + integration + E2E smoke tests

data/                      GAIA dataset and run artifacts
  download_gaia.py           Dataset downloader (HuggingFace → JSONL)
  test.jsonl                 GAIA tasks (downloaded separately, not in git)

scripts/                   Deployment tooling
  deploy.sh                  Full deploy: sync → 4 tmux services → optional benchmark
  setup_gcp_vm.sh            One-time VM setup (Chrome, venvs, Playwright)

# Legacy (preserved pending GAIA score verification — see MIGRATION.md)
agent/                     Legacy ReAct agent implementations
engine/core/               Legacy runtime framework
llm/                       Legacy LLM client
test/                      Legacy benchmark runner (test_run.py)

License

Apache License 2.0 — see LICENSE and NOTICE.

Attribution

Portions of browser tooling adapted from HuggingFace open-source libraries (Apache 2.0).

About

No description, website, or topics provided.

Resources

Readme

License

Apache-2.0 license
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages