A local, auditable multi-agent system for automotive failure investigation.
AFAMA orchestrates a pipeline of eight specialized AI agents, all running locally via Ollama or a self-hosted vLLM server, to investigate failure scenarios, synthesize expert engineering opinions, and produce structured, traceable reports. No external API keys or cloud services are required, ensuring complete data privacy and intellectual property protection.
- Overview
- Key Features
- Architecture and Workflow
- Agent Roster
- Prerequisites
- Installation
- Configuration
- Usage
- Project Structure
- Production Notes
- License
AFAMA is designed for automotive engineers, quality assurance teams, and compliance officers who require a repeatable, traceable investigation workflow. Given a failure scenario identifier (for example, SC-042), the system executes the following steps:
- Evidence Retrieval. Searches a local ChromaDB knowledge base (RAG) for similar past incidents, applicable regulatory requirements, and relevant test reports.
- Specialist Analysis. Runs three independent analyses — Technical, Quality, and Compliance — either concurrently (vLLM) or sequentially (Ollama), depending on backend configuration.
- Critique. Cross-examines the three specialist opinions to surface contradictions, weak claims, and evidence gaps.
- Synthesis. Unifies all findings into a single, balanced investigation report.
- Validation. Automatically quality-checks the synthesis against the source evidence, retrying up to two times if it falls below threshold.
- Human Approval. Presents the finished report for human review and approval before any automated action is taken.
- Automation. Upon approval, saves the report to disk and creates a structured engineering follow-up ticket.
Every stage writes its inputs, outputs, and agent decisions to an append-only structured audit log, so any report generated by AFAMA can be traced back to the exact evidence and reasoning that produced it.
| Feature | Description |
|---|---|
| Fully Local Execution | Runs against Ollama (Llama 3.1, Mistral, etc.) or a self-hosted vLLM server. No data leaves the host machine. |
| Pluggable LLM Backends | Switch between ollama and vllm via a single environment variable. Each backend exposes an identical interface. |
| LangGraph Orchestration | Graph-based workflow with a strongly-typed InvestigationState TypedDict. Conditional edges handle validation retries and approval routing. |
| Model Context Protocol (MCP) | Capability servers (Knowledge, Analysis, Automation) expose tools via FastMCP, decoupling retrieval and file I/O from agent logic. |
| ChromaDB RAG | Documents are embedded using nomic-embed-text and stored in a persistent ChromaDB collection. Embedding model version is stamped in collection metadata; a model change raises an explicit error instead of silently corrupting retrieval. |
| Pydantic v2 Contracts | All agent handoffs use validated Pydantic models defined in core/schemas.py. No loose dictionaries cross agent boundaries. |
| Structured Audit Logging | Every agent action is written to audit/events.jsonl via a thread-safe rotating file handler. Reads and writes share a lock to prevent partial-line races under concurrent execution. |
| Prompt Separation | Agent personas and instructions are stored as Markdown files under skills/. Engineers can tune agent behaviour without modifying Python code. |
| Rate Limiting and Backoff | A configurable semaphore (LLM_MAX_CONCURRENT) caps simultaneous LLM calls. Retries use exponential backoff with full jitter to avoid thundering-herd recovery. |
| SIGTERM Handling | A signal handler flushes the audit log and exits cleanly on SIGTERM, suitable for containerized and systemd-managed deployments. |
The workflow is managed by LangGraph, which passes a strongly-typed InvestigationState between nodes. Whether the three specialist agents run concurrently or sequentially is controlled by the PARALLEL_SPECIALISTS setting, which defaults to False for Ollama (single-request server) and True for vLLM (continuous batching).
User Request
|
v
Orchestrator (LangGraph)
|
+-> Evidence Agent (RAG search: incidents, requirements, test reports, model cards)
| |
| v
+-> [Parallel or Sequential]
| +-> Technical Agent (Root cause, failure mode, sensor behaviour)
| +-> Quality Agent (Process gaps, test coverage, engineering risk)
| +-> Compliance Agent (Safety standards, regulatory requirements, traceability)
| |
| v
+-> Critic Agent (Agreements, contradictions, weak and unsupported claims)
| |
| v
+-> Synthesis Agent (Unified, balanced investigation report)
| |
| v
+-> Validator Agent (Evidence-backed quality check; retries synthesis up to 2x)
| |
| v (conditional)
+-> Human Approval (Interactive approve / reject step)
| |
| v (conditional)
+-> Automation Agent (Saves report, creates engineering ticket)
|
v
END
The synthesis-validation loop is bounded to two retries (MAX_RETRIES = 2). If validation still fails after both retries, the pipeline proceeds to human review with a warning recorded in the audit log, so the operator can make an informed decision rather than receiving a silently degraded output.
| # | Agent | Skill File | Role |
|---|---|---|---|
| 1 | Evidence Agent | skills/evidence_agent.md |
Retrieval-augmented search over the local knowledge base. Produces an EvidenceReport. |
| 2 | Technical Agent | skills/technical_agent.md |
Root-cause hypothesis generation, failure mode analysis, sensor behaviour review. |
| 3 | Quality Agent | skills/quality_agent.md |
Process and quality management system assessment, test coverage gap analysis. |
| 4 | Compliance Agent | skills/compliance_agent.md |
ISO 26262 / SOTIF requirement traceability and regulatory alignment check. |
| 5 | Critic Agent | skills/critic_agent.md |
Cross-examination of the three specialist opinions. Produces a ComparativeReview. |
| 6 | Synthesis Agent | skills/synthesis_agent.md |
Merges all findings into a single FinalSynthesis, rejecting unsupported claims. |
| 7 | Validator Agent | skills/validator_agent.md |
Verifies every accepted finding against the source evidence. Produces a ValidationReport. |
| 8 | Automation Agent | skills/automation_agent.md |
Writes the approved report to workspace/reports/ and creates a ticket in workspace/tickets/. |
All specialist agents (Technical, Quality, Compliance) inherit from agents/base_specialist.py. New agents must follow the same inheritance pattern and register a corresponding skill file and Pydantic schema pair.
Before installing AFAMA, ensure the following are available on the host system:
- Ollama installed and running locally (
ollama serve), or a vLLM server reachable at a configured base URL. - At least one compatible model pulled, for example:
ollama pull llama3.1 ollama pull nomic-embed-text
- Python 3.11 or later.
pipfor dependency installation.- Sufficient local compute resources (CPU or GPU, and RAM) to serve the selected model.
-
Clone the repository:
git clone https://github.com/<your-org>/autoagent.git cd autoagent
-
Create and activate a virtual environment:
python -m venv venv source venv/bin/activate # Windows: venv\Scripts\activate
-
Install dependencies:
pip install -r requirements.txt
-
Pull the required Ollama models (Ollama backend only):
ollama pull llama3.1 ollama pull nomic-embed-text
-
Copy the example environment file and set values as required:
cp .env.example .env
-
Ingest the knowledge base into ChromaDB:
python main.py --ingest
AFAMA is configured exclusively through environment variables, read from a .env file in the project root. Refer to .env.example for all available options. The principal variables are listed below.
| Variable | Default | Description |
|---|---|---|
LLM_BACKEND |
ollama |
Backend to use. Set to vllm for self-hosted vLLM. |
LLM_MODEL |
llama3.1 |
Ollama model name. |
LLM_TEMPERATURE |
0.1 |
Sampling temperature applied to all agents. |
LLM_MAX_TOKENS |
4096 |
Maximum output tokens per request. |
LLM_MAX_CONCURRENT |
1 (Ollama) / 4 (vLLM) |
Maximum simultaneous LLM calls across all threads. |
OLLAMA_BASE_URL |
http://localhost:11434 |
Ollama server address. Override for remote or containerized deployments. |
VLLM_BASE_URL |
http://localhost:8000/v1 |
vLLM OpenAI-compatible endpoint. Override for cloud or Kubernetes deployments. |
VLLM_MODEL |
meta-llama/Llama-3.1-8B-Instruct |
Model identifier passed to the vLLM server. |
VLLM_API_KEY |
(none) | Required only if the vLLM server has authentication enabled. |
| Variable | Default | Description |
|---|---|---|
PARALLEL_SPECIALISTS |
false (Ollama), true (vLLM) |
Whether the three specialist agents run concurrently. Set to false when using Ollama to avoid connection errors from the single-request server. |
| Variable | Default | Description |
|---|---|---|
EMBED_MODEL |
nomic-embed-text |
Ollama embedding model. Must match the model used during ingestion; a mismatch raises an explicit error at startup. |
CHROMA_COLLECTION |
autoagent_knowledge |
ChromaDB collection name. |
RETRIEVAL_TOP_K |
5 |
Number of documents returned per RAG query. |
The port constants below are provided for documentation purposes. Pass them when launching MCP servers as independent processes.
| Variable | Default |
|---|---|
KNOWLEDGE_MCP_PORT |
8100 |
ANALYSIS_MCP_PORT |
8101 |
AUTOMATION_MCP_PORT |
8102 |
Agent personas are maintained as Markdown files under skills/. Edit these files to tune agent behaviour, reframe perspectives, or adjust output format requirements without modifying any Python source file.
python main.pyThis investigates scenario SC-042 using the built-in demonstration request.
python main.py --query "Investigate the camera false-negative rate under heavy rain conditions."python main.py --model llama3 --task-id INV-2026-001Run this after adding new documents to data/ or after changing the embedding model:
python main.py --ingest- The RAG pipeline is built and, if the collection is empty, documents are ingested automatically.
- The investigation graph is constructed and invoked with the supplied request.
- All eight agents run in sequence (or with specialists in parallel, if configured).
- The finished report is displayed in the terminal for human review.
- On approval, the report is written to
workspace/reports/<task-id>.mdand a ticket is created inworkspace/tickets/. - All events are appended to
audit/events.jsonl.
autoagent/
|
+-- agents/ Agent implementations
| +-- base_specialist.py Base class for Technical, Quality, and Compliance agents
| +-- evidence_agent.py
| +-- technical_agent.py
| +-- quality_agent.py
| +-- compliance_agent.py
| +-- critic_agent.py
| +-- synthesis_agent.py
| +-- validator_agent.py
| +-- automation_agent.py
|
+-- core/ Shared infrastructure
| +-- schemas.py All Pydantic contracts (single source of truth)
| +-- llm.py Backend-agnostic LLM client with retry and rate limiting
| +-- backends.py OllamaBackend and VLLMBackend implementations
| +-- skill_loader.py Loads agent personas from skills/
| +-- logger.py Thread-safe structured audit logger
|
+-- mcp_servers/ FastMCP capability servers
| +-- knowledge_server.py RAG search tools
| +-- analysis_server.py Scenario metadata and sensor comparison tools
| +-- automation_server.py Report and ticket persistence tools
|
+-- orchestrator/ LangGraph pipeline
| +-- graph.py DAG construction and compilation
| +-- nodes.py One function per graph node; all agent calls with error guards
| +-- routing.py Conditional edge logic (validation retry, human approval)
| +-- state.py InvestigationState TypedDict
| +-- context.py PipelineContext dependency container (thread-safe, no globals)
|
+-- rag/ Retrieval-augmented generation
| +-- adapter.py KnowledgeAdapter (unified retrieval interface)
| +-- embeddings/ OllamaEmbedder
| +-- vectorstores/ ChromaStore with embedding version check and query timeout
| +-- ingestion/ Document loader (Markdown, JSON, and MarkItDown for other formats)
| +-- retrieval/ Retriever (thin adapter over ChromaStore.search)
| +-- setup.py Build the full RAG pipeline and return a KnowledgeAdapter
|
+-- skills/ Agent persona Markdown files (never modify Python to change prompts)
+-- data/ Source knowledge base (incidents/, requirements/, scenarios/, etc.)
+-- workspace/ Output directory
| +-- reports/ Generated investigation reports (.md)
| +-- tickets/ Engineering follow-up tickets (.json)
|
+-- audit/ Append-only structured event log
| +-- events.jsonl
|
+-- config.py Central configuration (env-var driven, no hardcoded secrets)
+-- main.py CLI entry point with SIGTERM handling
+-- requirements.txt Python dependencies
+-- .env.example Reference environment file
Embedding model changes. If EMBED_MODEL is changed after an initial ingestion, the ChromaDB collection metadata will not match and AFAMA will raise a RuntimeError at startup. Run python main.py --ingest to rebuild the index with the new model.
Concurrent execution. Set PARALLEL_SPECIALISTS=true only when using vLLM, which supports concurrent batching. Setting it with Ollama causes connection drops because Ollama processes one request at a time per model by default.
Rate limiting. LLM_MAX_CONCURRENT controls the semaphore that gates all LLM calls across threads. The default is 1 for Ollama and 4 for vLLM. Raise the vLLM value based on available GPU memory and server configuration.
Containerized deployments. Set OLLAMA_BASE_URL and VLLM_BASE_URL to the appropriate service addresses. The localhost defaults will fail silently in Kubernetes or Docker Compose environments where the LLM server is a separate container.
Audit log integrity. The audit/events.jsonl file is append-only. Do not modify or truncate it during a running investigation. Log rotation is handled automatically by RotatingFileHandler (10 MB per file, five backup files retained).
Report backups. If a report file for a given task identifier already exists in workspace/reports/, the previous version is automatically backed up with a UTC timestamp suffix before the new file is written.
MCP servers as independent processes. In the default CLI mode, MCP tool functions are called in-process. To run the servers independently (for example, as microservices), start each server with its configured port:
uvicorn mcp_servers.knowledge_server:mcp --port 8100
uvicorn mcp_servers.analysis_server:mcp --port 8101
uvicorn mcp_servers.automation_server:mcp --port 8102Replace the in-process function calls in orchestrator/nodes.py with FastMCP client calls pointing to the appropriate service addresses.
See the LICENSE file for details.