A local RAG-powered document Q&A application. Upload documents, ask questions, and get AI-generated answers grounded in your document content — all running locally with no API keys required.
- Bun (runtime and package manager)
- Ollama (optional — the app falls back to mock AI responses if Ollama isn't running)
bun install
bun devOpen http://localhost:3000. No environment variables, no database setup — SQLite creates itself automatically in data/.
Ollama is optional. Without it, the app runs with deterministic mock responses — useful for development and testing, but not for real Q&A.
To enable real AI responses:
-
Pull the required models:
ollama pull llama3.2 ollama pull nomic-embed-text
-
Make sure Ollama is running (
http://localhost:11434) before starting the app
The app auto-detects Ollama on startup and falls back to mocks if it's unavailable.
| Script | Description |
|---|---|
bun dev |
Start the development server |
bun run build |
Production build |
bun start |
Start the production server |
bun run lint |
Lint and format check (Biome) |
bun test |
Run tests |
bun run test:watch |
Run tests in watch mode |
- Upload PDF, TXT, or Markdown files (max 10MB) through the sidebar
- Documents are chunked, embedded, and stored in a local SQLite database
- Ask questions in the chat — a LangGraph RAG agent handles the flow:
- Classify intent — determines if the message is a question, greeting, or off-topic
- Retrieve — embeds the query and finds the most similar document chunks
- Grade relevance — checks if retrieved chunks meet a similarity threshold
- Rephrase & retry — if chunks aren't relevant and retries remain, rephrases the query
- Generate — produces a response with source citations
- Use @mentions in the chat input to scope searches to specific documents
- Responses stream in progressively with full Markdown rendering
┌─────────────────────────────────────────┐
│ START │
└────────────────┬────────────────────────┘
│
▼
┌───────────────────┐
│ classifyIntent │
└─────────┬─────────┘
│
┌─────────────────────┼────────────────────┐
│ │ │
"greeting" "question" "off_topic"
│ │ │
│ ▼ │
│ ┌─────────────────┐ │
│ │ retrieve │ │
│ └────────┬────────┘ │
│ │ │
│ ▼ │
│ ┌────────────────────┐ │
│ │ gradeRelevance │ │
│ └─────────┬──────────┘ │
│ │ │
│ ┌────────────┼────────────┐ │
│ │ │ │ │
│ sim >= 0.4 sim < 0.4 sim < 0.4 │
│ │ retries > 0 retries = 0 │
│ │ │ │ │
│ │ ▼ │ │
│ │ ┌────────────┐ │ │
│ │ │ rephrase │ │ │
│ │ └──────┬─────┘ │ │
│ │ │ │ │
│ │ └──► retrieve │
│ │ (retry loop) │
│ │ │ │
│ ▼ ▼ │
│ ┌────────────────────────────────┐ │
└──► │ generate │ ◄──┘
└───────────────┬────────────────┘
│
▼
END
| Category | Technology |
|---|---|
| Framework | Next.js 16, React 19, TypeScript |
| AI / Agent | LangGraph + LangChain, AI SDK, Ollama (llama3.2 + nomic-embed-text) |
| Database | SQLite (better-sqlite3), custom vector store |
| Styling | Tailwind CSS, shadcn/ui |
| Data Fetching | TanStack React Query |
| Testing | Vitest |
| Linting | Biome |
The challenge prompt specified LangChain/LangGraph for agent orchestration with a choice between Node.js and Python for the server side. I chose Next.js because it's where I'm most productive, but in hindsight, LangChain and LangGraph are clearly more mature on the Python side — more examples, better documentation, and broader community support.
The AI SDK + LangChain/LangGraph combination in Next.js was friction-heavy. These libraries weren't designed to work together seamlessly, and the integration points (streaming LangGraph output through AI SDK's chat transport layer, reconciling message formats, filtering internal structured output from the stream) required workarounds that added complexity without adding value. For this project's scope, AI SDK alone could have handled the use cases cleanly without LangGraph.
That said, if this were a production system with plans for resumability, parallel branches, long-running workflows, or human-in-the-loop steps, I'd go with a Python backend using LangGraph/LangChain — that's where the ecosystem's real strength is. The graph-based orchestration model is powerful, but the Node.js implementation isn't where you want to be building on it long-term.
-
Custom minimal vector store built on SQLite — Rather than using a dedicated vector database, I built a minimal vector store from scratch inside SQLite. Embeddings are stored as JSON-serialized float arrays in a TEXT column, and similarity search is brute-force cosine similarity computed in JavaScript — every chunk is loaded into memory, scored, and sorted on each query. The entire app was designed to run fully local with zero external services, which is a nice developer experience but not scalable. In production, I'd use a proper vector database like pgvector hosted on Supabase or a managed service like Pinecone, which gives you indexed approximate nearest neighbor search out of the box and removes the need to hand-roll any of this.
-
Rephrase step limited by model capabilities — The rephrase node exists in the graph and technically works, but llama3.2 (3B) tends to mangle queries rather than improve them. With a more capable model (GPT-4o, Claude, larger Llama variants), query rephrasing would meaningfully improve retrieval on follow-up questions and pronoun resolution.
-
Threshold-based relevance grading instead of LLM-based — Relevance grading uses a numeric cosine similarity threshold (0.4) rather than asking the LLM to judge relevance. This was changed because the local model was unreliable at structured relevance judgments. A stronger model could do LLM-based grading for more nuanced decisions.
-
No conversation persistence — Chat history lives only in the browser via the AI SDK's
useChathook. Refreshing the page loses the conversation. Production would use a LangGraph checkpointer or a database-backed session store. -
No tRPC/RPC layer — API routes use plain fetch with no shared type safety between client and server. There are only a few API endpoints so this felt like overkill, but in a production codebase with more routes, tRPC or a similar RPC layer would be a must for type safety across the stack.
-
No logging or metrics — There's no observability beyond
console.logstatements. In production, I'd use PostHog for product analytics and PostHog's LLM analytics for tracking token usage, latency, and response quality across the agent pipeline.