adding the initial structure

.claude/CLAUDE.md

@@ -0,0 +1,152 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code when working with code in this repository.
+
+## Project Overview
+
+Kaapi is an AI platform built with FastAPI and PostgreSQL, containerized with Docker. It provides AI capabilities including multi-provider LLM access (OpenAI, Anthropic, Google, plus speech providers like Sarvam/ElevenLabs — orchestrated via `litellm`), fine-tuning, document processing, collection management, and evaluation runs. Async work runs on Celery (RabbitMQ broker, Redis backend); observability via Langfuse, OpenTelemetry, and Sentry.
+
+## Key Commands
+
+### Development
+
+**The Python project root is `backend/`, not the repo root.** Every command below assumes `cd backend` first (`pyproject.toml`, `uv.lock`, `alembic.ini`, and `scripts/` all live there). Paths like `app/...` in this doc are relative to `backend/`.
+
+```bash
+# Bring up the full stack (Postgres, RabbitMQ, Redis, backend, celery, Adminer, Flower) — from repo root
+docker compose watch
+
+# --- the rest run from backend/ ---
+cd backend
+
+# Start development server with auto-reload (stop the docker `backend` service first)
+fastapi run --reload app/main.py
+
+# Run the Celery worker locally (separate terminal)
+uv run celery -A app.celery.celery_app worker --loglevel=info
+
+# Lint / format (also wired into pre-commit)
+bash scripts/lint.sh      # ruff check + mypy strict
+bash scripts/format.sh    # ruff format + import sort
+uv run pre-commit run --all-files
+
+# Generate database migration.
+# Compute <next_rev_id> at runtime as the latest existing revision ID + 1,
+# zero-padded to 3 digits (check the highest NNN in app/alembic/versions/NNN_*.py).
+uv run alembic revision --autogenerate -m "Description" --rev-id <next_rev_id>
+uv run alembic upgrade head
+
+# Seed database with test data
+uv run python -m app.seed_data.seed_data
+
+# Internal admin/data CLI (entrypoint: app.cli.main:cli)
+uv run ai-cli --help
+```
+
+### Testing
+
+Tests use `.env.test` for environment-specific configuration (env var `ENVIRONMENT=testing`). Run against a **real Postgres** — the suite runs `tests_pre_start.py` + `alembic upgrade head` before pytest; do not mock the DB session.
+
+```bash
+# Full suite with coverage, from backend/ (applies test migrations, then pytest)
+uv run bash scripts/tests-start.sh
+
+# A single test (after migrations are applied), from backend/
+uv run pytest app/tests/path/to/test_file.py::test_name -x
+
+# Dockerized run (from repo root): builds, brings up the stack, runs the suite, tears down
+bash scripts/test.sh
+```
+
+## Architecture
+
+### Backend Structure
+
+The backend follows a layered architecture located in `backend/app/`:
+
+- **Models** (`models/`): SQLModel entities representing database tables and domain objects
+
+- **CRUD** (`crud/`): Database access layer for all data operations
+
+- **Routes** (`api/`): FastAPI REST endpoints organized by domain
+
+- **Core** (`core/`): Core functionality and utilities
+  - Configuration and settings
+  - Database connection and session management
+  - Security (JWT, password hashing, API keys)
+  - Cloud storage (`cloud/storage.py`)
+  - Document transformation (`doctransform/`)
+  - Fine-tuning utilities (`finetune/`)
+  - Langfuse observability integration (`langfuse/`)
+  - Exception handlers and middleware
+
+- **Services** (`services/`): Business logic services
+  - Response service (`response/`): OpenAI Responses API integration, conversation management, and job execution
+
+- **Celery** (`celery/`): Asynchronous task processing with RabbitMQ and Redis
+  - Task definitions (`tasks/`)
+  - Celery app configuration with priority queues
+  - Beat scheduler and worker configuration
+
+
+### Authentication & Security
+
+- JWT-based authentication
+- API key support for programmatic access
+- Organization and project-level permissions
+
+## Environment Configuration
+
+The application uses different environment files:
+- `.env` - Application environment configuration (use `.env.example` as template)
+- `.env.test` - Test environment configuration
+
+
+## Testing Strategy
+
+- Tests located in `app/tests/`
+- Factory pattern for test fixtures
+- Automatic coverage reporting
+
+## Code Standards
+
+- Python 3.12+ with type hints (`requires-python = ">=3.12"`; mypy `strict`, ruff target `py312`)
+- Pre-commit hooks for linting and formatting
+
+## Coding Conventions
+
+Layer-specific conventions live in **`.claude/conventions/*.md`** — one doc per layer (`model`, `crud`, `service`, `route`, `migration`, `celery`, `test`, `error-handling`) plus `cross-cutting.md`. The **`backend-conventions`** skill indexes them, and the `feature-builder` skill loads each layer's doc when *building* a feature. Same source of truth, so design and code never drift.
+
+### Cross-cutting rules
+
+These are a terse summary for quick reference; `.claude/conventions/cross-cutting.md` is the authoritative, fuller version (update that file, not just this list).
+
+- **Type hints** on every parameter and return value. `-> Any` is not an annotation — narrow it or drop it.
+- **Logging prefix:** every log line starts with the function name in square brackets.
+  ```python
+  logger.info(f"[function_name] Message | key: {value}")
+  ```
+- **`uv` is the runner**, not `pip`. Examples: `uv run pytest`, `uv run alembic ...`, `uv run pre-commit run --all-files`.
+- **No magic values** in code — extract repeated literals to constants / `Enum` / settings.
+- **Comments explain *why*, not *what*.** Don't restate what the code already says (`i += 1  # increment i`), don't narrate self-evident lines, and don't pad docstrings/migration descriptions with obvious recaps of the operations. A comment earns its place only by adding non-obvious context — rationale, a gotcha, a link, a constraint. When in doubt, delete it; clear code needs fewer comments, not more.
+- **Naming:** `list_*` for plural fetch, `get_*` for singletons; snake_case funcs/vars, PascalCase classes, UPPER_SNAKE constants; `Enum` suffix on enum classes.
+- **Timestamps** are `inserted_at` / `updated_at` (not `created_at`).
+
+## Skills (the feature lifecycle)
+
+Work is driven by skills, not specialist subagents. The lifecycle:
+
+| Stage | Skill | Output |
+|---|---|---|
+| Product spec | `start-prd` | `features/<slug>/PRD.md` |
+| Software spec | `srd-creator` | `features/<slug>/SRD.md` (includes blast-radius impact analysis via `docs/domain-map.md`) |
+| Execute SRD (build) | `feature-builder` | the code: model → crud → service → route, then migration / celery / tests |
+| Review | `/pr-review` | convention / security / correctness gate on the diff |
+
+The `backend-conventions` skill is the conventions index that `feature-builder` loads per layer while building.
+
+### Building a feature
+
+The `feature-builder` skill walks the dependency spine **model → crud → service → route** in the current context, loading each layer's convention doc (via `backend-conventions`) right before writing that layer, then the migration / Celery / tests as needed, and finishes by running `/pr-review` on the diff. Only build the layers the feature touches; a single-layer change just loads that one doc.
+
+For a large feature where context size is a concern, you may dispatch a **general-purpose subagent per phase** (schema-spine → migration → tests) and pass only the artifacts forward (signatures, file paths, next migration rev-id) — the `feature-builder` skill works the same whether run inline or inside a dispatched subagent.

.claude/README.md

@@ -0,0 +1,93 @@
+# AI-assisted development (`.claude/`)
+
+How features get designed and built in kaapi-backend with Claude Code. The system is **skills +
+convention docs** — no specialist subagents. Skills carry the *workflow*; convention docs carry
+the *house style*; a domain map carries *impact analysis*. Every stage loads only what it needs.
+
+## The feature lifecycle
+
+A feature moves through four stages, each driven by a skill. Each stage consumes the previous
+one's output — run them in order, but skip any that don't apply (a small change can go straight to
+`feature-builder`).
+
+| Stage | Skill | Produces | Answers |
+|---|---|---|---|
+| 1. PRD (product spec) | `start-prd` | `features/<slug>/PRD.md` | *why* are we building this, and for whom? (no tech) |
+| 2. SRD (software spec) | `srd-creator` | `features/<slug>/SRD.md` | *what* must the software do — endpoints, schema, scope, **+ blast radius** |
+| 3. Execute SRD (build) | `feature-builder` | the code | model → crud → service → route, then migration / celery / tests |
+| 4. Review | `/pr-review` | review notes | convention / security / correctness gate on the diff |
+
+Everything for one feature lives together in `features/<slug>/` (PRD, SRD), reusing the same
+kebab-case slug.
+
+## How to run it
+
+Invoke a skill by name (e.g. type `/srd-creator`), or just describe the task — the main agent
+routes to the matching skill. A typical end-to-end run:
+
+1. **Discuss the idea**, then `/start-prd` → writes `features/<slug>/PRD.md`.
+2. `/srd-creator` → writes `features/<slug>/SRD.md` (the testable spec). As part of this it reads
+   `docs/domain-map.md` and does **blast-radius analysis** — it stops and asks you about every
+   impacted surface the spec didn't address (in scope / deferred / out of scope), so a downstream
+   surface never gets silently dropped. `/grill-me` to stress-test the spec.
+3. `/feature-builder` → **executes the SRD**: walks the dependency spine, loading each layer's
+   convention doc before writing that layer, then the migration / Celery tasks / tests.
+4. `/pr-review` → the pre-merge gate.
+
+For a large feature you can dispatch a **general-purpose subagent per phase** (schema-spine →
+migration → tests), passing forward only signatures, paths, and the next migration rev-id — the
+`feature-builder` skill behaves the same inline or inside a subagent.
+
+## Code conventions
+
+All house style lives in **`.claude/conventions/*.md`** — one doc per layer. The
+**`backend-conventions`** skill is the index: it maps each layer to its doc and tells you to load
+the relevant ones *before* writing or reviewing that layer. `feature-builder` loads them while
+building — **one source of truth, so the code never drifts from house style.**
+
+| Concern | Doc | Concern | Doc |
+|---|---|---|---|
+| Cross-cutting (always) | `cross-cutting.md` | Migrations | `migration.md` |
+| Models | `model.md` | Celery tasks | `celery.md` |
+| CRUD | `crud.md` | Tests | `test.md` |
+| Services | `service.md` | Error handling | `error-handling.md` |
+| Routes | `route.md` | | |
+
+The terse cross-cutting summary in `CLAUDE.md` defers to `cross-cutting.md` — edit the doc, not
+just the summary.
+
+## The domain map (`docs/domain-map.md`)
+
+The source of truth for **blast-radius analysis**: which product surfaces consume which, so a
+change to one surface doesn't silently break a downstream one. `srd-creator` traverses its
+`Consumed by` edges (1-hop then 2-hop) while writing the SRD's **Impact / Blast Radius** section.
+It's a dated snapshot — `srd-creator` reconciles it against live code and flags drift, but refresh
+it as surfaces are added.
+
+## Directory map
+
+```
+.claude/
+├── README.md            ← this file
+├── CLAUDE.md            project context loaded into every session (terse; defers to the docs)
+├── skills/
+│   ├── start-prd/       PRD writer
+│   ├── srd-creator/     SRD writer + blast-radius analysis (+ reference/ template & guide)
+│   ├── backend-conventions/  conventions index/loader
+│   ├── feature-builder/ the build workflow (executes an SRD)
+│   └── grill-me/        stress-test a spec/design
+└── conventions/         cross-cutting + per-layer code conventions (the source of truth)
+docs/
+├── domain-map.md        product surfaces + consumer edges (blast radius)
+└── architecture/        deep-dive architecture docs per subsystem
+features/<slug>/         PRD.md · SRD.md for each feature
+```
+
+## Maintaining the system
+
+- **Convention changed?** Edit the doc in `.claude/conventions/`. Then sync the `/pr-review`
+  checklist, which mirrors these docs as a self-contained review list.
+- **New product surface?** Add it to `docs/domain-map.md` (surface + its consumes/consumed-by
+  edges) so future blast-radius analysis sees it.
+- **New layer or skill?** Add the convention doc, register it in the `backend-conventions` index,
+  and reference it from `feature-builder`.