Redis LangCache Demo

Semantic caching for LLM responses with Redis LangCache + OpenAI, featuring scoped isolation by Company / Business Unit / Person and a guided, presentation-ready web UI.

On a cache hit the answer comes straight from LangCache in tens of milliseconds, skipping the LLM call entirely: lower latency, lower token spend.

Presenting to an audience? Follow DEMO_SCRIPT.md, a step-by-step walkthrough of the 5 guided steps. Portuguese versions of the docs exist too: README_PT-BR.md and roteiro_demo.md.

Quickstart (5 minutes)

1. Clone the repo:

   git clone https://github.com/Redislabs-Solution-Architects/redis-langcache-python-example.git
   cd redis-langcache-python-example

2. Create a .env from .env.EXAMPLE and fill it in:

   cp .env.EXAMPLE .env

   Variable	Where to get it
   LANGCACHE_BASE_URL	Your LangCache service on Redis Cloud
   LANGCACHE_CACHE_ID	Same LangCache service page
   LANGCACHE_SERVICE_KEY	Same LangCache service page (or use LANGCACHE_API_KEY)
   OPENAI_API_KEY	Your OpenAI account
   APP_PASSWORD	Pick one; it is the UI login password (default secret42)

3. Run it, either way:

   docker compose up -d --build

   or

   pip install -r requirements.txt && python app.py

4. Open http://localhost:7860 and log in with your APP_PASSWORD.

What the demo shows

Feature	How it works
Semantic cache hits	LangCache search with similarity threshold and EXACT to SEMANTIC fallback
Scoped isolation	LangCache attributes (company, business_unit, person), switchable live in the UI
Identity handling	Role stored under an exact key per scope ("My role is X", PT-BR forms also accepted); the user's name is never cached
Domain disambiguation	Ambiguous prompts ("deploy", "pipeline") are rewritten with the domain inferred from the persona's BU
Cross-language reuse	A question in Portuguese hits the English cached answer at company scope
Observability	Per-message trace (search / LLM / set latencies), similarity meter, search strategy, global counters

The right rail contains a clickable demo script: 5 steps that walk through person-scoped identity, BU-scoped context, company-wide memory and the multilingual cache hit. See DEMO_SCRIPT.md for the full narration.

Architecture

static/  (vanilla HTML/CSS/JS, no build step)
   |
   v
app.py  (FastAPI)  ----->  LangCache (Redis Cloud)   # search / set / delete_query
   |                            ^
   +----->  OpenAI  ------------+                    # only on cache miss, answer stored back

Path	Purpose
app.py	FastAPI backend: UI + JSON API
static/	Web UI (Redis brand theme, chat A/B, guided script, observability panel)
DEMO_SCRIPT.md	Presentation-ready walkthrough of the 5 guided steps
deploy_gce.sh	Reference deploy script (Docker build on a GCE VM behind Caddy)
legacy/	Earlier standalone examples (CLI, Gradio UIs, RAG), see legacy/README.md

API

Endpoint	Description
POST /api/login	Password gate, sets the session cookie
POST /api/chat	Cache-first answer with full telemetry (source, similarity, strategy, latencies, trace)
POST /api/flush	Delete cache entries by scope attributes (the index is never dropped)
GET /api/meta	Service metadata + global counters
GET /healthz	Liveness probe (no auth)

Prerequisites

1. A LangCache service on Redis Cloud: you need the base URL, cache ID and service key.
2. An OpenAI API key.

Configuration

Create a .env in the repo root (see .env.EXAMPLE):

OPENAI_API_KEY=sk-proj-...
OPENAI_MODEL=gpt-5.4-mini

LANGCACHE_SERVICE_KEY=...        # or LANGCACHE_API_KEY
LANGCACHE_CACHE_ID=...
LANGCACHE_BASE_URL=https://aws-us-east-1.langcache.redis.io

APP_PASSWORD=secret42            # UI password (change it)

The server listens on port 7860 by default (override with PORT).

Run

Local:

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
python app.py                    # http://localhost:7860

Docker:

docker build -t langcache-demo .
docker run --env-file .env -p 7860:7860 langcache-demo

Docker Compose:

docker compose up -d --build

Demo walkthrough

Use the guided script in the right rail, in order (full narration in DEMO_SCRIPT.md):

1. Identity per person: Gabs and Diego each store their role ("My role is Software Engineer." / "My role is Financial Analyst.").
2. Cache per person: "What is my role at the company?" returns each person's own role (exact hit, 100% similarity).
3. BU context: "What does the term deploy mean?" returns software deployment for engineering and capital allocation for finance.
4. Company memory: switch the scope card to Company and cache one English answer for everyone ("Explain what machine learning is.").
5. Cross-language hit: ask "O que é aprendizado de máquina?" and watch the semantic hit on the English answer cached in the previous step.

Useful controls:

• Scope cards decide who shares cached answers (Person / BU / Company / Global).
• The trash icon on each chat clears only that scope's entries.
• Advanced settings (collapsed): similarity threshold, TTL, token pricing for the savings estimate.

Notes

• The session cookie rotates when the server restarts; just log in again.
• Cost savings are an estimate: chars/4 tokens priced with the configurable per-1M rates.
• Cache writes always store the neutral (depersonalized) answer, so entries are safe to share across scopes.