Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 68 additions & 9 deletions examples/langfuse/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,50 @@ reflection — and exports them over OTLP to any backend, including
[Langfuse](https://langfuse.com). There is **no wrapper and no extra
instrumentation code**: enable it in config and the traces appear.

## Enable
Two ways to look at it:

| | What it is | What you need |
| --- | --- | --- |
| [Replay a recording](#replay-a-recording-no-everos-needed) | A trace a real EverOS server produced, pushed into your Langfuse project | Langfuse keys only |
| [Trace your own server](#trace-your-own-server) | Your EverOS, your data, live | An EverOS server |

## Replay a recording (no EverOS needed)

`recorded_trace.json` is a capture of one real `demo.py` run against EverOS
1.2.1: 237 spans over 60 traces. Eleven conversations are ingested and flushed,
each with its LLM extraction and OME strategies nested underneath; reflection
then consolidates two of them and deprecates what they superseded; and five
questions are asked of the resulting memory, with their recall scores.
`replay.py` pushes it into your own Langfuse project, so you can see what the
integration looks like before deploying anything.

```bash
pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
export LANGFUSE_PUBLIC_KEY="pk-lf-..."
export LANGFUSE_SECRET_KEY="sk-lf-..."
export LANGFUSE_HOST="https://cloud.langfuse.com" # US: https://us.cloud.langfuse.com
python replay.py
```

Then open Langfuse → **Tracing** and filter on the `replay` tag.

Span names, attributes, token usage, parent/child structure and durations are
EverOS's own output, replayed verbatim. Three things are rewritten: trace and
span ids are minted fresh so repeated runs do not collide, timestamps are
shifted so the trace lands at the current time, and root spans carry a `replay`
tag so a recording is never mistaken for live traffic.

Two things in the trace list are not self-explanatory. The short keyword
searches beyond the five questions are `demo.py` waiting for each conversation
to become searchable. And the OME spans outlast the `flush` span they hang
under, because reflection continues after the request returns and re-attaches
to the originating trace through its `traceparent`.

Recall scores are per-method scales: read HYBRID against HYBRID, not against
AGENTIC. Agent cases and skills are not in this recording; the span and score
contract is the same when they appear.

## Trace your own server

1. Install the optional OpenTelemetry extra:

Expand All @@ -29,15 +72,26 @@ instrumentation code**: enable it in config and the traces appear.
Container/CI equivalent via env vars: `EVEROS_OBSERVABILITY__ENABLED=true`,
`EVEROS_OBSERVABILITY__LANGFUSE_PUBLIC_KEY=...`, and so on.

3. Run EverOS normally:
3. Run EverOS normally, then drive one memory lifecycle through it:

```bash
everos server start
python demo.py # add -> flush -> search against 127.0.0.1:8000
```

`demo.py` uses only the standard library and contains no instrumentation
code; the spans come from the server. It ingests eleven conversations, nudges
reflection (a weekly cron otherwise), then asks five questions, so the
traces show recall choosing between memories rather than returning the only
one there is.

Off by default — with `enabled = false` (or the `otel` extra absent) there is
zero tracing overhead.

The signal is plain OTLP/HTTP and vendor-neutral, so the same config exports to
an OpenTelemetry Collector or any other OTLP backend. The `langfuse_*` keys are
just a shortcut that fills in the endpoint and auth header for you.

## What you get

| EverOS operation | Langfuse observation |
Expand All @@ -48,7 +102,8 @@ zero tracing overhead.
| markdown persistence | span `everos.persist.markdown` |
| `POST /api/v2/memory/search` | retriever `everos.memory.search` → `recall` / `rank` |
| query / recall embedding | embedding `everos.embedding` |
| OME reflection strategies | agent `everos.ome.<strategy>` (linked to the triggering request's trace) |
| OME extraction strategies | agent `everos.ome.<strategy>` (linked to the triggering request's trace) |
| reflection consolidating a cluster | span `everos.reflect.consolidate` under `everos.ome.reflect_episodes` |

`langfuse.session.id` / `langfuse.user.id` group the traces. Recall quality is
pushed as Langfuse scores, split by whether the method's score is calibrated:
Expand All @@ -58,17 +113,21 @@ cosine values are on a different scale and must not be averaged in with the
calibrated ones. Query and memory text are captured only when
`capture_content = true`.

## Try it
## Re-recording the fixture

With a server running and `[observability]` enabled:
`record_trace.py` is the maintainer-side tool that produced
`recorded_trace.json`. It stands in for Langfuse's two ingestion endpoints on
localhost, so a real EverOS server exports its spans *and* its recall scores
there instead of to Langfuse. Nothing about the recording is synthesized.

```bash
python demo.py
python record_trace.py # sink on :4318; writes the fixture on Ctrl-C
```

It drives one add → flush → search (keyword / hybrid / agentic) cycle against
`http://127.0.0.1:8000` using only the standard library, then tells you to open
Langfuse → **Tracing** filtered to `session.id = langfuse_demo`.
Point `[observability].langfuse_host` at `http://127.0.0.1:4318`, start the
server, run `demo.py`, then stop the sink. Only worth redoing when the span
contract changes (a span added, renamed, or given new attributes); ordinary
releases do not invalidate a recording.

## Learn more

Expand Down
Loading
Loading