Francis — a self-verifying daily security digest

Francis is a scheduled AI agent that turns a day's worth of SIEM alerts into a short, trustworthy security briefing and delivers it to a chat channel every morning. It is built to do one thing well: summarize only what actually happened, and refuse to invent the rest.

The interesting part is not "LLM writes a summary." It's the two-stage, grounded pipeline that makes the summary safe to act on: a synthesis pass drafts the digest, and a second independent verification pass checks that draft against the raw alert data and flags anything the draft glossed over or got wrong.

---

What it produces

A daily markdown digest, for example:

• alert volume and severity distribution for the previous 24h
• the noisy rules (so routine churn is named and dismissed, not hidden)
• every critical alert (high severity) restated factually from its log line
• a configuration-drift delta from the host hardening / compliance scans: which checks newly failed and which flipped state during the day

If the verifier finds the draft omitted or misstated something, the digest is delivered with a clearly marked "⚠️ Verifier flagged" block at the top, so the operator sees both the discrepancy and the original draft.

---

Architecture

  ┌─────────────┐   nightly    ┌──────────────────────┐
  │ systemd      │  timer fires │ Wazuh exporter (py)  │
  │ timer        ├─────────────►│ reads yesterday's    │
  └─────────────┘              │ alerts, summarizes    │
                                │ by level/rule/agent,  │
                                │ extracts criticals +  │
                                │ an SCA (CIS) delta    │
                                └───────────┬──────────┘
                                            │ writes one JSON summary
                                            ▼
  ┌──────────────────────────────────────────────────────────────┐
  │ n8n workflow (cron, ~15 min later)                             │
  │                                                                │
  │  read summary ─► build prompt ─► [SYNTHESIS]  Claude drafts    │
  │                                       │        the digest      │
  │                                       ▼                         │
  │                  build verifier req ─► [VERIFY] Claude checks   │
  │                                       │   draft vs. raw data    │
  │                                       ▼                         │
  │                  apply verifier ─► format ─► deliver to chat    │
  └──────────────────────────────────────────────────────────────┘

Stage 1 — Synthesis. A persona-scoped prompt ("terse security analyst") receives the structured summary and drafts the digest. The system prompt enforces hard grounding rules: every claim must come from the supplied data; no cross-source inference; quote rule IDs / agent names / IP:port verbatim; say "investigate" rather than guess a root cause.

Stage 2 — Grounded verification. A separate request hands the draft plus the raw rule/level/agent breakdown to the model acting as a verifier. It applies ordered, threshold-based rules — e.g. any rule at severity ≥ 10 with a hit that the draft did not name explicitly is flagged; auth-anomaly rules (failed login, invalid user, brute force) above a small threshold must be named, not silently bucketed; any per-agent count in the draft that contradicts the raw data is flagged. The verifier returns OK or a terse bulleted list of discrepancies. Only then is the digest formatted and sent.

This split is deliberate: the synthesis pass optimizes for readability, and the verification pass optimizes for faithfulness to the data. Separating them keeps a single model from grading its own homework in the same breath it writes it.

---

Components

Component	Role
francis-export-wazuh.py	Reads the SIEM's daily alert archive, summarizes by level/rule/agent, captures full records for critical alerts, and computes an SCA (CIS hardening) delta — checks ending failed + same-day pass/fail transitions. Emits one JSON summary.
francis-export-wazuh.service / .timer	systemd oneshot + timer that runs the exporter shortly before the digest is built.
workflow.json	The n8n workflow: read summary → build prompt → synthesis → build verifier request → verification → apply verifier → format → deliver.

The LLM calls go to the Anthropic Messages API (Claude Sonnet). API credentials and the destination chat ID are supplied via the workflow's credential store and environment — none are committed here.

---

Engineering notes

• Anti-hallucination by construction. The exporter attaches the observed severity level to every rule in the summary. An earlier version omitted it, and the verifier inferred severity from a rule's description text — flagging a low-severity rule as critical because its name contained the word "EXPLOIT." Carrying the real level removed that whole class of false flags.
• Signal vs. noise in compliance scans. Some hardening checks flap between pass/fail purely as a scan-resolution artifact (e.g. sshd -T rendering of UsePAM / MACs / PermitUserEnvironment). The prompt explicitly teaches the model to distinguish these known flaps from real configuration drift, so the digest doesn't cry wolf every morning.
• Fail-soft delivery. If the verifier disagrees with the draft, the digest still goes out — with the discrepancies surfaced on top — rather than being suppressed. A flagged digest is more useful than a missing one.

---

Running it

The exporter is path-configurable via environment variables:

export WAZUH_ALERTS_DIR=/var/ossec/logs/alerts      # SIEM alert archive
export FRANCIS_OUT_FILE=/data/francis/wazuh-daily.json
./francis-export-wazuh.py

Install the systemd units, point the n8n workflow's "read summary" node at the same output file, set an Anthropic API credential and a destination chat ID, and schedule the workflow a few minutes after the timer.

---

Stack

Python 3 · systemd · n8n · Anthropic Claude (Messages API) · Wazuh SIEM (alert source). No external dependencies beyond the Python standard library in the exporter.