Skip to content

test(sessions): add session/memory multi-backend replay consistency framework#125

Open
coder-mtj wants to merge 10 commits into
trpc-group:mainfrom
coder-mtj:feat/replay-consistency-python
Open

test(sessions): add session/memory multi-backend replay consistency framework#125
coder-mtj wants to merge 10 commits into
trpc-group:mainfrom
coder-mtj:feat/replay-consistency-python

Conversation

@coder-mtj

Copy link
Copy Markdown

Description | 描述

构建 Session/Memory/Summary 多后端回放一致性测试框架。
驱动 InMemory 和 SQLite 后端用同一组确定性 replay case 归一化比较,
输出结构化 diff 报告。Redis 后端通过 REDIS_URL env var 按需启用。

Closes #89

Change Type | 修改类型

  • New feature | 新功能

Files Changed

  • tests/sessions/replay_consistency/ — 模块化框架
    • cases.py — 10 个 ReplayCase 定义 + JSONL load/save
    • normalizer.py — Event/Snapshot 归一化
    • comparator.py — 递归 diff 引擎
    • fixtures/ — 10 个 .jsonl replay fixture 文件
    • test_normalizer.py — 15 个归一化单元测试
    • test_comparator.py — 15 个比较器单元测试
    • test_cases.py — 9 个 case 定义/fixture 测试
  • tests/sessions/test_replay_consistency.py — 主测试(InMemory vs SQLite E2E)
  • tests/sessions/test_replay_redis.py — Redis 后端测试(env-var gated)

Test Coverage | 测试覆盖

  • 379 passed, 3 skipped (Redis not available)
  • 15 normalizer tests — event/snapshot normalization correctness
  • 15 comparator tests — recursive diff on dicts/lists/primitives/nested
  • 9 cases tests — fixture integrity + JSONL round-trip
  • 4 E2E tests — 10-case InMemory vs SQLite comparison
  • 3 Redis tests — gracefully skipped when REDIS_URL not set

Self-test Checklist | 自测清单

  • Verified locally | 本地验证通过
  • No existing features affected | 无影响现有功能
  • Redis gracefully skipped when unavailable

coder-mtj added 4 commits July 6, 2026 08:53
15 tests covering event normalization (author, text, state_delta)
and snapshot normalization (session_id, state, events, memories, summaries).

Signed-off-by: coder-mtj <coder-mtj@users.noreply.github.com>
15 tests covering recursive diff on dicts, lists, primitives,
nested structures, and full session snapshots.

Signed-off-by: coder-mtj <coder-mtj@users.noreply.github.com>
10 replay scenarios defined in cases.py with round-trip
JSONL fixture support. Mirrors Go trpc-agent-go session/replaytest.

Signed-off-by: coder-mtj <coder-mtj@users.noreply.github.com>
- test_replay_consistency.py: InMemory vs SQLite 10-case E2E
- test_replay_redis.py: env-var-gated Redis vs InMemory/SQLite
- 379 passed, 3 skipped (Redis), 0 new failures

Signed-off-by: coder-mtj <coder-mtj@users.noreply.github.com>
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@099b571). Learn more about missing BASE report.

Additional details and impacted files
@@            Coverage Diff             @@
##             main        #125   +/-   ##
==========================================
  Coverage        ?   87.51534%           
==========================================
  Files           ?         467           
  Lines           ?       44006           
  Branches        ?           0           
==========================================
  Hits            ?       38512           
  Misses          ?        5494           
  Partials        ?           0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@raychen911

Copy link
Copy Markdown
Contributor

建议给出一些设计文档,使用说明,方便复现效果,另外也可以提供一下完成这个项目的提示词,如果提示词不方便展示,可以邮件发送我

- design-en.md: English architecture design document
- usage.md: Usage guide with reproduction steps
- ai-prompts.md: AI-assisted development prompts (4 rounds)
- Include existing design.md, test-plan.md, summary.md

Response to reviewer feedback from raychen911.
@coder-mtj

Copy link
Copy Markdown
Author

建议给出一些设计文档,使用说明,方便复现效果,另外也可以提供一下完成这个项目的提示词,如果提示词不方便展示,可以邮件发送我

@raychen911 感谢 review!已补充设计文档和 AI 提示词:

新增文档(docs/issue-89-replay-consistency/)

文件 说明
design.md 中文架构设计文档(模块拆分、归一化策略、JSONL fixture 格式)
design-en.md English architecture design document
usage.md 使用说明 + 快速复现步骤 + 添加新 case 指南
test-plan.md 测试计划(6 维度,379 tests)
summary.md 实现总结
ai-prompts.md AI 辅助开发提示词(4 轮完整对话)

AI 提示词要点

项目架构(10 个 ReplayCase 设计、normalize_event/normalize_snapshot 归一化规则、recursive_diff 算法、JSONL fixture 格式、多后端 gating 策略)均由我亲自设计,AI 执行编码。4 轮对话:

  1. 数据模型+Normalizer+Comparator → AI 实现正常化器和比较器
  2. JSONL Fixtures+Cases → AI 生成 10 个测试场景和 fixture 文件
  3. E2E 集成测试 → AI 实现 InMemory vs SQLite 对比 + Redis env-var gated
  4. 格式修复+验证 → yapf 格式化,379 tests passed

完整提示词见 PR 中的 docs/issue-89-replay-consistency/ai-prompts.md。

@coder-mtj

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

Rook1ex added a commit to trpc-group/cla-database that referenced this pull request Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

构建 Session / Memory 多后端回放一致性测试框架

2 participants