diff --git a/.gitignore b/.gitignore
index 8a45ae5c..59782e89 100644
--- a/.gitignore
+++ b/.gitignore
@@ -16,6 +16,7 @@ benchmarks/Astrodynamics/MannedLunarLanding/results.txt
benchmarks/Astrodynamics/MannedLunarLanding/outputlog.txt
.vscode
*.csv
+!leaderboard/*.csv
*.h5ad
*.h5ad.tmp
*.h5ad.tmp.*
@@ -46,6 +47,7 @@ outputlog.txt
frontier_eval/conf/batch/*
!frontier_eval/conf/batch/example_matrix.yaml
!frontier_eval/conf/batch/v1.yaml
+!frontier_eval/conf/batch/v1_lite.yaml
metrics.json
artifacts.json
debug-*.log
@@ -54,3 +56,6 @@ debug-*.log
**/outputs/
**/artifacts/
**/last_eval.json
+
+# IndexOptimization generated data
+benchmarks/ComputerSystems/IndexOptimization/data/tpch_sf1/tpch_sf1.dump
diff --git a/README.md b/README.md
index 0fdf60bf..98acef96 100644
--- a/README.md
+++ b/README.md
@@ -14,6 +14,10 @@ Frontier-Eng is a benchmark for **generative optimization**: agents iteratively
The benchmark currently covers **47 tasks** across computing, quantum information, operations research, robotics and control, optics and communications, and physical sciences. The project homepage and paper frame it as a missing evaluation axis between pass/fail coding benchmarks and real engineering work: most engineering problems start from a feasible baseline and reward iterative improvement, not one-shot correctness.
+## News
+
+- **2026-06-30** — **New scoring metric: the Medal Score (gold/silver/bronze).** Alongside average rank, we now release a peer-relative *Medal Score* (normalized to `[0,1]`). On each task the top-3 best-feasible scores in the v1 snapshot are frozen as gold/silver/bronze baselines; a model earns 1.00 / 0.67 / 0.33 for reaching each, averaged over the task set, and is reported on both v1 (47 tasks) and v1-lite (10 tasks). It rewards only reaching each task's frontier and ignores negligible long-tail margins, making cross-task aggregation fairer. Per-task podium values and the leaderboard live in [`leaderboard/`](leaderboard/README.md).
+- **2026-06-30** — **`v1-lite` released.** A 10-task representative subset of `v1` covering all five categories with distinct benchmark families, selected for tasks whose scores climb gradually under budget (not one-shot-saturated or all-or-nothing). Run it with `frontier_eval/conf/batch/v1_lite.yaml`.
## 0. Host Requirements
@@ -97,6 +101,14 @@ bash scripts/batch/validate_v1_task_envs.sh
That command runs the batch config for the `v1` problem set with `algorithm.iterations=0`, which evaluates each task's shipped baseline instead of asking an LLM to improve it.
+### `v1-lite` quick subset
+
+For fast iteration and ablations, use the 10-task `v1-lite` matrix
+([`frontier_eval/conf/batch/v1_lite.yaml`](frontier_eval/conf/batch/v1_lite.yaml))
+instead of the full `v1` config. It spans all five categories with distinct
+benchmark families and favors tasks whose scores improve gradually under budget,
+so a short run still exercises the full optimization loop.
+
If you want the full `v1` problem set with normal optimization runs later, see [`run.md`](run.md).
## Where To Go Next
@@ -108,18 +120,20 @@ If you want the full `v1` problem set with normal optimization runs later, see [
## Leaderboard
-Detailed leaderboard: [lab.einsia.ai/frontier-eng/leaderboard.html](https://lab.einsia.ai/frontier-eng/leaderboard.html)
-
-| Rank | Model | Average Rank |
-| :--: | :--- | --: |
-| 1 | GPT-5.4 | 3.54 |
-| 2 | Claude Opus 4.6 | 3.63 |
-| 3 | GLM-5 | 4.34 |
-| 4 | DeepSeek V3.2 | 4.76 |
-| 5 | Gemini 3.1 Pro Preview | 5.53 |
-| 6 | Grok 4.20 | 5.82 |
-| 7 | SEED 2.0 Pro | 5.86 |
-| 8 | Qwen3 Coder Next | 6.71 |
+Detailed leaderboard (incl. average rank): [lab.einsia.ai/frontier-eng/leaderboard](https://lab.einsia.ai/frontier-eng/leaderboard). Released score tables and the per-task medal podium: [`leaderboard/`](leaderboard/README.md).
+
+**Medal Score** (gold/silver/bronze podium, higher is better, normalized to `[0,1]` = mean per-task podium credit). On each task the top-3 best scores in the **v1 snapshot (2026-04-14)** are frozen as gold/silver/bronze baselines; a model earns 1.00 / 0.67 / 0.33 for reaching each. Reported on both the full **v1** set (47 tasks) and the **v1-lite** subset (10 tasks); gold/silver/bronze counts are for v1 (see [`leaderboard/`](leaderboard/README.md)):
+
+| Rank | Model | Medal (v1) | Medal (v1-lite) | 🥇 | 🥈 | 🥉 |
+| :--: | :--- | --: | --: | --: | --: | --: |
+| 1 | GPT-5.4 | 0.596 | 0.667 | 24 | 5 | 2 |
+| 2 | Claude Opus 4.6 | 0.490 | 0.501 | 9 | 18 | 6 |
+| 3 | GLM-5 | 0.312 | 0.233 | 4 | 10 | 12 |
+| 4 | DeepSeek V3.2 | 0.248 | 0.166 | 3 | 9 | 8 |
+| 5 | Gemini 3.1 Pro Preview | 0.213 | 0.200 | 3 | 6 | 9 |
+| 6 | Seed 2.0 Pro | 0.185 | 0.100 | 3 | 7 | 3 |
+| 7 | Grok 4.20 | 0.184 | 0.133 | 3 | 6 | 5 |
+| 8 | Qwen3 Coder Next | 0.121 | 0.000 | 3 | 3 | 2 |
## Contributing
diff --git a/README_zh-CN.md b/README_zh-CN.md
index 75ddd213..26e66e5b 100644
--- a/README_zh-CN.md
+++ b/README_zh-CN.md
@@ -8,6 +8,11 @@ Frontier-Eng 是一个面向 **generative optimization** 的 benchmark：Agent
当前版本包含 **47 个任务**，覆盖计算系统、量子信息、运筹优化、机器人控制、光学通信、物理与工程设计。主页和论文的核心观点是：真实工程问题通常从一个可行 baseline 出发，价值来自持续优化，而不是 pass/fail。
+## News
+
+- **2026-06-30** — **新增评测指标：金银铜 Medal Score。** 在 average rank 之外，我们发布同侪相对的 *Medal Score*（归一化到 `[0,1]`）：每道题取 v1 snapshot 中最好的前三名分数冻结为金/银/铜 baseline，模型达到金/银/铜分别得 1.00 / 0.67 / 0.33，对题集求均值；同时汇报 v1（47 题）与 v1-lite（10 题）。它只奖励"达到该题最前沿（领奖台）"，忽略长尾里可忽略的微小差距，使跨题汇总更公平。每题 podium 分数与榜单见 [`leaderboard/`](leaderboard/README.md)。
+- **2026-06-30** — **发布 `v1-lite`。** `v1` 的 10 题代表性子集，覆盖全部五大类、family 各不相同，专选"分数随预算逐步提升（而非一步做满或非高即低）"的题，配置见 `frontier_eval/conf/batch/v1_lite.yaml`。
+
## 这个 benchmark 在测什么
和传统 agent benchmark 相比，Frontier-Eng 更关注三件事：
@@ -110,18 +115,20 @@ bash scripts/batch/validate_v1_task_envs.sh
## Leaderboard
-详细榜单见 [lab.einsia.ai/frontier-eng/leaderboard.html](https://lab.einsia.ai/frontier-eng/leaderboard.html)。
-
-| 排名 | Model | Average Rank |
-| :--: | :--- | --: |
-| 1 | Claude Opus 4.6 | 3.18 |
-| 2 | GLM-5 | 4.02 |
-| 3 | DeepSeek V3.2 | 4.41 |
-| 4 | Gemini 3.1 Pro Preview | 5.34 |
-| 5 | Grok 4.20 | 5.60 |
-| 6 | SEED 2.0 Pro | 5.63 |
-| 7 | GPT-5.4 | 5.68 |
-| 8 | Qwen3 Coder Next | 6.68 |
+详细榜单（含 average rank）见 [lab.einsia.ai/frontier-eng/leaderboard](https://lab.einsia.ai/frontier-eng/leaderboard)。发布的分数表与每题金银铜 podium 见 [`leaderboard/`](leaderboard/README.md)。
+
+**Medal Score**（金银铜 podium，越高越好，归一化到 `[0,1]`，即每题领奖台得分的均值）。每题取 **v1 snapshot (2026-04-14)** 的前三名分数冻结为金/银/铜 baseline，模型达到金/银/铜分别得 1.00 / 0.67 / 0.33。同时汇报 **v1**（47 题）与 **v1-lite**（10 题）两个集合；金银铜次数为 v1（`gpt-5.4` 采用其 47 题全量重测结果）：
+
+| 排名 | Model | Medal (v1) | Medal (v1-lite) | 🥇 | 🥈 | 🥉 |
+| :--: | :--- | --: | --: | --: | --: | --: |
+| 1 | GPT-5.4 | 0.596 | 0.667 | 24 | 5 | 2 |
+| 2 | Claude Opus 4.6 | 0.490 | 0.501 | 9 | 18 | 6 |
+| 3 | GLM-5 | 0.312 | 0.233 | 4 | 10 | 12 |
+| 4 | DeepSeek V3.2 | 0.248 | 0.166 | 3 | 9 | 8 |
+| 5 | Gemini 3.1 Pro Preview | 0.213 | 0.200 | 3 | 6 | 9 |
+| 6 | Seed 2.0 Pro | 0.185 | 0.100 | 3 | 7 | 3 |
+| 7 | Grok 4.20 | 0.184 | 0.133 | 3 | 6 | 5 |
+| 8 | Qwen3 Coder Next | 0.121 | 0.000 | 3 | 3 | 2 |
## 贡献
diff --git a/TASK_DETAILS.md b/TASK_DETAILS.md
index b331f7d8..c89789af 100644
--- a/TASK_DETAILS.md
+++ b/TASK_DETAILS.md
@@ -204,7 +204,7 @@ We welcome new engineering problem ideas — even without complete verification
| Polarization-multiplexed holography |
- | ComputerSystems |
+ ComputerSystems |
MallocLab |
High-performance C memory allocator (utilization & throughput) |
@@ -212,6 +212,10 @@ We welcome new engineering problem ideas — even without complete verification
DuckDBWorkloadOptimization |
Index / materialized-view selection and query rewriting on official DuckDB workloads |
+
+ CacheReplacement |
+ CPU cache replacement policy optimization via ChampSim on SPEC CPU traces |
+
| EngDesign |
CY_03, WJ_01, XY_05, AM_02, AM_03, YJ_02, YJ_03 |
diff --git a/benchmarks/ComputerSystems/CacheReplacement/.gitignore b/benchmarks/ComputerSystems/CacheReplacement/.gitignore
new file mode 100644
index 00000000..4a14c86f
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/.gitignore
@@ -0,0 +1,9 @@
+data/traces/
+data/ChampSim/
+data/baseline_ipcs/
+references/generated_baseline.json
+__pycache__/
+*.pyc
+*.log
+temp/
+.DS_Store
diff --git a/benchmarks/ComputerSystems/CacheReplacement/README.md b/benchmarks/ComputerSystems/CacheReplacement/README.md
new file mode 100644
index 00000000..522a60d1
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/README.md
@@ -0,0 +1,284 @@
+# CacheReplacement — CPU Cache Replacement Policy Optimization
+
+An AI agent designs a CPU cache replacement policy by modifying C++ code. The policy is compiled into the [ChampSim](https://github.com/ChampSim/ChampSim) simulator and evaluated on SPEC CPU 2017 memory access traces. The agent's goal is to outperform the classic LRU baseline.
+
+| Quick Links | |
+|---|---|
+| Task specification | [Task.md](Task.md) / [Task_zh-CN.md](Task_zh-CN.md) |
+| Engineering domain | Computer Architecture — microarchitectural optimization |
+| Existing benchmarks comparison | MallocLab (OS), DuckDB (Database) → **CacheReplacement (Architecture)** |
+| Competition background | [Cache Replacement Championship (CRC-2, ISCA 2017)](https://crc2.ece.tamu.edu/) |
+
+---
+
+## Architecture Pipeline
+
+```
+ Agent edits replacement/my_policy.cc (within EVOLVE-BLOCK markers)
+ │
+ ▼
+ ┌──────────────────────────────────────────────────────┐
+ │ Layer 1: Source Validation │
+ │ · Best-effort static scan │
+ │ · Forbidden headers: , , │
+ │ · Forbidden APIs: fopen, mmap, system, fork │
+ │ · No randomness: rand(), mt19937, time() │
+ │ · No dynamic memory: new, delete, malloc, free │
+ │ · Storage budget ≤ 64 KB (approximate check) │
+ │ · EVOLVE-BLOCK integrity check │
+ │ FAIL → valid=0, score=0 │
+ └──────────────────────┬───────────────────────────────┘
+ │ PASS
+ ▼
+ ┌──────────────────────────────────────────────────────┐
+ │ Layer 2: Compile into ChampSim │
+ │ · config.sh + make -j4 (incremental build) │
+ │ · Environment: g++-13, C++17, -O3 -fno-exceptions │
+ │ · Timeout: 300s │
+ │ FAIL → valid=0, score=0 │
+ └──────────────────────┬───────────────────────────────┘
+ │ PASS
+ ▼
+ ┌──────────────────────────────────────────────────────┐
+ │ Layer 3: Quick Verify │
+ │ · Run on verify_trace.xz (10M warmup + 20M sim) │
+ │ · Checks: no crash, valid IPC value │
+ │ · Purpose: fast rejection of broken policies │
+ │ · NOT used for scoring │
+ │ · Timeout: 120s │
+ │ FAIL → valid=0, score=0 │
+ └──────────────────────┬───────────────────────────────┘
+ │ PASS
+ ▼
+ ┌──────────────────────────────────────────────────────┐
+ │ Layer 4: Full Evaluation │
+ │ · Quick mode: 3 traces, ~15 min │
+ │ · Full mode: 10 traces, ~30-60 min │
+ │ · Per-trace: 50M warmup + 100M simulation │
+ │ · IPC extracted from ChampSim stdout │
+ │ · score = GMEAN(IPC_candidate / IPC_LRU) │
+ │ · Failed traces are skipped (remaining continue) │
+ └──────────────────────┬───────────────────────────────┘
+ │
+ ▼
+ Output: {valid, score, per_trace, error}
+```
+
+---
+
+## Scoring
+
+```
+score = geometric_mean( IPC_candidate[t] / IPC_LRU[t] ) over all completed traces t
+
+ score = 1.0 → parity with LRU baseline
+ score > 1.0 → better than LRU
+ score ∈ (0,1) → worse than LRU but functionally valid
+ score = 0 → invalid (safety/compile/runtime failure)
+```
+
+Baseline LRU vs LRU always gives score = 1.0. Score > 1.0 means the agent discovered measurable improvement over a decades-old heuristic.
+
+### Metrics output (metrics.json)
+
+```json
+{
+ "valid": true,
+ "score": 1.15,
+ "benchmark_version": "1.0.0",
+ "mode": "full",
+ "compile_time_s": 7.8,
+ "simulation_time_s": 914.7,
+ "num_traces_completed": 10,
+ "error": null
+}
+```
+
+### On failure
+
+```json
+{
+ "valid": false,
+ "score": 0.0,
+ "error": "Static validation: Forbidden header: "
+}
+```
+
+---
+
+## Evaluation Modes
+
+| Mode | Traces | Warmup | Simulation | Expected runtime | Use case |
+|---|---|---|---|---|---|
+| **Quick** (`--mode quick`) | 3 (mcf, x264, cactuBSSN) | 10M | 20M | ~15 min | Agent iteration |
+| **Full** (`--mode full`) | All 10 | 50M | 100M | ~30-60 min | Final scoring |
+
+---
+
+## File Structure
+
+```
+CacheReplacement/
+├── replacement/
+│ ├── my_policy.h # Class declaration (EVOLVE-BLOCK for member vars)
+│ └── my_policy.cc # Method implementation (EVOLVE-BLOCK for methods) ← Agent target
+├── verification/
+│ ├── evaluator.py # 4-layer evaluation pipeline
+│ ├── test_validation.py # 10 Layer-1 validation tests
+│ ├── Dockerfile # Ubuntu 22.04, gcc-13, cmake 3.22
+│ └── requirements.txt # pyyaml
+├── baseline/
+│ ├── lru.cc # LRU reference (official baseline)
+│ ├── random.cc # Random policy (sanity check only, score < 1.0)
+│ ├── metrics.json # Local test results
+│ └── result_log.txt # Test environment & results
+├── frontier_eval/ # Frontier-Eng integration metadata
+├── references/
+│ ├── constants.json # Parameter definitions
+│ ├── problem_config.json # Trace lists, timeout config
+│ ├── generated_baseline.json # Auto-generated LRU IPC cache
+│ └── related_work.md # Academic references
+└── data/ # Auto-downloaded (NOT in git)
+ ├── download_traces.sh # Automated download script
+ ├── checksums.txt # SHA256 checksums
+ ├── traces/ # 10 SPEC CPU 2017 .xz traces
+ └── ChampSim/ # Simulator source (commit 51588e1d)
+```
+
+---
+
+## EVOLVE-BLOCK Contract Summary
+
+The agent modifies **two files**, each with `// EVOLVE-BLOCK-START` / `// EVOLVE-BLOCK-END` markers:
+
+### `replacement/my_policy.h` — Member variables (within EVOLVE-BLOCK)
+
+```cpp
+// EVOLVE-BLOCK-START
+// Agent may add fixed-size arrays here:
+uint8_t history_table[1024]; // 1 KB
+uint16_t access_counters[64]; // 128 bytes
+// Total ≤ 64 KB (checked by evaluator)
+// EVOLVE-BLOCK-END
+```
+
+### `replacement/my_policy.cc` — Method implementations (within EVOLVE-BLOCK)
+
+```cpp
+// EVOLVE-BLOCK-START
+// Agent may modify:
+// - find_victim() body → eviction decision logic
+// - replacement_cache_fill() → fill handler
+// - update_replacement_state() → state update
+//
+// Agent must NOT modify:
+// - function signatures
+// - code outside EVOLVE-BLOCK
+// EVOLVE-BLOCK-END
+```
+
+### Forbidden (causes immediate rejection with valid=0)
+
+| Category | Examples | Detection |
+|---|---|---|
+| **Headers** | ``, ``, ``, ``, ``, ``, `` | Header scan |
+| **File I/O** | `fopen`, `fread`, `fwrite`, `open(`, `read(`, `write(`, `mmap(`, `munmap(` | API scan |
+| **System** | `system(`, `popen`, `fork` | API scan |
+| **Threading** | `pthread_`, `std::thread`, `std::async` | API scan |
+| **Dynamic memory** | `new`, `delete`, `malloc(`, `free(`, `calloc(`, `realloc(` | Regex scan |
+| **Dynamic containers** | `std::vector`, `std::map`, `std::unordered_map`, `std::set`, `std::list`, `std::deque` | Container scan |
+| **Non-determinism** | `rand()`, `srand(`, `random_device`, `mt19937`, `time(`, `clock(` | Function scan |
+| **Exceptions/RTTI** | `throw`, `try`, `catch`, `dynamic_cast`, `typeid` | Keyword scan |
+| **Storage overflow** | Total declared arrays > 64 KB | Static size sum |
+| **Interface violation** | Missing EVOLVE-BLOCK markers, modified read-only code | Diff check |
+
+---
+
+## Setup
+
+### Prerequisites
+
+- **Python 3.10+**
+- **C++17 capable compiler** (g++-12+ or clang)
+- **CMake ≥ 3.22**
+- **~10 GB free disk space** (for trace download + ChampSim build)
+
+### Step-by-step
+
+```bash
+# 1. Clone ChampSim simulator
+git clone https://github.com/ChampSim/ChampSim.git data/ChampSim
+cd data/ChampSim
+git checkout 51588e1d6f97875fe8de1a3621d28668bff83fcf
+git submodule update --init
+./vcpkg/bootstrap-vcpkg.sh
+./vcpkg/vcpkg install
+cd ../..
+
+# 2. Download SPEC CPU 2017 traces
+bash data/download_traces.sh
+
+# 3. Install Python dependencies
+pip install -r verification/requirements.txt
+
+# 4. Run baseline test (quick mode)
+python verification/evaluator.py replacement/my_policy.cc --mode quick
+```
+
+### Docker
+
+```bash
+docker build -t champsim-eval -f verification/Dockerfile .
+docker run --rm -v $(pwd):/benchmark champsim-eval
+```
+
+---
+
+## Validation Tests
+
+```bash
+python verification/test_validation.py
+```
+
+Expected output: all 10 tests pass:
+
+```
+ [PASS] 1. Normal LRU policy: clean
+ [PASS] 2. Forbidden header : Forbidden header:
+ [PASS] 3. rand() usage: Non-deterministic function: rand()
+ [PASS] 4. fopen() file I/O: Forbidden API: fopen
+ [PASS] 5. std::vector in EVOLVE-BLOCK: Dynamic container: std::vector
+ [PASS] 6. new operator: Dynamic memory: ...
+ [PASS] 7. mt19937 randomness: Non-deterministic function: mt19937
+ [PASS] 8. system() call: Forbidden API: system(
+ [PASS] 9. Missing EVOLVE-BLOCK-END: Missing '// EVOLVE-BLOCK-END' in .cc file
+ [PASS] 10. Budget exceeded: Storage budget exceeded: ~100000 B > 65536 B
+```
+
+---
+
+## Integration with Frontier-Eng
+
+```bash
+python -m frontier_eval task=unified task.benchmark=ComputerSystems/CacheReplacement algorithm.iterations=0
+```
+
+---
+
+## Differences from existing ComputerSystems benchmarks
+
+| Benchmark | Engineering domain | Agent's task | Language |
+|---|---|---|---|
+| `MallocLab` | OS / Memory management | Implement malloc/free/realloc | C |
+| `DuckDBWorkloadOptimization` | Database / Query optimization | Select indexes + rewrite SQL | Python |
+| **`CacheReplacement`** | **Computer architecture** | **Design CPU cache eviction policy** | **C++** |
+
+---
+
+## References
+
+- Cache Replacement Championship (CRC-2, ISCA 2017): https://crc2.ece.tamu.edu/
+- Data Prefetching Championship (DPC-3, ISCA 2019): https://dpc3.compas.cs.stonybrook.edu/
+- ChampSim simulator: https://github.com/ChampSim/ChampSim
+- SPEC CPU 2017 traces: https://dpc3.compas.cs.stonybrook.edu/champsim-traces/speccpu/
+- Related work: [references/related_work.md](references/related_work.md)
diff --git a/benchmarks/ComputerSystems/CacheReplacement/README_zh-CN.md b/benchmarks/ComputerSystems/CacheReplacement/README_zh-CN.md
new file mode 100644
index 00000000..d83a23ec
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/README_zh-CN.md
@@ -0,0 +1,202 @@
+# CacheReplacement — CPU 缓存替换策略优化
+
+AI agent 通过修改 C++ 代码设计 CPU 缓存替换策略，编译到 [ChampSim](https://github.com/ChampSim/ChampSim) 模拟器中，在 SPEC CPU 2017 内存访问 trace 上进行评测。目标是超越经典的 LRU 基线。
+
+| 快速链接 | |
+|---|---|
+| 任务说明 | [Task_zh-CN.md](Task_zh-CN.md) / [Task.md](Task.md) |
+| 工程领域 | 计算机体系结构 — 微架构优化 |
+| 对比已有 benchmark | MallocLab (OS)、DuckDB (数据库) → **CacheReplacement (体系结构)** |
+| 竞赛背景 | [Cache Replacement Championship (CRC-2, ISCA 2017)](https://crc2.ece.tamu.edu/) |
+
+---
+
+## 评测管线
+
+```
+ Agent 修改 replacement/my_policy.cc（EVOLVE-BLOCK 标记内）
+ │
+ ▼
+ ┌──────────────────────────────────────────────────────┐
+ │ Layer 1: 源码验证 │
+ │ · 最佳努力静态扫描 │
+ │ · 禁止头文件：, , │
+ │ · 禁止API：fopen, mmap, system, fork │
+ │ · 禁止随机：rand(), mt19937, time() │
+ │ · 禁止动态内存：new, delete, malloc, free │
+ │ · 存储预算 ≤ 64 KB（近似检查） │
+ │ · EVOLVE-BLOCK 完整性检查 │
+ │ 失败 → valid=0, score=0 │
+ └──────────────────────┬───────────────────────────────┘
+ │ 通过
+ ▼
+ ┌──────────────────────────────────────────────────────┐
+ │ Layer 2: 编译到 ChampSim │
+ │ · config.sh + make -j4（增量编译） │
+ │ · 环境：g++-13, C++17, -O3 -fno-exceptions │
+ │ · 超时：300s │
+ │ 失败 → valid=0, score=0 │
+ └──────────────────────┬───────────────────────────────┘
+ │ 通过
+ ▼
+ ┌──────────────────────────────────────────────────────┐
+ │ Layer 3: 快速验证 │
+ │ · 运行 verify_trace（10M warmup + 20M sim） │
+ │ · 检查：不崩溃、IPC合法 │
+ │ · 目的：快速拒绝无效策略，不参与评分 │
+ │ · 超时：120s │
+ │ 失败 → valid=0, score=0 │
+ └──────────────────────┬───────────────────────────────┘
+ │ 通过
+ ▼
+ ┌──────────────────────────────────────────────────────┐
+ │ Layer 4: 正式评测 │
+ │ · Quick 模式：3 traces，~15 min │
+ │ · Full 模式：10 traces，~30-60 min │
+ │ · 每个 trace：50M warmup + 100M simulation │
+ │ · 从 ChampSim 输出提取 IPC │
+ │ · score = GMEAN(IPC_candidate / IPC_LRU) │
+ │ · 失败的 trace 跳过，剩余继续 │
+ └──────────────────────┬───────────────────────────────┘
+ │
+ ▼
+ 输出：{valid, score, per_trace, error}
+```
+
+---
+
+## 评分
+
+```
+score = geometric_mean( IPC_candidate[t] / IPC_LRU[t] )
+
+ score = 1.0 → 等价于 LRU 基线
+ score > 1.0 → 优于 LRU
+ score ∈ (0,1) → 差于 LRU 但功能有效
+ score = 0 → 无效（安全/编译/运行时失败）
+```
+
+### metrics.json 输出示例
+
+```json
+{
+ "valid": true,
+ "score": 1.15,
+ "benchmark_version": "1.0.0",
+ "mode": "full",
+ "num_traces_completed": 10
+}
+```
+
+失败时：
+
+```json
+{
+ "valid": false,
+ "score": 0.0,
+ "error": "Static validation: Forbidden header: "
+}
+```
+
+---
+
+## 评测模式
+
+| 模式 | Traces | Warmup | Simulation | 预期时间 | 用途 |
+|---|---|---|---|---|---|
+| **Quick** (`--mode quick`) | 3 (mcf, x264, cactuBSSN) | 10M | 20M | ~15 min | Agent 迭代 |
+| **Full** (`--mode full`) | 全部 10 个 | 50M | 100M | ~30-60 min | 最终评分 |
+
+---
+
+## EVOLVE-BLOCK 契约
+
+Agent 修改 **两个文件**，每个都有 `// EVOLVE-BLOCK-START` / `// EVOLVE-BLOCK-END` 标记：
+
+### `replacement/my_policy.h` — 成员变量
+
+```cpp
+// EVOLVE-BLOCK-START
+uint8_t history_table[1024]; // 1 KB
+uint16_t counters[64]; // 128 bytes
+// 总计 ≤ 64 KB
+// EVOLVE-BLOCK-END
+```
+
+### `replacement/my_policy.cc` — 方法实现
+
+```cpp
+// EVOLVE-BLOCK-START
+// 可修改：find_victim()、replacement_cache_fill()、update_replacement_state()
+// 不可修改：函数签名、EVOLVE-BLOCK 外的代码
+// EVOLVE-BLOCK-END
+```
+
+### 禁止行为（立即拒绝，valid=0）
+
+| 类别 | 示例 | 检测方式 |
+|---|---|---|
+| **头文件** | ``, ``, ``, ``, `` | 头文件扫描 |
+| **文件 I/O** | `fopen`, `fread`, `mmap(`, `open(` | API 扫描 |
+| **系统调用** | `system(`, `popen`, `fork` | API 扫描 |
+| **线程** | `pthread_`, `std::thread`, `std::async` | API 扫描 |
+| **动态内存** | `new`, `delete`, `malloc(`, `free(` | 正则扫描 |
+| **动态容器** | `std::vector`, `std::map`, `std::unordered_map` | 容器扫描 |
+| **非确定性** | `rand()`, `srand(`, `random_device`, `mt19937`, `time()` | 函数扫描 |
+| **存储溢出** | 声明的数组总大小 > 64 KB | 静态求和 |
+| **接口违规** | 删除 EVOLVE-BLOCK 标记、修改只读区域 | diff 检测 |
+
+---
+
+## 安装与运行
+
+### 前置条件
+
+- Python 3.10+
+- 支持 C++17 的编译器（g++-12+ 或 clang）
+- CMake ≥ 3.22
+- ~10 GB 磁盘空间
+
+### 安装步骤
+
+```bash
+# 1. 克隆 ChampSim
+git clone https://github.com/ChampSim/ChampSim.git data/ChampSim
+cd data/ChampSim && git checkout 51588e1d && git submodule update --init && ./vcpkg/bootstrap-vcpkg.sh && ./vcpkg/vcpkg install && cd ../..
+
+# 2. 下载 traces
+bash data/download_traces.sh
+
+# 3. 安装 Python 依赖
+pip install -r verification/requirements.txt
+
+# 4. 运行基线测试
+python verification/evaluator.py replacement/my_policy.cc --mode quick
+```
+
+### 验证测试
+
+```bash
+python verification/test_validation.py
+```
+
+预期：全部 10 个测试通过。
+
+---
+
+## 与现有 ComputerSystems benchmark 对比
+
+| Benchmark | 工程领域 | Agent 操作 | 语言 |
+|---|---|---|---|
+| `MallocLab` | 操作系统内存管理 | 实现 malloc/free/realloc | C |
+| `DuckDBWorkloadOptimization` | 数据库查询优化 | 选择索引 + 改写 SQL | Python |
+| **`CacheReplacement`** | **计算机体系结构** | **设计缓存替换策略** | **C++** |
+
+---
+
+## 参考资料
+
+- Cache Replacement Championship (CRC-2): https://crc2.ece.tamu.edu/
+- Data Prefetching Championship (DPC-3): https://dpc3.compas.cs.stonybrook.edu/
+- ChampSim: https://github.com/ChampSim/ChampSim
+- SPEC CPU 2017 traces: https://dpc3.compas.cs.stonybrook.edu/champsim-traces/speccpu/
diff --git a/benchmarks/ComputerSystems/CacheReplacement/Task.md b/benchmarks/ComputerSystems/CacheReplacement/Task.md
new file mode 100644
index 00000000..2f74963e
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/Task.md
@@ -0,0 +1,368 @@
+# CacheReplacement Task Specification
+
+## 1. Background
+
+### 1.1 The Engineering Problem
+
+CPU caches are finite-capacity hardware structures. On every cache miss, the hardware must decide which existing line to evict to make room for the incoming line. This eviction decision — the **replacement policy** — directly determines cache hit rate and application performance.
+
+- A 1% increase in LLC hit rate translates to 5–15% application speedup on modern CPUs
+- Wrong eviction decisions cause cache pollution, thrashing, and severe performance degradation
+- Engineers at Intel, AMD, ARM, and Apple have designed replacement policies for decades
+- The problem remains open — no single policy dominates all workloads
+
+### 1.2 Why LRU is the Baseline
+
+LRU (Least Recently Used) is the classical replacement policy, used as the default baseline in virtually all replacement research (Cache Replacement Championship, Data Prefetching Championship, academic papers). It assumes temporal locality: recently accessed lines are likely to be accessed again soon.
+
+**Returning LRU does not mean optimization is successful.** Score = 1.0 means parity; score > 1.0 means the agent discovered measurable improvement over a decades-old heuristic.
+
+### 1.3 Competition Background
+
+- **Cache Replacement Championship (CRC-2)**, ISCA 2017: https://crc2.ece.tamu.edu/
+- **Data Prefetching Championship (DPC-3)**, ISCA 2019: https://dpc3.compas.cs.stonybrook.edu/
+- Winners: SHIP (signature-based hit prediction), Hawkeye (optimal reference simulation)
+
+### 1.4 Optimization Search Space
+
+| Dimension | What the agent designs |
+|---|---|
+| **Metadata structure** | What per-line state to track (counters, signatures, reuse distance, history) |
+| **Recency / frequency** | How to balance temporal vs access-frequency signals |
+| **Victim selection** | Decision logic in `find_victim()` given current state |
+| **Adaptivity** | Whether to detect workload phases and switch strategies |
+| **State compression** | How to pack metadata within 64 KB budget |
+
+---
+
+## 2. Task Definition
+
+The agent modifies `replacement/my_policy.cc` (within `// EVOLVE-BLOCK-START` / `// EVOLVE-BLOCK-END` markers) to implement a custom cache replacement policy. The evalutor performs 4 layers of validation and evaluation:
+
+| Layer | Check | Outcome on failure |
+|---|---|---|
+| **L1: Source Validation** | Forbidden headers/APIs, randomness, dynamic memory, storage budget, EVOLVE-BLOCK integrity | `valid=0, score=0` |
+| **L2: Compile** | Incremental build via ChampSim build system, timeout 300s | `valid=0, score=0` |
+| **L3: Quick Verify** | Run on verify_trace.xz, check for crashes, timeout 120s | `valid=0, score=0` |
+| **L4: Full Evaluation** | Run on 3 (quick) or 10 (full) SPEC traces, compute GMEAN score | Failed traces skipped |
+
+### 2.1 Input
+
+- `replacement/my_policy.h` — Class declaration (EVOLVE-BLOCK for member variables, read-only otherwise)
+- `replacement/my_policy.cc` — Method implementations (EVOLVE-BLOCK for method bodies, read-only otherwise)
+
+### 2.2 Output
+
+The evaluator produces `metrics.json`:
+
+```json
+{
+ "valid": true,
+ "score": 1.15,
+ "benchmark_version": "1.0.0",
+ "mode": "full",
+ "compile_time_s": 7.8,
+ "simulation_time_s": 914.7,
+ "num_traces_completed": 10,
+ "num_traces_total": 10,
+ "error": null
+}
+```
+
+On failure:
+
+```json
+{
+ "valid": false,
+ "score": 0.0,
+ "error": "Static validation: Forbidden header: "
+}
+```
+
+---
+
+## 3. EVOLVE-BLOCK Contract
+
+### 3.1 `replacement/my_policy.h` — Member Variable Declaration
+
+The EVOLVE-BLOCK in the header file allows the agent to **add** member variables that store replacement policy metadata:
+
+```cpp
+// EVOLVE-BLOCK-START
+// Agent may ADD member variables here (fixed-size arrays only):
+// uint8_t name[SIZE];
+// std::array name;
+//
+// Agent must NOT remove or modify existing base members:
+// long NUM_WAY, std::vector last_used_cycles, uint64_t cycle
+//
+// Total EVOLVE-BLOCK storage ≤ 64 KB (checked by evaluator).
+// EVOLVE-BLOCK-END
+```
+
+**Allowed**: fixed-size primitive arrays, `std::array`
+**Prohibited**: `std::vector`, `std::map`, dynamic containers, global/static variables
+
+### 3.2 `replacement/my_policy.cc` — Method Implementation
+
+The EVOLVE-BLOCK in the implementation file allows the agent to modify **three method bodies**:
+
+```cpp
+// EVOLVE-BLOCK-START
+// Agent may modify method bodies below. Keep function signatures unchanged.
+
+long my_policy::find_victim(uint32_t triggering_cpu, uint64_t instr_id,
+ long set, const champsim::cache_block* current_set,
+ champsim::address ip, champsim::address full_addr,
+ access_type type)
+{
+ // Return way index to evict: [0, NUM_WAY)
+ // Default LRU: evict way with oldest last-use cycle
+ auto begin = std::next(std::begin(last_used_cycles), set * NUM_WAY);
+ auto end = std::next(begin, NUM_WAY);
+ auto victim = std::min_element(begin, end);
+ return static_cast(std::distance(begin, victim));
+}
+
+void my_policy::replacement_cache_fill(uint32_t triggering_cpu, long set,
+ long way, champsim::address full_addr,
+ champsim::address ip,
+ champsim::address victim_addr,
+ access_type type)
+{
+ // Called when a new line is filled into the cache
+ last_used_cycles.at(static_cast(set * NUM_WAY + way)) = cycle++;
+}
+
+void my_policy::update_replacement_state(uint32_t triggering_cpu, long set,
+ long way, champsim::address full_addr,
+ champsim::address ip,
+ champsim::address victim_addr,
+ access_type type, uint8_t hit)
+{
+ // Called on every cache access. Update policy state.
+ if (hit && access_type{type} != access_type::WRITE)
+ last_used_cycles.at(static_cast(set * NUM_WAY + way)) = cycle++;
+}
+
+// EVOLVE-BLOCK-END
+```
+
+### 3.3 Available Base Members
+
+The following member variables are available for use in method implementations:
+
+| Member | Type | Description |
+|---|---|---|
+| `NUM_WAY` | `long` | Cache associativity (number of ways per set) |
+| `last_used_cycles` | `std::vector` | Per-line last-use cycle timestamps (size = sets × ways) |
+| `cycle` | `uint64_t` | Monotonically increasing cycle counter |
+
+### 3.4 Forbidden (complete list)
+
+| Category | Banned items |
+|---|---|
+| **Headers** | ``, ``, ``, ``, ``, ``, ``, ``, ``, ``, ``, ``, ``, ``, ``, `` |
+| **File I/O** | `fopen`, `fread`, `fwrite`, `fclose`, `open(`, `read(`, `write(`, `close(`, `mmap(`, `munmap(` |
+| **System** | `system(`, `popen`, `fork` |
+| **Threading** | `pthread_*`, `std::thread`, `std::async` |
+| **Dynamic memory** | `new`, `delete`, `malloc(`, `free(`, `calloc(`, `realloc(` |
+| **Dynamic containers** | `std::vector`, `std::map`, `std::unordered_map`, `std::set`, `std::list`, `std::deque` |
+| **Non-determinism** | `rand()`, `srand(`, `random_device`, `mt19937`, `default_random_engine`, `time(`, `clock(` |
+| **Exceptions** | `throw`, `try`, `catch` |
+| **RTTI** | `dynamic_cast`, `typeid` |
+| **Storage overflow** | Total declared arrays in header EVOLVE-BLOCK > 64 KB |
+| **Interface violation** | Modifying function signatures, EVOLVE-BLOCK markers, or code outside EVOLVE-BLOCK |
+
+---
+
+## 4. Scoring
+
+### 4.1 Formula
+
+```
+For each completed trace t:
+ ratio[t] = IPC_candidate[t] / IPC_LRU[t]
+
+final_score = exp( (1/N) * Σ ln(max(ratio[t], 0.001)) )
+ where N = number of successfully completed traces
+```
+
+### 4.2 Interpretation
+
+| Score | Meaning |
+|---|---|
+| 1.0 | Parity with LRU (baseline) |
+| > 1.0 | Better than LRU |
+| (0, 1.0) | Worse than LRU but functionally valid |
+| 0 | Invalid (any layer failure) |
+
+### 4.3 Edge Cases
+
+| Scenario | Handling |
+|---|---|
+| Candidate IPC ≤ 0 | Trace skipped |
+| Simulation crashes | Trace skipped |
+| Per-trace timeout | Trace skipped |
+| All traces skipped | `valid=0, score=0` |
+| Baseline IPC ≤ 0 | `valid=0, error="Bad environment"` |
+
+---
+
+## 5. Evaluation Modes
+
+### Quick Mode (`--mode quick`)
+
+| Parameter | Value |
+|---|---|
+| Traces | `605.mcf_s-665B`, `625.x264_s-18B`, `607.cactuBSSN_s-2421B` |
+| Warmup instructions | 10,000,000 |
+| Simulation instructions | 20,000,000 |
+| Expected runtime | ~15 minutes |
+| Per-trace timeout | 600 seconds |
+| Purpose | Agent iteration / rapid feedback |
+
+### Full Mode (`--mode full`)
+
+| Parameter | Value |
+|---|---|
+| Traces | 10 (all SPEC CPU 2017) |
+| Warmup instructions | 50,000,000 |
+| Simulation instructions | 100,000,000 |
+| Expected runtime | 30–60 minutes |
+| Per-trace timeout | 1800 seconds |
+| Purpose | Leaderboard / final scoring |
+
+---
+
+## 6. Timeout Behavior
+
+| Timeout type | Value | Effect |
+|---|---|---|
+| Compile | 300s | `valid=0, score=0` |
+| Quick Verify | 120s | `valid=0, score=0` |
+| Per-trace (Quick) | 600s | Trace skipped, remaining continue |
+| Per-trace (Full) | 1800s | Trace skipped, remaining continue |
+
+---
+
+## 7. Baselines
+
+### Official baseline: `baseline/lru.cc`
+
+LRU (Least Recently Used). Score = 1.0. Used for computing IPC normalization.
+
+### Sanity check: `baseline/random.cc`
+
+Random replacement policy. Expected score < 1.0. **Not an official baseline.** Not included in leaderboard. Used to verify that the evaluator can distinguish policy quality:
+
+```bash
+# Expected: score < 1.0 (random is worse than LRU)
+python verification/evaluator.py baseline/random.cc --mode quick
+```
+
+---
+
+## 8. Examples
+
+### Example: Well-formed policy (LRU)
+
+This is the default implementation. It should produce `valid=1, score≈1.0`:
+
+```cpp
+long my_policy::find_victim(...) {
+ auto begin = std::next(std::begin(last_used_cycles), set * NUM_WAY);
+ auto end = std::next(begin, NUM_WAY);
+ auto victim = std::min_element(begin, end);
+ return static_cast(std::distance(begin, victim));
+}
+```
+
+### Example: Rejected (iostream)
+
+```cpp
+// EVOLVE-BLOCK-START
+#include // ← Layer 1: Rejected. Forbidden header.
+long my_policy::find_victim(...) { return 0; }
+// EVOLVE-BLOCK-END
+```
+
+### Example: Rejected (rand)
+
+```cpp
+// EVOLVE-BLOCK-START
+long my_policy::find_victim(...) {
+ return rand() % NUM_WAY; // ← Layer 1: Rejected. Non-deterministic.
+}
+// EVOLVE-BLOCK-END
+```
+
+### Example: Rejected (deepseek vector)
+
+```cpp
+// EVOLVE-BLOCK-START
+std::vector extra; // ← Layer 1: Rejected. Dynamic container.
+long my_policy::find_victim(...) { return 0; }
+// EVOLVE-BLOCK-END
+```
+
+---
+
+## 9. Usage
+
+```bash
+# Quick mode (agent iteration, ~15 min)
+python verification/evaluator.py replacement/my_policy.cc --mode quick
+
+# Full mode (final scoring, ~30-60 min)
+python verification/evaluator.py replacement/my_policy.cc --mode full
+
+# Validation tests
+python verification/test_validation.py
+
+# Frontier-Eng integration
+python -m frontier_eval task=unified task.benchmark=ComputerSystems/CacheReplacement algorithm.iterations=0
+```
+
+---
+
+## 10. Data Management
+
+Traces are **ChampSim trace files generated from SPEC CPU 2017 workloads**, sourced from the [DPC-3 public repository](https://dpc3.compas.cs.stonybrook.edu/champsim-traces/speccpu/).
+
+> ⚠️ **License notice**: This benchmark repository does **not** redistribute SPEC CPU traces. The repository only contains metadata, download instructions, and checksums. Users must download trace files from the original source via `bash data/download_traces.sh`.
+
+### Trace specification
+
+| Trace | Domain | Quick | Full | Compressed |
+|---|---|---|---|---|
+| `603.bwaves_s-3699B` | Float (scientific) | — | ✓ | ~50 MB |
+| `605.mcf_s-665B` | Integer (combinatorial) | ✓ | ✓ | ~60 MB |
+| `625.x264_s-18B` | Integer (video) | ✓ | ✓ | ~80 MB |
+| `654.roms_s-842B` | Float (ocean) | — | ✓ | ~55 MB |
+| `657.xz_s-3167B` | Integer (compression) | — | ✓ | ~70 MB |
+| `600.perlbench_s-210B` | Integer (scripting) | — | ✓ | ~60 MB |
+| `602.gcc_s-734B` | Integer (compiler) | — | ✓ | ~75 MB |
+| `607.cactuBSSN_s-2421B` | Float (physics) | ✓ | ✓ | ~90 MB |
+| `621.wrf_s-575B` | Float (weather) | — | ✓ | ~65 MB |
+| `631.deepsjeng_s-928B` | Integer (AI/game) | — | ✓ | ~55 MB |
+
+### Checksum verification
+
+```bash
+cd data/traces && sha256sum -c ../checksums.txt
+```
+
+---
+
+## 11. References
+
+- Cache Replacement Championship (CRC-2, ISCA 2017): https://crc2.ece.tamu.edu/
+- Data Prefetching Championship (DPC-3, ISCA 2019): https://dpc3.compas.cs.stonybrook.edu/
+- ChampSim simulator: https://github.com/ChampSim/ChampSim
+- SPEC CPU 2017: https://www.spec.org/cpu2017/
+- SRRIP/DRRIP: Jaleel et al., ISCA 2010
+- SHIP: Wu et al., MICRO 2011
+- Hawkeye: Jain and Lin, ASPLOS 2016
+- TAGE-SC-L: Seznec, CBP 2016
+- See `references/related_work.md` for full citations.
diff --git a/benchmarks/ComputerSystems/CacheReplacement/Task_zh-CN.md b/benchmarks/ComputerSystems/CacheReplacement/Task_zh-CN.md
new file mode 100644
index 00000000..e7d93a1e
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/Task_zh-CN.md
@@ -0,0 +1,139 @@
+# CacheReplacement 任务说明
+
+## 1. 背景
+
+### 1.1 工程问题
+
+CPU 缓存容量有限。当新数据载入缓存时，必须选择驱逐哪条旧数据。这个决策——**替换策略（Replacement Policy）**——直接影响缓存命中率和性能。
+
+- LLC 命中率提升 1% → 5–15% 应用加速
+- 错误驱逐会导致缓存污染和性能严重下降
+- Intel、AMD、ARM、Apple 研究数十年，至今无统一最优解
+
+### 1.2 为什么 LRU 是基线
+
+LRU 是经典替换策略，几乎所有学术研究以 LRU 为默认基线。
+
+**返回 LRU 不代表优化成功。** Score = 1.0 = 等价；score > 1.0 = 超越了数十年启发式算法。
+
+### 1.3 优化搜索空间
+
+| 维度 | 设计内容 |
+|---|---|
+| metadata 结构 | 跟踪哪些状态（计数器、签名、重用距离） |
+| 最近/频率 | 平衡时间局部性和频率 |
+| 驱逐选择 | `find_victim()` 决策逻辑 |
+| 自适应性 | 检测 workload 阶段切换 |
+| 状态压缩 | 64 KB 预算内压缩 |
+
+---
+
+## 2. 任务定义
+
+修改 `replacement/my_policy.cc`（EVOLVE-BLOCK 内）实现替换策略。评估器 4 层验证：
+
+| 层 | 检查 | 失败结果 |
+|---|---|---|
+| L1: 源码验证 | 禁止头文件/API、动态内存、预算、标记完整性 | `valid=0, score=0` |
+| L2: 编译 | ChampSim 增量编译，超时 300s | `valid=0, score=0` |
+| L3: 快速验证 | verify_trace 运行不崩溃 | `valid=0, score=0` |
+| L4: 正式评测 | 3/10 个 SPEC trace 计算 GMEAN | 失败 trace 跳过 |
+
+### 2.1 输入/输出
+
+- 输入：`replacement/my_policy.h`（成员变量）、`replacement/my_policy.cc`（方法体）
+- 输出：`metrics.json`（valid, score, per_trace）
+
+---
+
+## 3. EVOLVE-BLOCK 契约
+
+### 3.1 `.h` 文件 — 成员变量
+
+```cpp
+// EVOLVE-BLOCK-START
+// 可添加固定大小数组，总 ≤ 64 KB
+uint8_t history_table[1024];
+// EVOLVE-BLOCK-END
+```
+
+### 3.2 `.cc` 文件 — 方法体
+
+可修改三个方法体，不可修改函数签名：
+
+- `find_victim()` — 返回 [0, NUM_WAY)
+- `replacement_cache_fill()` — 缓存填充
+- `update_replacement_state()` — 状态更新
+
+### 3.3 可用成员
+
+`NUM_WAY`（long）、`last_used_cycles`（vector）、`cycle`（uint64_t）
+
+### 3.4 禁止项
+
+| 类别 | 禁止项 |
+|---|---|
+| 头文件 | ``, ``, ``, ``, ``, `` |
+| 文件 I/O | `fopen`, `fread`, `fwrite`, `open(`, `read(`, `mmap(` |
+| 系统调用 | `system(`, `popen`, `fork` |
+| 线程 | `pthread_`, `std::thread`, `std::async` |
+| 动态内存 | `new`, `delete`, `malloc(`, `free(` |
+| 动态容器 | `std::vector`, `std::map`, `std::unordered_map`, `std::set`, `std::list` |
+| 非确定性 | `rand()`, `srand(`, `random_device`, `mt19937`, `time(` |
+| 存储溢出 | 数组总大小 > 64 KB |
+| 接口违规 | 修改函数签名、EVOLVE-BLOCK 标记、只读区域 |
+
+---
+
+## 4. 评分
+
+### 4.1 公式
+
+```
+score = exp( (1/N) * Σ ln(max(IPC_candidate/ IPC_LRU, 0.001)) )
+```
+
+### 4.2 解释
+
+| 分数 | 含义 |
+|---|---|
+| 1.0 | LRU 等价（基线） |
+| > 1.0 | 优于 LRU |
+| (0, 1.0) | 差于但有效 |
+| 0 | 无效 |
+
+---
+
+## 5. 评测模式
+
+| 模式 | Traces | Warmup | Sim | 时间 | 用途 |
+|---|---|---|---|---|---|
+| Quick | 3 | 10M | 20M | ~15 min | 迭代 |
+| Full | 10 | 50M | 100M | ~30-60 min | 评分 |
+
+---
+
+## 6. 使用方法
+
+```bash
+# Quick 模式
+python verification/evaluator.py replacement/my_policy.cc --mode quick
+
+# Full 模式
+python verification/evaluator.py replacement/my_policy.cc --mode full
+
+# 验证测试
+python verification/test_validation.py
+
+# Frontier-Eng 集成
+python -m frontier_eval task=unified task.benchmark=ComputerSystems/CacheReplacement algorithm.iterations=0
+```
+
+---
+
+## 7. 参考资料
+
+- CRC-2: https://crc2.ece.tamu.edu/
+- DPC-3: https://dpc3.compas.cs.stonybrook.edu/
+- ChampSim: https://github.com/ChampSim/ChampSim
+- 学术论文：`references/related_work.md`
diff --git a/benchmarks/ComputerSystems/CacheReplacement/baseline/lru.cc b/benchmarks/ComputerSystems/CacheReplacement/baseline/lru.cc
new file mode 100644
index 00000000..4e85ccf0
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/baseline/lru.cc
@@ -0,0 +1,21 @@
+// baseline/lru.cc — LRU reference (official baseline)
+
+#include "cache.h"
+#include "modules.h"
+
+struct lru : public champsim::modules::replacement {
+ long NUM_WAY;
+ std::vector last_used_cycles;
+ uint64_t cycle = 0;
+ lru(CACHE* cache) : lru(cache, cache->NUM_SET, cache->NUM_WAY) {}
+ lru(CACHE* cache, long sets, long ways) : replacement(cache), NUM_WAY(ways), last_used_cycles(static_cast(sets * ways), 0) {}
+ long find_victim(uint32_t, uint64_t, long set, const champsim::cache_block* b, champsim::address, champsim::address, access_type) {
+ return static_cast(std::distance(std::begin(last_used_cycles) + set * NUM_WAY, std::min_element(std::begin(last_used_cycles) + set * NUM_WAY, std::begin(last_used_cycles) + (set + 1) * NUM_WAY)));
+ }
+ void replacement_cache_fill(uint32_t, long set, long way, champsim::address, champsim::address, champsim::address, access_type) {
+ last_used_cycles.at(static_cast(set * NUM_WAY + way)) = cycle++;
+ }
+ void update_replacement_state(uint32_t, long set, long way, champsim::address, champsim::address, champsim::address, access_type, uint8_t hit) {
+ if (hit) last_used_cycles.at(static_cast(set * NUM_WAY + way)) = cycle++;
+ }
+};
diff --git a/benchmarks/ComputerSystems/CacheReplacement/baseline/random.cc b/benchmarks/ComputerSystems/CacheReplacement/baseline/random.cc
new file mode 100644
index 00000000..6df3093e
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/baseline/random.cc
@@ -0,0 +1,11 @@
+// baseline/random.cc — sanity check only. Not official baseline.
+#include "cache.h"
+#include "modules.h"
+struct random_policy : public champsim::modules::replacement {
+ long NUM_WAY;
+ random_policy(CACHE* cache) : random_policy(cache, cache->NUM_SET, cache->NUM_WAY) {}
+ random_policy(CACHE* cache, long sets, long ways) : replacement(cache), NUM_WAY(ways) {}
+ long find_victim(uint32_t, uint64_t, long, const champsim::cache_block*, champsim::address, champsim::address, access_type) { return rand() % NUM_WAY; }
+ void replacement_cache_fill(uint32_t, long, long, champsim::address, champsim::address, champsim::address, access_type) {}
+ void update_replacement_state(uint32_t, long, long, champsim::address, champsim::address, champsim::address, access_type, uint8_t) {}
+};
diff --git a/benchmarks/ComputerSystems/CacheReplacement/baseline/result_log.txt b/benchmarks/ComputerSystems/CacheReplacement/baseline/result_log.txt
new file mode 100644
index 00000000..85e78267
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/baseline/result_log.txt
@@ -0,0 +1,53 @@
+# CacheReplacement Baseline Test Results
+# Generated: 2026-07-15 (fresh run after all fixes)
+#
+# This file records the local test results of the LRU baseline
+# policy (replacement/my_policy.cc default implementation).
+# Score should be 1.0 (LRU vs LRU — baseline self-test).
+
+## Environment
+- Host: macOS 14.5 (Apple Silicon M3)
+- Python: 3.12.7
+- Compiler: Apple Clang 16 (C++17, via Xcode)
+- ChampSim commit: 51588e1d6f97875fe8de1a3621d28668bff83fcf
+- Evaluator: verification/evaluator.py v1.0.0
+
+## Quick Mode Results
+- Mode: quick (3 traces)
+- Traces: 605.mcf_s-665B, 625.x264_s-18B, 607.cactuBSSN_s-2421B
+- Warmup: 10M instructions | Simulation: 20M instructions
+- Total runtime: 1882.3s
+- Compile time: 7.1s
+- Simulation time: 931.3s
+
+## Score
+- Score: 1.0 (baseline LRU vs LRU = 1.0)
+- Valid: true
+- Traces completed: 3/3
+
+## Per-Trace Results
+See baseline/artifacts.json for complete per-trace IPC data.
+
+## Baseline LRU IPC (Quick Mode)
+See references/generated_baseline.json for cached values.
+
+## Validation Tests
+All 10 Layer-1 static validation tests pass:
+ 1. Normal LRU policy PASS
+ 2. Forbidden header PASS
+ 3. rand() usage PASS
+ 4. fopen() file I/O PASS
+ 5. Dynamic container PASS
+ 6. new operator PASS
+ 7. mt19937 randomness PASS
+ 8. system() call PASS
+ 9. Missing EVOLVE-BLOCK-END PASS
+ 10. Budget exceeded PASS
+
+## Reproducibility Notes
+- ChampSim is pinned to commit 51588e1d via git checkout
+- vcpkg dependencies managed via submodule (pinned)
+- verify_trace.xz is a fixed file (75 KB, ~25s quick validation)
+- Trace checksums are verified via data/checksums.txt (SHA256)
+- Docker image uses pinned Ubuntu 22.04 + gcc-13 + cmake 3.22
+- On macOS, evaluator auto-detects vcpkg include path in _build_champsim()
diff --git a/benchmarks/ComputerSystems/CacheReplacement/data/README.md b/benchmarks/ComputerSystems/CacheReplacement/data/README.md
new file mode 100644
index 00000000..fc6203a6
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/data/README.md
@@ -0,0 +1,59 @@
+# CacheReplacement — Data
+
+This directory contains external data required by the CacheReplacement benchmark.
+
+## Traces
+
+ChampSim trace files generated from SPEC CPU 2017 workloads.
+
+**Source**: [DPC-3 public repository](https://dpc3.compas.cs.stonybrook.edu/champsim-traces/speccpu/)
+
+> ⚠️ **License**: This benchmark does NOT redistribute SPEC CPU traces.
+> Users must download traces from the original source and verify integrity.
+
+### Download
+
+```bash
+# Automated download + checksum verification
+bash data/download_traces.sh
+```
+
+### Verify
+
+```bash
+# Verify all 10 traces match expected checksums
+cd data/traces && sha256sum -c ../checksums.txt
+```
+
+### Trace list
+
+| Trace | Domain | Quick | Full | Size (xz) |
+|---|---|---|---|---|
+| `603.bwaves_s-3699B` | Float (scientific) | — | ✓ | ~50 MB |
+| `605.mcf_s-665B` | Integer (combinatorial) | ✓ | ✓ | ~60 MB |
+| `625.x264_s-18B` | Integer (video) | ✓ | ✓ | ~80 MB |
+| `654.roms_s-842B` | Float (ocean) | — | ✓ | ~55 MB |
+| `657.xz_s-3167B` | Integer (compression) | — | ✓ | ~70 MB |
+| `600.perlbench_s-210B` | Integer (scripting) | — | ✓ | ~60 MB |
+| `602.gcc_s-734B` | Integer (compiler) | — | ✓ | ~75 MB |
+| `607.cactuBSSN_s-2421B` | Float (physics) | ✓ | ✓ | ~90 MB |
+| `621.wrf_s-575B` | Float (weather) | — | ✓ | ~65 MB |
+| `631.deepsjeng_s-928B` | Integer (AI/game) | — | ✓ | ~55 MB |
+
+**Total**: ~660 MB compressed, ~3 GB decompressed.
+
+## Simulator
+
+ChampSim is auto-cloned to `data/ChampSim/` at commit `51588e1d`.
+Source: https://github.com/ChampSim/ChampSim
+
+## Structure
+
+```
+data/
+├── README.md ← this file
+├── download_traces.sh ← automated download script
+├── checksums.txt ← SHA256 checksums for all traces
+├── traces/ ← .xz trace files (NOT tracked in git)
+└── ChampSim/ ← simulator source (NOT tracked in git)
+```
diff --git a/benchmarks/ComputerSystems/CacheReplacement/data/checksums.txt b/benchmarks/ComputerSystems/CacheReplacement/data/checksums.txt
new file mode 100644
index 00000000..c8b72455
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/data/checksums.txt
@@ -0,0 +1,14 @@
+# CacheReplacement — Trace SHA256 checksums
+# Generated: 2026-07-15
+# Download: bash data/download_traces.sh
+# Verify: cd data/traces && sha256sum -c ../checksums.txt
+ac576730cff4f13d70384dff1939da83250bc0e3de1d9d994fdc637fd1b7cf4d 600.perlbench_s-210B.champsimtrace.xz
+a32be73c53946748a1393cca4999f374c5d9d3737a414437d81bb20ed6311bd5 602.gcc_s-734B.champsimtrace.xz
+107f5d325cb4f02a144657883176767fbe3153911e37e955201fff1fc6474594 603.bwaves_s-3699B.champsimtrace.xz
+b7bef3f68180923fa4b88e1e8a6749487beef99099efce78500a03e3a3e8334e 605.mcf_s-665B.champsimtrace.xz
+ae2cf28160028687bd3bdecf9135aeace9553b506d731230176087e1cee26559 607.cactuBSSN_s-2421B.champsimtrace.xz
+03f8b049cfedf871715493513f505a726194fd09f10b0177f04113be80bead18 621.wrf_s-575B.champsimtrace.xz
+e2782e14cff1fe6c2968f7527aaa4d5172eabc77a56e656cd6773c79696ee4ef 625.x264_s-18B.champsimtrace.xz
+a4bf5970bf96d95e35007af9772989c4eaffb9312146f0bc6a13919270400e56 631.deepsjeng_s-928B.champsimtrace.xz
+2377f1d782da5f4c158265d3a2379338903e15a56ce0aab013032539c181a82a 654.roms_s-842B.champsimtrace.xz
+69b5c7c62231b4b116c6a49674e65af9d19ec3fd7bb6157c735d114c23e420ef 657.xz_s-3167B.champsimtrace.xz
diff --git a/benchmarks/ComputerSystems/CacheReplacement/data/download_traces.sh b/benchmarks/ComputerSystems/CacheReplacement/data/download_traces.sh
new file mode 100755
index 00000000..03a3d557
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/data/download_traces.sh
@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+BASE_URL="https://dpc3.compas.cs.stonybrook.edu/champsim-traces/speccpu"
+cd "$(dirname "$0")/traces"
+for t in 603.bwaves_s-3699B 605.mcf_s-665B 625.x264_s-18B 654.roms_s-842B 657.xz_s-3167B 600.perlbench_s-210B 602.gcc_s-734B 607.cactuBSSN_s-2421B 621.wrf_s-575B 631.deepsjeng_s-928B; do
+ f="${t}.champsimtrace.xz"; [ -f "$f" ] && echo "Exists: $f" || { echo "Downloading $f..."; curl -sSL -o "$f" "${BASE_URL}/${f}" || wget -q -O "$f" "${BASE_URL}/${f}"; }
+done
+sha256sum -c ../checksums.txt 2>/dev/null && echo "Checksums OK" || echo "Checksums not verified"
diff --git a/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/agent_files.txt b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/agent_files.txt
new file mode 100644
index 00000000..da6e439e
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/agent_files.txt
@@ -0,0 +1,18 @@
+README.md
+README_zh-CN.md
+Task.md
+Task_zh-CN.md
+replacement/my_policy.cc
+replacement/my_policy.h
+verification/evaluator.py
+verification/test_validation.py
+baseline/lru.cc
+baseline/random.cc
+references/problem_config.json
+references/constants.json
+references/related_work.md
+data/README.md
+data/download_traces.sh
+data/checksums.txt
+frontier_eval/constraints.txt
+.gitignore
diff --git a/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/artifact_files.txt b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/artifact_files.txt
new file mode 100644
index 00000000..76dc893a
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/artifact_files.txt
@@ -0,0 +1,2 @@
+metrics.json
+artifacts.json
diff --git a/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/candidate_destination.txt b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/candidate_destination.txt
new file mode 100644
index 00000000..443fd8c7
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/candidate_destination.txt
@@ -0,0 +1 @@
+replacement/my_policy.cc
diff --git a/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/constraints.txt b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/constraints.txt
new file mode 100644
index 00000000..c2ae9d5a
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/constraints.txt
@@ -0,0 +1,8 @@
+CacheReplacement constraints:
+1) Only modify code inside EVOLVE-BLOCK in replacement/my_policy.cc and replacement/my_policy.h.
+2) Function signatures are read-only.
+3) No dynamic memory, dynamic containers, file I/O, randomness, threading, system calls.
+4) Storage budget: ≤64 KB for EVOLVE-BLOCK member variables.
+5) Implementation must be deterministic.
+6) Baseline: LRU. Score = geometric_mean(IPC_candidate / IPC_LRU). Baseline = 1.0.
+7) Read-only: evaluator.py, baseline/*, references/*, data/ChampSim/*
diff --git a/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/copy_files.txt b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/copy_files.txt
new file mode 100644
index 00000000..9c558e35
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/copy_files.txt
@@ -0,0 +1 @@
+.
diff --git a/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/eval_command.txt b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/eval_command.txt
new file mode 100644
index 00000000..584b4178
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/eval_command.txt
@@ -0,0 +1 @@
+{python} verification/evaluator.py {candidate} --metrics-out metrics.json --artifacts-out artifacts.json --mode full
diff --git a/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/eval_cwd.txt b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/eval_cwd.txt
new file mode 100644
index 00000000..9c558e35
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/eval_cwd.txt
@@ -0,0 +1 @@
+.
diff --git a/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/initial_program.txt b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/initial_program.txt
new file mode 100644
index 00000000..443fd8c7
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/initial_program.txt
@@ -0,0 +1 @@
+replacement/my_policy.cc
diff --git a/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/readonly_files.txt b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/readonly_files.txt
new file mode 100644
index 00000000..3e9ff41a
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/frontier_eval/readonly_files.txt
@@ -0,0 +1,14 @@
+replacement/my_policy.h
+verification/evaluator.py
+verification/test_validation.py
+baseline/lru.cc
+baseline/random.cc
+references/problem_config.json
+references/constants.json
+references/related_work.md
+data/README.md
+data/download_traces.sh
+frontier_eval/constraints.txt
+frontier_eval/agent_files.txt
+frontier_eval/artifact_files.txt
+.gitignore
diff --git a/benchmarks/ComputerSystems/CacheReplacement/references/constants.json b/benchmarks/ComputerSystems/CacheReplacement/references/constants.json
new file mode 100644
index 00000000..57129114
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/references/constants.json
@@ -0,0 +1,17 @@
+{
+ "benchmark_version": "1.0.0",
+ "simulator": {
+ "name": "ChampSim",
+ "repo": "https://github.com/ChampSim/ChampSim",
+ "commit": "51588e1d6f97875fe8de1a3621d28668bff83fcf",
+ "build_command": "./config.sh .json && make -j4"
+ },
+ "evaluation": {
+ "llc_size": "2 MB",
+ "llc_associativity": 16,
+ "cache_line_size": 64,
+ "num_cores": 1
+ },
+ "storage": { "budget_bytes": 65536, "validation": "approximate static check" },
+ "scoring": { "formula": "GMEAN(IPC_candidate / IPC_LRU)", "baseline_score": 1.0 }
+}
diff --git a/benchmarks/ComputerSystems/CacheReplacement/references/problem_config.json b/benchmarks/ComputerSystems/CacheReplacement/references/problem_config.json
new file mode 100644
index 00000000..50662e69
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/references/problem_config.json
@@ -0,0 +1,17 @@
+{
+ "benchmark_id": "cache_replacement",
+ "benchmark_version": "1.0.0",
+ "traces": {
+ "source": "https://dpc3.compas.cs.stonybrook.edu/champsim-traces/speccpu/",
+ "quick": ["605.mcf_s-665B", "625.x264_s-18B", "607.cactuBSSN_s-2421B"],
+ "full": [
+ "603.bwaves_s-3699B", "605.mcf_s-665B", "625.x264_s-18B",
+ "654.roms_s-842B", "657.xz_s-3167B", "600.perlbench_s-210B",
+ "602.gcc_s-734B", "607.cactuBSSN_s-2421B", "621.wrf_s-575B",
+ "631.deepsjeng_s-928B"
+ ]
+ },
+ "quick": { "warmup_instructions": 10000000, "sim_instructions": 20000000, "expected_runtime_min": "<10" },
+ "full": { "warmup_instructions": 50000000, "sim_instructions": 100000000, "expected_runtime_min": "30-60" },
+ "timeout": { "compile": 300, "quick_verify": 120, "per_trace_quick": 600, "per_trace_full": 1800 }
+}
diff --git a/benchmarks/ComputerSystems/CacheReplacement/references/related_work.md b/benchmarks/ComputerSystems/CacheReplacement/references/related_work.md
new file mode 100644
index 00000000..1a1869b5
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/references/related_work.md
@@ -0,0 +1,61 @@
+# Related Work — Cache Replacement Policies
+
+## Key Papers
+
+### LRU (Least Recently Used)
+
+Classic baseline replacement policy. Assumes temporal locality. Used as default baseline in virtually all replacement research.
+
+### SRRIP / DRRIP (Re-Reference Interval Prediction)
+
+Jaleel, A., Theobald, K. B., Steely, S. C., & Emer, J.
+**"High Performance Cache Replacement Using Re-Reference Interval Prediction (RRIP)."**
+*Proceedings of the 37th Annual International Symposium on Computer Architecture (ISCA)*, 2010.
+DOI: [10.1145/1815961.1815971](https://doi.org/10.1145/1815961.1815971)
+
+### SHIP (Signature-based Hit Predictor)
+
+Wu, C.-J., Jaleel, A., Hasenplaugh, W., Martonosi, M., Steely, S. C., & Emer, J.
+**"SHiP: Signature-based Hit Predictor for High Performance Caching."**
+*Proceedings of the 44th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)*, 2011.
+DOI: [10.1145/2155620.2155671](https://doi.org/10.1145/2155620.2155671)
+
+### Hawkeye
+
+Jain, A., & Lin, C.
+**"Hawkeye: Efficiently Identifying Access Patterns for Optimal Cache Replacement."**
+*Proceedings of the Twenty-First International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS)*, 2016.
+DOI: [10.1145/2872362.2872406](https://doi.org/10.1145/2872362.2872406)
+
+### TAGE-SC-L
+
+Seznec, A.
+**"Tage-SC-L Branch Predictors."**
+*Championship Branch Prediction (CBP)*, 2016.
+
+### ChampSim Simulator
+
+Gober, N., Chacon, G., Wang, L., Gratz, P. V., Jimenez, D. A., Teran, E., Pugsley, S., & Kim, J.
+**"The Championship Simulator: Architectural Simulation for Education and Competition."**
+arXiv:2210.14324, 2022.
+DOI: [10.48550/arXiv.2210.14324](https://doi.org/10.48550/arXiv.2210.14324)
+GitHub: https://github.com/ChampSim/ChampSim
+
+## Competitions
+
+- **CRC-2** (Cache Replacement Championship 2, ISCA 2017)
+ https://crc2.ece.tamu.edu/
+
+- **DPC-3** (Data Prefetching Championship 3, ISCA 2019)
+ https://dpc3.compas.cs.stonybrook.edu/
+
+- **CBP-6** (Championship Branch Prediction 6, ISCA 2025)
+ https://ericrotenberg.wordpress.ncsu.edu/cbp2025-simulator-framework/
+
+## Trace Sources
+
+- SPEC CPU 2017 traces for ChampSim (DPC-3 public mirror):
+ https://dpc3.compas.cs.stonybrook.edu/champsim-traces/speccpu/
+
+- CRC-2 traces:
+ http://bit.ly/2t2nkUj
diff --git a/benchmarks/ComputerSystems/CacheReplacement/replacement/my_policy.cc b/benchmarks/ComputerSystems/CacheReplacement/replacement/my_policy.cc
new file mode 100644
index 00000000..0070cb9e
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/replacement/my_policy.cc
@@ -0,0 +1,29 @@
+// CacheReplacement — my_policy implementation
+// Code outside EVOLVE-BLOCK is READ-ONLY.
+
+#include "my_policy.h"
+#include
+#include
+
+my_policy::my_policy(CACHE* cache) : my_policy(cache, cache->NUM_SET, cache->NUM_WAY) {}
+my_policy::my_policy(CACHE* cache, long sets, long ways) : replacement(cache), NUM_WAY(ways), last_used_cycles(static_cast(sets * ways), 0) {}
+
+// EVOLVE-BLOCK-START
+// Agent may modify method bodies below. Keep function signatures unchanged.
+
+long my_policy::find_victim(uint32_t, uint64_t, long set, const champsim::cache_block* b, champsim::address, champsim::address, access_type) {
+ auto begin = std::next(std::begin(last_used_cycles), set * NUM_WAY);
+ auto end = std::next(begin, NUM_WAY);
+ auto victim = std::min_element(begin, end);
+ return static_cast(std::distance(begin, victim));
+}
+
+void my_policy::replacement_cache_fill(uint32_t, long set, long way, champsim::address, champsim::address, champsim::address, access_type) {
+ last_used_cycles.at(static_cast(set * NUM_WAY + way)) = cycle++;
+}
+
+void my_policy::update_replacement_state(uint32_t, long set, long way, champsim::address, champsim::address, champsim::address, access_type, uint8_t hit) {
+ if (hit) last_used_cycles.at(static_cast(set * NUM_WAY + way)) = cycle++;
+}
+
+// EVOLVE-BLOCK-END
diff --git a/benchmarks/ComputerSystems/CacheReplacement/replacement/my_policy.h b/benchmarks/ComputerSystems/CacheReplacement/replacement/my_policy.h
new file mode 100644
index 00000000..bc58fa7c
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/replacement/my_policy.h
@@ -0,0 +1,20 @@
+#ifndef REPLACEMENT_MY_POLICY_H
+#define REPLACEMENT_MY_POLICY_H
+#include
+#include "cache.h"
+#include "modules.h"
+struct my_policy : public champsim::modules::replacement {
+ long NUM_WAY;
+ std::vector last_used_cycles;
+ uint64_t cycle = 0;
+ // EVOLVE-BLOCK-START
+ // Agent may ADD member variables here (fixed-size arrays only)
+ // Total EVOLVE-BLOCK storage ≤ 64 KB (checked by evaluator)
+ // EVOLVE-BLOCK-END
+ my_policy(CACHE* cache);
+ my_policy(CACHE* cache, long sets, long ways);
+ long find_victim(uint32_t, uint64_t, long, const champsim::cache_block*, champsim::address, champsim::address, access_type);
+ void replacement_cache_fill(uint32_t, long, long, champsim::address, champsim::address, champsim::address, access_type);
+ void update_replacement_state(uint32_t, long, long, champsim::address, champsim::address, champsim::address, access_type, uint8_t);
+};
+#endif
diff --git a/benchmarks/ComputerSystems/CacheReplacement/verification/Dockerfile b/benchmarks/ComputerSystems/CacheReplacement/verification/Dockerfile
new file mode 100644
index 00000000..d1148e8b
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/verification/Dockerfile
@@ -0,0 +1,84 @@
+FROM ubuntu:22.04 AS champsim-builder
+
+# Optional: use APT mirror for faster builds (e.g., ARG APT_MIRROR=mirrors.ustc.edu.cn)
+ARG APT_MIRROR="archive.ubuntu.com"
+# Optional: use GitHub mirror for cloning ChampSim (e.g., ARG GIT_BASE=https://gitclone.com/github.com/)
+ARG GIT_BASE="https://github.com/"
+
+# Fixed toolchain (pinned versions for reproducibility)
+RUN sed -i "s|archive.ubuntu.com|${APT_MIRROR}|g" /etc/apt/sources.list && \
+ apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
+ build-essential \
+ cmake=3.22.1-1ubuntu1 \
+ g++-13=13.2.0-23ubuntu4 \
+ gcc-13=13.2.0-23ubuntu4 \
+ git=1:2.34.1-1ubuntu1.12 \
+ wget=1.21.2-2ubuntu1 \
+ xz-utils \
+ python3=3.10.12-1~22.04 \
+ python3-pip \
+ && rm -rf /var/lib/apt/lists/*
+
+RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-13 100 \
+ && update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-13 100
+
+# Clone ChampSim at fixed commit (supports GIT_BASE mirror)
+WORKDIR /opt
+RUN git clone ${GIT_BASE}ChampSim/ChampSim.git && \
+ cd ChampSim && \
+ git checkout 51588e1d6f97875fe8de1a3621d28668bff83fcf && \
+ git submodule update --init && \
+ ./vcpkg/bootstrap-vcpkg.sh && \
+ ./vcpkg/vcpkg install
+
+# Env vars for ChampSim builds
+ENV CXXFLAGS="-O3 -std=c++17 -fno-exceptions -fno-rtti"
+
+# Pre-build LRU baseline (caches build artifacts)
+RUN cd /opt/ChampSim && \
+ python3 config.sh config/my_policy_config.json && \
+ mkdir -p config && \
+ echo '{"executable_name":"champsim","block_size":64,"page_size":4096,"heartbeat_frequency":10000000,"num_cores":1,"ooo_cpu":[{"frequency":4000,"ifetch_buffer_size":64,"decode_buffer_size":32,"dispatch_buffer_size":32,"register_file_size":128,"rob_size":352,"lq_size":128,"sq_size":72,"fetch_width":6,"decode_width":6,"dispatch_width":6,"execute_width":4,"lq_width":2,"sq_width":2,"retire_width":5,"mispredict_penalty":1,"scheduler_size":128,"decode_latency":1,"dispatch_latency":1,"schedule_latency":0,"execute_latency":0,"branch_predictor":"bimodal","btb":"basic_btb"}],"L1I":{"sets":64,"ways":8,"rq_size":64,"wq_size":64,"pq_size":8,"mshr_size":8,"latency":4,"max_tag_check":2,"max_fill":2},"L1D":{"sets":64,"ways":12,"rq_size":64,"wq_size":64,"pq_size":8,"mshr_size":16,"latency":5,"max_tag_check":2,"max_fill":2,"prefetch_as_load":false,"virtual_prefetch":false,"replacement":"lru"},"L2C":{"sets":1024,"ways":8,"rq_size":32,"wq_size":32,"pq_size":16,"mshr_size":32,"latency":10,"max_tag_check":1,"max_fill":1,"prefetch_as_load":false,"virtual_prefetch":false,"replacement":"lru"},"LLC":{"sets":2048,"ways":16,"rq_size":32,"wq_size":32,"pq_size":32,"mshr_size":64,"latency":20,"max_tag_check":1,"max_fill":1,"prefetch_as_load":false,"virtual_prefetch":false,"replacement":"lru"}}' > config/lru_baseline.json && \
+ python3 config.sh config/lru_baseline.json && \
+ make -j4
+
+# Build the my_policy module as well
+RUN cd /opt/ChampSim && \
+ echo '{"executable_name":"champsim","block_size":64,"page_size":4096,"heartbeat_frequency":10000000,"num_cores":1,"ooo_cpu":[{"frequency":4000,"ifetch_buffer_size":64,"decode_buffer_size":32,"dispatch_buffer_size":32,"register_file_size":128,"rob_size":352,"lq_size":128,"sq_size":72,"fetch_width":6,"decode_width":6,"dispatch_width":6,"execute_width":4,"lq_width":2,"sq_width":2,"retire_width":5,"mispredict_penalty":1,"scheduler_size":128,"decode_latency":1,"dispatch_latency":1,"schedule_latency":0,"execute_latency":0,"branch_predictor":"bimodal","btb":"basic_btb"}],"L1I":{"sets":64,"ways":8,"rq_size":64,"wq_size":64,"pq_size":8,"mshr_size":8,"latency":4,"max_tag_check":2,"max_fill":2},"L1D":{"sets":64,"ways":12,"rq_size":64,"wq_size":64,"pq_size":8,"mshr_size":16,"latency":5,"max_tag_check":2,"max_fill":2,"prefetch_as_load":false,"virtual_prefetch":false,"replacement":"my_policy"},"L2C":{"sets":1024,"ways":8,"rq_size":32,"wq_size":32,"pq_size":16,"mshr_size":32,"latency":10,"max_tag_check":1,"max_fill":1,"prefetch_as_load":false,"virtual_prefetch":false,"replacement":"my_policy"},"LLC":{"sets":2048,"ways":16,"rq_size":32,"wq_size":32,"pq_size":32,"mshr_size":64,"latency":20,"max_tag_check":1,"max_fill":1,"prefetch_as_load":false,"virtual_prefetch":false,"replacement":"my_policy"}}' > config/my_policy_config.json && \
+ python3 config.sh config/my_policy_config.json && \
+ make -j4
+
+# Record environment versions
+RUN echo "gcc: $(gcc --version | head -1)" > /opt/champsim.env && \
+ echo "cmake: $(cmake --version | head -1)" >> /opt/champsim.env && \
+ echo "python: $(python3 --version)" >> /opt/champsim.env && \
+ echo "champsim: $(cd /opt/ChampSim && git rev-parse HEAD)" >> /opt/champsim.env
+
+# ── Evaluation stage ──
+FROM ubuntu:22.04
+ARG APT_MIRROR="archive.ubuntu.com"
+RUN sed -i "s|archive.ubuntu.com|${APT_MIRROR}|g" /etc/apt/sources.list && \
+ apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
+ python3=3.10.12-1~22.04 \
+ python3-pip \
+ xz-utils \
+ && rm -rf /var/lib/apt/lists/*
+
+COPY --from=champsim-builder /opt/ChampSim /opt/ChampSim
+COPY --from=champsim-builder /opt/champsim.env /opt/champsim.env
+
+RUN pip3 install --no-cache-dir pyyaml
+
+WORKDIR /benchmark
+COPY . .
+
+# Symlink ChampSim
+RUN ln -s /opt/ChampSim data/ChampSim
+
+# Download traces (continue on error)
+RUN bash data/download_traces.sh 2>/dev/null; exit 0
+
+# Generate baseline IPC
+RUN python3 verification/evaluator.py replacement/my_policy.cc --mode quick --metrics-out /tmp/init_check.json 2>/dev/null; exit 0
+
+CMD ["python3", "verification/evaluator.py", "replacement/my_policy.cc", "--mode", "full"]
diff --git a/benchmarks/ComputerSystems/CacheReplacement/verification/evaluator.py b/benchmarks/ComputerSystems/CacheReplacement/verification/evaluator.py
new file mode 100644
index 00000000..25fec24f
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/verification/evaluator.py
@@ -0,0 +1,223 @@
+"""
+CacheReplacement evaluator — 4-layer pipeline for ChampSim cache policy optimization.
+"""
+
+import argparse, json, math, os, re, shutil, subprocess, sys, time, traceback
+from pathlib import Path
+
+BENCHMARK_VERSION = "1.0.0"
+CHAMPSIM_COMMIT = "51588e1d6f97875fe8de1a3621d28668bff83fcf"
+STORAGE_BUDGET_BYTES = 65536
+TIMEOUT_COMPILE = 300; TIMEOUT_QUICK_VERIFY = 120
+TIMEOUT_PER_TRACE_QUICK = 600; TIMEOUT_PER_TRACE_FULL = 1800
+WARMUP_QUICK = 10_000_000; SIM_QUICK = 20_000_000
+WARMUP_FULL = 50_000_000; SIM_FULL = 100_000_000
+
+FORBIDDEN_HEADERS = ["iostream","fstream","sstream","thread","mutex","future","shared_mutex","filesystem","unistd.h","fcntl.h","sys/stat.h","sys/mman.h","dlfcn.h","execinfo.h","signal.h","setjmp.h"]
+FORBIDDEN_APIS = ["fopen","fread","fwrite","fclose","open(","read(","write(","close(","mmap(","munmap(","dlopen","dlsym","system(","popen","fork","pthread_","std::thread","std::async"]
+FORBIDDEN_RANDOM = ["rand()","srand(","random_device","mt19937","default_random_engine","steady_clock","system_clock","time(","clock("]
+
+# Layer 1: Source Validation
+
+def _static_validation(cc_path: Path, h_path: Path | None) -> list[str]:
+ errors = []
+ cc_text = cc_path.read_text()
+ if "// EVOLVE-BLOCK-START" not in cc_text: errors.append("Missing '// EVOLVE-BLOCK-START' in .cc file")
+ if "// EVOLVE-BLOCK-END" not in cc_text: errors.append("Missing '// EVOLVE-BLOCK-END' in .cc file")
+ s = cc_text.find("// EVOLVE-BLOCK-START"); e = cc_text.find("// EVOLVE-BLOCK-END")
+ if s >= 0 and e >= 0 and s >= e: errors.append("EVOLVE-BLOCK-START must precede EVOLVE-BLOCK-END")
+ for h in FORBIDDEN_HEADERS:
+ if f"#include <{h}>" in cc_text: errors.append(f"Forbidden header: <{h}>")
+ for api in FORBIDDEN_APIS:
+ if api in cc_text: errors.append(f"Forbidden API: {api}")
+ for rf in FORBIDDEN_RANDOM:
+ if rf in cc_text: errors.append(f"Non-deterministic function: {rf}")
+ dm_patterns = [r'\bnew\s+\w+', r'\bdelete\s+\[\]', r'\bdelete\s+\w+', r'\bmalloc\s*\(', r'\bfree\s*\(', r'\bcalloc\s*\(', r'\brealloc\s*\(']
+ for pat in dm_patterns:
+ if re.search(pat, cc_text): errors.append(f"Dynamic memory: {pat}")
+ for dc in ["std::vector","std::map","std::unordered_map","std::set","std::list","std::deque"]:
+ if dc in cc_text: errors.append(f"Dynamic container: {dc}")
+ if h_path and h_path.is_file():
+ h_text = h_path.read_text()
+ hs = h_text.find("// EVOLVE-BLOCK-START"); he = h_text.find("// EVOLVE-BLOCK-END")
+ if hs >= 0 and he >= 0:
+ total = 0; type_sizes = {"uint8_t":1,"int8_t":1,"uint16_t":2,"int16_t":2,"uint32_t":4,"int32_t":4,"uint64_t":8,"int64_t":8,"bool":1,"char":1,"float":4,"double":8}
+ block = h_text[hs+len("// EVOLVE-BLOCK-START"):he]
+ for m in re.finditer(r'(uint\d+_t|int\d+_t|bool|char|float|double)\s+\w+\[(\d+)\]', block):
+ total += type_sizes.get(m.group(1),4) * int(m.group(2))
+ for m in re.finditer(r'std::array\s*<\s*(\w+)\s*,\s*(\d+)\s*>', block):
+ total += type_sizes.get(m.group(1),4) * int(m.group(2))
+ if total > STORAGE_BUDGET_BYTES:
+ errors.append(f"Storage budget exceeded: ~{total} B > {STORAGE_BUDGET_BYTES} B (approximate check)")
+ return errors
+
+# Layer 2: Compile (cross-platform)
+
+def _build_champsim(cd: Path, config_name: str = "my_policy_config") -> bool:
+ """Configure and build ChampSim. Handles vcpkg on macOS and Linux."""
+ cfg = cd / "config" / f"{config_name}.json"
+ # Step 1: Run config.sh (generates Makefile + .csconfig)
+ p1 = subprocess.run([sys.executable, str(cd/"config.sh"), str(cfg)], cwd=cd, capture_output=True, text=True, timeout=60)
+ if p1.returncode != 0: return False
+ # Step 2: Ensure vcpkg include path is in absolute.options
+ # On Linux config.sh handles this; on macOS it may not.
+ abs_opts = cd / "absolute.options"
+ if abs_opts.is_file():
+ vcpkg_incs = [str(p) for p in sorted((cd / "vcpkg_installed").glob("*/include"))]
+ for inc in vcpkg_incs:
+ if inc not in abs_opts.read_text():
+ with open(abs_opts, "a") as f:
+ f.write(f" -isystem {inc}")
+ # Step 3: Build
+ p2 = subprocess.run(["make", "-j4"], cwd=cd, capture_output=True, text=True, timeout=TIMEOUT_COMPILE)
+ return p2.returncode == 0
+
+def _ensure_config(cd: Path, name: str, replacement: str):
+ cfg = {"executable_name":"champsim","block_size":64,"page_size":4096,"heartbeat_frequency":10000000,"num_cores":1,
+ "ooo_cpu":[{"frequency":4000,"ifetch_buffer_size":64,"decode_buffer_size":32,"dispatch_buffer_size":32,"register_file_size":128,"rob_size":352,"lq_size":128,"sq_size":72,"fetch_width":6,"decode_width":6,"dispatch_width":6,"execute_width":4,"lq_width":2,"sq_width":2,"retire_width":5,"mispredict_penalty":1,"scheduler_size":128,"decode_latency":1,"dispatch_latency":1,"schedule_latency":0,"execute_latency":0,"branch_predictor":"bimodal","btb":"basic_btb"}],
+ "L1I":{"sets":64,"ways":8,"rq_size":64,"wq_size":64,"pq_size":8,"mshr_size":8,"latency":4,"max_tag_check":2,"max_fill":2},
+ "L1D":{"sets":64,"ways":12,"rq_size":64,"wq_size":64,"pq_size":8,"mshr_size":16,"latency":5,"max_tag_check":2,"max_fill":2,"prefetch_as_load":False,"virtual_prefetch":False,"replacement":replacement},
+ "L2C":{"sets":1024,"ways":8,"rq_size":32,"wq_size":32,"pq_size":16,"mshr_size":32,"latency":10,"max_tag_check":1,"max_fill":1,"prefetch_as_load":False,"virtual_prefetch":False,"replacement":replacement},
+ "LLC":{"sets":2048,"ways":16,"rq_size":32,"wq_size":32,"pq_size":32,"mshr_size":64,"latency":20,"max_tag_check":1,"max_fill":1,"prefetch_as_load":False,"virtual_prefetch":False,"replacement":replacement}}
+ (cd/"config").mkdir(exist_ok=True); (cd/"config"/f"{name}.json").write_text(json.dumps(cfg,indent=2))
+
+# Layer 3: Quick Verify (uses verify_trace.xz, ~2M instructions, ~25s)
+
+def _quick_verify(binary: Path, verify_trace: Path) -> tuple[bool, str]:
+ """Quick sanity check. NOT used for scoring."""
+ cmd = [str(binary), "--warmup-instructions", "1000000", "--simulation-instructions", "2000000", str(verify_trace)]
+ p = subprocess.run(cmd, capture_output=True, text=True, timeout=TIMEOUT_QUICK_VERIFY)
+ if p.returncode != 0: return False, p.stderr[:500]
+ ipc = _parse_ipc(p.stdout)
+ return (True, p.stdout) if ipc and ipc > 0 else (False, f"Invalid IPC: {ipc}")
+
+# Layer 4: Full Evaluation
+
+def _run_trace(binary: Path, trace_path: Path, warmup: int, sim: int, timeout: int) -> dict:
+ cmd = [str(binary), "--warmup-instructions", str(warmup), "--simulation-instructions", str(sim), str(trace_path)]
+ try:
+ p = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout)
+ if p.returncode != 0: return {"status":"failed","error":p.stderr[:200]}
+ ipc = _parse_ipc(p.stdout)
+ return ({"status":"ok","ipc":ipc} if ipc and ipc > 0 else {"status":"failed","error":f"invalid IPC: {ipc}"})
+ except subprocess.TimeoutExpired: return {"status":"timeout"}
+
+def _parse_ipc(output: str) -> float | None:
+ for line in output.splitlines():
+ m = re.search(r'cumulative IPC:\s+([\d.]+)', line)
+ if m:
+ try: return float(m.group(1))
+ except ValueError: return None
+ return None
+
+def _get_baseline_ipcs(base_dir: Path, traces: list[str], warmup: int, sim: int) -> dict:
+ ref_dir = base_dir / "references"; cache_file = ref_dir / "generated_baseline.json"
+ if cache_file.is_file():
+ data = json.loads(cache_file.read_text())
+ if data.get("champsim_commit") == CHAMPSIM_COMMIT: return data.get("ipc_results", {})
+ cd = base_dir / "data" / "ChampSim"
+ _ensure_config(cd, "lru_baseline", "lru")
+ if not _build_champsim(cd, "lru_baseline"): raise RuntimeError("Failed to build LRU baseline")
+ binary = cd / "bin" / "champsim"; td = base_dir / "data" / "traces"
+ to = TIMEOUT_PER_TRACE_FULL if sim == SIM_FULL else TIMEOUT_PER_TRACE_QUICK
+ results = {}
+ for t in traces:
+ tf = td / f"{t}.champsimtrace.xz"
+ if not tf.is_file(): continue
+ r = _run_trace(binary, tf, warmup, sim, to)
+ if r["status"] == "ok": results[t] = r["ipc"]
+ ref_dir.mkdir(parents=True, exist_ok=True)
+ cache_file.write_text(json.dumps({"benchmark_version":BENCHMARK_VERSION,"champsim_commit":CHAMPSIM_COMMIT,"warmup":warmup,"simulation":sim,"ipc_results":results},indent=2))
+ return results
+
+# Main evaluate
+
+def evaluate(candidate_path: str, *, mode: str = "quick", repo_root: Path | None = None) -> dict:
+ start = time.time()
+ metrics = {"benchmark_version":BENCHMARK_VERSION,"simulator_commit":CHAMPSIM_COMMIT,"valid":0.0,"score":0.0,"runtime_s":0.0,"compile_time_s":0.0,"simulation_time_s":0.0,"l1_static_pass":0.0,"l2_compile_pass":0.0,"l3_quick_verify_pass":0.0,"l4_eval_pass":0.0,"num_traces_completed":0,"num_traces_total":0,"mode":mode,"error":None}
+ artifacts = {"per_trace_results":{}}
+ try:
+ repo = repo_root or _find_repo_root()
+ base = repo / "benchmarks" / "ComputerSystems" / "CacheReplacement"
+ cand = Path(candidate_path).expanduser().resolve()
+ artifacts["candidate_path"] = str(cand)
+ if mode == "quick":
+ traces = ["605.mcf_s-665B","625.x264_s-18B","607.cactuBSSN_s-2421B"]
+ warmup, sim, tto = WARMUP_QUICK, SIM_QUICK, TIMEOUT_PER_TRACE_QUICK
+ else:
+ traces = ["603.bwaves_s-3699B","605.mcf_s-665B","625.x264_s-18B","654.roms_s-842B","657.xz_s-3167B","600.perlbench_s-210B","602.gcc_s-734B","607.cactuBSSN_s-2421B","621.wrf_s-575B","631.deepsjeng_s-928B"]
+ warmup, sim, tto = WARMUP_FULL, SIM_FULL, TIMEOUT_PER_TRACE_FULL
+ if not cand.is_file(): metrics["error"] = "Candidate not found"; return _wrap(metrics, artifacts)
+ h_path = (cand.parent / f"{cand.stem}.h") if (cand.parent / f"{cand.stem}.h").is_file() else cand.with_suffix(".h")
+ se = _static_validation(cand, h_path if h_path.is_file() else None)
+ artifacts["static_errors"] = se
+ if se: metrics["error"] = f"Static validation: {'; '.join(se)}"; return _wrap(metrics, artifacts)
+ metrics["l1_static_pass"] = 1.0
+ cd = base / "data" / "ChampSim"; md = cd / "replacement" / "my_policy"
+ md.mkdir(parents=True, exist_ok=True)
+ shutil.copy2(str(cand), str(md / "my_policy.cc"))
+ if h_path.is_file(): shutil.copy2(str(h_path), str(md / "my_policy.h"))
+ else: (md / "my_policy.h").write_text('#ifndef REPLACEMENT_MY_POLICY_H\n#define REPLACEMENT_MY_POLICY_H\n#include \n#include "cache.h"\n#include "modules.h"\nstruct my_policy;\n#endif\n')
+ t0 = time.perf_counter()
+ _ensure_config(cd, "my_policy_config", "my_policy")
+ if not _build_champsim(cd, "my_policy_config"): metrics["error"] = "Compilation failed"; return _wrap(metrics, artifacts)
+ metrics["compile_time_s"] = time.perf_counter() - t0; metrics["l2_compile_pass"] = 1.0
+ binary = cd / "bin" / "champsim"
+ vt = base / "verification" / "verify_trace.xz"
+ if vt.is_file():
+ qok, qlog = _quick_verify(binary, vt); artifacts["quick_verify_log"] = qlog
+ if not qok: metrics["error"] = "Quick verification failed"; return _wrap(metrics, artifacts)
+ metrics["l3_quick_verify_pass"] = 1.0
+ baseline_ipcs = _get_baseline_ipcs(base, traces, warmup, sim)
+ if not baseline_ipcs: metrics["error"] = "Baseline generation failed"; return _wrap(metrics, artifacts)
+ td = base / "data" / "traces"; metrics["num_traces_total"] = float(len(traces))
+ t1 = time.perf_counter(); results = {}
+ for trace_name in traces:
+ tf = td / f"{trace_name}.champsimtrace.xz"
+ results[trace_name] = _run_trace(binary, tf, warmup, sim, tto) if tf.is_file() else {"status":"failed","error":"trace not found"}
+ metrics["simulation_time_s"] = time.perf_counter() - t1; artifacts["per_trace_results"] = results
+ completed = {t:r for t,r in results.items() if r.get("status")=="ok"}
+ metrics["num_traces_completed"] = float(len(completed))
+ if completed:
+ ratios = []
+ for t,r in completed.items():
+ bl = baseline_ipcs.get(t)
+ if bl and bl > 0 and r.get("ipc",0) > 0: ratios.append(r["ipc"]/bl)
+ if ratios:
+ metrics["score"] = math.exp(sum(math.log(max(x,0.001)) for x in ratios)/len(ratios))
+ metrics["valid"] = 1.0; metrics["l4_eval_pass"] = 1.0
+ else: metrics["error"] = "No valid trace results"
+ else: metrics["error"] = "No traces completed"
+ except subprocess.TimeoutExpired as exc: metrics["error"] = f"Timeout: {exc}"
+ except Exception as exc: metrics["error"] = str(exc); artifacts["traceback"] = traceback.format_exc()
+ finally: metrics["runtime_s"] = time.time() - start
+ return _wrap(metrics, artifacts)
+
+def _find_repo_root() -> Path:
+ here = Path(__file__).resolve()
+ for p in [here.parent, *here.parents]:
+ if (p / "benchmarks").is_dir() and (p / "frontier_eval").is_dir(): return p
+ return Path.cwd()
+
+def _wrap(m: dict, a: dict) -> dict:
+ try:
+ from openevolve.evaluation_result import EvaluationResult as ER
+ return ER(metrics=m, artifacts=a)
+ except ImportError: return {"metrics": m, "artifacts": a}
+
+def main() -> int:
+ p = argparse.ArgumentParser(description="CacheReplacement: evaluate a cache replacement policy via ChampSim simulation.")
+ p.add_argument("candidate", help="Path to the candidate my_policy.cc file (e.g., replacement/my_policy.cc)")
+ p.add_argument("--mode", choices=["quick", "full"], default="quick",
+ help="'quick' (3 traces, ~15 min) or 'full' (10 traces, ~30-60 min)")
+ p.add_argument("--metrics-out", default="", help="Optional path to write metrics JSON")
+ p.add_argument("--artifacts-out", default="", help="Optional path to write artifacts JSON")
+ args = p.parse_args()
+ result = evaluate(args.candidate, mode=args.mode)
+ r = result if isinstance(result,dict) else {"metrics":dict(result.metrics or {}),"artifacts":dict(result.artifacts or {})}
+ if args.metrics_out: Path(args.metrics_out).write_text(json.dumps(r["metrics"],indent=2))
+ if args.artifacts_out: Path(args.artifacts_out).write_text(json.dumps(r["artifacts"],indent=2,default=str))
+ print(json.dumps(r["metrics"],indent=2))
+ return 0
+
+if __name__ == "__main__": raise SystemExit(main())
diff --git a/benchmarks/ComputerSystems/CacheReplacement/verification/requirements.txt b/benchmarks/ComputerSystems/CacheReplacement/verification/requirements.txt
new file mode 100644
index 00000000..3aecde93
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/verification/requirements.txt
@@ -0,0 +1 @@
+pyyaml>=6.0
diff --git a/benchmarks/ComputerSystems/CacheReplacement/verification/test_validation.py b/benchmarks/ComputerSystems/CacheReplacement/verification/test_validation.py
new file mode 100644
index 00000000..247a0259
--- /dev/null
+++ b/benchmarks/ComputerSystems/CacheReplacement/verification/test_validation.py
@@ -0,0 +1,55 @@
+"""CacheReplacement Layer 1 validation tests (fast, no compile needed)."""
+
+import sys, tempfile, json
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+sys.path.insert(0, str(HERE))
+import evaluator as ev
+
+HEADER = """#ifndef REPLACEMENT_MY_POLICY_H
+#define REPLACEMENT_MY_POLICY_H
+#include
+#include "cache.h"
+#include "modules.h"
+struct my_policy : public champsim::modules::replacement {
+ long NUM_WAY; std::vector last_used_cycles; uint64_t cycle = 0;
+ // EVOLVE-BLOCK-START
+ // EVOLVE-BLOCK-END
+ my_policy(CACHE*); my_policy(CACHE*,long,long);
+ long find_victim(uint32_t,uint64_t,long,const champsim::cache_block*,champsim::address,champsim::address,access_type);
+ void replacement_cache_fill(uint32_t,long,long,champsim::address,champsim::address,champsim::address,access_type);
+ void update_replacement_state(uint32_t,long,long,champsim::address,champsim::address,champsim::address,access_type,uint8_t);
+};
+#endif"""
+
+PREFIX = '#include "my_policy.h"\n#include \n#include \nmy_policy::my_policy(CACHE*c):my_policy(c,c->NUM_SET,c->NUM_WAY){}\nmy_policy::my_policy(CACHE*c,long s,long w):replacement(c),NUM_WAY(w),last_used_cycles(static_cast(s*w),0){}\n// EVOLVE-BLOCK-START\n'
+SUFFIX = "\n// EVOLVE-BLOCK-END\n"
+def cc(body): return PREFIX + body + SUFFIX
+
+def test(name, cc_text, hdr_text, expect_errors):
+ with tempfile.TemporaryDirectory() as td:
+ p = Path(td); (p/"my_policy.cc").write_text(cc_text); (p/"my_policy.h").write_text(hdr_text or HEADER)
+ errs = ev._static_validation(p/"my_policy.cc", p/"my_policy.h")
+ ok = (len(errs) > 0) == expect_errors
+ detail = errs[0][:80] if errs else "clean"
+ print(f" [{'PASS' if ok else 'FAIL'}] {name}: {detail}")
+ return ok
+
+all_pass = True
+print("="*50+"\nCacheReplacement — Static Validation Tests\n"+"="*50+"\n")
+
+all_pass &= test("1. Normal LRU", cc("long my_policy::find_victim(...){return 0;}void my_policy::replacement_cache_fill(...){}void my_policy::update_replacement_state(...){}"), HEADER, False)
+all_pass &= test("2. Forbidden header ", cc('#include \nlong my_policy::find_victim(...){}'), HEADER, True)
+all_pass &= test("3. rand() usage", cc("long my_policy::find_victim(...){return rand()%4;}"), HEADER, True)
+all_pass &= test("4. fopen() file I/O", cc('long my_policy::find_victim(...){FILE*f=fopen("x","r");return 0;}'), HEADER, True)
+all_pass &= test("5. std::vector in EVOLVE-BLOCK", cc("std::vector v;\nlong my_policy::find_victim(...){return 0;}"), HEADER, True)
+all_pass &= test("6. new operator", cc("long my_policy::find_victim(...){int*p=new int;return 0;}"), HEADER, True)
+all_pass &= test("7. mt19937 randomness", cc("long my_policy::find_victim(...){std::mt19937 g;return 0;}"), HEADER, True)
+all_pass &= test("8. system() call", cc('long my_policy::find_victim(...){system("ls");return 0;}'), HEADER, True)
+all_pass &= test("9. Missing EVOLVE-BLOCK-END", "// EVOLVE-BLOCK-START\n", HEADER, True)
+all_pass &= test("10. Budget exceeded", cc("long my_policy::find_victim(...){return 0;}"),
+ HEADER.replace("// EVOLVE-BLOCK-START\n // EVOLVE-BLOCK-END","// EVOLVE-BLOCK-START\nuint8_t huge[100000];\n// EVOLVE-BLOCK-END"), True)
+
+print(f"\n{'ALL TESTS PASSED' if all_pass else 'SOME FAILED'}")
+sys.exit(0 if all_pass else 1)
diff --git a/benchmarks/ComputerSystems/CacheReplacement/verification/verify_trace.xz b/benchmarks/ComputerSystems/CacheReplacement/verification/verify_trace.xz
new file mode 100644
index 00000000..9f505423
Binary files /dev/null and b/benchmarks/ComputerSystems/CacheReplacement/verification/verify_trace.xz differ
diff --git a/benchmarks/ComputerSystems/README.md b/benchmarks/ComputerSystems/README.md
index 411962ac..8ad7b593 100644
--- a/benchmarks/ComputerSystems/README.md
+++ b/benchmarks/ComputerSystems/README.md
@@ -3,5 +3,6 @@
Includes computer-systems engineering optimization tasks:
- `MallocLab`: dynamic memory allocation.
- `DuckDBWorkloadOptimization`: analytical SQL workload tuning (index/materialized-view selection + query rewrite).
+- `CacheReplacement`: CPU cache replacement policy optimization. (Replaces the deleted `IndexOptimization`, which overlapped with `DuckDBWorkloadOptimization`.)
Note for contributors: ensure the evolved baseline source file contains `EVOLVE-BLOCK-START` / `EVOLVE-BLOCK-END` markers (use `// ...` in C/C++).
diff --git a/benchmarks/ComputerSystems/README_zh-CN.md b/benchmarks/ComputerSystems/README_zh-CN.md
index 0d12dfc1..364d6188 100644
--- a/benchmarks/ComputerSystems/README_zh-CN.md
+++ b/benchmarks/ComputerSystems/README_zh-CN.md
@@ -3,5 +3,6 @@
包含以下计算机系统工程优化任务：
- `MallocLab`：动态内存分配。
- `DuckDBWorkloadOptimization`：分析型 SQL 负载调优（索引/物化视图选择 + 查询改写）。
+- `CacheReplacement`：CPU 缓存替换策略优化。（替代已删除的 `IndexOptimization`，因其与 `DuckDBWorkloadOptimization` 功能重叠。）
贡献提示：请确保被 evolve 的 baseline 源码文件包含 `EVOLVE-BLOCK-START` / `EVOLVE-BLOCK-END` 标记（C/C++ 中使用 `// ...`）。
diff --git a/frontier_eval/conf/batch/v1_lite.yaml b/frontier_eval/conf/batch/v1_lite.yaml
new file mode 100644
index 00000000..284cc45a
--- /dev/null
+++ b/frontier_eval/conf/batch/v1_lite.yaml
@@ -0,0 +1,83 @@
+# Frontier-Eng v1-lite batch matrix.
+#
+# v1-lite is a 10-task representative subset of the full 47-task v1 set, chosen
+# for fast iteration and ablation. Tasks are selected to (a) cover all five
+# engineering categories with distinct benchmark families and (b) reward
+# incremental optimization -- their best-feasible scores climb gradually with
+# budget instead of saturating in one shot or splitting all-or-nothing. The
+# selection rule (top-graduality task per family, 2 per category) is documented
+# and reproduced by Einsia-lab/scripts/build_frontier_metrics.py.
+#
+# Overrides for each task mirror frontier_eval/conf/batch/v1.yaml exactly.
+#
+version: 1
+tasks:
+- name: unified
+ label: ComputerSystems/MallocLab
+ overrides:
+ - task.benchmark=ComputerSystems/MallocLab
+- name: unified
+ label: QuantumComputing/task_01_routing_qftentangled
+ overrides:
+ - task.benchmark=QuantumComputing/task_01_routing_qftentangled
+ - task.runtime.env_name=frontier-v1-main
+- name: unified
+ label: JobShop/abz
+ overrides:
+ - task.benchmark=JobShop/abz
+ - task.runtime.python_path=uv-env:frontier-v1-main
+- name: unified
+ label: InventoryOptimization/disruption_eoqd
+ overrides:
+ - task.benchmark=InventoryOptimization/disruption_eoqd
+ - task.runtime.env_name=frontier-v1-main
+- name: unified
+ label: EnergyStorage/BatteryFastChargingSPMe
+ overrides:
+ - task.benchmark=EnergyStorage/BatteryFastChargingSPMe
+- name: unified
+ label: Robotics/RobotArmCycleTimeOptimization
+ overrides:
+ - task.benchmark=Robotics/RobotArmCycleTimeOptimization
+ - task.runtime.env_name=frontier-v1-main
+ - algorithm.oe.evaluator.timeout=600
+- name: unified
+ label: Optics/holographic_multiplane_focusing
+ overrides:
+ - task.benchmark=Optics/holographic_multiplane_focusing
+ - task.runtime.env_name=frontier-v1-main
+ - algorithm.oe.evaluator.timeout=600
+- name: unified
+ label: WirelessChannelSimulation/HighReliableSimulation
+ overrides:
+ - task.benchmark=WirelessChannelSimulation/HighReliableSimulation
+- name: unified
+ label: ReactionOptimisation/snar_multiobjective
+ overrides:
+ - task.benchmark=ReactionOptimisation/snar_multiobjective
+ - task.runtime.python_path=uv-env:frontier-v1-summit
+ - algorithm.oe.evaluator.timeout=600
+- name: unified
+ label: StructuralOptimization/TopologyOptimization
+ overrides:
+ - task.benchmark=StructuralOptimization/TopologyOptimization
+ - task.runtime.env_name=frontier-v1-main
+algorithms:
+- name: openevolve
+ overrides:
+ - algorithm.iterations=100
+ - algorithm.oe.evaluator.timeout=300
+llms:
+- name: v1
+ api_base: ${oc.env:OPENAI_API_BASE,https://api.openai.com/v1}
+ model: ${oc.env:OPENAI_MODEL,gpt-4o-mini}
+ api_key_env: OPENAI_API_KEY
+llm_config: openai_compatible
+common_overrides:
+- llm.temperature=0.7
+- llm.timeout=60
+run:
+ name: v1_lite
+ base_dir: runs/batch
+ max_parallel: 4
+ fail_fast: false
diff --git a/frontier_eval/conf/task/database_index_optimization.yaml b/frontier_eval/conf/task/database_index_optimization.yaml
new file mode 100644
index 00000000..9cf1abca
--- /dev/null
+++ b/frontier_eval/conf/task/database_index_optimization.yaml
@@ -0,0 +1,5 @@
+name: unified
+benchmark: ComputerSystems/IndexOptimization
+runtime:
+ isolation_mode: docker
+ docker_image: frontier-pg-index:latest
diff --git a/leaderboard/README.md b/leaderboard/README.md
new file mode 100644
index 00000000..25e039b9
--- /dev/null
+++ b/leaderboard/README.md
@@ -0,0 +1,62 @@
+# Leaderboard & Medal Score
+
+Released score artifacts for the Frontier-Eng `v1` set (Experiment 1: foundation
+models under `openevolve`, 100 iterations, same initial programs and frozen
+verifiers; `gpt-5.4` uses its full 47-task retest).
+
+| File | Contents |
+|---|---|
+| `medal_podium.csv` | Frozen per-task **gold / silver / bronze** threshold scores and the model that set each. |
+| `medal_leaderboard.csv` | Per-model normalized **Medal Score** on v1 and v1-lite, with gold/silver/bronze counts. |
+| `exp1_models_raw.csv` | Best-feasible score of each model on each of the 47 tasks (higher is better); source of the podium. |
+| `score_submission.py` | Scores a new submission against the frozen podium. |
+| `submission_example.csv` | Example submission (claude-opus-4.6) — scoring it reproduces its leaderboard line. |
+
+## Medal Score
+
+On each task the top-3 best scores in the **v1 snapshot (2026-04-14)** are frozen
+as peer baselines — gold (1st), silver (2nd), bronze (3rd). A model earns
+**1.00** for reaching the gold score, **0.67** for silver, **0.33** for bronze,
+otherwise 0; its Medal Score is the **mean** of this credit over a task set
+(normalized to `[0,1]`). It credits only reaching each task's frontier (the
+podium) and ignores negligible margins in the long tail — a fairer aggregate
+than crediting every ordinal rank when the question is "how often does a model
+reach the best-known solutions?" We report it on both the full **v1** set
+(47 tasks) and the **v1-lite** subset (10 tasks). (Average rank and other
+diagnostics are on the [website leaderboard](https://lab.einsia.ai/frontier-eng/leaderboard).)
+
+> `gpt-oss-120b` is part of the paper's 9-model rank tables, but its per-task raw
+> scores were not retained; the released podium is therefore computed over the 8
+> models with available raw scores.
+
+## Medal leaderboard (normalized; gold/silver/bronze counts are for v1)
+
+| Rank | Model | Medal (v1) | Medal (v1-lite) | 🥇 | 🥈 | 🥉 |
+| :--: | :--- | --: | --: | --: | --: | --: |
+| 1 | gpt-5.4 | 0.596 | 0.667 | 24 | 5 | 2 |
+| 2 | claude-opus-4.6 | 0.490 | 0.501 | 9 | 18 | 6 |
+| 3 | glm-5 | 0.312 | 0.233 | 4 | 10 | 12 |
+| 4 | deepseek-v3.2 | 0.248 | 0.166 | 3 | 9 | 8 |
+| 5 | gemini-3.1-pro-preview | 0.213 | 0.200 | 3 | 6 | 9 |
+| 6 | seed-2.0-pro | 0.185 | 0.100 | 3 | 7 | 3 |
+| 7 | grok-4.20 | 0.184 | 0.133 | 3 | 6 | 5 |
+| 8 | qwen3-coder-next | 0.121 | 0.000 | 3 | 3 | 2 |
+
+## Score your own model
+
+Put your model's best score per task in a CSV (`Task,Score`, one row per task,
+task names as in `medal_podium.csv`), then:
+
+```bash
+python leaderboard/score_submission.py your_scores.csv
+# -> Medal Score (v1, 47 tasks) : 0.xxx (gold .., silver .., bronze ..)
+# Medal Score (v1-lite, 10 tasks) : 0.xxx
+```
+
+Sanity check (reproduces claude-opus-4.6's line, 0.490 / 0.501):
+
+```bash
+python leaderboard/score_submission.py leaderboard/submission_example.csv
+```
+
+Interactive view: [lab.einsia.ai/frontier-eng/leaderboard](https://lab.einsia.ai/frontier-eng/leaderboard)
diff --git a/leaderboard/exp1_models_raw.csv b/leaderboard/exp1_models_raw.csv
new file mode 100644
index 00000000..1b720953
--- /dev/null
+++ b/leaderboard/exp1_models_raw.csv
@@ -0,0 +1,48 @@
+Task,Baseline,claude-opus-4.6_best,deepseek-v3.2_best,gemini-3.1-pro-preview_best,glm-5_best,gpt-5.4_best,grok-4.20_best,qwen3-coder-next_best,seed-2.0-pro_best,,,,,,,,,,
+Aerodynamics_CarAerodynamicsSensing,0.9617,0.9624,0.9632,0.9632,0.9628,0.9630695838481188,0.9624,0.9632,0.9624,,,,,,,,,,
+Astrodynamics_MannedLunarLanding,4577.437,6027.3126,6079.2455,4674.9462,6839.0331,6660.942428,4577.437,4577.437,4733.0435,,,,,,,,,,
+ComputerSystems_MallocLab,28,96,53,48,86,28,57,32,38,,,,,,,,,,
+Cryptographic_AES-128,7.5209,11.8617,12.4591,10.2396,7.9669,39.824967043300866,10.8615,5.5501,7.9481,,,,,,,,,,
+Cryptographic_SHA-256,9.8274,16.7955,9.718,9.942,15.1655,26.34045367870492,17.2504,9.8475,15.2838,,,,,,,,,,
+Cryptographic_SHA3-256,16.0932,17.4003,17.0749,16.2255,17.5778,37.44512785396786,16.0594,16.5292,18.3478,,,,,,,,,,
+EnergyStorage_BatteryFastChargingProfile,71.2806,120.8025,111.4518,116.6532,118.7678,121.99136502281442,99.6875,89.8416,115.6882,,,,,,,,,,
+EnergyStorage_BatteryFastChargingSPMe,66.1636,71.8225,91.0079,92.3198,78.0896,122.94304361063023,76.4657,79.0273,76.4122,,,,,,,,,,
+EngDesign,1.3571,1.3571,21.7143,27,25.5714,1.3571428571428572,27,25.5714,27,,,,,,,,,,
+InventoryOptimization_disruption_eoqd,0.3642,0.6473,0.6381,0.639,0.6303,1,0.6359,0.6225,0.6321,,,,,,,,,,
+InventoryOptimization_finite_horizon_dp,0.3673,0.9596,0.8025,0.7559,0.7965,0.9606835281410351,0.8547,0.4413,0.7323,,,,,,,,,,
+InventoryOptimization_general_meio,0.1825,0.9929,0.9893,0.9839,0.9165,0.9999999999999999,0.9236,0.7819,0.6973,,,,,,,,,,
+InventoryOptimization_joint_replenishment,0.3034,0.8822,0.8822,0.8822,0.8822,1,0.8822,0.8821,0.8822,,,,,,,,,,
+InventoryOptimization_tree_gsm_safety_stock,0.3813,0.75,0.6606,0.6606,0.6606,1,0.6606,0.6606,0.6606,,,,,,,,,,
+JobShop_abz,80.5042,96.1035,88.3614,86.751,88.4924,91.23143065488635,87.6717,85.603,86.672,,,,,,,,,,
+JobShop_swv,81.6325,89.4966,82.3575,82.3141,87.1611,87.33430826602005,85.5068,82.6129,82.4153,,,,,,,,,,
+JobShop_ta,78.8,90.8322,84.9043,85.7065,86.8095,86.16070055174835,84.9136,85.5489,83.9694,,,,,,,,,,
+KernelEngineering_FlashAttention,55.2957,983.5001,987.2034,991.8896,381.6257,182687.44188255747,324.919,525.5567,1218.5163,,,,,,,,,,
+KernelEngineering_MLA,0.7828,1000.3859,0.8936,1253.2017,20.1972,1132.0659025372765,19.8651,0.9271,19.987,,,,,,,,,,
+KernelEngineering_TriMul,47.1274,357.1636,85.5923,54.5774,110.8785,47.88292233116043,165.0294,49.1232,84.9069,,,,,,,,,,
+Optics_adaptive_fault_tolerant_fusion,0.3959,0.6398,0.64,0.6398,0.6398,0.455046169,0.6398,0.6398,0.6398,,,,,,,,,,
+Optics_adaptive_temporal_smooth_control,0.3152,0.8419,0.8419,0.8419,0.8417,0.841880414,0.842,0.8421,0.8421,,,,,,,,,,
+Optics_fiber_guardband_spectrum_packing,0.3861,0.6692,0.657,0.6629,0.6692,0.6754289215686274,0.6629,0.657,0.657,,,,,,,,,,
+Optics_fiber_mcs_power_scheduling,0.3297,0.6542,0.5182,0.4796,0.6491,0.6608370951757289,0.4557,0.4458,0.6491,,,,,,,,,,
+Optics_fiber_wdm_channel_power_allocation,0.3255,0.6675,0.6679,0.6619,0.6686,0.6964207451370852,0.6664,0.6666,0.6654,,,,,,,,,,
+Optics_holographic_multifocus_power_ratio,0.3927,0.8072,0.8265,0.5368,0.711,0.9999999999663148,0.4058,0.5875,0.5626,,,,,,,,,,
+Optics_holographic_multiplane_focusing,0.3302,0.6002,0.7196,0.4398,0.4516,0.9999999999886867,0.474,0.5631,0.5303,,,,,,,,,,
+Optics_phase_dammann_uniform_orders,26.8969,99.7995,97.3436,97.9498,97.8709,99.99999999999999,94.4055,95.9998,69.0576,,,,,,,,,,
+Optics_phase_fourier_pattern_holography,32.6457,82.1276,74.5838,76.6371,76.0127,99.99998936790779,74.217,67.3393,72.4578,,,,,,,,,,
+PyPortfolioOpt_robust_mvo_rebalance,32.9804,99.9946,84.941,77.165,82.8015,99.99460428985267,99.983,85.5194,83.0681,,,,,,,,,,
+QuantumComputing_task_01_routing_qftentangled,0.209,5.0479,3.6155,0.209,3.7681,6.507945106686525,3.7655,3.2471,3.6783,,,,,,,,,,
+QuantumComputing_task_02_clifford_t_synthesis,1.7134,1.6633,1.7134,1.7134,7.4236,1.7133669376223557,1.6633,1.7134,1.7134,,,,,,,,,,
+QuantumComputing_task_03_cross_target_qaoa,2.4149,2.5781,5.103,2.9782,5.0301,2.4149139615375192,2.6363,2.4517,2.9782,,,,,,,,,,
+ReactionOptimisation_mit_case1_mixed,87.3082,98.6621,98.6041,96.5437,95.9314,98.66214557690091,87.3082,95.3732,95.4297,,,,,,,,,,
+ReactionOptimisation_reizman_suzuki_pareto,63.5202,82.3427,82.0329,79.473,82.9901,82.24612252072882,63.5202,81.4666,79.7011,,,,,,,,,,
+ReactionOptimisation_snar_multiobjective,57.5234,87.3657,82.7881,80.1521,81.7614,100,72.3909,72.8477,79.427,,,,,,,,,,
+Robotics_DynamicObstacleAvoidanceNavigation,0.0722,0.086,0.0856,0.0834,0.0857,0.08571428571428559,0.0817,0.0765,0.0855,,,,,,,,,,
+Robotics_PIDTuning,0.0366,0.1632,0.151,0.1521,0.1515,0.1511172761100511,0.1585,0.1422,0.1514,,,,,,,,,,
+Robotics_QuadrupedGaitOptimization,0.0218,0.0219,0.0749,0.0218,0.1085,0.022154337029969478,0.0227,0.0232,0.0218,,,,,,,,,,
+Robotics_RobotArmCycleTimeOptimization,0.2922,0.4158,0.3923,0.4305,0.4219,0.4356212836221511,0.3923,0.3155,0.3256,,,,,,,,,,
+Robotics_UAVInspectionCoverageWithWind,28.8519,28.8519,38.8024,28.8519,35.1121,30.121714802877325,55.9109,32.8468,32.1552,,,,,,,,,,
+SingleCellAnalysis_predict_modality,0.5467,0.5467,0.5467,0.5467,0.5467,1,0.5467,0.5467,0.5467,,,,,,,,,,
+StructuralOptimization_ISCSO2015,-5401.589,-968.4567,-1120.212,-5401.589,-1139.3354,-5401.589002,-1318.7566,-1308.2575,-1302.2288,,,,,,,,,,
+StructuralOptimization_ISCSO2023,-77813242.9,-16477799.48,-55182772.3,-20092179.33,-17840974.17,-77813242.9,-30028112.28,-66126744.97,-42625693.78,,,,,,,,,,
+StructuralOptimization_TopologyOptimization,-195.9153,-190.1498,-190.3706,-189.3039,-188.4673,-195.9152621,-185.7983,-192.8488,-190.0603,,,,,,,,,,
+SustainableDataCenterControl_hand_written_control,8.3294,21.5657,15.292,12.9088,19.5978,8.5903,14.2432,30.1873,29.2868,,,,,,,,,,
+WirelessChannelSimulation_HighReliableSimulation,192.5193,292.3228,291.9451,232.9071,248.0119,231.22403446412542,245.7082,259.9776,304.0437,,,,,,,,,,
\ No newline at end of file
diff --git a/leaderboard/medal_leaderboard.csv b/leaderboard/medal_leaderboard.csv
new file mode 100644
index 00000000..20b35004
--- /dev/null
+++ b/leaderboard/medal_leaderboard.csv
@@ -0,0 +1,9 @@
+Rank,Model,Medal_v1,Medal_v1lite,Gold,Silver,Bronze
+1,gpt-5.4,0.596,0.667,24,5,2
+2,claude-opus-4.6,0.49,0.501,9,18,6
+3,glm-5,0.312,0.233,4,10,12
+4,deepseek-v3.2,0.248,0.166,3,9,8
+5,gemini-3.1-pro-preview,0.213,0.2,3,6,9
+6,seed-2.0-pro,0.185,0.1,3,7,3
+7,grok-4.20,0.184,0.133,3,6,5
+8,qwen3-coder-next,0.121,0.0,3,3,2
diff --git a/leaderboard/medal_podium.csv b/leaderboard/medal_podium.csv
new file mode 100644
index 00000000..8059fced
--- /dev/null
+++ b/leaderboard/medal_podium.csv
@@ -0,0 +1,48 @@
+Task,Baseline,Gold,Gold_model,Silver,Silver_model,Bronze,Bronze_model
+Aerodynamics_CarAerodynamicsSensing,0.9617,0.9632,deepseek-v3.2/gemini-3.1-pro-preview/qwen3-coder-next,0.9632,deepseek-v3.2/gemini-3.1-pro-preview/qwen3-coder-next,0.9632,deepseek-v3.2/gemini-3.1-pro-preview/qwen3-coder-next
+Astrodynamics_MannedLunarLanding,4577.437,6839.0331,glm-5,6660.942428,gpt-5.4,6079.2455,deepseek-v3.2
+ComputerSystems_MallocLab,28,96.0,claude-opus-4.6,86.0,glm-5,57.0,grok-4.20
+Cryptographic_AES-128,7.5209,39.824967043300866,gpt-5.4,12.4591,deepseek-v3.2,11.8617,claude-opus-4.6
+Cryptographic_SHA-256,9.8274,26.34045367870492,gpt-5.4,17.2504,grok-4.20,16.7955,claude-opus-4.6
+Cryptographic_SHA3-256,16.0932,37.44512785396786,gpt-5.4,18.3478,seed-2.0-pro,17.5778,glm-5
+EnergyStorage_BatteryFastChargingProfile,71.2806,121.99136502281442,gpt-5.4,120.8025,claude-opus-4.6,118.7678,glm-5
+EnergyStorage_BatteryFastChargingSPMe,66.1636,122.94304361063023,gpt-5.4,92.3198,gemini-3.1-pro-preview,91.0079,deepseek-v3.2
+EngDesign,1.3571,27.0,gemini-3.1-pro-preview/grok-4.20/seed-2.0-pro,27.0,gemini-3.1-pro-preview/grok-4.20/seed-2.0-pro,27.0,gemini-3.1-pro-preview/grok-4.20/seed-2.0-pro
+InventoryOptimization_disruption_eoqd,0.3642,1.0,gpt-5.4,0.6473,claude-opus-4.6,0.639,gemini-3.1-pro-preview
+InventoryOptimization_finite_horizon_dp,0.3673,0.9606835281410351,gpt-5.4,0.9596,claude-opus-4.6,0.8547,grok-4.20
+InventoryOptimization_general_meio,0.1825,0.9999999999999999,gpt-5.4,0.9929,claude-opus-4.6,0.9893,deepseek-v3.2
+InventoryOptimization_joint_replenishment,0.3034,1.0,gpt-5.4,0.8822,claude-opus-4.6/deepseek-v3.2/gemini-3.1-pro-preview/glm-5/grok-4.20/seed-2.0-pro,0.8822,claude-opus-4.6/deepseek-v3.2/gemini-3.1-pro-preview/glm-5/grok-4.20/seed-2.0-pro
+InventoryOptimization_tree_gsm_safety_stock,0.3813,1.0,gpt-5.4,0.75,claude-opus-4.6,0.6606,deepseek-v3.2/gemini-3.1-pro-preview/glm-5/grok-4.20/qwen3-coder-next/seed-2.0-pro
+JobShop_abz,80.5042,96.1035,claude-opus-4.6,91.23143065488635,gpt-5.4,88.4924,glm-5
+JobShop_swv,81.6325,89.4966,claude-opus-4.6,87.33430826602005,gpt-5.4,87.1611,glm-5
+JobShop_ta,78.8,90.8322,claude-opus-4.6,86.8095,glm-5,86.16070055174835,gpt-5.4
+KernelEngineering_FlashAttention,55.2957,182687.44188255747,gpt-5.4,1218.5163,seed-2.0-pro,991.8896,gemini-3.1-pro-preview
+KernelEngineering_MLA,0.7828,1253.2017,gemini-3.1-pro-preview,1132.0659025372765,gpt-5.4,1000.3859,claude-opus-4.6
+KernelEngineering_TriMul,47.1274,357.1636,claude-opus-4.6,165.0294,grok-4.20,110.8785,glm-5
+Optics_adaptive_fault_tolerant_fusion,0.3959,0.64,deepseek-v3.2,0.6398,claude-opus-4.6/gemini-3.1-pro-preview/glm-5/grok-4.20/qwen3-coder-next/seed-2.0-pro,0.6398,claude-opus-4.6/gemini-3.1-pro-preview/glm-5/grok-4.20/qwen3-coder-next/seed-2.0-pro
+Optics_adaptive_temporal_smooth_control,0.3152,0.8421,qwen3-coder-next/seed-2.0-pro,0.8421,qwen3-coder-next/seed-2.0-pro,0.842,grok-4.20
+Optics_fiber_guardband_spectrum_packing,0.3861,0.6754289215686274,gpt-5.4,0.6692,claude-opus-4.6/glm-5,0.6692,claude-opus-4.6/glm-5
+Optics_fiber_mcs_power_scheduling,0.3297,0.6608370951757289,gpt-5.4,0.6542,claude-opus-4.6,0.6491,glm-5/seed-2.0-pro
+Optics_fiber_wdm_channel_power_allocation,0.3255,0.6964207451370852,gpt-5.4,0.6686,glm-5,0.6679,deepseek-v3.2
+Optics_holographic_multifocus_power_ratio,0.3927,0.9999999999663148,gpt-5.4,0.8265,deepseek-v3.2,0.8072,claude-opus-4.6
+Optics_holographic_multiplane_focusing,0.3302,0.9999999999886867,gpt-5.4,0.7196,deepseek-v3.2,0.6002,claude-opus-4.6
+Optics_phase_dammann_uniform_orders,26.8969,99.99999999999999,gpt-5.4,99.7995,claude-opus-4.6,97.9498,gemini-3.1-pro-preview
+Optics_phase_fourier_pattern_holography,32.6457,99.99998936790779,gpt-5.4,82.1276,claude-opus-4.6,76.6371,gemini-3.1-pro-preview
+PyPortfolioOpt_robust_mvo_rebalance,32.9804,99.99460428985267,gpt-5.4,99.9946,claude-opus-4.6,99.983,grok-4.20
+QuantumComputing_task_01_routing_qftentangled,0.209,6.507945106686525,gpt-5.4,5.0479,claude-opus-4.6,3.7681,glm-5
+QuantumComputing_task_02_clifford_t_synthesis,1.7134,7.4236,glm-5,1.7134,deepseek-v3.2/gemini-3.1-pro-preview/qwen3-coder-next/seed-2.0-pro,1.7134,deepseek-v3.2/gemini-3.1-pro-preview/qwen3-coder-next/seed-2.0-pro
+QuantumComputing_task_03_cross_target_qaoa,2.4149,5.103,deepseek-v3.2,5.0301,glm-5,2.9782,gemini-3.1-pro-preview/seed-2.0-pro
+ReactionOptimisation_mit_case1_mixed,87.3082,98.66214557690091,gpt-5.4,98.6621,claude-opus-4.6,98.6041,deepseek-v3.2
+ReactionOptimisation_reizman_suzuki_pareto,63.5202,82.9901,glm-5,82.3427,claude-opus-4.6,82.24612252072882,gpt-5.4
+ReactionOptimisation_snar_multiobjective,57.5234,100.0,gpt-5.4,87.3657,claude-opus-4.6,82.7881,deepseek-v3.2
+Robotics_DynamicObstacleAvoidanceNavigation,0.0722,0.086,claude-opus-4.6,0.08571428571428559,gpt-5.4,0.0857,glm-5
+Robotics_PIDTuning,0.0366,0.1632,claude-opus-4.6,0.1585,grok-4.20,0.1521,gemini-3.1-pro-preview
+Robotics_QuadrupedGaitOptimization,0.0218,0.1085,glm-5,0.0749,deepseek-v3.2,0.0232,qwen3-coder-next
+Robotics_RobotArmCycleTimeOptimization,0.2922,0.4356212836221511,gpt-5.4,0.4305,gemini-3.1-pro-preview,0.4219,glm-5
+Robotics_UAVInspectionCoverageWithWind,28.8519,55.9109,grok-4.20,38.8024,deepseek-v3.2,35.1121,glm-5
+SingleCellAnalysis_predict_modality,0.5467,1.0,gpt-5.4,0.5467,claude-opus-4.6/deepseek-v3.2/gemini-3.1-pro-preview/glm-5/grok-4.20/qwen3-coder-next/seed-2.0-pro,0.5467,claude-opus-4.6/deepseek-v3.2/gemini-3.1-pro-preview/glm-5/grok-4.20/qwen3-coder-next/seed-2.0-pro
+StructuralOptimization_ISCSO2015,-5401.589,-968.4567,claude-opus-4.6,-1120.212,deepseek-v3.2,-1139.3354,glm-5
+StructuralOptimization_ISCSO2023,-77813242.9,-16477799.48,claude-opus-4.6,-17840974.17,glm-5,-20092179.33,gemini-3.1-pro-preview
+StructuralOptimization_TopologyOptimization,-195.9153,-185.7983,grok-4.20,-188.4673,glm-5,-189.3039,gemini-3.1-pro-preview
+SustainableDataCenterControl_hand_written_control,8.3294,30.1873,qwen3-coder-next,29.2868,seed-2.0-pro,21.5657,claude-opus-4.6
+WirelessChannelSimulation_HighReliableSimulation,192.5193,304.0437,seed-2.0-pro,292.3228,claude-opus-4.6,291.9451,deepseek-v3.2
diff --git a/leaderboard/score_submission.py b/leaderboard/score_submission.py
new file mode 100644
index 00000000..1c25561e
--- /dev/null
+++ b/leaderboard/score_submission.py
@@ -0,0 +1,124 @@
+#!/usr/bin/env python3
+"""Score a submission against the frozen Frontier-Eng Medal podium.
+
+The gold/silver/bronze baselines are frozen at the v1 snapshot (2026-04-14) and
+shipped in ``medal_podium.csv``. This script takes a new model's best-feasible
+score on each task and reports its Medal Score, so anyone can be scored against
+the released benchmark without rerunning the reference models.
+
+Usage
+-----
+ python leaderboard/score_submission.py [--verbose]
+
+Submission CSV format (header required): two columns, ``Task,Score``, one row
+per task, using the task names from ``medal_podium.csv`` (e.g. ``JobShop_abz``).
+Higher score is better on every task. Missing tasks score 0. See
+``submission_example.csv`` (the claude-opus-4.6 column) for a working example;
+scoring it reproduces its leaderboard line (Medal v1 = 0.490, v1-lite = 0.501).
+
+Metric
+------
+On each task a submission earns 1.00 / 0.67 / 0.33 for reaching the gold /
+silver / bronze score, else 0. The Medal Score is the mean of this credit,
+normalized to [0, 1], reported on the full v1 set (47 tasks) and the v1-lite
+subset (10 tasks).
+"""
+
+import argparse
+import csv
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+GOLD, SILVER, BRONZE = 1.00, 0.67, 0.33
+
+# v1-lite: 10-task representative subset (frontier_eval/conf/batch/v1_lite.yaml).
+V1_LITE = {
+ "QuantumComputing_task_01_routing_qftentangled", "ComputerSystems_MallocLab",
+ "JobShop_abz", "InventoryOptimization_disruption_eoqd",
+ "EnergyStorage_BatteryFastChargingSPMe", "Robotics_RobotArmCycleTimeOptimization",
+ "Optics_holographic_multiplane_focusing", "WirelessChannelSimulation_HighReliableSimulation",
+ "ReactionOptimisation_snar_multiobjective", "StructuralOptimization_TopologyOptimization",
+}
+
+
+def load_podium(path):
+ """task -> (gold, silver, bronze) thresholds (higher is better)."""
+ podium = {}
+ with open(path, encoding="utf-8-sig") as f:
+ for row in csv.DictReader(f):
+ podium[row["Task"]] = (
+ float(row["Gold"]), float(row["Silver"]), float(row["Bronze"]))
+ return podium
+
+
+def load_submission(path):
+ """task -> score. Accepts a 'Task,Score' header or any two-column CSV."""
+ scores = {}
+ with open(path, encoding="utf-8-sig") as f:
+ reader = csv.reader(f)
+ first = next(reader)
+ if not (first[1].strip().lower() in ("score", "best", "value")):
+ f.seek(0) # no recognizable header -> treat all rows as data
+ reader = csv.reader(f)
+ for row in reader:
+ if len(row) < 2 or not row[0].strip():
+ continue
+ try:
+ scores[row[0].strip()] = float(row[1])
+ except ValueError:
+ continue # skip header/garbage rows
+ return scores
+
+
+def tier(score, gold, silver, bronze):
+ if score >= gold:
+ return GOLD, "gold"
+ if score >= silver:
+ return SILVER, "silver"
+ if score >= bronze:
+ return BRONZE, "bronze"
+ return 0.0, None
+
+
+def score(podium, submission, verbose=False):
+ per_task = {}
+ counts = {"gold": 0, "silver": 0, "bronze": 0}
+ missing = []
+ for task, (g, s, b) in podium.items():
+ if task not in submission:
+ per_task[task] = 0.0
+ missing.append(task)
+ continue
+ pts, name = tier(submission[task], g, s, b)
+ per_task[task] = pts
+ if name:
+ counts[name] += 1
+ medal_v1 = sum(per_task.values()) / len(podium)
+ lite = [t for t in podium if t in V1_LITE]
+ medal_lite = sum(per_task[t] for t in lite) / len(lite)
+
+ print(f"Medal Score (v1, 47 tasks) : {medal_v1:.3f}"
+ f" (gold {counts['gold']}, silver {counts['silver']}, bronze {counts['bronze']})")
+ print(f"Medal Score (v1-lite, 10 tasks) : {medal_lite:.3f}")
+ if missing:
+ print(f"\n[warn] {len(missing)} task(s) absent from submission (scored 0): "
+ f"{', '.join(missing[:5])}{' ...' if len(missing) > 5 else ''}")
+ if verbose:
+ print("\nper-task credit:")
+ for task in podium:
+ print(f" {per_task[task]:.2f} {task}")
+ return medal_v1, medal_lite
+
+
+def main():
+ ap = argparse.ArgumentParser(description="Score a submission against the frozen Medal podium.")
+ ap.add_argument("submission", help="CSV with columns Task,Score (one row per task)")
+ ap.add_argument("--podium", default=str(HERE / "medal_podium.csv"),
+ help="frozen gold/silver/bronze baselines (default: leaderboard/medal_podium.csv)")
+ ap.add_argument("--verbose", action="store_true", help="print per-task medal credit")
+ args = ap.parse_args()
+ score(load_podium(args.podium), load_submission(args.submission), args.verbose)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/leaderboard/submission_example.csv b/leaderboard/submission_example.csv
new file mode 100644
index 00000000..82f4c1a8
--- /dev/null
+++ b/leaderboard/submission_example.csv
@@ -0,0 +1,48 @@
+Task,Score
+Aerodynamics_CarAerodynamicsSensing,0.9624
+Astrodynamics_MannedLunarLanding,6027.3126
+ComputerSystems_MallocLab,96
+Cryptographic_AES-128,11.8617
+Cryptographic_SHA-256,16.7955
+Cryptographic_SHA3-256,17.4003
+EnergyStorage_BatteryFastChargingProfile,120.8025
+EnergyStorage_BatteryFastChargingSPMe,71.8225
+EngDesign,1.3571
+InventoryOptimization_disruption_eoqd,0.6473
+InventoryOptimization_finite_horizon_dp,0.9596
+InventoryOptimization_general_meio,0.9929
+InventoryOptimization_joint_replenishment,0.8822
+InventoryOptimization_tree_gsm_safety_stock,0.75
+JobShop_abz,96.1035
+JobShop_swv,89.4966
+JobShop_ta,90.8322
+KernelEngineering_FlashAttention,983.5001
+KernelEngineering_MLA,1000.3859
+KernelEngineering_TriMul,357.1636
+Optics_adaptive_fault_tolerant_fusion,0.6398
+Optics_adaptive_temporal_smooth_control,0.8419
+Optics_fiber_guardband_spectrum_packing,0.6692
+Optics_fiber_mcs_power_scheduling,0.6542
+Optics_fiber_wdm_channel_power_allocation,0.6675
+Optics_holographic_multifocus_power_ratio,0.8072
+Optics_holographic_multiplane_focusing,0.6002
+Optics_phase_dammann_uniform_orders,99.7995
+Optics_phase_fourier_pattern_holography,82.1276
+PyPortfolioOpt_robust_mvo_rebalance,99.9946
+QuantumComputing_task_01_routing_qftentangled,5.0479
+QuantumComputing_task_02_clifford_t_synthesis,1.6633
+QuantumComputing_task_03_cross_target_qaoa,2.5781
+ReactionOptimisation_mit_case1_mixed,98.6621
+ReactionOptimisation_reizman_suzuki_pareto,82.3427
+ReactionOptimisation_snar_multiobjective,87.3657
+Robotics_DynamicObstacleAvoidanceNavigation,0.086
+Robotics_PIDTuning,0.1632
+Robotics_QuadrupedGaitOptimization,0.0219
+Robotics_RobotArmCycleTimeOptimization,0.4158
+Robotics_UAVInspectionCoverageWithWind,28.8519
+SingleCellAnalysis_predict_modality,0.5467
+StructuralOptimization_ISCSO2015,-968.4567
+StructuralOptimization_ISCSO2023,-16477799.48
+StructuralOptimization_TopologyOptimization,-190.1498
+SustainableDataCenterControl_hand_written_control,21.5657
+WirelessChannelSimulation_HighReliableSimulation,292.3228