Add FPGA Placement Optimization benchmark

[y-ji24]

Summary

This PR introduces the FPGA Placement Optimization benchmark.

The benchmark evaluates an agent's ability to design and improve placement algorithms through an editable artifact (scripts/init.py).

Design

• Editable artifact:
  scripts/init.py

• Baseline:
  reference only, not modified by agents

• Evaluation:
  candidate-generated solution.pl is evaluated independently

Benchmark Features

• Bookshelf format parsing
• Placement legality verification
• HPWL-based scoring
• Multiple benchmark support

Dataset

The default benchmark is bundled.

ISPD 2016 benchmarks are supported as optional external datasets.

Validation

Tested:

• default benchmark evaluation
• ISPD benchmark loading
• legality checks
• invalid benchmark handling

[github-actions]

🤖 AI Code Review (gemini-3-flash-preview)

🇬🇧 English Analysis

1. Executive Summary

• Core Purpose: This PR introduces a new evaluation metric ("Medal Score"), a lightweight benchmark subset (v1-lite), and comprehensive design/validation documentation for a new FPGA placement optimization task.
• Modified File Structure & Modifications:
  • .gitignore: Whitelisted leaderboard/*.csv and the new v1_lite.yaml configuration.
  • README.md & README_zh-CN.md: Updated with "News" regarding the Medal Score and v1-lite. Updated the leaderboard table to reflect these new metrics and future-dated (2026) project milestones.
  • benchmarks/FPGA/FPGAPlacementOptimization/ClockAwarePlacement_DesignValidation.md: A new, highly detailed audit of the aug-elfPlace baseline, identifying discrepancies between the "Clock-Aware" naming and actual code implementation (e.g., dead code in C++ parsers).
  • benchmarks/FPGA/FPGAPlacementOptimization/ClockAwarePlacement_Design_Report.md: A new architectural blueprint for the FPGA placement task, outlining scoring logic (HPWL-based), validation gates, and implementation roadmaps.

2. AI Content Analysis

• Estimated AI Component: 25%
• Reasoning & Evidence: The README updates and leaderboard data are clearly manual entries. The two new Markdown reports (DesignValidation.md and Design_Report.md) exhibit a highly structured, LLM-like formatting (tables, clear headers, pros/cons lists). However, the content is extremely domain-specific and references internal code locations (e.g., PlaceDB.cpp:805-813, lut_ff_legalization.py:240). This suggests an engineer used an AI to structure and draft findings based on a very thorough manual code audit.

3. Engineering & Economic Assessment

• Engineering Reality Check:
  • High Reality: The "Medal Score" addresses a sophisticated benchmarking problem: how to aggregate performance across diverse tasks without letting "long-tail" marginal gains in easy tasks skew the average.
  • Critical Audit: The DesignValidation.md is a standout piece of engineering work. It correctly identifies that the baseline lacks actual clock-aware optimization despite the naming, preventing the creation of a "toy" benchmark based on false assumptions.
• Economic Value: High.
  • Efficiency: The v1-lite subset significantly reduces the compute cost and time for developer iterations.
  • Technical Debt: The FPGA audit prevents the implementation of a flawed benchmark, saving future engineering hours that would have been spent debugging non-functional "clock" constraints.

4. Quality Assurance

• Verification & Testing:
  • frontier_eval Integration: Yes.
  • task_name: v1-lite (as a batch configuration) and ClockAwarePlacement (proposed in design docs).
  • Execution & Dependencies: The README explicitly provides the command to run the new v1-lite subset (frontier_eval/conf/batch/v1_lite.yaml). The design report correctly identifies the need for a pure-Python evaluator to avoid heavy C++/CUDA dependencies.
• Documentation Quality: Excellent. The design reports are thorough, and the README updates are clear. Note: The dates are set in June 2026, which appears to be a project roadmap convention rather than a typo.
• Organizational Structure: Logical. The design and validation documents are co-located with the specific benchmark they describe.

5. Security & Privacy Check

• Sensitive Files: Clean. No .env, .vscode, or cache files were included.
• Absolute Paths: None detected. The design report explicitly recommends using relative paths and pure-Python scoring to maintain portability.

---

🇨🇳 中文分析

1. 摘要

• 核心目的: 此 PR 引入了新的评分指标（“勋章分” Medal Score）、轻量级评测子集（v1-lite），以及针对新 FPGA 布局优化任务的详细设计与验证文档。
• 修改的文件结构与变更摘要:
  • .gitignore: 将 leaderboard/*.csv 和新的 v1_lite.yaml 配置列入白名单。
  • README.md & README_zh-CN.md: 更新了关于 Medal Score 和 v1-lite 的“新闻”动态。更新了排行榜表格以反映这些新指标，并标注了 2026 年的项目里程碑。
  • benchmarks/FPGA/FPGAPlacementOptimization/ClockAwarePlacement_DesignValidation.md: 一份全新的、极其详尽的 aug-elfPlace 基准线审计文档，指出了“时钟感知”命名与实际代码实现之间的差异（例如 C++ 解析器中的死代码）。
  • benchmarks/FPGA/FPGAPlacementOptimization/ClockAwarePlacement_Design_Report.md: FPGA 布局任务的新架构蓝图，概述了评分逻辑（基于 HPWL）、验证门槛和实施路线图。

2. AI 成分分析

• 预估 AI 含量: 25%
• 判断依据与证据: README 的更新和排行榜数据显然是手动录入的。两份新的 Markdown 报告（DesignValidation.md 和 Design_Report.md）表现出高度结构化、类似 LLM 的格式（表格、清晰的标题、优缺点列表）。然而，内容具有极强的领域专业性，并引用了内部代码位置（如 PlaceDB.cpp:805-813, lut_ff_legalization.py:240）。这表明工程师在进行了非常彻底的手动代码审计后，利用 AI 进行了结构化整理和草拟。

3. 工程与经济评估

• 工程现实检验:
  • 高度现实: “勋章分”解决了基准测试中的一个复杂问题：如何在不让简单任务中的“长尾”边际收益扭曲平均值的情况下，汇总不同任务的性能。
  • 批判性审计: DesignValidation.md 是工程实践中的亮点。它准确识别出基准线虽然命名为“时钟感知”，但实际上缺乏相关优化逻辑，从而避免了基于错误假设构建“玩具级”基准测试。
• 经济价值: 高。
  • 效率: v1-lite 子集显著降低了开发人员迭代的计算成本和时间。
  • 技术债务: FPGA 审计防止了缺陷基准测试的实施，节省了未来用于调试无效“时钟”约束的工程时间。

4. 质量保证

• 验证与测试:
  • frontier_eval 集成: 是。
  • task_name: v1-lite（作为批处理配置）和 ClockAwarePlacement（在设计文档中提出）。
  • 运行与依赖: README 明确提供了运行新 v1-lite 子集的命令（frontier_eval/conf/batch/v1_lite.yaml）。设计报告正确识别了对纯 Python 评估器的需求，以避免沉重的 C++/CUDA 依赖。
• 文档质量: 优秀。设计报告非常详尽，README 更新清晰。注：日期设定在 2026 年 6 月，这似乎是项目路线图的惯例而非拼写错误。
• 组织结构: 符合逻辑。设计和验证文档与它们所描述的特定基准测试存放在一起。

5. 安全与隐私检查

• 敏感文件: 未发现异常。未包含 .env、.vscode 或缓存文件。
• 绝对路径: 未检测到。设计报告明确建议使用相对路径和纯 Python 评分以保持可移植性。