feat(WindEnergy): add WakeSteeringControl benchmark

[Yunn1633]

Background

This task optimizes a deterministic yaw-control policy for a nine-turbine wind farm. A frozen verifier evaluates expected farm power over fixed wind directions and speeds while enforcing yaw, runtime, and determinism constraints.

Source

The evaluator uses FLORIS 4.6.6, its built-in NREL 5 MW turbine model, and the fixed farm configuration under references/. Dependency and configuration provenance are documented in the task README files.

Validation

From the benchmark directory:

../../../.venvs/frontier-wake-steering/bin/python verification/evaluator.py scripts/init.py

From the repository root:

python -m frontier_eval task=unified task.benchmark=WindEnergy/WakeSteeringControl task.runtime.python_path=uv-env:frontier-wake-steering algorithm=openevolve algorithm.iterations=0

[github-actions]

🤖 AI Code Review (gemini-3-flash-preview)

🇬🇧 English Analysis

1. Executive Summary

• Core Purpose: This PR introduces a new benchmark category, WindEnergy, featuring a specific task: WakeSteeringControl. The task requires an agent to optimize the yaw angles of a nine-turbine wind farm to maximize energy production while minimizing control effort and maintaining strategy smoothness, using the physics-based FLORIS (v4.6.6) model.
• Modified File Structure & Modifications:
  • benchmarks/WindEnergy/README.md & README_zh-CN.md: Category-level documentation in English and Chinese.
  • benchmarks/WindEnergy/WakeSteeringControl/README.md & README_zh-CN.md: Task-specific setup, environment, and execution guides.
  • benchmarks/WindEnergy/WakeSteeringControl/Task.md & Task_zh-CN.md: Detailed task description, objective, and scoring logic.
  • benchmarks/WindEnergy/WakeSteeringControl/scripts/init.py: The entry point for the agent, containing the yaw_policy function and EVOLVE markers.
  • benchmarks/WindEnergy/WakeSteeringControl/verification/evaluator.py: The core evaluation engine using the floris library to simulate wind farm physics and calculate scores.
  • benchmarks/WindEnergy/WakeSteeringControl/verification/run_candidate.py: A wrapper script to execute candidate code in isolation with network blocking and output capturing.
  • benchmarks/WindEnergy/WakeSteeringControl/references/farm_config.json: Static configuration for the wind farm layout and turbine models.
  • benchmarks/WindEnergy/WakeSteeringControl/frontier_eval/: Metadata files (e.g., agent_files.txt, constraints.txt, run_eval.sh) for integration with the frontier_eval framework.

2. AI Content Analysis

• Estimated AI Component: 15%
• Reasoning & Evidence: The code structure is highly professional and follows a standardized template seen in other frontier_eval tasks. While the boilerplate (e.g., run_candidate.py network blocking, evaluator.py argument parsing) might be AI-assisted or templated, the domain-specific logic—such as the _smoothness calculation in evaluator.py (lines 100-110) and the specific wind condition arrays in _build_conditions (lines 47-68)—demonstrates deep integration with the FLORIS library and wind energy domain knowledge that is unlikely to be purely generic AI output.

3. Engineering & Economic Assessment

• Engineering Reality Check: This is a high-quality, production-grade engineering problem. It utilizes a real-world physics engine (NREL's FLORIS), handles complex multi-objective optimization (energy vs. wear-and-tear), and implements robust verification features like nondeterminism checks and subprocess isolation.
• Economic Value: High. Wind farm wake steering is a significant area of research in renewable energy. Optimizing these controls can lead to measurable increases in annual energy production (AEP) for real wind plants, directly impacting revenue and operational efficiency.

4. Quality Assurance

• Verification & Testing:
  • frontier_eval Integration: Yes
  • task_name: WindEnergy/WakeSteeringControl
  • Execution & Dependencies: The README.md clearly documents the environment setup using venv and pip, and provides the exact commands for both direct and unified evaluation.
• Documentation Quality: Excellent. The PR provides comprehensive bilingual documentation. The Task.md clearly defines inputs, outputs, and the mathematical components of the score. No significant redundant information or grammatical errors were detected.
• Organizational Structure: The structure is logical and modular. It separates the agent's workspace (scripts/), the evaluation logic (verification/), and the framework metadata (frontier_eval/).

5. Security & Privacy Check

• Sensitive Files: Clean. No .env, API keys, or IDE-specific files were found.
• Absolute Paths: None detected. The code uses Path(__file__).resolve() to handle paths relative to the script location.

---

🇨🇳 中文分析

1. 摘要

• 核心目的: 此 PR 引入了一个新的基准测试类别 WindEnergy（风能），并包含一个具体任务：WakeSteeringControl（尾流偏航控制）。该任务要求 Agent 优化九台风机组成的风电场的偏航角，以在最小化控制代价并保持策略平滑性的前提下，利用基于物理的 FLORIS (v4.6.6) 模型最大化发电量。
• 修改的文件结构与变更摘要:
  • benchmarks/WindEnergy/README.md & README_zh-CN.md: 类别级中英文文档。
  • benchmarks/WindEnergy/WakeSteeringControl/README.md & README_zh-CN.md: 任务特定的设置、环境和运行指南。
  • benchmarks/WindEnergy/WakeSteeringControl/Task.md & Task_zh-CN.md: 详细的任务描述、目标和评分逻辑。
  • benchmarks/WindEnergy/WakeSteeringControl/scripts/init.py: Agent 的入口文件，包含 yaw_policy 函数和 EVOLVE 标记。
  • benchmarks/WindEnergy/WakeSteeringControl/verification/evaluator.py: 核心评估引擎，使用 floris 库模拟风场物理并计算分数。
  • benchmarks/WindEnergy/WakeSteeringControl/verification/run_candidate.py: 包装脚本，用于在隔离环境中执行候选代码，具备网络屏蔽和输出捕获功能。
  • benchmarks/WindEnergy/WakeSteeringControl/references/farm_config.json: 风场布局和风机模型的静态配置。
  • benchmarks/WindEnergy/WakeSteeringControl/frontier_eval/: 用于集成到 frontier_eval 框架的元数据文件（如 agent_files.txt, constraints.txt, run_eval.sh）。

2. AI 成分分析

• 预估 AI 含量: 15%
• 判断依据与证据: 代码结构非常专业，遵循了 frontier_eval 任务的标准模板。虽然部分样板代码（如 run_candidate.py 中的网络屏蔽、evaluator.py 中的参数解析）可能是 AI 辅助生成或基于模板的，但领域特定的逻辑——例如 evaluator.py 中的 _smoothness 计算（第 100-110 行）以及 _build_conditions 中特定的风况数组（第 47-68 行）——展示了与 FLORIS 库的深度集成和风能领域知识，不太可能是纯粹的通用 AI 输出。

3. 工程与经济评估

• 工程现实检验: 这是一个高质量、生产级别的工程问题。它使用了真实的物理引擎（NREL 的 FLORIS），处理复杂的多目标优化（发电量 vs. 损耗），并实现了健壮的验证功能，如非确定性检查和子进程隔离。
• 经济价值: 高。风电场尾流控制是可再生能源领域的重要研究方向。优化这些控制策略可以显著提高实际风电场的年发电量 (AEP)，直接影响营收和运营效率。

4. 质量保证

• 验证与测试:
  • frontier_eval 集成: 是
  • task_name: WindEnergy/WakeSteeringControl
  • 运行与依赖: README.md 清晰地记录了使用 venv 和 pip 进行环境配置的步骤，并提供了直接评测和统一评测的确切命令。
• 文档质量: 优秀。PR 提供了详尽的双语文档。Task.md 明确定义了输入、输出和评分的数学组成部分。未发现明显的冗余信息或语法错误。
• 组织结构: 结构逻辑清晰且模块化。它将 Agent 的工作区 (scripts/)、评估逻辑 (verification/) 和框架元数据 (frontier_eval/) 进行了有效分离。

5. 安全与隐私检查

• 敏感文件: 未发现异常。未发现 .env、API 密钥或 IDE 特定文件。
• 绝对路径: 未检测到。代码使用 Path(__file__).resolve() 来处理相对于脚本位置的路径。

[wrh-human]

Review — WakeSteeringControl (PR #92)

Thank you for contributing this benchmark. Wind farm wake steering control is a real engineering domain with practical value, and using FLORIS as the physical model is a reasonable choice. Several issues need to be addressed before merging.

---

1. Domain, Economic Value, and Frontier-Eng Fit ✅

Wind farm wake steering control is a genuine wind energy engineering problem. Upstream turbine yaw can redirect wakes to reduce downstream losses, thereby increasing total farm power output. The task uses NREL's FLORIS 4.6.6 as the wake simulation model — an open-source tool widely used in academia. Optimizing turbine yaw angles directly translates to increased wind farm revenue, giving the task clear economic value.

2. Search space ✅

24 wind conditions × 9 turbines = 216 continuous parameters, each in [-25, 25]. Parameter interactions through wake physics (changing one turbine's yaw affects all downstream turbines) make the design space large enough that brute-force search is infeasible.

3. Evaluator and engineering verification ✅

The evaluator uses FLORIS for realistic wake simulation, computing farm power output under the candidate's yaw strategy and comparing against a zero-yaw baseline. Verification includes:

• Output shape check (n_conditions, n_turbines)
• Yaw angle range check [-25, 25]
• Finite value check (NaN/inf detection)
• Determinism check (candidate called twice with identical inputs)
• Subprocess isolation (candidate runs in a separate process via run_candidate.py)
• JSON-only communication interface

4. Constraints and verifier enforcement ⚠️ Partially implemented

constraints.txt defines reasonable constraints, but the evaluator does not fully enforce them (see issues below).

5. Baseline experiment ⚠️ Partially complete

result_log.json is included, showing valid=1.0 and combined_score=0.0 (zero-yaw baseline scores 0, which is correct). However, environment information and a headroom check are missing.

---

Issues to address

Issue 1 (most critical): Missing EVOLVE-BLOCK boundary validation

Neither the evaluator nor run_candidate.py validates the EVOLVE-BLOCK boundary. constraints.txt explicitly requires "Only modify the EVOLVE block in scripts/init.py", but the evaluator never enforces this. An agent could modify the yaw_policy() function signature, add code outside the EVOLVE-BLOCK, or modify/remove the EVOLVE markers themselves without triggering any checks.

Suggestion: add EVOLVE-BLOCK boundary validation before importing the candidate source.

Issue 2: Candidate process lacks resource isolation

_run_candidate() uses subprocess.run() to launch the candidate without any resource limits:

• No RLIMIT_NPROC (can fork child processes)
• No RLIMIT_AS (no memory limit)
• No RLIMIT_CPU (no CPU time limit, relies solely on wall-clock timeout)
• No start_new_session=True (no process group isolation)
• No preexec_fn

run_candidate.py replaces socket.socket and socket.create_connection to block network access, but this is easily bypassed — the agent can access external resources via subprocess.run("curl ..."), os.system(), or C-level network functions.

Suggestion: add preexec_fn to subprocess.run with RLIMIT_NPROC, RLIMIT_AS, and RLIMIT_CPU, and use start_new_session=True.

Issue 3: No test file

No test_evaluator.py exists. At minimum, the following should be added:

• A test verifying the baseline returns valid=1.0 and combined_score=0.0
• A test rejecting yaw angles outside the allowed range
• A test rejecting EVOLVE-BLOCK boundary violations (after Issue 1 is fixed)
• A test verifying candidate timeout is reported correctly

Issue 4: result_log.json lacks environment details and headroom check

The log only contains aggregate metrics. It is missing:

• Environment information (CPU model, FLORIS version, numpy version, etc.)
• Per-condition power output data
• A headroom check demonstrating optimization potential beyond the zero-yaw baseline (e.g., results from a simple heuristic yaw strategy)

Issue 5: Single scenario design risks overfitting

Only one turbine layout (9 turbines) and one set of wind conditions (24 variants) are evaluated. A single layout makes the results susceptible to overfitting — the agent could optimize specifically for this 9-turbine layout without generalizing. Suggestion: evaluate against multiple distinct farm layouts to detect overfitting.

Issue 6: Engineering reasoning density is relatively low

The agent's output is 216 numerical values (24 conditions × 9 turbine yaw angles). The yaw_policy() function is essentially a numerical mapping from wind parameters to a matrix. The agent could approximate an optimal solution via a hardcoded lookup table or a simple numerical formula, without needing to understand wake physics, turbine coupling, or FLORIS's simulation logic. Compared to tasks that require writing CUDA kernels, AIG rewrite rules, or codecs — which demand structural engineering reasoning — this task is weaker in terms of forcing the agent to understand the engineering problem.

Suggestion: consider enriching the task's information dimensionality so the agent must do more than numerical regression. For example, making the farm layout a variable input, requiring a physics-based analytical control strategy, or including textual descriptions of physical quantities in the observation that require interpretation.

---

Non-blocking suggestions

1. Setting both NO_PROXY and no_proxy is redundant — os.environ is case-insensitive.
2. FLORIS is multi-threaded (numpy/jax). Consider setting OMP_NUM_THREADS=1 in the evaluator to avoid measurement variance from parallelism.
3. The candidate timeout has no wall-clock safety margin. Consider adding a 1-second buffer to the subprocess.run timeout to account for process cleanup overhead.

---

Summary

The core direction is sound, and the use of FLORIS as the physical model is a strength. Issues 1 (missing EVOLVE-BLOCK validation) and 2 (missing process resource isolation) are the highest priority — they directly affect benchmark integrity and rigor. The review can proceed once the above issues are addressed.