Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions benchmarks/WindEnergy/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Wind Energy

Wind-energy benchmarks evaluate control and design decisions with executable,
physics-based wind-plant models.

## Tasks

- [`WakeSteeringControl`](WakeSteeringControl/README.md): optimize turbine yaw
control to increase weighted farm energy while limiting control effort.
8 changes: 8 additions & 0 deletions benchmarks/WindEnergy/README_zh-CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# 风能工程

风能工程任务使用可执行的风电场物理模型,评测控制与设计策略的优化能力。

## 任务

- [`WakeSteeringControl`](WakeSteeringControl/README_zh-CN.md):优化风机偏航控制,
在限制控制代价的同时提升风电场加权发电量。
69 changes: 69 additions & 0 deletions benchmarks/WindEnergy/WakeSteeringControl/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Wake Steering Control

This benchmark asks an agent to improve a deterministic yaw-control policy for
a nine-turbine wind farm. A frozen FLORIS 4.6.6 verifier evaluates the policy
over multiple wind directions and speeds.

## Structure

```text
scripts/init.py editable yaw policy
references/farm_config.json public farm and score contract
verification/evaluator.py frozen FLORIS evaluator
verification/run_candidate.py isolated candidate runner
frontier_eval/ unified-task metadata
baseline/result_log.json measured zero-yaw baseline
```

## Environment

From the Frontier-Engineering repository root, create the task runtime. The task
is tested with Python 3.12 on Linux.

```bash
python3.12 -m venv .venvs/frontier-wake-steering
.venvs/frontier-wake-steering/bin/python -m pip install \
-r benchmarks/WindEnergy/WakeSteeringControl/verification/requirements.txt
```

The task is CPU-only, and FLORIS itself does not require a GPU. A baseline
evaluation typically completes within seconds. No external dataset, Docker, or
runtime network access is required.

## Direct Evaluation

From this benchmark directory:

```bash
../../../.venvs/frontier-wake-steering/bin/python \
verification/evaluator.py scripts/init.py
```

The evaluator writes `metrics.json` and `artifacts.json`. The shipped zero-yaw
policy is feasible and has a combined score of approximately zero by design.

## Unified Evaluation

The unified benchmark ID is `WindEnergy/WakeSteeringControl`:

```bash
python -m frontier_eval \
task=unified \
task.benchmark=WindEnergy/WakeSteeringControl \
task.runtime.python_path=uv-env:frontier-wake-steering \
algorithm=openevolve \
algorithm.iterations=0
```

Optimization runs require the normal model API configuration used by
Frontier Eval. Baseline-only validation does not require an API key.

## Reproducibility and Security

- FLORIS is pinned to version 4.6.6.
- The verifier uses the built-in NREL 5 MW turbine model and a fixed farm.
- Candidate code runs in a separate subprocess with a wall-clock timeout.
- The unified task checks the verifier and public contract as read-only and
invalidates candidates that modify them.
- Process isolation is not a hardened security sandbox. Run untrusted candidate
code in a container or disposable host.
62 changes: 62 additions & 0 deletions benchmarks/WindEnergy/WakeSteeringControl/README_zh-CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# 风电场尾流偏航控制

本任务要求 Agent 优化一个确定性的偏航控制策略。冻结的 FLORIS 4.6.6
验证器在多种风向和风速下评估九台风机组成的风电场。

## 目录

```text
scripts/init.py 可编辑偏航策略
references/farm_config.json 公开风场与评分契约
verification/evaluator.py 冻结的 FLORIS 验证器
verification/run_candidate.py 候选代码隔离运行器
frontier_eval/ unified-task 元数据
baseline/result_log.json 零偏航基线结果
```

## 环境

在 Frontier-Engineering 仓库根目录执行:

```bash
python -m venv .venvs/frontier-wake-steering
.venvs/frontier-wake-steering/bin/python -m pip install \
-r benchmarks/WindEnergy/WakeSteeringControl/verification/requirements.txt
```

本任务仅需 CPU,FLORIS 不要求 GPU。

## 直接评测

在任务目录中执行:

```bash
../../../.venvs/frontier-wake-steering/bin/python \
verification/evaluator.py scripts/init.py
```

验证器生成 `metrics.json` 和 `artifacts.json`。仓库自带的零偏航策略保证可行,
其综合分数按定义约为零。

## Unified 评测

任务 ID 为 `WindEnergy/WakeSteeringControl`:

```bash
python -m frontier_eval \
task=unified \
task.benchmark=WindEnergy/WakeSteeringControl \
task.runtime.python_path=uv-env:frontier-wake-steering \
algorithm=openevolve \
algorithm.iterations=0
```

零迭代验证不需要模型 API Key;正式优化使用 Frontier Eval 的常规模型配置。

## 复现与安全

- FLORIS 固定为 4.6.6。
- 验证器使用内置 NREL 5 MW 风机和固定风场。
- 候选代码在带超时的独立子进程中运行。
- unified task 检查验证器与公开契约是否保持只读,修改这些文件的候选会被判为无效。
- 进程隔离不是强安全沙箱;不可信候选应在容器或一次性主机中运行。
63 changes: 63 additions & 0 deletions benchmarks/WindEnergy/WakeSteeringControl/Task.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Task: Wind-Farm Wake-Steering Control

## Background

An upstream turbine extracts energy and leaves a slower wake. When turbines are
aligned with the wind, that wake can reduce the production of downstream
turbines. Deliberately yawing an upstream rotor can redirect its wake. The
upstream turbine loses some power, but the farm may gain more power overall.

## Objective

Implement `yaw_policy()` in `scripts/init.py`. For each wind condition, return
one yaw angle per turbine. Maximize frequency-weighted farm energy relative to
the zero-yaw baseline while avoiding excessive yaw and abrupt changes between
nearby wind directions.

## Inputs

The policy receives Python lists:

- `wind_directions_deg`: meteorological wind directions in degrees;
- `wind_speeds_mps`: wind speeds in metres per second;
- `turbulence_intensities`: dimensionless turbulence intensity values;
- `layout_x_m`, `layout_y_m`: fixed turbine coordinates in metres.

There are `n_conditions` wind conditions and `n_turbines` layout points.

## Output

Return a rectangular nested list or compatible numeric array with shape:

```text
(n_conditions, n_turbines)
```

Every value is a yaw angle in degrees and must lie in `[-25, 25]`. Positive and
negative signs represent opposite steering directions under FLORIS conventions.

## Score

The verifier computes weighted expected farm power for the candidate and the
zero-yaw baseline. The higher-is-better score is:

```text
relative energy gain in percentage points
- 0.01 * frequency-weighted mean absolute yaw in degrees
- 0.002 * mean adjacent-direction yaw change in degrees
```

The zero-yaw baseline therefore scores approximately `0.0`. Raw energy and
penalty components are reported separately.

## Hard Constraints

- Return the exact required shape and only finite numeric values.
- Keep every yaw angle within `[-25, 25]` degrees.
- Return identical results for identical inputs.
- Complete within the candidate timeout.
- Do not modify benchmark, reference, or verifier files.
- Keep the function signature and EVOLVE markers unchanged.

The verifier determines feasibility and score. During unified evaluation,
Frontier Eval also invalidates candidates that modify files declared read-only.
59 changes: 59 additions & 0 deletions benchmarks/WindEnergy/WakeSteeringControl/Task_zh-CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# 任务:风电场尾流偏航控制

## 背景

上游风机提取能量后会形成低速尾流。当风机沿风向排列时,尾流会降低下游风机
发电量。上游风机主动偏航可以改变尾流方向;虽然上游风机会损失部分功率,整个
风电场的总功率可能增加。

## 目标

实现 `scripts/init.py` 中的 `yaw_policy()`。针对每个风况,为每台风机返回一个
偏航角。在限制偏航幅度和相邻风向策略突变的同时,相对零偏航基线最大化加权
风电场发电量。

## 输入

策略接收以下 Python 列表:

- `wind_directions_deg`:气象风向,单位为度;
- `wind_speeds_mps`:风速,单位为米每秒;
- `turbulence_intensities`:无量纲湍流强度;
- `layout_x_m`、`layout_y_m`:固定风机坐标,单位为米。

风况数量为 `n_conditions`,风机数量为 `n_turbines`。

## 输出

返回矩形嵌套列表或兼容数值数组,形状必须为:

```text
(n_conditions, n_turbines)
```

每个值表示偏航角,范围为 `[-25, 25]` 度。正负号表示 FLORIS 约定下两个相反
的尾流转向方向。

## 评分

验证器分别计算候选策略和零偏航基线的加权期望功率。综合分数越高越好:

```text
相对发电量增益(百分点)
- 0.01 * 频率加权平均绝对偏航角
- 0.002 * 相邻风向平均偏航变化
```

因此零偏航基线分数约为 `0.0`。验证器会分别报告原始发电量和惩罚项。

## 硬约束

- 返回精确形状和有限数值;
- 所有偏航角均在 `[-25, 25]` 度内;
- 相同输入必须返回相同结果;
- 必须在候选超时前完成;
- 不得修改任务、参考资料及验证器文件;
- 不得改变函数签名和 EVOLVE 标记。

验证器负责判定可行性和最终分数。在 unified 评测中,Frontier Eval 还会将修改
只读声明文件的候选判为无效。
15 changes: 15 additions & 0 deletions benchmarks/WindEnergy/WakeSteeringControl/baseline/result_log.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
"baseline_expected_power_w": 14562813.187886339,
"candidate_expected_power_w": 14562813.187886339,
"candidate_runtime_s": 0.05653033405542374,
"combined_score": 0.0,
"evaluator_runtime_s": 0.2937989681959152,
"n_conditions": 24.0,
"n_turbines": 9.0,
"relative_energy_gain_pct": 0.0,
"valid": 1.0,
"weighted_mean_abs_yaw_deg": 0.0,
"yaw_effort_penalty": 0.0,
"yaw_smoothness_deg": 0.0,
"yaw_smoothness_penalty": 0.0
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
README.md
README_zh-CN.md
Task.md
Task_zh-CN.md
references/README.md
references/farm_config.json
frontier_eval/constraints.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
metrics.json
artifacts.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
scripts/init.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
WakeSteeringControl constraints:
1) Only modify the EVOLVE block in `scripts/init.py`.
2) Keep the `yaw_policy()` signature unchanged and return shape `(n_conditions, n_turbines)`.
3) Return only finite yaw angles in `[-25, 25]` degrees.
4) Keep the policy deterministic and complete within the candidate timeout.
5) Do not modify task, reference, or verification files; unified evaluation checks them as read-only.
6) Maximize weighted farm-energy gain while limiting yaw effort and directional discontinuity.
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
README.md
README_zh-CN.md
Task.md
Task_zh-CN.md
references
verification
scripts
frontier_eval/run_eval.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
bash frontier_eval/run_eval.sh {python} {candidate}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
scripts/init.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
README.md
README_zh-CN.md
Task.md
Task_zh-CN.md
references
verification
frontier_eval/run_eval.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/usr/bin/env bash
set -euo pipefail

PYTHON_CMD="${1:?missing Python interpreter}"
CANDIDATE_PATH="${2:?missing candidate path}"

exec "${PYTHON_CMD}" verification/evaluator.py "${CANDIDATE_PATH}" \
--json-out metrics.json \
--artifacts-out artifacts.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Public Configuration

`farm_config.json` defines the public farm geometry, yaw limits, timeout, and
score coefficients. Coordinates are expressed in rotor-diameter multiples and
are converted to metres using the NREL 5 MW rotor diameter loaded by FLORIS.

The evaluator owns the deterministic wind-condition ensemble. A candidate
receives every condition it must control through the documented policy inputs.
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"floris_version": "4.6.6",
"turbine_model": "nrel_5MW",
"layout_rotor_diameters": [
[0.0, 0.0],
[5.0, 0.0],
[10.0, 0.0],
[0.0, 5.0],
[5.0, 5.0],
[10.0, 5.0],
[0.0, 10.0],
[5.0, 10.0],
[10.0, 10.0]
],
"yaw_min_deg": -25.0,
"yaw_max_deg": 25.0,
"candidate_timeout_s": 5.0,
"yaw_effort_coefficient": 0.01,
"smoothness_coefficient": 0.002
}
Loading
Loading