Skip to content

Add adaptive compressed telemetry execution benchmark#1

Closed
hyizhak wants to merge 1 commit into
mainfrom
agent/adaptive-compressed-telemetry
Closed

Add adaptive compressed telemetry execution benchmark#1
hyizhak wants to merge 1 commit into
mainfrom
agent/adaptive-compressed-telemetry

Conversation

@hyizhak

@hyizhak hyizhak commented Jul 13, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add AdaptiveCompressedTelemetryExecution, an execution-backed C++20 systems benchmark.
  • Compile and run candidate encode, decode, and compressed-query implementations on deterministic telemetry workloads.
  • Measure encoded bytes and pinned-CPU timings, verify exact decoding and query answers with an independent oracle, and convert the measured results into a storage/compute economic score.
  • Add a raw starter implementation, baseline measurements, resource limits, unified Frontier metadata, and English/Chinese task index entries.

Why

This adds a realistic optimization task with economic value that cannot be solved by returning an analytical estimate. Candidate code is actually executed and can earn continuous partial credit by improving compression and CPU performance while preserving exact correctness.

Validation

  • Raw starter: valid=1, correctness=1, combined_score=1.0 across three scenarios.
  • Unified zero-iteration Frontier evaluation: passed with score 1.0.
  • Alternate-seed evaluation: passed.
  • Invalid decode and timeout diagnostics: correctly rejected.
  • Simple optimized diagnostic codec: exact results, 0.232 compression ratio, and repeat scores of 2.88x and 2.90x, demonstrating optimization headroom.
  • Staged diff whitespace check, JSON validation, and starter/baseline identity check: passed.

@hyizhak

hyizhak commented Jul 13, 2026

Copy link
Copy Markdown
Owner Author

Superseded by the upstream contribution at EinsiaLab#94.

@hyizhak hyizhak closed this Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant