Skip to content

Add adversarial safety fixtures for service runtime (shared helpers services) #142

Description

@haasonsaas

Summary

Exercise prompt/tool/data poisoning and fail-closed behavior for the repo's most sensitive agent-facing path.

This issue was generated from an org-wide EvalOps mining pass on 2026-05-10 07:57 UTC. It combines live GitHub repo signals with a per-repo arXiv search. Treat the research links as grounding for a concrete implementation, not as a request for a literature review.

Repo Evidence

  • Repository description: Shared Go runtime helpers for EvalOps services
  • Tree signals: 0 docs files, 4 workflows, 1 proto files, 11 test-like files.
  • README.md:47 includes latent-spec language: The automation intentionally stays conservative and only advances the patch line. If maintainers need to start a new minor or major line, cut that seed tag manually first and the workflow will continue from there on later merges.
  • README.md:91 includes latent-spec language: Use this package directly when a service needs retry behavior but still wants to own the actual bootstrap logic and logging. This is the pattern used by
  • README.md:94 includes latent-spec language: services that want retry logs around each failed database attempt, but still need one place to register shutdown work such as tracing flushes.
  • README.md:203 includes latent-spec language: Use OpenAndInit when a service needs to run bootstrap logic after the DB is reachable, such as:
  • README.md:298 includes latent-spec language: Helpers for the shared EvalOps mTLS contract.
  • README.md:308 includes latent-spec language: Use this package when a service needs the same client/server TLS file-path contract that memory, registry, meter, and audit share today.

Research Grounding

Repo axes: infra, governance, security, evaluation

Search keywords: ctx, err, startup, helpers, when, bootstrap, use, cfg, client, context, mtls, testutil

  • arXiv:2604.03262v1 AI Governance Control Stack for Operational Stability: Achieving Hardened Governance in AI Systems (Horatio Morgan), 2026.
  • arXiv:2503.15577v1 Navigating MLOps: Insights into Maturity, Lifecycle, Tools, and Careers (Jasper Stone, Raj Patel, Farbod Ghiasi, Sudip Mittal, Shahram Rahimi), 2025.
  • arXiv:2601.20415v1 An Empirical Evaluation of Modern MLOps Frameworks (Jon Marcos-Mercadé, Unai Lopez-Novoa, Mikel Egaña Aranguren), 2026.
  • arXiv:2604.24801v2 Architectural Observability Collapse in Transformers (Thomas Carmichael), 2026.
  • arXiv:2502.15859v4 AI Governance InternationaL Evaluation Index (AGILE Index) 2024 (Yi Zeng, Enmeng Lu, Xin Guan, Cunqing Huangfu, Zizhe Ruan, Ammar Younas), 2025.
  • arXiv:2510.21203v1 The Nuclear Analogy in AI Governance Research (Sophia Hatz), 2025.
  • arXiv:2305.14865v1 A Game-Theoretic Framework for AI Governance (Na Zhang, Kun Yue, Chao Fang), 2023.
  • arXiv:2407.01557v1 AI Governance and Accountability: An Analysis of Anthropic's Claude (Aman Priyanshu, Yash Maurya, Zuofei Hong), 2024.
  • arXiv:2512.11541v1 A Multi-Criteria Automated MLOps Pipeline for Cost-Effective Cloud-Based Classifier Retraining in Response to Data Distribution Shifts (Emmanuel K. Katalay, David O. Dimandja, Jordan F. Masakuna), 2025.
  • arXiv:1902.03439v1 PoliFi: Airtime Policy Enforcement for WiFi (Toke Høiland-Jørgensen, Per Hurtig, Anna Brunstrom), 2019.

What To Build

  • Add adversarial fixtures for deployment drift, credentials, and privileged workflow inputs.
  • Document the intended fail-closed behavior and any allowed degraded-mode fallback.
  • Add regression coverage that proves unsafe inputs do not silently reach the privileged path.

Acceptance Criteria

  • A short design note names the repo-specific workflow, threat or correctness model, and the research assumptions being adopted.
  • A runnable check, fixture, or verifier exercises the new contract in CI or an equivalent local command documented in the repo.
  • The implementation emits or stores enough evidence for a downstream agent/operator to cite inputs, decisions, and outputs.
  • At least one negative/degraded-mode case is covered so failures are observable rather than silently accepted.
  • Documentation links the new behavior to the relevant EvalOps platform primitive or explicitly records why this repo remains standalone.

Notes

  • Generated issue 3/5 for evalops/service-runtime by evalops_org_miner.py.
  • Before implementation, confirm the sampled latent-spec snippets still match main; this issue intentionally cites exact file paths/lines where the mining pass saw them.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions