Wrap sandbox single_person results to close FRS guard bypass

[anth-volk]

Fixes #160

Summary

The run_python sandbox (backend/engine/sandbox.py) enforces the FRS row-level guard by ensuring sandboxed code can only ever hold SafeSimulation-wrapped objects and never the raw compiled Simulation class. A factory classmethod that returned a raw, unwrapped simulation_cls instance would break that invariant: sandboxed code could recover the unguarded class via type(...) and call RawSimulation(dataset="frs").run_microdata(), defeating the guard and reading row-level FRS microdata.

SafeSimulation.single_person previously forwarded straight to the raw class:

@classmethod
def single_person(cls, *args, **kwargs):
    return simulation_cls.single_person(*args, **kwargs)

This change makes factory classmethods wrap any raw simulation_cls instance they might return:

• Adds an internal SafeSimulation._wrap classmethod that guards an already-built raw instance via object.__new__ + object.__setattr__ (no re-run), with _dataset=None and _is_synthetic=True for single_person (synthetic by construction).
• single_person now wraps the result when it is a raw Simulation instance, and otherwise passes it through unchanged.

The documented use is unchanged: under the pinned compiled contract (policyengine-uk-compiled >= 0.44) single_person is a staticmethod returning (persons, benunits, households) DataFrame frames, which are not simulation_cls instances and pass through untouched. The existing test_allows_synthetic_microdata still passes.

Scope / honesty note

Under the pinned/installed compiled contract single_person (and couple) return DataFrame frames, so type(...) yields tuple and the literal exploit is currently latent. This PR is defense-in-depth: it enforces the "sandboxed code never holds the raw class" invariant for factory classmethods, closing the vector against contract drift. The genuinely-reproducible class-recovery vectors in the current environment (e.g. ().__class__.__bases__[0].__subclasses__() and bound-method __self__ walks) are general CPython sandbox escapes tracked under #136, not the specific FRS-guard-via-factory-return issue this PR addresses.

Self-review

• [medium — transparency] The literal single_person exploit is inert under the pinned tuple contract, and broad class-recovery escapes are run_python is risky #136 territory; this PR is hardening + regression coverage rather than a live-exploit patch. Addressed by documenting explicitly here and in the issue.
• [low-medium] The sandbox-level test test_single_person_return_cannot_recover_frs_capable_class passes vacuously under the current tuple contract. Mitigated by the white-box test_factory_wrap_returns_guarded_simulation, which depends on _wrap (only present with the fix) and proves the guard in this environment. Both kept.
• [low] _wrap is reachable as Simulation._wrap from the sandbox (class-attribute access bypasses the instance __getattribute__). No escalation: it only wraps an object the caller already holds, never returns the raw instance/class, and cannot unwrap a legitimately-obtained guarded FRS object (the inner _simulation stays underscore-blocked and re-guards on delegate). Left as-is.
• [low] couple is not wrapped, but it is also not exposed on SafeSimulation (class lookup fails in the sandbox), so there is no leak; noted for future exposure. Left as-is.

No high-importance items; the one medium item is documentation/transparency, handled above.

Tests

• PYTHONPATH=backend CI=true python -m pytest backend/tests/test_agent_tools.py::TestRunPython -v — 9 passed (includes the 2 new tests).
• Full backend suite (PYTHONPATH=backend python -m pytest backend/tests -q): 199 passed, 10 skipped, 7 failed. All 7 failures are pre-existing on clean main (verified via git stash) and stem from the local compiled package being 0.35.0 (older than the pinned >=0.44.0 contract), which produces the older output shape (KeyError: 'income_tax' in TestCalculateHousehold, plus one chat-route logging test). They are unrelated to this change; the changed-area tests all pass.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
policyengine-uk-chat	Ready	Preview, Comment	Jul 2, 2026 2:20pm

[github-actions]

Beta preview is ready.

• Frontend: open preview
• Backend: open backend

[vahid-ahmadi]

The narrow change is correct and the tests pass, so no objection to the mechanics — but two things should be resolved before this is treated as "the FRS guard is closed," and I'd reword the title/description accordingly.

1. The FRS guard remains bypassable regardless of this fix — please track it separately.
Because type, getattr, and real pandas/numpy objects are exposed in the exec namespace, sandboxed run_python code can recover the unguarded raw Simulation class via the standard object-subclasses walk (starting from a DataFrame's base class), construct it directly with dataset="frs", and call run_microdata() to read real FRS row-level microdata. I verified this end-to-end through the actual run_python tool — it returns a genuine MicrodataResult, and single_person isn't involved at all. (Happy to share the repro privately rather than in a public comment.) So SafeSimulation is best-effort hardening, not a security boundary, and this PR doesn't change that.

2. test_single_person_return_cannot_recover_frs_capable_class is effectively vacuous.
Under the pinned compiled contract, single_person returns a (persons, benunits, households) tuple, so isinstance(result, simulation_cls) is always False and the new _wrap branch is never exercised by that test — it passes only via tuple(dataset="frs") → TypeError. Only test_factory_wrap_returns_guarded_simulation (which builds a raw instance manually) covers the wrapping logic. Consider a test that forces the raw-instance path, or drop the "closes the bypass" framing.

Minor: _wrap is callable from the sandbox as Simulation._wrap(...) — the leading-underscore __getattribute__ block only guards instance access, not class-attribute access — and its is_synthetic=True default makes it a guard-launder for any raw sim. Worth not exposing it on the sandbox-visible class if the guard is meant to be meaningful.

test_agent_tools.py → 96 passed locally. The defensive wrap itself is harmless, so I'm fine merging it as incremental hardening as long as the framing is adjusted and the real bypass is tracked.