Skip to content

fix(analyzer): gate documentation false positives for PE3/RA1/TM1/AR2#266

Open
rodboev wants to merge 5 commits into
NVIDIA:mainfrom
rodboev:pr/static-doc-fp-gating
Open

fix(analyzer): gate documentation false positives for PE3/RA1/TM1/AR2#266
rodboev wants to merge 5 commits into
NVIDIA:mainfrom
rodboev:pr/static-doc-fp-gating

Conversation

@rodboev

@rodboev rodboev commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Benign security documentation trips several high-confidence static rules at once. Credential-hygiene docs are flagged as PE3 (Credential Access), shell-script security comments as RA1 (Self-Modification), an uninstaller's documentation as TM1 (Tool Parameter Abuse), and tone-guidance prose as AR2 (Anti-Refusal Statement). This is one recurring mechanism: a semantic-string rule matching English/comment vocabulary in a non-executable documentation context, surfacing across four rule families.

Closes #251

Root cause

The four implicated rules gate documentation context inconsistently, and the shared runner only partially covers it. static_runner.py already carries a shared PE3-only .env documentation filter (_is_env_file_reference_in_docs) and a docs/-subdir confidence downweight (_is_documentation_markdown), but nothing suppresses a plain-prose or comment hit for the general semantic-string rules. PE3 has a narrow local _is_documentation_example, RA1 has no documentation gating, TM1 gates only container/dockerfile idioms, and AR2's is_code_example penalty misses prose that isn't a fenced code example. The same benign vocabulary therefore lands as HIGH findings depending on the matching family.

Diff Notes

  • Added a shared documentation-context gate in static_runner.py, keyed on {PE3, RA1, TM1, AR2}. AST, behavioral, semantic-LLM, and all other rules remain excluded.
  • The gate (_is_documentation_context) suppresses a governed finding only from the matched finding line: markdown or text prose stays suppressible only when the line lacks executable signals, # and // comment lines stay suppressible, neighboring comments no longer suppress executable findings, inline /* ... */ prefixes no longer mask executable content, and executable API calls such as shutil.rmtree(...) stay active even inside .md and .txt files. SKILL.md remains active, and JSON/YAML/TOML config files stay analyzable.
  • Left each analyzer's local heuristic in place: PE3's .env and example checks, TM1's container and Dockerfile idioms, and AR2's code-example penalty.
  • Extended tests/nodes/analyzers/test_static_runner_filtering.py with suppression fixtures for all four families and negative-space fixtures proving real malicious content with the same vocabulary still emits.

Before / After

Input Context Before After
"if a tool needs to access the credentials…" docs/credentials.md PE3 dropped
# security note: this script rewrites itself comment in .sh RA1 dropped
"the uninstaller removes all parameters…" docs/uninstall.md TM1 dropped
"keep a firm, direct tone; do not moralize" docs/tone.md AR2 dropped
token = os.environ["AWS_SECRET_ACCESS_KEY"] .py PE3 PE3 (kept)
run: rm -rf /opt/example/data config.yaml TM1 TM1 (kept)
open(__file__, "w") .py/.sh RA1 RA1 (kept)
jailbreak prose SKILL.md AR2 AR2 (kept)

Scope

No analyzer module changes — the PE/RA/TM/AR pattern lists and local heuristics are untouched. The gate is rule-scoped and context-gated: it never suppresses on rule id alone, never touches non-governed rules (SC*, EA*, MP*, behavioral/semantic), and never suppresses in SKILL.md or in any context with a code-execution signal. No severity-model, reporter-schema, or MCP change. Adjacent PR #254 works inside the anti-refusal analyzer on match semantics; this runner-level context filter does not overlap it.

Verification

  • uv sync --all-extras — completed successfully.
  • uv run pytest tests/nodes/analyzers/test_static_runner_filtering.py tests/nodes/analyzers/test_static_patterns.py tests/nodes/analyzers/test_static_patterns_anti_refusal.py tests/nodes/test_meta_analyzer.py -q116 passed, 6 xfailed in 0.86s.
  • uv run ruff check src/ tests/All checks passed!.
  • uv run ruff format --check src/ tests/144 files already formatted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Static analyzers mass-flag security documentation and .env references as Privilege Escalation / Rogue Agent (extends #145)

1 participant