fix(analyzer): gate documentation false positives for PE3/RA1/TM1/AR2#266
Open
rodboev wants to merge 5 commits into
Open
fix(analyzer): gate documentation false positives for PE3/RA1/TM1/AR2#266rodboev wants to merge 5 commits into
rodboev wants to merge 5 commits into
Conversation
…NVIDIA#251) Signed-off-by: Rod Boev <rod.boev@gmail.com>
Signed-off-by: Rod Boev <rod.boev@gmail.com>
Signed-off-by: Rod Boev <rod.boev@gmail.com>
Signed-off-by: Rod Boev <rod.boev@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Benign security documentation trips several high-confidence static rules at once. Credential-hygiene docs are flagged as PE3 (Credential Access), shell-script security comments as RA1 (Self-Modification), an uninstaller's documentation as TM1 (Tool Parameter Abuse), and tone-guidance prose as AR2 (Anti-Refusal Statement). This is one recurring mechanism: a semantic-string rule matching English/comment vocabulary in a non-executable documentation context, surfacing across four rule families.
Closes #251
Root cause
The four implicated rules gate documentation context inconsistently, and the shared runner only partially covers it.
static_runner.pyalready carries a shared PE3-only.envdocumentation filter (_is_env_file_reference_in_docs) and adocs/-subdir confidence downweight (_is_documentation_markdown), but nothing suppresses a plain-prose or comment hit for the general semantic-string rules. PE3 has a narrow local_is_documentation_example, RA1 has no documentation gating, TM1 gates only container/dockerfile idioms, and AR2'sis_code_examplepenalty misses prose that isn't a fenced code example. The same benign vocabulary therefore lands as HIGH findings depending on the matching family.Diff Notes
static_runner.py, keyed on{PE3, RA1, TM1, AR2}. AST, behavioral, semantic-LLM, and all other rules remain excluded._is_documentation_context) suppresses a governed finding only from the matched finding line: markdown or text prose stays suppressible only when the line lacks executable signals,#and//comment lines stay suppressible, neighboring comments no longer suppress executable findings, inline/* ... */prefixes no longer mask executable content, and executable API calls such asshutil.rmtree(...)stay active even inside.mdand.txtfiles.SKILL.mdremains active, and JSON/YAML/TOML config files stay analyzable..envand example checks, TM1's container and Dockerfile idioms, and AR2's code-example penalty.tests/nodes/analyzers/test_static_runner_filtering.pywith suppression fixtures for all four families and negative-space fixtures proving real malicious content with the same vocabulary still emits.Before / After
docs/credentials.md# security note: this script rewrites itself.shdocs/uninstall.mddocs/tone.mdtoken = os.environ["AWS_SECRET_ACCESS_KEY"].pyrun: rm -rf /opt/example/dataconfig.yamlopen(__file__, "w").py/.shSKILL.mdScope
No analyzer module changes — the PE/RA/TM/AR pattern lists and local heuristics are untouched. The gate is rule-scoped and context-gated: it never suppresses on rule id alone, never touches non-governed rules (SC*, EA*, MP*, behavioral/semantic), and never suppresses in SKILL.md or in any context with a code-execution signal. No severity-model, reporter-schema, or MCP change. Adjacent PR #254 works inside the anti-refusal analyzer on match semantics; this runner-level context filter does not overlap it.
Verification
uv sync --all-extras— completed successfully.uv run pytest tests/nodes/analyzers/test_static_runner_filtering.py tests/nodes/analyzers/test_static_patterns.py tests/nodes/analyzers/test_static_patterns_anti_refusal.py tests/nodes/test_meta_analyzer.py -q—116 passed, 6 xfailed in 0.86s.uv run ruff check src/ tests/—All checks passed!.uv run ruff format --check src/ tests/—144 files already formatted.