Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions examples/skills_code_review_agent/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
output/
output_clean/
*.db
*.sqlite
*.sqlite3
__pycache__/
.pytest_cache/
159 changes: 159 additions & 0 deletions examples/skills_code_review_agent/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
# Skills Code Review Agent

This example implements the early phases of issue #92 as a deterministic,
local-only code review loop. It reads a unified diff, scans added lines with a
small static rule set, redacts likely secrets, and writes JSON and Markdown
reports. When `--db-path` is provided, it also persists the review task,
findings, report metadata, sandbox run metadata, and filter decisions into
SQLite.

This is a lightweight deterministic baseline, not a full sandbox/scanner
implementation. It intentionally does not call an LLM, remote sandbox, Docker,
Cube, E2B, external scanner, or SDK core extension. The sandbox runner in this
example is a fake dry-run runner: it records what would have happened, but
never executes untrusted code.

## Run

From the repository root:

```bash
python examples/skills_code_review_agent/run_agent.py --diff-file examples/skills_code_review_agent/fixtures/security.diff --output-dir examples/skills_code_review_agent/output --dry-run
```

Expected output files:

```text
examples/skills_code_review_agent/output/
review_report.json
review_report.md
```

You can also run the clean fixture:

```bash
python examples/skills_code_review_agent/run_agent.py --diff-file examples/skills_code_review_agent/fixtures/clean.diff --output-dir examples/skills_code_review_agent/output_clean --dry-run
```

## Run From Stdin

`--diff-file -` reads a unified diff from stdin and records the report input as
`<stdin>`:

```bash
git diff | python examples/skills_code_review_agent/run_agent.py --diff-file - --output-dir examples/skills_code_review_agent/output --dry-run
```

## CI Failure Gate

By default the command exits with `0` even when findings are present. Use
`--fail-on-severity` to make a lightweight CI gate:

```bash
python examples/skills_code_review_agent/run_agent.py --diff-file examples/skills_code_review_agent/fixtures/security.diff --output-dir examples/skills_code_review_agent/output --dry-run --fail-on-severity high
```

Values are `never`, `low`, `medium`, and `high`. For example, `medium` fails on
medium or high findings, while `high` fails only on high findings.

## List Rules

```bash
python examples/skills_code_review_agent/run_agent.py --list-rules
```

This prints the deterministic rule id, category, default severity, description,
and known limitations without reading a diff.

## Run With SQLite

```bash
python examples/skills_code_review_agent/run_agent.py --diff-file examples/skills_code_review_agent/fixtures/security.diff --output-dir examples/skills_code_review_agent/output --db-path examples/skills_code_review_agent/output/reviews.sqlite3 --dry-run
```

When `--db-path` is set, the command prints the database path and generated
task id. The database contains these tables:

- `review_tasks`
- `findings`
- `reports`
- `sandbox_runs`
- `filter_decisions`

## Verify SQLite Records

Using Python:

```bash
python -c "import sqlite3; db='examples/skills_code_review_agent/output/reviews.sqlite3'; con=sqlite3.connect(db); print(con.execute('select task_id,total_findings from review_tasks order by created_at desc limit 1').fetchone()); print(con.execute('select severity,category,file,line,title from findings order by id limit 5').fetchall())"
```

Using the `sqlite3` CLI:

```bash
sqlite3 examples/skills_code_review_agent/output/reviews.sqlite3 "select severity, category, file, line, title from findings order by id;"
```

## Run Tests

```bash
python -m pytest examples/skills_code_review_agent/tests
```

The tests run only local parsing, rules, redaction, report generation, and
dedupe checks, fake sandbox status, filter decisions, telemetry summaries, and
SQLite persistence. They also cover the lightweight CI failure gate, stdin diff
input, and rule listing. They do not call an LLM, Docker, Cube, remote network,
external scanners, or any external service.

## Fixtures

- `clean.diff`: safe changes; should produce no high-severity findings.
- `security.diff`: mixed security sample covering secret, missing timeout, broad exception, SQL risk, and resource lifecycle.
- `sql_injection.diff`: SQL string concatenation.
- `missing_timeout.diff`: `httpx` request without `timeout=`.
- `broad_except.diff`: broad `except Exception` with swallowed failure.
- `resource_leak.diff`: `open(...)` without a context manager.
- `duplicate.diff`: duplicate hunk producing the same finding, used to verify dedupe.
- `secret_redaction.diff`: multiple secret-like values, used to verify reports omit plaintext secrets.

## Current Scope

- Parses unified diff hunks and added line numbers.
- Runs five deterministic rules:
- hardcoded secret / token / password
- SQL string concatenation risk
- `requests` / `httpx` calls missing `timeout=`
- broad `except Exception` or simple error swallowing
- `open(...)` without `with`
- Redacts likely API keys, tokens, secrets, and passwords before writing reports.
- Produces `review_report.json` and `review_report.md`.
- Supports reading a unified diff from stdin with `--diff-file -`.
- Supports a lightweight exit-code gate with `--fail-on-severity`.
- Prints rule metadata with `--list-rules`.
- Optionally persists review tasks, findings, and report metadata to SQLite.
- Records minimal filter decisions: `allow`, `deny`, or `needs_human_review`.
- Records a fake dry-run sandbox result without executing untrusted code.
- Adds telemetry summary fields to JSON and Markdown reports.
- Includes local pytest coverage for the deterministic rule fixtures.

## Sandbox Notes

The current `dry-run` sandbox runner is deliberately fake and safe for local
development. It simulates a static-check pass and records status/timing fields.
It does not run shell commands, install dependencies, invoke Docker, or reach a
remote sandbox service.

A true local runner should only be used as a development fallback because it
does not isolate untrusted code. Later phases can replace this example runner
with the project-level `ContainerWorkspaceRuntime` or `CubeWorkspaceRuntime`
without changing the report and storage shape introduced here.

## Not Implemented Yet

- Real LLM review.
- Real remote sandbox or container execution.
- External scanner integration such as Bandit, Ruff, detect-secrets, or Semgrep.
- SDK core integration.
- Production-grade filter governance and telemetry export.
- Multi-agent orchestration.
2 changes: 2 additions & 0 deletions examples/skills_code_review_agent/agent/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
"""Phase-1 deterministic code review example helpers."""

132 changes: 132 additions & 0 deletions examples/skills_code_review_agent/agent/diff_parser.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
"""Minimal unified diff parser for deterministic review fixtures."""

from __future__ import annotations

import re
from dataclasses import dataclass
from typing import Iterable


_HUNK_RE = re.compile(
r"^@@ -(?P<old_start>\d+)(?:,(?P<old_count>\d+))? \+(?P<new_start>\d+)(?:,(?P<new_count>\d+))? @@(?P<context>.*)$"
)


@dataclass(frozen=True)
class ChangedLine:
"""A line changed by a unified diff hunk."""

file_path: str
line_number: int
content: str
hunk_header: str
change_type: str


@dataclass(frozen=True)
class ParsedDiff:
"""Parsed unified diff data used by the static rule scanner."""

files: list[str]
changed_lines: list[ChangedLine]


def _normalize_diff_path(raw: str) -> str:
value = raw.strip()
if not value or value == "/dev/null":
return value
path = value.split("\t", maxsplit=1)[0].split(" ", maxsplit=1)[0]
if path.startswith("a/") or path.startswith("b/"):
return path[2:]
return path


def _unique_in_order(values: Iterable[str]) -> list[str]:
seen: set[str] = set()
out: list[str] = []
for value in values:
if value and value not in seen:
seen.add(value)
out.append(value)
return out


def parse_unified_diff(diff_text: str) -> ParsedDiff:
"""Parse file paths, hunk headers, and changed lines from unified diff text.

The parser records added and removed lines, but phase-1 rules only scan
additions. Context lines are used only to keep line counters accurate.
"""

files: list[str] = []
changed_lines: list[ChangedLine] = []

current_file = ""
pending_file = ""
hunk_header = ""
old_line = 0
new_line = 0
in_hunk = False

for raw_line in diff_text.splitlines():
if raw_line.startswith("diff --git "):
parts = raw_line.split()
if len(parts) >= 4:
pending_file = _normalize_diff_path(parts[3])
current_file = pending_file
files.append(current_file)
in_hunk = False
hunk_header = ""
continue

if raw_line.startswith("+++ "):
path = _normalize_diff_path(raw_line[4:])
if path != "/dev/null":
current_file = path
files.append(current_file)
elif pending_file:
current_file = pending_file
continue

match = _HUNK_RE.match(raw_line)
if match:
hunk_header = raw_line
old_line = int(match.group("old_start"))
new_line = int(match.group("new_start"))
in_hunk = True
continue

if not in_hunk or not current_file:
continue

if raw_line.startswith("+") and not raw_line.startswith("+++"):
changed_lines.append(
ChangedLine(
file_path=current_file,
line_number=new_line,
content=raw_line[1:],
hunk_header=hunk_header,
change_type="add",
)
)
new_line += 1
elif raw_line.startswith("-") and not raw_line.startswith("---"):
changed_lines.append(
ChangedLine(
file_path=current_file,
line_number=old_line,
content=raw_line[1:],
hunk_header=hunk_header,
change_type="delete",
)
)
old_line += 1
elif raw_line.startswith(" "):
old_line += 1
new_line += 1
elif raw_line.startswith("\\"):
continue
else:
in_hunk = False

return ParsedDiff(files=_unique_in_order(files), changed_lines=changed_lines)
56 changes: 56 additions & 0 deletions examples/skills_code_review_agent/agent/filtering.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
"""Minimal filter decision abstraction for the code review example."""

from __future__ import annotations

import re
from dataclasses import asdict
from dataclasses import dataclass

from .diff_parser import ParsedDiff


_HIGH_RISK_COMMAND_RE = re.compile(
r"\b("
r"rm\s+-rf\s+/|"
r"curl\b[^|;&]*\|\s*(?:sh|bash)|"
r"wget\b[^|;&]*\|\s*(?:sh|bash)|"
r"chmod\s+777|"
r"Invoke-WebRequest\b[^|;&]*\|\s*iex|"
r"iwr\b[^|;&]*\|\s*iex"
r")",
re.IGNORECASE,
)
_MAX_DIFF_BYTES = 200_000
_MAX_CHANGED_LINES = 2_000


@dataclass(frozen=True)
class FilterDecision:
"""A minimal filter decision for review governance."""

decision: str
reason: str

def to_dict(self) -> dict[str, str]:
return asdict(self)


def evaluate_filter_decision(diff_text: str, parsed_diff: ParsedDiff) -> FilterDecision:
"""Evaluate whether the review can proceed automatically."""

if len(diff_text.encode("utf-8")) > _MAX_DIFF_BYTES:
return FilterDecision(
decision="needs_human_review",
reason=f"diff is larger than {_MAX_DIFF_BYTES} bytes",
)
if len(parsed_diff.changed_lines) > _MAX_CHANGED_LINES:
return FilterDecision(
decision="needs_human_review",
reason=f"diff changes more than {_MAX_CHANGED_LINES} lines",
)
if _HIGH_RISK_COMMAND_RE.search(diff_text):
return FilterDecision(
decision="needs_human_review",
reason="diff contains a high-risk command pattern",
)
return FilterDecision(decision="allow", reason="diff is within dry-run review limits")
24 changes: 24 additions & 0 deletions examples/skills_code_review_agent/agent/findings.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
"""Finding data model for deterministic code review."""

from __future__ import annotations

from dataclasses import asdict
from dataclasses import dataclass


@dataclass(frozen=True)
class Finding:
"""A structured code review finding."""

severity: str
category: str
file: str
line: int
title: str
evidence: str
recommendation: str
confidence: float
source: str

def to_dict(self) -> dict[str, object]:
return asdict(self)
Loading
Loading