Skip to content

algorithm.md

Latest commit

 

History

History
164 lines (112 loc) · 10.4 KB

algorithm.md

File metadata and controls

164 lines (112 loc) · 10.4 KB
Raw

How CodeHeat works — algorithms & internals

This is the deep dive behind the README: how each score is computed, the design trade-offs, the output schema, and the known limitations. If you just want to run it, the README is enough — read this when you want to trust (or challenge) the numbers.

Smell Score

Cyclomatic complexity (CCN) only counts branches, so a 3-deep nested if scores the same as three flat ones — even though the nested version is far harder to read. Borrowing the core intuition of cognitive complexity (nesting hurts most), CodeHeat gives each function:

smell = CCN + nesting penalty + parameter penalty + length penalty
  • Each penalty is penalty-free up to a tolerance — normal functions aren't punished for having a couple of parameters or one level of nesting.
  • Nesting is weighted heaviest, matching how much deep nesting actually costs a reader.
  • The nesting term is capped, so a single pathologically deep function (or a parser artifact — see limitations) can't dominate the whole ranking.

The file's score is its worst function, and that function's name is reported as hotspot_function — a semantic pointer to where to start. The raw max_nesting_depth and max_param_count are reported alongside so you can see what drove the score.

Nesting depth comes from lizard's nd (nesting-depth) extension, so it's language-agnostic — the same scoring works across Python, JS/TS, Go, etc.

Files are ranked by Smell Score descending; ties break on CCN.

TODO detection

TODO/FIXME detection is token-based, not a naive grep. Pygments tokenizes the source and only real comment tokens are inspected for markers. That means all of these are correctly excluded:

  • docstring prose that happens to say "todo"
  • regex literals and argument names
  • fake comments inside string literals — e.g. s = "# TODO" is a string, not a task

When Pygments can't find a lexer for a file, a line-anchored regex fallback kicks in (it still masks single-line string literals to avoid the obvious false positives).

Age in days comes from git log -S (pickaxe) on the TODO text — so you can rank by "this TODO has been rotting for 400 days", not just its presence.

Ownership scoring

The goal is matching, not blame: find the person who can fix a hotspot fastest, i.e. whoever has the most domain knowledge of it.

score(contributor) =   Σ    recency_weight(commit) × change_weight(commit)
                   commits by them

recency_weight = 0.5 ^ (age_days / 365)          # 1-year half-life
change_weight  = 1 + max_CCN(touched functions)  # function-level (preferred)
               = 1 + ln(1 + churn)               # fallback when CCN isn't computable
  • Recency (recency_weight) — a commit's weight halves every 365 days: today = 1.0, one year ago = 0.5, two years ago = 0.25. age_days is clamped at ≥ 0. Recent work dominates.
  • Change magnitude (change_weight) is function-level. The commit's diff hunks (git show --unified=0) are intersected with the file's function ranges at that commit, and only the max CCN of the functions it actually touched is credited as 1 + ccn. So you are not tagged an expert on a 3,000-line file because you fixed one unrelated typo in it. When the function-level CCN can't be computed (unparseable revision, pure rename with zero churn, or a commit outside the newest-N window), it falls back to 1 + ln(1 + churn) on the commit's changed-line count.
  • Contributors are aggregated by email (%aE, .mailmap-aware); merge commits are excluded (--no-merges).

Scaling to large histories

Per-commit complexity needs a git show, so a file with tens of thousands of commits can bottleneck on I/O. Two guards keep it bounded:

  1. Newest-N window — the expensive function-level complexity path only runs for the newest N commits (--complexity-limit, default 200). Older commits have near-zero recency weight anyway, so they fall back to cheap churn.
  2. Run-scoped blob cache — each (commit, path) blob analysis is memoized, so overlapping work (e.g. a PR's head commit vs. its ownership history) isn't recomputed.

--churn-only skips the complexity path entirely for maximum speed.

Parsing & failure handling

Every external parse is defensive — a failure degrades gracefully instead of crashing:

  • lizard (complexity) — a file or historical revision lizard can't parse (unsupported extension, or a syntactically broken blob at some old commit) simply yields no functions. _analyze_blob catches any exception and returns None, and that commit falls back to churn weighting. The None result is cached in the run-scoped blob cache, so a bad revision isn't re-parsed on every pass.
  • Pygments (TODO tokenizing) — when no lexer matches a file, or lexing raises, TODO detection falls back to the line-anchored regex (which still masks single-line string literals). Mainstream languages always take the precise Pygments path; the fallback only affects exotic/unknown extensions.
  • git — every git call runs with a timeout and is treated as best-effort: on a non-zero exit or timeout it returns None. TODO age then becomes null, and complexity deltas fall back to churn. A missing/broken git never aborts the run.
  • Ollama (insights) — a short pre-flight socket check fails fast (≈3 s) with a clear "start ollama serve" message instead of blocking on the 120 s generation timeout when the local server is down (see the CLI's default backend).

AI insights — metadata only

The insights stage bundles the stage 1+2 JSON, hands it to an LLM, and gets back a refactor priority and a "ask who, about what" per file. Source code is never sent — only numbers and metadata.

To keep the model from parroting generic "it's complex, refactor it" advice while still never sending code, the prompt carries opaque metadata as context:

  • the hotspot_function name
  • the file's other function names
  • imported libraries
  • its directory location

That's enough for the model to infer what the file is about (a router? a training loop? an auth flow?) and phrase a specific question — without a single line of the source body.

Backends:

  • ollama (default) — stdlib-only, zero extra deps, free and local.
  • anthropic — uses claude-opus-4-8 with structured output to strictly enforce the schema. Needs pip install codeheat[llm] + ANTHROPIC_API_KEY.

--dry-run prints the exact assembled prompt without calling any LLM, so you can audit precisely what would leave your machine.

PR bot — rule-based filtering

Don't be a nagging bot. A CCN that ticked up by one shouldn't fire a warning on every push — that's how teams end up muting the bot.

A change is only critical when:

  • the resulting CCN clears an absolute threshold (--min-ccn, default 15) and got hotter, or
  • the delta jumps by --critical-delta (default +5) in a single change.

With no critical files, the comment softens to a "for reference" summary and, by default, no new comment is posted — but an existing one is still updated, so a past warning self-heals to "resolved". Flags:

  • --always-comment — force a comment even when nothing is critical.
  • --fail-on-critical — exit non-zero to block the merge.

The workflow reads PR context from the event payload + GITHUB_* env vars, comments with GITHUB_TOKEN, and upserts via a hidden marker so repeated pushes update one comment instead of spamming. Comment posting uses stdlib urllib only (zero extra deps). Checkout must use fetch-depth: 0 — the base...head delta needs full history.

Output schema

smell_report.json, sorted by Smell Score descending:

{
  "repo_path": ".",
  "file_count": 3,
  "files": [
    {
      "file": "codeheat/static_scan.py",
      "complexity": 6,
      "avg_complexity": 2.5,
      "function_count": 7,
      "loc": 90,
      "todos": [{ "line": 42, "text": "TODO: ...", "age_days": 3 }],
      "duplication_ratio": 0.0,
      "max_nesting_depth": 4,
      "max_param_count": 3,
      "smell_score": 12.0,
      "hotspot_function": "build_smell_reports",
      "function_names": ["scan_complexity", "find_todos", "build_smell_reports"],
      "imports": ["lizard", "os", "re", "subprocess"],
      "oldest_todo_days": 3
    }
  ]
}

Each stage emits JSON that the next consumes, and the schema is built so the LLM layer only ever sees numbers and metadata, never code.

Known limitations

Being honest about the edges:

Scan

  • TODO detection is token-based; the regex fallback's only blind spot is fake comments inside multi-line string literals (mainstream languages take the Pygments path and aren't affected).
  • duplication_ratio is computed per function; git log -S (pickaxe) uses only the first 40 chars of the TODO text, so very generic text may mismatch. age_days is null when git is unavailable / times out / the file is untracked.
  • exclude is directory-level only (node_modules/.git/venv/.venv) plus build outputs (.next/out/dist/build).

Ownership

  • Change magnitude uses function-level matching, but attribution still relies on lizard being able to parse the file at that historical commit; unparseable revisions fall back to churn.
  • Per-commit complexity runs git show (in-memory, no temp files). It's bounded by --complexity-limit and a run-scoped blob cache, but a file with thousands of recent commits is still work — --churn-only skips the complexity path entirely.

PR bot

  • Temperature is file-level max CCN, so splitting a function may not lower it if another function is more complex. Critical-gating is threshold-based, not per-function. Non-code files (.md/.json/.toml) are excluded. A brand-new file shows "🆕 new" instead of a delta.

Smell Score

  • Nesting depth comes from lizard's nd extension, which can over-count on modern TS/JS operators (??, ?., ternary chains). The score caps the nesting term to blunt this, but the raw max_nesting_depth field can still read high on such files — treat it as a signal, not a measurement. (tree-sitter-based analysis is on the roadmap for a more accurate source.)