Skip to content

Host test suite + coverage report (2/2, stacked on #2)#3

Closed
d33mobile wants to merge 19 commits into
hakierspejs:masterfrom
d33mobile:chore/host-tests
Closed

Host test suite + coverage report (2/2, stacked on #2)#3
d33mobile wants to merge 19 commits into
hakierspejs:masterfrom
d33mobile:chore/host-tests

Conversation

@d33mobile

@d33mobile d33mobile commented Jul 8, 2026

Copy link
Copy Markdown

Scope — host test suite + coverage (2/2)

Second of two split PRs. Stacked on #2 (the clang-format/shellcheck tooling). GitHub can't base a cross-fork PR on a fork branch, so until #2 merges this PR's diff also contains its formatting commits — please review #2 first; the test-specific commits are everything after 62cab50.

Contents

  • test/ — host harnesses (native Linux ELF) for the hardware-independent logic: harness_secret_qr.c (secret → base32 → otpauth URI → QR, mirroring serial/commands_keys.c) and harness_base64.c (base64 roundtrip). test/Makefile targets asan / valgrind / coverage, needing a single host stub (test/stub/pico/rand.h).
  • ./ci --check=test — --action=check gates on ASan+UBSan + Valgrind; coverage is report-only.
  • CI test job + pages.yml — ubuntu-latest, submodules recursive, valgrind + lcov; publishes the coverage report to GitHub Pages.
  • Bug fix — UBSan caught a signed-overflow UB in libs/base32/base32.c (int → uint32_t accumulator); the harness is the regression test.

Coverage report

lcov + genhtml over the modules the harnesses exercise (base32, base64, random): overall 98.4% lines, 84.8% branches. Live from my fork: https://d33mobile.github.io/hslock/

CI status

master has no workflows yet, so GitHub runs no pull_request checks here. Green on my fork (build / deploy / lint / test): d33mobile#3

🤖 Generated with Claude Code

Claude and others added 19 commits July 8, 2026 04:56
@claude
Add clang-format config + CI format check, reformat sources
1727bba 
Introduce a `.clang-format` tuned to the existing hand-written style
(4-space indent, spaces-only, K&R braces, 100-col) and a GitHub Actions
job that enforces it on our sources (vendored `libs/` and generated
`version.h` are excluded).

The config keeps `SortIncludes: false` and enables the `AlignConsecutive*`
options so most of the author's manual column alignment (aligned #defines,
struct members) survives. Remaining reflow is alignment inside call-args
and brace-init tables, which clang-format cannot preserve.

The accompanying reformat is provably whitespace-only: with all whitespace
stripped, every touched file's token stream is byte-identical to HEAD.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
Preserve keypad KEY_MAP as a 4x4 grid via clang-format off
1eacca4 
The keypad map mirrors the physical 4x4 button layout; keeping it on one
row per matrix row is more readable than the collapsed single line
clang-format produces. Guard it with clang-format off/on.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
Add ./ci entrypoint (lint fix / check)
2392e3f 
Minimal dispatcher. Default `--check=lint --action=fix` formats sources
in place; `--action=check` does a non-zero-on-diff dry run for a git
pre-commit hook or CI. Same file set as the workflow (excludes libs/ and
version.h).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
Add shellcheck to ./ci lint; unify CI on ./ci --action=check
12f0ccc 
- ./ci now shellchecks shell scripts (discovered by shebang) alongside
  clang-format, under the same lint check.
- build.sh: quote $(nproc) to fix the sole pre-existing SC2046 warning.
- CI now runs ./ci --action=check (single source of truth) instead of a
  separate clang-format action; clang-format is pinned to 19.1.7 via pipx,
  shellcheck is preinstalled on the runner. Drops format.yml for ci.yml.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
Actually add ci.yml + shellcheck integration (fix prior partial commit)
62cab50 
The previous commit only recorded the format.yml deletion; the new
ci.yml workflow, the shellcheck-enabled ./ci, and the build.sh nproc
quoting were left out due to a staging error. Add them now.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
Fix signed-overflow UB in base32_encode accumulator
9542062 
The `buffer` accumulator was declared `int`. For a 20-byte input the
running value grows past 2^23 and the `buffer << 8` step then shifts
into / past the sign bit of a 32-bit `int`, which is undefined behavior
in C (UBSan: "left shift of N by 8 places cannot be represented in type
'int'"). Widen the accumulator to `uint32_t`, where left shifts are
well-defined modular arithmetic. Output is byte-identical (the low bits
consumed via `>> bits_left & 0x1F` are unaffected).

The host harness under -fsanitize=undefined -fno-sanitize-recover
(test/harness_secret_qr.c) is the regression: it aborted on the old
code and passes on the fixed code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
Add host test skeleton (test/): ASan/UBSan/valgrind/coverage harnesses
2d11fdf 
Firmware is cross-compiled for the RP2040 and cannot run natively, but
the hardware-independent logic can. This adds a standalone host test
suite in test/, separate from the Pico CMake build:

- test/stub/pico/rand.h: host stub for the Pico SDK RNG header; each
  harness supplies a deterministic get_rand_64().
- test/harness_secret_qr.c: mirrors serial/commands_keys.c
  (secret -> base32 -> otpauth URI -> qrcodegen), asserting base32 len,
  charset, and QR size, against the real shared/random, libs/base32 and
  libs/qrcodegen sources.
- test/harness_base64.c: libs/base64 encode+decode roundtrip over empty
  input and every length residue mod 3 (branch coverage of the tail /
  padding paths) plus invalid-input rejection.
- test/Makefile: targets asan (ASan+UBSan, -fno-sanitize-recover, CI
  gate), valgrind, coverage, clean. Binaries under test/build/.
- .gitignore: ignore generated coverage / gcov artifacts.

`make -C test asan` is green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
Add whole-codebase host coverage (lcov/genhtml) + valgrind + blind-sp…
16dfbf0 
…ot summary

test/Makefile coverage target now builds the ENTIRE first-party codebase so
untested files surface at their true 0% instead of being filtered out:
  - real harnesses (base32 / base64 / random) run and report real coverage;
  - every other host-compilable first-party .c is compiled with --coverage to
    emit a .gcno (0% blind spot) via cheap Pico-SDK stubs under test/stub/;
  - lcov --initial baseline + post-run capture, combined, filtered to
    first-party (qrcodegen/littlefs/test/usr removed), branch coverage on,
    rendered to coverage/html/ with per-line highlighting.

test/coverage-summary.sh (shellcheck-clean) emits coverage/summary.{txt,json}
and a summary.html surfaced at the top of the report: overall first-party
line%/branch% over all 19 files (blind spots folded in at 0%), the 0%-coverage
file list, files below 100% branch coverage worst-first, and an explicit
BLIND SPOTS list. Three files remain non-host-compilable blind spots
(network/ntp.c, network/wifi.c: cyw43/lwip; storage/storage.c: flash+littlefs).

valgrind target already runs each harness under
--error-exitcode=99 --leak-check=full -q and is clean.

harness_secret_qr.c reflowed to satisfy clang-format (ci lint).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
Wire --check=test into ./ci (asan+valgrind+coverage)
c18957f 
test.check runs asan + valgrind + coverage; the hard gate is
ASan/UBSan + Valgrind (correctness). Coverage is report-only: the
whole-codebase denominator keeps line% intentionally low (blind spots
folded in), so no coverage floor gates CI. test.fix regenerates the
HTML report + summary via make -C test coverage.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
ci: add host test job (asan+valgrind+coverage)
d2de8fc 
Runs ./ci --check=test --action=check on ubuntu-latest with valgrind+lcov
installed and submodules checked out recursively (qrcodegen/littlefs are
needed for the coverage build). The hard gate is ASan+UBSan + Valgrind;
coverage is report-only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
test: drop 'range' from lcov --ignore-errors for CI lcov 2.0 compat
189324e 
GitHub's ubuntu-latest ships an lcov 2.0 build whose pure-lcov combine
step (--add-tracefile) rejects the geninfo-only 'range' error category
with 'unknown argument for --ignore-errors: range', failing the coverage
step. Removing it keeps coverage identical locally (same 5.5% line /
12.7% branch, no new lcov messages) and works across lcov 1.x/2.x.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
test: separate genhtml --ignore-errors list (drop 'gcov') for CI lcov
d6ef84e 
Ubuntu 24.04's lcov 2.0 genhtml rejects the 'gcov' error category
('unknown argument for --ignore-errors: gcov') because genhtml has no
gcov phase, while lcov/geninfo accept it. Split into LCOV_IGNORE
(capture/combine, keeps 'gcov') and GENHTML_IGNORE (no 'gcov'). All
categories exist in lcov 2.0.0, so it stays robust across Debian and
Ubuntu lcov builds. Verified end-to-end in an ubuntu:24.04 container
(lcov 2.0-1, gcov 13.3.0) matching the CI runner: ./ci --check=test
--action=check exits 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
ci: publish coverage report to GitHub Pages
106ba65 
Add a standalone pages.yml workflow that regenerates the host-test
coverage HTML (./ci --check=test) and deploys test/coverage/html/ to
GitHub Pages via actions/upload-pages-artifact + actions/deploy-pages.
Pages is enabled through the job token (configure-pages enablement),
so no admin CLI scope is required. Kept separate from ci.yml.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
test: derive coverage first-party filter from repo root, not literal …
f27cc47 
…"hslock"

The coverage step extracted first-party files with a hardcoded glob
'*/hslock/*', so `./ci --check=test --action=check` only worked when the
checkout dir was literally named "hslock". In any other dir the extract
matched nothing and genhtml aborted with "unexpected empty worklist"
(exit 2). CI happened to use a dir named hslock, masking the fragility.

Derive REPO_ROOT at run time (git rev-parse --show-toplevel, falling back
to the parent dir for non-git checkouts) and extract "$(REPO_ROOT)/*".
Coverage is built with -fprofile-abs-path so the tracefiles carry absolute
paths; filtering by the absolute repo root is dir-name-independent. The
existing --remove calls (submodules, /usr, test/) are unchanged.
coverage-summary.sh already derived the repo root via git, so no change
there.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
test: report coverage on every first-party file, drop blind-spot list
19d5cb7 
Add host stubs (cyw43_arch, lwip udp/dns/pbuf/err/ip_addr/arch, pico/flash,
hardware/flash, hardware/sync) so network/wifi.c, network/ntp.c and
storage/storage.c compile host-side. They now emit a .gcno and appear in the
lcov/genhtml report at their true 0% with per-line detail, like every other
untested file - so all 19 first-party files are reported uniformly.

Drop the separate hand-maintained BLIND SPOTS list and the source-line
approximation from coverage-summary.sh: overall line% / branch% are now derived
entirely from lcov (991 real instrumented lines, was ~1099 approximated).
sleep_ms/sleep_us/time_us_64 move into pico/time.h (mirroring the SDK) so both
network/ntp.c and files pulling pico/stdlib.h resolve them without redefinition.

Trigger the Pages redeploy from chore/host-tests too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
test: clang-format the new cyw43/lwip host stubs
fc8c4f3 
Fix macro-alignment and function-signature wrapping in the stubs added in the
previous commit so `./ci --check=lint` passes on the now-tracked headers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
test: drop coverage-summary.sh; rely on genhtml's own report
cbe63ec 
The summary script was a pure presentation layer (whole-codebase % headline
+ curated worst-first blind-spot list) on top of the lcov data. genhtml's
own index already shows the 19-file overall totals and a sortable per-file
table with the 0% files visible, so the report still covers everything.
Removing the 199-line script keeps the patch minimal.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
test: remove coverage-summary.sh call from Makefile (fix prior partia…
1fcedcc 
…l commit)

The previous commit recorded only the script deletion (git add aborted on the
already-removed pathspec), leaving the Makefile still invoking the deleted
script. Drop the invocation so the coverage target ends at genhtml.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude
test: slim coverage to harness-exercised modules only
59b56e8 
Drop the whole-codebase 0%-coverage "fold-in" scaffolding. The coverage
build previously compiled every first-party .c (via ~18 host stub headers
under test/stub/) purely so untested files surfaced at 0% in the report.

Coverage now reports exactly the modules the harnesses exercise: base32.c,
base64.c, random.c. The HW-only firmware (ntp/wifi/storage/serial/...) is no
longer built host-side and is simply absent from the report -- a focused,
honest denominator instead of a padded one.

Changes:
- Remove COV_ZERO_SRCS / COV_ZERO_DEFS and the loop compiling untested files
  to .gcno, plus the --initial baseline capture and base/total.info merge that
  only existed to surface those compiled-but-unrun files.
- Delete every test/stub header except pico/rand.h, the only stub the real
  harnesses need (shared/random.h -> pico/rand.h -> get_rand_64).

Kept: REPO_ROOT derivation, branch coverage, lcov 2.x --ignore-errors
handling, harness INCLUDES (still -Istub for pico/rand.h), asan/valgrind.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@d33mobile
d33mobile marked this pull request as draft July 8, 2026 10:07
@d33mobile

d33mobile commented Jul 8, 2026

Copy link
Copy Markdown
Author

Replaced by #4 — a fully independent version (base master, zero files shared with the format PR #2). It has its own test.yml workflow instead of extending the format PR's ci.yml, so the two PRs are now independent and mergeable in any order.

@d33mobile d33mobile closed this Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@d33mobile