Skip to content

Credit Spine Compiler codegen execution to coverage#317

Merged
alexander-yevsyukov merged 7 commits into
masterfrom
credit-codegen-coverage
Jul 6, 2026
Merged

Credit Spine Compiler codegen execution to coverage#317
alexander-yevsyukov merged 7 commits into
masterfrom
credit-codegen-coverage

Conversation

@alexander-yevsyukov

Copy link
Copy Markdown
Collaborator

What & why

The java and context modules — the Spine Compiler plugins, renderers, and option
generators — execute in the forked launch*SpineCompiler JVM during the build, which
Kover/JaCoCo do not instrument. They reported ~0% line coverage even though the tests/*
.proto fixtures exercise them on every build, holding the repo total near 12% (matching
Codecov).

This attaches the standalone JaCoCo agent to the forked compiler JVM, harvests its .exec, and
merges it into the root Kover report (the one Codecov ingests) — mirroring the existing
enableTestKitCoverage() pipeline for Gradle TestKit workers.

How

  • enableSpineCompilerCoverage() (new buildSrc/.../testing/SpineCompilerCoverage.kt)
    attaches -javaagent:<Jacoco.agent>=destfile=…/<task>.exec,append=false to every
    launch*SpineCompiler JavaExec task, with a per-task destfile.
  • The .exec is declared as a task output, so the launch tasks stay cacheable: a
    JavaExec blocks until the fork exits and the agent flushes the exec on exit, so the file is on
    disk when the task action returns and can be a declared output — unlike a lingering TestKit
    worker daemon (which is why enableTestKitCoverage() still disables caching). Coverage therefore
    survives both an UP-TO-DATE skip and a FROM-CACHE hit.
  • KoverConfig feeds the exec into the root report's additionalBinaryReports, and orders
    kover*Report/kover*Verify after the launch tasks — the root report does not otherwise depend
    on the non-Kover tests/* modules that produce the exec.
  • The JaCoCo agent coordinate is centralized as Jacoco.agent (shared with the TestKit helper).
  • Applied via subprojects { enableSpineCompilerCoverage() }; a no-op where no launch task exists.

Class IDs match by construction: the compiler CLI shadow jar does no relocate() and loads
consumer plugin classes as their original artifacts (verified in the compiler repo, Phases 0–1).

Result (verified locally)

./gradlew clean build koverXmlReport → root report 12% → 86.3% line coverage
(covered 1894 / 2195), reconciling exactly with Codecov's 12.35% baseline. Ten .exec files are
produced across all three launch-task variants. Generators flip 0% → covered:
MinGenerator/MaxGenerator/RangeGenerator 100%, JavaValidationRenderer 97.8%,
BoundedFieldGenerator 88%. ValidatorGenerator stays 0% — a genuine gap (no fixture declares
a custom @Validator), confirming the measure is discriminating rather than blanket-crediting.

Caching verified: a no-change re-run is UP-TO-DATE; deleting the exec makes the task
FROM-CACHE and restores the exec; the full build reports 1 from cache.

Caveats / follow-ups

  • This is execution coverage of codegen, not assertion-backed unit coverage — genuine (the code
    runs against the fixtures) but more generous. A follow-up should report it under a separate
    Codecov flag
    (codegen vs runtime) and upstream the two buildSrc helpers to config.
  • The version bump's dependency/license-report refresh was not run separately; a report-update
    commit from the working tree is included, but if CI's report check flags a .448 mismatch, run
    the report task.

Pre-PR

PASS — ./gradlew build dokkaGenerate green; spine-code-review + kotlin-engineer APPROVE.

Note on commits

Opened as-is per request. Beyond the coverage feature, the version bump, and a review-feedback
commit (cacheable exec + Jacoco.agent), this PR also carries three commits from local-master
drift: a team-memory note (.agents/memory/*), a dependency-report update (docs/dependencies/*),
and an IDE-config change (.idea/misc.xml).

🤖 Generated with Claude Code

alexander-yevsyukov and others added 6 commits July 6, 2026 18:01
The `java` and `context` modules — the compiler plugins, renderers, and
option generators — execute in the forked `launch*SpineCompiler` JVM during
the build, which Kover/JaCoCo do not instrument. They therefore reported
~0% line coverage even though the `tests/*` `.proto` fixtures exercise them
on every build, holding the repo total near 12%.

Attach the standalone JaCoCo agent to the forked compiler JVM, harvest its
`.exec`, and merge it into the root Kover report (the one Codecov ingests),
mirroring the existing `enableTestKitCoverage()` pipeline:

- `enableSpineCompilerCoverage()` attaches `-javaagent` to every
  `launch*SpineCompiler` task with a per-task destfile, and marks those
  tasks non-cacheable so a cache hit never drops the exec.
- `KoverConfig` feeds the exec into the root report's
  `additionalBinaryReports` and orders the report/verify tasks after the
  launch tasks (the root report does not otherwise depend on the non-Kover
  test modules that produce the exec).
- Applied to all subprojects; a no-op where no launch task exists.

This is execution coverage of codegen, not assertion-backed unit coverage;
a follow-up will report it under a separate Codecov flag. Locally the root
report rises from ~12% to ~86% line coverage, with the codegen classes
flipping from 0% to their exercised percentage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…oco.agent`

Address review feedback on the compiler-coverage helper:

- Declare each `launch*SpineCompiler` task's `.exec` as a task output and drop
  `outputs.cacheIf { false }`, restoring build-cache and up-to-date behavior. A
  `JavaExec` blocks until the forked JVM exits and the JaCoCo agent flushes the
  exec on that exit, so — unlike a lingering TestKit worker daemon — the file is
  on disk when the task action returns and can be a declared output. The build
  cache then stores and restores it, so coverage survives both an UP-TO-DATE skip
  and a FROM-CACHE hit. Switch the agent to `append=false` (each run overwrites
  its own per-task file), which removes the need for the guarded one-shot wipe.
  Verified: re-run is UP-TO-DATE; deleting the exec restores it FROM-CACHE.

- Extract the JaCoCo agent coordinate to `Jacoco.agent`, used by both
  `enableSpineCompilerCoverage()` and `enableTestKitCoverage()`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 6, 2026 18:02
@alexander-yevsyukov alexander-yevsyukov self-assigned this Jul 6, 2026
@alexander-yevsyukov alexander-yevsyukov moved this to 🏗 In progress in v2.0 Jul 6, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to credit out-of-process Spine Compiler plugin/codegen execution to the root Kover/JaCoCo coverage report by instrumenting the forked launch*SpineCompiler JVM and merging its .exec files into the aggregated report consumed by Codecov.

Changes:

  • Add build logic to attach the JaCoCo agent to launch*SpineCompiler JavaExec tasks and collect per-task .exec outputs for cache-friendly coverage attribution.
  • Feed the collected compiler .exec files into the root Kover aggregation via additionalBinaryReports, and order reporting tasks after compiler launch tasks.
  • Bump the snapshot version and refresh generated dependency report artifacts (plus a few repo-state drift changes).

Reviewed changes

Copilot reviewed 8 out of 11 changed files in this pull request and generated no comments.

Show a summary per file
File Description
version.gradle.kts Bumps published snapshot version to 2.0.0-SNAPSHOT.448.
docs/dependencies/pom.xml Updates generated dependency POM content (dependency list changes).
docs/dependencies/dependencies.md Updates generated dependency/license report timestamps (but still references .447 in headers).
buildSrc/src/main/kotlin/io/spine/gradle/testing/TestKitCoverage.kt Switches TestKit coverage helper to reuse a centralized JaCoCo agent coordinate.
buildSrc/src/main/kotlin/io/spine/gradle/testing/SpineCompilerCoverage.kt Introduces Spine Compiler forked-JVM coverage instrumentation and .exec output capture.
buildSrc/src/main/kotlin/io/spine/gradle/report/coverage/KoverConfig.kt Merges compiler-produced .exec files into the root Kover report inputs.
buildSrc/src/main/kotlin/io/spine/dependency/test/Jacoco.kt Centralizes the standalone JaCoCo agent Maven coordinate.
build.gradle.kts Enables the Spine Compiler coverage hook across subprojects.
.idea/misc.xml Updates IntelliJ project settings (ProjectRootManager default flag).
.agents/memory/project/codegen-coverage-blind-spot.md Adds team memory note documenting the historical coverage blind spot and rationale.
.agents/memory/MEMORY.md Registers the new memory entry.
Files not reviewed (1)
  • .idea/misc.xml: Generated file
Comments suppressed due to low confidence (1)

docs/dependencies/dependencies.md:1116

  • The dependencies reports were regenerated (timestamps updated), but they still reference 2.0.0-SNAPSHOT.447 in the section headers, while version.gradle.kts bumps the project to .448. docs/dependencies/pom.xml also still declares <version>2.0.0-SNAPSHOT.447</version>, so the published version and the dependency-report docs will be inconsistent (and may fail any report/version consistency checks).
This report was generated on **Mon Jul 06 16:20:12 WEST 2026** using 
[Gradle-License-Report plugin](https://github.com/jk1/Gradle-License-Report) by Evgeny Naumenko, licensed under 
[Apache 2.0 License](https://github.com/jk1/Gradle-License-Report/blob/master/LICENSE).




# Dependencies of `io.spine.tools:validation-context-tests:2.0.0-SNAPSHOT.447`

@alexander-yevsyukov alexander-yevsyukov moved this from 🏗 In progress to In Review in v2.0 Jul 6, 2026
The dependency/license reports were carried into the branch from an earlier
commit at `.447`; regenerate them so the report headers and `pom.xml` match the
bumped `version.gradle.kts`. Resolves the version/report inconsistency flagged
by the Copilot PR review.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@alexander-yevsyukov

Copy link
Copy Markdown
Collaborator Author

Addressed the dependency-report inconsistency noted in the Copilot review: the docs/dependencies/ reports had been carried into the branch at 2.0.0-SNAPSHOT.447 from an earlier commit, while version.gradle.kts bumps to .448. Regenerated them for .448 in c92dc9d — report headers and pom.xml are now consistent (0 remaining .447 refs).

🤖 Addressed by Claude Code

@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.64%. Comparing base (9c12903) to head (c92dc9d).

Additional details and impacted files
@@              Coverage Diff              @@
##             master     #317       +/-   ##
=============================================
+ Coverage     12.35%   82.64%   +70.29%     
- Complexity       84      332      +248     
=============================================
  Files            82       82               
  Lines          2040     2040               
  Branches        161      161               
=============================================
+ Hits            252     1686     +1434     
+ Misses         1786      289     -1497     
- Partials          2       65       +63     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@alexander-yevsyukov alexander-yevsyukov merged commit ec8c202 into master Jul 6, 2026
12 checks passed
@alexander-yevsyukov alexander-yevsyukov deleted the credit-codegen-coverage branch July 6, 2026 18:24
@github-project-automation github-project-automation Bot moved this from In Review to ✅ Done in v2.0 Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants