X Tutup
Skip to content

Improve coverage gate: hybrid regression check + method-level detail reporting#4324

Merged
andimarek merged 4 commits intomasterfrom
claude/coverage-gate-hybrid-check-efh0A
Mar 9, 2026
Merged

Improve coverage gate: hybrid regression check + method-level detail reporting#4324
andimarek merged 4 commits intomasterfrom
claude/coverage-gate-hybrid-check-efh0A

Conversation

@andimarek
Copy link
Member

@andimarek andimarek commented Mar 9, 2026

Summary

  • Hybrid regression detection: The coverage gate now requires BOTH a percentage drop (>0.05%) AND an increase in the absolute number of missed items. This eliminates false positives when well-covered code is extracted or moved out of a class — the percentage drops due to a ratio shift even though no test coverage was actually lost.
  • Method-level detail in PR comment: When a class shows a coverage decrease, the PR report comment now includes a collapsible <details> section listing exactly which methods are uncovered or changed, with line and branch coverage percentages and deltas vs baseline.
  • Method-level detail in gate failure: The coverage gate failure message now includes indented method-level context below each class regression, showing which specific methods are uncovered, new, removed, or changed.
  • JaCoCo parser enhancement: parse-jacoco.js now extracts per-method counters (name, descriptor, line/branch/method coverage) from the JaCoCo XML. This data automatically flows into the baseline via master.yml on the next merge, enabling method-level before/after comparisons.

Files changed

File Change
.github/scripts/parse-jacoco.js Extract per-method counters from JaCoCo XML
.github/workflows/pull_request.yml Hybrid regression check + method detail in gate failure
.github/workflows/pr-report.yml Collapsible method-detail sections in PR comment

Test plan

  • Verify gate still passes on a PR with no coverage regressions
  • Verify gate fails when a class has both a percentage drop and increased missed count, and that the failure message includes method-level detail
  • Verify gate does NOT fail when a class has a percentage drop but no increase in missed count (refactoring/move scenario)
  • Verify PR comment includes collapsible method-detail sections for classes with coverage decreases
  • Verify method-level baseline data is stored after next master merge

https://claude.ai/code/session_01XVBWUo2VM4B82ssZm1f2mu

The coverage gate previously flagged any percentage drop > 0.05% as a
regression. This caused false positives when well-covered code was
extracted/moved out of a class — the percentage drops due to a ratio
shift even though no test coverage was actually lost.

Now a regression requires BOTH a percentage drop AND an increase in the
absolute number of missed items. This correctly distinguishes real
regressions (new uncovered code) from cosmetic ratio changes caused by
refactoring.

https://claude.ai/code/session_01XVBWUo2VM4B82ssZm1f2mu
@github-actions
Copy link
Contributor

github-actions bot commented Mar 9, 2026

Test Report

Test Results

Java Version Total Passed Failed Errors Skipped
Java 11 5698 (±0) 5642 (±0) 0 (±0) 0 (±0) 56 (±0)
Java 17 5698 (±0) 5641 (±0) 0 (±0) 0 (±0) 57 (±0)
Java 21 5698 (±0) 5641 (±0) 0 (±0) 0 (±0) 57 (±0)
Java 25 5698 (±0) 5641 (±0) 0 (±0) 0 (±0) 57 (±0)
jcstress 32 (±0) 32 (±0) 0 (±0) 0 (±0) 0 (±0)
Total 22824 (±0) 22597 (±0) 0 (±0) 0 (±0) 227 (±0)

Code Coverage (Java 25)

Metric Covered Missed Coverage vs Master
Lines 28727 3121 90.2% ±0.0%
Branches 8345 1511 84.7% ±0.0%
Methods 7684 1222 86.3% ±0.0%

No per-class coverage changes detected.

Full HTML report: build artifact jacoco-html-report

Updated: 2026-03-09 18:53:27 UTC

When a class shows a coverage decrease, the PR report comment now includes
a collapsible <details> section showing exactly which methods are uncovered
or changed. The coverage gate failure message also includes method-level
context. The JaCoCo XML parser is enhanced to extract per-method counters
(name, descriptor, line/branch coverage) within each class.

https://claude.ai/code/session_01XVBWUo2VM4B82ssZm1f2mu
@andimarek andimarek changed the title Fix false positive coverage regressions on code extraction Improve coverage gate: hybrid regression check + method-level detail reporting Mar 9, 2026
claude added 2 commits March 9, 2026 18:52
When baseline method data is available, the method-level detail sections
now only show methods whose coverage actually changed or that are new/
removed. Previously the hasMissed filter would show every partially-covered
method, creating noise that buried the actual regressions. Falls back to
showing all uncovered methods when no baseline method data exists yet.

https://claude.ai/code/session_01XVBWUo2VM4B82ssZm1f2mu
The buildMethodDetails function now uses the same hybrid check as the
coverage gate: percentage drop AND missed count increase. Previously it
used a percentage-only check, which would show misleading detail sections
for classes where code was merely extracted or moved.

https://claude.ai/code/session_01XVBWUo2VM4B82ssZm1f2mu
@andimarek andimarek merged commit 5e83c1b into master Mar 9, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

X Tutup