Quality Assurance

cats-eo is mostly a type-level library: a large part of its correctness is discharged by the compiler before any test runs. The quality signals it tracks reflect that, in roughly increasing cost-to-fool order:

1. Types as tests — the composition matrix pins, at compile time, exactly which optic families compose with which (and at what strength), and which combinations are deliberately rejected. A regression that loosened or broke the lattice fails to compile.
2. Discipline law suites — cats-eo-laws defines the optic and typeclass laws; cats-eo-tests and the integration modules run them against concrete instances.
3. Statement / branch coverage (scoverage) — the project's primary runtime-quality signal. See the CLAUDE.md coverage note for why ~70–80 % is the expected ceiling: the remainder is pure type-level machinery with no runtime footprint, or code reachable only once a downstream carrier instance is added.
4. Mutation testing (stryker4s) — the strongest and most expensive signal, and the one that historically did not pay its way here. It was reintroduced once the schemes module grew real runtime machinery (the ArrayDeque fold machine, the effectful M-drivers, PSVec) and the opaque-carrier compose work added runtime dispatch to core.

The numbers below are regenerated from the live reports by site/tools/gen-qa-report.py; see Regenerating these numbers.

Composition matrix

Every (outer family ∘ inner family) pair either composes — import-free and without a type ascription, landing at the strength described in the optic taxonomy — or it does not compile, because the cell is void by design (building through a non-invertible optic, writing through a read-only one, reading through a write-only one). The grid is the pass/fail projection of CompositionMatrixSpec, which is the single source of truth — if a cell starts needing an import or an ascription, that spec goes red.

✓ composes import-free at the strength shown in the optic taxonomy; ✗ does not compile (void by design — building through a read-only optic, reading through a write-only one, etc.). 87 composing / 34 void cells, pinned by CompositionMatrixSpec.

Coverage

Statement and branch coverage per package, from the cross-module scoverage aggregate (sbt coverageAll). BC/SC is the branch-coverage ÷ statement-coverage ratio — a value well below 1 flags a package whose conditionals are under-exercised relative to its straight-line code, even when its statement coverage looks healthy. Packages with no branch statements show —.

Mutation testing

stryker4s mutates each module's main sources and re-runs that module's own test suite per mutant. Score (total) counts no-coverage mutants against the score; Score (covered) is restricted to mutants on exercised lines. Timeout mutants count as detected (the mutant visibly broke the run). Compile err mutants — mutations that don't type-check, common in this codebase's match-type / opaque-carrier code — are excluded from both scores.

StringLiteral mutants are excluded build-wide (ThisBuild / strykerExcludedMutations): in this codebase string literals are error messages, vestigial-arm labels, and discipline rule-set names — nothing any suite asserts on, so they survive as pure noise and drown the genuine survivors (in laws, 99 of 101 unfiltered survivors were rule-set name labels). They appear as Ignored in the HTML reports and are counted in no score.

Caveats that keep the table honest:

• core — its behavioural suite lives in the separate tests/ module (which depends on laws and generics, so core can't dependsOn it back — that's a project cycle). mutationAll instead task-borrows the compiled suite: it appends tests/Test/definedTests and tests/Test/fullClasspath to core's Test scope for the mutation run only. This is sound because stryker compiles every mutant into core's classes behind runtime switches (binary-compatible), so specs compiled against unmutated core still exercise the mutated bytecode. Core's row below is scored against the full cross-module suite.
• generics is macro code: it expands at compile time, so its mutants leave no runtime footprint for a test run to cover. Mutation testing structurally can't score it — the derived-optic laws in generics/test guard it instead.
• laws has no in-module tests, so it borrows the tests/ suite the same way core does. Mutating the law definitions is the "who tests the tests" probe: a killed mutant means the discipline suites notice a corrupted law; a survivor pinpoints a law whose discriminating power nothing exercises. The surviving mutants are all law-weakening mutations (missIsEmpty's guard → false in AffineFoldLaws, && → || in ModifyFLaws.functorIdentity / functorComposition): every instance under test satisfies the weakened law too, so nothing fails. Killing those requires negative fixtures — deliberately unlawful instances pinned to fail the suite — which is the concrete follow-up this table surfaces.

Two "can't be scored" caveats that used to sit here are retired — both were re-tested on 2026-09-18 and neither reproduces:

• jsoniter was recorded as un-mutatable because instrumenting PathParser.parseField supposedly overflowed the JVM's 64 KB per-method bytecode limit. It does not: the module mutates and runs clean end to end (420 mutants, 0 compile errors), and PathParser.scala itself yields 24 mutants. Nothing needs splitting or excluding.
• avro was recorded as unscoreable because stryker's forked test-runner failed to initialise in the sandbox. It now completes in about two minutes and produces a full report. The row's numbers are real.

What is structural and permanent — do not spend a test-writing budget on it:

• generics — 86 NoCoverage mutants, every one inside a quoted macro (LensMacro, MacroSelectors, PlateMacro, PrismMacro). Macro bodies run inside the compiler, so stryker's runtime coverage probe never observes them. The only tests that can reach them are must not compile negative specs, which register no coverage. The module's score is 0 % by construction.
• avro's AvroPrismMacro — 17 NoCoverage mutants, identical cause.
• kyo's RecordIsoMacro — 28 NoCoverage mutants, identical cause; it is the entire reason kyo's total score reads far below its covered score.

The high-signal rows are core and laws (via the borrowed suite), schemes, circe, jsoniter, avro and zio — modules whose mutated code is genuinely exercised at run time by the suite stryker runs.

The table is regenerated only on release tags, so between releases it lags the tree. The most recent full sweep (2026-09-18, JDK 25, project <m>; stryker per module), after the survivor-killing pass of the same day, measured: core 193 K / 4 T / 19 S, laws 85 K / 0 S, schemes 48 K / 8 S, circe 44 K / 6 S, jsoniter 328 K / 4 T / 58 S, avro 391 K / 78 S, generics 0 K / 86 NC, zio 37 K / 6 S, kyo 30 K / 5 S / 28 NC.

The two effect-system integrations are the newest rows in that table, and each needs one line of reading:

• zio is a clean, high-signal row: 37 killed / 6 survived, no no-coverage block at all. Everything stryker can mutate in the module — the DynamicValue and zio.json tree navigation, the JsonCursor write descent, the Chunk index guards — is genuinely executed by the suite. (It was not always: before the ecosystem optics landed, the module was ZEnvironment / ZLayer / Ref wiring that delegated straight into ZIO's own API and offered a mutator no operator, literal or branch to change.)
• kyo's low total score is a macro artefact, not a coverage hole: every one of its 28 no-coverage mutants is in RecordIsoMacro, quoted-macro code that expands at compile time — the same structural reason generics scores 0 %. The covered column, 85.7 % (30/35, the rest on schema/StructureOptics.scala), is the one that describes the hand-written optics.

Both modules score at all only because their single-method extension clauses are braced. Re-printed by stryker4s, a significant-indentation extension clause with a Scaladoc'd body loses its method to column 0, the file stops compiling, and the whole module run aborts before scoring anything — see the comment on kyo/schema/StructureOptics.scala for the mechanism. Any new single-method extension in these modules needs the same treatment.

The generator distinguishes a module that produced no mutants at all from one with no report: the first renders n/a, the second an em-dash. Nothing currently hits the n/a path, but printing 0.0 % for a module a mutator cannot touch would read as a test-quality failure where there is no bar to fail.

Known equivalent mutants

A mutation score is not a coverage target: some mutants are equivalent — the mutated program computes the same observable value as the original, so no test can kill them. Chasing them is how a suite grows LOC without gaining kill capacity. The 40 survivors below were each read against the source and carry a checkable reason. Re-triaging them is wasted work; if one of them is ever killed, the equivalence claim was wrong and this table is the bug report.

Keyed (file, line:column, mutator → replacement) so a key survives renumbering only as far as the next edit to the file — re-verify a row whose line has moved rather than trusting it.

core — 19 (all of core's survivors)

Every one is a perf fast path whose general path computes the same value.

A deep-transform test is still worth having as a stack-safety test — but it will not kill the four Plated:127 mutants, so it must be justified on its own terms.

circe — 6 (all of circe's survivors)

The other copy of the array-bounds check in this file (JsonWalk.scala:56, readPath) is not equivalent — it survived for years because JsonIndexBoundsSpec drove only the write twin, and a differential oracle killed all six of its variants in 2026-09.

schemes — 8 (all of schemes' survivors)

jsoniter — 7 of 58

Not certified here: the remaining 51 JsonPathScanner.scala survivors and JsoniterPrism.scala:130:8. They were triaged into classes (end-of-input >= → == on cursors that advance by one; empty-container fast paths), but one class was measured wrong in 2026-09: 254:13 >= → > was called equivalent and is in fact killable, because with > the left disjunct of kpos >= bytes.length || bytes(kpos) != '"' is dead-false and the right disjunct then indexes at kpos == bytes.length. Any guard whose right-hand side indexes the array needs the reachability of pos == length checked individually before it may be called equivalent — which is why those rows are absent from the table above.

Timeout oscillation

A handful of the builder mutants (IntArrBuilder:17:8, ObjArrBuilder:17:12, IntArrBuilder:28:10 → true, ObjArrBuilder:40:10 → true) alternate between Timeout and Survived between runs: forcing a grow on every append makes the builder quadratic, which sometimes trips the per-mutant timeout and sometimes does not. Both statuses are equivalent-mutant outcomes, but only Timeout counts as detected, so a module's survivor count can move by one or two with no change to the suite. Do not read such a delta as a regression.

Regenerating these numbers

# Statement / branch coverage (cross-module aggregate):
SBT_OPTS="-Xmx6g" sbt coverageAll
#   → target/scala-3.8.3/scoverage-report/  (HTML + scoverage.xml)

# Mutation testing across the runtime-logic modules:
SBT_OPTS="-Xmx6g" sbt mutationAll
#   → <module>/target/stryker4s-report/<ts>/  (index.html + report.json)

# Rewrite the tables above from those reports + CompositionMatrixSpec:
python3 site/tools/gen-qa-report.py            # in place
python3 site/tools/gen-qa-report.py --check     # CI: non-zero if stale

Both aliases relax the always-on -Werror (tlFatalWarnings) first, since instrumented sources can surface -Wunused warnings; the larger heap is because the set reapply re-evaluates the Laika docs settings. Mutation runs with project <m>; stryker (not <m>/stryker) so specs2 is visible to the test runner — see project/plugins.sbt.

The quality.yml workflow runs all three on every release tag (and on demand via workflow_dispatch), uploading the HTML reports and the regenerated tables as build artifacts.