feat: Add vitest coverage to ~most~ packages by jmuzina · Pull Request #247 · canonical/pragma

jmuzina · 2025-06-06T03:30:54Z

Done

Enables Vitest coverage reports on pushes to main and PRs. Prerequisite for enabling TICS.
Adds Vitest to TS/JS-based packages that didn't have it yet. Otherwise, these packages would not be included in the coverage reports.
Enables Vitest projects to orchestrate all of the coverage tests under a single process
Adds a coverage dashboard which will show the current code coverage. Ex on my fork. This is also used by maas-ui. After merging, it will be available here.
Fixes some bugs in asynchronous React hooks that were causing coverage tests to fail due to the test environment being cleaned up while async options (setTimeout, debounce) were still in progress. Added cleanups and/or cancel operations as necessary.

Fixes WD-22631

TODO:

Definitively identify preferred coverage provider. Options are istanbul and v8. PR is currently using v8. The choice should be documented and added to testing/arch docs.
As a team, Solidify our desired process for when coverage is run and how it is reported

QA

Review CI, see that coverage is reported.
This PR should also have coverage reported as a comment.

PR readiness check

PR should have one of the following labels:
- Feature 🎁, Breaking Change 💣, Bug 🐛, Documentation 📝, Maintenance 🔨.
PR title follows the Conventional Commits format.
All packages define the required scripts in package.json:
- All packages: check, check:fix, and test.
- Packages with a build step: build.

.github/workflows/pr.yml

github-actions · 2025-06-06T03:32:58Z

Coverage Report

Status	Category	Percentage	Covered / Total
🔵	Lines	40.21%	2591 / 6443
🔵	Statements	40.21%	2591 / 6443
🔵	Functions	64.11%	109 / 170
🔵	Branches	76.22%	404 / 530

File Coverage

No changed files found.

Generated in workflow #8 for commit f01172f by the Vitest Coverage Report Action

jmuzina · 2025-06-07T01:28:07Z

Regarding the choice of coverage provider between istanbul and v8:

I believe we should go for v8. Vitest lays out a good comparison of the two here. Pulling the most important points:

Running Istanbul requires a transpilation step before running the coverage tests.
V8 is generally faster and uses less memory
Coverage report accuracy is approximately the same as Istanbul. This took some time to achieve. For a while the choice of Istanbul vs v8 was generally "Istanbul for quality, v8 for speed". However, this is no longer the case, as v8 has matured quite a bit and their speed is approximately the same since vitest 3.2.0 (ref)
Running coverage with Istanbul reports this error for the React core package, which does not occur when using v8:

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Error ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Error: Failed to resolve entry for package "@canonical/react-ds-core". The package may have incorrect main/module/exports specified in its package.json.
279|  cov_1bn0o15k31();
280|  import { Fragment, jsxDEV } from "react/jsx-dev-runtime";
281|  import { Button, TooltipArea } from "@canonical/react-ds-core";
   |                                       ^
282|  import { Suspense, useState, lazy } from "react";
283|  import canonicalLogo from "./assets/canonical.svg";

Another possible option is Bun coverage testing, but this does not seem to be suitably mature for our purposes yet. It does not seem to support workspace-level codecoverage reporting, like Vitest does, and it doesn't seem to support HTML code coverage reporting, which is a useful tool for seeing the current state of code coverage.

packages/typography/vitest.config.ts

Copilot

Pull Request Overview

This PR enables Vitest coverage reporting across most packages, sets up Vitest in TS/JS packages, and updates GitHub workflows to run and publish coverage.

Adds vitest.setup.ts and vitest.config.ts to integrate JSDOM, jest-dom matchers, and DOM cleanup.
Updates package.json files to include Vitest, testing-library, and coverage dependencies.
Introduces GitHub Actions workflows for coverage runs and GitHub Pages deployment.

Reviewed Changes

Copilot reviewed 19 out of 28 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
packages/storybook-addon-baseline-grid/vitest.setup.ts	Add DOM cleanup after each test
packages/storybook-addon-baseline-grid/vitest.config.ts	Configure Vitest with JSDOM and setup files
packages/storybook-addon-baseline-grid/package.json	Add Vitest and testing-library dependencies
packages/react/ssr/vitest.setup.ts	Add DOM cleanup after each test
packages/react/ssr/vitest.config.ts	Configure Vitest for SSR package
packages/react/ssr/package.json	Add Vitest and testing-library dependencies
packages/generator-ds/vitest.config.ts	Add Vitest configuration
packages/generator-ds/package.json	Add Vitest dependency
package.json	Add coverage script and Vitest dependencies
configs/storybook/vitest.config.ts	Add shared Vitest config for Storybook
configs/storybook/package.json	Add Vitest dependency
apps/react/demo/vitest.setup.ts	Add DOM cleanup after each test
apps/react/demo/vitest.config.ts	Configure Vitest with JSDOM and setup files
apps/react/demo/package.json	Add Vitest and testing-library dependencies
apps/react/boilerplate-vite/vitest.setup.ts	Add DOM cleanup and ResizeObserver mock
apps/react/boilerplate-vite/vitest.config.ts	Configure Vitest with JSDOM and setup files
apps/react/boilerplate-vite/package.json	Add Vitest and coverage-v8 dependency
.github/workflows/push.yml	Update CI to run coverage action
.github/workflows/coverage.yml	Add new coverage workflow

Comments suppressed due to low confidence (2)

apps/react/boilerplate-vite/vitest.setup.ts:11

Vitest's mocking API uses vi.fn(), not vitest.fn(); this line will error. Replace vitest.fn() with vi.fn().

global.ResizeObserver = vitest.fn().mockImplementation(() => ({

packages/storybook-addon-baseline-grid/vitest.setup.ts

apps/react/demo/vitest.setup.ts

apps/react/boilerplate-vite/vitest.setup.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

advl

I approve the approach. Minor comments and questions related to some of your choices.

Also would something like this help ?

https://nx.dev/technologies/build-tools/vite/api/generators/vitest

advl · 2025-06-20T17:06:47Z

.github/workflows/coverage.yml

@@ -0,0 +1,52 @@
+name: "Coverage"
+on:
+  pull_request:


I sense that these should be two workflows - maybe with a reusable bit ? (A reusable workflow)
This would allow for the conditional check further down which seems unidiomatic

What do you propose the two workflows to be - the running of the coverage test and the creation of the coverage dashboard? Can you speak a bit to your thinking re: splitting those actions up into separate workflows? To my mind, coverage seems to be nicely covered by one workflow, which always runs the coverage test, and in some cases (merges to main) also uploads a coverage report - not sure if there's a case for making an additional reusable action unless there are foreseeable scenarios where we'd want to re-use it outside of this scope.

My rationale is that this workflow runs on both

pr

push on main

and the second job (upload coverage pages) applies what is to me an undiomatic if

It would likely be better that we have two workflows

pr (runs only test coverage)

push (tests coverage and uploads artifact)
Maybe benefiting from a shared action

.github/workflows/coverage.yml

advl · 2025-06-20T17:13:58Z

.github/workflows/coverage.yml

+      pages: write
+    steps:
+      - name: Download coverage report
+        uses: actions/download-artifact@v4


I wonder if when splitting the workflow in two we could avoid the step of uploading and downloading the artifact to simplify

advl · 2025-06-20T17:15:47Z

configs/storybook/package.json

    "@canonical/biome-config": "^0.9.0-experimental.12",
    "@canonical/typescript-config-react": "^0.9.0-experimental.12",
+    "jsdom": "^26.0.0",
+    "vitest": "^3.0.9",


Please review S9 best practices - I believe vitest is already a default. I am also unsure of the side effects of pinning a version here

advl · 2025-06-20T17:16:00Z

configs/storybook/package.json

    "@biomejs/biome": "^1.9.4",
    "@canonical/biome-config": "^0.9.0-experimental.12",
    "@canonical/typescript-config-react": "^0.9.0-experimental.12",
+    "jsdom": "^26.0.0",


See below comment

advl · 2025-06-20T17:24:34Z

packages/generator-ds/vitest.config.ts

Doesn't necessarily need to be tackled here but : is there a way we could reuse or modularize vitest configs ?

advl · 2025-06-20T17:26:19Z

packages/storybook/addon-baseline-grid/vitest.setup.ts

Similarly might be worth adding to the msw addon

advl · 2025-06-20T17:27:16Z

packages/utils/vitest.config.ts

+  test: {
+    // include vite globals for terser test code
+    globals: true,
+    include: ["src/**/*.tests.ts", "src/**/*.tests.tsx"],


Minus tsx as this doesn't contain react code

advl · 2025-06-20T17:34:16Z

vitest.config.ts

+import { defineConfig } from 'vitest/config'
+
+export default defineConfig({
+  test: {


I wonder how this top level guest config would play out with Adam's Pr ?

advl · 2025-06-20T17:36:20Z

vitest.config.ts

+    environment: "jsdom",
+    coverage: {
+      provider: "v8",
+      reporter: ["text", "json-summary", "json", "html", "lcov"],


Could you comment briefly on why these are sensible defaults ?

wip: Add vitest coverage to ~most~ packages

2027d27

jmuzina commented Jun 6, 2025

View reviewed changes

.github/workflows/pr.yml Outdated Show resolved Hide resolved

jmuzina added 3 commits June 6, 2025 15:10

coverage is in separate workflow. added upload to pages.

d5cedea

reporters

1781c2c

fix intermittent coverage failures by adding a debounce cancel

dc291d0

jmuzina changed the title ~~wip: Add vitest coverage to ~most~ packages~~ feat: Add vitest coverage to ~most~ packages Jun 7, 2025

jmuzina added 3 commits June 6, 2025 21:03

Fix useDelayedToggle window undefined error in coverage test environment

af23e76

rm extra comment

0e25251

gitignore codecov

59b3c84

jmuzina requested a review from advl June 7, 2025 01:30

jmuzina marked this pull request as ready for review June 7, 2025 01:30

jmuzina mentioned this pull request Jun 10, 2025

feat: Create Svelte app-launchpad #246

Closed

5 tasks

Merge remote-tracking branch 'origin/main' into vitest-coverage

2d374e5

jmuzina commented Jun 11, 2025

View reviewed changes

packages/typography/vitest.config.ts Outdated Show resolved Hide resolved

remove extra jsdom dependencies

99d2b63

jmuzina requested a review from Copilot June 18, 2025 22:24

Copilot AI reviewed Jun 18, 2025

View reviewed changes

packages/storybook-addon-baseline-grid/vitest.setup.ts Show resolved Hide resolved

apps/react/demo/vitest.setup.ts Outdated Show resolved Hide resolved

apps/react/boilerplate-vite/vitest.setup.ts Outdated Show resolved Hide resolved

jmuzina and others added 2 commits June 20, 2025 13:21

Merge remote-tracking branch 'origin/main' into vitest-coverage

6bf7ef2

Apply suggestions from code review

f01172f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

advl requested changes Jun 20, 2025

View reviewed changes

jmuzina self-assigned this Sep 30, 2025

Conversation

jmuzina commented Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Done

QA

PR readiness check

Uh oh!

Uh oh!

github-actions bot commented Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Coverage Report

Uh oh!

jmuzina commented Jun 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

advl left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jmuzina commented Jun 6, 2025 •

edited

Loading

github-actions bot commented Jun 6, 2025 •

edited

Loading

jmuzina commented Jun 7, 2025 •

edited

Loading