-
Notifications
You must be signed in to change notification settings - Fork 9
Description
Describe the bug
If a user has a Storybook configured in their development environment, specifically for the latest major releases (e.g., 7.6 and 8.0), they automatically have the option to enable the Visual testing addon to run visual tests on their components locally. However, if they are working with Chromatic E2E integration (i.e., Playwright, Cypress), they can enable the Visual tests addon to run visual tests on their components when that should not be the case, whether intentionally or not. This is something that should be avoided as much as possible, as this can lead to unnecessary visual tests being run on components that are not intended to be tested, polluting the build screen with unnecessary builds, and depleting the allowed snapshots for the user's account.
It would be in our best interest to either:
- Filter the projects during the onboarding workflow to prevent this issue
- If the above is impossible, prevent the build from going through, as this is not the correct type, and showcase a message in the addon's panel to reflect this fact
If none of the above is possible, I'm open to discussing this further as a potential solution.
For context, I've opened an internal ticket in our internal system to track this so that both teams are aware of the issue and can work on a solution to prevent it from happening.
cc @ghengeveld , @thafryer
To reproduce
- Create a project with the framework of choice
- Enable Storybook
- Add an E2E solution to the project and the Chromatic counterpart
- Run tests
- Run Storybook and enable Storybook's Visual testing addon
- Link it and run a build
Environment
Tests were performed with Storybook 7.6 but can be replicated in the latest stable Storybook version.
Storybook Environment Info:
System:
OS: macOS 14.2
CPU: (8) arm64 Apple M3
Shell: 5.9 - /bin/zsh
Binaries:
Node: 18.19.0 - ~/.nvm/versions/node/v18.19.0/bin/node
Yarn: 1.22.21 - ~/.nvm/versions/node/v18.19.0/bin/yarn <----- active
npm: 10.2.3 - ~/.nvm/versions/node/v18.19.0/bin/npm
pnpm: 9.1.4 - ~/.nvm/versions/node/v18.19.0/bin/pnpm
Browsers:
Chrome: 125.0.6422.142
Edge: 125.0.2535.85
Safari: 17.2
npmPackages:
@storybook/addon-a11y: ^7.6.6 => 7.6.19
@storybook/addon-essentials: ^7.6.6 => 7.6.6
@storybook/addon-interactions: ^7.6.6 => 7.6.6
@storybook/addon-links: ^7.6.6 => 7.6.6
@storybook/blocks: ^7.6.6 => 7.6.6
@storybook/react: ^7.6.6 => 7.6.6
@storybook/react-vite: ^7.6.6 => 7.6.6
@storybook/test: ^7.6.5 => 7.6.6
@storybook/test-runner: ^0.18.2 => 0.18.2
chromatic: ^11.5.3 => 11.5.3
eslint-plugin-storybook: ^0.6.15 => 0.6.15
msw-storybook-addon: ^1.10.0 => 1.10.0
storybook: ^7.6.6 => 7.6.6```