Technical Requirements for a Test Automation Framework for End-to-End API Testing

And how Playwright satisfies these requirements:

1. Use only standard components (yes, Playwright):
   • Easy to install dependencies (a single NPM package)
   • Easy to configure (config file)
   • Easy to use HTTP library (suitable for API testing. API testing with Playwright does not require browsers installation. However, browser will be needed if you decide to run the tests in UI Mode for debugging)
2. Well tests organization:
   • Logical division of tests into suites, steps, and nested steps (test function)
   • Global preconditions and postconditions (global setup and teardown)
   • Preconditions and postconditions for a single test (by beforeAll and afterAll hooks)
   • Easy to use assertions (only Playwright’s non-retrying assertions are valuable for API testing)
   • Custom error messages (custom expect message) and custom assertions (this can be implemented through fixtures)
   • Table-driven tests (parameterize tests)
   • Contract testing (supported by third-party libraries like Ajv or Zod)
3. Various ways to bypass the instability of the fragile tested infrastructure (unexpected errors, timeouts) which leads to false-positive fails:
   • Retry HTTP request based on conditions (Playwright does not support this feature, but it can be implemented through expect.poll or expect.toPass methods)
   • Retry tests during the test run (retries feature)
   • Re-run only failed tests after a test run (--last-failed CLI option)
   • Individual timeouts for different test cases (multiple configurable timeouts and test.slow method)
   • Do not terminate test execution in case of a failed assertion (soft assertions)
4. Scaling:
   • Parallel execution (parallelism) and sequential launch if needed (this is configurable at the test and workes level)
   • Filtering tests: «running by tag», exclude tests from a run, run only a single test (by --grep and --grep-invert CLI options or by the test.only method, as well as on the the describe level)
   • Skipping tests (by test.skip and test.fixme methods, as well as on the describe level)
5. Good test reports:
   • Good build-in reporters (especially HTML one)
   • Adding your own debug information to the report (test.info method, even with attachments) — for API tests you definitely need cURL command for manual HTTP debugging
   • Easy to add Allure report (optional)

Stack

A basic set of packages for API testing with Playwright:

• Playwright — testing framework
• Zod — JSON schema validator
• Daj.js — minimalist date utility library
• Prettier — code formatter
• ESLint — code linter
• Vitest — unit testing framework (optional)

Example API for testing: NASA API

How to Use

You have to have Node.js >= 24 (LTS):

1. Clone repository
2. Install dependencies: npm install
3. Run tests: npm run test

CLI Options

• A different tested host can be passed to the tests through an environment variable (it can be useful for testing different environments):

BASE_URL=https://api.nasa.gov npm run test

• An individual API key can be passed to the tests through an environment variable; otherwise, the DEMO_KEY value will be used.

API_KEY={api_key} npm run test

• Run a single test or tests that match a specific criteria:

npm run test -- --grep @nasa

• Run tests in UI Mode:

npm run test -- --ui

Concepts

• All interactions with environment variables are carried out in /utils/env.ts file
• All constants (endpoints, timeouts, etc.) are contained in their respective directories
• Each request and response is logged in the report (read more about it)
• Expect messages should be written in BDD (Behavior-Driven Development) style, such as «something should have something»
• Most utils/helpers functions should have unit tests (optional)