Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 22 additions & 25 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,19 +86,25 @@ jobs:
## 🚀 swagger-coverage-cli v${{ env.NEW_VERSION }}

### ✨ New Features
- **🌐 Multi-Protocol Support**: Native support for REST (OpenAPI/Swagger), gRPC (Protocol Buffers), and GraphQL schemas
- **🔄 Mixed API Analysis**: Process multiple API specifications with different protocols in a single run
- **🎯 Protocol-Aware Matching**: Intelligent request matching tailored to each API protocol's characteristics
- **📊 Unified Reporting**: Generate consolidated HTML reports with protocol-specific insights and color coding
- **⚡ Universal CLI**: Single interface works across all supported protocols with consistent syntax
- **🛡️ Flexible Spec Validation**: New `--disable-spec-validation` flag to process specs with validation or reference issues

### 🎨 Enhanced Features
- **Smart Endpoint Mapping**: Intelligent endpoint matching with status code prioritization enabled by default
- **Enhanced Path Matching**: Improved handling of path parameters with different naming conventions
- **Confidence Scoring**: Match quality assessment with 0.0-1.0 confidence scores
- **Status Code Intelligence**: Prioritizes successful (2xx) codes over error codes for better coverage
- **Multi-API Support**: Process multiple API specifications in a single run
- **Enhanced HTML Reports**: Interactive reports with protocol identification and color coding
- **Legacy API Support**: Work with incomplete or invalid specs using `--disable-spec-validation`

### 🛡️ Spec Validation Control

The new `--disable-spec-validation` flag allows you to analyze coverage even when specs have validation issues:

```bash
# Skip validation for specs with broken references or validation errors
swagger-coverage-cli api.yaml collection.json --disable-spec-validation
```

**Use cases:**
- Legacy APIs with incomplete specifications
- Specs with external references that can't be resolved
- APIs in development where specs aren't fully complete
- Quick coverage checks without fixing all spec issues first

### 🎯 Protocol Support

Expand Down Expand Up @@ -147,11 +153,12 @@ jobs:
```

### 🧪 Quality Assurance
- **147 Tests**: Comprehensive test suite covering all protocols and scenarios
- **19 Test Suites**: Dedicated test coverage for each protocol and integration scenarios
- **Edge Case Coverage**: Robust handling of malformed URLs, missing data, and complex scenarios
- **183 Tests**: Comprehensive test suite covering all protocols and scenarios including spec validation control
- **22 Test Suites**: Dedicated test coverage for each protocol, integration scenarios, and validation features
- **Edge Case Coverage**: Robust handling of malformed URLs, missing data, broken references, and complex scenarios
- **Performance Tested**: Validated with large datasets and mixed protocol specifications
- **Protocol Isolation**: Each protocol's parsing and matching logic is independently tested
- **Validation Testing**: 16 new tests for `--disable-spec-validation` flag covering unit and CLI integration

---

Expand Down Expand Up @@ -184,15 +191,5 @@ jobs:
echo "- **GitHub Release:** [v${{ env.NEW_VERSION }}](https://github.com/${{ github.repository }}/releases/tag/v${{ env.NEW_VERSION }})" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "### 🎯 Key Features" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Multi-protocol support (REST, gRPC, GraphQL)" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Protocol-aware matching logic" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Mixed API analysis in single run" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Smart endpoint mapping (enabled by default)" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Status code prioritization" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Enhanced path matching" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Confidence scoring" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Multi-API support" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Newman report support" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Enhanced HTML reports with protocol identification" >> $GITHUB_STEP_SUMMARY
echo "- ✅ YAML, JSON, CSV, .proto, .graphql support" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Flexible spec validation with --disable-spec-validation flag" >> $GITHUB_STEP_SUMMARY
echo "- ✅ Backwards compatibility" >> $GITHUB_STEP_SUMMARY
2 changes: 1 addition & 1 deletion auto-detect-newman.html
Original file line number Diff line number Diff line change
Expand Up @@ -384,7 +384,7 @@ <h1>Swagger Coverage Report</h1>
&#128262; <!-- flashlight icon -->
</button>
<div class="meta-info">
<p><strong>Timestamp:</strong> 9/18/2025, 2:32:58 PM</p>
<p><strong>Timestamp:</strong> 10/9/2025, 8:30:30 AM</p>
<p><strong>API Spec:</strong> Test API</p>

<p><strong>Postman Collection:</strong> Test Newman Collection</p>
Expand Down
5 changes: 3 additions & 2 deletions cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,12 @@ program
.option("-v, --verbose", "Show verbose debug info")
.option("--strict-query", "Enable strict validation of query parameters")
.option("--strict-body", "Enable strict validation of requestBody (JSON)")
.option("--disable-spec-validation", "Disable OpenAPI/Swagger spec validation (useful for specs with validation or reference issues)")
.option("--output <file>", "HTML report output file", "coverage-report.html")
.option("--newman", "Treat input file as Newman run report instead of Postman collection")
.action(async (apiFiles, postmanFile, options) => {
try {
const { verbose, strictQuery, strictBody, output, newman } = options;
const { verbose, strictQuery, strictBody, output, newman, disableSpecValidation } = options;

// Parse comma-separated API files
const files = apiFiles.includes(',') ?
Expand Down Expand Up @@ -80,7 +81,7 @@ program
}
} else {
// Original OpenAPI/Swagger flow
const spec = await loadAndParseSpec(apiFile);
const spec = await loadAndParseSpec(apiFile, { disableValidation: disableSpecValidation });
specName = spec.info.title;
protocol = 'rest';
if (verbose) {
Expand Down
7 changes: 6 additions & 1 deletion lib/swagger.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ const SwaggerParser = require('@apidevtools/swagger-parser');
/**
* Загрузка и парсинг Swagger/OpenAPI (v2/v3) файла.
*/
async function loadAndParseSpec(filePath) {
async function loadAndParseSpec(filePath, options = {}) {
if (!fs.existsSync(filePath)) {
throw new Error(`Spec file not found: ${filePath}`);
}
Expand All @@ -24,6 +24,11 @@ async function loadAndParseSpec(filePath) {
doc = JSON.parse(raw);
}

// If validation is disabled, just return the parsed document without validation
if (options.disableValidation) {
return doc;
}

// Валидируем и нормализуем через SwaggerParser
const parsed = await SwaggerParser.validate(doc);
return parsed;
Expand Down
56 changes: 56 additions & 0 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ swagger-coverage-cli "api.yaml,service.proto" collection.json --verbose --strict
- **🏗️ Enterprise Ready**: Perfect for microservices architectures using diverse API protocols
- **🎨 Smart Endpoint Mapping**: Intelligent endpoint matching with status code prioritization and enhanced path matching
- **🔒 Strict Matching (Optional)**: Enforce strict checks for query parameters, request bodies, and more
- **🛡️ Flexible Validation**: Skip spec validation with `--disable-spec-validation` for legacy APIs or specs with reference issues
- **📈 Enhanced HTML Reports**: Generates interactive `coverage-report.html` with protocol identification
- **🧩 Extensible**: Modular code structure allows customization of matching logic and protocol support
- **📋 CSV Support**: Flexible API documentation format for teams preferring spreadsheet-based docs
Expand Down Expand Up @@ -260,6 +261,7 @@ npm swagger-coverage-cli "users-api.yaml,products-api.yaml" newman-report.json -
- `--newman`: Treat input file as Newman run report instead of Postman collection.
- `--strict-query`: Enforce strict checks on query parameters (e.g., required params, `enum`, `pattern`, etc.).
- `--strict-body`: Verify that `application/json` request bodies in the spec match raw JSON bodies in Postman requests.
- `--disable-spec-validation`: Disable OpenAPI/Swagger spec validation (useful for specs with validation or reference issues).
- `--output <file>`: Customize the name of the HTML report file (default is `coverage-report.html`).

### Run via NPM Script
Expand Down Expand Up @@ -539,6 +541,60 @@ Beyond basic percentage, consider these quality indicators:

---

## Handling Specs with Validation Issues

Sometimes API specifications may have validation errors or broken references, especially in legacy systems or during development. The `--disable-spec-validation` flag allows you to analyze coverage even when specs have issues.

### When to Use

Use `--disable-spec-validation` when:
- Working with legacy APIs that have incomplete specifications
- Specs contain broken `$ref` references that can't be resolved
- External references aren't available in your CI/CD environment
- You need quick coverage analysis without fixing all spec issues first
- API specifications are still in development

### Usage

```bash
# Standard usage (validation enabled - will fail on invalid specs)
swagger-coverage-cli api.yaml collection.json

# Disable validation for specs with issues
swagger-coverage-cli api.yaml collection.json --disable-spec-validation

# Works with all other flags
swagger-coverage-cli api.yaml collection.json --disable-spec-validation --verbose --strict-body
```

### Example

**Without the flag (validation enabled):**
```bash
$ swagger-coverage-cli broken-spec.yaml collection.json
Error: Token "NonExistentSchema" does not exist.
```

**With the flag (validation disabled):**
```bash
$ swagger-coverage-cli broken-spec.yaml collection.json --disable-spec-validation
=== Swagger Coverage Report ===
Total operations in spec(s): 12
Matched operations in Postman/Newman: 9
Coverage: 75.00%

HTML report saved to: coverage-report.html
```

### Important Notes

- When validation is disabled, the tool parses the spec without validating references
- Coverage can still be calculated for the operations that are defined
- The tool won't catch structural issues in the spec
- Default behavior (with validation enabled) ensures spec quality

---

## Detailed Matching Logic

**swagger-coverage-cli** tries to match each **operation** from the spec with a **request** in Postman. An operation is considered **covered** if:
Expand Down
Loading