Phase 0.2 | Add Dry-run test SDK generation across all languages #172
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
November 3, 2025 14:09
- Implements dry-run testing for all 6 SDK languages (Java, Ruby, Python, JavaScript, C#, Go) - Validates OpenAPI spec can generate SDKs without publishing - Runs on PR changes to mx_platform_api.yml only - Includes comprehensive file structure validation and basic syntax checking - Provides clear pass/fail status with error reporting - Temporary files cleaned up after validation Addresses Phase 0 AC for SDK generation pre-checks
…ource files don't exist
added 15 commits
November 4, 2025 14:51
Gvieve
approved these changes
Nov 5, 2025
Collaborator
Gvieve
left a comment
Gvieve
left a comment
There was a problem hiding this comment.
It looks absolutely byoo-tiful! 🎉 😍
nickitza
added a commit
that referenced
this pull request
Nov 5, 2025
* feat(api): add 35 missing schemas from models.yaml
Phase 1 of synchronization: Added all missing schemas inline to
mx_platform_api.yml with internal references.
Added schemas:
- ACHResponse, ACHReturnCreateRequest, ACHReturnCreateRequestBody
- ACHReturnResponseBody, ACHReturnsResponseBody
- AllVerifications, AllVerificationsResponse
- InsightUpdateRequestBody
- InvestmentHoldingResponse, InvestmentHoldingResponseBody
- InvestmentHoldingsDeactivation, InvestmentHoldingsResponseBody
- MemberElements, MicrodepositElements
- NotificationResponse, NotificationResponseBody, NotificationsResponseBody
- PaymentAccount, PaymentAccountBody
- PreInitiateMicrodeposit
- ProcessorAccountNumber, ProcessorAccountNumberBody
- ProcessorOwner, ProcessorOwnerBody
- ProcessorTransaction, ProcessorTransactionBody
- RepeatingTransactionResponse, RepeatingTransactionsResponseBody
- RewardElements
- TokenRequestBody, TokenResponse, TokenResponseBody
- TransactionIncludesResponse, TransactionsResponseBodyIncludes
- VCResponse
All external references converted to internal format:
$ref: '#/SchemaName' → $ref: '#/components/schemas/SchemaName'
File changes:
- Before: 9,341 lines (177 schemas)
- After: 10,342 lines (212 schemas)
- Added: 1,002 lines (35 schemas)
Verification:
- Missing schemas: 0 (confirmed via comparison tool)
- YAML valid: passes Redocly preview server
- Internal $ref format: all converted
Tools created for reusability:
- tmp/sync_schemas.rb: Dynamic schema synchronization script
- tmp/compare_openapi_specs.rb: Comparison tool
- tmp/SYNC_SCHEMAS_README.md: Usage documentation
Refs: #DEVX-7466
* feat(api): synchronize schema fields with models.yaml
Phase 2: Add missing fields and remove extra fields to achieve exact parity
Added missing fields (15 total):
- ConnectWidgetRequest: enable_app2app
- InsightResponse: has_associated_categories
- InstitutionResponse: instructional_text_steps, is_disabled_by_client, iso_country_code
- MemberCreateRequest: use_cases
- MemberUpdateRequest: use_cases
- TransactionResponse: is_manual
- TransactionUpdateRequest: date, memo, category_guid
- WidgetRequest: connections_use_case_filter, enable_app2app, iso_country_code, use_cases
Removed extra fields (10 total):
- ImageOptionResponse: guid (removed)
- MemberResponse: error (removed)
- MicrodepositResponse: account_name, account_number, account_type, email, first_name, last_name, routing_number (all removed)
- OptionResponse: guid (removed)
Changes: +77 lines, -33 lines
Script: tmp/sync_fields.rb (reusable for v20250224)
Status after Phase 2:
- Missing schemas: 0 ✅
- Missing fields: 0 ✅
- Extra fields: 0 ✅
- Extra schemas: 9 (Phase 3)
- Missing parameters: 72 (Phase 4)
- Missing paths: 25 (Phase 5)
- Extra paths: 10 (Phase 6)
* feat(params): consolidate parameters to components.parameters library
Phase 3: Parameter Synchronization and Consolidation
This commit implements atomic parameter synchronization between v20111101.yaml
and mx_platform_api.yml, adding missing parameters to the components library
and converting inline parameter definitions to $ref where possible.
Changes:
- Added sync_parameters.rb script with atomic 3-phase operation:
* Part 1: Add missing parameters to components.parameters
* Part 2: Remove extra parameters from components.parameters
* Part 3: Convert inline parameters to $ref (NEW)
- Added SYNC_PARAMETERS_README.md with comprehensive documentation
- Added sync_fields.rb and SYNC_FIELDS_README.md for field synchronization
Phase 3a - Library Creation:
- Added 72 parameters to components.parameters section
- Created section before securitySchemes for proper structure
- Converted external refs to internal format
- Total library size: 519 lines (5201-5719)
Phase 3b - Inline Conversion:
- Converted 352 inline parameter definitions to $ref
- Affected 46 unique parameters across 144 operations
- Net reduction: 1,466 lines (14% file size reduction)
- File size: 10,386 → 8,920 lines
- 11 inline parameters intentionally kept (variant descriptions or extra paths)
Inline Parameters Kept:
- page, records_per_page: Minor description variants
- user_guid, account_guid: Context-specific descriptions
- tax_document_guid: From extra paths (will be removed in Phase 6)
Script Features:
- Atomic operation ensures consistency
- Set-based tracking prevents duplicates
- Precise regex with lookahead prevents tag/response capture
- Comprehensive statistics and error reporting
- Runtime: 5-10 seconds typical
Verification:
- 0 missing parameters (verified via compare_openapi_specs.rb)
- 0 extra parameters (verified via compare_openapi_specs.rb)
- 352 $ref conversions (verified via grep)
- YAML structure validated
Breaking Changes: None
Related: DEVX-7466
* feat(paths): sync API paths with v20111101.yaml using yq
Phase 4: Path Synchronization (BREAKING)
This commit synchronizes all API paths between v20111101.yaml and
mx_platform_api.yml using yq to preserve YAML formatting and avoid
unwanted reformatting side effects.
Changes:
- Added sync_paths.rb script using yq for format-preserving YAML manipulation
- Added SYNC_PATHS_README.md with comprehensive documentation
- Added TECHNOLOGY_STACK.md enforcing Ruby-only policy
- Removed compare_openapi_specs.py (Python - violates Ruby-only policy)
- Modified mx_platform_api.yml with path synchronization
Phase 4a - Add Missing Paths (25):
- ACH returns: /ach_returns, /ach_returns/{ach_return_guid}
- Investment holdings (6 paths):
* /users/{user_guid}/investment_holdings
* /users/{user_guid}/investment_holdings/{holding_guid}
* /users/{user_guid}/investment_holdings_deactivate
* /users/{user_guid}/accounts/{account_guid}/investment_holdings
* /users/{user_guid}/members/{member_guid}/investment_holdings
- Account operations: /account/account_numbers, /account/check_balance, /account/transactions
- Notifications: /users/{user_guid}/notifications, /users/{user_guid}/notifications/{notification_guid}
- Repeating transactions (2 paths)
- VC endpoints (3 paths)
- Micro deposit verify: /micro_deposits/{micro_deposit_guid}/verify
- Account verifications, tokens, payment accounts
- Transaction insights, spending plan current iteration
- Fixed path: /users/{user_guid}/insights/{insight_guid}
Phase 4b - Remove Extra Paths (10 - BREAKING):
- Holdings endpoints → Investment holdings (4 paths removed)
- Tax documents feature removed entirely (4 paths)
- Path typo fixes:
* /insights{insight_guid} → /insights/{insight_guid}
* /microdeposit_guid → /micro_deposit_guid
Technology Improvements:
- Installed yq v4.48.1 (brew install yq)
- Uses yq for YAML manipulation (preserves formatting)
- No quote style changes, no date format changes
- Minimal whitespace cleanup (trailing spaces, array formatting)
- Ruby-only policy enforced (removed Python scripts)
Net Result:
- Path count: 99 → 114 (matches v20111101.yaml)
- File changes: +900 lines, -891 lines (net +9 lines)
- Format preserved: no massive reformatting issues
Breaking Changes:
1. Holdings API renamed to Investment Holdings
- Update SDK method names and client code
2. Tax documents feature removed (4 endpoints)
- No longer available in API
3. Path structure fixes may affect hardcoded URLs
Verification:
- 0 missing paths
- 0 extra paths
- 114 paths in perfect sync with v20111101.yaml
Related: DEVX-7466
* feat(tags): synchronize OpenAPI tags for improved navigation
Replace generic 'mx_platform' tag with 25 domain-specific tags to improve
API documentation organization and user experience in Swagger/Redoc UI.
Changes:
- Update 90 path operation tags from 'mx_platform' to specific domains
- Replace top-level tags definition with all 25 domain tags
- Eliminate generic catch-all tag for better endpoint discoverability
Tag domains added:
- ach return, accounts, budgets, categories, goals
- insights, institutions, investment holdings, managed data
- members, merchants, microdeposits, monthly cash flow profile
- notifications, processor token, rewards, spending plan
- statements, taggings, tags, transaction rules
- transactions, users, verifiable credentials, widgets
Impact:
- Non-breaking: documentation organization only
- Improved navigation: endpoints logically grouped by domain
- Better UX: 25 focused sections instead of 99-item generic list
- Matches v20111101.yaml tag structure
Files changed:
- openapi/mx_platform_api.yml (124 insertions, 100 deletions)
- tmp/sync_tags.rb (Ruby script using yq for format preservation)
- tmp/SYNC_TAGS_README.md (complete phase documentation)
- tmp/PHASES_OVERVIEW.md (project phase tracking)
Phase 5 complete: Tag synchronization successful
* feat(phase6): fix schema structures and remove deprecated schemas
Fix 3 schema structures to match models.yaml using Elements composition:
- RewardResponseBody: Use MemberElements + RewardElements (allOf)
- RewardsResponseBody: Use MemberElements + RewardElements (allOf)
- MicrodepositRequestBody: Use MicrodepositElements
Remove 9 deprecated schemas not present in models.yaml:
- HoldingResponse, HoldingResponseBody, HoldingsResponseBody
- MicrodepositRequest
- RewardResponse, RewardsResponse
- TaxDocumentResponse, TaxDocumentResponseBody, TaxDocumentsResponseBody
Impact: 257 lines changed (7 insertions, 250 deletions)
Schema count: 211 → 202 (-9 schemas)
BREAKING CHANGE: Deprecated schemas removed. These were already replaced by newer versions in v20111101.yaml (InvestmentHoldings, Elements patterns).
* feat(phase7): fix type mismatches from integer to number
Fix 3 fields to use number type instead of integer to match models.yaml:
- MicrodepositVerifyRequest.deposit_amount_1: integer → number
- MicrodepositVerifyRequest.deposit_amount_2: integer → number
- MonthlyCashFlowResponse.estimated_goals_contribution: integer → number
Financial amounts and deposit values require decimal precision (cents).
Impact: 3 type changes, minimal breaking risk (number is more permissive than integer)
* feat(phase8): internalize external references for self-contained spec
Convert all external file references to internal component references,
making mx_platform_api.yml completely self-contained and portable.
Changes:
- Converted 33 schema references (25 unique schemas)
- Converted 60 parameter references (25 unique parameters)
- Total: 93 reference conversions
- Pattern: './schemas/models.yaml#/X' → '#/components/schemas/X'
- Pattern: './schemas/parameters.yaml#/X' → '#/components/parameters/X'
Validation:
- Verified all 25 unique schemas exist in components/schemas
- Verified all 25 unique parameters exist in components/parameters
- YAML structure validated post-conversion
- Preview server operational without external dependencies
Cleanup:
- Removed temporary openapi/schemas/ directory
- File is now completely self-contained with no external dependencies
Progress: 8 of 9 phases complete (89%)
Next: Phase 9 - Final Validation
Refs: devx-7466
* feat(phase9): final validation and complete parity achieved
Add missing delete operation and comprehensive validation script to achieve
100% parity with v20111101.yaml source specification.
Changes:
- Added missing delete operation to /users/{user_guid}/transactions/{transaction_guid}
- Created comprehensive validation script (phase9_final_validation.rb)
- Validates schemas, parameters, paths, operations, types, tags, and structure
Validation Results (100% Pass):
✓ Schema parity: 202/202 schemas match
✓ Parameter parity: 72/72 parameters match
✓ Path parity: 114/114 paths with all operations
✓ Type consistency: All types match models.yaml
✓ No external references (fully self-contained)
✓ Tag synchronization: 25/25 tags properly assigned
✓ File structure: All required sections present
Project Complete:
- 9 of 9 phases complete (100%)
- mx_platform_api.yml fully synchronized with v20111101.yaml
- Specification is production-ready, self-contained, and validated
Refs: devx-7466
* fully replace mx_platform_api.yml with v20111101.yml, remove tmp files
* fix(parameters): add missing schema definitions and in property
Fix 6 parameters that were missing required properties causing validation errors:
- userIsDisabled: Added type: boolean, in: query
- topLevelCategoryGuidArray: Added type: array with items, in: query
- topLevelCategoryGuid: Added type: string, in: query
- includes: Added type: string, in: query
- categoryGuidQueryArray: Added type: array with items, in: query
- categoryGuidQuery: Added type: string, in: query
All schema definitions verified against source parameters.yaml file.
Fixes validation errors:
- #/components/parameters/{param}/schema must be object
- #/components/parameters/{param} must have required property 'in'
- #/paths/~1users/get/parameters/4/schema must be object
Refs: devx-7466
* fix(structure): reorder YAML keys to OpenAPI spec requirements
Fix "Cannot read properties of null (reading 'swagger')" validation error
by ensuring top-level keys are in the correct order.
OpenAPI specifications must have keys in this order:
1. openapi
2. info
3. servers
4. security
5. tags
6. paths
7. components
The file had all required fields but in wrong order (components was first),
causing validators to fail when reading line-by-line expecting 'openapi' or
'swagger' as the first key.
File now starts with proper header:
openapi: 3.0.0
info:
...
Refs: devx-7466
* fix(parameters): add missing properties to includes parameter
Add missing name, example, and required properties to includes parameter
to fix validation error.
Also refactor phase3_sync_parameters.rb to prevent this issue in future runs:
- Changed from regex text extraction to YAML parsing
- Now syncs ALL properties from source (name, description, in, required, example, schema)
- Uses yq commands to add each property individually
- Validates required properties after adding
This ensures parameters are copied completely from parameters.yaml source,
preventing missing property issues that required post-fix patches.
Fixes validation error:
#/components/parameters/includes must have required property 'name'
Root cause: Original phase3 script used regex that didn't capture all
properties, especially for parameters with multi-line descriptions.
Refs: devx-7466
* tmp file clean up, rename mx_platform_api.yml > v20111101.yaml
* chore(ci): update workflows for v20111101.yml filename
Update GitHub Actions workflows to reference v20111101.yml instead of
mx_platform_api.yml after file rename for version clarity.
Also fix .gitignore to allow workflow files while blocking AI toolkit:
- Changed from broad .github/* pattern to specific subdirectories
- Now ignores: .github/instructions/*, .github/chatmodes/*, .github/prompts/*
- Allows: .github/workflows/* (CI/CD pipelines)
Files updated:
- .github/workflows/validate.yml
- .github/workflows/version.yml
- .github/workflows/update.yml
- .gitignore
Refs: devx-7466
* fix(ci): correct file extension to .yaml in workflows
The OpenAPI file is v20111101.yaml (not .yml), update all workflow
references to use the correct extension.
* fix(ci): restore original version.yml workflow logic
Accidentally replaced workflow logic when only the filename should have
been updated. Restored original structure with tj-actions/changed-files
and mheap/github-action-required-labels actions, only updating filename
from mx_platform_api.yml to v20111101.yaml.
* revert filename change: v20111101.yaml back to mx_platform_api.yml and updated references inside of workflows
* fix(spec): add missing bearerAuth scheme and transaction parameters
Fix OpenAPI validation errors discovered by Redocly linter:
1. Add bearerAuth security scheme definition
- Was present in source v20111101.yaml but not synced during phases
- Required by 4 processor token endpoints
2. Add missing parameters to DELETE /users/{user_guid}/transactions/{transaction_guid}
- Added transactionGuid and userGuid parameter refs
- Pre-existing issue from source file
These fixes resolve all 6 critical errors. Spec now validates successfully
with 'npx @redocly/cli lint' (231 warnings remain, non-critical).
Root cause: Phase scripts didn't sync securitySchemes section from source.
* fix(spec): sync 21 missing operationIds from source
Sync missing operationIds from v20111101.yaml to mx_platform_api.yml for
existing path operations that were not updated during Phase 4.
Root cause: Phase 4 only added/removed entire paths but didn't update
properties on paths that already existed in both files. Operations existed
in mx_platform_api.yml before synchronization but lacked operationIds.
Solution: Created fix_missing_operationIds.rb and refactored logic into
phase4_sync_paths.rb as Part 3 to prevent future occurrences.
Validation: Resolved all 21 operation-operationId warnings
- Before: 231 warnings (21 operationId + 210 others)
- After: 210 warnings (pre-existing from source: missing 4xx, descriptions)
* fix(openapi): sync missing parameters and response schemas from source
Sync 123 operation property updates to achieve parity with v20111101.yaml:
- Add 118 missing parameter references across 54 operations
- Fix 5 response schemas to use TransactionsResponseBodyIncludes
This addresses Phase 4 gaps where existing operations were not fully
synchronized with the source file. Operations now have complete parameter
lists and correct response schema references for enhanced transaction data.
Impact on validation warnings:
- Eliminates 21 "missing operationId" warnings (synced separately)
- Eliminates 15 "unused component" warnings (14 params + 1 schema now used)
- Remaining 3 unused schemas match source (inherited, not introduced)
Refs: DEVX-7466
* remove redundant refs
* fix(openapi): add required items field to iso_country_code array schema
Add items field to WidgetRequest.iso_country_code to comply with OpenAPI 3.0
specification requirement that array types must define their element types.
This change makes the schema technically valid without affecting documentation
display or user request construction, as the example array already demonstrates
the expected string values ["US", "CA"].
* fix(openapi): sync version and deprecation flag with source spec
Update mx_platform_api.yml to match v20111101.yaml source file:
- Change version from 0.1.0 to '20111101' to match source specification
- Add missing deprecated: true flag to deprecatedRequestPaymentProcessorAuthorizationCode operation
These changes ensure the compiled spec is a carbon copy of the source file.
* re-apply unused tags for parity to v20111101.yaml
* mx_platform_api generated by tiny mx from v20111101
* Phase 0.2 | Add Dry-run test SDK generation across all languages (#172)
* Add SDK generation validation workflow
- Implements dry-run testing for all 6 SDK languages (Java, Ruby, Python, JavaScript, C#, Go)
- Validates OpenAPI spec can generate SDKs without publishing
- Runs on PR changes to mx_platform_api.yml only
- Includes comprehensive file structure validation and basic syntax checking
- Provides clear pass/fail status with error reporting
- Temporary files cleaned up after validation
Addresses Phase 0 AC for SDK generation pre-checks
* downgrade versions to v3 to match existing usages in sdks
* remove intensive full project validation and replae with failing if source files don't exist
* on: pull_request, remove path
* fix javascript references to be node and point to typescript-axios generator
* paths array
* restore matrix.generator
* Add simple test workflow to debug trigger
* Remove paths filter to test workflow trigger
* Remove test workflow and fix main SDK validation workflow trigger
* test removing setup
* trigger test
* remove test comment
* test comment to test trigger
* remove java@v3 setup
* silence file logs, stop after finding 1
* quit after finding 1
* remove comments
---------
Co-authored-by: Nicki Nixon <nicki.nixon@mx.com>
---------
Co-authored-by: Nicki Nixon <nicki.nixon@mx.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Using the feature branch from https://mxcom.atlassian.net/browse/DEVX-7466 we need to validate that the consolidated OpenAPI specification works with our SDK generation process. This ticket implements basic dry-run testing as a required CI check to prevent broken SDK generation from reaching production. This also provides basic testing validation for the SDKs which is a major gap we have today.
Acceptance Criteria:
CI Pipeline Setup:
Configure as required status check for PR mergeability
Trigger on PRs affecting mx_platform_api.yml only
Clear pass/fail status with error logs if generation fails
Basic Validation Testing:
NOTE: This is already part of validation but ensure it is retained or moved into CI pipeline accordingly
Test OpenAPI Generator can process specification for all 6 languages (Java, Ruby, Python, Node.js, C#, Go)
Verify expected files are generated to temporary directories
Clean up temporary files after validation
Merge feature branch and confirm all SDKs are successfully generated, published and released
Safety & Performance NOTES:
Ensure no artifacts published, no repository modifications
Target execution time under 5 minutes total. For example, you can use GitHub Actions matrix strategy for parallel execution.
Note
The Test SDK workflow does not show up in this MR because it is configured to only run on a PR on the mx_platform_api.yml. Since there were no changes to that file here, it didnt run. I did validate that when there is a change to that file, the workflow successfully ran and every SDK check passed. Below are screenshots of when a change was made to the mx_platform_api.yml: