diff --git a/.claude/agents/README.md b/.claude/agents/README.md new file mode 100644 index 0000000..b6d493d --- /dev/null +++ b/.claude/agents/README.md @@ -0,0 +1,75 @@ +# Agent Workflows + +Token-efficient AI agent workflows for Goblet development tasks. + +## Available Agents + +| Agent | Purpose | Key Tasks | +|-------|---------|-----------| +| **release.md** | Manage releases | Version determination, GoReleaser workflow, artifact verification | +| **testing.md** | Run test suites | CI pipeline, offline mode tests, race detection | +| **offline-mode.md** | Verify offline features | Fallback testing, thread safety, monitoring | +| **documentation.md** | Maintain docs | Standards, structure, conventional commits | + +## Usage + +Each agent provides: +- **Core Workflow**: Step-by-step process for the task +- **Quick Commands**: Copy-paste command reference +- **Key Files**: Related source files and documentation +- **Verification Steps**: How to validate success + +## Design Principles + +1. **Token Efficient**: Concise, actionable content +2. **Self-Contained**: Each agent is independently usable +3. **Standards-Based**: Uses project conventions (GoReleaser, conventional commits) +4. **Practical**: Real commands, not abstract concepts + +## When to Use + +**Release Agent**: Before creating a version tag +**Testing Agent**: Before committing code changes +**Offline Mode Agent**: When modifying cache/offline features +**Documentation Agent**: When updating project docs + +## Integration with Development + +These agents complement: +- `Taskfile.yml` - Development task automation +- `.github/workflows/` - CI/CD pipelines +- `RELEASING.md` - Full release documentation +- `testing/TEST_COVERAGE.md` - Test details + +## Contributing Agent Workflows + +When adding new agents: +1. Keep under 2 pages (200-300 lines) +2. Lead with core workflow +3. Include quick command reference +4. Link to detailed documentation +5. Focus on actionable steps +6. Use existing agents as templates + +## Example Usage + +```bash +# Before creating a release +cat .claude/agents/release.md +# Follow the workflow: verify CI, check commits, create tag + +# When writing offline mode tests +cat .claude/agents/offline-mode.md +# Use the test commands and verification steps + +# For test-driven development +cat .claude/agents/testing.md +# Run baseline tests, implement, verify with CI +``` + +## Project Context + +**Repository**: github-cache-daemon (Goblet) +**Purpose**: Git caching proxy server +**Key Technologies**: Go, GoReleaser, GitHub Actions +**Main Features**: Offline mode, multi-platform releases, comprehensive testing diff --git a/.claude/agents/documentation.md b/.claude/agents/documentation.md new file mode 100644 index 0000000..f319344 --- /dev/null +++ b/.claude/agents/documentation.md @@ -0,0 +1,142 @@ +# Documentation Agent + +Maintain comprehensive documentation following project standards. + +## Documentation Structure + +``` +├── README.md # Main project overview, usage +├── RELEASING.md # Release process (GoReleaser) +├── CHANGELOG.md # Keep a Changelog format +├── CONTRIBUTING.md # Contribution guidelines +├── TESTING.md # Test infrastructure overview +├── testing/TEST_COVERAGE.md # Detailed test coverage +├── testing/README.md # Test details +└── .claude/agents/*.md # Agent workflows +``` + +## Documentation Standards + +**Format**: GitHub-flavored Markdown +**Style**: +- Clear, concise, actionable +- Code blocks with language hints +- Examples before explanations +- Commands before concepts + +**Structure**: +1. Quick start / TL;DR section +2. Detailed explanations +3. Examples and code snippets +4. Troubleshooting (if applicable) +5. Related resources + +## Conventional Commits for Docs + +```bash +# Documentation changes +git commit -m "docs: update offline mode configuration examples" +git commit -m "docs: add troubleshooting section to RELEASING.md" +git commit -m "docs: clarify semantic versioning guidelines" + +# Documentation fixes +git commit -m "fix(docs): correct Docker image registry URL" +``` + +## Key Documentation Areas + +### 1. README.md +- Project overview and purpose +- Quick start usage +- Offline mode features +- Testing instructions +- Basic configuration + +### 2. RELEASING.md +- Complete release process +- Conventional commit guidelines +- GoReleaser workflow +- Troubleshooting releases +- Release checklist + +### 3. CHANGELOG.md +- Follow [Keep a Changelog](https://keepachangelog.com/) +- Semantic versioning links +- Group by: Added, Changed, Fixed, Deprecated, Removed, Security +- Update on each release (automated by GoReleaser) + +### 4. Testing Documentation +- Test coverage details (`testing/TEST_COVERAGE.md`) +- Test infrastructure (`testing/README.md`) +- Quick test commands in README + +## Documentation Workflow + +1. **When adding features** + - Update README.md with usage + - Add examples and configuration + - Update test documentation if applicable + - Consider agent workflow updates + +2. **When fixing bugs** + - Add troubleshooting section if useful + - Update examples if bug was in docs + +3. **When releasing** + - Verify CHANGELOG.md updated (automatic) + - Check version references + - Verify all new features documented + +4. **For breaking changes** + - Add migration guide + - Update UPGRADING.md + - Clear examples of before/after + - Use `BREAKING CHANGE:` in commit + +## Documentation Verification + +```bash +# Check markdown formatting +markdownlint **/*.md + +# Verify links +markdown-link-check README.md + +# Test code examples +# Extract and run code blocks to verify accuracy +``` + +## Writing Guidelines + +**Code Examples**: +- Include full commands, not fragments +- Show expected output +- Use realistic paths and values +- Test before documenting + +**Structure**: +- Use headers (##, ###) for organization +- Bullet points for lists +- Numbered lists for sequences +- Code fences with language hints + +**Cross-references**: +- Link to related docs +- Reference specific files with full paths +- Use anchor links for long documents + +## Agent Workflow Documentation + +Agent files (`.claude/agents/*.md`) should: +- Be concise (1-2 pages max) +- Lead with core workflow +- Include quick command reference +- Link to detailed docs +- Focus on actionable steps + +## Related Resources + +- [Conventional Commits](https://www.conventionalcommits.org/) +- [Keep a Changelog](https://keepachangelog.com/) +- [Semantic Versioning](https://semver.org/) +- [GitHub Flavored Markdown](https://github.github.com/gfm/) diff --git a/.claude/agents/offline-mode.md b/.claude/agents/offline-mode.md new file mode 100644 index 0000000..a951013 --- /dev/null +++ b/.claude/agents/offline-mode.md @@ -0,0 +1,94 @@ +# Offline Mode Verification Agent + +Verify offline mode functionality and resilience features. + +## Core Capabilities + +Goblet automatically falls back to local cache when upstream is unavailable: +- ls-refs requests served from local git repository +- Graceful degradation during outages +- Thread-safe configuration with atomic operations +- Staleness warnings (>5 minutes old refs) + +## Verification Workflow + +1. **Test automatic fallback** + ```bash + # Integration test with warm cache + go test ./testing -v -run TestOfflineModeWithWarmCache + + # Upstream failure scenarios + go test ./testing -v -run TestUpstreamFailureFallback + ``` + +2. **Test thread safety** + ```bash + # Race detection for concurrent access + go test -race ./testing -run TestConcurrentOfflineRequests + go test -race ./testing -run "Offline" + ``` + +3. **Verify configuration** + - Default: `UpstreamEnabled` = true with auto-fallback + - Testing: Use `SetUpstreamEnabled(&false)` to disable upstream entirely + - Thread-safe: Uses atomic operations for concurrent access + +## Expected Behaviors + +**Normal operation** (upstream available): +- Forwards ls-refs to upstream +- Caches response locally +- Serves fetches from cache when possible + +**Upstream failure** (network down): +- Detects failure on ls-refs +- Reads refs from local git repo +- Logs fallback event +- Serves cached refs to client + +**Staleness warnings**: +- Logs warning if refs >5 minutes old +- Format: "Warning: serving stale ls-refs for /path (last update: Xm ago)" + +## Monitoring Log Patterns + +```bash +# Fallback events +"Upstream ls-refs failed (connection refused), attempting local fallback" + +# Stale cache warnings +"Warning: serving stale ls-refs for /cache/path (last update: 10m ago)" +``` + +## Limitations + +- Fetch operations for uncached objects still fail (expected) +- Cold cache (no prior fetches) will error when upstream down (expected) +- Only ls-refs can be served offline, not fetch with new objects + +## Key Configuration + +```go +// Production (default) - auto-fallback enabled +config := &goblet.ServerConfig{ + LocalDiskCacheRoot: "/path/to/cache", + // UpstreamEnabled defaults to true +} + +// Testing - disable upstream entirely +falseValue := false +config.SetUpstreamEnabled(&falseValue) +``` + +## Test Coverage + +- 4 integration tests: End-to-end offline scenarios +- 8 unit tests: Edge cases, concurrency, filtering +- Thread safety verified with race detector +- See `testing/TEST_COVERAGE.md` for details + +## Related Documentation + +- `README.md` - Offline mode features and configuration +- `testing/TEST_COVERAGE.md` - Test details +- `testing/README.md` - Test infrastructure diff --git a/.claude/agents/release.md b/.claude/agents/release.md new file mode 100644 index 0000000..90a6a09 --- /dev/null +++ b/.claude/agents/release.md @@ -0,0 +1,58 @@ +# Release Agent + +Guide releases using GoReleaser with semantic versioning and conventional commits. + +## Core Workflow + +1. **Verify readiness** + - Check `gh run list --branch main --limit 1` for CI status + - Run `git log --oneline -10` to verify conventional commits + - Run `task ci` locally to ensure all checks pass + +2. **Version determination** + - MAJOR (v2.0.0): Breaking changes, incompatible API + - MINOR (v1.1.0): New features, backwards compatible + - PATCH (v1.0.1): Bug fixes only + +3. **Create release** + ```bash + git tag -a vX.Y.Z -m "Release vX.Y.Z" + git push origin vX.Y.Z + gh run watch + ``` + +4. **Verify artifacts** + - Check `gh release view vX.Y.Z` for all 5 platform archives + - Verify checksums.txt exists + - Test Docker image: `docker pull ghcr.io/jrepp/goblet-server:X.Y.Z` + +## Conventional Commit Types + +- `feat:` → Features (MINOR bump) +- `fix:` → Bug fixes (PATCH bump) +- `feat!:` or `BREAKING CHANGE:` → Major version bump +- `perf:`, `docs:`, `test:` → Included in changelog +- `chore:`, `ci:` → Excluded from changelog + +## Quick Commands + +```bash +# Test locally before release +goreleaser build --snapshot --clean +goreleaser check + +# Create pre-release +git tag -a v1.0.0-rc.1 -m "Release candidate" +git push origin v1.0.0-rc.1 + +# Delete failed release +gh release delete vX.Y.Z +git push origin :refs/tags/vX.Y.Z +git tag -d vX.Y.Z +``` + +## Key Files + +- `.goreleaser.yml` - Build configuration +- `.github/workflows/release.yml` - CI pipeline +- `RELEASING.md` - Full documentation diff --git a/.claude/agents/testing.md b/.claude/agents/testing.md new file mode 100644 index 0000000..083f655 --- /dev/null +++ b/.claude/agents/testing.md @@ -0,0 +1,86 @@ +# Testing Agent + +Run and verify test suites with focus on offline mode and integration tests. + +## Core Test Commands + +```bash +# Full CI pipeline +task ci # fmt + lint + test + build + +# Quick tests (38s, default for CI) +task test-short +go test ./... -short + +# Full tests including long-running +go test ./... + +# With race detection (thread safety) +go test -race ./... +``` + +## Offline Mode Testing + +```bash +# All offline functionality +go test ./testing -v -run "Offline|Upstream|LsRefsLocal" + +# Specific scenarios +go test ./testing -v -run TestOfflineModeWithWarmCache +go test ./testing -v -run TestUpstreamFailureFallback +go test ./testing -v -run TestConcurrentOfflineRequests + +# Thread safety verification +go test -race ./testing -run "Offline" +``` + +## Test Coverage + +Current coverage: +- **4 integration tests**: End-to-end with real git operations +- **8 unit tests**: Edge cases, concurrency, symbolic refs +- **38 total tests**: Full suite + +See `testing/TEST_COVERAGE.md` for details. + +## Test Workflow + +1. **Before code changes** + - Run `task test-short` to establish baseline + - Note any existing failures + +2. **After implementation** + - Run affected test suite specifically + - Run `task ci` for full verification + - Check race detector: `go test -race ./...` + +3. **For offline mode changes** + - Must test warm cache scenarios + - Must test upstream failure fallback + - Must test concurrent requests (race detector) + - Verify staleness warnings in logs + +## Quick Verification + +```bash +# Build only +task build + +# Format check +task fmt-check + +# Lint +task lint + +# Single package +go test ./testing -v + +# Single test +go test ./testing -v -run TestSpecificFunction +``` + +## Key Test Files + +- `testing/` - Integration test suite +- `testing/TEST_COVERAGE.md` - Coverage documentation +- `testing/README.md` - Test infrastructure details diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..fd1ff0b --- /dev/null +++ b/.dockerignore @@ -0,0 +1,9 @@ +.git +.github +.gitignore +*.md +docker-compose.yml +Dockerfile +.dockerignore +testing/ +*.test diff --git a/.editorconfig b/.editorconfig new file mode 100644 index 0000000..3a3dd94 --- /dev/null +++ b/.editorconfig @@ -0,0 +1,26 @@ +root = true + +[*] +charset = utf-8 +end_of_line = lf +insert_final_newline = true +trim_trailing_whitespace = true + +[*.go] +indent_style = tab +indent_size = 4 + +[*.{yml,yaml}] +indent_style = space +indent_size = 2 + +[*.{json,md}] +indent_style = space +indent_size = 2 + +[Makefile] +indent_style = tab + +[*.sh] +indent_style = space +indent_size = 2 diff --git a/.env.example b/.env.example new file mode 100644 index 0000000..9fd1b6d --- /dev/null +++ b/.env.example @@ -0,0 +1,36 @@ +# Docker Compose environment variables +# Copy this file to .env and customize + +# Architecture for Docker build (amd64 or arm64) +ARCH=amd64 + +# Authentication configuration +AUTH_MODE=oidc # oidc or google +OIDC_ISSUER=http://dex:5556/dex +OIDC_CLIENT_ID=goblet-server +OIDC_CLIENT_SECRET=goblet-secret-key-change-in-production + +# Dex configuration +DEX_PORT=5556 + +# Minio configuration +MINIO_ROOT_USER=minioadmin +MINIO_ROOT_PASSWORD=minioadmin +MINIO_BUCKET=goblet-backups + +# Goblet configuration +GOBLET_CACHE_ROOT=/cache +GOBLET_PORT=8888 +GOBLET_BACKUP_MANIFEST=dev + +# Storage provider configuration +STORAGE_PROVIDER=s3 +BACKUP_MANIFEST_NAME=dev + +# S3 configuration (for Minio) +S3_ENDPOINT=minio:9000 +S3_BUCKET=goblet-backups +S3_ACCESS_KEY=minioadmin +S3_SECRET_KEY=minioadmin +S3_REGION=us-east-1 +S3_USE_SSL=false diff --git a/.github/WORKFLOWS.md b/.github/WORKFLOWS.md new file mode 100644 index 0000000..7568771 --- /dev/null +++ b/.github/WORKFLOWS.md @@ -0,0 +1,188 @@ +# GitHub Actions Workflows + +## CI Workflow (`.github/workflows/ci.yml`) + +The CI workflow parallelizes the `task ci` command into separate jobs for optimal performance. + +### Workflow Structure + +``` +Pull Request / Push to main +│ +├─── format-check ────┐ +├─── tidy-check ──────┤ +├─── lint ────────────┤──> ci-complete (status check) +├─── test-unit ───────┤ +├─── build ───────────┘ +│ +├─── build-multi (matrix: 4 platforms) +│ +└─── integration-test (conditional: main branch or label) +``` + +### Job Details + +#### Parallel CI Jobs (No Docker Required) + +| Job | Task Equivalent | Duration | Description | +|-----|----------------|----------|-------------| +| `format-check` | `task fmt-check` | ~10s | Validates code formatting with gofmt and goimports | +| `tidy-check` | `task tidy-check` | ~15s | Checks go.mod and go.sum are tidy | +| `lint` | `task lint` | ~45s | Runs golangci-lint, staticcheck, and go vet | +| `test-unit` | `task test-unit` | ~30s | Unit tests with race detector, uploads coverage | +| `build` | `task build` | ~20s | Builds for current platform, uploads artifact | + +#### Matrix Build Job + +| Job | Platforms | Description | +|-----|-----------|-------------| +| `build-multi` | linux-amd64, linux-arm64, darwin-amd64, darwin-arm64 | Cross-platform builds in parallel | + +#### Status Check Job + +| Job | Dependencies | Description | +|-----|-------------|-------------| +| `ci-complete` | All parallel jobs | Provides single PR status check | + +#### Integration Test Job + +| Job | When | Docker | Description | +|-----|------|--------|-------------| +| `integration-test` | main branch or label | ✅ Yes | Runs `task test-integration-go` | + +### Triggering Integration Tests on PRs + +To run integration tests on a pull request, add the `run-integration-tests` label: + +```bash +# Via GitHub CLI +gh pr edit --add-label "run-integration-tests" + +# Via GitHub UI +Add label: run-integration-tests +``` + +### Local Testing + +Run the same checks locally: + +```bash +# Fast CI checks (no Docker) +task ci + +# Quick feedback (no race detector) +task ci-quick + +# Full CI with integration tests (requires Docker) +task ci-full + +# Individual checks +task fmt-check +task tidy-check +task lint +task test-unit +task build +``` + +### Performance Comparison + +| Approach | Duration | Parallelization | +|----------|----------|----------------| +| Sequential (`task ci`) | ~2min | ❌ No | +| GitHub Actions (parallel) | ~45s | ✅ Yes (5 jobs) | + +### Workflow Features + +✓ **Parallel Execution** - All CI checks run simultaneously +✓ **Fast Feedback** - Get results in ~45s instead of ~2min +✓ **Granular Status** - See which specific check failed +✓ **Artifact Uploads** - Build artifacts and coverage reports saved +✓ **Conditional Integration Tests** - Only run when needed +✓ **Go Caching** - Dependencies cached between runs +✓ **Multi-platform Builds** - Cross-compile for 4 platforms in parallel + +### Codecov Integration + +The workflow uploads coverage reports to Codecov: + +- **Unit tests**: `coverage-unit.out` → flag: `unittests` +- **Integration tests**: `coverage-integration.out` → flag: `integration` + +**Note:** Requires `CODECOV_TOKEN` secret to be configured in repository settings. + +### Customization + +#### Change Go Version + +Edit the environment variable in `.github/workflows/ci.yml`: + +```yaml +env: + GO_VERSION: '1.21' # Change this +``` + +#### Skip Integration Tests + +Integration tests are automatically skipped on PRs unless: +- The PR has the `run-integration-tests` label +- The push is to main/master branch + +#### Adjust Parallel Jobs + +To add/remove jobs from the `ci-complete` dependency list: + +```yaml +ci-complete: + needs: + - format-check + - tidy-check + - lint + - test-unit + - build + # Add new jobs here +``` + +## Workflow Best Practices + +1. **All CI checks must pass** - The `ci-complete` job provides a single status check +2. **Integration tests optional on PRs** - Use label to run when needed +3. **Coverage uploaded automatically** - View reports on Codecov +4. **Artifacts retained for 7 days** - Download builds from GitHub Actions UI +5. **Test locally first** - Run `task ci` before pushing + +## Troubleshooting + +### Job Fails: "goimports not found" + +The `format-check` job installs goimports automatically. If it fails, the Go tools cache may be corrupted. + +**Solution:** Re-run the job or clear the cache. + +### Job Fails: "golangci-lint not found" + +The `lint` job runs `task install-tools` to install linters. If it fails: + +**Solution:** Check that `task install-tools` works locally. + +### Integration Tests Skipped on PR + +Integration tests only run when: +- On main/master branch, OR +- PR has `run-integration-tests` label + +**Solution:** Add the label to your PR. + +### Coverage Upload Fails + +Requires `CODECOV_TOKEN` secret. + +**Solution:** Add the token in repository settings: +1. Go to repository Settings → Secrets and variables → Actions +2. Add new secret: `CODECOV_TOKEN` +3. Get token from https://codecov.io + +### All Jobs Pending + +GitHub Actions may be queueing jobs. + +**Solution:** Wait for runners to become available, or check GitHub Actions status page. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..fd15c92 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,252 @@ +name: CI + +on: + push: + branches: [ main, master ] + pull_request: + branches: [ main, master ] + +env: + GO_VERSION: '1.21' + +jobs: + # Format check - runs in parallel + format-check: + name: Format Check + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up Go + uses: actions/setup-go@v5 + with: + go-version: ${{ env.GO_VERSION }} + cache: true + + - name: Install Task + uses: arduino/setup-task@v1 + with: + version: 3.x + repo-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Install goimports + run: go install golang.org/x/tools/cmd/goimports@latest + + - name: Check formatting + run: task fmt-check + + # Tidy check - runs in parallel + tidy-check: + name: Go Mod Tidy Check + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up Go + uses: actions/setup-go@v5 + with: + go-version: ${{ env.GO_VERSION }} + cache: true + + - name: Install Task + uses: arduino/setup-task@v1 + with: + version: 3.x + repo-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Download dependencies + run: task deps + + - name: Check go.mod tidiness + run: task tidy-check + + # Lint - runs in parallel + lint: + name: Lint + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up Go + uses: actions/setup-go@v5 + with: + go-version: ${{ env.GO_VERSION }} + cache: true + + - name: Install Task + uses: arduino/setup-task@v1 + with: + version: 3.x + repo-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Install linting tools + run: task install-tools + + - name: Download dependencies + run: task deps + + - name: Run linters + run: task lint + + # Unit tests - runs in parallel + test-unit: + name: Unit Tests + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up Go + uses: actions/setup-go@v5 + with: + go-version: ${{ env.GO_VERSION }} + cache: true + + - name: Install Task + uses: arduino/setup-task@v1 + with: + version: 3.x + repo-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Download dependencies + run: task deps + + - name: Run unit tests + run: task test-unit + + - name: Upload unit test coverage + uses: codecov/codecov-action@v4 + if: always() + with: + file: ./coverage-unit.out + flags: unittests + name: codecov-unit + token: ${{ secrets.CODECOV_TOKEN }} + + # Build - runs in parallel + build: + name: Build + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up Go + uses: actions/setup-go@v5 + with: + go-version: ${{ env.GO_VERSION }} + cache: true + + - name: Install Task + uses: arduino/setup-task@v1 + with: + version: 3.x + repo-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Download dependencies + run: task deps + + - name: Build for current platform + run: task build + + - name: Upload build artifact + uses: actions/upload-artifact@v4 + with: + name: goblet-server + path: build/goblet-server + retention-days: 7 + + # Multi-platform builds - runs in parallel, separate from main build + build-multi: + name: Build ${{ matrix.platform }} + runs-on: ubuntu-latest + strategy: + matrix: + platform: + - linux-amd64 + - linux-arm64 + - darwin-amd64 + - darwin-arm64 + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up Go + uses: actions/setup-go@v5 + with: + go-version: ${{ env.GO_VERSION }} + cache: true + + - name: Install Task + uses: arduino/setup-task@v1 + with: + version: 3.x + repo-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Build ${{ matrix.platform }} + run: task build-${{ matrix.platform }} + + - name: Upload artifacts + uses: actions/upload-artifact@v4 + with: + name: goblet-${{ matrix.platform }} + path: build/goblet-server-${{ matrix.platform }}* + retention-days: 7 + + # CI Status Check - depends on all parallel jobs + ci-complete: + name: CI Complete + runs-on: ubuntu-latest + needs: + - format-check + - tidy-check + - lint + - test-unit + - build + steps: + - name: CI Pipeline Complete + run: echo "✓ All CI checks passed!" + + # Integration tests - separate workflow that requires Docker + integration-test: + name: Integration Tests + runs-on: ubuntu-latest + # Only run integration tests on main branch or when explicitly requested + if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master' || contains(github.event.pull_request.labels.*.name, 'run-integration-tests') + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up Go + uses: actions/setup-go@v5 + with: + go-version: ${{ env.GO_VERSION }} + cache: true + + - name: Install Task + uses: arduino/setup-task@v1 + with: + version: 3.x + repo-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Download dependencies + run: task deps + + - name: Run Go integration tests + run: task test-integration-go + + - name: Stop Docker services + if: always() + run: task docker-test-down + + - name: Upload integration test coverage + uses: codecov/codecov-action@v4 + if: always() + with: + file: ./coverage-integration.out + flags: integration + name: codecov-integration + token: ${{ secrets.CODECOV_TOKEN }} diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 0000000..fb88b22 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,51 @@ +name: Release + +on: + push: + tags: + - 'v*' # Trigger on version tags like v1.0.0, v2.1.3, etc. + +permissions: + contents: write # Required to create releases and upload assets + packages: write # Required to push Docker images to GHCR + id-token: write # Required for OIDC token + +jobs: + goreleaser: + name: Release with GoReleaser + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v4 + with: + fetch-depth: 0 # Fetch all history for proper changelog generation + + - name: Set up Go + uses: actions/setup-go@v5 + with: + go-version-file: go.mod + cache: true + + - name: Set up QEMU + uses: docker/setup-qemu-action@v3 + with: + platforms: arm64 + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + + - name: Log in to GitHub Container Registry + uses: docker/login-action@v3 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Run GoReleaser + uses: goreleaser/goreleaser-action@v6 + with: + distribution: goreleaser + version: latest + args: release --clean + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.gitignore b/.gitignore index 0992301..08d0471 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,51 @@ +# Binaries /goblet-server/goblet-server +goblet-server +*.exe +*.exe~ +*.dll +*.so +*.dylib + +# Build artifacts +/build/ +/dist/ + +# Test binary, built with `go test -c` +*.test + +# Output of the go coverage tool +*.out +coverage.html +coverage.txt + +# Dependency directories +vendor/ + +# Go workspace file +go.work + +# Environment files +.env +*.local + +# IDE +.vscode/ +.idea/ +*.swp +*.swo +*~ + +# OS +.DS_Store +Thumbs.db + +# Bazel (legacy) /bazel-* + +# Temporary files +tmp/ +*.tmp + +# Logs +*.log diff --git a/.golangci.yml b/.golangci.yml new file mode 100644 index 0000000..cc6fc38 --- /dev/null +++ b/.golangci.yml @@ -0,0 +1,79 @@ +run: + timeout: 5m + tests: true + modules-download-mode: readonly + +linters: + enable: + - errcheck + - gosimple + - govet + - ineffassign + - staticcheck + - typecheck + - unused + - gofmt + - goimports + - misspell + - unconvert + - unparam + - goconst + - gocyclo + - godot + - gosec + - gocritic + +linters-settings: + gocyclo: + min-complexity: 15 + goconst: + min-len: 3 + min-occurrences: 3 + misspell: + locale: US + godot: + scope: declarations + capital: true + staticcheck: + checks: ["all", "-SA1019"] + +issues: + exclude-rules: + - path: _test\.go + linters: + - gocyclo + - gosec + - goconst + - path: testing/ + linters: + - gosec + # Exclude deprecated import warnings until dependencies are updated + - text: "SA1019" + linters: + - staticcheck + # Exclude ifElseChain style suggestions (often less readable as switch) + - linters: + - gocritic + text: "ifElseChain" + # Exclude exitAfterDefer in main function (acceptable for startup code) + - path: goblet-server/main\.go + linters: + - gocritic + text: "exitAfterDefer" + # Exclude high complexity warnings for known complex functions + - path: git_protocol_v2_handler\.go + linters: + - gocyclo + text: "handleV2Command" + - path: goblet-server/main\.go + linters: + - gocyclo + text: "main" + + max-issues-per-linter: 0 + max-same-issues: 0 + +output: + formats: colored-line-number + print-issued-lines: true + print-linter-name: true diff --git a/.goreleaser.yml b/.goreleaser.yml new file mode 100644 index 0000000..1719023 --- /dev/null +++ b/.goreleaser.yml @@ -0,0 +1,180 @@ +# GoReleaser configuration for Goblet +# Documentation: https://goreleaser.com + +version: 2 + +# Before hooks - run before building +before: + hooks: + - go mod tidy + - go mod download + +# Build configuration +builds: + - id: goblet-server + main: ./goblet-server + binary: goblet-server + env: + - CGO_ENABLED=0 + goos: + - linux + - darwin + - windows + goarch: + - amd64 + - arm64 + # Ignore arm64 on windows (not commonly used) + ignore: + - goos: windows + goarch: arm64 + ldflags: + - -s -w + - -X main.Version={{.Version}} + - -X main.Commit={{.Commit}} + - -X main.Date={{.Date}} + - -X main.BuiltBy=goreleaser + flags: + - -trimpath + +# Archive configuration +archives: + - id: goblet + format: tar.gz + format_overrides: + - goos: windows + format: zip + name_template: >- + {{ .ProjectName }}_ + {{- .Version }}_ + {{- .Os }}_ + {{- .Arch }} + files: + - LICENSE* + - README* + - CHANGELOG* + +# Checksum configuration +checksum: + name_template: 'checksums.txt' + algorithm: sha256 + +# Snapshot builds (non-tagged commits) +snapshot: + version_template: "{{ incpatch .Version }}-next" + +# Changelog configuration +changelog: + use: github + sort: asc + abbrev: 7 + groups: + - title: Features + regexp: '^.*?feat(\([[:word:]]+\))??!?:.+$' + order: 0 + - title: 'Bug Fixes' + regexp: '^.*?fix(\([[:word:]]+\))??!?:.+$' + order: 1 + - title: 'Performance Improvements' + regexp: '^.*?perf(\([[:word:]]+\))??!?:.+$' + order: 2 + - title: 'Documentation' + regexp: '^.*?docs(\([[:word:]]+\))??!?:.+$' + order: 3 + - title: 'Tests' + regexp: '^.*?test(\([[:word:]]+\))??!?:.+$' + order: 4 + - title: Others + order: 999 + filters: + exclude: + - '^chore:' + - '^ci:' + - '^style:' + - '^refactor:' + - Merge pull request + - Merge branch + +# GitHub Release configuration +release: + github: + owner: jrepp + name: github-cache-daemon + draft: false + prerelease: auto # Automatically detect pre-releases based on semver + mode: replace + header: | + ## Goblet {{ .Tag }} ({{ .Date }}) + + Welcome to this new release of Goblet! + + footer: | + ## Docker Images + + Multi-arch Docker images are available: + + ```bash + docker pull ghcr.io/jrepp/goblet-server:{{ .Tag }} + docker pull ghcr.io/jrepp/goblet-server:latest + ``` + + ## Verification + + Verify the integrity of downloaded binaries using the checksums file: + + ```bash + sha256sum -c checksums.txt + ``` + + **Full Changelog**: https://github.com/jrepp/github-cache-daemon/compare/{{ .PreviousTag }}...{{ .Tag }} + +# Docker images +dockers: + - image_templates: + - "ghcr.io/jrepp/goblet-server:{{ .Tag }}-amd64" + - "ghcr.io/jrepp/goblet-server:latest-amd64" + use: buildx + dockerfile: Dockerfile + build_flag_templates: + - "--platform=linux/amd64" + - "--label=org.opencontainers.image.created={{.Date}}" + - "--label=org.opencontainers.image.title={{.ProjectName}}" + - "--label=org.opencontainers.image.revision={{.FullCommit}}" + - "--label=org.opencontainers.image.version={{.Version}}" + - "--label=org.opencontainers.image.source={{.GitURL}}" + extra_files: + - LICENSE + + - image_templates: + - "ghcr.io/jrepp/goblet-server:{{ .Tag }}-arm64" + - "ghcr.io/jrepp/goblet-server:latest-arm64" + use: buildx + dockerfile: Dockerfile + build_flag_templates: + - "--platform=linux/arm64" + - "--label=org.opencontainers.image.created={{.Date}}" + - "--label=org.opencontainers.image.title={{.ProjectName}}" + - "--label=org.opencontainers.image.revision={{.FullCommit}}" + - "--label=org.opencontainers.image.version={{.Version}}" + - "--label=org.opencontainers.image.source={{.GitURL}}" + goarch: arm64 + extra_files: + - LICENSE + +# Docker manifests for multi-arch images +docker_manifests: + - name_template: ghcr.io/jrepp/goblet-server:{{ .Tag }} + image_templates: + - ghcr.io/jrepp/goblet-server:{{ .Tag }}-amd64 + - ghcr.io/jrepp/goblet-server:{{ .Tag }}-arm64 + + - name_template: ghcr.io/jrepp/goblet-server:latest + image_templates: + - ghcr.io/jrepp/goblet-server:latest-amd64 + - ghcr.io/jrepp/goblet-server:latest-arm64 + +# Announce releases (optional - configure as needed) +# announce: +# skip: '{{gt .Patch 0}}' # Only announce major and minor releases +# discord: +# enabled: true +# message_template: 'Goblet {{ .Tag }} is out! Check it out at {{ .ReleaseURL }}' diff --git a/BUILD b/BUILD deleted file mode 100644 index 3ded448..0000000 --- a/BUILD +++ /dev/null @@ -1,31 +0,0 @@ -load("@io_bazel_rules_go//go:def.bzl", "go_library") -load("@bazel_gazelle//:def.bzl", "gazelle") - -# gazelle:prefix github.com/google/goblet -# gazelle:build_file_name BUILD -gazelle(name = "gazelle") - -go_library( - name = "go_default_library", - srcs = [ - "git_protocol_v2_handler.go", - "goblet.go", - "http_proxy_server.go", - "io.go", - "managed_repository.go", - "reporting.go", - ], - importpath = "github.com/google/goblet", - visibility = ["//visibility:public"], - deps = [ - "@com_github_go_git_go_git_v5//:go_default_library", - "@com_github_go_git_go_git_v5//plumbing:go_default_library", - "@com_github_google_gitprotocolio//:go_default_library", - "@com_github_grpc_ecosystem_grpc_gateway//runtime:go_default_library", - "@io_opencensus_go//stats:go_default_library", - "@io_opencensus_go//tag:go_default_library", - "@org_golang_google_grpc//codes:go_default_library", - "@org_golang_google_grpc//status:go_default_library", - "@org_golang_x_oauth2//:go_default_library", - ], -) diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..6cbd3f1 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,47 @@ +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [Unreleased] + +### Added +- GitHub Actions automated release pipeline +- Multi-platform binary builds (Linux, macOS, Windows) +- Automated release notes generation +- SHA256 checksums for all release binaries +- Docker multi-arch image builds and publishing +- Comprehensive offline mode documentation with testing guides + +### Changed +- Enhanced README with offline mode configuration, monitoring, and testing sections + +## Template for New Releases + +When creating a new release, copy the following template and fill in the details: + +```markdown +## [X.Y.Z] - YYYY-MM-DD + +### Added +- New features and capabilities + +### Changed +- Changes to existing functionality + +### Deprecated +- Features that will be removed in future releases + +### Removed +- Features that have been removed + +### Fixed +- Bug fixes + +### Security +- Security-related changes and fixes +``` + +[Unreleased]: https://github.com/jrepp/goblet/compare/main...HEAD diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..1ae4f69 --- /dev/null +++ b/Dockerfile @@ -0,0 +1,27 @@ +# Dockerfile for x86_64 (amd64) architecture +# Build the binary first with: task build-linux-amd64 +FROM alpine:latest + +# Install runtime dependencies +RUN apk --no-cache add ca-certificates git + +WORKDIR / + +# Copy the pre-built binary +# Default to amd64, override with --build-arg ARCH=arm64 for ARM +ARG ARCH=amd64 +COPY build/goblet-server-linux-${ARCH} /goblet-server + +# Ensure binary is executable +RUN chmod +x /goblet-server + +# Create cache directory +RUN mkdir -p /cache + +# Health check +HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \ + CMD wget --no-verbose --tries=1 --spider http://localhost:8080/healthz || exit 1 + +EXPOSE 8080 + +ENTRYPOINT ["/goblet-server"] diff --git a/Dockerfile.build b/Dockerfile.build new file mode 100644 index 0000000..061f113 --- /dev/null +++ b/Dockerfile.build @@ -0,0 +1,43 @@ +# Multi-stage Dockerfile that builds from source +# Use this if you want to build inside Docker +FROM golang:1.24-alpine AS builder + +# Install build dependencies +RUN apk add --no-cache git ca-certificates make + +WORKDIR /build + +# Copy go mod files first for better caching +COPY go.mod go.sum ./ +RUN go mod download + +# Copy source code +COPY . . + +# Build the application +ARG TARGETOS=linux +ARG TARGETARCH=amd64 +RUN CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} \ + go build -ldflags="-w -s" -trimpath -o goblet-server ./goblet-server + +# Runtime stage +FROM alpine:latest + +# Install runtime dependencies +RUN apk --no-cache add ca-certificates git + +WORKDIR / + +# Copy the binary from builder +COPY --from=builder /build/goblet-server /goblet-server + +# Create cache directory +RUN mkdir -p /cache + +# Health check +HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \ + CMD wget --no-verbose --tries=1 --spider http://localhost:8080/healthz || exit 1 + +EXPOSE 8080 + +ENTRYPOINT ["/goblet-server"] diff --git a/OFFLINE_MODE_PLAN.md b/OFFLINE_MODE_PLAN.md new file mode 100644 index 0000000..eee3575 --- /dev/null +++ b/OFFLINE_MODE_PLAN.md @@ -0,0 +1,580 @@ +# Implementation Plan: Offline ls-refs Support + +## Overview +Enable Goblet to serve ls-refs requests from cache when the upstream server is unavailable, making the proxy resilient to upstream failures. + +## Current Limitation +From `README.md:28-31`: +> Note that Goblet forwards the ls-refs traffic to the upstream server. If the upstream server is down, Goblet is effectively down. Technically, we can modify Goblet to serve even if the upstream is down, but the current implementation doesn't do such thing. + +## Goals +1. ✅ Cache ls-refs responses for offline serving +2. ✅ Serve from cache when upstream is unavailable +3. ✅ Add configuration to enable/disable upstream (for testing) +4. ✅ Maintain backward compatibility +5. ✅ Provide clear metrics and health status + +--- + +## Architecture Changes + +### 1. Configuration Extension (`ServerConfig`) + +**File**: `server_config.go` or inline in relevant files + +Add new configuration options: + +```go +type ServerConfig struct { + // ... existing fields ... + + // Offline mode configuration + EnableOfflineMode bool // Enable ls-refs cache fallback + UpstreamEnabled bool // For testing: disable upstream completely + LsRefsCacheTTL time.Duration // How long to trust cached ls-refs (default: 5m) + LsRefsCachePath string // Path to persist ls-refs cache (optional) +} +``` + +**Default values**: +- `EnableOfflineMode`: `true` (enable resilience) +- `UpstreamEnabled`: `true` (production default) +- `LsRefsCacheTTL`: `5 * time.Minute` +- `LsRefsCachePath`: `{LocalDiskCacheRoot}/.ls-refs-cache` + +### 2. ls-refs Cache Structure + +**File**: `ls_refs_cache.go` (new file) + +```go +type LsRefsCache struct { + mu sync.RWMutex + entries map[string]*LsRefsCacheEntry + diskPath string +} + +type LsRefsCacheEntry struct { + RepoPath string // Repository identifier + Refs map[string]string // ref name -> commit hash + SymRefs map[string]string // symbolic refs (HEAD -> refs/heads/main) + Timestamp time.Time // When cached + RawResponse []byte // Original protocol response + UpstreamURL string // Source upstream +} +``` + +**Operations**: +- `Get(repoPath string) (*LsRefsCacheEntry, bool)` +- `Set(repoPath string, entry *LsRefsCacheEntry) error` +- `IsStale(entry *LsRefsCacheEntry, ttl time.Duration) bool` +- `LoadFromDisk() error` +- `SaveToDisk() error` +- `Invalidate(repoPath string)` + +### 3. Modified Request Flow + +**File**: `git_protocol_v2_handler.go` + +Current flow: +``` +ls-refs request + ↓ +lsRefsUpstream() ──[error]──> return error to client + ↓ +return upstream response +``` + +New flow: +``` +ls-refs request + ↓ +Check if UpstreamEnabled == false (test mode) + ↓ [false] + Serve from cache or error + + ↓ [true] +Try lsRefsUpstream() + ↓ + ├─ [success] ──> Cache response ──> Return to client + │ + └─ [error] + ↓ + Check EnableOfflineMode + ↓ + ├─ [false] ──> Return error (current behavior) + │ + └─ [true] + ↓ + Check cache for valid entry + ↓ + ├─ [found & fresh] ──> Serve from cache (with warning header) + ├─ [found & stale] ──> Serve from cache (with staleness warning) + └─ [not found] ──> Return error (no cached data) +``` + +--- + +## Implementation Steps + +### Phase 1: Configuration and Cache Infrastructure + +#### 1.1 Add Configuration Options +**File**: `server_config.go` or where `ServerConfig` is defined + +```go +type ServerConfig struct { + // ... existing fields ... + + // Offline mode support + EnableOfflineMode bool + UpstreamEnabled bool + LsRefsCacheTTL time.Duration + LsRefsCachePath string +} +``` + +#### 1.2 Create ls-refs Cache Manager +**File**: `ls_refs_cache.go` (new) + +Implement: +- In-memory cache with mutex protection +- Disk persistence (JSON or protobuf format) +- TTL checking +- Atomic updates + +**File format** (JSON example): +```json +{ + "github.com/user/repo": { + "timestamp": "2025-11-06T10:30:00Z", + "upstream_url": "https://github.com/user/repo", + "refs": { + "refs/heads/main": "abc123...", + "refs/heads/feature": "def456...", + "refs/tags/v1.0.0": "789abc..." + }, + "symrefs": { + "HEAD": "refs/heads/main" + }, + "raw_response": "base64-encoded-protocol-response" + } +} +``` + +#### 1.3 Initialize Cache on Server Start +**File**: `http_proxy_server.go` + +In `StartServer()` or similar: +```go +lsRefsCache, err := NewLsRefsCache(config.LsRefsCachePath) +if err != nil { + return fmt.Errorf("failed to initialize ls-refs cache: %w", err) +} +if err := lsRefsCache.LoadFromDisk(); err != nil { + log.Printf("Warning: could not load ls-refs cache: %v", err) +} +``` + +### Phase 2: Upstream Interaction Changes + +#### 2.1 Modify `lsRefsUpstream` +**File**: `managed_repository.go:129-170` + +Add caching after successful upstream response: + +```go +func (repo *managedRepository) lsRefsUpstream(command *gitprotocolio.ProtocolV2Command) (...) { + // Check if upstream is disabled (test mode) + if !repo.config.UpstreamEnabled { + return nil, status.Error(codes.Unavailable, "upstream disabled for testing") + } + + // ... existing upstream call ... + + // On success, cache the response + if repo.config.EnableOfflineMode { + entry := &LsRefsCacheEntry{ + RepoPath: repo.localDiskPath, + Refs: refs, // parsed from response + SymRefs: symrefs, + Timestamp: time.Now(), + RawResponse: rawResponse, + UpstreamURL: repo.upstreamURL.String(), + } + if err := lsRefsCache.Set(repo.localDiskPath, entry); err != nil { + log.Printf("Warning: failed to cache ls-refs: %v", err) + } + } + + return refs, rawResponse, nil +} +``` + +#### 2.2 Add Fallback Method +**File**: `managed_repository.go` (new method) + +```go +func (repo *managedRepository) lsRefsFromCache() (map[string]string, []byte, error) { + if !repo.config.EnableOfflineMode { + return nil, nil, status.Error(codes.Unavailable, "offline mode disabled") + } + + entry, found := lsRefsCache.Get(repo.localDiskPath) + if !found { + return nil, nil, status.Error(codes.NotFound, "no cached ls-refs available") + } + + // Check staleness + isStale := lsRefsCache.IsStale(entry, repo.config.LsRefsCacheTTL) + + // Optionally add warning to response + if isStale { + log.Printf("Warning: serving stale ls-refs for %s (age: %v)", + repo.localDiskPath, time.Since(entry.Timestamp)) + } + + return entry.Refs, entry.RawResponse, nil +} +``` + +#### 2.3 Update ls-refs Handler +**File**: `git_protocol_v2_handler.go:54-83` + +Modify the ls-refs handling: + +```go +case "ls-refs": + var refs map[string]string + var rawResponse []byte + var err error + + // Try upstream first + refs, rawResponse, err = repo.lsRefsUpstream(command) + + // If upstream fails, try cache fallback + if err != nil && repo.config.EnableOfflineMode { + log.Printf("Upstream ls-refs failed, attempting cache fallback: %v", err) + refs, rawResponse, err = repo.lsRefsFromCache() + if err == nil { + // Successfully served from cache + repo.config.RequestLogger(req, "ls-refs", "cache-fallback", ...) + } + } + + if err != nil { + return err // No fallback available + } + + // ... rest of existing logic ... +``` + +### Phase 3: Metrics and Observability + +#### 3.1 Add Metrics +**File**: `reporting.go` or new `metrics.go` + +Add counters/gauges: +```go +var ( + lsRefsCacheHits = /* counter */ + lsRefsCacheMisses = /* counter */ + lsRefsServedStale = /* counter */ + upstreamAvailable = /* gauge: 0 or 1 */ +) +``` + +#### 3.2 Update Health Check +**File**: `health_check.go` (if exists) or `http_proxy_server.go` + +Add to health check response: +```json +{ + "status": "healthy", + "upstream_status": "unavailable", + "offline_mode": "active", + "cached_repos": 42, + "cache_stats": { + "hits": 150, + "misses": 3, + "stale_serves": 12 + } +} +``` + +### Phase 4: Integration Testing + +#### 4.1 Test Helper: Disable Upstream +**File**: `testing/test_helpers.go` or similar + +```go +func NewTestServerWithoutUpstream(t *testing.T) *httpProxyServer { + config := &ServerConfig{ + // ... standard test config ... + EnableOfflineMode: true, + UpstreamEnabled: false, // Key: disable upstream + LsRefsCacheTTL: 5 * time.Minute, + } + return newServer(config) +} +``` + +#### 4.2 Test: Offline Mode with Warm Cache +**File**: `testing/offline_integration_test.go` (new) + +```go +func TestLsRefsOfflineWithCache(t *testing.T) { + server := NewTestServer(t) + + // Step 1: Populate cache with real upstream + client := git.NewClient(server.URL) + refs1, err := client.LsRefs("github.com/user/repo") + require.NoError(t, err) + + // Step 2: Disable upstream + server.config.UpstreamEnabled = false + + // Step 3: Verify cache serves refs + refs2, err := client.LsRefs("github.com/user/repo") + require.NoError(t, err) + assert.Equal(t, refs1, refs2, "cached refs should match") +} +``` + +#### 4.3 Test: Offline Mode with Cold Cache +**File**: `testing/offline_integration_test.go` + +```go +func TestLsRefsOfflineWithoutCache(t *testing.T) { + server := NewTestServerWithoutUpstream(t) + + client := git.NewClient(server.URL) + _, err := client.LsRefs("github.com/user/repo") + + // Should fail: no cache, no upstream + assert.Error(t, err) + assert.Contains(t, err.Error(), "no cached ls-refs available") +} +``` + +#### 4.4 Test: Stale Cache Serving +**File**: `testing/offline_integration_test.go` + +```go +func TestLsRefsStaleCache(t *testing.T) { + server := NewTestServer(t) + server.config.LsRefsCacheTTL = 1 * time.Second + + // Populate cache + client := git.NewClient(server.URL) + _, err := client.LsRefs("github.com/user/repo") + require.NoError(t, err) + + // Wait for cache to become stale + time.Sleep(2 * time.Second) + + // Disable upstream + server.config.UpstreamEnabled = false + + // Should still serve from stale cache + _, err = client.LsRefs("github.com/user/repo") + require.NoError(t, err) + + // Verify metrics show stale serve + assert.Equal(t, 1, server.metrics.LsRefsServedStale) +} +``` + +#### 4.5 Test: Upstream Recovery +**File**: `testing/offline_integration_test.go` + +```go +func TestLsRefsUpstreamRecovery(t *testing.T) { + server := NewTestServer(t) + + // Populate cache + client := git.NewClient(server.URL) + refs1, err := client.LsRefs("github.com/user/repo") + require.NoError(t, err) + + // Simulate upstream failure + server.config.UpstreamEnabled = false + refs2, err := client.LsRefs("github.com/user/repo") + require.NoError(t, err) + assert.Equal(t, refs1, refs2) + + // Simulate upstream recovery + server.config.UpstreamEnabled = true + updateUpstreamRefs(t, "github.com/user/repo", "new-commit") + + // Should fetch fresh refs + refs3, err := client.LsRefs("github.com/user/repo") + require.NoError(t, err) + assert.NotEqual(t, refs2, refs3, "refs should be updated") +} +``` + +### Phase 5: Documentation + +#### 5.1 Update README.md +**File**: `README.md:28-31` + +Replace limitation note with: + +```markdown +### Offline Mode and Resilience + +Goblet can now serve ls-refs requests from cache when the upstream server is unavailable: + +- **Automatic fallback**: When upstream is down, Goblet serves cached ref listings +- **Configurable TTL**: Control cache freshness (default: 5 minutes) +- **Testing support**: Disable upstream connectivity for integration tests +- **Metrics**: Track cache hits, misses, and stale serves + +Configure offline mode: +```go +config := &ServerConfig{ + EnableOfflineMode: true, // Enable cache fallback + LsRefsCacheTTL: 5 * time.Minute, // Cache freshness + LsRefsCachePath: "/path/to/cache", +} +``` + +For testing without upstream: +```go +config.UpstreamEnabled = false // Disable all upstream calls +``` +``` + +#### 5.2 Add Configuration Guide +**File**: `docs/CONFIGURATION.md` (if exists) or add section to README + +Document all new configuration options with examples. + +--- + +## Testing Strategy + +### Unit Tests +- `ls_refs_cache_test.go`: Cache operations (Get, Set, TTL, persistence) +- `managed_repository_test.go`: Cache fallback logic +- Mock upstream responses + +### Integration Tests +1. ✅ **Warm cache offline**: Upstream populated cache, then disabled +2. ✅ **Cold cache offline**: No cache, upstream disabled (should fail) +3. ✅ **Stale cache serving**: Expired cache still serves when upstream down +4. ✅ **Upstream recovery**: Cache updates when upstream comes back +5. ✅ **Concurrent access**: Multiple clients with cache fallback +6. ✅ **Cache persistence**: Server restart preserves cache + +### Manual Testing +- Deploy with upstream Github down +- Verify git clone/fetch works from cache +- Monitor metrics and logs +- Test cache invalidation + +--- + +## Rollout Strategy + +### Phase 1: Feature Flag (Week 1) +- Deploy with `EnableOfflineMode: false` (disabled) +- Monitor cache population +- No behavior change + +### Phase 2: Canary (Week 2) +- Enable for 10% of traffic +- Monitor error rates, cache hit ratios +- Compare latency: cache vs upstream + +### Phase 3: Full Rollout (Week 3+) +- Enable for all traffic +- Update documentation +- Announce feature + +--- + +## Risks and Mitigations + +### Risk 1: Stale Cache Serving Wrong Refs +**Impact**: Clients fetch outdated commits + +**Mitigation**: +- Conservative default TTL (5 minutes) +- Log warnings for stale serves +- Metric tracking for monitoring + +### Risk 2: Cache Size Growth +**Impact**: Disk space exhaustion + +**Mitigation**: +- LRU eviction policy +- Configurable max cache size +- Periodic cleanup job + +### Risk 3: Upstream Never Recovers +**Impact**: Perpetually stale cache + +**Mitigation**: +- Health check reports upstream status +- Alert on prolonged upstream unavailability +- Manual cache invalidation API + +### Risk 4: Race Conditions +**Impact**: Concurrent requests corrupt cache + +**Mitigation**: +- RWMutex protection for all cache operations +- Atomic file writes for disk persistence +- Integration tests for concurrency + +--- + +## Success Metrics + +1. **Availability**: Proxy remains operational during upstream outages +2. **Cache Hit Ratio**: >80% of ls-refs served from cache (eventually) +3. **Latency**: Cache-served ls-refs <10ms (vs ~100ms upstream) +4. **Error Rate**: Zero increase in client errors during upstream outages +5. **Test Coverage**: >90% for new code + +--- + +## Future Enhancements + +1. **Smart Cache Invalidation**: Webhook-based cache updates +2. **Multi-Tier Caching**: Redis/Memcached for distributed deployments +3. **Partial Offline Mode**: Serve cached refs, but fail fetch if objects missing +4. **Circuit Breaker**: Automatically detect upstream failure patterns +5. **Admin API**: Manual cache inspection and invalidation endpoints + +--- + +## Files to Modify/Create + +### New Files +- `ls_refs_cache.go`: Cache manager implementation +- `ls_refs_cache_test.go`: Unit tests +- `testing/offline_integration_test.go`: Integration tests +- `OFFLINE_MODE_PLAN.md`: This document + +### Modified Files +- `server_config.go`: Add configuration options +- `managed_repository.go`: Add cache fallback methods +- `git_protocol_v2_handler.go`: Update ls-refs handling +- `http_proxy_server.go`: Initialize cache on startup +- `health_check.go`: Add cache status +- `reporting.go`: Add offline mode metrics +- `README.md`: Update documentation + +--- + +## Timeline Estimate + +- **Phase 1** (Config + Cache Infrastructure): 2-3 days +- **Phase 2** (Upstream Integration): 2-3 days +- **Phase 3** (Metrics + Observability): 1-2 days +- **Phase 4** (Integration Testing): 2-3 days +- **Phase 5** (Documentation): 1 day + +**Total**: ~8-12 days for full implementation and testing diff --git a/PLAN_REVIEW.md b/PLAN_REVIEW.md new file mode 100644 index 0000000..6c7a869 --- /dev/null +++ b/PLAN_REVIEW.md @@ -0,0 +1,350 @@ +# Staff Engineer Review: Offline ls-refs Implementation Plan + +## Executive Summary +**Recommendation**: Simplify the implementation significantly. We're over-engineering the solution. + +**Key insight**: We already have a local git repository on disk that IS the cache. We don't need a separate ls-refs cache layer. + +--- + +## Critical Issues with Current Plan + +### 1. Over-Engineering: Unnecessary Cache Layer ❌ + +**Problem**: The plan introduces a new cache layer (`LsRefsCache`) with: +- In-memory storage (`map[string]*LsRefsCacheEntry`) +- Disk persistence (JSON files) +- TTL management +- Cache invalidation logic +- ~300+ lines of new code + +**Why this is wrong**: We already have the refs cached in the local git repository at `{LocalDiskCacheRoot}/{host}/{path}`. The local git repo already maintains refs in `.git/refs/` and `.git/packed-refs`. + +**Evidence**: +- `managed_repository.go:251-268` already reads refs from local repo using `go-git` library +- `hasAnyUpdate()` uses `git.PlainOpen()` and `g.Reference()` to read refs +- Local repo is kept up-to-date by `fetchUpstream()` (already exists) + +### 2. Testing Complexity ❌ + +**Current plan requires**: +- Mock cache state +- Manage TTL expiration +- Test cache persistence/loading +- Handle cache corruption +- Test race conditions in cache access + +**This is 5x more test surface area than needed.** + +### 3. Configuration Bloat ❌ + +Four new config options: +```go +EnableOfflineMode bool // Do we need this? +UpstreamEnabled bool // OK for testing +LsRefsCacheTTL time.Duration // Unnecessary if using local repo +LsRefsCachePath string // Unnecessary +``` + +**We only need one**: `UpstreamEnabled` for testing. + +--- + +## Simplified Architecture + +### Core Insight +**The local git repository IS the cache.** We just need to read from it when upstream is unavailable. + +### Implementation (3 simple changes) + +#### Change 1: Add `lsRefsLocal()` method +**File**: `managed_repository.go` (new method, ~30 lines) + +```go +func (r *managedRepository) lsRefsLocal(command *gitprotocolio.ProtocolV2Command) (map[string]plumbing.Hash, []byte, error) { + // Open local git repo + g, err := git.PlainOpen(r.localDiskPath) + if err != nil { + return nil, nil, status.Errorf(codes.Unavailable, "local repo not available: %v", err) + } + + // List all refs + refs, err := g.References() + if err != nil { + return nil, nil, status.Errorf(codes.Internal, "failed to read refs: %v", err) + } + + // Convert to map and protocol response + refMap := make(map[string]plumbing.Hash) + var buf bytes.Buffer + + refs.ForEach(func(ref *plumbing.Reference) error { + // Apply ls-refs filters from command (ref-prefix, etc.) + if shouldIncludeRef(ref, command) { + refMap[ref.Name().String()] = ref.Hash() + fmt.Fprintf(&buf, "%s %s\n", ref.Hash(), ref.Name()) + } + return nil + }) + + // Add symrefs (HEAD -> refs/heads/main) + head, _ := g.Head() + if head != nil { + fmt.Fprintf(&buf, "symref-target:%s %s\n", head.Name(), "HEAD") + } + + buf.WriteString("0000") // Protocol delimiter + return refMap, buf.Bytes(), nil +} +``` + +#### Change 2: Update `handleV2Command` for ls-refs +**File**: `git_protocol_v2_handler.go:54-83` (modify existing) + +```go +case "ls-refs": + var refs map[string]plumbing.Hash + var rawResponse []byte + var err error + var source string + + // Try upstream first (if enabled) + if repo.config.UpstreamEnabled { + refs, rawResponse, err = repo.lsRefsUpstream(command) + source = "upstream" + + if err != nil { + // Upstream failed, try local fallback + log.Printf("Upstream ls-refs failed (%v), falling back to local", err) + refs, rawResponse, err = repo.lsRefsLocal(command) + source = "local-fallback" + } + } else { + // Testing mode: serve from local only + refs, rawResponse, err = repo.lsRefsLocal(command) + source = "local" + } + + if err != nil { + return err + } + + // Log staleness warning if serving from local + if source != "upstream" && time.Since(repo.lastUpdate) > 5*time.Minute { + log.Printf("Warning: serving stale ls-refs for %s (last update: %v ago)", + repo.localDiskPath, time.Since(repo.lastUpdate)) + } + + // ... rest of existing logic (hasAnyUpdate check, etc.) + repo.config.RequestLogger(req, "ls-refs", source, ...) +``` + +#### Change 3: Add single config option +**File**: `server_config.go` or inline + +```go +type ServerConfig struct { + // ... existing fields ... + + // Testing: set false to disable all upstream calls + UpstreamEnabled bool // default: true +} +``` + +**That's it.** Three changes, ~60 lines of code total. + +--- + +## Why This is Better + +### 1. Simplicity ✅ +- **No new data structures**: Uses existing local git repo +- **No cache management**: Git handles ref storage +- **No TTL logic**: Just check `lastUpdate` timestamp (already exists) +- **No persistence code**: Git already persists refs to disk + +### 2. Testability ✅ + +**Unit tests** (simple mocks): +```go +func TestLsRefsLocal(t *testing.T) { + // Create test git repo + repo := createTestRepo(t) + + // Write some refs + writeRef(repo, "refs/heads/main", "abc123") + writeRef(repo, "refs/tags/v1.0", "def456") + + // Read via lsRefsLocal + mr := &managedRepository{localDiskPath: repo.Path()} + refs, _, err := mr.lsRefsLocal(nil) + + require.NoError(t, err) + assert.Equal(t, "abc123", refs["refs/heads/main"]) + assert.Equal(t, "def456", refs["refs/tags/v1.0"]) +} +``` + +**Integration tests** (no mocking needed): +```go +func TestLsRefsOfflineMode(t *testing.T) { + // Step 1: Normal operation (populate local cache) + server := NewTestServer(t) + client := NewGitClient(server.URL) + + refs1, err := client.LsRefs("github.com/user/repo") + require.NoError(t, err) + + // Step 2: Disable upstream + server.config.UpstreamEnabled = false + + // Step 3: Should still work (serves from local) + refs2, err := client.LsRefs("github.com/user/repo") + require.NoError(t, err) + assert.Equal(t, refs1, refs2) +} + +func TestLsRefsNoLocalCache(t *testing.T) { + // Start server with upstream disabled + server := NewTestServer(t) + server.config.UpstreamEnabled = false + + client := NewGitClient(server.URL) + + // Should fail: no local cache exists + _, err := client.LsRefs("github.com/never/cached") + assert.Error(t, err) + assert.Contains(t, err.Error(), "local repo not available") +} +``` + +### 3. Maintenance ✅ +- **Fewer bugs**: Less code = fewer bugs +- **No cache invalidation bugs**: Git handles consistency +- **No cache corruption**: Git is battle-tested +- **No synchronization bugs**: We already lock `managedRepository` + +### 4. Performance ✅ +- **Fast**: Reading from local git repo is ~1-2ms +- **No extra memory**: No in-memory cache needed +- **No extra I/O**: No separate cache file writes + +--- + +## Comparison: Lines of Code + +| Component | Original Plan | Simplified | +|-----------|---------------|------------| +| Cache manager | ~150 lines | 0 | +| Cache persistence | ~80 lines | 0 | +| TTL management | ~40 lines | 0 | +| Configuration | ~20 lines | ~5 lines | +| Core logic change | ~50 lines | ~35 lines | +| Unit tests | ~200 lines | ~50 lines | +| Integration tests | ~150 lines | ~50 lines | +| **Total** | **~690 lines** | **~140 lines** | + +**5x reduction in code and complexity.** + +--- + +## What We Still Get + +✅ **Offline resilience**: Serves ls-refs when upstream is down +✅ **Testing support**: `UpstreamEnabled = false` for tests +✅ **Staleness tracking**: Use existing `lastUpdate` timestamp +✅ **Zero config**: Works out of the box, no tuning needed +✅ **Observability**: Log source (upstream/local-fallback/local) + +--- + +## What We Lose (Intentionally) + +❌ **Separate cache file**: Don't need it, git repo is the cache +❌ **Configurable TTL**: Use `lastUpdate`, warn if > 5min +❌ **Cache warming**: Happens naturally via `fetchUpstream()` +❌ **Circuit breaker**: Can add later if needed (YAGNI) + +None of these are necessary for the core requirement. + +--- + +## Implementation Plan (Simplified) + +### Phase 1: Core Implementation (1 day) +1. Add `lsRefsLocal()` method to `managed_repository.go` +2. Modify `handleV2Command` to try local on upstream failure +3. Add `UpstreamEnabled` config option + +### Phase 2: Testing (1 day) +1. Unit test `lsRefsLocal()` with various ref scenarios +2. Integration test: offline mode with warm cache +3. Integration test: offline mode with cold cache +4. Integration test: stale cache warning + +### Phase 3: Documentation (0.5 days) +1. Update README.md limitation note +2. Add example test usage + +**Total: 2.5 days** (vs 8-12 days in original plan) + +--- + +## Recommended Changes to Plan + +### Remove These Sections +- ❌ Section 2.2: "ls-refs Cache Structure" - unnecessary +- ❌ Section 2.3: "Modified Request Flow" - over-complicated +- ❌ Phase 1.2: "Create ls-refs Cache Manager" - don't need it +- ❌ Phase 1.3: "Initialize Cache on Server Start" - nothing to initialize +- ❌ Phase 2.1: Caching in `lsRefsUpstream` - just rely on `fetchUpstream` +- ❌ Section 3.1: Complex metrics - simple counters are enough +- ❌ "Risks and Mitigations" section - most risks gone with simpler design + +### Keep These (Simplified) +- ✅ `UpstreamEnabled` config option +- ✅ Basic integration tests +- ✅ README update +- ✅ Request logging with source indicator + +--- + +## Questions to Answer + +### Q: "What if the local repo is corrupted?" +**A**: Same as today - the repo is already critical infrastructure. Git corruption is extremely rare and already a failure mode for fetch operations. + +### Q: "What about cache staleness?" +**A**: We already track `lastUpdate` timestamp. Just log warnings if serving refs older than 5 minutes. No TTL needed. + +### Q: "What if refs are deleted upstream?" +**A**: Next `fetchUpstream()` will sync. Until then, serving stale refs is better than being completely down. This is acceptable for a cache. + +### Q: "How do we force cache refresh?" +**A**: Already exists: `fetchUpstream()` is called when `hasAnyUpdate()` detects changes. No new code needed. + +--- + +## Summary + +**Original plan**: 690 lines, 8-12 days, complex cache layer +**Simplified plan**: 140 lines, 2.5 days, leverage existing git repo + +**Staff engineer principle**: Use existing infrastructure. The local git repository is already a perfect cache for refs. Adding another cache layer is textbook over-engineering. + +**Recommendation**: +1. Implement the 3-change simplified version +2. Ship it and gather metrics +3. Only add complexity if data shows it's needed (it won't be) + +--- + +## Next Steps + +If you agree with this review: +1. Archive `OFFLINE_MODE_PLAN.md` as reference +2. Create `OFFLINE_MODE_PLAN_V2.md` with simplified approach +3. Start implementation with Phase 1 (core logic) +4. Write tests as we go (TDD) + +**Estimated delivery**: 2-3 days vs 2-3 weeks diff --git a/README.md b/README.md index b7d01bd..29b8f8b 100644 --- a/README.md +++ b/README.md @@ -23,9 +23,151 @@ Goblet is intended to be used as a library. You would need to write some glue code. This repository includes the glue code for googlesource.com. See `goblet-server` and `google` directories. +## Offline Mode and Resilience + +Goblet can serve ls-refs requests from the local cache when the upstream server is unavailable, providing resilience during upstream outages: + +### Features + +- **Automatic fallback**: When upstream is down or unreachable, Goblet automatically serves cached ref listings from the local git repository +- **Graceful degradation**: Git operations continue to work with cached data during upstream outages +- **Thread-safe configuration**: Uses atomic operations for concurrent read/write access to configuration +- **Staleness tracking**: Logs warnings when serving refs older than 5 minutes, helping identify stale cache scenarios +- **Testing support**: Upstream connectivity can be disabled entirely for integration testing +- **Zero configuration**: Works out of the box - automatic fallback requires no configuration changes + +### How It Works + +1. **Normal operation** (upstream available): + - Goblet forwards ls-refs requests to upstream + - Caches the response locally + - Serves subsequent fetch requests from cache when possible + +2. **Upstream failure** (network down, server unreachable): + - Goblet detects upstream failure on ls-refs request + - Automatically reads refs from local git repository cache + - Logs fallback event for monitoring + - Serves refs to client from cache + +3. **Upstream recovery**: + - Next ls-refs request attempts upstream again + - On success, cache is updated with latest refs + - System returns to normal operation + +### Configuration + +#### Production Mode (Default) + +By default, Goblet operates with automatic fallback enabled. No configuration needed: + +```go +config := &goblet.ServerConfig{ + LocalDiskCacheRoot: "/path/to/cache", + URLCanonializer: canonicalizer, + TokenSource: tokenSource, + // UpstreamEnabled defaults to true with automatic fallback +} +``` + +#### Testing Mode (Disable Upstream) + +For integration testing where you want to disable upstream connectivity entirely: + +```go +falseValue := false +config := &goblet.ServerConfig{ + LocalDiskCacheRoot: "/path/to/cache", + // ... other config ... +} +config.SetUpstreamEnabled(&falseValue) // Thread-safe: disable all upstream calls +``` + +Or during server initialization: + +```go +falseValue := false +ts := NewTestServer(&TestServerConfig{ + // ... other config ... + UpstreamEnabled: &falseValue, // Start with upstream disabled +}) +``` + +### Monitoring + +Goblet logs important offline mode events: + +``` +# Fallback to local cache +Upstream ls-refs failed (connection refused), attempting local fallback for /cache/path + +# Stale cache warning (>5 minutes old) +Warning: serving stale ls-refs for /cache/path (last update: 10m ago) +``` + +Use these logs to: +- Track upstream availability issues +- Identify when cache is being served +- Monitor cache staleness +- Set up alerts for extended offline periods + +## Testing + +### Quick Start + +Run the full test suite: + +```bash +# Run all tests (short mode, ~38s) +task test-short + +# Or with go directly +go test ./... -short +``` + +### Testing Offline Functionality + +Test the offline mode features specifically: + +```bash +# Run all offline-related tests +go test ./testing -v -run "Offline|Upstream|LsRefsLocal" + +# Test with race detector (verifies thread safety) +go test -race ./testing -run "Offline" + +# Test specific scenarios +go test ./testing -v -run TestOfflineModeWithWarmCache +go test ./testing -v -run TestUpstreamFailureFallback +go test ./testing -v -run TestConcurrentOfflineRequests +``` + +### CI Pipeline + +Run the complete CI pipeline locally: + +```bash +# Full CI (format, lint, test, build) +task ci + +# Individual steps +task fmt-check # Check code formatting +task lint # Run linters +task test-short # Run tests +task build # Build binary +``` + +### Test Coverage + +The offline mode implementation includes comprehensive test coverage: + +- **4 integration tests**: End-to-end scenarios with real git operations +- **8 unit tests**: Edge cases, concurrency, filtering, symbolic refs +- **38 total tests**: All existing tests continue to pass + +See [testing/TEST_COVERAGE.md](testing/TEST_COVERAGE.md) for detailed test documentation. + ## Limitations -Note that Goblet forwards the ls-refs traffic to the upstream server. If the -upstream server is down, Goblet is effectively down. Technically, we can modify -Goblet to serve even if the upstream is down, but the current implementation -doesn't do such thing. +While Goblet can serve ls-refs from cache during upstream outages, fetch operations for objects not already in the cache will still fail if the upstream is unavailable. This is expected behavior as Goblet cannot serve content it doesn't have cached. + +**Important**: The local cache must be populated before offline mode can serve requests. A cold cache (no prior fetches) will result in appropriate errors when upstream is unavailable. diff --git a/RELEASING.md b/RELEASING.md new file mode 100644 index 0000000..43740c0 --- /dev/null +++ b/RELEASING.md @@ -0,0 +1,433 @@ +# Release Process + +This document describes how to create a new release of Goblet. + +## Overview + +Goblet uses **[GoReleaser](https://goreleaser.com/)** for automated, standardized releases. GoReleaser is the industry-standard tool for Go project releases and provides: + +- ✅ **Automatic semantic versioning** from git tags +- ✅ **Multi-platform binary builds** (Linux, macOS, Windows) +- ✅ **Automatic changelog generation** from git commits +- ✅ **SHA256 checksum generation** +- ✅ **GitHub release creation** with all artifacts +- ✅ **Multi-arch Docker images** (amd64, arm64) +- ✅ **Archive generation** (tar.gz, zip) + +## Prerequisites + +- Write access to the GitHub repository +- Clean working directory on the `main` branch +- All CI checks passing on `main` +- Follow [Conventional Commits](https://www.conventionalcommits.org/) for automatic changelog generation + +## Release Workflow Overview + +When you push a version tag, GoReleaser automatically: + +1. Builds binaries for all supported platforms +2. Generates SHA256 checksums for verification +3. Creates archives (tar.gz for Unix, zip for Windows) +4. Generates changelog from git history using conventional commits +5. Creates a GitHub release with all binaries attached +6. Builds and pushes multi-arch Docker images to GitHub Container Registry (GHCR) + +## Supported Platforms + +The release pipeline builds binaries for: + +- **Linux**: amd64, arm64 +- **macOS**: amd64 (Intel), arm64 (Apple Silicon) +- **Windows**: amd64 + +## Conventional Commits for Automatic Changelogs + +GoReleaser generates changelogs automatically from git commit messages. Follow the [Conventional Commits](https://www.conventionalcommits.org/) specification: + +### Commit Message Format + +``` +(): + + + +