Add Circuit Breaker and Retries for AI Provider

[AnkanMisra]

Problem

The Gateway calls OpenRouter via a standard http.DefaultClient.

If OpenRouter experiences downtime, high latency, or intermittent failures, the Gateway will:

1. Hang until the 30s timeout, tying up resources.
2. Spam the failing API repeatedly, worsening the issue.
3. Fail immediately on transient network blips without retrying.

stateDiagram-v2
    [*] --> Closed
    
    Closed --> Open : Failure Threshold Reached
    Open --> HalfOpen : Timeout Expired
    
    HalfOpen --> Closed : Success
    HalfOpen --> Open : Failure
    
    state Closed {
        [*] --> Request
        Request --> Success
        Request --> Failure
    }
    
    state Open {
        [*] --> FastFail
    }

Loading

Solution

Implement the Circuit Breaker pattern (to stop calling a dead service) and Exponential Backoff Retries (for transient errors).

Implementation

1. Add dependency: github.com/sony/gobreaker
2. Wrap callOpenRouter logic:

var cb *gobreaker.CircuitBreaker

func init() {
    cb = gobreaker.NewCircuitBreaker(gobreaker.Settings{
        Name:        "OpenRouter",
        MaxRequests: 5,
        Timeout:     30 * time.Second,
        ReadyToTrip: func(counts gobreaker.Counts) bool {
            return counts.ConsecutiveFailures > 3
        },
    })
}

Acceptance Criteria

• Gateway stops calling OpenRouter after 3 consecutive failures (Circuit Open)
• Gateway returns "503 Service Unavailable" immediately when Circuit is Open
• Gateway retries 500/502/503 responses from OpenRouter up to 3 times
• Gateway recovers automatically (Half-Open -> Closed)

Testing

• Mock OpenRouter to return 500s -> Verify Circuit Open.
• Mock OpenRouter to return success after 1 failure -> Verify Retry success.

[navin-oss]

This is a solid and important improvement, especially for protecting the gateway during OpenRouter instability.

I’d like to work on this issue under apertre3.0 if possible.
I’m comfortable implementing the circuit breaker using gobreaker, adding exponential backoff retries, and covering the acceptance criteria with appropriate tests.

Please assign this issue to me if that works.

[AnkanMisra]

sure @navin-oss
assigned

[navin-oss]

Implemented circuit breaker and exponential backoff retries for OpenRouter calls.

• Added gobreaker-based circuit breaker
• Added retry logic for transient 5xx/network errors
• Returns 503 immediately when circuit is open
• All tests passing (go build + go test ./...)

PR: #113
Ready for review.

[Omkarop0808]

hi @AnkanMisra i this issue is still open i would like to work on this issue please assign it to me.