Skip to content

refactor: migrate deepinfra provider to AI SDK#11270

Open
daniel-lxs wants to merge 2 commits intomainfrom
migrate/deepinfra-ai-sdk
Open

refactor: migrate deepinfra provider to AI SDK#11270
daniel-lxs wants to merge 2 commits intomainfrom
migrate/deepinfra-ai-sdk

Conversation

@daniel-lxs
Copy link
Member

@daniel-lxs daniel-lxs commented Feb 6, 2026

Migrates the DeepInfra provider from legacy RouterProvider (direct openai SDK) to AI SDK using the official @ai-sdk/deepinfra package.

Changes

src/api/providers/deepinfra.ts

  • Extends BaseProvider directly instead of RouterProvider (follows the deepseek/groq pattern)
  • Uses createDeepInfra from @ai-sdk/deepinfra with streamText/generateText from ai
  • Uses AI SDK transform utilities (convertToAiSdkMessages, convertToolsForAiSdk, processAiSdkStreamPart, mapToolChoice, handleAiSdkError)

Preserved Features

  • Dynamic model fetching: getModels/getModelsFromCache integration with fetchModel() override
  • reasoning_effort: Passed via providerOptions.deepinfra
  • prompt_cache_key: Passed via providerOptions.deepinfra when model supports prompt caching
  • Custom cost calculation: calculateApiCostOpenAI with cache token details in processUsageMetrics()
  • Temperature control: Handled via getModelParams (o3-mini models get undefined temperature)
  • max_completion_tokens: Only included when includeMaxTokens is true
  • Custom headers: X-Deepinfra-Source: roo-code and X-Deepinfra-Version: 2025-08-25
  • isAiSdkProvider(): Returns true

Dependencies

  • Added @ai-sdk/deepinfra (^2.0.31) to src/package.json

Tests

  • Rewrote src/api/providers/__tests__/deepinfra.spec.ts (21 tests, all passing)
  • Tests cover: constructor, createProvider, getModel, createMessage (streaming, reasoning, usage/cost, tools, tool calls, errors), completePrompt, processUsageMetrics, isAiSdkProvider

Important

Refactors DeepInfra provider to use AI SDK, updating deepinfra.ts and tests, and adds @ai-sdk/deepinfra dependency.

  • Behavior:
    • Migrates DeepInfraHandler in deepinfra.ts to use AI SDK, replacing RouterProvider with BaseProvider.
    • Utilizes createDeepInfra, streamText, and generateText from AI SDK.
    • Implements AI SDK utilities for message conversion and error handling.
  • Preserved Features:
    • Dynamic model fetching with getModels and getModelsFromCache.
    • Supports reasoning_effort, prompt_cache_key, and custom cost calculation.
    • Maintains temperature control and max_completion_tokens logic.
    • Custom headers X-Deepinfra-Source and X-Deepinfra-Version.
    • isAiSdkProvider() returns true.
  • Dependencies:
    • Adds @ai-sdk/deepinfra to package.json.
  • Tests:
    • Rewrites deepinfra.spec.ts to cover new AI SDK-based implementation.

This description was created by Ellipsis for 305b26c. You can customize this summary. It will automatically update as commits are pushed.

Migrates the DeepInfra provider from legacy RouterProvider (direct openai SDK)
to AI SDK using the official @ai-sdk/deepinfra package.

- Use createDeepInfra from @ai-sdk/deepinfra with streamText/generateText
- Extend BaseProvider directly (like deepseek/groq pattern)
- Preserve dynamic model fetching via getModels/getModelsFromCache
- Pass reasoning_effort and prompt_cache_key via providerOptions
- Keep custom cost calculation with calculateApiCostOpenAI
- Temperature handling via getModelParams (handles o3-mini suppression)
- Custom headers: X-Deepinfra-Source and X-Deepinfra-Version
- isAiSdkProvider() returns true
@daniel-lxs daniel-lxs requested review from cte, jr and mrubens as code owners February 6, 2026 19:31
@dosubot dosubot bot added the size:XL This PR changes 500-999 lines, ignoring generated files. label Feb 6, 2026
@roomote
Copy link
Contributor

roomote bot commented Feb 6, 2026

Rooviewer Clock   See task

Both issues from the previous review have been addressed. No new issues found.

  • Missing DEFAULT_HEADERS: Standard Roo Code identification headers (HTTP-Referer, X-Title, User-Agent) are dropped in the migration. All other AI SDK providers include them.
  • Provider re-created per request: createProvider() runs on every API call instead of being cached in the constructor like DeepSeek and Groq.
Previous reviews

Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues.

return createDeepInfra({
apiKey: this.options.deepInfraApiKey ?? "not-provided",
baseURL: this.options.deepInfraBaseUrl || DEEPINFRA_DEFAULT_BASE_URL,
headers: DEEPINFRA_HEADERS,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old RouterProvider-based implementation merged DEFAULT_HEADERS (HTTP-Referer, X-Title, User-Agent) with the DeepInfra-specific headers. This migration drops those standard headers. Every other AI SDK provider in the codebase (DeepSeek, Groq, Fireworks, Mistral, xAI, etc.) includes DEFAULT_HEADERS in their provider creation. Without them, DeepInfra requests no longer carry the Roo Code identification headers that other providers send.

Suggested change
headers: DEEPINFRA_HEADERS,
headers: { ...DEFAULT_HEADERS, ...DEEPINFRA_HEADERS },

You'll also need to add import { DEFAULT_HEADERS } from "./constants" to the imports.

Fix it with Roo Code or mention @roomote and request a fix.

Comment on lines 89 to 92
protected getLanguageModel(modelId: string) {
const provider = this.createProvider()
return provider(modelId)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

createProvider() is called inside getLanguageModel(), which runs on every createMessage() and completePrompt() invocation. This creates a new @ai-sdk/deepinfra provider instance per API call. The DeepSeek and Groq handlers both create the provider once in the constructor and store it as this.provider. Consider caching it the same way for consistency and to avoid the repeated setup cost.

Fix it with Roo Code or mention @roomote and request a fix.

Address review feedback:
- Import and include DEFAULT_HEADERS (HTTP-Referer, X-Title, User-Agent) alongside
  DeepInfra-specific headers, matching all other AI SDK providers.
- Cache the provider instance in the constructor instead of re-creating it
  per request, following the DeepSeek/Groq pattern.
- Add test for provider caching behavior.
- Update test assertions to verify merged headers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant