Skip to content

Comments

fix(ai21): update chat completions to use OpenAI-compatible Jamba API (/v1/chat/completions)#1529

Open
cluster2600 wants to merge 1 commit intoPortkey-AI:mainfrom
cluster2600:fix/ai21-jamba-openai-compatible-api
Open

fix(ai21): update chat completions to use OpenAI-compatible Jamba API (/v1/chat/completions)#1529
cluster2600 wants to merge 1 commit intoPortkey-AI:mainfrom
cluster2600:fix/ai21-jamba-openai-compatible-api

Conversation

@cluster2600
Copy link

What

Updates the AI21 provider integration to route chat completion requests
to the new OpenAI-compatible /v1/chat/completions endpoint used by
the Jamba 1.5 / 1.6 model family, instead of the legacy
/v1/{model}/chat endpoint.

Also replaces the hand-rolled request/response format in
chatComplete.ts with the shared chatCompleteParams +
responseTransformers helpers from open-ai-base, exactly like the
Cerebras provider does.

Why

Fixes #1215.

The previous implementation targeted the legacy AI21 API that was
designed for older Jurassic-style models. AI21's current Jamba models
(jamba-instruct, jamba-1.5-mini, jamba-1.5-large, jamba-1.6-mini,
jamba-1.6-large) expose an OpenAI-compatible chat endpoint at
POST /v1/chat/completions. Sending requests to the old
/{model}/chat route returns a 404 for any Jamba model.

Reference: https://docs.ai21.com/reference/jamba-1-6-api-ref

How

src/providers/ai21/api.ts

  • chatComplete case now returns /chat/completions (model is passed
    in the request body, as per the OpenAI spec).
  • complete case is unchanged — legacy Jurassic-2 models still use
    /{model}/complete.

src/providers/ai21/chatComplete.ts

  • Removed the old AI21ChatCompleteConfig (non-OpenAI param mapping),
    AI21ChatCompleteResponse interface, and
    AI21ChatCompleteResponseTransform.
  • Kept only AI21ErrorResponseTransform (still used by embed.ts).

src/providers/ai21/index.ts

  • chatComplete now uses chatCompleteParams() from open-ai-base,
    excluding the handful of OpenAI params the Jamba API does not support:
    logit_bias, logprobs, top_logprobs, service_tier,
    parallel_tool_calls.
  • responseTransforms.chatComplete is provided by
    responseTransformers(AI21, { chatComplete: true }), which correctly
    stamps the provider name and delegates error handling to the standard
    OpenAI error transformer.

Testing

  • TypeScript compiles without new errors (npx tsc --noEmit — pre-existing errors in bytez and open-ai-base are unchanged)
  • npm run build succeeds (build log shows only pre-existing warnings)
  • Manual curl against a local gateway with an AI21 API key against a Jamba model returns a valid chat completion response

Checklist

  • Follows existing provider pattern (matches Cerebras, Groq, etc.)
  • Backward-compatible: legacy Jurassic-2 complete and embed endpoints untouched
  • No new TypeScript errors introduced
  • Prettier formatting applied

The AI21 integration was using the legacy endpoint /{model}/chat which
only worked with older API versions. The current AI21 Jamba 1.5+ models
use an OpenAI-compatible /v1/chat/completions endpoint.

Changes:
- api.ts: route chatComplete to /chat/completions instead of /{model}/chat
- chatComplete.ts: remove old non-OpenAI request/response format
- index.ts: use chatCompleteParams + responseTransformers from open-ai-base
  (same pattern as the Cerebras provider) with unsupported params excluded

Legacy Jurassic-2 text completion (complete) and embed endpoints are
unchanged.

Closes Portkey-AI#1215
Reference: https://docs.ai21.com/reference/jamba-1-6-api-ref
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update ai21 integration

1 participant