fix(ai21): update chat completions to use OpenAI-compatible Jamba API (/v1/chat/completions)#1529
Open
cluster2600 wants to merge 1 commit intoPortkey-AI:mainfrom
Open
Conversation
The AI21 integration was using the legacy endpoint /{model}/chat which
only worked with older API versions. The current AI21 Jamba 1.5+ models
use an OpenAI-compatible /v1/chat/completions endpoint.
Changes:
- api.ts: route chatComplete to /chat/completions instead of /{model}/chat
- chatComplete.ts: remove old non-OpenAI request/response format
- index.ts: use chatCompleteParams + responseTransformers from open-ai-base
(same pattern as the Cerebras provider) with unsupported params excluded
Legacy Jurassic-2 text completion (complete) and embed endpoints are
unchanged.
Closes Portkey-AI#1215
Reference: https://docs.ai21.com/reference/jamba-1-6-api-ref
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Updates the AI21 provider integration to route chat completion requests
to the new OpenAI-compatible
/v1/chat/completionsendpoint used bythe Jamba 1.5 / 1.6 model family, instead of the legacy
/v1/{model}/chatendpoint.Also replaces the hand-rolled request/response format in
chatComplete.tswith the sharedchatCompleteParams+responseTransformershelpers fromopen-ai-base, exactly like theCerebras provider does.
Why
Fixes #1215.
The previous implementation targeted the legacy AI21 API that was
designed for older Jurassic-style models. AI21's current Jamba models
(jamba-instruct, jamba-1.5-mini, jamba-1.5-large, jamba-1.6-mini,
jamba-1.6-large) expose an OpenAI-compatible chat endpoint at
POST /v1/chat/completions. Sending requests to the old/{model}/chatroute returns a 404 for any Jamba model.Reference: https://docs.ai21.com/reference/jamba-1-6-api-ref
How
src/providers/ai21/api.tschatCompletecase now returns/chat/completions(model is passedin the request body, as per the OpenAI spec).
completecase is unchanged — legacy Jurassic-2 models still use/{model}/complete.src/providers/ai21/chatComplete.tsAI21ChatCompleteConfig(non-OpenAI param mapping),AI21ChatCompleteResponseinterface, andAI21ChatCompleteResponseTransform.AI21ErrorResponseTransform(still used byembed.ts).src/providers/ai21/index.tschatCompletenow useschatCompleteParams()fromopen-ai-base,excluding the handful of OpenAI params the Jamba API does not support:
logit_bias,logprobs,top_logprobs,service_tier,parallel_tool_calls.responseTransforms.chatCompleteis provided byresponseTransformers(AI21, { chatComplete: true }), which correctlystamps the provider name and delegates error handling to the standard
OpenAI error transformer.
Testing
npx tsc --noEmit— pre-existing errors inbytezandopen-ai-baseare unchanged)npm run buildsucceeds (build log shows only pre-existing warnings)curlagainst a local gateway with an AI21 API key against a Jamba model returns a valid chat completion responseChecklist
completeandembedendpoints untouched