Skip to content

feat: support using RG scoped prompt templates #722

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
n-o-u-r-h-a-n wants to merge 30 commits into
base: main
Choose a base branch
from
+455 −31
Open

feat: support using RG scoped prompt templates #722

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
7f9397b
Updating TemplateConfig.java.
n-o-u-r-h-a-n Nov 28, 2025
8f9b5ca
Merge branch 'main' into support-using-RG
n-o-u-r-h-a-n Dec 4, 2025
9f6138c
Merge branch 'main' into support-using-RG
n-o-u-r-h-a-n Dec 11, 2025
60881ea
Merge branch 'main' into support-using-RG
n-o-u-r-h-a-n Dec 23, 2025
731a02a
Merge branch 'main' into support-using-RG
n-o-u-r-h-a-n Jan 15, 2026
3a84ad7
rolling back
n-o-u-r-h-a-n Jan 15, 2026
35004bb
Merge branch 'main' into support-using-RG
n-o-u-r-h-a-n Jan 20, 2026
49be7aa
Orchestration Convenience RG-Scoped Prompt Templates
n-o-u-r-h-a-n Jan 21, 2026
230fe5d
Merge branch 'main' into support-using-RG
n-o-u-r-h-a-n Jan 21, 2026
9065cbf
Merge branch 'main' into support-using-RG
n-o-u-r-h-a-n Jan 21, 2026
fe3817a
Merge branch 'main' into support-using-RG
n-o-u-r-h-a-n Jan 21, 2026
cb1ac38
Triggering
n-o-u-r-h-a-n Jan 21, 2026
0642127
Merge remote-tracking branch 'origin/support-using-RG' into support-u…
n-o-u-r-h-a-n Jan 21, 2026
c288e4d
Triggering 2
n-o-u-r-h-a-n Jan 21, 2026
536a917
Rolling back renaming avoiding API breakage.
n-o-u-r-h-a-n Jan 21, 2026
3c6f4e0
Formatting
bot-sdk-js Jan 21, 2026
d7a360f
Merge branch 'main' into support-using-RG
n-o-u-r-h-a-n Jan 21, 2026
3b97f7c
Changing RG id + minor format
n-o-u-r-h-a-n Jan 21, 2026
abdd0fb
Changing API Approach
n-o-u-r-h-a-n Jan 22, 2026
89c6f96
Formatting
bot-sdk-js Jan 22, 2026
f6ea67a
Adding JavaDocs + Formatting
n-o-u-r-h-a-n Jan 22, 2026
8d72f4e
Merge remote-tracking branch 'origin/support-using-RG' into support-u…
n-o-u-r-h-a-n Jan 22, 2026
416e8c9
Merge branch 'main' into support-using-RG
n-o-u-r-h-a-n Jan 26, 2026
66b27ed
Removing LLM with Image Support
n-o-u-r-h-a-n Jan 26, 2026
e7ceaa0
Merge remote-tracking branch 'origin/support-using-RG' into support-u…
n-o-u-r-h-a-n Jan 26, 2026
acf6003
Updating release notes
n-o-u-r-h-a-n Jan 26, 2026
8c09db8
Making exception error message clearer
n-o-u-r-h-a-n Jan 26, 2026
4cb9d58
Merge branch 'main' into support-using-RG
n-o-u-r-h-a-n Jan 26, 2026
6787c53
Making exception error message clearer
n-o-u-r-h-a-n Jan 26, 2026
258acfe
Merge remote-tracking branch 'origin/support-using-RG' into support-u…
n-o-u-r-h-a-n Jan 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions core-services/prompt-registry/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,10 @@
</scm>
<properties>
<project.rootdir>${project.basedir}/../../</project.rootdir>
<coverage.complexity>84%</coverage.complexity>
<coverage.complexity>73%</coverage.complexity>
<coverage.line>90%</coverage.line>
<coverage.instruction>92%</coverage.instruction>
<coverage.branch>100%</coverage.branch>
<coverage.branch>78%</coverage.branch>
<coverage.method>80%</coverage.method>
<coverage.class>100%</coverage.class>
</properties>
Expand Down
2 changes: 1 addition & 1 deletion docs/release_notes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

### 📈 Improvements

-
-[Orchestration] Added new API OrchestrationTemplateReference#withScope to support RG-scoped prompt templates.

### 🐛 Fixed Issues

Expand Down
5 changes: 5 additions & 0 deletions orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationClientException.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,11 @@ public class OrchestrationClientException extends ClientException {
(message, clientError, cause) -> {
final var details = extractInputFilterDetails(clientError);
if (details.isEmpty()) {
if (message.equals(
"Request failed with status 404 (Not Found): 404 - Templating Module: No Prompt Template found in the Prompt Registry.")) {
message +=
"\n Please verify that the provided resource group id is correct and matches the provided template reference details if the template is referenced from a resource-group scope, otherwise use the tenant scope without resource group id header.";
}
return new OrchestrationClientException(message, cause).setClientError(clientError);
}
return new Input(message, cause).setFilterDetails(details).setClientError(clientError);
Expand Down
26 changes: 25 additions & 1 deletion orchestration/src/main/java/com/sap/ai/sdk/orchestration/OrchestrationTemplateReference.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,15 @@
import com.google.common.annotations.Beta;
import com.sap.ai.sdk.orchestration.model.PromptTemplatingModuleConfigPrompt;
import com.sap.ai.sdk.orchestration.model.TemplateRef;
import com.sap.ai.sdk.orchestration.model.TemplateRefByID;
import com.sap.ai.sdk.orchestration.model.TemplateRefByScenarioNameVersion;
import com.sap.ai.sdk.orchestration.model.TemplateRefTemplateRef;
import javax.annotation.Nonnull;
import lombok.AccessLevel;
import lombok.AllArgsConstructor;
import lombok.EqualsAndHashCode;
import lombok.Value;
import lombok.With;

/**
* A reference to a template to use in {@link OrchestrationModuleConfig}.
Expand All @@ -22,6 +25,8 @@
public class OrchestrationTemplateReference extends TemplateConfig {
@Nonnull TemplateRefTemplateRef reference;

@With @Nonnull ScopeEnum scope;

/**
* Create a low-level representation of the template.
*
Expand All @@ -30,6 +35,25 @@ public class OrchestrationTemplateReference extends TemplateConfig {
@Nonnull
@Override
protected PromptTemplatingModuleConfigPrompt toLowLevel() {
return TemplateRef.create().templateRef(reference);
if (reference instanceof TemplateRefByID idRef) {
final var valueById = TemplateRefByID.ScopeEnum.valueOf(scope.name());
idRef.setScope(valueById);
return TemplateRef.create().templateRef(idRef);
} else if (reference instanceof TemplateRefByScenarioNameVersion scenarioRef) {
final var valueByScenario = TemplateRefByScenarioNameVersion.ScopeEnum.valueOf(scope.name());
scenarioRef.setScope(valueByScenario);
return TemplateRef.create().templateRef(scenarioRef);
} else {
throw new IllegalStateException(
"Unsupported template reference type: " + reference.getClass());
}
}

/** The scope of the template reference. */
public enum ScopeEnum {
/** Template is resolved within the current tenant scope. */
TENANT,
/** Template is resolved within the configured resource group scope. */
RESOURCE_GROUP
}
}
15 changes: 11 additions & 4 deletions orchestration/src/main/java/com/sap/ai/sdk/orchestration/TemplateConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package com.sap.ai.sdk.orchestration;

import static com.sap.ai.sdk.orchestration.OrchestrationTemplateReference.ScopeEnum.TENANT;

import com.google.common.annotations.Beta;
import com.sap.ai.sdk.orchestration.model.PromptTemplatingModuleConfigPrompt;
import com.sap.ai.sdk.orchestration.model.TemplateRefByID;
Expand Down Expand Up @@ -35,28 +37,33 @@ public static OrchestrationTemplate create() {
}

/**
* Build a template reference.
* Build a template reference with tenant level scope.
*
* @return An intermediate object to build the template reference.
*/
@Nonnull
public static ReferenceBuilder reference() {
final var templ = TemplateRefByScenarioNameVersion.create();
return s -> n -> v -> new OrchestrationTemplateReference(templ.scenario(s).name(n).version(v));

return scenario ->
name ->
version ->
new OrchestrationTemplateReference(
templ.scenario(scenario).name(name).version(version), TENANT);
}

/** Intermediate object to build a template reference. */
@FunctionalInterface
public interface ReferenceBuilder {
/**
* Build a template reference with the given id.
* Build a template reference with the given id for tenant scope.
*
* @param id The id of the template.
* @return A template reference with the given id.
*/
@Nonnull
default OrchestrationTemplateReference byId(@Nonnull final String id) {
return new OrchestrationTemplateReference(TemplateRefByID.create().id(id));
return new OrchestrationTemplateReference(TemplateRefByID.create().id(id), TENANT);
}

/**
Expand Down
50 changes: 47 additions & 3 deletions ...stration/src/test/java/com/sap/ai/sdk/orchestration/OrchestrationConvenienceUnitTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package com.sap.ai.sdk.orchestration;

import static com.sap.ai.sdk.orchestration.OrchestrationTemplateReference.ScopeEnum.RESOURCE_GROUP;
import static com.sap.ai.sdk.orchestration.OrchestrationTemplateReference.ScopeEnum.TENANT;
import static com.sap.ai.sdk.orchestration.model.UserChatMessage.RoleEnum.USER;
import static org.assertj.core.api.Assertions.assertThat;

Expand Down Expand Up @@ -160,20 +162,35 @@ void testTemplateConstruction() {
void testTemplateReferenceConstruction() {
var templateReferenceId = TemplateConfig.reference().byId("id");
var expectedTemplateReferenceId =
new OrchestrationTemplateReference(TemplateRefByID.create().id("id"));
new OrchestrationTemplateReference(TemplateRefByID.create().id("id"), TENANT);
var templateReferenceIdLowLevel =
TemplateRef.create().templateRef(TemplateRefByID.create().id("id"));
assertThat(templateReferenceId).isEqualTo(expectedTemplateReferenceId);
assertThat(templateReferenceId.toLowLevel()).isEqualTo(templateReferenceIdLowLevel);

templateReferenceId = TemplateConfig.reference().byId("id").withScope(RESOURCE_GROUP);
expectedTemplateReferenceId =
new OrchestrationTemplateReference(TemplateRefByID.create().id("id"), RESOURCE_GROUP);
templateReferenceIdLowLevel =
TemplateRef.create()
.templateRef(
TemplateRefByID.create().id("id").scope(TemplateRefByID.ScopeEnum.RESOURCE_GROUP));
assertThat(templateReferenceId).isEqualTo(expectedTemplateReferenceId);
assertThat(templateReferenceId.toLowLevel()).isEqualTo(templateReferenceIdLowLevel);

var templateReferenceScenarioNameVersion =
TemplateConfig.reference().byScenario("scenario").name("name").version("version");
TemplateConfig.reference()
.byScenario("scenario")
.name("name")
.version("version")
.withScope(TENANT);
var expectedTemplateReferenceScenarioNameVersion =
new OrchestrationTemplateReference(
TemplateRefByScenarioNameVersion.create()
.scenario("scenario")
.name("name")
.version("version"));
.version("version"),
TENANT);
var templateReferenceScenarioNameVersionLowLevel =
TemplateRef.create()
.templateRef(
Expand All @@ -185,6 +202,33 @@ void testTemplateReferenceConstruction() {
.isEqualTo(expectedTemplateReferenceScenarioNameVersion);
assertThat(templateReferenceScenarioNameVersion.toLowLevel())
.isEqualTo(templateReferenceScenarioNameVersionLowLevel);

templateReferenceScenarioNameVersion =
TemplateConfig.reference()
.byScenario("scenario")
.name("name")
.version("version")
.withScope(RESOURCE_GROUP);
var scopeScenario = TemplateRefByScenarioNameVersion.ScopeEnum.RESOURCE_GROUP;
expectedTemplateReferenceScenarioNameVersion =
new OrchestrationTemplateReference(
TemplateRefByScenarioNameVersion.create()
.scenario("scenario")
.name("name")
.version("version"),
RESOURCE_GROUP);
templateReferenceScenarioNameVersionLowLevel =
TemplateRef.create()
.templateRef(
TemplateRefByScenarioNameVersion.create()
.scenario("scenario")
.name("name")
.version("version")
.scope(scopeScenario));
assertThat(templateReferenceScenarioNameVersion)
.isEqualTo(expectedTemplateReferenceScenarioNameVersion);
assertThat(templateReferenceScenarioNameVersion.toLowLevel())
.isEqualTo(templateReferenceScenarioNameVersionLowLevel);
}

@Test
Expand Down
79 changes: 77 additions & 2 deletions orchestration/src/test/java/com/sap/ai/sdk/orchestration/OrchestrationUnitTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,11 @@
import static com.github.tomakehurst.wiremock.client.WireMock.verify;
import static com.sap.ai.sdk.orchestration.AzureFilterThreshold.ALLOW_SAFE;
import static com.sap.ai.sdk.orchestration.AzureFilterThreshold.ALLOW_SAFE_LOW_MEDIUM;
import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.GEMINI_2_5_FLASH;
import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.GPT_4O;
import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.GPT_4O_MINI;
import static com.sap.ai.sdk.orchestration.OrchestrationAiModel.Parameter.*;
import static com.sap.ai.sdk.orchestration.OrchestrationTemplateReference.ScopeEnum.RESOURCE_GROUP;
import static com.sap.ai.sdk.orchestration.model.AzureThreshold.NUMBER_0;
import static com.sap.ai.sdk.orchestration.model.AzureThreshold.NUMBER_4;
import static com.sap.ai.sdk.orchestration.model.AzureThreshold.NUMBER_6;
Expand Down Expand Up @@ -1259,7 +1261,7 @@ void testResponseFormatText() throws IOException {
}

@Test
void testTemplateFromPromptRegistryById() throws IOException {
void testTemplateFromPromptRegistryByIdTenant() throws IOException {
{
stubFor(
post(anyUrl())
Expand All @@ -1285,7 +1287,44 @@ void testTemplateFromPromptRegistryById() throws IOException {
}

@Test
void testTemplateFromPromptRegistryByScenario() throws IOException {
void testTemplateFromPromptRegistryByIdResourceGroup() throws IOException {
{
stubFor(
post(anyUrl())
.willReturn(
aResponse()
.withBodyFile("templateReferenceResourceGroupResponse.json")
.withHeader("Content-Type", "application/json")));

var template =
TemplateConfig.reference()
.byId("8bf72116-11ab-41bb-8933-8be56f59cb67")
.withScope(RESOURCE_GROUP);
var config =
new OrchestrationModuleConfig()
.withLlmConfig(GEMINI_2_5_FLASH.withParam(TEMPERATURE, 0.0));
var configWithTemplate = config.withTemplateConfig(template);

var inputParams =
Map.of(
"categories",
"Finance, Tech, Sports",
"inputExample",
"What's the latest news on the stock market?");
var prompt = new OrchestrationPrompt(inputParams);

final var response = client.chatCompletion(prompt, configWithTemplate);
assertThat(response.getContent()).startsWith("Finance");
assertThat(response.getOriginalResponse().getIntermediateResults().getTemplating())
.hasSize(2);

final String request = fileLoaderStr.apply("templateReferenceResourceGroupByIdRequest.json");
verify(postRequestedFor(anyUrl()).withRequestBody(equalToJson(request)));
}
}

@Test
void testTemplateFromPromptRegistryByScenarioTenant() throws IOException {
stubFor(
post(anyUrl())
.willReturn(
Expand All @@ -1307,6 +1346,42 @@ void testTemplateFromPromptRegistryByScenario() throws IOException {
verify(postRequestedFor(anyUrl()).withRequestBody(equalToJson(request)));
}

@Test
void testTemplateFromPromptRegistryByScenarioResourceGroup() throws IOException {
stubFor(
post(anyUrl())
.willReturn(
aResponse()
.withBodyFile("templateReferenceResourceGroupResponse.json")
.withHeader("Content-Type", "application/json")));

var template =
TemplateConfig.reference()
.byScenario("categorization")
.name("example-prompt-template")
.version("0.0.1")
.withScope(RESOURCE_GROUP);
var config =
new OrchestrationModuleConfig().withLlmConfig(GEMINI_2_5_FLASH.withParam(TEMPERATURE, 0.0));
var configWithTemplate = config.withTemplateConfig(template);

var inputParams =
Map.of(
"categories",
"Finance, Tech, Sports",
"inputExample",
"What's the latest news on the stock market?");
var prompt = new OrchestrationPrompt(inputParams);

final var response = client.chatCompletion(prompt, configWithTemplate);
assertThat(response.getContent()).startsWith("Finance");
assertThat(response.getOriginalResponse().getIntermediateResults().getTemplating()).hasSize(2);

final String request =
fileLoaderStr.apply("templateReferenceResourceGroupByScenarioRequest.json");
verify(postRequestedFor(anyUrl()).withRequestBody(equalToJson(request)));
}

@Test
void testTemplateFromInput() throws IOException {
stubFor(
Expand Down
69 changes: 69 additions & 0 deletions orchestration/src/test/resources/__files/templateReferenceResourceGroupResponse.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
{
"request_id": "921f38b7-3434-9171-85df-27be1b7fca3c",
"intermediate_results": {
"templating": [
{
"role": "system",
"content": "You classify input text into the two following categories: Finance, Tech, Sports"
},
{
"content": "What's the latest news on the stock market?",
"role": "user"
}
],
"llm": {
"id": "",
"object": "chat.completion",
"created": 1769424215,
"model": "gemini-2.5-flash",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Finance"
},
"finish_reason": "stop"
}
],
"usage": {
"completion_tokens": 154,
"prompt_tokens": 26,
"total_tokens": 180,
"prompt_tokens_details": {
"cached_tokens": 0
},
"completion_tokens_details": {
"reasoning_tokens": 153
}
}
}
},
"final_result": {
"id": "",
"object": "chat.completion",
"created": 1769424215,
"model": "gemini-2.5-flash",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Finance"
},
"finish_reason": "stop"
}
],
"usage": {
"completion_tokens": 154,
"prompt_tokens": 26,
"total_tokens": 180,
"prompt_tokens_details": {
"cached_tokens": 0
},
"completion_tokens_details": {
"reasoning_tokens": 153
}
}
}
}
Loading