Skip to content

Conversation

@apascal07
Copy link
Collaborator

@apascal07 apascal07 commented Dec 11, 2025

This PR adds Go iterator-based streaming support for model generation and prompt execution, enabling real-time consumption of model output chunks.

New Features

1. GenerateStream / GenerateDataStream - Stream model responses using Go's range-over-func pattern:

type StoryOutput struct { Title, Content string }

for result, err := range genkit.GenerateDataStream[StoryOutput, StoryOutput](ctx, g,
    ai.WithModelName("googleai/gemini-2.5-flash"),
    ai.WithPrompt("Tell me a story"),
) {
    if err != nil {
        return err
    }
    if result.Done {
        fmt.Println("Final:", result.Output.Title)
    } else {
        fmt.Print(result.Chunk.Content) // Stream chunks as they arrive
    }
}

2. Prompt.ExecuteStream - Stream prompt execution:

prompt := genkit.LookupPrompt(g, "myPrompt")
for result, err := range prompt.ExecuteStream(ctx, ai.WithInput(input)) {
    if err != nil {
        return err
    }
    if !result.Done {
        cb(ctx, result.Chunk) // Forward chunks to flow callback
    }
}

3. DataPrompt with typed streaming - Strongly-typed prompts with streaming:

type StoryInput struct { Topic string `json:"topic"` }
type StoryOutput struct { Title, Content string }

storyPrompt := genkit.DefineDataPrompt[StoryInput, StoryOutput, StoryOutput](
    g, "story",
    ai.WithModelName("googleai/gemini-2.5-flash"),
    ai.WithPrompt("Write a story about {{topic}}"),
)

// Alternatively, look up and wrap a .prompt file:
// storyPrompt := genkit.LookupDataPrompt[StoryInput, StoryOutput, StoryOutput](g, "story")

for result, err := range storyPrompt.ExecuteStream(ctx, StoryInput{Topic: "dragons"}) {
    if err != nil {
        return err
    }
    if result.Done {
        fmt.Printf("Title: %s\n", result.Output.Title)
    }
}

@github-actions github-actions bot added the go label Dec 11, 2025
@apascal07 apascal07 changed the base branch from main to ap/go-structured-streaming December 13, 2025 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant