Skip to content

video Indexer in auto-clipper MMI-3401 #2550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JacobWang-bc wants to merge 13 commits into
base: dev
Choose a base branch
from
Open

video Indexer in auto-clipper MMI-3401 #2550

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
27f172d
Update .gitignore to include additional media file types and output d…
JacobWang-bc Jan 28, 2026
4bdfd0a
Enhance .env.sample for Auto Clipper
JacobWang-bc Jan 28, 2026
085bde6
Add Video Indexer support to AutoClipper Harness
JacobWang-bc Jan 28, 2026
bdf50ef
feat(auto-clipper): extend TimestampedTranscript with speaker info
JacobWang-bc Jan 29, 2026
553b1ab
feat(auto-clipper): add Video Indexer configuration options
JacobWang-bc Jan 29, 2026
de40a45
feat(auto-clipper): implement Azure Video Indexer client
JacobWang-bc Jan 29, 2026
03c595c
feat(auto-clipper): integrate Video Indexer into transcription pipeline
JacobWang-bc Jan 29, 2026
de097fb
feat(auto-clipper): add speaker prefix to transcript output
JacobWang-bc Jan 29, 2026
101cb79
feat(auto-clipper): add CHAN station config with Video Indexer
JacobWang-bc Jan 29, 2026
1e956cb
fix(auto-clipper): strip markdown code fences from LLM response
JacobWang-bc Jan 29, 2026
feaf320
feat(auto-clipper): update CHAN station config
JacobWang-bc Feb 3, 2026
801dc2f
docs(auto-clipper): update README to reflect changes in transcription…
JacobWang-bc Feb 3, 2026
53fe92e
feat(auto-clipper): add Azure Video Indexer configuration to deployme…
JacobWang-bc Feb 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions openshift/kustomize/services/auto-clipper/base/deploy.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,26 @@ spec:
name: azure-openai
key: AZURE_OPENAI_KEY

# Azure Video Indexer Configuration (optional - for speaker identification)
- name: Service__AzureVideoIndexerAccountId
valueFrom:
secretKeyRef:
name: azure-video-indexer
key: AZURE_VIDEO_INDEXER_ACCOUNT_ID
optional: true
- name: Service__AzureVideoIndexerApiKey
valueFrom:
secretKeyRef:
name: azure-video-indexer
key: AZURE_VIDEO_INDEXER_API_KEY
optional: true
- name: Service__AzureVideoIndexerLocation
valueFrom:
secretKeyRef:
name: azure-video-indexer
key: AZURE_VIDEO_INDEXER_LOCATION
optional: true

# Service Configuration
- name: Service__MaxFailLimit
valueFrom:
Expand Down
3 changes: 3 additions & 0 deletions openshift/kustomize/services/auto-clipper/overlays/dev/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ secretGenerator:
- name: azure-openai
type: stringData
env: openai.env
- name: azure-video-indexer
type: stringData
env: video-indexer.env

patches:
- target:
Expand Down
3 changes: 3 additions & 0 deletions openshift/kustomize/services/auto-clipper/overlays/prod/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ secretGenerator:
- name: azure-openai
type: stringData
env: openai.env
- name: azure-video-indexer
type: stringData
env: video-indexer.env

patches:
- target:
Expand Down
3 changes: 3 additions & 0 deletions openshift/kustomize/services/auto-clipper/overlays/test/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ secretGenerator:
- name: azure-openai
type: stringData
env: openai.env
- name: azure-video-indexer
type: stringData
env: video-indexer.env

patches:
- target:
Expand Down
21 changes: 15 additions & 6 deletions services/net/auto-clipper/AutoClipperManager.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -635,23 +635,32 @@ private static void CleanupTemporaryFiles(bool isSyncedToS3, params string[] fil

/// <summary>
/// Format the transcript to include newlines.
/// When speaker information is available (from Video Indexer), includes speaker prefix.
/// Azure Speech transcripts have no speaker info and will output plain text.
/// </summary>
/// <param name="transcript"></param>
/// <returns></returns>
/// <param name="segments">Transcript segments with optional speaker information.</param>
/// <returns>Formatted transcript string.</returns>
private static string BuildTranscriptDocument(IReadOnlyList<TimestampedTranscript> segments)
{
if (segments == null || segments.Count == 0) return string.Empty;

var sb = new StringBuilder();
var index = 1;
foreach (var segment in segments)
{
if (string.IsNullOrWhiteSpace(segment.Text)) continue;
// sb.AppendLine(index.ToString(CultureInfo.InvariantCulture));
// sb.AppendLine($"{FormatTimestamp(segment.Start)} --> {FormatTimestamp(segment.End)}");

// Add speaker prefix if available (Video Indexer provides this)
if (!string.IsNullOrWhiteSpace(segment.SpeakerName))
{
sb.Append($"{segment.SpeakerName}: ");
}
else if (segment.SpeakerId.HasValue)
{
sb.Append($"speaker{segment.SpeakerId}: ");
}

sb.AppendLine(segment.Text.Trim());
sb.AppendLine();
index++;
}

return sb.ToString().Trim();
Expand Down
8 changes: 8 additions & 0 deletions services/net/auto-clipper/AutoClipperService.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,14 @@ protected override IServiceCollection ConfigureServices(IServiceCollection servi
services.AddSingleton<IAzureSpeechTranscriptionService, AzureSpeechTranscriptionService>();
services.AddHttpClient<IClipSegmentationService, ClipSegmentationService>();

// Register Video Indexer client if configured
var videoIndexerAccountId = this.Configuration.GetSection("Service")["AzureVideoIndexerAccountId"];
var videoIndexerApiKey = this.Configuration.GetSection("Service")["AzureVideoIndexerApiKey"];
if (!string.IsNullOrWhiteSpace(videoIndexerAccountId) && !string.IsNullOrWhiteSpace(videoIndexerApiKey))
{
services.AddHttpClient<IAzureVideoIndexerClient, AzureVideoIndexerClient>();
}

// TODO: Figure out how to validate without resulting in aggregating the config values.
// services.AddOptions<AutoClipperOptions>()
// .Bind(this.Configuration.GetSection("Service"))
Expand Down
Loading
Loading