From 73f8d22c98bcf94814a2e433d588205c8ed994ec Mon Sep 17 00:00:00 2001 From: Dan Coffey Date: Wed, 11 Feb 2026 15:13:21 -0500 Subject: [PATCH 1/3] examples for timecode metadata --- millicast/playback/frame-metadata.md | 65 +++++++++++++++++++++++++--- 1 file changed, 59 insertions(+), 6 deletions(-) diff --git a/millicast/playback/frame-metadata.md b/millicast/playback/frame-metadata.md index 4d653d1dbe50..45a6286d3431 100644 --- a/millicast/playback/frame-metadata.md +++ b/millicast/playback/frame-metadata.md @@ -30,7 +30,7 @@ import MetadataFeature from '../assets/img/metadata-feature.png'; Metadata is encoded into the frame at broadcast and then extracted by the Client SDK at playback time. -- There is a **timecode** that will be supplied by the broadcaster and carried through the platform and available at playback. +- There is a **timecode** that will be supplied by the broadcaster and carried through the platform and available at playback. (See [Timecode Metadata](#timecode-metadata)) - Application supplied metadata is incorporated as **unregistered** SEI messages. These are received as `Byte` arrays if unable to be converted into JSON objects. - It is recommended to limit the amount of data that is packaged with the frame as this increases the size of each frame for distribution (impacting latency and bandwidth costs). - There may be marginal overhead in encoding and decoding messages, so you must enable **metadata** when establishing a connection if you want to send or receive. @@ -41,14 +41,67 @@ With WebRTC and a UDP connection, latency is prioritized over reliable delivery When receiving metadata there is a `uuid` attribute that can be used to uniquely identify the source of the metadata in cases of multiple publishing sources. The table includes a few examples: -| UUID | Description | -| :----------------------------------- | :--------------------------------------------------- | -| 9a21f3be-31f0-4b78-b0be-c7f7dbb97250 | SEI metadata inserted from AMF OnFi message feeds. | -| d40e38ea-d419-4c62-94ed-20ac37b4e4fa | SEI metadata inserted by the Web SDK. | -| dc45e9bd-e6d9-48b7-962c-d820d923eeef | SEI metadata inserted by libavc such as with ffmpeg. | +| UUID | Description | +| :----------------------------------- | :------------------------------------------------- | +| 9a21f3be-31f0-4b78-b0be-c7f7dbb97250 | SEI metadata inserted from AMF OnFi message feeds. | +| d40e38ea-d419-4c62-94ed-20ac37b4e4fa | SEI metadata inserted by the Web SDK. | For **PIC_TIMING** SEI messages that are inserted by various encoders, there will not be a UUID assigned and included with the frame. +### Timecode Metadata + +There are several ways to send and recieve timecode into the service which can be extracted in the player. + +1. Embed pic_timing (part of SEI metadata) in the `h264` video stream (this is automatically inserted with some encoders) +1. Send RTMP with onFi (part of OnMetaData in AMF metadata). The service will read this value from the RTMP and insert it into the SEI metadata of the timecode (See [Metadata Source Identification](#metadata-source-identification)) + +Here is an example of a frame's metadata that has pic_timing from the SEI in the `metadata` callback: + +```json +{ + "seiPicTimingTimeCodeArray": [ + { + "n_frames": 20, + "seconds_value": 57, + "minutes_value": 51, + "hours_value": 19, + "time_offset": 0 + } + ], + "mid": "0", + "track": {} +} +``` + +If you have metadata from onFi, here is what that metadata will look like (note the UUID indicating the source from onFi): + +```json +{ + "uuid": "9a21f3be-31f0-4b78-b0be-c7f7dbb97250", + "timecode": "2026-02-11T19:49:16.299Z", + "seiPicTimingTimeCodeArray": [ + { + "n_frames": 9, + "seconds_value": 16, + "minutes_value": 49, + "hours_value": 19, + "time_offset": 0 + } + ], + "mid": "0", + "track": {} +} +``` + +Here is an example of consuming this metadata in JavaScript: + +```javascript +millicastView.on('metadata', (metadata) => { + updateMetadataDisplay(metadata); + processTimecodeDisplay(metadata); +}); +``` + ## Web SDK When using the [Web SDK](/millicast/playback/players-sdks/web/sdk/index.mdx) to set and get frame metadata, you must include the `metadata` option to the `connect()` method on both [Publish](https://millicast.github.io/millicast-sdk/Publish.html#connect) and [View](https://millicast.github.io/millicast-sdk/View.html#connect) connections.. From a72817d4b3c5ae86ffbeffcbddb272b982004f12 Mon Sep 17 00:00:00 2001 From: Daniel Coffey Date: Thu, 12 Feb 2026 13:50:43 -0500 Subject: [PATCH 2/3] Update millicast/playback/frame-metadata.md Co-authored-by: Fabien Lavocat <4154532+FabienLavocat@users.noreply.github.com> --- millicast/playback/frame-metadata.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/millicast/playback/frame-metadata.md b/millicast/playback/frame-metadata.md index 45a6286d3431..fba2d57281bd 100644 --- a/millicast/playback/frame-metadata.md +++ b/millicast/playback/frame-metadata.md @@ -50,7 +50,7 @@ For **PIC_TIMING** SEI messages that are inserted by various encoders, there wil ### Timecode Metadata -There are several ways to send and recieve timecode into the service which can be extracted in the player. +There are several ways to send and receive timecode into the service which can be extracted in the player. 1. Embed pic_timing (part of SEI metadata) in the `h264` video stream (this is automatically inserted with some encoders) 1. Send RTMP with onFi (part of OnMetaData in AMF metadata). The service will read this value from the RTMP and insert it into the SEI metadata of the timecode (See [Metadata Source Identification](#metadata-source-identification)) From 49d025ba313920e99148c10bc865d0cb5ef410d8 Mon Sep 17 00:00:00 2001 From: Daniel Coffey Date: Thu, 12 Feb 2026 13:50:57 -0500 Subject: [PATCH 3/3] Update millicast/playback/frame-metadata.md Co-authored-by: Fabien Lavocat <4154532+FabienLavocat@users.noreply.github.com> --- millicast/playback/frame-metadata.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/millicast/playback/frame-metadata.md b/millicast/playback/frame-metadata.md index fba2d57281bd..c7d07598e195 100644 --- a/millicast/playback/frame-metadata.md +++ b/millicast/playback/frame-metadata.md @@ -53,7 +53,7 @@ For **PIC_TIMING** SEI messages that are inserted by various encoders, there wil There are several ways to send and receive timecode into the service which can be extracted in the player. 1. Embed pic_timing (part of SEI metadata) in the `h264` video stream (this is automatically inserted with some encoders) -1. Send RTMP with onFi (part of OnMetaData in AMF metadata). The service will read this value from the RTMP and insert it into the SEI metadata of the timecode (See [Metadata Source Identification](#metadata-source-identification)) +1. Send RTMP with onFi (part of OnMetaData in AMF metadata). The service will read this value from the RTMP stream and insert it into the SEI metadata of the timecode (See [Metadata Source Identification](#metadata-source-identification)) Here is an example of a frame's metadata that has pic_timing from the SEI in the `metadata` callback: