Skip to content

Audio Dubbing feature #387

@jmavescodex-arch

Description

@jmavescodex-arch

Problem / Gaps:
Per-cell recorded audio, timestamps, and video playback are not synchronized end-to-end, causing misaligned narration. Overlaps between adjacent cells are blocked, and VTT exports with overlaps are inconsistently ingested by downstream systems.

Hypothesis:
If we enable timecode-locked per-cell audio with video playback sync, allow intentional overlaps with clear UI, and provide VTT export options to preserve or trim overlaps, then dubbing teams will complete alignment and export steps more smoothly with fewer reworks because audio, timing, and outputs stay coordinated.

Test / Measurement Plan:
Verify that pressing Play within a cell starts the video at the cell start and plays the cell’s recorded audio in sync, including after timestamp edits. Confirm users can create and review overlaps without blockers, and that both preserve and trim VTT exports are valid and ingest cleanly in downstream systems with no regressions in non-overlap projects.

Criteria for Resolution:

  • Pressing Play in a cell starts video at the cell’s start time and plays the cell’s recorded audio in sync, with an option to reduce/duck original video audio
  • Changing a cell’s start/end remaps its recorded audio to the new timestamps for playback even before saving. This allow the user to fine tune the start and end time and see/hear the result without having to save the edit.
  • Users can create overlaps between adjacent cells; UI clearly indicates overlap regions without blocking edits; playback honors overlaps(this would mean multiple cell audio could be playing at once with the video when previewing playback)
  • VTT export offers Preserve Overlaps and Trim Overlaps; preserve exports cues as authored; trim produces non-overlapping cues that ingest successfully in downstream systems
  • clean up video player: currently there is a drag and clunky drop time stamp editor that is displayed under the video. I don't think this is necessary any more because each cell can edit its own timestamp. However it might help the user as a source of navigation. It would be nice if we had a system like this that the user could use to scrub through the video and click on the sub title. Clicking on the subtitle could scroll to the cell and open it.

Interface / Airtable URL:
https://airtable.com/appfEUXIPganSIP89/pagyewiWtbGAM67M4/recj03dF1l9UiVc9x?home=pag3CmPvhPQfuxCwC

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions