Skip to content

Add WHIP/WHEP support for ultra-low-latency streaming, ingest, and playback#225

Closed
anishmenon wants to merge 1 commit intoeerimoq:mainfrom
nexhe:main
Closed

Add WHIP/WHEP support for ultra-low-latency streaming, ingest, and playback#225
anishmenon wants to merge 1 commit intoeerimoq:mainfrom
nexhe:main

Conversation

@anishmenon
Copy link

Summary

Adds full WHIP and WHEP support to Moblin, enabling ultra-low-latency WebRTC streaming in all three directions:

  • WHIP Publishing (Stream out): Stream from Moblin to any WHIP-compatible endpoint (e.g., Cloudflare, Dolby.io, self-hosted servers) — just like RTMP/SRT but with sub-second latency. Available as a stream protocol option under Settings → Streams.
  • WHIP Server (Ingest in): Run a local WHIP server so external apps like OBS or FFmpeg can publish streams into Moblin over the local network. Ingested streams appear as selectable camera sources in scenes. Available under Settings → Ingests.
  • WHEP Client (Playback): Subscribe to remote WebRTC streams via WHEP and use them as camera sources within Moblin. Available under Settings → Ingests.

Motivation

Addresses community feature requests for ultra-low-latency streaming using WebRTC. WHIP/WHEP (RFC 9725 / draft) are the standard HTTP-based signaling protocols for WebRTC, supported by OBS, FFmpeg 8+, Cloudflare, and many other platforms. Compared to RTMP, WHIP delivers significantly lower latency (sub-second vs. 2+ seconds).

What's included

Publishing (stream out):

  • WHIP as a stream protocol alongside RTMP, SRT, and RIST
  • Accepts https://, http://, whip://, and whips:// endpoint URLs
  • H.264 video + Opus audio encoding via WebRTC
  • Configurable ICE servers and retry settings

Ingest (stream in):

  • Built-in WHIP server with HTTP endpoint for receiving streams
  • Full ICE negotiation, trickle ICE (PATCH), and session teardown (DELETE)
  • RTMP-style video decode pipeline (PTS retiming before decode) for smooth real-time playback
  • Configurable per-stream latency buffer

Playback (subscribe):

  • WHEP client for subscribing to external WebRTC sources
  • Configurable URL, latency, and enable/disable per stream

Shared:

  • WHIP/WHEP streams appear as selectable camera sources in scene and widget configuration
  • Vendored HaishinKit.swift with patches for H.264 STAP-A (NAL type 24) decoding and recvonly track lifecycle management in RTCHaishinKit
  • Decode session auto-recovery on persistent errors

Test plan

  • Publish from Moblin via WHIP to a WHIP-compatible server and verify playback on the receiving end
  • Stream from OBS via WHIP to Moblin's WHIP server and verify real-time video/audio in a scene
  • Subscribe to a remote WebRTC source via WHEP and verify playback as a camera source
  • Verify trickle ICE works with clients that send candidates via PATCH
  • Verify session teardown (DELETE) and reconnection for all three modes
  • Verify WHIP/WHEP camera sources are selectable in scene and widget configuration

The key change is making the three use cases (publish, ingest, playback) explicit and distinct so reviewers and users immediately understand what each one does.

…IP publishing as a stream protocol for ultra-low-latencystreaming to WHIP-compatible endpoints. Add a built-in WHIP serverfor receiving WebRTC streams from OBS/FFmpeg as camera sources.Add a WHEP client for subscribing to remote WebRTC streams.- WHIP publishing alongside RTMP, SRT, RIST in stream settings- WHIP server with ICE negotiation, trickle ICE, session teardown- WHEP client with configurable URL and latency per stream- RTMP-style video decode pipeline for smooth ingest playback- H.264 STAP-A decoding and recvonly track fixes in RTCHaishinKit- WHIP/WHEP streams as selectable camera sources in scenes
@datagutt
Copy link

datagutt commented Feb 9, 2026

This seems to overwrite a lot of existing files with older versions. Almost as if this is based on an old Moblin version and not rebased
Also it includes a duplicate copy of Haishinkit in Vendor/ instead of integrating WHIP/WHEP into the existing fork of it.

Oh and it is one big commit so very hard to follow the changes done.

@anishmenon anishmenon closed this Feb 9, 2026
@anishmenon
Copy link
Author

anishmenon commented Feb 9, 2026

Hey, you're absolutely right — sorry about that. The original PR was based on an older fork and included the entire vendored HaishinKit library plus a bunch of unrelated upstream changes mixed in with the WHIP/WHEP code. Totally my mistake.
I've cleaned it up and opened a new PR: #227

It's now split into 2 clean commits on top of latest upstream main:
Commit 1: Vendors HaishinKit.swift as a local package (separate commit so reviewers can skip past it)
Commit 2: The actual WHIP/WHEP feature — 35 files, ~2300 lines. No unrelated changes.
Zero Scoreboard/YouTube/AppleWatch/RemoteControl/Chat noise. Just the WHIP/WHEP code.
I'll close #227. Would appreciate another look at #227 when you get a chance.

#227

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants