VideoPlayback (Next)

The value channel carries one of two payloads, discriminated structurally:

• a video-atlas manifest from video-atlas-encode (video-utils), which is the current path and is described below;
• a legacy adaptive video catalogue from adaptive-video-encode, which still renders the original stub.

An atlas is ONE composited video stream containing every seat as a tile, so the browser uses a single hardware decoder regardless of seat count. The widget clips each seat’s letterboxed content rect onto its own canvas.

Summary

Migration status

migrated , reference: tools/arbex/reference/next/VideoPlayback.json. See Dataservice migration status.

Layout

Clock sync

When syncWithClock is true (default):

1. Inbound: scrubber follows realm clock elapsedMs / play state from outside this widget.
2. Outbound: scrub / optional play-pause call clock.seek / start / pause.
3. No feedback loop: seeks and play actions initiated by VideoPlayback set a short-lived flag so the matching clock echo tick/state is ignored.
4. Origin: elapsed time maps onto the atlas through the realm clock’s originMs (wall-clock at elapsed 0). The dataservice sets it from the scenario instance for the default clock, and for a channel-derived range from the range start: used as it is when it is already an epoch value, or added to the scenario start when the producer emits session-relative times (the OdySSEy Lite convention, where sessionFromIso is that zero point). When the clock carries no origin the player assumes the manifest epochMs is the realm start, which is exact only on a dimension the atlas alone defines. This is what lets the same channel play inside a realm-filtered layout (a vignette modal) as well as on its own dimension.

When syncWithClock is false, scrubber state is local only.

Inside a ComposableChart

The same manifest channel can render as a video_playback slice inside a ComposableChart. The slice draws the seat tiles as a non-synced panel that follows the chart’s realm clock, and it pairs with a playback slice for the scrubber and events_lane slices for per-seat gaze lanes.

Pick the slice inside a vignette modal (a realm-filtered layout with defaultRealm set to the vignette realm) so playback, seeking and buffering stay within the vignette. Pick the column when the page is the atlas’s own dimension and the user should scrub the whole session. Both consume the manifest unchanged; do not re-base epochMs for either.

Gaze overlay (inside a ComposableChart)

The video_playback slice can draw a gaze marker on each seat tile. Set videoPlayback.gazeChannel on the slice element to the URI (<realm:>id:name, or a bare id when id and name are identical) of a value channel holding a gaze-points payload, and list that channel on the chart column channels. The standalone VideoPlayback column does not bind this channel; the overlay is slice only. Omit gazeChannel for video without the overlay.

{
  "formatVersion": 1,
  "eye": "left",
  "bucketMs": 200,
  "projection": { "hFovDeg": 90, "vFovDeg": 90, "model": "perspective", "forward": "-z", "vFrom": "bottom" },
  "byOrigin": {
    "<originUuid>": { "t": [0, 200, 400], "u": [0.48, 0.51, 0.55], "v": [0.52, 0.5, 0.47] }
  }
}

Publish the payload in every realm the slice is mounted in, clipped to that realm’s span. A seat with no track shows no marker; a placeholder region still gets one. In the OdySSEy Lite template the channel id and name are both gaze_points, group video.

Data contract: video-atlas manifest

{
  "formatVersion": 1,
  "geometryGeneration": 0,
  "epochMs": 1787564288000,
  "indexEpochMs": 1787564288000,
  "windowMs": 8000,
  "leadInMs": 1000,
  "fps": 15,
  "codecMime": "video/mp4; codecs=\"avc1.4D401F\"",
  "avccMd5": "dfdac36713332fe923b91977cc1f8aa8",
  "geometry": {
    "atlasWidth": 960, "atlasHeight": 480, "cols": 2, "rows": 1,
    "regions": [
      {
        "originUuid": "d6c024dc-...", "streamName": "eye.left",
        "entityName": "Varjo_Data", "label": "Aircraft 1",
        "sourceWidth": 1088, "sourceHeight": 1072,
        "tile": [0, 0, 480, 480], "content": [0, 4, 480, 472]
      }
    ]
  },
  "initOutputId": 341439,
  "windows": [
    {
      "index": 0, "startMs": 1787564288000, "endMs": 1787564296000,
      "empty": false, "blank": false, "outputId": 341433,
      "inputHash": "b3478b7293930c76dc6ee85af3dc26fb",
      "residency": { "d6c024dc-.../eye.left": { "frameCount": 718, "firstMs": 0, "lastMs": 0, "coveredMs": 0 } }
    }
  ]
}

Fields that carry contract weight

Region fields

One entry per seat in geometry.regions. Region order is display order: the player lays tiles out in array order, and reordering is safe because window residency is keyed by originUuid/streamName, not by array position. Producers that know the seating may set label and sort mapped seats first in role order. For example the OL EWFT script takes the label from the origin-to-station binding in the session model (each TFTD capture origin republishes its own simulator’s DIS, so the application id is the station), which is why OL tiles read “Aircraft N” rather than the device name. Producers may also append placeholder regions for seats they know about but have no video for. Label, order and placeholders are the only reshaping of the manifest a producer may do; every other field passes through unchanged.

The timeline is contiguous

Every window between the first and last carries media, black where there is no data. This is deliberate: a hole in the MSE buffer does not read as “no video” to a browser. Playback stalls at readyState 1 and does not resume on its own. Blank windows compress to a few KB each, which is a far smaller cost than an unplayable timeline.

A window is likewise never withheld because one of the N seats is missing. Dropouts are per-seat and are not simultaneous, so a partial window renders the seats that have data and blacks the ones that do not.

Windows may be sparse and negative

Window index is relative to indexEpochMs, which is pinned once and never moves. A retroactive upload of start-of-session data therefore produces NEGATIVE indices. Clients must not assume indices start at zero, are contiguous in the array, or are sorted; select windows by time range (startMs / endMs), never by index arithmetic, and compute span from max(endMs) rather than the last array element.

Fragment bytes

Fragments and the init segment are fetched by outputId through the v2 node-output passthrough (/api/v2/node-instance/outputs/{id}/data), or by relative media URL in the offline development rig. Exactly one of the two is set.

Data contract: legacy adaptive video catalogue

{
  "artefactKind": "adaptive-video-catalogue",
  "status": "pending",
  "streams": [
    {
      "name": "eye.left",
      "codec": "H264",
      "width": 1280,
      "height": 720,
      "chunks": [],
      "hlsManifestUrl": null,
      "dashManifestUrl": null
    }
  ],
  "durationMs": 900000,
  "note": "Chunks are short H264 segments; ladders produced after stitch/catalogue."
}

Capture video arrives as chunked multi-stream H264 (StreamComponent / VideoSource), not a single contiguous MP4. Ladders and chunk lists are filled by video-utils over time (pending → partial → ready).

Example

Maximal layout + channel: VideoPlayback.json. RAG: tools/arbex/rag/reference/components/VideoPlayback.md.

Related

• Next components index
• Dataservice migration status
• ComposableChart video_playback slice (bounded playback inside a chart stack)
• Video Utils runner
• video-atlas-encode (the producer: topology, tuning, memory)