Skip to main content

TN7002: Time Synchronization

Introduction​

For video playback, Recative System split the video file into two different file: the muted video file and the audio file. For most browsers:

  • Muted video files can be played automatically without users' interaction.

  • By interactive the page once (click or tap), the AudioContext could be activated, then Recative System could play any audio files without any limitation.

So we can achieve unlimited auto play by playing muted video and its sound in AudioContext.

Since they are played with two different Web API, the time of two different files could be uncoordinated. So we need a tool to synchronize the time of different channel.

This is managed by the @recative/time-schedule.

Architecture​

Timeline​

For each asset, there is an instance handle all the time synchronization related tasks. This timeline will manage audio, video and logic produced by the act point.

Tracks​

Each timeline managed a lot of Tracks. For all type of asset, there is a monitor track:

  • AudioTrack: Audio track will track the loading and the progress of audio elements for most cases, audio tracks would not be synchronized.

  • RemoteTrack: Remote track will handle general progress management logic for the logic which is hard to abstract, Recative System use Remote Track to manage the progress of video playback and animation of act point.

  • MonitorTrack: Monitor track do not report any progress, but trigger events while time updated and the content stuck.

While adding the track to the timeline, developers could specify a priority for that track, while there is out of sync happened, the time of all other tracks will follow the time of the track with the highest priority. This is designed to minimize the user's perception while time synchronization happens.

Compared to visual information, users are more sensitive to the glitch of auditory information, so we tend to give higher priority to audio information and adjust the timing of visual components rather than sound when the tracks appear uncoordinated.

Integration​

Each ContentInstance have a member called timeline

Most tracks will be added to the timeline while the asset is initialized, but some track could be added later.