Samples

Raw unencoded audio and video samples

VideoSample

Represents a raw, unencoded video frame. Provides methods for creating, manipulating, and converting video frames across different formats.

Constructor

new VideoSample(init: VideoSampleInit)

init · VideoSampleInit · required

Initialization options for the video sample.

Creating VideoSamples

From VideoFrame

VideoSample.fromVideoFrame(frame: VideoFrame, options?: VideoSampleInit): VideoSample

Creates a VideoSample from a VideoFrame.

From Canvas

VideoSample.fromCanvas(
  canvas: HTMLCanvasElement | OffscreenCanvas,
  options?: VideoSampleInit
): VideoSample

Creates a VideoSample by rendering a canvas element.

From Raw Pixels

VideoSample.fromPixels(
  data: AllowSharedBufferSource,
  options: SetRequired<VideoSampleInit, 'format' | 'codedWidth' | 'codedHeight'>
): VideoSample

Creates a VideoSample from raw pixel data.

Properties

format · VideoSamplePixelFormat

The internal pixel format in which the frame is stored. One of 21 supported formats including I420, NV12, RGBA, etc.

codedWidth · number

The width of the frame in pixels.

codedHeight · number

The height of the frame in pixels.

displayWidth · number

The display width accounting for rotation and pixel aspect ratio.

displayHeight · number

The display height accounting for rotation and pixel aspect ratio.

rotation · Rotation

The rotation of the frame in degrees (0, 90, 180, 270), clockwise.

timestamp · number

The presentation timestamp in seconds.

duration · number

The duration in seconds.

colorSpace · VideoSampleColorSpace

The color space information for the frame.

Methods

toVideoFrame()

Converts this sample to a VideoFrame.

toVideoFrame(): VideoFrame

copyToCanvas()

Draws this video sample onto a canvas.

copyToCanvas(
  canvas: HTMLCanvasElement | OffscreenCanvas,
  options?: CanvasRenderingOptions
): void

copyPixelsTo()

Copies the raw pixel data of this sample to a destination buffer.

copyPixelsTo(
  destination: AllowSharedBufferSource,
  options?: VideoSampleCopyToOptions
): Promise<void>

clone()

Creates a copy of this video sample.

clone(): VideoSample

close()

Releases underlying resources. You must call this when you’re done using the sample.

close(): void

Always call close() on VideoSamples when you’re done using them to prevent memory leaks. VideoSamples hold underlying resources that won’t be freed automatically.

AudioSample

Represents a raw, unencoded audio buffer. Provides methods for creating, manipulating, and converting audio data.

Constructor

new AudioSample(init: AudioSampleInit)

init · AudioSampleInit · required

Initialization options for the audio sample.

Creating AudioSamples

From AudioData

AudioSample.fromAudioData(data: AudioData, options?: AudioSampleInit): AudioSample

Creates an AudioSample from an AudioData.

From AudioBuffer

AudioSample.fromAudioBuffer(
  buffer: AudioBuffer,
  options?: AudioSampleInit
): AudioSample

Creates an AudioSample from an AudioBuffer.

From Raw Audio Data

AudioSample.fromData(
  data: AllowSharedBufferSource,
  options: SetRequired<AudioSampleInit, 'format' | 'numberOfChannels' | 'numberOfFrames' | 'sampleRate'>
): AudioSample

Creates an AudioSample from raw audio data.

Properties

format · AudioSampleFormat

The internal format of the audio data (e.g., ‘s16’, ‘f32’, ‘s32’, ‘f32-planar’).

numberOfChannels · number

The number of audio channels (e.g., 2 for stereo).

numberOfFrames · number

The number of audio frames in this sample.

sampleRate · number

The sample rate in Hz (e.g., 48000).

timestamp · number

The presentation timestamp in seconds.

duration · number

The duration in seconds.

Methods

toAudioData()

Converts this sample to an AudioData.

toAudioData(): AudioData

toAudioBuffer()

Converts this sample to an AudioBuffer.

toAudioBuffer(context?: BaseAudioContext): Promise<AudioBuffer>

copyDataTo()

Copies the raw audio data to a destination buffer.

copyDataTo(
  destination: AllowSharedBufferSource,
  options?: AudioSampleCopyToOptions
): void

clone()

Creates a copy of this audio sample.

clone(): AudioSample

close()

Releases underlying resources. You must call this when you’re done using the sample.

close(): void

Always call close() on AudioSamples when you’re done using them to prevent memory leaks.

VideoSamplePixelFormat

The internal pixel format with which a VideoSample is stored. See WebCodecs pixel formats for more details.

type VideoSamplePixelFormat =
  | 'I420' | 'I420P10' | 'I420P12'      // 4:2:0 Y, U, V
  | 'I420A' | 'I420AP10' | 'I420AP12'   // 4:2:0 Y, U, V, A
  | 'I422' | 'I422P10' | 'I422P12'      // 4:2:2 Y, U, V
  | 'I422A' | 'I422AP10' | 'I422AP12'   // 4:2:2 Y, U, V, A
  | 'I444' | 'I444P10' | 'I444P12'      // 4:4:4 Y, U, V
  | 'I444A' | 'I444AP10' | 'I444AP12'   // 4:4:4 Y, U, V, A
  | 'NV12'                               // 4:2:0 Y, UV
  | 'RGBA' | 'RGBX'                      // 4:4:4 RGBA/RGBX
  | 'BGRA' | 'BGRX';                     // 4:4:4 BGRA/BGRX

VideoSampleColorSpace

Color space information for video samples.

type VideoSampleColorSpace = {
  primaries?: 'bt709' | 'bt470bg' | 'smpte170m' | 'bt2020' | 'smpte432';
  transfer?: 'bt709' | 'smpte170m' | 'iec61966-2-1' | 'linear' | 'pq' | 'hlg';
  matrix?: 'rgb' | 'bt709' | 'bt470bg' | 'smpte170m' | 'bt2020-ncl';
  fullRange?: boolean;
}

Example

import { VideoSample, AudioSample } from 'mediabunny';

// Create a video sample from a canvas
const canvas = document.createElement('canvas');
canvas.width = 1920;
canvas.height = 1080;
const ctx = canvas.getContext('2d');
ctx.fillStyle = 'blue';
ctx.fillRect(0, 0, canvas.width, canvas.height);

const videoSample = VideoSample.fromCanvas(canvas, {
  timestamp: 0,
  duration: 1/30
});

console.log(`Video: ${videoSample.codedWidth}x${videoSample.codedHeight}`);
console.log(`Format: ${videoSample.format}`);

// Always close when done
videoSample.close();

// Create an audio sample from an AudioBuffer
const audioContext = new AudioContext();
const audioBuffer = audioContext.createBuffer(2, 48000, 48000);

const audioSample = AudioSample.fromAudioBuffer(audioBuffer, {
  timestamp: 0
});

console.log(`Audio: ${audioSample.numberOfChannels} channels`);
console.log(`Sample rate: ${audioSample.sampleRate}Hz`);

// Always close when done
audioSample.close();

See also

• Packets - Encoded compressed media data
• Packets and samples concept - Understanding the difference
• VideoSampleSource - Add video samples to output
• AudioSampleSource - Add audio samples to output
• VideoSampleSink - Extract video samples from input
• AudioSampleSink - Extract audio samples from input