RTP CellB: RTP Payload Format of Sun's CellB Video Encoding
How to chop Sun's CellB video stream into RTP packets that survive packet loss.
Speer, M. and D. Hoffman, "RTP Payload Format of Sun's CellB Video Encoding", RFC 2029, DOI 10.17487/RFC2029, October 1996, <https://www.rfc-editor.org/info/rfc2029>.
@misc{rfc2029,
series = {Request for Comments},
number = 2029,
howpublished = {RFC 2029},
publisher = {RFC Editor},
doi = {10.17487/RFC2029},
url = {https://www.rfc-editor.org/info/rfc2029},
author = {M. Speer and D. Hoffman},
title = {{RTP Payload Format of Sun's CellB Video Encoding}},
pagetotal = 6,
year = 1996,
month = oct,
} RFC text
Contents
Network Working Group M. Speer
Request for Comment: 2029 D. Hoffman
Category: Standards Track Sun Microsystems, Inc.
October 1996
RTP Payload Format of Sun's CellB Video Encoding
Status of this Memo
This document specifies an Internet standards track protocol for the
Internet community, and requests discussion and suggestions for
improvements. Please refer to the current edition of the "Internet
Official Protocol Standards" (STD 1) for the standardization state
and status of this protocol. Distribution of this memo is unlimited.
Abstract
This memo describes a packetization scheme for the CellB video
encoding. The scheme proposed allows applications to transport CellB
video flows over protocols used by RTP. This document is meant for
implementors of video applications that want to use RTP and CellB.
1. Introduction
The Cell image compression algorithm is a variable bit-rate video
coding scheme. It provides "high" quality, low bit-rate image
compression at low computational cost. The bytestream that is
produced by the Cell encoder consists of instructional codes and
information about the compressed image.
For futher information on Cell compression technology, refer to [1].
Currently, there are two versions of the Cell compression technology:
CellA and CellB. CellA is primarily designed for the encoding of
stored video intended for local display, and will not be discussed in
this memo.
CellB, derived from CellA, has been optimized for network-based video
applications. It is computationally symmetric in both encode and
decode. CellB utilizes a fixed colormap and vector quantization
techniques in the YUV color space to achieve compression.
2. Network Packetization and Encapsulation
2.1 RTP Usage
The RTP timestamp is in units of 90KHz. The same timestamp value is
used for all packet payloads of a frame. The RTP maker bit denotes
the end of a frame.
2.2 CellB Header
The packetization of the CellB bytestream is designed to make the
resulting packet stream robust to packet loss. To achieve this goal,
an additional header is added to each RTP packet to uniquely identify
the location of the first cell of the packet within the current
frame. In addition, the width and height of the frame in pixels is
carried in each CellB packet header. Although the size can only
change between frames, it is carried in every packet to simplify the
packet encoding.
0 1 2 3
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Cell X Location | Cell Y Location |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Width of Image | Height of Image |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| Compressed CellB Data |
| .... |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
All fields are 16-bit unsigned integers in network byte order, and
are placed at the beginning of the payload for each RTP packet. The
Cell X and the Cell Y Location coordinates are expressed as cell
coordinates, not pixel coordinates. Since cells represent 4x4 blocks
of pixels, the X or Y dimension of the cell coordinates range in
value from 0 through 1/4 of the of the same dimension in pixel
coordinates.
2.3 Packetization Rules
A packet can be of any size chosen by the implementor, up to a full
frame. All multi-byte codes must be completely contained within a
packet. In general, the implementor should avoid packet sizes that
result in fragmentation by the network.
3. References
1. "Cell Image Compression Byte Stream Description,"
ftp://playground.sun.com:/pub/multimedia/video/
cellbytestream.ps.Z
2. Turletti, T., and C. Huitema, "RTP Payload Format
for H.261 Video Streams", RFC 2032, October 1996.
3. Schulzrinne, H., Casner, S., Frederick, R., and
V. Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", RFC 1889, January 1996.
4. Schulzrinne, H., "RTP Profile for Audio and Video
Conferences with Minimal Control", RFC 1890,
January 1996.
4 Authors' Addresses
Michael F. Speer
Sun Microsystems Computer Corporation
2550 Garcia Ave MailStop UMPK14-305
Mountain View, CA 94043
Voice: +1 415 786 6368
Fax: +1 415 786 6445
EMail: michael.speer@eng.sun.com
Don Hoffman
Sun Microsystems Computer Corporation
2550 Garcia Ave MailStop UMPK14-305
Mountain View, CA 94043
Voice: +1 415 786 6370
Fax: +1 415 786 6445
EMail: don.hoffman@eng.sun.com
Appendix A - Structure of the CellB Video Stream
The CellB bytestream consists of cell codes, skip codes and
quantization-table specific codes. These are now described.
A.1 CellB Cell Code
Cell codes are 4 bytes in length, and describe a 4x4 pixel cell.
There are two possible luminance (Y) levels for each cell, but only
one pair of chrominance (UV) values. The CellB cell code is shown
below:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 M M M M M M M M M M M M M M M|U V U V U V U V|Y Y Y Y Y Y Y Y|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
4x4 Bitmask U/V Code Y/Y Code
The first two bytes of the cell code are a bitmask. Each bit in the
mask represents a pixel in a 16-pixel cell. Bit 0 represents the
value of the upper right-hand pixel of the cell, and subsequent bits
represent the pixels in row-major order. If a pixel's bit is set in
the 4x4 Bitmask, then the pixel will be rendered with the pixel value
<Y(1), U, V>. If the pixel's bit is not set in the bitmask, then the
pixel's value will be rendered with the value <Y(0), U, V>. The
bitmask for the cell is normalized so that the most significant bit
is always 0 (i.e., corresponding to <Y(0), U, V>).
The U/V field of the cell code represents the chrominance component.
This code is in index into a table of vectors that represents two
independent components of chrominance.
The Y/Y field of the cell code represents two luminance values (Y(0)
and Y(1)). This code is an index into a table of two-compoment
luminance vectors.
The derivation of the U/V and Y/Y tables is outside the scope of this
memo. A complete discussion can be found in [1].
A.2 CellB Skip Code
The single byte CellB skip code tells the CellB decoder to skip the
next S+1 cells in the current video frame being decoded. The maximum
number of cells that can be skipped is 32. The format of the skip
code is shown below.
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|1 0 0 S S S S S|
+-+-+-+-+-+-+-+-+
A.3 CellB Y/Y Table Code
The single byte "new Y/Y table" code is used to tell the decoder that
the next 512 bytes are a new Y/Y quantization table. The code and
the representation of the table are shown below. The sample
encoder/decoder pair in this document do not implement this feature
of the CellB compression. However, future CellB codecs may implement
this feature.
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|1 1 1 1 1 1 1 0|
+-+-+-+-+-+-+-+-+
The format of the new Y/Y table is:
0 1 2 3
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Y1_000 | Y2_000 | Y1_001 | Y2_001 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
.
.
.
0 1 2 3
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Y1_254 | Y2_254 | Y1_255 | Y2_255 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
A.4 CellB U/V Table Code
The single byte "new U/V table" code is used to tell the decoder that
the next 512 bytes represent a new U/V quantization table. The code
is shown below. The sample encoder/decoder pair provided in this
document do not implement this feature of the CellB compression.
However, future CellB codecs may implement this feature.
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|1 1 1 1 1 1 1 1|
+-+-+-+-+-+-+-+-+
The bytes of the new U/V quantization table are arranged as:
0 1 2 3
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| U_000 | V_000 | U_001 | V_001 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
.
.
.
0 1 2 3
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| U_254 | V_254 | U_255 | V_255 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Appendix B - Availability of CellB
It is the viewpoint of Sun Microsystems, Inc, that CellB is
publically available for use without any license.
AI ELI5
On this page
§1What CellB Is
RFC 2029 describes a packetization scheme for the CellB video encoding from Sun Microsystems. It is meant for implementors of video applications that want to use RTP and CellB.
The Cell image compression algorithm is a variable bit-rate video coding scheme. It aims at high quality and low bit-rate at low computational cost. There are two versions, CellA and CellB. CellA is designed for stored video shown locally and is not covered. CellB is derived from CellA and optimized for network-based video. It is computationally symmetric, so encoding and decoding cost about the same. It uses a fixed colormap and vector quantization in the YUV color space.
CellB is a way of squeezing video so it is small enough to send over a network. This document explains how to put that squeezed video into packets.
§2.1How RTP Is Used
The RTP timestamp for CellB is in units of 90KHz. All packets that carry parts of the same frame use the same timestamp value. The RTP marker bit marks the end of a frame, so a receiver knows when a frame is complete.
Every packet from the same picture gets the same time label. The last packet of a picture gets a special flag that says it is the end.
§2.2The CellB Header
To make the packet stream robust to packet loss, an extra header is added to each RTP packet. It identifies where the first cell of the packet sits within the current frame. It also carries the width and height of the frame in pixels. The size can only change between frames, but it is sent in every packet to keep the encoding simple.
The header has four fields, Cell X Location, Cell Y Location, Width of Image and Height of Image. All are 16-bit unsigned integers in network byte order, placed at the start of each RTP payload, followed by the compressed CellB data.
The location fields are in cell coordinates, not pixel coordinates. Cells are 4x4 blocks of pixels, so each cell coordinate ranges from 0 up to 1/4 of the same dimension in pixels.
Each packet starts with a note saying which tile of the picture comes first and how big the whole picture is. Then a packet that arrives alone still knows where it belongs.
| Field | Name | Size | Value | Description |
|---|---|---|---|---|
CELL-X | Cell X Location | 2 bytes | Column of the first cell in this packet, in cell coordinates. | |
CELL-Y | Cell Y Location | 2 bytes | Row of the first cell in this packet, in cell coordinates. | |
WIDTH | Width of Image | 2 bytes | Frame width in pixels. | |
HEIGHT | Height of Image | 2 bytes | Frame height in pixels. |
§2.3Packetization Rules
A packet can be any size the implementor chooses, up to a full frame. The one hard rule is that all multi-byte codes must be completely contained within a single packet, so a code is never split across two packets.
As a general guideline, the implementor should avoid packet sizes that cause fragmentation by the network.
You can put as many tiles in a packet as you like, but you must never cut one tile's instructions in half between two packets.
§5The CellB Byte Stream
The appendix describes the stream inside the packets. It consists of cell codes, skip codes and quantization-table codes.
A cell code is 4 bytes and describes one 4x4 pixel cell. The first two bytes are a bitmask with one bit per pixel. A set bit means the pixel uses luminance Y(1), and a clear bit means Y(0). The bitmask is normalized so its most significant bit is always 0. The next byte is a U/V code, an index into a table of chrominance vectors. The last byte is a Y/Y code, an index into a table of two-component luminance vectors. Deriving these tables is outside the scope of the memo.
A one-byte skip code tells the decoder to skip the next S+1 cells in the frame, and at most 32 cells can be skipped.
Two one-byte codes announce a new quantization table: 11111110 for a new Y/Y table and 11111111 for a new U/V table. Each is followed by 512 bytes of table data. The sample encoder and decoder do not implement these, though future CellB codecs may.
Each tile is described by a small code: which pixels are light or dark and which colour to use. There is also a shortcut code that says to skip tiles that did not change.
| Field | Name | Size | Value | Description |
|---|---|---|---|---|
ZERO | Zero bit | 1 bits | Always 0; the bitmask is normalized. | |
MASK | Bitmask | 15 bits | Which pixels of the 4x4 cell use Y(1) rather than Y(0). | |
UV | U/V Code | 1 byte | Index into the chrominance vector table. | |
YY | Y/Y Code | 1 byte | Index into the luminance vector table. |
§6Availability and Status
Appendix B records the view of Sun Microsystems that CellB is publicly available for use without any license. The RFC is a Standards Track document, and its metadata lists it as a Proposed Standard.
The company that made CellB says anyone can use it for free.