RTP pointers: RTP Payload Format for Real-Time Pointers
An RTP payload format that sends a presenter's pointer position separately from the slides, in four bytes.
Civanlar, M. and G. Cash, "RTP Payload Format for Real-Time Pointers", RFC 2862, DOI 10.17487/RFC2862, June 2000, <https://www.rfc-editor.org/info/rfc2862>.
@misc{rfc2862,
series = {Request for Comments},
number = 2862,
howpublished = {RFC 2862},
publisher = {RFC Editor},
doi = {10.17487/RFC2862},
url = {https://www.rfc-editor.org/info/rfc2862},
author = {M. Civanlar and G. Cash},
title = {{RTP Payload Format for Real-Time Pointers}},
pagetotal = 7,
year = 2000,
month = jun,
} RFC text
Contents
Network Working Group M. Civanlar
Request for Comments: 2862 G. Cash
Category: Standards Track AT&T
June 2000
RTP Payload Format for Real-Time Pointers
Status of this Memo
This document specifies an Internet standards track protocol for the
Internet community, and requests discussion and suggestions for
improvements. Please refer to the current edition of the "Internet
Official Protocol Standards" (STD 1) for the standardization state
and status of this protocol. Distribution of this memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2000). All Rights Reserved.
Abstract
This document describes an RTP [1] payload format for transporting
the coordinates of a dynamic pointer that may be used during a
presentation. Although a mouse can be used as the pointer, this
payload format is not intended and may not have all functionalities
needed to implement a general mouse event transmission mechanism.
1. Introduction
In most presentations, significant information is conveyed through
the use of viewgraphs and a pointer. This makes accurate transmission
of them vital in remote conferencing applications. Using regular
video of a presenter's display for this purpose is problematic
because, while the viewgraphs require a high spatial resolution, the
pointer movements need to be sampled and transmitted at a high
temporal resolution so that the presenter's pointing actions can be
displayed synchronously with the corresponding audio and video
signals. In many instances, this synchronization carries vital
information. As an example, consider a speaker pointing at two
alternatives on a viewgraph in sequence and saying "this one is
better than this". To satisfy both high spatial and high temporal
resolution requirements, at least S-VHS quality video may need to be
used. Codecs that can compress S-VHS video effectively in real-time
are expensive for this purpose, and transmitting such video
uncompressed requires very high bandwidths.
A much simpler and economical system can be designed by capturing and
transmitting the pointer coordinates separately [2]. The pointer
coordinates with respect to a displayed viewgraph can easily be
obtained in electronic presentation systems. For presentations
prepared for optical systems, such as transparencies for overhead
projectors, an arrangement where the viewgraph is captured in a frame
buffer on a computer can be used to associate the pointer coordinates
with the displayed viewgraph. For capturing transparencies, printed
material, or even three dimensional objects, a document camera and a
personal computer or workstation based video capture card can be
used. This arrangement can handle electronic viewgraphs by feeding
the video output of the computer that displays them to the video
capture card through an appropriate converter also. A side benefit of
this is that it allows using a presenter's own computer to transmit
electronic viewgraphs without connecting it to, for example, an
intranet. The captured image is then displayed along with the
capturing computer's mouse pointer on the presenter's display using a
projector. The presenter moves the pointer on the display using a
regular or maybe a wireless mouse whose location can easily be
captured by appropriate software running on the capturing computer.
This document describes an RTP payload format to transmit the pointer
coordinates captured in one of the ways described above using RTP.
Although, a mouse can be used as the pointer, this payload format is
not intended and may not have all functionalities needed to implement
a general mouse event transmission mechanism.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [3].
2. Payload Format
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC |M| PT | sequence number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: contributing source (CSRC) identifiers :
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
|L|M|R| | x-coordinate | | PIN | y-coordinate |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
MBZ MBZ
Figure 1 - An RTP packet for Real-Time Pointer
Fig. 1 shows an RTP packet carrying real-time pointer coordinates.
This payload format does not have a payload specific header.
2.1. RTP Header Usage:
Payload Type (PT): The assignment of an RTP payload type for this new
packet format is outside the scope of this document, and will not be
specified here. It is expected that the RTP profile for a particular
class of applications will assign a payload type for this encoding,
or if that is not done then a payload type in the dynamic range shall
be chosen.
Marker (M) bit: Set to one if the pointer icon is changed in this
packet.
Extension (X) bit: Defined by the RTP profile used.
Sequence Number: Set as described in RFC1889 [1].
Timestamp: The sampling time for the pointer location measured by a
90kHz clock.
SSRC: Set as described in RFC1889 [1].
CC and CSRC fields are used as described in RFC 1889 [1].
RTCP SHOULD be used as defined in RFC 1889 [1].
2.2. Payload:
The pointer's x and y coordinates are measured from the upper left
corner of the associated display window. They are represented as a
fraction of the corresponding edge length of the display window using
12 bits, positive, fixed point numbers between 0 and (1 - 2^-12).
L (left), R (right) and/or M (middle) bits are pointer special
effects flags. Their use is application dependent and MUST be
established out-of-band. Applications MAY ignore these bits.
PIN: Pointer Icon Number (3 bits) selects a pointer icon. The
association between the PIN numbers and the icon pictures MUST be
established out-of-band. PIN = 0 represents a default pointer icon.
Applications which only support a single pointer icon SHOULD set the
PIN field to zero. Applications MAY ignore non-zero PIN values on
reception, and display a default icon.
3. MIME Media Type Registrations
This document defines a new RTP payload name, "pointer," and
associated MIME subtype, "video/pointer."
3.1. Registration of MIME media type video/pointer
MIME media type name: video
MIME subtype name: pointer
Required parameters: None
Optional parameters: None
Encoding considerations: Pointer video can be transmitted with RTP
as specified in this document.
Security considerations: As described in this document.
Interoperability considerations: None
Published specification: this document.
Applications which use this media type: Videoconferencing systems
that transmit VUgraphs with a real-time pointer.
Additional information: None
Magic number(s): None
File extension(s): None
Macintosh File Type Code(s): None
Person & email address to contact for further information:
M. Reha Civanlar
e-mail: civanlar@research.att.com
Intended usage: COMMON Author/Change controller:
M. Reha Civanlar
e-mail: civanlar@research.att.com
4. Security Considerations
RTP packets using the payload format defined in this specification
are subject to the security considerations discussed in the RTP
specification [1].
This payload type does not exhibit any significant non-uniformity in
the receiver side computational complexity for packet processing to
cause a potential denial-of-service threat.
5. References
[1] Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson,
"RTP: A Transport Protocol for Real Time Applications", RFC 1889,
January 1996.
[2] M. R. Civanlar, G. L. Cash, "Networked Viewgraphs - NetVG"
Proceedings of The 9th Int. Workshop on Packet Video,
http://www.research.att.com/~mrc/PacketVideo99.html.
[3] Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997.
6. Authors' Addresses
M. Reha Civanlar
AT&T Labs - Research
100 Schultz Drive, Room 3-205
Red Bank, NJ 07701, USA
EMail: civanlar@research.att.com
Glenn L. Cash
AT&T Labs - Research
100 Schultz Drive, Room 3-213
Red Bank, NJ 07701, USA
EMail: glenn@research.att.com
7. Full Copyright Statement
Copyright (C) The Internet Society (2000). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFC Editor function is currently provided by the
Internet Society.
AI ELI5
On this page
§1Why Send the Pointer Separately
RFC 2862 defines an RTP payload format for the coordinates of a dynamic pointer used during a presentation. Presentations convey much of their information through viewgraphs and a pointer, so both must be transmitted accurately in remote conferencing.
The problem with plain video of the presenter's display is that viewgraphs need high spatial resolution, while pointer movements must be sampled at high temporal resolution so the pointing shows up in sync with the audio and video. The RFC's example is a speaker pointing at two alternatives in turn while saying "this one is better than this". Doing both with one video stream may need at least S-VHS quality video, and the codecs for that are expensive, while uncompressed video needs very high bandwidth.
The proposed alternative is a simpler and cheaper system that captures and transmits the pointer coordinates separately. The document also says that a mouse can be the pointer, but the format is not intended for, and may lack the functions needed by, a general mouse event transmission mechanism.
Sending a whole movie just to show a moving dot is wasteful. It is cheaper to send only where the dot is, many times a second.
§2How the Coordinates Are Captured
The RFC describes several ways to get pointer coordinates relative to a displayed viewgraph. In electronic presentation systems they are easy to obtain. For optical systems such as overhead transparencies, the viewgraph can be captured into a frame buffer on a computer. A document camera and a video capture card can capture transparencies, printed material or even three dimensional objects, and electronic viewgraphs can be fed in through the capture card too.
A side benefit is that a presenter's own computer can transmit electronic viewgraphs without connecting to, for example, an intranet. The captured image is shown with the capturing computer's mouse pointer on the presenter's display through a projector. The presenter moves the pointer with a regular or wireless mouse, and software on the capturing computer captures its location.
A computer watches the slide and also notices where the presenter's mouse dot is. It sends the dot's position to everyone else.
§3The Payload Format
An RTP packet for this format has the normal RTP header followed directly by the pointer data. The format has no payload specific header.
The payload packs the following into 32 bits: three flag bits L, M and R, a 12-bit x-coordinate, a 3-bit PIN (Pointer Icon Number), and a 12-bit y-coordinate, with the unused bits marked as must-be-zero in the figure.
The x and y values are measured from the upper left corner of the associated display window. Each is a fraction of the corresponding edge length of the window, held as a 12-bit, positive, fixed point number between 0 and (1 - 2^-12).
The L (left), R (right) and M (middle) bits are pointer special effects flags. Their use is application dependent and MUST be established out-of-band, and applications MAY ignore them. The PIN selects a pointer icon; the mapping from numbers to pictures MUST also be set up out-of-band. PIN 0 is the default icon, applications with one icon SHOULD set it to zero, and receivers MAY ignore non-zero values and show a default icon.
Each message is like a tiny note saying how far across and how far down the dot is, which button effects are on, and which dot picture to use.
| Field | Name | Size | Value | Description |
|---|---|---|---|---|
L | L | 1 bits | Left effects flag; application dependent. | |
M | M | 1 bits | Middle effects flag; application dependent. | |
R | R | 1 bits | Right effects flag; application dependent. | |
MBZ1 | MBZ | 1 bits | Must be zero. | |
X | x-coordinate | 12 bits | Fraction of window width from the left edge. | |
MBZ2 | MBZ | 1 bits | Must be zero. | |
PIN | PIN | 3 bits | Pointer Icon Number; 0 is the default icon. | |
Y | y-coordinate | 12 bits | Fraction of window height from the top edge. |
§4RTP Header Usage
Most RTP header fields follow the base RTP specification (RFC 1889): the sequence number, SSRC, CC and CSRC fields are used as described there, and RTCP SHOULD be used as defined there.
The format-specific rules are few. The timestamp is the sampling time of the pointer location, measured by a 90kHz clock. The marker (M) bit is set to one if the pointer icon changes in this packet. The extension (X) bit is defined by the RTP profile in use.
The payload type is not assigned here. The RFC expects the RTP profile for a class of applications to assign one, or else a payload type in the dynamic range to be chosen.
The envelope follows the usual RTP rules. The time stamp says when the dot was at that spot, and a special flag shouts when the dot picture changes.
§5MIME Registration
The document defines a new RTP payload name, pointer, and the associated MIME subtype video/pointer. The registration lists no required or optional parameters and no magic numbers or file extensions. It names videoconferencing systems that transmit viewgraphs with a real-time pointer as the applications that use the type, and gives its intended usage as COMMON.
The format gets an official name tag, video/pointer, so programs can recognize it.
§6Security Considerations
Packets in this format are subject to the security considerations of the RTP specification. The RFC adds that the payload type does not show any significant non-uniformity in receiver computational complexity for packet processing, so it does not create a potential denial-of-service threat from that source.
Handling these small messages is always about equally easy, so nobody can easily overload a receiver with them. Other safety rules come from RTP itself.