RFC 5610 PROPOSED STANDARD FLOW EXPORT · IPFIX · MONITORING
IPFIX type info: Exporting Type Information for IP Flow Information Export (IPFIX) Information Elements
How an IPFIX exporter can describe its own custom data fields inside the message stream.
RFC EDITOR
Boschi, E., Trammell, B., Mark, L., and T. Zseby, "Exporting Type Information for IP Flow Information Export (IPFIX) Information Elements", RFC 5610, DOI 10.17487/RFC5610, July 2009, <https://www.rfc-editor.org/info/rfc5610>.
BIBTEX
@misc{rfc5610,
series = {Request for Comments},
number = 5610,
howpublished = {RFC 5610},
publisher = {RFC Editor},
doi = {10.17487/RFC5610},
url = {https://www.rfc-editor.org/info/rfc5610},
author = {E. Boschi and B. Trammell and L. Mark and T. Zseby},
title = {{Exporting Type Information for IP Flow Information Export (IPFIX) Information Elements}},
pagetotal = 20,
year = 2009,
month = jul,
} RFC text
Contents
Network Working Group E. Boschi
Request for Comments: 5610 B. Trammell
Category: Standards Track Hitachi Europe
L. Mark
Fraunhofer IFAM
T. Zseby
Fraunhofer FOKUS
July 2009
Exporting Type Information for
IP Flow Information Export (IPFIX) Information Elements
Abstract
This document describes an extension to the IP Flow Information
Export (IPFIX) protocol, which is used to represent and transmit data
from IP flow measurement devices for collection, storage, and
analysis, to allow the encoding of IPFIX Information Model properties
within an IPFIX Message stream. This enables the export of extended
type information for enterprise-specific Information Elements and the
storage of such information within IPFIX Files, facilitating
interoperability and reusability among a wide variety of applications
and tools.
Status of This Memo
This document specifies an Internet standards track protocol for the
Internet community, and requests discussion and suggestions for
improvements. Please refer to the current edition of the "Internet
Official Protocol Standards" (STD 1) for the standardization state
and status of this protocol. Distribution of this memo is unlimited.
Copyright Notice
Copyright (c) 2009 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents in effect on the date of
publication of this document (http://trustee.ietf.org/license-info).
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. IPFIX Documents Overview . . . . . . . . . . . . . . . . . 4
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Type Information Export . . . . . . . . . . . . . . . . . . . 5
3.1. informationElementDataType . . . . . . . . . . . . . . . . 5
3.2. informationElementDescription . . . . . . . . . . . . . . 6
3.3. informationElementName . . . . . . . . . . . . . . . . . . 7
3.4. informationElementRangeBegin . . . . . . . . . . . . . . . 7
3.5. informationElementRangeEnd . . . . . . . . . . . . . . . . 7
3.6. informationElementSemantics . . . . . . . . . . . . . . . 8
3.7. informationElementUnits . . . . . . . . . . . . . . . . . 9
3.8. privateEnterpriseNumber . . . . . . . . . . . . . . . . . 9
3.9. Information Element Type Options Template . . . . . . . . 10
3.10. Data Type and Semantics Restrictions . . . . . . . . . . . 12
4. Security Considerations . . . . . . . . . . . . . . . . . . . 13
5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 14
6. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 15
7. References . . . . . . . . . . . . . . . . . . . . . . . . . . 15
7.1. Normative References . . . . . . . . . . . . . . . . . . . 15
7.2. Informative References . . . . . . . . . . . . . . . . . . 16
Appendix A. Examples . . . . . . . . . . . . . . . . . . . . . . 17
1. Introduction
IP Flow Information Export (IPFIX) provides a template mechanism for
the flexible description of Data Records, by defining a record as a
collection of Information Elements defined in an IANA registry,
However, these Templates provide limited information about the type
of described data; indeed, they encode only the size of the fields
defined by these Information Elements. There presently exists no
mechanism to provide full type information for these Information
Elements, as is defined for the Information Elements in the IPFIX
Information Model [RFC5102].
This especially limits the interoperability of enterprise-specific
Information Elements. It is not possible to use analysis tools on
IPFIX records containing these partially defined Information Elements
that have not been developed with a priori knowledge of their types,
since such tools will not be able to decode them; these tools can
only treat and store them as opaque octet arrays. However, if richer
information is available, additional operations such as efficient
storage, display, and limited analysis of records containing
enterprise-specific Information Elements become possible, even for
Collecting Processes that have not been specifically developed to
understand them.
This document defines a general mechanism to encode the full set of
properties available for the definition of Information Elements
within the IPFIX Information Model inline within an IPFIX Message
stream using IPFIX Options. This mechanism may be used to fully
define type information for Information Elements used within a
message stream, without resorting to an external reference or
reliance on out-of-band configuration, thereby improving the
interoperability of enterprise-specific Information Elements.
Note that the solution described in this document is not intended as
a replacement for registration with IANA of generally useful
Information Elements. It introduces overhead and does not lead to
real interoperability as provided by standardization. Therefore, we
highly recommend standardizing all new generally useful Information
Elements by registering them with IANA. Standardization is
straightforward, and the type information that needs to be specified
in order to support the proposed solution provides a perfect basis
for the description required for standardizing the Information
Element.
It might happen that an Information Element previously described by
the mechanism in this document later becomes an IANA-registered,
standard Information Element. In such environments, old and new
versions of the Information Element can coexist. A translation
between Information Elements expressed by the described solution and
standardized Information Elements is therefore not necessary and is
out of scope for this document.
1.1. IPFIX Documents Overview
"Specification of the IP Flow Information Export (IPFIX) Protocol for
the Exchange of IP Traffic Flow Information" [RFC5101] (informally,
the IPFIX Protocol document) and its associated documents define the
IPFIX Protocol, which provides network engineers and administrators
with access to IP traffic flow information.
"Architecture for IP Flow Information Export" [RFC5470] (the IPFIX
Architecture document) defines the architecture for the export of
measured IP flow information out of an IPFIX Exporting Process to an
IPFIX Collecting Process, and the basic terminology used to describe
the elements of this architecture, per the requirements defined in
"Requirements for IP Flow Information Export" [RFC3917]. The IPFIX
Protocol document [RFC5101] then covers the details of the method for
transporting IPFIX Data Records and Templates via a congestion-aware
transport protocol from an IPFIX Exporting Process to an IPFIX
Collecting Process.
"Information Model for IP Flow Information Export" [RFC5102]
(informally, the IPFIX Information Model document) describes the
Information Elements used by IPFIX, including details on Information
Element naming, numbering, and data type encoding.
This document references the Protocol and Architecture documents for
terminology and extends the IPFIX Information Model to provide new
Information Elements for the representation of Information Element
properties. It draws data type definitions and data type semantics
definitions from the Information Model; the encodings of these data
types are defined in [RFC5101].
2. Terminology
Terms used in this document that are defined in the Terminology
section of the IPFIX Protocol [RFC5101] document are to be
interpreted as defined there.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119].
3. Type Information Export
This section describes the mechanism used to encode Information
Element type information within an IPFIX Message stream. This
mechanism consists of an Options Template Record used to define
Information Element type records, and a set of Information Elements
required by these type records. We first specify the necessary
Information Elements, followed by the structure of the Options
Template describing the type records.
Note that Information Element type records require one Information
Element, informationElementId, that is defined in the Packet Sampling
(PSAMP) Information Model [RFC5477]. This Information Element
supports references only to IANA-defined Information Elements; the
privateEnterpriseNumber Information Element is required alongside
informationElementId to describe enterprise-specific Information
Elements.
3.1. informationElementDataType
Description: A description of the abstract data type of an IPFIX
information element. These are taken from the abstract data types
defined in Section 3.1 of the IPFIX Information Model [RFC5102];
see that section for more information on the types described
below. This field may take the values defined in Table 1 below.
+-------+----------------------+
| Value | Description |
+-------+----------------------+
| 0 | octetArray |
| 1 | unsigned8 |
| 2 | unsigned16 |
| 3 | unsigned32 |
| 4 | unsigned64 |
| 5 | signed8 |
| 6 | signed16 |
| 7 | signed32 |
| 8 | signed64 |
| 9 | float32 |
| 10 | float64 |
| 11 | boolean |
| 12 | macAddress |
| 13 | string |
| 14 | dateTimeSeconds |
| 15 | dateTimeMilliseconds |
| 16 | dateTimeMicroseconds |
| 17 | dateTimeNanoseconds |
| 18 | ipv4Address |
| 19 | ipv6Address |
+-------+----------------------+
Table 1: IE Data Type Values
These types are registered in the IANA IPFIX Information Element
Data Type subregistry. This subregistry is intended to assign
numbers for type names, not to provide a mechanism for adding data
types to the IPFIX Protocol, and as such requires a Standards
Action [RFC5226] to modify.
Abstract Data Type: unsigned8
ElementId: 339
Status: current
Reference: Section 3.1 of the IPFIX Information Model [RFC5102]
3.2. informationElementDescription
Description: A UTF-8 [RFC3629] encoded Unicode string containing a
human-readable description of an Information Element. The content
of the informationElementDescription MAY be annotated with one or
more language tags [RFC4646], encoded in-line [RFC2482] within the
UTF-8 string, in order to specify the language in which the
description is written. Description text in multiple languages
MAY tag each section with its own language tag; in this case, the
description information in each language SHOULD have equivalent
meaning. In the absence of any language tag, the "i-default"
[RFC2277] language SHOULD be assumed. See the Security
Considerations (Section 4) for notes on string handling for
Information Element type records.
Abstract Data Type: string
ElementId: 340
Status: current
3.3. informationElementName
Description: A UTF-8 [RFC3629] encoded Unicode string containing
the name of an Information Element, intended as a simple
identifier. See the Security Considerations (Section 4) for notes
on string handling for Information Element type records.
Abstract Data Type: string
ElementId: 341
Status: current
3.4. informationElementRangeBegin
Description: Contains the inclusive low end of the range of
acceptable values for an Information Element.
Abstract Data Type: unsigned64
Data Type Semantics: quantity
ElementId: 342
Status: current
3.5. informationElementRangeEnd
Description: Contains the inclusive high end of the range of
acceptable values for an Information Element.
Abstract Data Type: unsigned64
Data Type Semantics: quantity
ElementId: 343
Status: current
3.6. informationElementSemantics
Description: A description of the semantics of an IPFIX Information
Element. These are taken from the data type semantics defined in
Section 3.2 of the IPFIX Information Model [RFC5102]; see that
section for more information on the types described below. This
field may take the values in Table 2 below. The special value
0x00 (default) is used to note that no semantics apply to the
field; it cannot be manipulated by a Collecting Process or File
Reader that does not understand it a priori.
+-------+--------------+
| Value | Description |
+-------+--------------+
| 0 | default |
| 1 | quantity |
| 2 | totalCounter |
| 3 | deltaCounter |
| 4 | identifier |
| 5 | flags |
+-------+--------------+
Table 2: IE Semantics Values
These semantics are registered in the IANA IPFIX Information
Element Semantics subregistry. This subregistry is intended to
assign numbers for semantics names, not to provide a mechanism for
adding semantics to the IPFIX Protocol, and as such requires a
Standards Action [RFC5226] to modify.
Abstract Data Type: unsigned8
ElementId: 344
Status: current
Reference: Section 3.2 of the IPFIX Information Model [RFC5102]
3.7. informationElementUnits
Description: A description of the units of an IPFIX Information
Element. These correspond to the units implicitly defined in the
Information Element definitions in Section 5 of the IPFIX
Information Model [RFC5102]; see that section for more information
on the types described below. This field may take the values in
Table 3 below; the special value 0x00 (none) is used to note that
the field is unitless.
+-------+---------------+---------------------------+
| Value | Name | Notes |
+-------+---------------+---------------------------+
| 0 | none | |
| 1 | bits | |
| 2 | octets | |
| 3 | packets | |
| 4 | flows | |
| 5 | seconds | |
| 6 | milliseconds | |
| 7 | microseconds | |
| 8 | nanoseconds | |
| 9 | 4-octet words | for IPv4 header length |
| 10 | messages | for reliability reporting |
| 11 | hops | for TTL |
| 12 | entries | for MPLS label stack |
+-------+---------------+---------------------------+
Table 3: IE Units Values
These types are registered in the IANA IPFIX Information Element
Units subregistry; new types may be added on a First Come First
Served [RFC5226] basis.
Abstract Data Type: unsigned16
ElementId: 345
Status: current
Reference: Section 5 of the IPFIX Information Model [RFC5102]
3.8. privateEnterpriseNumber
Description: A private enterprise number, as assigned by IANA.
Within the context of an Information Element Type record, this
element can be used along with the informationElementId element to
scope properties to a specific Information Element. To export
type information about an IANA-assigned Information Element, set
the privateEnterpriseNumber to 0, or do not export the
privateEnterpriseNumber in the type record. To export type
information about an enterprise-specific Information Element,
export the enterprise number in privateEnterpriseNumber, and
export the Information Element number with the Enterprise bit
cleared in informationElementId. The Enterprise bit in the
associated informationElementId Information Element MUST be
ignored by the Collecting Process.
Abstract Data Type: unsigned32
Data Type Semantics: identifier
ElementId: 346
Status: current
Reference: Sections 3.2 and 3.4.1 of the IPFIX Protocol [RFC5101];
Section 8.2.3 of the PSAMP Information Model [RFC5477].
3.9. Information Element Type Options Template
The Information Element Type Options Template attaches type
information to Information Elements used within Template Records, as
scoped to an Observation Domain within a Transport Session. This
provides a mechanism for representing an IPFIX Information Model
inline within an IPFIX Message stream. Data Records described by
this template are referred to as Information Element type records.
In deployments in which interoperability across vendor
implementations of IPFIX is important, an Exporting Process exporting
data using Templates containing enterprise-specific Information
Elements SHOULD export an Information Element type record for each
enterprise-specific Information Element it exports. Collecting
Processes MAY use these type records to improve handling of unknown
enterprise-specific Information Elements. Exporting Processes using
enterprise-specific Information Elements to implement proprietary
features MAY omit type records for those Information Elements.
Information Element type records MUST be handled by Collecting
Processes as scoped to the Transport Session in which they are sent;
this facility is not intended to provide a method for the permanent
definition of Information Elements.
Similarly, for security reasons, type information for a given
Information Element MUST NOT be redefined by Information Element type
records, and a Collecting Process MUST NOT allow an Information
Element type record to replace its own internal definition of an
Information Element. Information Element type records SHOULD NOT be
duplicated in a given Observation Domain within a Transport Session.
Once an Information Element type record has been exported for a given
Information Element within a given Transport Session, all subsequent
type records for that Information Element MUST be identical.
Information Elements for which a Collecting Process receives
conflicting semantic or type information MUST be ignored.
Note that while this template MAY be used to export information about
any Information Element, including those registered with IANA,
Exporting Processes SHOULD NOT export any type records that could be
reasonably assumed to duplicate type information available at the
Collecting Process. This mechanism is not intended as a replacement
for Exporting and Collecting Processes keeping up to date with
changes to the IANA registry; such an update mechanism is out of
scope for this document.
The template SHOULD contain the Information Elements in Table 4,
below, as defined in the PSAMP Information Model [RFC5477] and in
this document, above.
+-------------------------------+-----------------------------------+
| IE | Description |
+-------------------------------+-----------------------------------+
| informationElementId [scope] | The Information Element |
| | identifier of the Information |
| | Element described by this type |
| | record. This Information Element |
| | MUST be defined as a Scope Field. |
| | See the PSAMP Information Model |
| | [RFC5477] for a definition of |
| | this field. |
| privateEnterpriseNumber | The Private Enterprise number of |
| [scope] | the Information Element described |
| | by this type record. This |
| | Information Element MUST be |
| | defined as a Scope Field. |
| informationElementDataType | The storage type of the specified |
| | Information Element. |
| informationElementSemantics | The semantic type of the |
| | specified Information Element. |
| informationElementUnits | The units of the specified |
| | Information Element. This |
| | element SHOULD be omitted if the |
| | Information Element is a unitless |
| | quantity, or a not a quantity or |
| | counter. |
+-------------------------------+-----------------------------------+
| IE (Continued) | Description (Continued) |
+-------------------------------+-----------------------------------+
| informationElementRangeBegin | The low end of the range of |
| | acceptable values for the |
| | specified Information Element. |
| | This element SHOULD be omitted if |
| | the beginning of the Information |
| | Element's acceptable range is |
| | defined by its data type. |
| informationElementRangeEnd | The high end of the range of |
| | acceptable values for the |
| | specified Information Element. |
| | This element SHOULD be omitted if |
| | the end Information Element's |
| | acceptable range is defined by |
| | its data type. |
| informationElementName | The name of the specified |
| | Information Element. |
| informationElementDescription | A human-readable description of |
| | the specified Information |
| | Element. This element MAY be |
| | omitted in the interest of export |
| | efficiency. |
+-------------------------------+-----------------------------------+
Table 4: IE Type Options
3.10. Data Type and Semantics Restrictions
Note that the informationElementSemantics values defined in Section
3.2 of [RFC5102] are primarily intended to differentiate semantic
interpretation of numeric values, and that not all combinations of
the informationElementDataType and informationElementSemantics
Information Elements are valid; e.g., a counter cannot be encoded as
an IPv4 address. The following are acceptable values of
informationElementSemantics:
o Any value is valid for unsigned informationElementDataType values
("unsigned8", "unsigned16", "unsigned32", or "unsigned64").
o Any value except "flags" is valid for signed
informationElementDataType values ("signed8", "signed16",
"signed32", or "signed64").
o Any value except "identifier" or "flags" is valid for floating-
point informationElementDataType values ("float32" or "float64").
o Only "default" is valid for all other informationElementDataType
values ("octetArray", "boolean", "macAddress", "string",
"dateTimeSeconds", "dateTimeMilliseconds", "dateTimeMicroseconds",
"dateTimeNanoseconds", "ipv4Address", or "ipv6Address").
Information Element type records containing invalid combinations of
informationElementSemantics and informationElementDataType MUST NOT
be sent by Exporting Processes, and MUST be ignored by Collecting
Processes.
Future Standards Actions that modify the Information Element Data
Type subregistry or the Information Element Semantics subregistry
should contain a Data Type and Semantics Restrictions section such as
this one to define allowable combinations of type and semantics
information.
4. Security Considerations
The same security considerations as for the IPFIX Protocol [RFC5101]
apply.
In addition, attention must be paid to the handling of Information
Element type records at the Collecting Process. Type information
precedence rules defined above (a Collecting Process' current
knowledge overrides type records; types are not redefinable during a
session) are designed to minimize the opportunity for an attacker to
maliciously redefine the data model.
Note that Information Element type records may contain two strings
describing Information Elements: informationElementName and
informationElementDescription. IPFIX strings on the wire are length-
prefixed and UTF-8 [RFC3629] encoded, most often within an IPFIX
variable-length Information Element, which mitigates the risk of
unterminated-string attacks against IPFIX Collecting Processes.
However, care should still be taken when handling strings within the
type system of the Collecting Process.
First, Collecting Processes should pay particular attention to buffer
sizes converting between length-prefixed and null-terminated strings.
Exporting Processes MUST NOT export, and Collecting Processes MUST
ignore, any informationElementName or informationElementDescription
content that contains null characters (U+0000) in order to ensure
buffer and string lengths are consistent.
Also, note that there is no limit to IPFIX string length beyond that
inherent in the protocol. The maximum IPFIX string length is 65512
octets (maximum message length (65535), minus message header (16),
minus set header (4), minus long variable length field (3)).
Specifically, although the informationElementName of all IANA
Information Elements at the time of this writing is less than about
40 octets, and the informationElementDescription is less than 4096
octets, either of these Information Elements may contain strings up
to 65512 octets long.
5. IANA Considerations
This document specifies several new IPFIX Information Elements in the
IPFIX Information Element registry as defined in Section 3 above.
IANA has assigned the following Information Element numbers for their
respective Information Elements as specified below:
o Information Element Number 339 for the informationElementDataType
Information Element
o Information Element Number 340 for the
informationElementDescription Information Element
o Information Element Number 341 for the informationElementName
Information Element
o Information Element Number 342 for the
informationElementRangeBegin Information Element
o Information Element Number 343 for the informationElementRangeEnd
Information Element
o Information Element Number 344 for the informationElementSemantics
Information Element
o Information Element Number 345 for the informationElementUnits
Information Element
o Information Element Number 346 for the privateEnterpriseNumber
Information Element
IANA has created an Information Element Data Type subregistry for the
values defined for the informationElementDataType Information
Element. Entries may be added to this subregistry subject to a
Standards Action [RFC5226].
IANA has created an Information Element Semantics subregistry for the
values defined for the informationElementSemantics Information
Element. Entries may be added to this subregistry subject to a
Standards Action [RFC5226].
IANA has created an Information Element Units subregistry for the
values defined for the informationElementUnits Information Element.
Entries may be added to this subregistry on an Expert Review
[RFC5226] basis.
6. Acknowledgements
Thanks to Paul Aitken and Gerhard Muenz for the detailed reviews, and
to David Moore for first raising this issue to the IPFIX mailing
list. Thanks to the PRISM project for its support of this work.
7. References
7.1. Normative References
[RFC5101] Claise, B., "Specification of the IP Flow Information
Export (IPFIX) Protocol for the Exchange of IP Traffic
Flow Information", RFC 5101, January 2008.
[RFC5102] Quittek, J., Bryant, S., Claise, B., Aitken, P., and J.
Meyer, "Information Model for IP Flow Information Export",
RFC 5102, January 2008.
[RFC5477] Dietz, T., Claise, B., Aitken, P., Dressler, F., and G.
Carle, "Information Model for Packet Sampling Exports",
RFC 5477, March 2009.
[RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO
10646", STD 63, RFC 3629, November 2003.
[RFC2277] Alvestrand, H., "IETF Policy on Character Sets and
Languages", BCP 18, RFC 2277, January 1998.
[RFC2482] Whistler, K. and G. Adams, "Language Tagging in Unicode
Plain Text", RFC 2482, January 1999.
[RFC4646] Phillips, A. and M. Davis, "Tags for Identifying
Languages", BCP 47, RFC 4646, September 2006.
[RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an
IANA Considerations Section in RFCs", BCP 26, RFC 5226,
May 2008.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
7.2. Informative References
[RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander,
"Requirements for IP Flow Information Export (IPFIX)",
RFC 3917, October 2004.
[RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek,
"Architecture for IP Flow Information Export", RFC 5470,
March 2009.
Appendix A. Examples
The following example illustrates how the type information extension
mechanism defined in this document may be used to describe the
semantics of enterprise-specific Information Elements. The
Information Elements used in this example are as follows:
o initialTCPFlags, an example private IE 14, 1 octet, the TCP flags
on the first TCP packet in the flow.
o unionTCPFlags, an example private IE 15, 1 octet, the union of the
TCP flags on all packets after the first TCP packet in the flow.
An Exporting Process exporting flows containing these Information
Elements might use a Template like the following:
1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 2 | Length = 52 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 256 | Field Count = 9 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| flowStartSeconds 150 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| sourceIPv4Address 8 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| destinationIPv4Address 12 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| sourceTransportPort 7 | Field Length = 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| destinationTransportPort 11 | Field Length = 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| octetTotalCount 85 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1| (initialTCPFlags) 14 | Field Length = 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Private Enterprise Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1| (unionTCPFlags) 15 | Field Length = 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Private Enterprise Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| protocolIdentifier 4 | Field Length = 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1: Template with Enterprise-Specific IEs
However, a Collecting Process receiving Data Sets described by this
Template can only treat the enterprise-specific Information Elements
as opaque octets; specifically, there is no hint to the collector
that they contain flag information. To use the type information
extension mechanism to address this problem, the Exporting Process
would first export the Information Element Type Options Template
described in Section 3.9 above:
1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 3 | Length = 26 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 257 | Field Count = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope Field Count = 2 |0| priv.EnterpriseNumber 346 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 |0| informationElementId 303 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 2 |0| inf.El.DataType 339 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 1 |0| inf.El.Semantics 344 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 1 |0| inf.El.Name 341 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 65536 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2: Example Information Element Type Options Template
Then, the Exporting Process would export two records described by the
Example Information Element Type Options Template to describe the
enterprise-specific Information Elements:
1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 257 | Length = 50 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Private Enterprise Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|X| IE 14 |0x01 unsigned8 |0x05 flags |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 15 length | |
+-+-+-+-+-+-+-+-+ |
| "initialTCPFlags" |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Private Enterprise Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|X| IE 15 |0x01 unsigned8 |0x05 flags |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 13 length | |
+-+-+-+-+-+-+-+-+ "unionTCPFlags" |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3: Type Record Example
Authors' Addresses
Elisa Boschi
Hitachi Europe
c/o ETH Zurich
Gloriastrasse 35
8092 Zurich
Switzerland
Phone: +41 44 632 70 57
EMail: elisa.boschi@hitachi-eu.com
Brian Trammell
Hitachi Europe
c/o ETH Zurich
Gloriastrasse 35
8092 Zurich
Switzerland
Phone: +41 44 632 70 13
EMail: brian.trammell@hitachi-eu.com
Lutz Mark
Fraunhofer Institute for Manufacturing Technology
and Applied Materials Research
Wiener Str. 12
28359 Bremen
Germany
Phone: +49 421 2246206
EMail: lutz.mark@ifam.fraunhofer.de
Tanja Zseby
Fraunhofer Institute for Open Communication Systems
Kaiserin-Augusta-Allee 31
10589 Berlin
Germany
Phone: +49 30 3463 7153
EMail: tanja.zseby@fokus.fraunhofer.de
AI ELI5
On this page
PROBLEMIPFIX templates only tell a collector the size of each field, so a collector cannot decode enterprise-specific Information Elements it has never heard of and must treat them as opaque octets.
MECHANISMRFC 5610 defines eight new Information Elements and an Options Template that lets the exporter send type records saying each element's name, data type, semantics, units, range and description.
WHY IT MATTERSCollectors and IPFIX Files can store, display and do limited analysis on unfamiliar elements. It does not replace registering useful elements with IANA.
§1The Problem With Opaque Fields
IP Flow Information Export (IPFIX) describes each Data Record with a Template: a list of Information Elements taken from an IANA registry. But a Template encodes only the size of each field. It carries no full type information, although the IPFIX Information Model in RFC 5102 defines such information for registered elements.
This mostly hurts enterprise-specific Information Elements. Analysis tools built without advance knowledge of them can only treat and store their values as opaque octet arrays. With richer information, efficient storage, display and limited analysis become possible even for a Collecting Process that was not written to understand those elements.
RFC 5610 therefore defines a way to encode the properties of Information Elements inline in an IPFIX Message stream using IPFIX Options, with no external reference or out-of-band configuration.
If a sender uses a made-up kind of number, the receiver only sees a pile of bytes. This RFC lets the sender explain what each pile means.
§1.1Scope and Standardization
The RFC says plainly that this is not a replacement for registering generally useful Information Elements with IANA. The mechanism adds overhead and does not give the real interoperability that standardization does, so the authors highly recommend standardizing all new generally useful elements. The type information needed here is a good basis for the description a registration requires.
If an element once described this way later becomes IANA-registered, old and new versions can coexist, and translating between them is out of scope. The document relies on RFC 5101 (the IPFIX Protocol) for terminology and encodings and on RFC 5102 for data types and semantics.
The note-passing trick is for private, custom numbers. If a number is useful to everybody, it should go in the official list instead.
§3The New Information Elements
The type records are built from eight Information Elements defined here. IANA assigned them numbers 339 to 346.
informationElementDataType(339): the abstract data type, with values 0 to 19 fromoctetArraytoipv6Address.informationElementDescription(340): a UTF-8 human-readable description. It MAY carry in-line language tags, and without one thei-defaultlanguage SHOULD be assumed.informationElementName(341): a UTF-8 name used as a simple identifier.informationElementRangeBegin(342) andinformationElementRangeEnd(343): the inclusive low and high ends of acceptable values.informationElementSemantics(344): values 0 to 5, fromdefaulttoflags.informationElementUnits(345): values 0 to 12, fromnonetoentries.privateEnterpriseNumber(346): scopes the properties to a specific enterprise element.
The record also needs informationElementId from RFC 5477, which can refer only to IANA-defined elements. To describe an enterprise-specific element, the exporter sends the enterprise number in privateEnterpriseNumber and the element number with the Enterprise bit cleared in informationElementId. The collector MUST ignore that Enterprise bit. For an IANA element, the enterprise number is 0 or is omitted.
Each note has slots: what the number is called, what kind of number it is, what it measures, and what range it can have.
- 0default — No semantics apply; the value cannot be manipulated by a collector that does not understand it in advance.
- 1quantity — The quantity semantics value, defined in the IPFIX Information Model.
- 2totalCounter — The total counter semantics value, defined in the IPFIX Information Model.
- 3deltaCounter — The delta counter semantics value, defined in the IPFIX Information Model.
- 4identifier — The identifier semantics value, defined in the IPFIX Information Model.
- 5flags — A set of flags.
§3.9The Type Options Template
An Information Element Type Options Template attaches type information to the elements used in Templates, scoped to an Observation Domain within a Transport Session. Records it describes are called Information Element type records. The template SHOULD contain informationElementId and privateEnterpriseNumber, which MUST both be Scope Fields, plus data type, semantics, units, range begin, range end, name and description. Units SHOULD be omitted for a unitless quantity or a non-quantity, range ends SHOULD be omitted when the data type already defines them, and the description MAY be omitted for export efficiency.
Where cross-vendor interoperability matters, an Exporting Process using enterprise-specific elements SHOULD export a type record for each one. Collectors MAY use them to handle unknown elements better. Exporters using enterprise elements for proprietary features MAY omit them.
Type records are scoped to the Transport Session and are not a way to define elements permanently. Exporters SHOULD NOT export records that duplicate information the collector can reasonably be assumed to have, such as for IANA elements.
The sender first announces the form for the notes, then sends one filled-in note per custom number. The notes only count for this one conversation.
§3.10Consistency and Valid Combinations
Once a type record has been exported for an element within a Transport Session, all later type records for it MUST be identical, and duplicates in a given Observation Domain SHOULD be avoided. Type information MUST NOT be redefined by type records, and a collector MUST NOT let a record replace its own internal definition. An element with conflicting semantic or type information MUST be ignored.
Not every pairing of data type and semantics is valid. Any semantics are valid for the unsigned types. Any except flags are valid for the signed types. Any except identifier or flags are valid for float32 and float64. All other types allow only default. Exporters MUST NOT send invalid combinations, and collectors MUST ignore them.
Once a note says what a number is, it cannot change its mind. Also, some labels make no sense together, like calling an address a counter.
| Data type | Valid semantics |
|---|---|
| unsigned8, 16, 32, 64 | Any value |
| signed8, 16, 32, 64 | Any except flags |
| float32, float64 | Any except identifier or flags |
| octetArray, boolean, macAddress, string, date and time types, ipv4Address, ipv6Address | Only default |
§7Worked Example: TCP Flags
The appendix uses two example private elements: initialTCPFlags (private IE 14, 1 octet), the TCP flags on the first TCP packet of the flow, and unionTCPFlags (private IE 15, 1 octet), the union of the flags on all later packets. A normal Template with these elements gives a collector no hint that they hold flags.
To fix that, the exporter first sends the Type Options Template, then two type records. Each record gives the enterprise number, the element number, the data type unsigned8 (0x01), the semantics flags (0x05) and the element name. A collector now knows to treat the byte as a set of flags.
A mystery byte arrives. The note says it is a row of on and off switches, so the receiver knows to read it as switches rather than as a plain number.
Exporting ProcessCollecting Process
- Template with private IEs 14 and 15Only field sizes are described.
- Type Options TemplateDefines the layout of type records.
- Two type recordsEach says unsigned8, flags, and the element name.
§4Security Considerations
The security considerations of RFC 5101 apply. In addition, the precedence rules (a collector's own knowledge wins, and types cannot be redefined during a session) are designed to limit the chance that an attacker maliciously redefines the data model.
The name and description are strings. IPFIX strings are length-prefixed and UTF-8 encoded, which reduces the risk of unterminated-string attacks, but collectors should still be careful converting to null-terminated strings. Exporters MUST NOT export, and collectors MUST ignore, any name or description containing a null character (U+0000). The maximum IPFIX string length is 65512 octets, so either string can in principle be that long.
Receivers should not let a note rewrite what they already know, and should reject notes with sneaky blank characters inside the text.
§8References
[RFC5101]RFC 5101Claise, B., "Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information", RFC 5101, January 2008.January 2008[RFC5102]RFC 5102Quittek, J., Bryant, S., Claise, B., Aitken, P., and J. Meyer, "Information Model for IP Flow Information Export", RFC 5102, January 2008.January 2008[RFC5477]RFC 5477Dietz, T., Claise, B., Aitken, P., Dressler, F., and G. Carle, "Information Model for Packet Sampling Exports", RFC 5477, March 2009.March 2009[RFC5470]RFC 5470Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, "Architecture for IP Flow Information Export", RFC 5470, March 2009.March 2009[RFC5226]RFC 5226Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008.May 2008