DetNet IP OAM: Operations, Administration, and Maintenance (OAM) for Deterministic Networking (DetNet) with the IP Data Plane
Testing and managing networks that promise to deliver packets on time.
Mirsky, G., Chen, M., and D. Black, "Operations, Administration, and Maintenance (OAM) for Deterministic Networking (DetNet) with the IP Data Plane", RFC 9634, DOI 10.17487/RFC9634, October 2024, <https://www.rfc-editor.org/info/rfc9634>.
@misc{rfc9634,
series = {Request for Comments},
number = 9634,
howpublished = {RFC 9634},
publisher = {RFC Editor},
doi = {10.17487/RFC9634},
url = {https://www.rfc-editor.org/info/rfc9634},
author = {G. Mirsky and M. Chen and D. Black},
title = {{Operations, Administration, and Maintenance (OAM) for Deterministic Networking (DetNet) with the IP Data Plane}},
pagetotal = 8,
year = 2024,
month = oct,
} RFC text
Contents
Internet Engineering Task Force (IETF) G. Mirsky
Request for Comments: 9634 Ericsson
Category: Informational M. Chen
ISSN: 2070-1721 Huawei
D. Black
Dell EMC
October 2024
Operations, Administration, and Maintenance (OAM) for Deterministic
Networking (DetNet) with the IP Data Plane
Abstract
This document discusses the use of existing IP Operations,
Administration, and Maintenance protocols and mechanisms in
Deterministic Networking networks that use the IP data plane.
Status of This Memo
This document is not an Internet Standards Track specification; it is
published for informational purposes.
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Not all documents
approved by the IESG are candidates for any level of Internet
Standard; see Section 2 of RFC 7841.
Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
https://www.rfc-editor.org/info/rfc9634.
Copyright Notice
Copyright (c) 2024 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Revised BSD License text as described in Section 4.e of the
Trust Legal Provisions and are provided without warranty as described
in the Revised BSD License.
Table of Contents
1. Introduction
2. Conventions Used in This Document
2.1. Terminology
3. Active OAM for DetNet Networks with the IP Data Plane
3.1. Mapping Active OAM and IP DetNet Flows
3.2. Active OAM Using IP-in-UDP Encapsulation
3.3. Active OAM Using DetNet-in-UDP Encapsulation
3.4. The Application of Y.1731/G.8013 Using GRE-in-UDP
Encapsulation
4. Active OAM for DetNet IP Interworking with OAM for Non-IP
DetNet Domains
5. IANA Considerations
6. Security Considerations
7. References
7.1. Normative References
7.2. Informative References
Authors' Addresses
1. Introduction
[RFC8655] introduces and explains the Deterministic Networking
(DetNet) architecture.
Operations, Administration, and Maintenance (OAM) protocols are used
to detect and localize defects in the network as well as to monitor
network performance. Some OAM functions (e.g., failure detection)
work in the network proactively, while others (e.g., defect
localization) are usually performed on demand. These tasks are
achieved by a combination of active and hybrid OAM methods, as
defined in [RFC7799].
[RFC9551] lists the OAM functional requirements for DetNet and
defines the principles for OAM use within DetNet networks utilizing
the IP data plane. The functional requirements can be compared
against current OAM tools to identify gaps and potential enhancements
required to enable proactive and on-demand path monitoring and
service validation.
This document discusses the use of existing IP OAM protocols and
mechanisms in DetNet networks that use the IP data plane.
2. Conventions Used in This Document
2.1. Terminology
The term "DetNet OAM" as used in this document means "a set of OAM
protocols, methods, and tools for Deterministic Networking".
DetNet: Deterministic Networking
OAM: Operations, Administration, and Maintenance
ICMP: Internet Control Message Protocol
Underlay Network or Underlay Layer: The network that provides
connectivity between DetNet nodes. MPLS networks providing Label
Switched Path (LSP) connectivity between DetNet nodes are an
example of the DetNet IP network underlay layer.
DetNet Node: A node that is an actor in the DetNet domain. DetNet
domain edge nodes and nodes that perform the Packet Replication,
Elimination, and Ordering Functions within the domain are examples
of a DetNet node.
3. Active OAM for DetNet Networks with the IP Data Plane
OAM protocols and mechanisms act within the data plane of the
particular networking layer. Thus, it is critical that the data
plane encapsulation support OAM mechanisms and enable them to be
configured so that DetNet OAM packets follow the same path
(unidirectional or bidirectional) through the network and receive the
same forwarding treatment in the DetNet forwarding sub-layer as the
DetNet flow being monitored.
The DetNet data plane encapsulation in a transport network with IP
encapsulations is specified in Section 6 of [RFC8939]. For the IP
underlay network, DetNet flows are identified by the ordered match to
the provisioned information set that, among other elements, includes
the IP protocol type, source port number, and destination port
number. Active IP OAM protocols like Bidirectional Forwarding
Detection (BFD) [RFC5880] or the Simple Two-way Active Measurement
Protocol (STAMP) [RFC8762] use UDP transport and the well-known UDP
port numbers as the respective destination ports. For BFD, the UDP
destination port is specific to the BFD variant, e.g., multihop BFD
uses port 4784 [RFC5883].
Thus, a DetNet node must be able to associate an IP DetNet flow with
the particular test session to ensure that test packets experience
the same treatment as the DetNet flow packets. For example, in a
network where path selection and DetNet functionality are based on
3-tuples (destination and source IP addresses in combination with the
Differentiated Services Code Point value), that association can be
achieved by having the OAM traffic use the same 3-tuple as the
monitored IP DetNet flow. In such a scenario, an IP OAM session
between the same pair of IP nodes would share the network treatment
with the monitored IP DetNet flow regardless of whether ICMP, BFD, or
STAMP is used.
In IP networks, the majority of on-demand failure detection and
localization is achieved through the use of ICMP, utilizing Echo
Request and Echo Reply messages, along with a set of defined error
messages such as Destination Unreachable, which provide detailed
information through assigned code points. [RFC0792] and [RFC4443]
define ICMP for IPv4 and IPv6 networks, respectively. To utilize
ICMP effectively for these purposes within DetNet, DetNet nodes must
establish the association of ICMP traffic between DetNet nodes with
IP DetNet traffic. This entails ensuring that such ICMP traffic
traverses the same interfaces and receives QoS treatment that is
identical to the monitored DetNet IP flow. Failure to do so may
result in ICMP being unable to detect and localize failures specific
to the DetNet IP data plane.
3.1. Mapping Active OAM and IP DetNet Flows
IP OAM protocols are used to detect failures (e.g., BFD [RFC5880])
and performance degradation (e.g., STAMP [RFC8762]) that affect an IP
DetNet flow. For active OAM to be useful, it is essential to ensure
that specially constructed OAM packets traverse the same set of nodes
and links and receive the same network QoS treatment as the monitored
data flow, e.g., a DetNet flow. When the UDP destination port number
used by the OAM protocol is assigned by IANA, judicious selection of
the UDP source port may help achieve co-routedness of OAM with the
monitored IP DetNet flow in multipath environments, e.g., Link
Aggregation Group or Equal Cost Multipath, via the use of a UDP
source port value that results in the OAM traffic and the monitored
IP DetNet flow hashing to the same path based on the packet header
hashes used for path selection. This does assume that forwarding
equipment along the multipath makes consistent hashing decisions,
which might not always be true in a heterogeneous environment. (That
also applies to the encapsulation techniques described in
Sections 3.2 and 3.3.) To ensure the accuracy of OAM results in
detecting failures and monitoring the performance of IP DetNet, it is
essential that test packets not only traverse the same path as the
monitored IP DetNet flow but also receive the same treatment by the
nodes, e.g., shaping, filtering, policing, and availability of the
pre-allocated resources, as experienced by the IP DetNet packet.
That correlation between the particular IP OAM session and the
monitored IP DetNet flow can be achieved by using DetNet provisioning
information (e.g., [RFC9633]). Each IP OAM protocol session is
presented as a DetNet application with related service and forwarding
sub-layers. The forwarding sub-layer of the IP OAM session is
identical to the forwarding sub-layer of the monitored IP DetNet
flow, except for information in the "ip-header" grouping item as
defined in [RFC9633].
3.2. Active OAM Using IP-in-UDP Encapsulation
As described above, active IP OAM is realized through several
protocols. Some protocols use UDP transport, while ICMP is a
network-layer protocol. The amount of operational work mapping IP
OAM protocols to the monitored DetNet flow can be reduced by using an
IP/UDP tunnel [RFC2003] to carry IP test packets. Then, to ensure
that OAM packets traverse the same set of nodes and links, the IP/UDP
tunnel must be mapped to the monitored DetNet flow. Note that in
such a case, the DetNet domain for the test packet is seen as a
single IP link. As a result, transit DetNet IP nodes cannot be
traced using the usual traceroute procedure, and a modification of
the traceroute may be required.
3.3. Active OAM Using DetNet-in-UDP Encapsulation
Active OAM in IP DetNet can be realized using DetNet-in-UDP
encapsulation. Using a DetNet-in-UDP tunnel between IP DetNet nodes
ensures that active OAM test packets follow the same path through the
network as the monitored IP DetNet flow packets and receive the same
forwarding treatment in the DetNet forwarding sub-layer (see
Section 4.1.2 of [RFC8655]) as the IP DetNet flow being monitored.
[RFC9566] describes how DetNet with the MPLS-over-UDP/IP data plane
[RFC9025] can be used to support Packet Replication, Elimination, and
Ordering Functions (PREOF) to potentially lower packet loss, improve
the probability of on-time packet delivery, and ensure in-order
packet delivery in IP DetNet's service sub-layer. To ensure that an
active OAM test packet follows the path of the monitored DetNet flow
in the DetNet service sub-layer, the encapsulation shown in Figure 1
is used.
+---------------------------------+
| |
| DetNet App-Flow |
| (original IP) Packet |
| |
+---------------------------------+ <--\
| DetNet ACH | |
+---------------------------------+ +--> PREOF-capable
| Service-ID (S-Label) | | DetNet IP data
+---------------------------------+ | plane encapsulation
| UDP Header | |
+---------------------------------+ |
| IP Header | |
+---------------------------------+ <--/
| Data-Link |
+---------------------------------+
| Physical |
+---------------------------------+
Figure 1: DetNet Associated Channel Header Format
* DetNet ACH - the DetNet Associated Channel Header as defined in
[RFC9546].
* PREOF - Packet Replication, Elimination, and Ordering Functions
used in the DetNet service sub-layer as defined in [RFC8655].
3.4. The Application of Y.1731/G.8013 Using GRE-in-UDP Encapsulation
[RFC8086] has defined the method of encapsulating GRE (Generic
Routing Encapsulation) headers in UDP. GRE-in-UDP encapsulation can
be used for IP DetNet OAM, as it eases the task of mapping an OAM
test session to a particular IP DetNet flow that is identified by an
N-tuple. Matching a GRE-in-UDP tunnel to the monitored IP DetNet
flow enables the use of Y.1731/G.8013 [ITU.Y1731] as a comprehensive
toolset of OAM. The Protocol Type field in the GRE header must be
set to 0x8902, assigned by IANA to the IEEE 802.1ag Connectivity
Fault Management (CFM) protocol / ITU-T Recommendation Y.1731.
Y.1731/G.8013 supports the necessary functions required for IP DetNet
OAM, i.e., continuity checks, one-way packet loss, and packet delay
measurements.
4. Active OAM for DetNet IP Interworking with OAM for Non-IP DetNet
Domains
A domain in which the IP data plane provides DetNet service could be
used in conjunction with a Time-Sensitive Networking (TSN) network
and a DetNet domain with the MPLS data plane to deliver end-to-end
service. In such scenarios, the ability to detect defects and
monitor performance using OAM is essential. [RFC9546] identifies two
OAM interworking models -- peering and tunneling. Interworking
between DetNet domains with IP and MPLS data planes is analyzed in
Section 4.2 of [RFC9546]. In addition, OAM interworking requirements
and recommendations that apply between a DetNet domain with the MPLS
data plane and an adjacent TSN network also apply between a DetNet
domain with the IP data plane and an adjacent TSN network.
5. IANA Considerations
This document has no IANA actions.
6. Security Considerations
This document describes the applicability of the existing Fault
Management and Performance Monitoring IP OAM protocols. It does not
raise any security concerns or issues in addition to those common to
networking or already documented in [RFC0792], [RFC4443], [RFC5880],
and [RFC8762] for the referenced DetNet and OAM protocols.
7. References
7.1. Normative References
[RFC0792] Postel, J., "Internet Control Message Protocol", STD 5,
RFC 792, DOI 10.17487/RFC0792, September 1981,
<https://www.rfc-editor.org/info/rfc792>.
[RFC2003] Perkins, C., "IP Encapsulation within IP", RFC 2003,
DOI 10.17487/RFC2003, October 1996,
<https://www.rfc-editor.org/info/rfc2003>.
[RFC4443] Conta, A., Deering, S., and M. Gupta, Ed., "Internet
Control Message Protocol (ICMPv6) for the Internet
Protocol Version 6 (IPv6) Specification", STD 89,
RFC 4443, DOI 10.17487/RFC4443, March 2006,
<https://www.rfc-editor.org/info/rfc4443>.
[RFC8086] Yong, L., Ed., Crabbe, E., Xu, X., and T. Herbert, "GRE-
in-UDP Encapsulation", RFC 8086, DOI 10.17487/RFC8086,
March 2017, <https://www.rfc-editor.org/info/rfc8086>.
[RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas,
"Deterministic Networking Architecture", RFC 8655,
DOI 10.17487/RFC8655, October 2019,
<https://www.rfc-editor.org/info/rfc8655>.
[RFC8939] Varga, B., Ed., Farkas, J., Berger, L., Fedyk, D., and S.
Bryant, "Deterministic Networking (DetNet) Data Plane:
IP", RFC 8939, DOI 10.17487/RFC8939, November 2020,
<https://www.rfc-editor.org/info/rfc8939>.
[RFC9025] Varga, B., Ed., Farkas, J., Berger, L., Malis, A., and S.
Bryant, "Deterministic Networking (DetNet) Data Plane:
MPLS over UDP/IP", RFC 9025, DOI 10.17487/RFC9025, April
2021, <https://www.rfc-editor.org/info/rfc9025>.
[RFC9546] Mirsky, G., Chen, M., and B. Varga, "Operations,
Administration, and Maintenance (OAM) for Deterministic
Networking (DetNet) with the MPLS Data Plane", RFC 9546,
DOI 10.17487/RFC9546, February 2024,
<https://www.rfc-editor.org/info/rfc9546>.
[RFC9633] Geng, X., Ryoo, Y., Fedyk, D., Rahman, R., and Z. Li,
"Deterministic Networking (DetNet) YANG Data Model",
RFC 9633, DOI 10.17487/RFC9633, October 2024,
<https://www.rfc-editor.org/info/rfc9633>.
7.2. Informative References
[ITU.Y1731]
ITU-T, "Operation, administration and maintenance (OAM)
functions and mechanisms for Ethernet-based networks",
ITU-T Recommendation G.8013/Y.1731, June 2023.
[RFC5880] Katz, D. and D. Ward, "Bidirectional Forwarding Detection
(BFD)", RFC 5880, DOI 10.17487/RFC5880, June 2010,
<https://www.rfc-editor.org/info/rfc5880>.
[RFC5883] Katz, D. and D. Ward, "Bidirectional Forwarding Detection
(BFD) for Multihop Paths", RFC 5883, DOI 10.17487/RFC5883,
June 2010, <https://www.rfc-editor.org/info/rfc5883>.
[RFC7799] Morton, A., "Active and Passive Metrics and Methods (with
Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799,
May 2016, <https://www.rfc-editor.org/info/rfc7799>.
[RFC8762] Mirsky, G., Jun, G., Nydell, H., and R. Foote, "Simple
Two-Way Active Measurement Protocol", RFC 8762,
DOI 10.17487/RFC8762, March 2020,
<https://www.rfc-editor.org/info/rfc8762>.
[RFC9551] Mirsky, G., Theoleyre, F., Papadopoulos, G., Bernardos,
CJ., Varga, B., and J. Farkas, "Framework of Operations,
Administration, and Maintenance (OAM) for Deterministic
Networking (DetNet)", RFC 9551, DOI 10.17487/RFC9551,
March 2024, <https://www.rfc-editor.org/info/rfc9551>.
[RFC9566] Varga, B., Farkas, J., and A. Malis, "Deterministic
Networking (DetNet) Packet Replication, Elimination, and
Ordering Functions (PREOF) via MPLS over UDP/IP",
RFC 9566, DOI 10.17487/RFC9566, April 2024,
<https://www.rfc-editor.org/info/rfc9566>.
Authors' Addresses
Greg Mirsky
Ericsson
Email: gregimirsky@gmail.com
Mach(Guoyi) Chen
Huawei
Email: mach.chen@huawei.com
David Black
Dell EMC
176 South Street
Hopkinton, MA 01748
United States of America
Email: david.black@dell.com
AI ELI5
On this page
§1Why OAM Matters in Deterministic Networks
Deterministic Networking guarantees that packets travel through the network in bounded time with minimal loss, making it suitable for critical applications: industrial automation, vehicle control systems, and real-time audio/video. Unlike the best-effort Internet, where some packets may be delayed or dropped and applications adapt, DetNet applications depend on these guarantees.
Operations, Administration, and Maintenance (OAM) tools detect failures, localize problems, and measure performance. In a best-effort network, OAM is nice to have. In DetNet, it is essential: operators must verify that the network is actually meeting its promises. A failure in a DetNet network is not just slow service; it could mean a missed deadline, a safety violation, or a broken system.
The challenge is that DetNet uses special forwarding rules to ensure bounded delay. OAM test packets must follow the same path as user data and receive the same priority, or the test results will be misleading. This document explains how to use existing IP OAM protocols in DetNet networks with the IP data plane.
A network that promises to deliver packets fast needs tools to check that the promise is kept. If you test with packets that get different treatment than real data, the test lies to you. This document shows how to test the right way.
§2OAM Protocols and Mapping to Flows
Existing IP OAM protocols include ICMP (used for ping and traceroute), BFD (Bidirectional Forwarding Detection, for rapid failure detection), and STAMP (Simple Two-way Active Measurement Protocol, for delay and loss measurement). These protocols work by sending specially crafted test packets that trigger responses, allowing operators to probe the network.
The fundamental challenge is that DetNet routers forward packets based on specific identifiers called flows. A flow might be identified by a tuple of (source IP, destination IP, protocol type, source port, destination port). User data packets in a DetNet flow follow a carefully provisioned path with reserved bandwidth and bounded delay.
For OAM to work, test packets must be mapped to the same flow as the user data. If an OAM packet uses different port numbers or IP addresses, it may take a different path or receive different treatment, making test results invalid. The solution is to carefully choose the IP and UDP headers of OAM packets so they match the DetNet flow being tested, or to encapsulate them in a tunnel that itself matches the flow.
A network treats packets differently depending on where they come from and where they go. To test a route, your test packet needs to look like the real packets so it gets the same treatment. Otherwise, your test does not tell you anything useful.
§3Encapsulation Techniques
The RFC describes several ways to encapsulate OAM packets so they follow DetNet flows:
Direct IP-level mapping: OAM protocols like BFD use UDP with well-known port numbers (e.g., port 4784 for multihop BFD). A router can identify a BFD test packet as belonging to a DetNet flow by matching the flow's source and destination IPs, protocol type, and UDP port numbers. This requires the least overhead but requires careful coordination of port numbers.
IP-in-UDP tunneling: An entire IP packet (including ICMP or other OAM protocol) can be encapsulated inside a UDP packet. The outer UDP/IP headers are chosen to match the DetNet flow, so the tunnel itself gets the right treatment. The DetNet network sees only the outer headers and forwards accordingly. This adds overhead but provides flexibility: the inner OAM packet can use any protocol.
DetNet-in-UDP tunneling: FROST (Packet Replication, Elimination, and Ordering Functions) encapsulation is used to ensure OAM test packets follow the exact same path within the DetNet service layer as user data. This is the most robust approach for networks using PREOF redundancy.
GRE-in-UDP for Y.1731: The Y.1731 Ethernet OAM protocol can be carried inside Generic Routing Encapsulation (GRE) inside UDP. This allows using a comprehensive OAM toolkit (continuity checks, loss measurement, delay measurement) while meeting DetNet routing requirements.
You can send OAM test packets inside a DetNet-compatible tunnel so they get routed the same way as real data. Different tunneling methods work for different situations: some are simple, others are more robust for special features like redundancy.
§4Connecting DetNet with Other Networks
A DetNet network does not stand alone. It might need to deliver packets through Time-Sensitive Networking (TSN) domains (for wired local-area connectivity in industrial settings) or MPLS DetNet domains (for WAN interconnection). When DetNet with the IP data plane interworks with these other technologies, OAM must also interwork.
The RFC identifies two interworking models: peering (where DetNet and the other network exchange information directly) and tunneling (where DetNet encapsulates traffic for the other network). In either case, OAM functions must be preserved: the ability to detect faults and measure performance must carry through the entire path.
For IP DetNet interworking with MPLS DetNet or TSN, the same OAM mapping principles apply: test packets must be provisioned so they follow the same path and receive the same treatment as user data across all network domains.
A DetNet network might connect to other special-delivery networks. When it does, the monitoring has to work end-to-end. A packet might go through DetNet, then TSN, then MPLS. The test packets need to follow the same route all the way.