RFC 3933 BEST CURRENT PRACTICE IETF · PROCESS · EXPERIMENTS

A Model for IETF Process Experiments

How the IETF can try out a change to its own rules for a limited time before making it permanent.

On rfc-editor.org ↗
PUBLISHED
November 2004
AUTHORS
J. Klensin, S. Dawkins
STREAM
IETF
BCP NUMBER
BCP 93
IETF LAST CALL
Four weeks
SUNSET TIMEOUT
Typically not more than one year after adoption
PUBLISHED AS
Experimental RFC
DOI
10.17487/RFC3933
ERRATA
Reported errata

RFC text

Contents
Original text · RFC Editor View on rfc-editor.org ↗
Network Working Group                                         J. Klensin
Request for Comments: 3933                                    S. Dawkins
BCP: 93                                                    November 2004
Category: Best Current Practice


                  A Model for IETF Process Experiments

Status of this Memo

   This document specifies an Internet Best Current Practices for the
   Internet Community, and requests discussion and suggestions for
   improvements.  Distribution of this memo is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (2004).

Abstract

   The IETF has designed process changes over the last ten years in one
   of two ways: announcement by the IESG, sometimes based on informal
   agreements with limited community involvement and awareness, and
   formal use of the same mechanism used for protocol specification.
   The first mechanism has often proven to be too lightweight, the
   second too heavyweight.

   This document specifies a middle-ground approach to the system of
   making changes to IETF process, one that relies heavily on a "propose
   and carry out an experiment, evaluate the experiment, and then
   establish permanent procedures based on operational experience" model
   rather than those previously attempted.

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2
   2.  Background and Specification . . . . . . . . . . . . . . . . . 2
   3.  Security Considerations  . . . . . . . . . . . . . . . . . . . 5
   4.  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 5
   5.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 5
       5.1.  Normative Reference. . . . . . . . . . . . . . . . . . . 5
       5.2.  Informative References . . . . . . . . . . . . . . . . . 5
   6.  Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 6
       Full Copyright Statement . . . . . . . . . . . . . . . . . . . 7

1.  Introduction

   This document specifies a middle-ground approach to the system of
   making changes to IETF process, one that relies heavily on a "propose
   and carry out an experiment, evaluate the experiment, and then
   establish permanent procedures based on operational experience" model
   rather than those previously attempted.

2.  Background and Specification

   Since the 1992 changes documented in [RFC1396], the IETF has used two
   mechanisms for process changes.

   1. IESG has adopted a number of procedural changes on its own
      initiative and documented them informally, utilizing the wide
      discretion implicitly granted to them by [RFC2026].  This provided
      a lightweight mechanism for change, but the lightness came with a
      cost: There was sometimes too little alignment with the larger
      IETF community.

   2. The IETF has also used the [RFC2026] protocol standards
      development process to identify a community of interest, hold one
      or more BoFs, charter a working group, discuss proposed changes
      within the community, develop IETF-wide consensus on the changes,
      and publish (usually) Best Current Practice specifications.  This
      provided full community involvement but also came with a cost in
      flexibility.  The IETF does not change its formal processes often
      (the IPR clarifications in [RFC3667, RFC3668] are the first
      documented changes to [RFC2026] since 1996), and the community is
      understandably reluctant to permanently alter or extend formally
      adopted processes with untried new procedures.

   There is a middle ground between BCP process updates and informal
   agreements.  This document specifies regularizing and formalizing the
   informal IESG mechanisms listed in 1 above as a means of moving
   forward with procedural changes that might prove valuable.

   The mechanisms outlined here add to the IESG's range of tools for
   dealing with process issues on an ongoing basis.  They supplement the
   existing tools rather than attempting to replace them with a single
   "magic bullet".  The choice of using the procedure outlined in this
   document or other mechanisms available to the IESG and the community
   -- present or future -- remains in the IESG's hands.  If the IESG
   does not exercise this discretion wisely, this document provides no
   additional remedies.

   Some have interpreted the current procedures as giving the IESG all
   of the capabilities outlined here.  If this were true, this document
   only encourages the IESG to use this type of mechanism more
   frequently in preference to less streamlined ones, and to more
   explicitly document its actions and decisions.

   This specification permits and encourages the IESG to adopt and
   institute "process experiments" by using the following procedure:

   1. An I-D is written describing the proposed new or altered
      procedure.  A statement of the problem expected to be resolved is
      desirable but not required (the intent is to keep the firm
      requirements for such an experiment as lightweight as possible).
      Similarly, specific experimental or evaluative criteria, although
      highly desirable, are not required -- for some of the process
      changes we anticipate, having the IESG reach a conclusion at the
      end of the sunset period that the community generally believes
      things to be better (or worse) will be both adequate and
      sufficient.  The I-D must state an explicit "sunset" timeout
      typically, not to exceed one year after adoption.

   2. If the IESG believes the proposal is plausible and plausibly
      useful, a four-week IETF Last Call is initiated.  The IESG can
      institute whatever procedures it wishes to make this determination
      and to avoid denial of service attacks from large numbers of
      spurious or unimportant proposals.  In particular, they might
      institute a procedure requiring a number of endorsements, or
      endorsements of a particular type, before the IESG considers the
      proposal.  The IESG is, however, expected to understand that
      procedures or review processes that act as a mechanism for
      significant delays do not fall within the intent of this
      specification.

   3. At the conclusion of the Last Call, the IESG reevaluates the
      plausibility and appropriateness of the proposal.  If they
      conclude that the proposed experiment is appropriate, a second I-D
      is generated (either by the IESG or by the original authors with
      IESG advice) that cleans up any definitional issues exposed in the
      Last Call and that explicitly identifies and responds to issues
      raised during the Last Call.

   4. The document and experiment are then announced, the experiment is
      begun, and the document is forwarded for publication as an
      Experimental RFC.

   The IESG is explicitly authorized to use this mechanism (based on
   Experimental RFCs) to gain experience with proposed changes to BCP
   specifications.  There is no requirement to approve a BCP
   specification for the experiment until the experiment is found to
   have value.

   The IESG could, of course, reach a "bad idea" conclusion at any stage
   in this process and abandon the experiment.  It might recommend
   publication of the experimental document, with a discussion of why it
   was a bad idea, but is not required to do so.  The list above is
   deliberately vague about where the I-Ds come from: a WG, design team,
   individual contribution, editing group, or other mechanism could be
   used in the first and/or third steps, but no specific mechanisms are
   required, and the IESG is explicitly permitted to generate such
   proposals internally.

   In each case, the IESG's decision to go forward (or not) with a
   procedural experiment, or the steps leading up to one, is expected to
   reflect their judgment of the existence of rough consensus in the
   community.  That judgment may be appealed using the usual procedures,
   but the IESG and the community are reminded that an experimental
   attempt to try something for a fixed period is typically a better
   engineering approach than extended philosophical discussion without
   any experience to back it up.

   Nothing above is to be construed as requiring an IETF-wide attempt
   for any given process experiment.  A proposal for such an experiment
   may specify selected areas, selected working groups, working groups
   meeting some specific criteria (e.g., those created after a
   particular time or of a specified age), or be specific in other ways.

   At or before the end of the "sunset" timeout, the IESG would either
   revise (or cause to be revised) the document into a BCP RFC or the
   procedure would expire and, presumably, not be tried again unless
   something changed radically.  A document describing why the
   experiment had succeeded or failed would be desirable but could not,
   realistically, be a requirement.  If the procedure went to BCP, the
   BCP would reflect what we would call "operational experience" in the
   real world.

   We note that, if the procedures the IESG has adopted (and the
   procedural exceptions it has made) over the last decade are
   legitimate, then the IESG has the authority to institute the changes
   specified here by bootstrapping this process.

3.  Security Considerations

   This document specifies a mechanism for evolving IETF procedures.  It
   does not raise or consider any protocol-specific security issues.  In
   considering experimental changes to procedures, the IESG should, of
   course, exercise due caution that such changes not reduce the quality
   of security review and consideration for protocols or, at least, that
   the process experiment proposals contain early detection and
   correction mechanisms should quality deterioration occur.

4.  Acknowledgements

   The first revision of this document benefited significantly from
   suggestions and comments from Avri Doria, Margaret Wasserman, and
   Harald Alvestrand, and from discussions with the General Area
   Directorate and at its open meeting during IETF 59.  After mailing
   list discussion, considerable explanatory material was removed to a
   separate document for the current version.

   The first version of this document was posted as an Internet Draft on
   February 7, 2004.

5.  References

5.1.  Normative References

   [RFC2026] Bradner, S., "The Internet Standards Process -- Revision
             3", BCP 9, RFC 2026, October 1996.

5.2.  Informative References

   [RFC1396] Crocker, S., "The Process for Organization of Internet
             Standards Working Group (POISED)", RFC 1396, January 1993.

   [RFC3667] Bradner, S., "IETF Rights in Contributions", BCP 78, RFC
             3667, February 2004.

   [RFC3668] Bradner, S., "Intellectual Property Rights in IETF
             Technology", BCP 79, RFC 3668, February 2004.

6.  Authors' Addresses

   John C Klensin
   1770 Massachusetts Ave, #322
   Cambridge, MA  02140
   USA

   Phone: +1 617 491 5735
   EMail: john-ietf@jck.com


   Spencer Dawkins
   1547 Rivercrest Blvd.
   Allen, TX  75002
   USA

   Phone: +1 469 330 3616
   EMail: spencer@mcsr-labs.org

Full Copyright Statement

   Copyright (C) The Internet Society (2004).

   This document is subject to the rights, licenses and restrictions
   contained in BCP 78, and at www.rfc-editor.org, and except as set
   forth therein, the authors retain all their rights.

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
   ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
   INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
   INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the ISOC's procedures with respect to rights in ISOC Documents can
   be found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at ietf-
   ipr@ietf.org.

Acknowledgement

   Funding for the RFC Editor function is currently provided by the
   Internet Society.

AI ELI5

On this page
PROBLEMThe IETF changed its process in two ways. Informal IESG announcements were too lightweight and sometimes out of step with the community, while the full standards process was too heavyweight for untried ideas.
MECHANISMA proposed change is written up as an Internet-Draft with a sunset date, goes through a four-week IETF Last Call, and is then published as an Experimental RFC and tried out.
WHY IT MATTERSAt or before the sunset the IESG either revises the document into a BCP RFC based on operational experience or lets the procedure expire.

§1A Middle Ground for Process Change

RFC 3933 is a Best Current Practice (BCP 93) about how the IETF changes itself, not about a network protocol. It specifies a middle-ground approach that relies on a model of proposing and carrying out an experiment, evaluating it, and then establishing permanent procedures based on operational experience.

The abstract describes the problem it fixes: earlier process changes were made either by announcement from the IESG, sometimes with limited community involvement, or through the same mechanism used for protocol specification. The first often proved too lightweight, the second too heavyweight.

A club wants to try a new rule but does not want to argue for months first. This RFC says: try it for a short time, then decide.

§2The Two Older Mechanisms

Since the 1992 changes documented in RFC 1396, the IETF has used two mechanisms for process changes.

  • The IESG adopted procedural changes on its own initiative and documented them informally, using the wide discretion implicitly granted to it by RFC 2026. This was a lightweight route, but there was sometimes too little alignment with the larger IETF community.
  • The RFC 2026 standards process was used to find a community of interest, hold BoFs, charter a working group and build IETF-wide consensus, usually ending in a BCP. This gave full community involvement but little flexibility.

The RFC notes that the IETF does not change its formal processes often. It cites the IPR clarifications in RFC 3667 and RFC 3668 as the first documented changes to RFC 2026 since 1996, and says the community is understandably reluctant to make permanent changes with untried procedures.

One way to change the rules was quick but not everyone heard about it. The other way involved everybody but took forever. Neither was great for testing a new idea.

§3The Process Experiment Procedure

The RFC regularizes and formalizes the informal IESG mechanism and permits and encourages the IESG to adopt process experiments in four steps.

  1. An Internet-Draft describes the proposed new or altered procedure. A problem statement and specific evaluation criteria are desirable but not required, to keep the requirements lightweight. The draft must state an explicit sunset timeout, typically not to exceed one year after adoption.
  2. If the IESG thinks the proposal is plausible and plausibly useful, a four-week IETF Last Call is started. The IESG may set up its own way of screening proposals, such as requiring endorsements, but review steps that cause significant delays are outside the intent of the specification.
  3. After the Last Call the IESG reevaluates the proposal. If it is appropriate, a second Internet-Draft is produced that cleans up definitional issues and responds to issues raised in the Last Call.
  4. The document and experiment are announced, the experiment begins, and the document is forwarded for publication as an Experimental RFC.

The RFC is deliberately vague about where the drafts come from: a working group, design team, individual, editing group or the IESG itself may write them.

Write down the idea and an end date, let everyone comment for four weeks, fix the write-up, then announce it and start the trial.

THE FOUR STEPS OF A PROCESS EXPERIMENT (RFC 3933)
Authors or IESGIESGIETF community
  1. I-D with sunsetDescribes the new or altered procedure and states an explicit sunset.
  2. Four-week Last CallStarted if the proposal is plausible and plausibly useful.
  3. Last Call commentsThe IESG reevaluates the proposal afterwards.
  4. Announce and startA second I-D is forwarded for publication as an Experimental RFC.

§4Scope, Consensus and Abandoning an Experiment

The IESG may use this mechanism, based on Experimental RFCs, to gain experience with proposed changes to BCP specifications. No BCP needs to be approved for an experiment until it is found to have value.

The IESG can reach a conclusion that an idea is bad at any stage and abandon the experiment. It may recommend publishing the experimental document with a discussion of why it was a bad idea, but is not required to.

An experiment does not have to apply to the whole IETF. A proposal may select areas, working groups, or working groups meeting some criterion, such as those created after a particular time or of a specified age.

The IESG's decision to go ahead or not is expected to reflect its judgment of rough consensus in the community. That judgment may be appealed using the usual procedures, but the RFC reminds everyone that trying something for a fixed period is typically a better engineering approach than extended philosophical discussion without experience.

The group can stop the trial at any time if it is a bad idea, and the trial can be run with just a few classes instead of the whole school.

§5How an Experiment Ends

At or before the sunset timeout, the IESG either revises, or causes to be revised, the document into a BCP RFC, or the procedure expires and presumably is not tried again unless something changes radically. If it becomes a BCP, that BCP reflects what the authors call operational experience in the real world.

A document explaining why the experiment succeeded or failed would be desirable, but the RFC says it could not realistically be a requirement.

The RFC also notes that if the procedures and procedural exceptions the IESG has adopted over the last decade are legitimate, then the IESG has the authority to institute these changes by bootstrapping this process.

When the trial time is up, the rule either becomes a real rule or quietly goes away.

§6Security Considerations

The RFC says it specifies a mechanism for evolving IETF procedures and does not raise or consider any protocol-specific security issues. It does ask the IESG to take care that process experiments do not reduce the quality of security review and consideration for protocols, or at least that proposals contain early detection and correction mechanisms should quality deteriorate.

When trying a new rule, make sure it does not make the safety checks worse, and have a way to notice if it does.

§7References