Response to RFC 467

Network Working Group                                           E. Meyer
Request for Comments: 492                                    MIT-Multics
NIC: 15357                                                 18 April 1973
                          RESPONSE TO RFC 467


   Jerry Burchfiel and Ray Tomlinson of Bolt, Beranek, and Newman, Inc,
   have issued a Network Request for Comments (#467) which proposes a
   solution to two problems which have been annoying to Network users.
   This document will briefly describe the problems and proposed
   solutions, and offer comments and alternative suggestions.

BACKGROUND

   To establish a data connection between two hosts through the network,
   the Host-Host protocol requires that one host send a Request for
   Connection and that the second Host reply affirmatively.  If the
   desired socket("port") at the target host is already in use, the
   target host replies negatively.  Once a connection is established,
   data transmission may proceed, controlled by data allocation messages
   dispatched by the host at the read end of the connection.  The host
   on the write side is constrained by protocol to send only as much
   data as has been permitted by the read side.  If it exhausts the
   allocation it must wait until a new data allocation control message
   is received.  Then it can send more.

   One of the problems arises from the fact that messages apparently are
   lost somewhere in the transmission path with a low but regular
   frequency.  If an allocate control message concerning an open
   connection is lost, a situation can occur in which data transmission
   over the connection ceases permanently.  This can happen because the
   host at the send side believes it has exhausted its allocation, and
   sits holding back data to end because it is waiting for a new data
   allocation message to come from the read side.  However, the read
   side has actually sent out the allocation, but it was lost.  It
   thinks that the send side may proceed and sits waiting for data to
   come in over the connection.  This is known as the "lost allocate"
   phenomenon.  However, similar symptoms can occur if a data message is
   lost and the send side exhausts its allocation before a new
   allocation is given by the read side.  The send side waits for a new
   allocation, but the read side has not received one of the data
   messages and believes there is still some allocation left.  In either
   case, the result is a permanently blocked connection.  This appears
   to happen with enough regularity to be annoying to users who connect
   typewriters to foreign hosts through the Network.  When it happens,
   the only current solution is to disconnect and to establish a new
   connection.

   The solution to this problem which RFC 467 proposes is to establish a
   pair of allocation-resetting control messages, one for use by the
   send side (RCS) and the other for the read side (RCR).  Whenever it
   wishes, either side may initiate the allocation-resetting sequence by
   setting its own allocation counter to zero and dispatching an RCS or
   RCR control message to the other side.  The host receiving it will
   set its own allocation counter for that connection to zero and send
   an RCR or RCS in reply.  Now the allocations for both sides are in
   synchronization (they are zero), and data transmission can begin
   again when a new allocation is sent by the receive side.  This
   procedure is intended to be initiated whenever either side thinks the
   connection has been quiescent for a suspiciously long time.  The
   actual specification of this control message pair in RFC 467 is more
   complex in that the pipeline between the two sides must be empty of
   data messages before the send side may dispatch an RCS control
   message.

   The second problem arises when the host at one side of an open
   connection crashes and purges its tables when it comes back up, while
   the host at the other end of the connection does not notice that
   anything has happened. (A similar situation occurs when the Network
   path temporarily fails between the two hosts, but only one host
   notices the failure and closes the connection.) If the host which
   crashed attempts to re-establish the connection, the host at the
   other end refuses to do so because the socket to which the connection
   request is targeted is seemingly already involved in an open
   connection.  Given the idiosyncrasies of the terminal support
   software of some systems, users at some consoles may be unable to
   reconnect to the distant system they were connected with when the
   local system supporting his terminal crashed.  This can continue
   indefinitely until the system which believes the original connections
   to be still open resets its internal state.  This is call the "half-
   closed" phenomenon, and a solution is proposed in RFC 467.  The basic
   principle of the RFC 467 proposal is that the side which has the open
   connection is able to detect an inconsistency whenever either side
   performs communication regarding this connection.  When it does, it
   is supposed to silently (without regard to normal protocol) close the
   connection and be ready to handle connection requests to the
   previously connected port.

   There are two types of interactions in which "half-closed"
   inconsistency is uncovered.  The first case occurs when the connected
   side sends a message over a write connection.  The side which has
   lost the connection receives this as a data message which does not
   correspond to an open connection and replies with an Error Report
   control message.  When the connected side receives it, it realizes
   that the connection actually no longer exists and deletes it from its
   own tables.  The second case occurs when the host which has lost the
   connection sends a connection request to the other host specifying
   the same sockets as were involved in the previous connection.  The
   host receiving this request recognizes the inconsistency, because not
   only is the local socket already connected, it is connected to the
   same foreign socket as specified in the connection request.  It
   internally deletes its record of the connection, making the local
   socket free, and responds to the connection request normally.

COMMENTS AND ALTERNATIVE PROPOSALS

   The Project MAC Computer Systems Research Division opposes both
   protocol change proposals in this RFC.  We have moderate opposition
   to the proposal to handle half-closed connections because it fails to
   consider all aspects of the problem and it further complicates the
   protocol, but very strong opposition to the proposal for allocation
   resynchronization because it attacks a symptom, not the disease, and
   furthermore tends to mask diagnosis of a potentially very serious
   network problem.

   RFC 467 proposes the addition of two control messages, Reset
   Connection by Sender (RCS) and Reset Connection by Receiver (RCR)
   whose sole purpose is to resynchronize the allocation counters at
   both ends of a connection.  In this way the "lost allocate"
   phenomenon, in which allocate (ALL) control messages somehow are lost
   in transmission so that the sending side is unable to continue
   transmitting data is solved.  If it were truly a "lost allocate"
   problem, this would be viable solution.  However, I feel that this is
   really a "lost message" problem, in which messages of all kinds are
   being lost in transmission, which is much more serious.  ALL messages
   may be very frequent in communications with some hosts and these may
   be the ones most often lost, but if messages are actually lost in the
   network, it may also be data messages that are being lost, which
   would provide similar symptoms.  A lost message in a Telnet
   connection can be detected and overcome by the human user, but an
   undetected lost message from the middle of a transmitted file can
   have disastrous consequences, especially because the invalid file, if
   ever detected, can perhaps not be corrected.  Because this "solution"
   tends to paper over the immediate problem and to propagate it to a
   point far removed in both space and time at which it appears as an
   incomprehensible disaster, it should be strongly opposed.

   The real problem appears to be the random undetected loss of messages
   somewhere in the transmission path.  A true solution to this problem
   is either a) to eliminate the cause of undetected loss of messages,
   or b) to move to a new protocol which is designed to cope with an
   unreliable physical transmission path.  Either of these solutions is
   some distance away.  A proposed interim solution which modifies the
   existing GVB and RET commands and which has the additional feature of
   simplifying them somewhat is outlined below.

   A receiving host may at an arbitrary time issue a Give-Back
   allocation (GVB) control message for a connection.

             8       8        8        8
         +-------+-------+--------+--------+
         |  GVB  | link  | f =255 | f =255 |
         |       |       |  m     |  b     |
         +-------+-------+--------+--------+

   The format of this GVB message is the same as that currently defined,
   except that the fraction fields f(m) and f(b) are required to all 1s.
   This is designed to provide a measure of upward compatibility.  A
   host operating under the modified protocol will ignore the fraction
   fields, but under the current protocol this message means return
   everything.  A sending host which receives a GVB control message
   immediately ceases transmission on the specified link.  When the RFNM
   from the last message transmitted is received (indicating an empty
   pipeline), the sending host issues a Return Allocation (RET) control
   message, returning the remaining allocation.

              8      8        16         32
          +------+------+-----------+-----------+
          | RET  | link | msg space | bit space |
          +------+------+-----------+-----------+

   The modified RET command has the same format as that currently
   defined.  The two differences are that it can not be sent until data
   transmission ceases and the last RFNM is received, and that it must
   return all remaining allocation for the send link (i.e., the
   allocation counters are set to zero).

   When the host on the read side of the connection receives the RET
   message, the allocation counters at the send side are zero and the
   pipeline is empty.  Therefore, if no error has occurred during the
   connection, the allocation returned in the RET message should be the
   same as the allocation in the counters of the read side of the
   connection.  If so, the read side can proceed to send a new
   allocation secure in the knowledge that no message has been lost.  If
   the two sets of values do not agree, some error in the transmitted
   data may have occurred.  What to do in that case is a local host
   option.  Some hosts may choose to close the connection, while others
   may choose to resume transmission by sending a new allocation to the
   sending side.  I feel that as a minimum a host should send a message
   indicating the error both to the user and to some human being at the
   host responsible for monitoring network performance.

   This modified control message pair is capable of both its originally
   intended function,and of detecting errors and resynchronizing
   allocations (if desired) when initiated by the receiving side.  I
   feel that the inability of this scheme to initiate allocation
   checking from either side is only a minor disadvantage which is more
   than compensated for by its positive features: this scheme gives
   positive indication that an error has occurred (the proposed RCS/RCR
   method conceals errors), and this minor change to the protocol may
   mean a correspondingly minor change to NCP's.

   I have negative feelings regarding the solution to the "half-closed"
   problem proposed in RFC 467.  To put additional burden on the RTS and
   STR commands not only unduly complicates the protocol, but in some
   sense can make operation less fail-safe and problems more obscure.
   My main objection concerns the action to be taken when control
   messages are received which conflict with the current state of the
   receiving NCP.  This proposal suggests that an NCP receiving an STR
   or RTS for a socket it believes to be connected assume something
   about the state of the foreign NCP (that the foreign NCP has closed
   the connection) and automatically change its own state to agree with
   the assumed state at the other end (close the connection at its end).
   This may work fine if the assumption is correct and the
   implementations are free from bugs.  However, the following
   situations could cause problems that are perhaps hard to diagnose: 1)
   the foreign NCP has a bug which causes it to send an RTS or STR for a
   connected socket, 2) the foreign NCP chooses to interpret the queuing
   option of the current protocol as permitting RFC's to be sent for
   already connected sockets, or 3) the local NCP has a bug which
   erroneously causes it to regard RFC's coming from a different host or
   from the particular foreign host but concerning a different foreign
   socket as pertaining to the open connection attached to the target
   socket.

   A second objection is that this proposal does not cover all
   possibilities.  Two likely possibilities are: another socket (from
   any host) attempts to connect to the socket involved in the dead
   connection.  Second, the host that lost a connection attached to one
   of its read sockets makes another connection with different sockets,
   but uses the same link number that implemented the previous
   connection.  The second case can be handled by additional
   complications to the protocol.  However, the first case is
   symptomatically identical to the situation in which an RFC is issued
   for a genuinely already-connected socket.  It can not be handled
   using this approach.

   I believe that a more rigorous use of the existing Reset Host (RST)
   control message would eliminate most of the causes of the "half-
   closed" phenomenon; viz. one of the hosts involved in a connection
   goes down without sending an RST when it comes back up; or the
   network between the two hosts partitions, and only one host notes it.
   If it were deemed necessary, a pair of Reset Link control commands to
   reset an individual link could be added to the protocol to cope with
   instance of the "half-closed" phenomenon due to other causes.

   I'd like to set down here a number of principles which I think are at
   least peripherally concerned with alleviating the "half-closed"
   phenomenon.  None of these is explicitly stated in the current Host-
   Host protocol document, but I believe that their enunciation would
   tend to alleviate confusion caused by network and host failures.

      1. A NCP which receives an Imp-to-Host message type 7 (Host Dead)
         concerning a host should consider all connections or connection
         attempts with that host as dead and should purge them from its
         tables.

      2. When after noting a foreign host as dead (by receiving a "Host
         Dead" Imp-to-Host message), an NCP receives any message from
         that host other than a Reset Host (RST) control message, it
         should delete the message and respond with an RST.  It should
         respond normally to a received RST.

      3. Two hosts must exchange the RST - RRP reset control message
         pair prior to any other form of communications.  An RST must
         first be sent by an NCP wishing to start communications with a
         foreign host if that host pair has not been previously reset
         since the local NCP came up or it noted the foreign NCP as
         down.  Note that this does not require an NCP to send resets to
         all other hosts each time it comes up.

      4. An NCP which receives an Imp-to-Host message type 9 (Incomplete
         Transmission) concerning a write link implementing an open
         connection, may at its option make several tries to retransmit
         the last message until a RFNM is received or the NCP gives up.
         However, unless the message is eventually successfully
         transmitted to the foreign host the NCP must abort the
         connection, sending out a CLS control message.  The successful
         implementation of retransmission depends on the retransmitting
         host to wait for a RFNM on a data link before sending a
         subsequent message and on all hosts to be able to discard
         messages which are not completely received.

      5. An NCP which receives a message from a foreign host that seems
         inconsistent with its current state should take no action to
         modify that state.  Rather it should send an ERR error control
         message specifying the type of inconsistency and discard the
         inconsistent message.  An NCP receiving an ERR message should
         log it for human inspection and is then allowed to silently
         modify its internal state or send out control messages in order
         to remove the inconsistency. (This is an extension of the
         proposal in RFC 467 that an NCP should delete a connection when
         it receives an ERR message specifying that the link involved is
         unknown.)


        [This RFC was put into machine readable form for entry]
   [into the online RFC archives by Helene Morin, Via Genie,12/1999]