RFC 58 UNKNOWN PROTOCOL · DESIGN

Logical Message Synchronization

A discussion of how to keep logical messages synchronized when they are split across physical network messages

On rfc-editor.org ↗
PUBLISHED
June 1970
AUTHORS
T.P. Skinner
STREAM
Legacy
IMP MESSAGE MAXIMUM LENGTH
8095 bits
DOI
10.17487/RFC0058

RFC text

Original text · RFC Editor View on rfc-editor.org ↗
Network Working Group                                      T. P. Skinner
Request for Comments: 58                                 MIT Project MAC
                                                               June 1970

                    Logical Message Synchronization


At the last network meeting, the question of logical and physical
message distinctions was raised.  An argument was made in favor of
never running two logical messages together as one or more physical
messages.  Another method of stating this is that a logical message
must begin on a physical message boundary.  This did not, however,
solve the problem of locating the end of a logical message.  A rather
poor technique was suggested by myself which consisted of using the
first partial physical message as an indication of the last physical
message of the logical message.  This technique was thrown out for a
number of very valid reasons.  The solution that seemed most pleasing
was the inclusion of some sort of a bit count or data type
specification to precede the logical message.  Most everyone seemed to
like this even though it was stated in a very general way.

As of this writing it appears that it is desired to completely sever
the relation between physical and logical messages.  This certainly is
aesthetically pleasing.  However, we are now forced to view the
network as a virtually infinite bit stream with no physical
delineations.  It may well do to transmit a logical header and bit
count for each message as long as there are no errors along the line.
If, however, a bit is dropped, the problem of synchronization is
compounded by the fact that we have no ability to search for the
beginning of a logical message other than brute force.  An error of
this type could be introduced by faulty host or user software/hardware
as well as the imp itself.  This would involve the shifting of the
message bit by bit and seeing if the data looked reasonable.  This
could certainly be time-consuming as well as introducing the
possibility of false synchronism.

I can think of several solutions to the problem at the moment.  None
of them seems to be very good.  Upon losing synchronism, a user could
send some form of error message to the other host.  The other host
could then in return cease sending and wait for a message to continue
from the troubled user.  This would allow the troubled user to flush
out all waiting input.  He would then be assured that the next bit
started a logical message.  The problems here are in assuring
synchrony due to input/output buffering in the network and at both
hosts.  How, for example, can the troubled host be assured he has all
the pending data?  Once he is sure, he can then resume input assuming
all is OK.

Another partial solution requires the original restriction that
logical messages always start on physical boundaries.  A user then
merely has to examine the beginning of each physical message to see if
it fits the pattern of a logical message header.  This technique is a
lot safer than examining the entire input stream as well as being
quite a bit faster.

I have not intended to suggest a solution to the problem, but merely
bring it to light.  If we want to restrict logical messages to begin
on physical boundaries we must plan this early in the game.  (It
probably works out that way in most cases anyway.)  Other schemes can
be tried later.  We must, however, face up to this problem fairly
soon.



       [ This RFC was put into machine readable form for entry ]
        [ into the online RFC archives by Carl Alexander 7/97 ]

AI ELI5

On this page
PROBLEMWhen the network transmits a logical message (an application's data unit) as multiple physical messages (network packets), how do receivers know where one logical message ends and the next begins? If a single bit is lost, synchronization fails, and recovery becomes extremely difficult.
MECHANISMSkinner outlines the tradeoff: one approach is to require logical messages to start on physical message boundaries, making recovery easier; another is to include a bit count or data type specification before each logical message. Each approach has drawbacks when bit errors occur.
DESIGN CHALLENGEThis RFC highlights a fundamental challenge in protocol design: the need to separate logical structure from physical transmission. The community ultimately adopted approaches that include length or boundary information.

§1The Synchronization Problem

Logical messages are the units of data that applications send (an email, a file, a command), while physical messages are the chunks the network actually transmits (8095-bit IMP messages). The problem: a single logical message might be split across many physical messages, or several logical messages might fit in one physical message. When a receiver gets a stream of physical messages, how does it know where the logical boundaries are?

If a single bit is dropped due to a transmission error or faulty software, the receiver loses synchronization. Recovery then requires either brute-force bit-shifting (examining the entire stream to find a valid message header) or stopping transmission and flushing buffers until the sender and receiver are back in sync. Both solutions are slow and error-prone.

If messages can be split up and reassembled, how do you know where one ends and the next starts? If even one bit gets lost, finding the next message boundary becomes very hard.

§2Proposed Solutions

Skinner presents three potential approaches, each with tradeoffs:

First approach: Require logical messages to start on physical message boundaries. This makes recovery simpler: the receiver only needs to check the beginning of each physical message. The downside: it may waste space if a logical message is shorter than a physical message capacity.

Second approach: Include a length or bit-count specification before each logical message. The receiver can then skip to the next logical message's start. The downside: a bit error in the count field causes loss of synchronization for all subsequent messages.

Third approach (implicit): Include a data-type field specifying the message format. This provides slightly more flexibility than a raw bit count.

Option 1: Start messages at message boundaries. Option 2: Put a length number in each message. Option 3: Label what type of data is in each message. Each has its own problems.

§3Unresolved Design Choice

Skinner does not recommend a solution; instead, he raises the issue for the working group to decide early. The choice affects the entire protocol: if the group wants to restrict logical messages to physical boundaries, that constraint must be planned from the beginning. Waiting too long to make this decision means redesigning the protocol later. Skinner notes that in practice, most implementations probably end up with logical messages starting on physical boundaries anyway. This RFC exemplifies the collaborative standards-development process: raising a problem, discussing approaches, and deferring decisions until consensus emerges.

This is an important choice that needs to be made early, because it affects how the whole protocol works. But nobody has agreed on the answer yet.