IMAP BINARY: IMAP4 Binary Content Extension
Lets an IMAP server decode attachments itself, so the client gets raw bytes instead of Base64 text.
Nerenberg, L., "IMAP4 Binary Content Extension", RFC 3516, DOI 10.17487/RFC3516, April 2003, <https://www.rfc-editor.org/info/rfc3516>.
@misc{rfc3516,
series = {Request for Comments},
number = 3516,
howpublished = {RFC 3516},
publisher = {RFC Editor},
doi = {10.17487/RFC3516},
url = {https://www.rfc-editor.org/info/rfc3516},
author = {L. Nerenberg},
title = {{IMAP4 Binary Content Extension}},
pagetotal = 8,
year = 2003,
month = apr,
} RFC text
Contents
Network Working Group L. Nerenberg
Request for Comments: 3516 Orthanc Systems
Category: Standards Track April 2003
IMAP4 Binary Content Extension
Status of this Memo
This document specifies an Internet standards track protocol for the
Internet community, and requests discussion and suggestions for
improvements. Please refer to the current edition of the "Internet
Official Protocol Standards" (STD 1) for the standardization state
and status of this protocol. Distribution of this memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2003). All Rights Reserved.
Abstract
This memo defines the Binary extension to the Internet Message Access
Protocol (IMAP4). It provides a mechanism for IMAP4 clients and
servers to exchange message body data without using a MIME content-
transfer-encoding.
1. Conventions Used in this Document
The key words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY"
in this document are to be interpreted as described in [KEYWORD].
The abbreviation "CTE" means content-transfer-encoding.
2. Introduction
The MIME extensions to Internet messaging allow for the transmission
of non-textual (binary) message content [MIME-IMB]. Since the
traditional transports for messaging are not always capable of
passing binary data transparently, MIME provides encoding schemes
that allow binary content to be transmitted over transports that are
not otherwise able to do so.
The overhead of MIME-encoding this content can be considerable in
some contexts (e.g., slow radio links, streaming multimedia).
Reducing the overhead associated with CTE schemes such as base64
can give a noticeable reduction in resource consumption. The Binary
extension lets the server perform CTE decoding prior to transmitting
message data to the client.
3. Content-Transfer-Encoding Considerations
Every IMAP4 body section has a MIME content-transfer-encoding.
(Those without an explicit Content-Transfer-Encoding header are
implicitly labeled as "7bit" content.) In the terminology of [MIME-
IMB], the CTE specifies both a decoding algorithm and the domain of
the decoded data. In this memo, "decoding" refers to the CTE
decoding step described in [MIME-IMB].
Certain CTEs use an identity encoding transformation. For these CTEs
there is no decoding required, however the domain of the underlying
data may not be expressible in the IMAP4 protocol (e.g., MIME
"binary" content containing NUL octets). To accommodate these cases
the Binary extension introduces a new type of literal protocol
element that is fully eight bit transparent.
Thus, server processing of the FETCH BINARY command involves two
logical steps:
1) perform any CTE-related decoding
2) determine the domain of the decoded data
Step 2 is necessary to determine which protocol element should be
used to transmit the decoded data. (See FETCH Response Extensions
for further details.)
4. Framework for the IMAP4 Binary Extension
This memo defines the following extensions to [IMAP4rev1].
4.1. CAPABILITY Identification
IMAP4 servers that support this extension MUST include "BINARY" in
the response list to the CAPABILITY command.
4.2. FETCH Command Extensions
This extension defines three new FETCH command data items.
BINARY<section-binary>[<partial>]
Requests that the specified section be transmitted after
performing CTE-related decoding.
The <partial> argument, if present, requests that a subset of
the data be returned. The semantics of a partial FETCH BINARY
command are the same as for a partial FETCH BODY command, with
the exception that the <partial> arguments refer to the DECODED
section data.
BINARY.PEEK<section-binary>[<partial>]
An alternate form of FETCH BINARY that does not implicitly set
the \Seen flag.
BINARY.SIZE<section-binary>
Requests the decoded size of the section (i.e., the size to
expect in response to the corresponding FETCH BINARY request).
Note: client authors are cautioned that this might be an
expensive operation for some server implementations.
Needlessly issuing this request could result in degraded
performance due to servers having to calculate the value every
time the request is issued.
4.3. FETCH Response Extensions
This extension defines two new FETCH response data items.
BINARY<section-binary>[<<number>>]
An <nstring> or <literal8> expressing the content of the
specified section after removing any CTE-related encoding. If
<number> is present it refers to the offset within the DECODED
section data.
If the domain of the decoded data is "8bit" and the data does
not contain the NUL octet, the server SHOULD return the data in
a <string> instead of a <literal8>; this allows the client to
determine if the "8bit" data contains the NUL octet without
having to explicitly scan the data stream for for NULs.
If the server does not know how to decode the section's CTE, it
MUST fail the request and issue a "NO" response that contains
the "UNKNOWN-CTE" extended response code.
BINARY.SIZE<section-binary>
The size of the section after removing any CTE-related
encoding. The value returned MUST match the size of the
<nstring> or <literal8> that will be returned by the
corresponding FETCH BINARY request.
If the server does not know how to decode the section's CTE, it
MUST fail the request and issue a "NO" response that contains
the "UNKNOWN-CTE" extended response code.
4.4. APPEND Command Extensions
The APPEND command is extended to allow the client to append data
containing NULs by using the <literal8> syntax. The server MAY
modify the CTE of the appended data, however any such transformation
MUST NOT result in a loss of data.
If the destination mailbox does not support the storage of binary
content, the server MUST fail the request and issue a "NO" response
that contains the "UNKNOWN-CTE" extended response code.
5. MIME Encoded Headers
[MIME-MHE] defines an encoding that allows for non-US-ASCII text in
message headers. This encoding is not the same as the content-
transfer-encoding applied to message bodies, and the decoding
transformations described in this memo do not apply to [MIME-MHE]
encoded header text. A server MUST NOT perform any conversion of
[MIME-MHE] encoded header text in response to any binary FETCH or
APPEND request.
6. Implementation Considerations
Messaging clients and servers have been notoriously lax in their
adherence to the Internet CRLF convention for terminating lines of
textual data in Internet protocols. When sending data using the
Binary extension, servers MUST ensure that textual line-oriented
sections are always transmitted using the IMAP4 CRLF line termination
syntax, regardless of the underlying storage representation of the
data on the server.
A server may choose to store message body binary content in a non-
encoded format. Regardless of the internal storage representation
used, the server MUST issue BODYSTRUCTURE responses that describe the
message as though the binary-encoded sections are encoded in a CTE
acceptable to the IMAP4 base specification. Furthermore, the results
of a FETCH BODY MUST return the message body content in the format
described by the corresponding FETCH BODYSTRUCTURE response.
While the server is allowed to modify the CTE of APPENDed <literal8>
data, this should only be done when it is absolutely necessary.
Gratuitous encoding changes will render useless most cryptographic
operations that have been performed on the message.
This extension provides an optimization that is useful in certain
specific situations. It does not absolve clients from providing
basic functionality (content transfer decoding) that should be
available in all messaging clients. Clients supporting this
extension SHOULD be prepared to perform their own CTE decoding
operations.
7. Formal Protocol Syntax
The following syntax specification uses the augmented Backus-Naur
Form (ABNF) notation as used in [ABNF], and incorporates by reference
the Core Rules defined in that document.
This syntax augments the grammar specified in [IMAP4rev1].
append =/ "APPEND" SP mailbox [SP flag-list]
[SP date-time] SP literal8
fetch-att =/ "BINARY" [".PEEK"] section-binary [partial]
/ "BINARY.SIZE" section-binary
literal8 = "~{" number "}" CRLF *OCTET
; <number> represents the number of OCTETs
; in the response string.
msg-att-static =/ "BINARY" section-binary SP (nstring / literal8)
/ "BINARY.SIZE" section-binary SP number
partial = "<" number "." nz-number ">"
resp-text-code =/ "UNKNOWN-CTE"
section-binary = "[" [section-part] "]"
8. Normative References
[ABNF] Crocker, D., Editor, and P. Overell, "Augmented BNF for
Syntax Specifications: ABNF", RFC 2234, November 1997.
[IMAP4rev1] Crispin, M., "Internet Message Access Protocol Version
4rev1", RFC 3501, March 2003.
[KEYWORD] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[MIME-IMB] Freed, N. and N. Borenstein, "Multipurpose Internet Mail
Extensions (MIME) Part One: Format of Internet Message
Bodies", RFC 2045, November 1996.
[MIME-MHE] Moore, K., "MIME (Multipurpose Internet Mail Extensions)
Part Three: Message Header Extensions for Non-ASCII
Text", RFC 2047, November 1996.
9. Security Considerations
There are no known additional security issues with this extension
beyond those described in the base protocol described in [IMAP4rev1].
10. Intellectual Property
The IETF takes no position regarding the validity or scope of any
intellectual property or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; neither does it represent that it
has made any effort to identify any such rights. Information on the
IETF's procedures with respect to rights in standards-track and
standards-related documentation can be found in BCP-11. Copies of
claims of rights made available for publication and any assurances of
licenses to be made available, or the result of an attempt made to
obtain a general license or permission for the use of such
proprietary rights by implementors or users of this specification can
be obtained from the IETF Secretariat.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights which may cover technology that may be required to practice
this standard. Please address the information to the IETF Executive
Director.
11. Author's Address
Lyndon Nerenberg
Orthanc Systems
1606 - 10770 Winterburn Road
Edmonton, Alberta
Canada T5S 1T6
EMail: lyndon@orthanc.ab.ca
12. Full Copyright Statement
Copyright (C) The Internet Society (2003). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFC Editor function is currently provided by the
Internet Society.
AI ELI5
On this page
§2Why Skip the Encoding
MIME lets email carry non-text content. Because traditional mail transports cannot always pass binary data transparently, MIME defines content-transfer-encodings (CTE), such as base64, that make the content safe to send.
That encoding has a cost. The RFC says the overhead can be considerable in some settings, for example slow radio links and streaming multimedia. The Binary extension to IMAP4 lets the server perform the CTE decoding before it sends message data to the client.
Wrapping a gift in lots of paper makes the parcel bigger. This extension lets the post office unwrap it first, so less has to be carried to you.
§3Decoding and the Domain of the Data
Every IMAP4 body section has a CTE. A section with no explicit Content-Transfer-Encoding header is implicitly labeled 7bit. The CTE specifies both a decoding algorithm and the domain of the decoded data.
Some CTEs are identity encodings, so no decoding is needed. Their data may still not be expressible in base IMAP4, for example MIME binary content that contains NUL octets. For these cases the extension introduces a new kind of literal that is fully eight bit transparent.
So the server does two logical steps for a FETCH BINARY: it performs any CTE-related decoding, then it determines the domain of the decoded data. The second step decides which protocol element carries the data.
First the server unwraps the parcel. Then it checks what kind of thing is inside, so it knows which box to put it in for the trip to you.
§4The FETCH and CAPABILITY Extensions
A server that supports the extension MUST include BINARY in its CAPABILITY response. The extension adds three FETCH data items:
BINARY[section]<partial>asks for the section after CTE decoding. A partial request works like a partial FETCH BODY, except that the offsets refer to the decoded data.BINARY.PEEKis the same but does not implicitly set the\Seenflag.BINARY.SIZEasks for the decoded size of the section, which is the size to expect from the matching FETCH BINARY.
The response carries the content as an nstring or a literal8. If the decoded data is 8bit and has no NUL octet, the server SHOULD use a normal string rather than a literal8, so the client can tell there are no NULs without scanning. The size returned by BINARY.SIZE MUST match the size of the data in the corresponding FETCH BINARY.
If the server cannot decode the section's CTE, it MUST fail the request with a NO response carrying the UNKNOWN-CTE response code.
BINARY.SIZE may be expensive for some servers, because they may have to calculate the value every time. Clients should not request it needlessly.
You can ask for the unwrapped file, for just a peek at it, or for how big it will be. If the post office does not know how to unwrap it, it says no and tells you why.
§4.4Appending Binary Data
The APPEND command is extended so a client can store data that contains NULs, by using the literal8 syntax. The server MAY change the CTE of the appended data, but the change MUST NOT lose any data.
If the destination mailbox cannot store binary content, the server MUST fail the request with a NO response carrying UNKNOWN-CTE.
The same trick works in the other direction. You can hand over a raw file to store, and the server must never damage it.
§5Headers, Line Endings and Client Duties
MIME encoded header text (the non-ASCII encoding from RFC 2047) is a different mechanism from the body CTE. A server MUST NOT convert it in response to any binary FETCH or APPEND.
Servers MUST send textual line-oriented sections with IMAP4 CRLF line endings, whatever the stored form. A server may store binary content unencoded, but it MUST still issue BODYSTRUCTURE responses as though binary sections were in a CTE the base specification accepts, and FETCH BODY MUST return content in the format BODYSTRUCTURE describes.
The RFC warns that changing the CTE of appended literal8 data should be done only when absolutely necessary, because gratuitous changes make most cryptographic operations on the message useless. It also says this is an optimization, not a replacement for basic client functionality, so clients that support it SHOULD be prepared to do their own CTE decoding.
The server must keep the labels on the parcel honest and not tamper with signed letters. Your mail program should still know how to unwrap things itself, just in case.
§7Formal Syntax
The grammar is given in ABNF and augments the IMAP4rev1 grammar from RFC 3501. It extends append, fetch-att, msg-att-static and resp-text-code, and defines literal8, partial and section-binary.
A literal8 begins with a tilde, then a brace-enclosed octet count, then CRLF, then that many octets of any value.
This part is the exact spelling rules, so every mail program writes the new commands the same way.
fetch-att =/ "BINARY" [".PEEK"] section-binary [partial]
/ "BINARY.SIZE" section-binary
literal8 = "~{" number "}" CRLF *OCTET
msg-att-static =/ "BINARY" section-binary SP (nstring / literal8)
/ "BINARY.SIZE" section-binary SP number
partial = "<" number "." nz-number ">"
resp-text-code =/ "UNKNOWN-CTE"
section-binary = "[" [section-part] "]"§9Security Considerations
The RFC states that there are no known additional security issues beyond those in the base IMAP4rev1 protocol. Its separate warning about gratuitous encoding changes breaking cryptographic operations on messages is covered above.
This extension does not add any new danger beyond what ordinary IMAP already has.