GeoJSON-Seq: GeoJSON Text Sequences
A way to write huge GeoJSON datasets as a stream of small, separately parsable items.
Gillies, S., "GeoJSON Text Sequences", RFC 8142, DOI 10.17487/RFC8142, April 2017, <https://www.rfc-editor.org/info/rfc8142>.
@misc{rfc8142,
series = {Request for Comments},
number = 8142,
howpublished = {RFC 8142},
publisher = {RFC Editor},
doi = {10.17487/RFC8142},
url = {https://www.rfc-editor.org/info/rfc8142},
author = {S. Gillies},
title = {{GeoJSON Text Sequences}},
pagetotal = 5,
year = 2017,
month = apr,
} RFC text
Contents
Internet Engineering Task Force (IETF) S. Gillies
Request for Comments: 8142 Mapbox
Category: Standards Track April 2017
ISSN: 2070-1721
GeoJSON Text Sequences
Abstract
This document describes the GeoJSON text sequence format and
"application/geo+json-seq" media type. This format is based on
JavaScript Object Notation (JSON) text sequences and GeoJSON, and it
makes arbitrarily large geographic datasets incrementally parseable
without restricting the form of GeoJSON texts within a sequence.
Status of This Memo
This is an Internet Standards Track document.
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by the
Internet Engineering Steering Group (IESG). Further information on
Internet Standards is available in Section 2 of RFC 7841.
Information about the current status of this document, any errata,
and how to provide feedback on it may be obtained at
http://www.rfc-editor.org/info/rfc8142.
Copyright Notice
Copyright (c) 2017 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License.
Table of Contents
1. Introduction ....................................................2
1.1. Requirements Language ......................................2
2. GeoJSON Text Sequence Format ....................................2
3. Security Considerations .........................................3
4. Interoperability Considerations .................................3
5. IANA Considerations .............................................3
6. References ......................................................4
6.1. Normative References .......................................4
6.2. Informative References .....................................5
Author's Address ...................................................5
1. Introduction
Arbitrarily large sequences of values pose a problem for JavaScript
Object Notation (JSON) [RFC7159] that is well explained in the
motivation for JSON text sequences [RFC7464]. The GeoJSON format
[RFC7946] faces the same kind of problem. Geographic datasets often
run to the tens of thousands or millions of features. The problem is
often amplified by the presence of large arrays of coordinates for
each of the features.
This document describes a specialization of JSON text sequences. A
GeoJSON text sequence is a document of arbitrarily large size
containing one or more GeoJSON objects (e.g., multiple GeoJSON texts
that can be produced and parsed incrementally) and not just a single
GeoJSON FeatureCollection, Feature, or Geometry.
1.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in
[RFC2119].
2. GeoJSON Text Sequence Format
Defined in prose similar to the description of the JSON text sequence
in [RFC7464], a GeoJSON text sequence is any number of GeoJSON
[RFC7946] texts, each encoded in UTF-8 [RFC3629], preceded by one
ASCII [RFC20] record separator (RS) character, and followed by a line
feed (LF).
The GeoJSON text sequence format conforms to all the rules of
[RFC7464] and adds the following constraint: each JSON text MUST
contain a single GeoJSON object as defined in [RFC7946].
Heterogeneous sequences containing a mix of GeoJSON Geometry,
Feature, and FeatureCollection objects are permitted. How producers
and parsers of GeoJSON text sequences communicate rules for allowed
GeoJSON types in exchanged sequences is not specified in this
document.
3. Security Considerations
GeoJSON text sequences have no security considerations beyond those
of JSON text sequences [RFC7464] and the GeoJSON format [RFC7946].
4. Interoperability Considerations
The advantage of using ASCII character RS "0x1e" to denote a text is
that sequence producers and parsers need not enforce a canonical form
of GeoJSON. Any valid GeoJSON, pretty-printed or compact, can be
used in a GeoJSON text sequence.
A variety of parsers designed for newline-delimited sequences of
compact JSON text are deployed on the internet today. While there is
no canonical form for JSON texts, and pretty-printed and compact
forms are equally valid, GeoJSON text sequences containing compact
GeoJSON texts with no internal newlines are more interoperable with
existing non-standardized parsers.
In a distributed system where order and exactly-once delivery of
messages are difficult to achieve, GeoJSON text sequences that do not
rely on order of texts for extra semantics are more interoperable
than those that do.
5. IANA Considerations
The MIME media type for GeoJSON text sequences is "application/
geo+json-seq" (which uses the suffix established in [RFC8091]). IANA
has registered it in the "Media Types" registry
<https://www.iana.org/assignments/media-types/>.
Type name: application
Subtype name: geo+json-seq
Required parameters: n/a
Optional parameters: n/a
Encoding considerations: binary
Security considerations: See Section 3 of RFC 8142
Interoperability considerations: See Section 4 of RFC 8142
Published specification: RFC 8142
Applications that use this media type: Geographic information
systems (GIS)
Additional information:
Deprecated alias names for this type: n/a
Magic number(s): n/a
File extension(s): n/a
Macintosh file type code(s): n/a
Person to contact for further information: Sean Gillies
(sean.gillies@gmail.com)
Intended usage: COMMON
Restrictions on usage: none
Author: Sean Gillies (sean.gillies@gmail.com)
Change controller: IETF
6. References
6.1. Normative References
[RFC20] Cerf, V., "ASCII format for network interchange", STD 80,
RFC 20, DOI 10.17487/RFC0020, October 1969,
<http://www.rfc-editor.org/info/rfc20>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>.
[RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO
10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November
2003, <http://www.rfc-editor.org/info/rfc3629>.
[RFC7159] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
Interchange Format", RFC 7159, DOI 10.17487/RFC7159, March
2014, <http://www.rfc-editor.org/info/rfc7159>.
[RFC7464] Williams, N., "JavaScript Object Notation (JSON) Text
Sequences", RFC 7464, DOI 10.17487/RFC7464, February 2015,
<http://www.rfc-editor.org/info/rfc7464>.
[RFC7946] Butler, H., Daly, M., Doyle, A., Gillies, S., Hagen, S.,
and T. Schaub, "The GeoJSON Format", RFC 7946,
DOI 10.17487/RFC7946, August 2016,
<http://www.rfc-editor.org/info/rfc7946>.
6.2. Informative References
[RFC8091] Wilde, E., "A Media Type Structured Syntax Suffix for JSON
Text Sequences", RFC 8091, DOI 10.17487/RFC8091, February
2017, <http://www.rfc-editor.org/info/rfc8091>.
Author's Address
Sean Gillies
Mapbox
Email: sean.gillies@gmail.com
URI: http://sgillies.net
AI ELI5
On this page
§1Why Sequences
Very large sequences of values are a problem for JSON, as explained in the motivation for JSON text sequences (RFC 7464). GeoJSON has the same problem: geographic datasets often run to the tens of thousands or millions of features, and big arrays of coordinates make it worse.
RFC 8142 specializes JSON text sequences. A GeoJSON text sequence is a document of arbitrary size holding one or more GeoJSON objects, not just a single FeatureCollection, Feature or Geometry.
A map with millions of places is too big to read in one go. This lets you split it into many small pieces.
§2The Sequence Format
A GeoJSON text sequence is any number of GeoJSON texts, each encoded in UTF-8, preceded by one ASCII record separator (RS) character and followed by a line feed (LF). It follows all the rules of RFC 7464 and adds one constraint: each JSON text MUST contain a single GeoJSON object.
Mixed sequences of Geometry, Feature and FeatureCollection objects are permitted. How producers and parsers agree on which GeoJSON types are allowed in a given exchange is not specified.
Each card starts with a special marker and ends with a line break, and each card holds one place or shape.
<RS>{"type":"Point","coordinates":[102.0,0.5]}<LF>
<RS>{"type":"Feature","geometry":null,"properties":{}}<LF>§3Interoperability Advice
Using the RS character 0x1e to mark each text means producers and parsers need not enforce a canonical form: any valid GeoJSON, pretty-printed or compact, can be used.
Many deployed parsers are built for newline-delimited sequences of compact JSON. Sequences of compact GeoJSON texts with no internal newlines are more interoperable with them. Where ordering and exactly-once delivery are hard to achieve, sequences that do not depend on the order of texts are more interoperable.
Short cards with no line breaks inside are easiest for other people's card readers. It also helps if the order of the cards does not matter.
§4Media Type and Security
IANA registered the media type application/geo+json-seq, which uses the suffix established in RFC 8091. Its encoding considerations are listed as binary, and its intended usage is COMMON.
GeoJSON text sequences have no security considerations beyond those of JSON text sequences and the GeoJSON format.
The format gets its own official label so programs know what is in the file. It adds no new safety worries.