From 2a84e7a084201d0f9ee3a9e5127212d7e25f1ef5 Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Mon, 15 Jun 2020 09:16:32 -0700 Subject: [PATCH 1/9] added null geometry recommendation (wip) --- best-practices.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/best-practices.md b/best-practices.md index adc3859fb..9e2d43406 100644 --- a/best-practices.md +++ b/best-practices.md @@ -5,6 +5,7 @@ * [Field and ID formatting](#field-and-id-formatting) * [Field selection and Metadata Linking](#field-selection-and-metadata-linking) * [Datetime selection](#datetime-selection) +* [Null Geometries](#null-geometries) * [Representing Vector Layers in STAC](#representing-vector-layers-in-stac) * [Common Use Cases of Additional Fields for Assets](#common-use-cases-of-additional-fields-for-assets) * [Static and Dynamic Catalogs](#static-and-dynamic-catalogs) @@ -69,6 +70,23 @@ might choose to have `datetime` be the start. The key is to put in a date and ti the focus of STAC. If `datetime` is set to `null` then it is strongly recommended to use it in conjunction with a content extension that explains why it should not be set for that type of data. +## Null Geometries + +Though the [GeoJSON standard](https://tools.ietf.org/html/rfc7946) allows null geometries, in STAC we strongly recommend +that every item have a geometry, since the general expectation one using a SpatioTemporal Catalog is to be able to query +all data by space and time. But there are some use cases where it can make sense to create a STAC Item before it gets +a geometry. The most common of these is 'level 1' satellite data, which is a picture downlinked before it has been +geospatially located. + +These follow the GeoJSON concept that it is an ['unlocated' feature](https://tools.ietf.org/html/rfc7946#section-3.2). +So if the catalog has data that is not located then it can follow GeoJSON and set the geometry to null. + +TODO: we have to figure out what to do with BBOX, it needs to not be required to make this work, since the GeoJSON schema +says a BBOX has minItems: 4. This waters down what is required in STAC even more, and potentially makes it so +validation won't catch if someone has filled out a geometry and not a bbox. + + + ## Representing Vector Layers in STAC Many implementors are tempted to try to use STAC for 'everything', using it as a universal catalog of all their 'stuff'. From 590ea1b1a62f03b8594c25f751a3abc33d8e9960 Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Tue, 23 Jun 2020 11:35:16 -0700 Subject: [PATCH 2/9] finished best practices for null geometries --- best-practices.md | 47 +++++++++++++++++++++++++++++++++++------------ 1 file changed, 35 insertions(+), 12 deletions(-) diff --git a/best-practices.md b/best-practices.md index 9e2d43406..3e0251f60 100644 --- a/best-practices.md +++ b/best-practices.md @@ -73,19 +73,42 @@ that explains why it should not be set for that type of data. ## Null Geometries Though the [GeoJSON standard](https://tools.ietf.org/html/rfc7946) allows null geometries, in STAC we strongly recommend -that every item have a geometry, since the general expectation one using a SpatioTemporal Catalog is to be able to query +that every item have a geometry, since the general expectation of someone using a SpatioTemporal Catalog is to be able to query all data by space and time. But there are some use cases where it can make sense to create a STAC Item before it gets -a geometry. The most common of these is 'level 1' satellite data, which is a picture downlinked before it has been -geospatially located. - -These follow the GeoJSON concept that it is an ['unlocated' feature](https://tools.ietf.org/html/rfc7946#section-3.2). -So if the catalog has data that is not located then it can follow GeoJSON and set the geometry to null. - -TODO: we have to figure out what to do with BBOX, it needs to not be required to make this work, since the GeoJSON schema -says a BBOX has minItems: 4. This waters down what is required in STAC even more, and potentially makes it so -validation won't catch if someone has filled out a geometry and not a bbox. - - +a geometry. The most common of these is 'level 1' satellite data, where an image is downlinked and cataloged before it has +been geospatially located. + +The recommendation for data that does not yet have a location is to follow the GeoJSON concept that it is an ['unlocated' +feature](https://tools.ietf.org/html/rfc7946#section-3.2). So if the catalog has data that is not located then it can follow +GeoJSON and set the geometry to null. Though normally required, in this case the `bbox` field should not be included. + +Note that this recommendation is only for cases where data does not yet have a geometry and it cannot be estimated. There +are further details on the two most commonly requested desired use cases for setting geometry to null: + +### Unrectified Satellite Data + +Most satellite data is downlinked without information that precisely describes where it is located on earth. A satellite +imagery processing pipeline will always attempt to locate it, but often that process takes a number of hours, or never +quite completes (like when it is too cloudy). It can be useful to start to populate the Item before it has a geometry. +In this case the recommendation is to use the 'estimated' position from the satellite, to populate at least the bounding box, +and use the same broad bounds for the geometry (or leaving it null) until there is precise ground lock. This estimation is +usually done by onboard equipment, like GPS or star trackers, but can be off by kilometers or more. But it is very useful for +STAC users to be able to at least find approximate area in their searches. A commonly used field for communicating ground lock +is not yet established, but likely should be (an extension proposal would be appreciated). If there is no way to provide an +estimate then the data then a null geometry with no `bbox` can be used, as described above. But the data will likely not +show up in STAC API searches, as most will at least implicitly use a geometry. Though this section is written with +satellite data in mind, one can easily imagine other data types that start with a less precise geometry but have it +refined after processing. + +### Non-STAC Items + +The other case that often comes up is people who love STAC and want to use it to catalog everything they have, even if its +not spatial. In this case our recommendation is to re-use the STAC structures, creating Items that have everything the same, +except they do not have a geometry. We have discussed mechanisms to make these 'non-spatial' Items differentiated in the +specification, like using an alternate link structure in catalogs. But there are no actual implementations of this, so if +you have one please get in touch and we can figure out how to support it. We likely will not consider these to be actually +STAC-compliant, as the contract in STAC is that Items should be those that represent an asset in space and time. But we +hope to design a solution that enables 'mixed' catalogs with compliant and non-compliant STAC items. ## Representing Vector Layers in STAC From 88d1e4597d9d46cd0b7cc366824483fb4c3faafe Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Tue, 23 Jun 2020 11:38:38 -0700 Subject: [PATCH 3/9] updated item spec to indicate bbox isnt required if null geometry is used --- item-spec/item-spec.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/item-spec/item-spec.md b/item-spec/item-spec.md index f8eb3560d..2bd2da5b4 100644 --- a/item-spec/item-spec.md +++ b/item-spec/item-spec.md @@ -34,7 +34,7 @@ inherited from GeoJSON. | id | string | **REQUIRED.** Provider identifier. As most geospatial assets are already defined by some identification scheme by the data provider it is recommended to simply use that ID. Data providers are advised to include sufficient information to make their IDs globally unique, including things like unique satellite IDs. | | type | string | **REQUIRED.** Type of the GeoJSON Object. MUST be set to `Feature`. | | geometry | [GeoJSON Geometry Object](https://tools.ietf.org/html/rfc7946#section-3.1) | **REQUIRED.** Defines the full footprint of the asset represented by this item, formatted according to [RFC 7946, section 3.1](https://tools.ietf.org/html/rfc7946#section-3.1). The footprint should be the default GeoJSON geometry, though additional geometries can be included. Coordinates are specified in Longitude/Latitude or Longitude/Latitude/Elevation based on [WGS 84](http://www.opengis.net/def/crs/OGC/1.3/CRS84). | -| bbox | \[number] | **REQUIRED.** Bounding Box of the asset represented by this item, formatted according to [RFC 7946, section 5](https://tools.ietf.org/html/rfc7946#section-5). | +| bbox | \[number] | **REQUIRED\*.** Bounding Box of the asset represented by this item, formatted according to [RFC 7946, section 5](https://tools.ietf.org/html/rfc7946#section-5). \* Note that GeoJSON allows null geometries - if a null geometry is used then the bbox is not required. | | properties | [Properties Object](#properties-object) | **REQUIRED.** A dictionary of additional metadata for the item. | | links | \[[Link Object](#link-object)] | **REQUIRED.** List of link objects to resources and related URLs. A link with the `rel` set to `self` is strongly recommended. | | assets | Map | **REQUIRED.** Dictionary of asset objects that can be downloaded, each with a unique key. | From 6a0bcfd4fe634abbdf5d9be9c52e5159e845aeab Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Tue, 23 Jun 2020 11:41:49 -0700 Subject: [PATCH 4/9] updated changelog --- CHANGELOG.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0fbdffc03..f970949a8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,10 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. ## [Unreleased] +### Changed + + - Clarification on null geometries, making bbox not required if a null geometry is used. + ## [v1.0.0-beta.1] - 2020-05-29 ### Removed From 1e967ce34a7c6c3782861ff2d652ef00be9f6219 Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Tue, 23 Jun 2020 12:00:35 -0700 Subject: [PATCH 5/9] Update item-spec/item-spec.md Co-authored-by: Matthias Mohr --- item-spec/item-spec.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/item-spec/item-spec.md b/item-spec/item-spec.md index 2bd2da5b4..b3c06ace1 100644 --- a/item-spec/item-spec.md +++ b/item-spec/item-spec.md @@ -34,7 +34,7 @@ inherited from GeoJSON. | id | string | **REQUIRED.** Provider identifier. As most geospatial assets are already defined by some identification scheme by the data provider it is recommended to simply use that ID. Data providers are advised to include sufficient information to make their IDs globally unique, including things like unique satellite IDs. | | type | string | **REQUIRED.** Type of the GeoJSON Object. MUST be set to `Feature`. | | geometry | [GeoJSON Geometry Object](https://tools.ietf.org/html/rfc7946#section-3.1) | **REQUIRED.** Defines the full footprint of the asset represented by this item, formatted according to [RFC 7946, section 3.1](https://tools.ietf.org/html/rfc7946#section-3.1). The footprint should be the default GeoJSON geometry, though additional geometries can be included. Coordinates are specified in Longitude/Latitude or Longitude/Latitude/Elevation based on [WGS 84](http://www.opengis.net/def/crs/OGC/1.3/CRS84). | -| bbox | \[number] | **REQUIRED\*.** Bounding Box of the asset represented by this item, formatted according to [RFC 7946, section 5](https://tools.ietf.org/html/rfc7946#section-5). \* Note that GeoJSON allows null geometries - if a null geometry is used then the bbox is not required. | +| bbox | \[number] | **REQUIRED if `geometry` is not `null`.** Bounding Box of the asset represented by this item, formatted according to [RFC 7946, section 5](https://tools.ietf.org/html/rfc7946#section-5). | | properties | [Properties Object](#properties-object) | **REQUIRED.** A dictionary of additional metadata for the item. | | links | \[[Link Object](#link-object)] | **REQUIRED.** List of link objects to resources and related URLs. A link with the `rel` set to `self` is strongly recommended. | | assets | Map | **REQUIRED.** Dictionary of asset objects that can be downloaded, each with a unique key. | From 174aaecd76f32834de5576beeae9dd0ca5e93ec6 Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Tue, 23 Jun 2020 12:09:18 -0700 Subject: [PATCH 6/9] updated section title, cleanup --- best-practices.md | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/best-practices.md b/best-practices.md index 3e0251f60..8739473e4 100644 --- a/best-practices.md +++ b/best-practices.md @@ -5,7 +5,7 @@ * [Field and ID formatting](#field-and-id-formatting) * [Field selection and Metadata Linking](#field-selection-and-metadata-linking) * [Datetime selection](#datetime-selection) -* [Null Geometries](#null-geometries) +* [Unlocated Items](#unlocated-items) * [Representing Vector Layers in STAC](#representing-vector-layers-in-stac) * [Common Use Cases of Additional Fields for Assets](#common-use-cases-of-additional-fields-for-assets) * [Static and Dynamic Catalogs](#static-and-dynamic-catalogs) @@ -70,7 +70,7 @@ might choose to have `datetime` be the start. The key is to put in a date and ti the focus of STAC. If `datetime` is set to `null` then it is strongly recommended to use it in conjunction with a content extension that explains why it should not be set for that type of data. -## Null Geometries +## Unlocated Items Though the [GeoJSON standard](https://tools.ietf.org/html/rfc7946) allows null geometries, in STAC we strongly recommend that every item have a geometry, since the general expectation of someone using a SpatioTemporal Catalog is to be able to query @@ -100,15 +100,14 @@ show up in STAC API searches, as most will at least implicitly use a geometry. T satellite data in mind, one can easily imagine other data types that start with a less precise geometry but have it refined after processing. -### Non-STAC Items +### Data that is not spatial -The other case that often comes up is people who love STAC and want to use it to catalog everything they have, even if its -not spatial. In this case our recommendation is to re-use the STAC structures, creating Items that have everything the same, -except they do not have a geometry. We have discussed mechanisms to make these 'non-spatial' Items differentiated in the -specification, like using an alternate link structure in catalogs. But there are no actual implementations of this, so if -you have one please get in touch and we can figure out how to support it. We likely will not consider these to be actually -STAC-compliant, as the contract in STAC is that Items should be those that represent an asset in space and time. But we -hope to design a solution that enables 'mixed' catalogs with compliant and non-compliant STAC items. +The other case that often comes up is people who love STAC and want to use it to catalog everything they have, even if it is +not spatial. This use case is not currently supported by STAC, as we are focused on data that is both temporal and spatial +in nature. The [OGC API - Records](https://github.com/opengeospatial/ogcapi-records) is an emerging standard that likely +will be able to handle a wider range of data to catalog than STAC. It builds on [OGC API - +Features](https://github.com/opengeospatial/ogcapi-features) just like [STAC API](https://github.com/radiantearth/stac-api-spec/) +does. ## Representing Vector Layers in STAC From a54081d8e51a10507de959a81f1db852f265f70b Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Tue, 23 Jun 2020 12:20:47 -0700 Subject: [PATCH 7/9] Update item-spec/item-spec.md Co-authored-by: Matthias Mohr --- item-spec/item-spec.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/item-spec/item-spec.md b/item-spec/item-spec.md index b3c06ace1..7ba640ad1 100644 --- a/item-spec/item-spec.md +++ b/item-spec/item-spec.md @@ -33,7 +33,7 @@ inherited from GeoJSON. | stac_extensions | \[string] | A list of extensions the Item implements. | | id | string | **REQUIRED.** Provider identifier. As most geospatial assets are already defined by some identification scheme by the data provider it is recommended to simply use that ID. Data providers are advised to include sufficient information to make their IDs globally unique, including things like unique satellite IDs. | | type | string | **REQUIRED.** Type of the GeoJSON Object. MUST be set to `Feature`. | -| geometry | [GeoJSON Geometry Object](https://tools.ietf.org/html/rfc7946#section-3.1) | **REQUIRED.** Defines the full footprint of the asset represented by this item, formatted according to [RFC 7946, section 3.1](https://tools.ietf.org/html/rfc7946#section-3.1). The footprint should be the default GeoJSON geometry, though additional geometries can be included. Coordinates are specified in Longitude/Latitude or Longitude/Latitude/Elevation based on [WGS 84](http://www.opengis.net/def/crs/OGC/1.3/CRS84). | +| geometry | [GeoJSON Geometry Object](https://tools.ietf.org/html/rfc7946#section-3.1) \| [null](https://tools.ietf.org/html/rfc7946#section-3.2) | **REQUIRED.** Defines the full footprint of the asset represented by this item, formatted according to [RFC 7946, section 3.1](https://tools.ietf.org/html/rfc7946#section-3.1). The footprint should be the default GeoJSON geometry, though additional geometries can be included. Coordinates are specified in Longitude/Latitude or Longitude/Latitude/Elevation based on [WGS 84](http://www.opengis.net/def/crs/OGC/1.3/CRS84). | | bbox | \[number] | **REQUIRED if `geometry` is not `null`.** Bounding Box of the asset represented by this item, formatted according to [RFC 7946, section 5](https://tools.ietf.org/html/rfc7946#section-5). | | properties | [Properties Object](#properties-object) | **REQUIRED.** A dictionary of additional metadata for the item. | | links | \[[Link Object](#link-object)] | **REQUIRED.** List of link objects to resources and related URLs. A link with the `rel` set to `self` is strongly recommended. | From 8f2a66e10e84c9aa2353a71f9745ecc55611475c Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Tue, 23 Jun 2020 12:39:58 -0700 Subject: [PATCH 8/9] added line about collection assets --- best-practices.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/best-practices.md b/best-practices.md index 8739473e4..cceece0e7 100644 --- a/best-practices.md +++ b/best-practices.md @@ -107,7 +107,7 @@ not spatial. This use case is not currently supported by STAC, as we are focused in nature. The [OGC API - Records](https://github.com/opengeospatial/ogcapi-records) is an emerging standard that likely will be able to handle a wider range of data to catalog than STAC. It builds on [OGC API - Features](https://github.com/opengeospatial/ogcapi-features) just like [STAC API](https://github.com/radiantearth/stac-api-spec/) -does. +does. The [collection assets extension](extensions/collection-assets) may also provide an option for some use cases. ## Representing Vector Layers in STAC From 0369fb094396677dd47fcc106d79cb70d1cf1c75 Mon Sep 17 00:00:00 2001 From: Matthias Mohr Date: Wed, 24 Jun 2020 10:10:21 +0200 Subject: [PATCH 9/9] Adapt JSON Schema --- CHANGELOG.md | 1 + item-spec/json-schema/item.json | 63 ++++++++++++++++++++++++--------- 2 files changed, 47 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5dc08a73f..f01c5c06f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. ### Changes - Clarification on null geometries, making bbox not required if a null geometry is used. + ### Removed ### Fixed diff --git a/item-spec/json-schema/item.json b/item-spec/json-schema/item.json index be5835ea1..595dfb458 100644 --- a/item-spec/json-schema/item.json +++ b/item-spec/json-schema/item.json @@ -34,6 +34,52 @@ { "$ref": "https://geojson.org/schema/Feature.json" }, + { + "oneOf": [ + { + "type": "object", + "required": [ + "geometry", + "bbox" + ], + "properties": { + "geometry": { + "$ref": "https://geojson.org/schema/Geometry.json" + }, + "bbox": { + "type": "array", + "oneOf": [ + { + "minItems": 4, + "maxItems": 4 + }, + { + "minItems": 6, + "maxItems": 6 + } + ], + "items": { + "type": "number" + } + } + } + }, + { + "type": "object", + "required": [ + "geometry" + ], + "properties": { + "geometry": { + "type": "null" + }, + "bbox": { + "not": {} + } + } + } + ] + }, { "type": "object", "required": [ @@ -41,7 +87,6 @@ "id", "links", "assets", - "bbox", "properties" ], "properties": { @@ -73,22 +118,6 @@ "description": "Provider item ID", "type": "string" }, - "bbox": { - "type": "array", - "oneOf": [ - { - "minItems": 4, - "maxItems": 4 - }, - { - "minItems": 6, - "maxItems": 6 - } - ], - "items": { - "type": "number" - } - }, "links": { "title": "Item links", "description": "Links to item relations",