Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filter Extension #128

Merged
merged 28 commits into from
May 26, 2021
Merged
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
6d51718
initial draft for Filter extension
philvarner May 10, 2021
3f51664
filter extension: update text with a few missing details
philvarner May 11, 2021
1d32f24
add more details around implementation requirements
philvarner May 11, 2021
ae9db54
draft openapi spec
philvarner May 11, 2021
26d22ce
update examples, and implementation section
philvarner May 12, 2021
facfd04
remove stray comma
philvarner May 12, 2021
e5031e8
wordsmith text
philvarner May 12, 2021
d22cda2
wordsmith text
philvarner May 12, 2021
c810d2c
wordsmith text
philvarner May 12, 2021
7edc330
improve openapi definition
philvarner May 12, 2021
eba4135
add text about synthetic queryables
philvarner May 13, 2021
7913862
Merge branch 'dev' into filter-extension
philvarner May 14, 2021
e8490d9
add better example, fix linting error in markdown and schemas
philvarner May 14, 2021
518184f
fix ref to cql yml
philvarner May 14, 2021
2bdf845
minor fixes for filter openapi
philvarner May 14, 2021
0b115eb
Merge branch 'dev' into filter-extension
philvarner May 19, 2021
e372618
clarification on queryables, line breaks, wordsmithing
philvarner May 19, 2021
9f9a3dd
fix formatting
philvarner May 19, 2021
7265eb8
redefine conformance classes
philvarner May 20, 2021
820a18e
change conformance class names back to item-search#
philvarner May 20, 2021
c201898
tweak language around proprietary extensions in queryables, add note …
philvarner May 20, 2021
96696c4
update list of conformance class uris in extension page
philvarner May 20, 2021
90c88da
add conformance class field to each extension
philvarner May 20, 2021
61a1f36
fix formatting of filter conformance classes
philvarner May 20, 2021
1fb71cb
fix formatting typo
philvarner May 20, 2021
9acf92d
clarify queryables
philvarner May 20, 2021
b9b7c83
add a getting started section
philvarner May 20, 2021
8e08794
modify text based on comments from Rob
philvarner May 24, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add text about synthetic queryables
  • Loading branch information
philvarner committed May 13, 2021
commit eba41352ec9bda279ac266313bc036d2e6a6faae
17 changes: 12 additions & 5 deletions fragments/filter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,28 +54,29 @@ OAFeat Part 3 CQL formally defines syntax for both a text format (cql-text) as a
## Limitations of Item Search
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section is more a justification for the extension, and perhaps doesn't need to be so prominent. It was a little confusing to me reading through the spec as I was expecting this section to begin talking about what this extensions supports. I'd recommend making a section lower down dedicated to talking about other parts of the spec in comparison or justifying this extension, and have the beginning of this document focused on specifying what the filter extension is and does.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll think about that. I like having the "motivation" section first, to adequately set the stage for where we're starting with this extension.


OAFeat defines a limited set of filtering capabilities. Filtering can only be done over a single collection and
with only a bbox and datetime parameter.
with only a single `bbox` (rectangular spatial filter) parameter and a single datetime (instant or interval) parameter.

The STAC Item Search specification extends the functionality of OAFeat in a few key ways:
- It allows cross-collection filtering, whereas OAFeat only allows filtering within a single collection. (`collections` parameter, accepting 0 or more collections)
- It allows filtering by Item ID (`ids` parameter)
- It allows filtering based on a single GeoJSON Geometry, rather than only a bbox (`intersects` parameter)

However, it does not contain a formalized way to filter based on arbitrary fields in an Item. For example, there is
no way to express the filter "item.properties.eo:cloud_cover is less than 10".
no way to express the filter "item.properties.eo:cloud_cover is less than 10". It also does not have a way to logically combine
multiple spatial or temporal filters.

## Filter expressiveness

This extension expands the capabilities of Item Search and the OAFeat Items resource with
[OAFeat Part 3 CQL](https://portal.ogc.org/files/96288)
by providing an expressive query language to construct more complex filter predicates. The operators are similar to
those provided by SQL. The Simple CQL conformance class requires the logical operators `AND`, `OR`, and `NOT`;
the comparison operators '=', `<`, `<=`, `>`, `>=`, `LIKE`, `IS NULL`, `BETWEEN`, `IN`; the spatial operator
`INTERSECTS` and the temporal operator `ANYINTERACTS`.
the comparison operators '=', `<`, `<=`, `>`, `>=`, `LIKE`, `IS NULL`, `BETWEEN`, and `IN`; the spatial operator
`INTERSECTS`; and the temporal operator `ANYINTERACTS`.

The ANYINTERACTS operator has effectively the same semantics as the existing `datetime` filter.

For example:
CQL enables these types of queries:
- Use of Item Property values in predicates (e.g., `item.properties.eo:cloud_cover`), using comparison operators
- Items whose `datetime` values are in the month of August of the years 2017-2021, using OR and ANYINTERACTS
- Items whose `geometry` values intersect any one of several Polygons, using OR and INTERSECTS
Expand Down Expand Up @@ -142,6 +143,8 @@ These queryable variables are mapped by the service to filter Items. For example

Queryables can be static or dynamically derived. For example, `cloud_cover` might be specified to have a value 0 to 100 or a field may have a set of enumerated values dynamically determined by an aggreation at runtime. This schema can be used by a UI or interactive client to dynamically expose to the user the fields that are available for filtering, and provide a precise group of options for the values of these variables.

Queryables can also be used to advertise synthesized property values. The only requirement in CQL is that the property have a type and evaluate to literal value of that type or NULL. For example, a filter like "Items must have an Asset with an eo:band with the common_name of 'nir'" can be expressed. A Queryable `assets_bands` could be defined to have a type of array of string and have the semantics that it contains all of `common_name` values across all assets and bands for an Item. This could then be filtered with the CQL expression `'nir' in assets_bands`. Implementations would then expand this expression into the appropriate query against its datastore. (TBD if this will actually work or not. This is also related to the upcoming restriction on property/literal comparisons)

## GET Query Parameters and POST JSON fields

This extension adds three GET query parameters or POST JSON fields to an Item Search request:
Expand Down Expand Up @@ -231,6 +234,10 @@ Queryables endpoint (`/queryables`) returns:
"gsd" : {
"title" : "Ground Sample Distance",
"$ref": "https://schemas.stacspec.org/v1.0.0/item-spec/json-schema/instrument.json#/properties/gsd"
},
"assets_bands" : {
"title" : "Asset eo:bands common names",
"$ref": "https://stac-extensions.github.io/eo/v1.0.0/schema.json#/properties/eo:bands/common_name"
}
}
}
Expand Down