Align STAC Filtering with OGC CQL

[cholmes]

STAC wrote its own filter language, since OGC API - Features had not established one. But the OGC community is now aligning around CQL, see http://docs.opengeospatial.org/DRAFTS/19-079.pdf and https://github.com/opengeospatial/ogcapi-features/tree/master/extensions/cql/

On the STAC Call of 7/20/20 everyone agreed that we should aim to drop our custom language and adopt CQL.

We should redo the specification to use CQL exactly, just as the features API does. And for STAC we want to require the use of CQL, so STAC is a profile on top of Features API that requires a bit more. Users who want to create a STAC implementation without supporting filtering can just implement STAC core (and point a more capable STAC API at it to ingest).

[cholmes]

Moving to beta.2, as OGC CQL is a bit unstable beneath us, see #51 (comment)

[dwilson1988]

@cholmes
Hey, I'm in the process of watching this moving target. Currently working on our next generation of STAC API (implemented in Go). Currently using the query extension as it stands in the repo, but while we are upgrading, this appears to be a good time to align with CQL.

I haven't touched CQL since my days working with Geoserver, it looks like it's changed up a bit from what I remember.

From what I can tell individual JSON filters are AND'd and OR'd together with a literal? Is that correct? Does STAC plan to do something a little more clear for the query extension, such as say {"and": [<filter_1>, <filter_2>]}? Will it be in the query extension or mirror OAFeat directly? I'm curious on the progress with this. Even if it's not fully settled, I would like to start an implementation on our end.

[cholmes]

Hey @dwilson1988 - the plan for the next STAC release is to fully align with the latest OGC CQL. They made some tweaks to their JSON since I wrote the PR. There are a couple of STAC implementations right now investigating CQL. The plan is to deprecate our STAC 'query' extension, since it never really got to fruition, and just fully adopt the OGC CQL (using the filter keyword). So yes, we'll just mirror OAFeat directly.

Filter & CQL is in public comment period, and we pushed them a bit to keep it open for longer, so it'll probably be open for the next couple months. They're still able to take feedback there. So it'd be great if you implemented CQL, and gave any feedback like about the AND and OR on the JSON. Indeed I think they just evolved their 'between' to be a bit more straightforward. Best place to follow and give feedback is https://github.com/opengeospatial/ogcapi-features/issues?q=is%3Aissue+is%3Aopen+label%3A%22Part+3%3A+CQL%22 Our goal in STAC is to work with them directly, and then have STAC just be a 'user' of CQL.

[dwilson1988]

Cool, thanks for the quick response!

[dwilson1988]

@cholmes, I'm assuming that means "search" will gain a "filter" parameter as well?

[cholmes]

@dwilson1988 - yeah, exactly. For beta.2 release we'll have both query and filter defined in 'fragments', and reused as classes in both search and features api (and for filter they will just be the exact OGC conformance classes). But query will be deprecated, with a plan to remove it fully by 1.0.0.

This is all subject to feedback though - if STAC community finds major problems with CQL we can not go that route. But the CQL team seems open to feedback, so my hope is that if we do find things they will be open to it. I think the main one would just be the ability to have the CQL 'core' requirements be a bit more minimal, which I think some other people have already brought up.

Keep us posted on your progress as you implement, and at some point I'll gather STAC feedback up to give to CQL. Or just give it directly on their tracker.

[cholmes]

@dwilson1988 - Note the other thing that I missed in the current PR was 'queryables'. It looks like it's a required part of the core conformance, and it does make sense to me, so I think we need to include it. But I think for STAC we can make some recommendations for how it works, since we have set fields, so we can just specify how they should be queried.

[dwilson1988]

@cholmes - Awesome. We'll be working on our implementation of CQL over the next few weeks. I had query working great, but it looks like CQL will be more complete. It does also appear they addressed booleans in JSON like how I had envisioned, I just missed that. Agree queryables will be nice to have, but a pain to implement appropriately. :)

STAC is core to the software we're building, so you'll probably see more of me in the future. When we have more of use for the community, we'll contribute back where we can.

[cholmes]

Cool. I'm super psyched on a Go implementation. Planet is all about Go these days. If you do an open-source a Go CQL library (no prob if that's not your intent) then I'd guess Planet would become a user and hopefully contributor down the line.

Take good notes on your implementation struggles, as I think the biggest thing STAC could do for CQL is to recommend an easier learning curve. Like if queryables or WKT parsing or anything is a big pain we can recommend they make it not required. Though I do think it's ok for some reasonable 'step up' as the core Features spec gives you BBOX and time filtering. The challenge is just to hit the right level of granularity.

And just checked out seer.ai - would love to learn more about what you're up to with STAC and in general. If you're up for it ping me on gitter - username cholmes.

[dwilson1988]

@cholmes - will do. Not sure if an open source Go CQL library will come out of this or not (not opposed to it either).