Find out how to include metadata into geospatial layers saved in GeoServer

[huard]

I've created a dataset using geopandas and saved it as a geopackage. This geopackage was then uploaded to GeoServer, and we can then access it using a WFS interface from QGIS. However, there is no accompanying metadata.

I'd like to know how to include information such as
Identifier, Title, Type, Language, Abstract, Keywords
so that users of the WFS endpoint know what they're getting and where it's coming from.

[Zeitsperre]

I think that in order to share metadata information for a particular layer in WFS, that information needs to be appended to the file at its source as there aren't any specific fields offered in the layer publishing section of GeoServer management. On inspection, the fields I can see in the layer preview when using the WFS service are the same ones visible for a general shapefile/GeoPackage:

I haven't explored much of the standards for metadata sharing in WMS/WMTS/WFS/WCS, so this could be a // TODO for me to look into.

There does exist an alternative means using a metadata link, but I get the impression this might only work for WMS and WMTS layers:

[huard]

@Zeitsperre Could you create a step-by-step guide on how to prepare a geopackage dataset so that the metadata information appears on GeoServer? Could be included in the pavics-sdi docs. If you need a deadline, I suggest end of February.

[huard]

See how to add metadata using Fiona:
https://github.com/Toblerity/Fiona/pull/821/files#diff-3121d1f70912305d7e409dfd557f34d9f5869a86d8d347b0a4cca1e1c124df42R15

Only thing remaining would be to match that with an appropriate metadata standard.

[Zeitsperre]

I've been working on this issue this afternoon and have come up against quite a few complications. It looks like, although the GeoPackage metadata standard is ISO certified, very few tools have been built to support it.

• I thought that QGIS might have this capability, but layer metadata is only saved in the GeoPackage if the project is saved within the gpkg (Save geopackage metadata inside the datasource qgis/QGIS#25119 and also Overhaul Metadata Management In QGIS qgis/QGIS-Enhancement-Proposals#91)
  • Some plugins exist for this capability, but they either require online account connections (?) or their documentation/support is threadbare (and they fail to install on Linux).
• The example above points to changes implemented in a version of Fiona not currently published (v1.9). The Fiona developers have been releasing maintainance patches of Fiona v1.8 to PyPI and Conda-forge (https://anaconda.org/conda-forge/fiona/files) semi-regularly, but this version still doesn't make use of GDAL's metadata standards. (Expose GDALs metadata Toblerity/Fiona#950 and Passing XML metadata (e.g. ISO 19135 standard) to GDB, or other shapes Toblerity/Fiona#566).
  • Good news is It looks as though v1.9a is on our doorstep (1.9.0 Release Toblerity/Fiona#1053).

If you're as curious as I am, we could try making some examples using the development version of Fiona (with the understanding that when v2.0 is released, we return to our documentation to revise it). Thoughts?

[Zeitsperre]

Good news: There is allegedly a community module available within GeoServer for modifying layer metadata (including bulk modifications) (https://docs.geoserver.org/2.19.x/en/user/community/metadata/index.html).
Bad news: It's not listed among the available plugins (https://sourceforge.net/projects/geoserver/files/GeoServer/2.19.0/extensions/).

Will keep poking around to see if I can find it (perhaps it's in a community-based repository?).

@tlvu what is the current procedure for adding modules to our GeoServer instances?

[Zeitsperre]

Found it: https://mvnrepository.com/artifact/org.geoserver.community/gs-metadata
(specific to our version of GS: https://mvnrepository.com/artifact/org.geoserver.community/gs-metadata/2.19.0)

There's a warning concerning some missing Java dependencies that could potentially break our instance when running this plugin. The fix is in a newer version of GeoServer, but the workaround is described here: https://osgeo-org.atlassian.net/browse/GEOS-10078

[huard]

If building Fiona from source is not painful, then yes, we could experiment with the master. I would be surprised if the mechanism we'll be relying on changed between now and 1.9. If the build is tricky, let's wait for the next release.

To be clear, what I want us to be able to do is

1. programatically feed metadata attributes into a gpkg, ideally we'd do this right when creating the gpkg with Fiona, and not as a separate operation on GeoServer;
2. confirm that those metadata are understood and displayed by GeoServer;
3. confirm that WFS requests to geoserver return this metadata with the response.

[Zeitsperre]

Yes, that would be ideal. Newer GDAL (>=3.0) allows for interfacing with the metadata fields using the ISO standards, but again there are very few tools currently supporting it. At the very least, the metadata plugin allows for bulk categorization using a template (useful for organisation name, contact info, etc.). I think we agree that the last we want to do is personalize non-exportable metadata in our GeoServer for 40+ layers.

[huard]

Agreed. I see two issues:

1. current layers on geoserver, most from 3rd parties, with little to no metadata.
2. WPS outputs. We have no process at the moment that publich gpkg, but I suspect we'll do sooner or later. This is the use case I had in mind, but you're right we'll need something for 1.

[Zeitsperre]

For reference: https://gis.stackexchange.com/questions/421124/what-are-the-geopackage-metadata-conventions-supported-in-geoserver

[tlvu]

@tlvu what is the current procedure for adding modules to our GeoServer instances?

Very simple if already bundle in the docker image we use, we just have to activate it: https://github.com/bird-house/birdhouse-deploy/blob/0329715a6f6ee81f5435257c98ea5b8eaee8acc1/birdhouse/default.env#L83-L85

We might be lucky, there is a "metadata" plugin available here https://github.com/kartoza/docker-geoserver/blob/034d477e02da3df637f68ef2b824ff0c18bcf26a/build_data/community_plugins.txt#L31 Hope it is the same as "gs-metadata".

[Zeitsperre]

@tlvu Can you enable gs-metadata plugin on the new GeoServer rebuild? It looks like that's the approach we need based on the repsonse I received from Andrea Aime (a GeoServer core dev) on StackExchange.

[tlvu]

@tlvu Can you enable gs-metadata plugin on the new GeoServer rebuild? It looks like that's the approach we need based on the repsonse I received from Andrea Aime (a GeoServer core dev) on StackExchange.

@Zeitsperre there is no gs-metadata in the list but only metadata so that's why I wasn't sure in my previous comment.

OK will enable this on my test VM and let you have access once it's done.

[tlvu]

@Zeitsperre

Here you go: https://lvupavics.ouranos.ca/geoserver, same admin passwd as prod since I synced prod data.

Data back from around October 2021 when I upgraded Geoserver. Let me know if enough or you need more recent data sync from prod.

Hope this screenshot is what you are looking for:

[tlvu]

@Zeitsperre Geoserver data synced with prod. You have latest data from prod on https://lvupavics.ouranos.ca/geoserver/.

[huard]

QGIS 3.24 can export metadata to GPKG. It appears as an XML document stored in the "tags".

{'DESCRIPTION': 'Absolute and relative changes between 1990-2010 and 2080-2100 for 1,000 and 10,000-year flood values, estimated from climate simulations from CESM1 and CanESM2 driving the GR4J-Cemaneige hydrological model.',
 'GPKG_METADATA_ITEM_1': '<!DOCTYPE qgis PUBLIC \'http://mrcc.com/qgis.dtd\' \'SYSTEM\'>\n<qgis version="3.24.0-Tisler">\n  <identifier>This is the identifier</identifier>\n  <parentidentifier>This is the parent identifier</parentidentifier>\n  <language>This is the Language</language>\n  <type>This is the Type</type>\n  <title>This is the title</title>\n  <abstract>Absolute and relative changes between 1990-2010 and 2080-2100 for 1,000 and 10,000-year flood values, estimated from climate simulations from CESM1 and CanESM2 driving the GR4J-Cemaneige hydrological model.</abstract>\n  <keywords vocabulary="gmd:topicCategory">\n    <keyword>Climatology Meteorology Atmosphere</keyword>\n    <keyword>Geoscientific Information</keyword>\n  </keywords>\n  <contact>\n    <contactAddress>\n      <type>postal</type>\n      <address>950 Sherbrooke St.</address>\n      <city>Montréal</city>\n      <administrativearea>Qc</administrativearea>\n      <postalcode>H3A1B9</postalcode>\n      <country>Canada</country>\n    </contactAddress>\n    <name>David Huard</name>\n    <organization>Ouranos</organization>\n    <position>Specialist</position>\n    <voice>514-282-2424</voice>\n    <fax></fax>\n    <email>[email protected]</email>\n    <role>owner</role>\n  </contact>\n  <links>\n    <link size="" name="undefined 1" mimeType="" url="" format="" description="" type=""/>\n    <link size="" name="undefined 2" mimeType="" url="" format="" description="" type=""/>\n  </links>\n  <history>history patate pouet</history>\n  <fees></fees>\n  <rights></rights>\n  <license>Creative Commons CC-BY 4.0</license>\n  <encoding></encoding>\n  <crs>\n    <spatialrefsys nativeFormat="Wkt">\n      <wkt></wkt>\n      <proj4></proj4>\n      <srsid>0</srsid>\n      <srid>0</srid>\n      <authid></authid>\n      <description></description>\n      <projectionacronym></projectionacronym>\n      <ellipsoidacronym></ellipsoidacronym>\n      <geographicflag>true</geographicflag>\n    </spatialrefsys>\n  </crs>\n  <extent>\n    <spatial crs="" maxx="0" maxy="0" dimensions="2" maxz="0" minx="0" minz="0" miny="0"/>\n    <temporal>\n      <period>\n        <start></start>\n        <end></end>\n      </period>\n    </temporal>\n  </extent>\n</qgis>\n',
 'IDENTIFIER': 'This is the identifier'}