fix: Do not use SENSING_TIME for Temporal/RangeDateTime field #50
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
When running HLS-VI jobs for Landsat data products I encountered an error during the STAC metadata generation,
This was caused by trying to parse the
SENSING_TIME
from the (non-VI) HLS version of the*cmr.xml
. I can't quite grok the origin of the bug here, but the metadata in the HDF has 1 extra digit (7 instead of 6) in the%f
part of the datetime stamp. We already fixed this as part of the HLS metadata creation code here,https://github.com/NASA-IMPACT/hls-metadata/blob/df1133d29a13426728ec2f7db644cd6b6af7e014/metadata_creator/metadata_creator.py#L283C41-L295
So this PR simply avoids the buggy
SENSING_TIME
definition because that issue is unique to Landsat (Sentinel-2 is fine). This way we don't have to conditionally fix for Landsat what has already been fixed upstream. This value seems safe because we're usingdatetime.strftime
to write the value inhls-landsat-tile
, but I could be missing some complication (e.g., that we tried to resolve in #35)To try to further confirm I pulled ~8,000 (and counting) of the Landsat
*cmr.xml
from our prod forward production bucket and inspected theTemporal/RangeDateTime/BeginningDateTime
field. All of these fields were formatted nicely as%Y-%m-%dT%H:%M:%S.%fZ
How I did it
Temporal/RangeDateTime
field in the HLS-VI version of theCMR.xml
file because we inherit the basics from the HLS product and don't need to override that.parse_sensing_time
code as cleanupHow you can test it
The authoritative way is via the container,
but this is quicker if you have a local dev setup,