Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue#16: Change apply opera treatment choice and lambda to be specific to HLS #24

Merged
merged 3 commits into from
May 30, 2024
Merged

Issue#16: Change apply opera treatment choice and lambda to be specific to HLS #24

merged 3 commits into from
May 30, 2024

Conversation

voxparcxls
Copy link
Contributor

@voxparcxls voxparcxls commented May 16, 2024 •
edited
Loading

Github Issue: #16

Description

Change apply opera treatment choice and lambda to be specific to HLS

Overview of work done

  • Update state machine
  • Rename lambda: apply_opera_hls_treatment
  • Updated dataset-config in SIT and Github: bignbit-config

Overview of verification done

SIT

"dataType": "OPERA_L3_DSWX-HLS_V1"

dataset-config:

"datasetConfigurationForBIG": {
          "config": {
            "sendToHarmony": false,
            "operaHLSTreatment": true,
            "imageFilenameRegex": ".*BROWSE.tif",
            "imgVariables": [
              {
                "id": "all"
              }
            ],
            "variables": []
          }
 }

Screenshot 2024-05-29 at 3 34 28 PM

Explain how this change was integration tested. Provide screenshots or logs if appropriate.

PR checklist:

  • Linted
  • Updated unit tests
  • Updated changelog
  • Integration testing

See Pull Request Review Checklist for pointers on reviewing this pull request

@voxparcxls voxparcxls marked this pull request as ready for review May 29, 2024 22:39
@frankinspace frankinspace merged commit 7bda932 into develop May 30, 2024
8 of 10 checks passed
@frankinspace frankinspace deleted the feature/update_operaHLSTreatment branch May 30, 2024 16:49
@frankinspace frankinspace linked an issue May 30, 2024 that may be closed by this pull request
Change apply opera treatment choice and lambda to be specific to HLS #16
Closed
frankinspace added a commit that referenced this pull request Feb 4, 2025
@frankinspace @voxparcxls
@torimcd @jake-herrmann @jamesfwood
Merge pull request #49 from podaac/release/0.2.0
43d56cb 
* /version 0.2.0a0

* feature: convertToPNG --> sendToHarmony (#21)

* update state_machine, sendToHarmony

* --changelog Issue-15

* moved to unreleased, changelog

* /version 0.2.0a1

* Update README.md (#22)

* /version 0.2.0a2

* Use -j to remove top-level directory in zip artifact (#25)

* /version 0.2.0a3

* Issue#16: Change apply opera treatment choice and lambda to be specific to HLS (#24)

* update for HLS -- lambda and state machine

* py lint error

* update file name

* /version 0.2.0a4

* /version 0.2.0a5

* Feature/issue 7 - Remove wait for GITC response (#30)

* fix type in readme

* change task token to uuid in send to gitc

* remove wait for task token & gitc response handler from state machine

* change gitc response handler to invoke save cma message

* fix typo

* fix context parameter

* convert uuid to string

* fix json typo

* remove uuid lib to use stdlib uuid

* change identifier to image set name instead of uuid

* remove uuid import

* fix json formatting of cma invoke

* add granule concept ID to identifier

* update image set name with granule conceptid

* remove whitespace

* move get umm json to utils and save gitc outgoing cnm

* remove unused import

* try setting cmr query env based on stage

* remove task token from tests

* fix granule index

* change how image set name defined

* add region to ssm client in utils

* add ssm parameters to gitc lambdas

* change region reference

* update how granule name referenced in send to gitc

* remove unused cmr var

* fix case

* fix cnm parsing

* reformat save cnm to separate step

* update tf vars for save cnm

* fix type in module definition

* fix cnm in cma

* fix collection reference in save cnm

* add debugging log statements

* fix input

* update save cnm input

* debugging

* debugging

* linting

* debugging

* debugging

* linting

* fix state machine

* update state machine

* reorg state machine

* missing comma

* move save cnm into map

* fix state machine transitions

* fix sm

* fix boolean

* debugging cnm input

* fix input

* change input

* debug input

* add cnm as config parameter

* lint

* fix cnm payload

* remove debugging statements

* add prefix to cnm path and fix gitc response

* increase handle gitc response timeout

* update changelog

* add original shortname as cnm prefix

* revert shortname and change collection ref to save cnm & cnm-r in same location

* remove save cma lambda no longer used

* remove EDL env params from sendtogitc

* change parsing of granule concept id

* /version 0.2.0a6

* Update README.md

* /version 0.2.0a7

* Support datetimes without microseconds (#37)

* Support datetimes without microseconds

* changelog

* Update generate_image_metadata.py (#39)

add docstring

* /version 0.2.0a8

* /version 0.2.0a9

* Issues/23: Enable Harmony write output to user bucket (#46)

* add s3 destination to harmony job

* syntax

* job success

* harmony block update

* --variable

* temp test in uat

* manual to uat

* test s1-v1

* lint

* deploy --sit

* Update to place harmony results in our bucket

* Update to place harmony results in our bucket

* Update to place harmony results in our bucket

* Update to place harmony results in our bucket

* Update to place harmony results in our bucket

* Update to place harmony results in our bucket

---------

Co-authored-by: Josh <[email protected]>

* /version 0.2.0a10

* fix deploy failure

* /version 0.2.0a11

* Add new message attribute response_topic_arn to all outgoing messages to GIBS (#47)

* /version 0.2.0a12

* prep for 0.2.0

* /version 0.2.0a13

* Fix build error

* /version 0.2.0a14

* /release 0.2.0

* Update graph

* /version 0.2.0rc1

* Turn prevent_destroy back on for gibs response topic

* /version 0.2.0rc2

* Improve error message when missing config

* /version 0.2.0rc3

* Improve error message when missing config

* /version 0.2.0rc4

* UAT is in east-1 region

* /version 0.2.0rc5

* fix gibs queue name

* /version 0.2.0rc6

* calculate md5sum for files from harmony

* calculate md5sum for files from harmony

* /version 0.2.0rc7

* bump chunk size to 100 MB

* /version 0.2.0rc8

* Update test message for UAT, add checksum to generated tif, add logging, fix type error

* Lint

* Lint

* /version 0.2.0rc9

* Change named variables for audit path from pobit to bignbit

* /version 0.2.0rc10

* Update docs

---------

Co-authored-by: frankinspace <[email protected]>
Co-authored-by: Josh Haile <[email protected]>
Co-authored-by: Victoria McDonald <[email protected]>
Co-authored-by: Jake Herrmann <[email protected]>
Co-authored-by: James Wood <[email protected]>
Co-authored-by: jamesfwood <[email protected]>
Co-authored-by: Josh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@frankinspace frankinspace frankinspace approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Change apply opera treatment choice and lambda to be specific to HLS
2 participants
@voxparcxls @frankinspace