Fix ValueError from chunks in time in existing workflows #128
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Error message is "Multiple chunks along the main adjustment dimension time is not supported". Ideally, we'd resolve this by changing chunking in upstream workflow steps, but `train_aiqpd` was written so we would not need to address these kinds of breaking changes until later. In other words, `train_aiqpd` needs to support bad input chunking to support legacy.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes bug from: 'ValueError: ... Multiple chunks along the main adjustment dimension time is not supported' when running
dodola train-aiqpd.We get this from running the new
dodola train-aiqpdin legacy workflows. Ideally, we'd resolve this by changing chunking in upstream workflow steps, buttrain_aiqpdwas written so we would not need to address these kinds of breaking changes until a later time. In other words,train_aiqpdneeds to support bad input dataset chunking to support legacy. This PR fixes it so that it does so.We're loading the input reference datasets before "training" AIQPD as a solution because this is the process used by the
apply-aiqpdservice. So it's consistent with existing behavior.