-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ERA5T Support - Cloud Run Job Migration #95
base: main
Are you sure you want to change the base?
Conversation
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
de52a0d
to
f200bb9
Compare
f200bb9
to
f63fc58
Compare
ENV PATH /opt/conda/envs/${CONDA_ENV_NAME}/bin:$PATH | ||
RUN pip install -e . | ||
|
||
ARG arco_era5_git_rev=era5t-support |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update the branch name before merging this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
@@ -15,8 +15,9 @@ name: era5 | |||
channels: | |||
- conda-forge | |||
dependencies: | |||
- python=3.9.17 | |||
- python=3.8.19 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using Python 3.8 is concerning to me. Python 3.8 is very old, already at end of life. This means it no longer receives security updates: https://devguide.python.org/versions/
Even Python 3.9 will soon be at end of life. We should strive to use newer versions of Python which are actively maintained.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed @shoyer. The plan is to remove this environment.yml
and Docker
entirely from the repository. As most of the code is working without these on dataflow
. Also we'll definitely upgrade with the newer python version and latest zarr
version (v3
) in future.
In the current ARCO-ERA5, files are updated from ECMWF on a monthly cadence (on roughly the 9th of each month) with a 3 month delay. This PR added support of ERA5T, in which files are updated on a daily cadence with a 6 days delay.
This new ARCO-ERA5 working into the 3 CRON-JOB.
Note: Actually we got the data 5 day behind the current_day but we took 1 day's buffer for the safety purposes.
Fix: #82