Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update time statistics functions #1189

Merged
merged 4 commits into from
Sep 9, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 15 additions & 9 deletions doc/sphinx/source/recipes/recipe_oceans.rst
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,7 @@ For this reason, we recommend extracting a small number of specific layers in
the preprocessor, using the `extract_layer` preprocessor.

This script can not process NetCDFs with multiple time steps. Please use the
`time_average` preprocessor to collapse the time dimension.
`climate_statistics` preprocessor to collapse the time dimension.

This diagnostic also includes the optional arguments, `threshold` and
`thresholds`.
Expand All @@ -311,7 +311,7 @@ For a Global 2D field:
.. code-block:: yaml

prep_map_1:
time_average:
climate_statistics:


For a regional 2D field:
Expand All @@ -324,7 +324,8 @@ For a regional 2D field:
end_longitude: 30.
start_latitude: -80.
end_latitude: 80.
time_average:
climate_statistics:
operator: mean

For a Global 3D field at the surface and 10m depth:

Expand All @@ -335,7 +336,8 @@ For a Global 3D field at the surface and 10m depth:
extract_levels:
levels: [0., 10.]
scheme: linear_horizontal_extrapolate_vertical
time_average:
climate_statistics:
operator: mean


For a multi-model comparison mean of 2D global fields including contour thresholds.
Expand All @@ -344,7 +346,8 @@ For a multi-model comparison mean of 2D global fields including contour threshol

prep_map_4:
custom_order: true
time_average:
climate_statistics:
operator: mean
regrid:
target_grid: 1x1
scheme: linear
Expand Down Expand Up @@ -390,7 +393,8 @@ An appropriate preprocessor for a 2D field would be:
.. code-block:: yaml

prep_quad_map:
time_average:
climate_statistics:
operator: mean

and an example of an appropriate diagnostic section of the recipe would be:

Expand Down Expand Up @@ -440,7 +444,7 @@ This diagnostic also includes the optional arguments, `maps_range` and
`diff_range` to manually define plot ranges. Both arguments are a list of two floats
to set plot range minimun and maximum values respectively for Model and Observations
maps (Top panels) and for the Model minus Observations panel (bottom left).
Note that if input data have negative values the Model over Observations map
Note that if input data have negative values the Model over Observations map
(bottom right) is not produced.

The scatter plots plot the matched model coordinate on the x axis, and the
Expand All @@ -457,7 +461,8 @@ An appropriate preprocessor for a 3D+time field would be:
extract_levels:
levels: [100., ]
scheme: linear_extrap
time_average:
climate_statistics:
operator: mean
regrid:
target_grid: 1x1
scheme: linear
Expand Down Expand Up @@ -573,7 +578,8 @@ An appropriate preprocessor for a 3D+time field would be:

.. code-block:: yaml

time_average:
climate_statistics:
operator: mean
extract_slice:
latitude: [-50.,50.]
longitude: 332.
Expand Down
2 changes: 1 addition & 1 deletion environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ channels:

dependencies:
# Python packages that cannot be installed from PyPI:
- esmvalcore>=2.0.0b1,<2.1
- esmvalcore>=2.0.0b2,<2.1
# Non-Python dependencies
- graphviz
- cdo
Expand Down
3 changes: 2 additions & 1 deletion esmvaltool/diag_scripts/ocean/diagnostic_maps.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@
extract_levels:
levels: [100., ]
scheme: linear_extrap
time_average:
climate_statistics:
operator: mean


Note that this recipe may not function on machines with no access to the
Expand Down
3 changes: 2 additions & 1 deletion esmvaltool/diag_scripts/ocean/diagnostic_maps_quad.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@
extract_levels:
levels: [100., ]
scheme: linear_extrap
time_average:
climate_statistics:
operator: mean

This diagnostic also requires the ``exper_model``, ``exper_model`` and
``observational_dataset`` keys in the recipe::
Expand Down
3 changes: 2 additions & 1 deletion esmvaltool/diag_scripts/ocean/diagnostic_model_vs_obs.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@
extract_levels:
levels: [100., ]
scheme: linear_extrap
time_average:
climate_statistics:
operator: mean
regrid:
target_grid: 1x1
scheme: linear
Expand Down
3 changes: 2 additions & 1 deletion esmvaltool/diag_scripts/ocean/diagnostic_transects.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@

preprocessors:
prep_transect:
time_average:
climate_statistics:
operator: mean
extract_slice: # Atlantic Meridional Transect
latitude: [-50.,50.]
longitude: 332.
Expand Down
12 changes: 6 additions & 6 deletions esmvaltool/diag_scripts/shared/_validation.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import os
import logging
import iris
from esmvalcore.preprocessor import time_average
from esmvalcore.preprocessor import climate_statistics
from esmvaltool.diag_scripts.shared import select_metadata


Expand Down Expand Up @@ -53,12 +53,12 @@ def get_control_exper_obs(short_name, input_data, cfg, cmip_type):


# apply supermeans: handy function that loads CONTROL, EXPERIMENT
# and OBS (if any) files and applies time_average() to mean the cubes
# and OBS (if any) files and applies climate_statistics() to mean the cubes
def apply_supermeans(ctrl, exper, obs_list):
"""
Apply supermeans on data components ie MEAN on time

This function is an extension of time_average() meant to ease the
This function is an extension of climate_statistics() meant to ease the
time-meaning procedure when dealing with CONTROL, EXPERIMENT and OBS
(if any) datasets.
ctrl: dictionary of CONTROL dataset
Expand All @@ -71,14 +71,14 @@ def apply_supermeans(ctrl, exper, obs_list):
exper_file = exper['filename']
ctrl_cube = iris.load_cube(ctrl_file)
exper_cube = iris.load_cube(exper_file)
ctrl_cube = time_average(ctrl_cube)
exper_cube = time_average(exper_cube)
ctrl_cube = climate_statistics(ctrl_cube)
exper_cube = climate_statistics(exper_cube)
if obs_list:
obs_cube_list = []
for obs in obs_list:
obs_file = obs['filename']
obs_cube = iris.load_cube(obs_file)
obs_cube = time_average(obs_cube)
obs_cube = climate_statistics(obs_cube)
obs_cube_list.append(obs_cube)
else:
obs_cube_list = None
Expand Down
9 changes: 3 additions & 6 deletions esmvaltool/recipes/recipe_ocean_Landschuetzer2016.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@ preprocessors:
# --------------------------------------------------
# For a 2D global surface map
prep_surface_map_2D:
time_average:
climate_statistics:
operator: mean
regrid:
target_grid: 2x2
scheme: linear
Expand Down Expand Up @@ -110,8 +111,4 @@ diagnostics:
script: ocean/diagnostic_maps.py
Global_Ocean_model_vs_obs:
script: ocean/diagnostic_model_vs_obs.py
observational_dataset: {dataset: Landschuetzer2016, project: OBS}




observational_dataset: {dataset: Landschutzer2016, project: OBS}
3 changes: 2 additions & 1 deletion esmvaltool/recipes/recipe_ocean_amoc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,8 @@ preprocessors:
end_year: 2003
end_month: 12
end_day: 31
time_average:
climate_statistics:
operator: mean

diagnostics:
# --------------------------------------------------
Expand Down
22 changes: 14 additions & 8 deletions esmvaltool/recipes/recipe_ocean_bgc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -89,40 +89,45 @@ preprocessors:

# For a 2D global surface map
prep_surface_map_2D:
time_average:
climate_statistics:
operator: mean

# For a 3D global surface map
prep_surface_map_3D:
extract_levels:
levels: [0., ]
scheme: linear_horizontal_extrapolate_vertical
time_average:
climate_statistics:
operator: mean

prep_surface_map_regrid_3D:
custom_order: true
extract_levels:
levels: [0., ]
scheme: linear_horizontal_extrapolate_vertical
time_average:
climate_statistics:
operator: mean
regrid:
target_grid: 1x1
scheme: linear


prep_global_profile:
annual_mean:
annual_statistics:
operator: mean
area_statistics:
operator: mean

prep_global_profile_decadal:
annual_mean:
decadal: true
decadal_statistics:
operator: mean
area_statistics:
operator: mean

prep_transect_AMT: # Atlantic Meridional Transect (28W)
custom_order: true
time_average:
climate_statistics:
operator: mean
extract_region:
start_longitude: 320.
end_longitude: 345.
Expand All @@ -138,7 +143,8 @@ preprocessors:
# prep_depth_integration:
# depth_integration:
# # new_units: kg m-2 # need to specify in advance, as cf_units has strange behaviour.
# time_average:
# climate_statistics:
# operator: mean
#
# # 2D map global depth integration time series maps
# prep_depth_integration_timeseries:
Expand Down
27 changes: 18 additions & 9 deletions esmvaltool/recipes/recipe_ocean_example.yml
Original file line number Diff line number Diff line change
Expand Up @@ -141,14 +141,16 @@ preprocessors:
# Map preprocessors - 2D fields
# --------------------------------------------------
prep_map_1: # For Global 2D fields
time_average:
climate_statistics:
operator: mean

prep_map_2: # For Global 2D fields with regridding
custom_order: true
regrid:
target_grid: 1x1
scheme: linear
time_average:
climate_statistics:
operator: mean
multi_model_statistics:
span: overlap
statistics: [mean ]
Expand All @@ -159,7 +161,8 @@ preprocessors:
extract_levels:
levels: [0., 10., 100., 1000.,]
scheme: linear_horizontal_extrapolate_vertical
time_average:
climate_statistics:
operator: mean
regrid:
target_grid: 1x1
scheme: linear
Expand All @@ -174,7 +177,8 @@ preprocessors:
end_longitude: 30.
start_latitude: -80.
end_latitude: 80.
time_average:
climate_statistics:
operator: mean
regrid:
target_grid: 1x1
scheme: linear
Expand All @@ -187,7 +191,8 @@ preprocessors:
extract_levels:
levels: [0., 10., 100., 1000.,]
scheme: linear_horizontal_extrapolate_vertical
time_average:
climate_statistics:
operator: mean
extract_region:
start_longitude: -80.
end_longitude: 30.
Expand All @@ -204,20 +209,23 @@ preprocessors:
# Transects preprocessors
# --------------------------------------------------
prep_transect_1: # For extracting along a trajectory
time_average:
climate_statistics:
operator: mean
extract_trajectory:
latitudes: [-5., 5.,]
longitudes: [332.,332.]
number_points: 4 # VERY SLOW with high numbers!

prep_transect_2: # For extracting a transect
time_average:
climate_statistics:
operator: mean
extract_transect: # Pacific Equator
latitude: 0.
longitude: [120., 300.]

prep_transect_3: # For extracting a transect
time_average:
climate_statistics:
operator: mean
extract_transect: # Atlantic Meridional Transect
latitude: [-50.,50.]
longitude: 332.
Expand Down Expand Up @@ -247,7 +255,8 @@ preprocessors:
# --------------------------------------------------
prep_depth_integration_1: # For a 2D map global profile
depth_integration:
time_average:
climate_statistics:
operator: mean


diagnostics:
Expand Down
3 changes: 2 additions & 1 deletion esmvaltool/recipes/recipe_ocean_quadmap.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@ preprocessors:
end_year: 2003
end_month: 12
end_day: 31
time_average:
climate_statistics:
operator: mean
regrid:
target_grid: 1x1
scheme: linear
Expand Down
2 changes: 1 addition & 1 deletion meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ requirements:
- cython
- eofs
- esmpy
- esmvalcore>=2.0.0b1,<2.1
- esmvalcore>=2.0.0b2,<2.1
- jinja2
- matplotlib
- nc-time-axis
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
'cython',
'jinja2',
'eofs',
'esmvalcore>=2.0.0b1,<2.1',
'esmvalcore>=2.0.0b2,<2.1',
'fiona',
'matplotlib<3',
'nc-time-axis', # needed by iris.plot
Expand Down