Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/develop' into feature_2769_tc_di…
Browse files Browse the repository at this point in the history
…ag_driver_v0.11.0
  • Loading branch information
JohnHalleyGotway committed Feb 1, 2024
2 parents 331d38c + 06d7825 commit d75c09d
Show file tree
Hide file tree
Showing 63 changed files with 3,696 additions and 2,268 deletions.
22 changes: 22 additions & 0 deletions docs/Users_Guide/config_options.rst
Original file line number Diff line number Diff line change
Expand Up @@ -515,6 +515,28 @@ When running with this option, users are advised to refer to section
:numref:`config_tmp_dir` and write temporary files to a personal location rather than
the default shared :code:`/tmp` directory.

.. _met_python_debug:

MET_PYTHON_DEBUG
----------------

The MET_PYTHON_DEBUG environment variable can be set to enable debugging log messages
related to Python embedding. These log messages are disabled by default. The environment
variable can be set to a value of :code:`all` for all log messages, :code:`dataplane`
for log messages when reading gridded data, or :code:`point` for log messages when
reading point data.

.. _met_python_tmp_format:

MET_PYTHON_TMP_FORMAT
---------------------

The MET_PYTHON_TMP_FORMAT environment variable defines whether temporary files for
Python embedding should be written as NetCDF files or using JSON/NumPy serialization.
By default, they are written using JSON for attributes and NumPy serialization for data
to avoid NetCDF library conflicts between MET and Python. Setting this environment
variable to :code:`netcdf` enables the use of temporary NetCDF files instead.

Settings Common to Multiple Tools
=================================

Expand Down
21 changes: 19 additions & 2 deletions docs/Users_Guide/mode.rst
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,23 @@ The **multivar_intensity_compare_fcst** and **multivar_intensity_compare_obs** c

When regridding to the FCST or OBS field (e.g. to_grid = FCST), the first field of the field array is used from the forecast and observation field dictionaries, respectively. All regridding is then done to that grid. Other regrid options described in :ref:`regrid` can also be used as normal.

"file_type" can be set independently for each input in multivariate mode. If not set for an input, MET uses file names and file content to determine the type.

When setting a threshold to a percentile, some choices require both an observation input and a forecast input. When this is the case, it's assumed the indices match, so for example if forecast input 1 has such a percentile setting, then observation input 1 will be used to compute the percentile. Percentiles in which this will happen are:

* SFP in an observation input.
* The matching forecast input will be used to determine the threshold. e.g. ">SFP33.3" in the 2nd observation input means greater than 33.3-rd percentile of the 2nd forecast input will be used as the threshold for that observation input.

* SOP in a forecast input.
* The matching observation input will be used to determine the threshold. e.g. ">SOP33.3" in the 2nd forecast input means greater than 33.3-rd percentile of the 2nd observation input will be used as the threshold for that forecast input.

* "==FBIAS" in an observation input.
* e.g. "==FBIAS1" in an observation input to automatically de-bias the data, using a simple threshold in the matching forecast input. For example, when observation input 3 has "==FBIAS1", and forecast input 3 has ">5.0", MET applies the >5.0 threshold to the forecast and then chooses an observation threshold which results in a frequency bias of 1. The frequency bias can be any float value > 0.0.

* "==FBIAS" in a forecast input.
* e.g. "==FBIAS1" in a forecast input to automatically de-bias the data, using a simple threshold in the matching observation input. For example, when forecast input 2 has "==FBIAS1", and observation input 2 has ">5.0", MET applies the >5.0 threshold to the observation and then chooses a forecast threshold which results in a frequency bias of 1. The frequency bias can be any float value > 0.0.


Practical Information
=====================

Expand Down Expand Up @@ -260,15 +277,15 @@ _____________________
multivar_name = "Super";
The **multivar_name** entry appears only in the **MODEMultivarConfig_default** file. This option is used only when the multivar option is enabled, and only when all **multivar_intensity_flag** values are FALSE. It can be thought of as an identifier for the multivariate super object. It shows up in output files names and content. It can be set separately for forecasts and observations or as a common value for both.
The **multivar_name** entry appears only in the **MODEMultivarConfig_default** file. This option is used only when the multivar option is enabled, and only when **multivar_intensity_compare_fcst** and **multivar_intensity_compare_obs** are empty. It can be thought of as an identifier for the multivariate super object. It shows up in output files names and content. It can be set separately for forecasts and observations or as a common value for both.

_____________________

.. code-block:: none
multivar_level = "LO";
The **multivar_level** entry appears only in the **MODEMultivarConfig_default** file. This option is used only when the multivar option is enabled, and only when all **multivar_intensity_flag** values are FALSE. It is the identifier for the multivariate super object as regards level. It shows up in output files names and content. If not set the default value is "NA". It can be set separately for forecasts and observations, or as a common value for both.
The **multivar_level** entry appears only in the **MODEMultivarConfig_default** file. This option is used only when the multivar option is enabled, and only when **multivar_intensity_compare_fcst** and **multivar_intensity_compare_obs** are empty. It is the identifier for the multivariate super object as regards level. It shows up in output files names and content. If not set the default value is "NA". It can be set separately for forecasts and observations, or as a common value for both.

_____________________

Expand Down
274 changes: 274 additions & 0 deletions internal/test_unit/config/MODEConfig_multivar
Original file line number Diff line number Diff line change
@@ -0,0 +1,274 @@
////////////////////////////////////////////////////////////////////////////////
//
// MODE configuration file.
//
// For additional information, see the MET_BASE/config/README file.
//
////////////////////////////////////////////////////////////////////////////////

//
// Output model name to be written
//
// model =
model = "HRRR";

//
// Output description to be written
//
// desc =

//
// Output observation type to be written
//
// obtype =
obtype = "ANALYSIS";

////////////////////////////////////////////////////////////////////////////////

//
// Verification grid
//
regrid = {to_grid = FCST;method = NEAREST;width = 1;vld_thresh = 0.5;}

////////////////////////////////////////////////////////////////////////////////

//
`// Approximate grid resolution (km)
//
// grid_res =
//

////////////////////////////////////////////////////////////////////////////////

//
// Run all permutations of radius and threshold
//
// quilt =
// ${METPLUS_QUILT}

//
// MODE Multivar boolean combination logic
//
//multivar_logic =
//${METPLUS_MULTIVAR_LOGIC}
multivar_logic = "#1 && #2 && #3";

// keep this around to compare to older versions, the new version doesn't read this
multivar_intensity_flag = [FALSE, TRUE, TRUE];

multivar_intensity_compare_fcst = [ 2, 3 ];
multivar_intensity_compare_obs = [ 2, 3 ];

//
// Forecast and observation fields to be verified
//
fcst = {
//${METPLUS_FCST_FIELD}
field = [{ name="CSNOW"; level="L0"; conv_radius = 0; conv_thresh = ==1; file_type = GRIB2; merge_flag = NONE; },
{ name="VIS"; level="L0"; conv_radius = 5; conv_thresh = <=804.672; merge_thresh = <=1207.008; merge_flag = THRESH; },
{ name="WIND"; level="Z10"; conv_radius = 5; conv_thresh = >=8.9408; merge_thresh = >=6.7056; merge_flag = THRESH; }];

//${METPLUS_FCST_CENSOR_THRESH}
//${METPLUS_FCST_CENSOR_VAL}
//${METPLUS_FCST_CONV_RADIUS}
//${METPLUS_FCST_CONV_THRESH}
//${METPLUS_FCST_VLD_THRESH}
//${METPLUS_FCST_FILTER_ATTR_NAME}
filter_attr_name = ["AREA"];

//${METPLUS_FCST_FILTER_ATTR_THRESH}
filter_attr_thresh = [>=25];

//${METPLUS_FCST_MERGE_THRESH}
//${METPLUS_FCST_MERGE_FLAG}
//${METPLUS_FCST_FILE_TYPE}
//file_type = GRIB2;
multivar_name = "Snow";
multivar_level = "LO";
}

obs = {
//${METPLUS_OBS_FIELD}
field = [{ name="PrecipFlag"; level="L0"; conv_radius = 0; conv_thresh = ==3; file_type = GRIB2; merge_flag = NONE; },
{ name="VIS"; level="L0"; conv_radius = 5; conv_thresh = <=804.672; merge_thresh = <=1207.008; merge_flag = THRESH; },
{ name="WIND"; level="Z10"; conv_radius = 5; conv_thresh = >=8.9408; merge_thresh = >=6.7056; merge_flag = THRESH; } ];

//${METPLUS_OBS_CENSOR_THRESH}
//${METPLUS_OBS_CENSOR_VAL}
//${METPLUS_OBS_CONV_RADIUS}
//${METPLUS_OBS_CONV_THRESH}
//${METPLUS_OBS_VLD_THRESH}

//${METPLUS_OBS_FILTER_ATTR_NAME}
filter_attr_name = ["AREA"];

//${METPLUS_OBS_FILTER_ATTR_THRESH}
filter_attr_thresh = [>=25];

// ${METPLUS_OBS_MERGE_THRESH}
// ${METPLUS_OBS_MERGE_FLAG}
// ${METPLUS_OBS_FILE_TYPE}
multivar_name = "Precip";
multivar_level = "LO";
}

////////////////////////////////////////////////////////////////////////////////

//
// Handle missing data
//
// mask_missing_flag =
// ${METPLUS_MASK_MISSING_FLAG}

//
// Match objects between the forecast and observation fields
//
//match_flag =
//${METPLUS_MATCH_FLAG}
match_flag = MERGE_BOTH;
//
// Maximum centroid distance for objects to be compared
//
//max_centroid_dist =
//${METPLUS_MAX_CENTROID_DIST}

////////////////////////////////////////////////////////////////////////////////

//
// Verification masking regions
//
//mask = {
//${METPLUS_MASK_DICT}

////////////////////////////////////////////////////////////////////////////////

//
// Fuzzy engine weights
//
//weight = {
//${METPLUS_WEIGHT_DICT}

////////////////////////////////////////////////////////////////////////////////

//
// Fuzzy engine interest functions
//
interest_function = {

//${METPLUS_INTEREST_FUNCTION_CENTROID_DIST}

//${METPLUS_INTEREST_FUNCTION_BOUNDARY_DIST}

//${METPLUS_INTEREST_FUNCTION_CONVEX_HULL_DIST}

angle_diff = (
( 0.0, 1.0 )
( 30.0, 1.0 )
( 90.0, 0.0 )
);

aspect_diff = (
( 0.00, 1.0 )
( 0.10, 1.0 )
( 0.75, 0.0 )
);

corner = 0.8;
ratio_if = (
( 0.0, 0.0 )
( corner, 1.0 )
( 1.0, 1.0 )
);

area_ratio = ratio_if;

int_area_ratio = (
( 0.00, 0.00 )
( 0.10, 0.50 )
( 0.25, 1.00 )
( 1.00, 1.00 )
);

curvature_ratio = ratio_if;

complexity_ratio = ratio_if;

inten_perc_ratio = ratio_if;
}

////////////////////////////////////////////////////////////////////////////////

//
// Total interest threshold for determining matches
//
//total_interest_thresh =
//${METPLUS_TOTAL_INTEREST_THRESH}

//
// Interest threshold for printing output pair information
//
print_interest_thresh = 0.0;

////////////////////////////////////////////////////////////////////////////////

//
// Plotting information
//
met_data_dir = "MET_BASE";

fcst_raw_plot = {
color_table = "MET_BASE/colortables/met_default.ctable";
plot_min = 0.0;
plot_max = 0.0;
}

obs_raw_plot = {
color_table = "MET_BASE/colortables/met_default.ctable";
plot_min = 0.0;
plot_max = 0.0;
}

object_plot = {
color_table = "MET_BASE/colortables/mode_obj.ctable";
}

//
// Boolean for plotting on the region of valid data within the domain
//
plot_valid_flag = FALSE;

//
// Plot polyline edges using great circle arcs instead of straight lines
//
plot_gcarc_flag = FALSE;

////////////////////////////////////////////////////////////////////////////////

//
// NetCDF matched pairs, PostScript, and contingency table output files
//
//ps_plot_flag =
//${METPLUS_PS_PLOT_FLAG}

//nc_pairs_flag = {
//${METPLUS_NC_PAIRS_FLAG_DICT}

//ct_stats_flag =
//${METPLUS_CT_STATS_FLAG}


////////////////////////////////////////////////////////////////////////////////

shift_right = 0; // grid squares

////////////////////////////////////////////////////////////////////////////////

//${METPLUS_OUTPUT_PREFIX}
//version = "V10.0";

//tmp_dir = "${MET_TMP_DIR}";
tmp_dir = "/tmp";

////////////////////////////////////////////////////////////////////////////////

//${METPLUS_MET_CONFIG_OVERRIDES}
Loading

0 comments on commit d75c09d

Please sign in to comment.