Feature 2706 mvmode obs fcst input size (#2742)

Co-authored-by: Tracy Hertneky <[email protected]>
Co-authored-by: John Halley Gotway <[email protected]>

data/config/MODEMultivarConfig_default

@@ -54,9 +54,11 @@ quilt = FALSE;
 multivar_logic = "#1 && #2 && #3";
 
 //
-// MODE Multivar intensity computation flag
+// MODE Multivar intensity comparison settings
 //
-multivar_intensity_flag = [FALSE, TRUE, TRUE];
+multivar_intensity_compare_fcst = [ 2, 3 ];
+multivar_intensity_compare_obs = [ 2, 3 ];
+
 
 //
 // Forecast and observation fields to be verified

docs/Users_Guide/mode.rst

@@ -112,11 +112,11 @@ Traditionally, MODE defines objects by smoothing and thresholding data from a si
 
 As described in :numref:`MODE-configuration-file`, the **field** entry in the forecast and observation dictionaries define the input data to be processed. If **field** is defined as a dictionary, the traditional method for running MODE is invoked, where objects are defined using a single input field. If **field** is defined as an array of dictionaries, each specifying a different input field, then the multi-variate MODE logic is invoked and requires the **multivar_logic** configuration entry to be set. Traditional MODE is run once for each input field to define objects for that field. Note that the object definition criteria can be defined separately for each field array entry. The objects from each input field are combined into forecast and observation data *super* objects 
 
-The **multivar_logic** and **multivar_intensity_flag** configuration entries, described in :numref:`MODE-configuration-file`, define the boolean logic for combining objects from multiple fields into *super* objects, and then optionally computing intensities for individual input fields when the input is masked to non-missing only inside the *super* objects. If the **multivar_logic** configuration option iset, there must be the same number of fields defined in an array of dictionaries for fcst and for obs as indicated in the **multivar_logic**. Note that the multi-variate MODE forecast and observation input fields and combination logic do not need to
-match. If a particular  **multivar_intensity_flag** value is TRUE the corresponding pair of fields (fcst and obs) are masked to non-missing inside the fcst and obs super objects, and traditional mode is run on that pair of masked inputs producing uniquely named outputs. If it is FALSE, mode is not run for that pair of inputs.
+The **multivar_logic** configuration entry, described in :numref:`MODE-configuration-file`, defines the boolean logic for combining objects from multiple fields into *super* objects. It can be defined once to apply to both the forecast and observation dictionaries if the field array lengths are the same, or defined separately within each dictionary. If defined separately within each dictionary, the field array lengths do not need to be the same for the forecast and observations. Note that the multi-variate MODE forecast and observation input fields and combination logic do not need to match.
 
-If all **multivar_intensity_flag** values are FALSE, the forecast and observation *super* objects are written to NetCDF, text, and postscript output files in the standard mode output format, but with no intensity information. 
+The **multivar_intensity_compare_fcst** and **multivar_intensity_compare_obs** configuration entries, described in :numref:`MODE-configuration-file`, define the field array indexes for which to optionally compare intensities for individual input fields when the input is masked to non-missing only inside the *super* objects and are required to be the same length. For example, if **multivar_intensity_compare_fcst = [ 1, 2 ];** and **multivar_intensity_compare_obs = [ 2, 3 ];**, then index 1 (2) of the forecast field array will be compared with index 2 (3) of the observation field array. If an intensity comparision is requested, the corresponding pair of fields (fcst and obs) are masked to non-missing inside the fcst and obs super objects, and traditional mode is run on that pair of masked inputs producing uniquely named outputs. If **multivar_intensity_compare_fcst** and **multivar_intensity_compare_obs** are empty, the forecast and observation *super* objects are written to NetCDF, text, and postscript output files in the standard mode output format, but with no intensity information.
 
+When regridding to the FCST or OBS field (e.g. to_grid = FCST), the first field of the field array is used from the forecast and observation field dictionaries, respectively. All regridding is then done to that grid. Other regrid options described in :ref:`regrid` can also be used as normal.
 
 Practical Information
 =====================
@@ -243,15 +243,16 @@ _____________________
 
 The **multivar_logic** entry appears only in the **MODEMultivarConfig_default** file. This option applies to running multi-variate MODE by setting **field** to an array of dictionaries to define multiple input fields. Objects are defined separately for each input field based on the configuration settings specified for each field array entry. The **multivar_logic** entry is a string which defines how objects for each field are combined into a final *super* object. The objects for each field are referred to as '#N' where N is the N-th field array entry. The '&&' and '||' strings define intersection and union logic, respectively. For example, "#1 && #2" is the intersection of the objects from the first and second fields. "(#1 && #2) || #3" is the union of that intersection with the objects from the third field.
 
-The **multivar_logic** entry is parsed separately from the **fcst** and **obs** dictionaries and can be defined differently in each.
+The **multivar_logic** entry is parsed separately from the **fcst** and **obs** dictionaries and can be defined differently in each. It does not require the same number of fields in the forecast and observation arrays.
 
 _____________________
 
 .. code-block:: none
 
-   multivar_intensity_flag = [FALSE, TRUE, TRUE];
+   multivar_intensity_compare_fcst = [1,2];
+   multivar_intensity_compare_obs = [2,3];
 
-The **multivar_intensity_flag** entry appears only in the **MODEMultivarConfig_default** file. This option is paired with the **multivar_logic** entry, and can take on a value of TRUE or FALSE for each **field**.  In the multivar case, super objects are created using the **multivar_logic** settings. For each input for which **multivar_intensity_flag** is TRUE, the input is masked to be non-missing only within the super objects, and traditional mode is run on that masked input.  For each input for which **multivar_intensity_flag** is FALSE, the input is skipped over.   If all the multivar_intensity_flag values are FALSE, traditional mode output is created for the super objects, but with no intensity information.
+The **multivar_intensity_compare_fcst** and **multivar_intensity_compare_obs** entries appear only in the **MODEMultivarConfig_default** file. These entries define an index in the field arrays to be compared for forecast and observation intensities and must be the same length. For example, in the above example, forecast field 1 will be compared to observation field 2 for computing intensity attribute statistics. If the **multivar_intensity_compare_fcst** and **multivar_intensity_compare_obs** are empty, traditional mode output is created for the super objects, but with no intensity information. 
 
 _____________________
 

internal/test_unit/config/MODEConfig_multivar_fake_data

@@ -54,9 +54,10 @@ quilt = FALSE;
 multivar_logic = "#1 && #2 && #3";
 
 //
-// MODE Multivar intensity logic
+// MODE Multivar intensity comparisons
 //
-multivar_intensity_flag = [FALSE, FALSE, FALSE];
+multivar_intensity_compare_fcst = [  ];
+multivar_intensity_compare_obs = [  ];
 
 //
 // Forecast and observation fields to be verified

internal/test_unit/config/MODEConfig_multivar_fake_data_with_intensities

@@ -56,7 +56,9 @@ multivar_logic = "#1 && #2 && #3";
 //
 // MODE Multivar intensity logic
 //
-multivar_intensity_flag = [TRUE, TRUE, TRUE];
+multivar_intensity_compare_fcst = [ 1, 2, 3 ];
+multivar_intensity_compare_obs = [ 1, 2, 3 ];
+
 
 //
 // Forecast and observation fields to be verified

src/basic/vx_config/config_constants.h

@@ -908,7 +908,8 @@ static const char conf_key_shift_right[]           = "shift_right";
 static const char conf_key_multivar_logic          [] = "multivar_logic";
 static const char conf_key_multivar_name           [] = "multivar_name";
 static const char conf_key_multivar_level          [] = "multivar_level";
-static const char conf_key_multivar_intensity_flag [] = "multivar_intensity_flag";
+static const char conf_key_fcst_multivar_compare_index [] = "multivar_intensity_compare_fcst";
+static const char conf_key_obs_multivar_compare_index [] = "multivar_intensity_compare_obs";
 
 //
 //  MTD specific parameter key names

src/basic/vx_config/config_file.cc

@@ -188,6 +188,14 @@ return;
 ////////////////////////////////////////////////////////////////////////
 
 
+void MetConfig::debug_dump(int depth) const
+{
+   dump(cout, depth);
+}
+
+////////////////////////////////////////////////////////////////////////
+
+
 void MetConfig::dump(ostream & out, int depth) const
 
 {

src/basic/vx_config/config_file.h

@@ -58,6 +58,8 @@ class MetConfig : public Dictionary {
 
       void dump(std::ostream &, int = 0) const;
 
+      void debug_dump(int = 0) const;
+
          //
          //  set stuff
          //

src/libcode/vx_shapedata/engine.cc

@@ -190,6 +190,8 @@ void ModeFuzzyEngine::init_from_scratch() {
 
    clear_features();
 
+   data_type = ModeDataType_Traditional;
+
    return;
 }
 
@@ -245,21 +247,26 @@ void ModeFuzzyEngine::clear_colors() {
 void ModeFuzzyEngine::set(const ShapeData &fcst_wd, const ShapeData &obs_wd)
 
 {
+   if (data_type == ModeDataType_MvMode_Fcst) {
+      set_fcst(fcst_wd);
+   } else if (data_type == ModeDataType_MvMode_Obs) {
+      set_obs(obs_wd);
+   } else {
+      ConcatString path;
 
-   ConcatString path;
-
-   clear_features();
-   clear_colors();
-   ctable.clear();
+      clear_features();
+      clear_colors();
+      ctable.clear();
 
-   collection.clear();
+      collection.clear();
 
-   set_fcst(fcst_wd);
-   set_obs(obs_wd);
+      set_fcst(fcst_wd);
+      set_obs(obs_wd);
 
-   path = replace_path(conf_info.object_pi.color_table.c_str());
+      path = replace_path(conf_info.object_pi.color_table.c_str());
 
-   ctable.read(path.c_str());
+      ctable.read(path.c_str());
+   }
 
    return;
 
@@ -271,21 +278,26 @@ void ModeFuzzyEngine::set(const ShapeData &fcst_wd, const ShapeData &obs_wd)
 void ModeFuzzyEngine::set_no_conv(const ShapeData &fcst_wd, const ShapeData &obs_wd)
 
 {
+   if (data_type == ModeDataType_MvMode_Fcst) {
+      set_fcst_no_conv(fcst_wd);
+   } else if (data_type == ModeDataType_MvMode_Obs) {
+      set_obs_no_conv  ( obs_wd);
+   } else {      
+      ConcatString path;
 
-   ConcatString path;
+      clear_features();
+      clear_colors();
+      ctable.clear();
 
-   clear_features();
-   clear_colors();
-   ctable.clear();
+      collection.clear();
 
-   collection.clear();
+      set_fcst_no_conv (fcst_wd);
+      set_obs_no_conv  ( obs_wd);
 
-   set_fcst_no_conv (fcst_wd);
-   set_obs_no_conv  ( obs_wd);
+      path = replace_path(conf_info.object_pi.color_table.c_str());
 
-   path = replace_path(conf_info.object_pi.color_table.c_str());
-
-   ctable.read(path.c_str());
+      ctable.read(path.c_str());
+   }
 
    return;
 
@@ -297,21 +309,27 @@ void ModeFuzzyEngine::set_only_split(const ShapeData &fcst_wd, const ShapeData &
 
 {
 
-   ConcatString path;
-
-   clear_features();
-   clear_colors();
-   ctable.clear();
+   if (data_type == ModeDataType_MvMode_Fcst) {
+      set_fcst_only_split (fcst_wd);
+   } else if (data_type == ModeDataType_MvMode_Obs) {
+      set_obs_only_split  ( obs_wd);
+   } else {
+      ConcatString path;
 
-   collection.clear();
+      clear_features();
+      clear_colors();
+      ctable.clear();
 
-   set_fcst_only_split (fcst_wd);
-   set_obs_only_split  ( obs_wd);
+      collection.clear();
 
-   path = replace_path(conf_info.object_pi.color_table.c_str());
+      set_fcst_only_split (fcst_wd);
+      set_obs_only_split  ( obs_wd);
 
-   ctable.read(path.c_str());
+      path = replace_path(conf_info.object_pi.color_table.c_str());
 
+      ctable.read(path.c_str());
+   }
+
    return;
 
 }
@@ -1579,7 +1597,7 @@ void ModeFuzzyEngine::do_fcst_merge_thresh(const ShapeData &merge_data) {
 ///////////////////////////////////////////////////////////////////////
 
 void ModeFuzzyEngine::do_obs_merge_thresh(const ShapeData &merge_data) {
-   const char *method_name = "ModeFuzzyEngine::do_fcst_merge_thresh() -> ";
+   const char *method_name = "ModeFuzzyEngine::do_obs_merge_thresh() -> ";
    int j, mid, oid;
    int n_obs_merge;
    ShapeData obs_merge_split;
@@ -1643,7 +1661,6 @@ void ModeFuzzyEngine::do_obs_merge_thresh(const ShapeData &merge_data) {
    // Calculate the composite object sets
    //
    for(mid=1; mid<=n_obs_merge; mid++) {
-
       // Ignore merge objects containing a single simple object
       if(mrg_to_obj_map[mid].size() <= 1) continue;
 
@@ -2482,6 +2499,14 @@ void ModeFuzzyEngine::do_cluster_features() {
 
 ///////////////////////////////////////////////////////////////////////
 
+void ModeFuzzyEngine::set_data_type(ModeDataType type)
+{
+   data_type = type;
+   conf_info.set_data_type(type);
+}
+
+///////////////////////////////////////////////////////////////////////
+
 int ModeFuzzyEngine::get_info_index(int pair_n) const {
    int i;
 
@@ -3939,7 +3964,7 @@ void calc_fcst_clus_ch_mask(const ModeFuzzyEngine &eng, ShapeData &mask) {
    Box bb;
 
    if(eng.need_fcst_clus_split) {
-      mlog << Error << "\ncalc_fcst_clus_ch_mask -> "
+      mlog << Error << "\ncalc_fcst_clus_ch_mask() -> "
            << "should not be called with need_fcst_clus_split set to true\n\n";
 
       exit(1);
@@ -4002,7 +4027,7 @@ void calc_obs_clus_ch_mask(const ModeFuzzyEngine &eng, ShapeData &mask) {
    Box bb;
 
    if(eng.need_obs_clus_split) {
-      mlog << Error << "\ncalc_obs_clus_ch_mask -> "
+      mlog << Error << "\ncalc_obs_clus_ch_mask() -> "
            << "should not be called with need_obs_clus_split set to true\n\n";
       exit(1);
    }

src/libcode/vx_shapedata/engine.h

@@ -14,6 +14,7 @@
 ///////////////////////////////////////////////////////////////////////////////
 
 #include "mode_conf_info.h"
+#include "mode_data_type.h"
 
 #include "interest.h"
 #include "set.h"
@@ -284,6 +285,8 @@ class ModeFuzzyEngine {
 
       void do_cluster_features();
 
+      void set_data_type(ModeDataType type);
+
          //
          // Configuration information
          //
@@ -368,8 +371,10 @@ class ModeFuzzyEngine {
       int get_matched_obs(int)    const;
       int get_unmatched_obs(int)  const;
 
+
       SetCollection collection;
 
+      ModeDataType data_type;
 };
 
 ///////////////////////////////////////////////////////////////////////////////