Update develop-ref after #1650 (#1652)

* Per 1646, one line fix for cut-and-paste error. (#1647) * Per #1643, redefine the contents of the existing AREA_RATIO output column from MODE. Define it as FCST/OBS object area instead of min/max. Update the User's Guide to note the change and also clarify that the MTD VOLUME_RATIO output really is FCST/OBS. (#1650) * Feature 1644 ps_log (#1651) * Per #1644, write rejection reason codes at verbosity 2 when there are 0 matched pairs. * Per #1644, add a few sentences to Point-Stat, Practical Information chapter about debugging 0 matched pairs. * The mode_conv.pl logic was slightly broken. MET PR #1650 should have broken the NB but it did not. Turns out the diffing logic is NOT properly distinguishing between single and pair object lines. It does this by looking for an underscore in the OBJECT_ID column. When we added FCST_UNITS and OBS_UNITS, that shifted OBJECT_ID up 2 spots, but the code was still checking the (0-based) 20th column instead of the 22nd. Fixing this now and will rerun NB20210202 to confirm it works again. * The diffing logic for MODE pair lines still was not correct. We'd added the ASPECT_DIFF and CURVATURE_RATIO columns a while ago, but they were missing from the diff logic. This logic really is not good. We need to make it more robust, reading the version-specific header columns from a table file instead of hard-coding them! Co-authored-by: John Halley Gotway <[email protected]>
dtcenter · Feb 3, 2021 · 35eadf2 · 35eadf2
1 parent 994f5a5
commit 35eadf2
Show file tree

Hide file tree

Showing 9 changed files with 52 additions and 33 deletions.
diff --git a/met/docs/Users_Guide/mode-td.rst b/met/docs/Users_Guide/mode-td.rst
@@ -622,7 +622,7 @@ The contents of the OBJECT_ID and OBJECT_CAT columns identify the objects using
     - Difference in object direction of movement
   * - 30
     - VOLUME_RATIO
-    - Ratio of object volumes
+    - Forecast object volume divided by observation object volume
   * - 31
     - START_TIME_DELTA
     - Difference in object starting time steps

diff --git a/met/docs/Users_Guide/mode.rst b/met/docs/Users_Guide/mode.rst
@@ -818,7 +818,8 @@ The contents of the columns in this ASCII file are summarized in :numref:`MODE_o
     - Absolute value of the difference between the aspect ratios of two objects (unitless)
   * - 50
     - AREA_RATIO
-    - Ratio of the areas of two objects defined as the lesser of the two divided by the greater of the two (unitless)
+    - The forecast object area divided by the observation object area (unitless) :raw-html:`<br />`
+      **NOTE:** Prior to met-10.0.0, defined as the lesser of the two object areas divided by the greater of the two
   * - 51
     - INTERSECTION :raw-html:`<br />` \_AREA
     - Intersection area of two objects (in grid squares)

diff --git a/met/docs/Users_Guide/point-stat.rst b/met/docs/Users_Guide/point-stat.rst
@@ -252,6 +252,8 @@ _____________________
 
 The Point-Stat tool is used to perform verification of a gridded model field using point observations. The gridded model field to be verified must be in one of the supported file formats. The point observations must be formatted as the NetCDF output of the point reformatting tools described in :numref:`reformat_point`. The Point-Stat tool provides the capability of interpolating the gridded forecast data to the observation points using a variety of methods as described in :numref:`matching-methods`. The Point-Stat tool computes a number of continuous statistics on the matched pair data as well as discrete statistics once the matched pair data have been thresholded.
 
+If no matched pairs are found for a particular verification task, a report listing counts for reasons why the observations were not used is written to the log output at the default verbosity level of 2. If matched pairs are found, this report is written at verbosity level 3. Inspecting these rejection reason counts is the first step in determining why Point-Stat found no matched pairs. The order of the log messages matches the order in which the processing logic is applied. Start from the last log message and work your way up, considering each of the non-zero rejection reason counts.
+
 point_stat usage
 ~~~~~~~~~~~~~~~~
 

diff --git a/met/src/libcode/vx_data2d/data2d_utils.cc b/met/src/libcode/vx_data2d/data2d_utils.cc
@@ -315,7 +315,7 @@ void set_attrs(const VarInfo *info, DataPlane &dp) {
       mlog << Debug(3) << "Resetting accumulation interval from "
            << sec_to_hhmmss(dp.accum()) << " to "
            << sec_to_hhmmss(info->accum_attr()) << ".\n";
-      dp.set_lead(info->accum_attr());
+      dp.set_accum(info->accum_attr());
    }
 
    return;

diff --git a/met/src/libcode/vx_shapedata/engine.cc b/met/src/libcode/vx_shapedata/engine.cc
@@ -3157,8 +3157,8 @@ void write_pair(ModeFuzzyEngine &eng, const Grid & grid, const int n_f, const in
    // Difference in aspect ratio
    at.set_entry(row, c++, eng.pair_single[n].aspect_diff);
 
-   // Area ratio
-   at.set_entry(row, c++, eng.pair_single[n].area_ratio);
+   // Fcst/Obs area ratio
+   at.set_entry(row, c++, eng.pair_single[n].fo_area_ratio);
 
    // Intersection area
    at.set_entry(row, c++,
@@ -3460,8 +3460,8 @@ void write_cluster_pair(ModeFuzzyEngine &eng, const Grid & grid, const int n,
    // Difference in aspect ratio
    at.set_entry(row, c++, eng.pair_cluster[n].aspect_diff);
 
-   // Area ratio
-   at.set_entry(row, c++, eng.pair_cluster[n].area_ratio);
+   // Fcst/Obs area ratio
+   at.set_entry(row, c++, eng.pair_cluster[n].fo_area_ratio);
 
    // Intersection area
    at.set_entry(row, c++,

diff --git a/met/src/libcode/vx_shapedata/interest.cc b/met/src/libcode/vx_shapedata/interest.cc
@@ -348,6 +348,7 @@ void PairFeature::clear()
    convex_hull_dist           = 0.0;
    angle_diff                 = 0.0;
    aspect_diff                = 0.0;
+   fo_area_ratio              = 0.0;
    area_ratio                 = 0.0;
    intersection_area          = 0.0;
    union_area                 = 0.0;
@@ -374,6 +375,7 @@ void PairFeature::assign(const PairFeature &p) {
    convex_hull_dist           = p.convex_hull_dist;
    angle_diff                 = p.angle_diff;
    aspect_diff                = p.aspect_diff;
+   fo_area_ratio              = p.fo_area_ratio;
    area_ratio                 = p.area_ratio;
    intersection_area          = p.intersection_area;
    union_area                 = p.union_area;
@@ -454,8 +456,11 @@ void PairFeature::set(const SingleFeature &fcst,
    //
    // Area ratio
    //
-   area_ratio = min( (Obs->area)/(Fcst->area),
-                     (Fcst->area)/(Obs->area) );
+   fo_area_ratio = (Fcst->area)/(Obs->area);
+
+   area_ratio    = min( (Obs->area)/(Fcst->area),
+                        (Fcst->area)/(Obs->area) );
+
 
    //
    // Intersection, union, and symmetric diff areas
@@ -575,6 +580,7 @@ ostream & operator<<(ostream & out, const PairFeature & p)
    out << "Convex Hull Distance              = " << (p.convex_hull_dist)           << "\n";
    out << "Angle Difference                  = " << (p.angle_diff)                 << "\n";
    out << "Aspect Difference                 = " << (p.aspect_diff)                << "\n";
+   out << "Fcst/Obs Area Ratio               = " << (p.fo_area_ratio)              << "\n";
    out << "Area Ratio                        = " << (p.area_ratio)                 << "\n";
    out << "Intersection Area                 = " << nint(p.intersection_area)      << "\n";
    out << "Union Area                        = " << nint(p.union_area)             << "\n";

diff --git a/met/src/libcode/vx_shapedata/interest.h b/met/src/libcode/vx_shapedata/interest.h
@@ -121,6 +121,7 @@ class PairFeature {
       double convex_hull_dist;
       double angle_diff;
       double aspect_diff;
+      double fo_area_ratio;
       double area_ratio;
       double intersection_area;
       double union_area;

diff --git a/met/src/tools/core/point_stat/point_stat.cc b/met/src/tools/core/point_stat/point_stat.cc
@@ -999,27 +999,34 @@ void process_scores() {
                     << shc.get_interp_pnts_str()
                     << "), using " << pd_ptr->n_obs << " matched pairs.\n";
 
-               // Dump out detailed information about why observations were rejected
-               mlog << Debug(3)
-                    << "Number of matched pairs   = " << pd_ptr->n_obs << "\n"
-                    << "Observations processed    = " << conf_info.vx_opt[i].vx_pd.n_try << "\n"
-                    << "Rejected: station id      = " << conf_info.vx_opt[i].vx_pd.rej_sid << "\n"
-                    << "Rejected: obs var name    = " << conf_info.vx_opt[i].vx_pd.rej_var << "\n"
-                    << "Rejected: valid time      = " << conf_info.vx_opt[i].vx_pd.rej_vld << "\n"
-                    << "Rejected: bad obs value   = " << conf_info.vx_opt[i].vx_pd.rej_obs << "\n"
-                    << "Rejected: off the grid    = " << conf_info.vx_opt[i].vx_pd.rej_grd << "\n"
-                    << "Rejected: topography      = " << conf_info.vx_opt[i].vx_pd.rej_topo << "\n"
-                    << "Rejected: level mismatch  = " << conf_info.vx_opt[i].vx_pd.rej_lvl << "\n"
-                    << "Rejected: quality marker  = " << conf_info.vx_opt[i].vx_pd.rej_qty << "\n"
-                    << "Rejected: message type    = " << conf_info.vx_opt[i].vx_pd.rej_typ[j][k][l] << "\n"
-                    << "Rejected: masking region  = " << conf_info.vx_opt[i].vx_pd.rej_mask[j][k][l] << "\n"
-                    << "Rejected: bad fcst value  = " << conf_info.vx_opt[i].vx_pd.rej_fcst[j][k][l] << "\n"
-                    << "Rejected: bad climo mean  = " << conf_info.vx_opt[i].vx_pd.rej_cmn[j][k][l] << "\n"
-                    << "Rejected: bad climo stdev = " << conf_info.vx_opt[i].vx_pd.rej_csd[j][k][l] << "\n"
-                    << "Rejected: duplicates      = " << conf_info.vx_opt[i].vx_pd.rej_dup[j][k][l] << "\n";
-
-               // Continue for no matched pairs
-               if(pd_ptr->n_obs == 0) continue;
+               // List counts for reasons why observations were rejected
+               cs << cs_erase
+                  << "Number of matched pairs   = " << pd_ptr->n_obs << "\n"
+                  << "Observations processed    = " << conf_info.vx_opt[i].vx_pd.n_try << "\n"
+                  << "Rejected: station id      = " << conf_info.vx_opt[i].vx_pd.rej_sid << "\n"
+                  << "Rejected: obs var name    = " << conf_info.vx_opt[i].vx_pd.rej_var << "\n"
+                  << "Rejected: valid time      = " << conf_info.vx_opt[i].vx_pd.rej_vld << "\n"
+                  << "Rejected: bad obs value   = " << conf_info.vx_opt[i].vx_pd.rej_obs << "\n"
+                  << "Rejected: off the grid    = " << conf_info.vx_opt[i].vx_pd.rej_grd << "\n"
+                  << "Rejected: topography      = " << conf_info.vx_opt[i].vx_pd.rej_topo << "\n"
+                  << "Rejected: level mismatch  = " << conf_info.vx_opt[i].vx_pd.rej_lvl << "\n"
+                  << "Rejected: quality marker  = " << conf_info.vx_opt[i].vx_pd.rej_qty << "\n"
+                  << "Rejected: message type    = " << conf_info.vx_opt[i].vx_pd.rej_typ[j][k][l] << "\n"
+                  << "Rejected: masking region  = " << conf_info.vx_opt[i].vx_pd.rej_mask[j][k][l] << "\n"
+                  << "Rejected: bad fcst value  = " << conf_info.vx_opt[i].vx_pd.rej_fcst[j][k][l] << "\n"
+                  << "Rejected: bad climo mean  = " << conf_info.vx_opt[i].vx_pd.rej_cmn[j][k][l] << "\n"
+                  << "Rejected: bad climo stdev = " << conf_info.vx_opt[i].vx_pd.rej_csd[j][k][l] << "\n"
+                  << "Rejected: duplicates      = " << conf_info.vx_opt[i].vx_pd.rej_dup[j][k][l] << "\n";
+
+               // Print report based on the number of matched pairs
+               if(pd_ptr->n_obs > 0) {
+                  mlog << Debug(3) << cs;
+               }
+               // Continue for zero matched pairs
+               else {
+                  mlog << Debug(2) << cs;
+                  continue;
+               }
 
                // Process percentile thresholds
                conf_info.vx_opt[i].set_perc_thresh(pd_ptr);

diff --git a/test/perl/mode_conv.pl b/test/perl/mode_conv.pl
@@ -34,8 +34,8 @@ ()
                    INTENSITY_90 INTENSITY_50 INTENSITY_SUM);
 
 my @fld_pairs = qw(N_VALID GRID_RES OBJECT_ID OBJECT_CAT CENTROID_DIST BOUNDARY_DIST CONVEX_HULL_DIST ANGLE_DIFF
-                   AREA_RATIO INTERSECTION_AREA UNION_AREA SYMMETRIC_DIFF INTERSECTION_OVER_AREA
-                   COMPLEXITY_RATIO PERCENTILE_INTENSITY_RATIO INTEREST);
+                   ASPECT_DIFF AREA_RATIO INTERSECTION_AREA UNION_AREA SYMMETRIC_DIFF INTERSECTION_OVER_AREA
+                   CURVATURE_RATIO COMPLEXITY_RATIO PERCENTILE_INTENSITY_RATIO INTEREST);
 
 my @fld_ctss  = qw(N_VALID GRID_RES FIELD TOTAL FY_OY FY_ON FN_OY FN_ON BASER FMEAN ACC FBIAS PODY PODN POFD FAR
                    CSI GSS HK HSS ODDS);
@@ -101,11 +101,13 @@ ()
       "%14s"  . # BOUNDARY_DIST
       "%17s"  . # CONVEX_HULL_DIST
       "%11s"  . # ANGLE_DIFF
+      "%11s"  . # ASPECT_DIFF
       "%11s"  . # AREA_RATIO
       "%18s"  . # INTERSECTION_AREA
       "%11s"  . # UNION_AREA
       "%15s"  . # SYMMETRIC_DIFF
       "%23s"  . # INTERSECTION_OVER_AREA
+      "%17s"  . # CURVATURE_RATIO
       "%17s"  . # COMPLEXITY_RATIO
       "%27s"  . # PERCENTILE_INTENSITY_RATIO
       "%12s";   # INTEREST
@@ -205,7 +207,7 @@ ()
   }
 
   # write a single object attribute line
-  elsif( $vals[20] !~ /_/ ){
+  elsif( $vals[22] !~ /_/ ){
     next if( $type eq "p");
     push @outs, (" MODE_SOA ", @vals[2,3,22 .. 43]);
     $fmt_val = $fmt_sing;