Add set_attr config file options to override the metadata read from input gridded data files.

[dwfncar]

This idea came up during a met_help exchange with Ying Lin at NCEP:


https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=85872

Here's the idea:

There is currently no way to override what Point-Stat writes to the FCST_VAR output column. There actually is logic for doing this in STAT-Analysis. You can use the "-set_hdr" option to explicitly specify the contents of the output header columns. For example, if you run STAT-Analysis to aggregate data where VX_MASK = EAST and VX_MASK = WEST, the output header column will, by default, be written as a concatenation of the unique input strings: VX_MASK = EAST,WEST. But you can manually override that by setting -set_hdr VX_MASK CONUS. And then EAST,WEST will be replaced by CONUS.

Perhaps we could add a similar config file option for Point-Stat, Grid-Stat, Ensemble-Stat, Wavelet-Stat, MODE, and MTD to do what STAT-Analysis is already doing:
   set_hdr_column = [ "FCST_VAR" ];
   set_hdr_value = [ "APCP_24" ];

Data is messy and this would give us an option for cleaning it up. [MET-1020] created by johnhg

[JohnHalleyGotway]

On 5/17/2019, Tracy pointed out that this functionality would be useful for AF T&E projects.

[TaraJensen]

Related to

[JohnHalleyGotway]

This really is a way of overriding the metadata for gridded input files. And this can be useful for tools which compute .stat output files as well as those that don't, like pcp_combine or regrid_data_plane. Instead of defining how the output columns should be set, just define the metadata, as shown below. This mirrors the "attr" dictionary used for python embedding. And these settings can be parsed into the VarInfoBase class so they apply to all derived gridded data file types:

set_attrs = {
valid = "timestring";
init = "timestring";
lead = "timestring";
accum = "timestring";
name = "string";
long_name = "string";
level = "string";
units = "string";
grid = "path to gridded data file or string defining the grid";

is_u_wind = boolean;
is_v_wind = boolean;
is_grid_relative = boolean;
is_wind_speed = boolean;
is_wind_direction = boolean;
is_precip = boolean;
is_specific_humidity = boolean;
}

Only the settings the user would like to override need to be defined. The boolean flags at the end are needed to enable python embedding to verify vector winds, as requested by NRL via met-help:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93123

[JohnHalleyGotway]

Need to specifically test to make sure that these changes enable python-embedding to compute VL1L2 and VCNT output.

Will need to add logic somewhere to handle...
(1) Using U and V to rotate winds from grid to earth relative.
(2) Using U and V and compute vector output stats.

[JohnHalleyGotway]

On 5/29/2020, NRL wrote met-help with a question about setting the OBS_UNITS column for Point-Stat output: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95402

Adding the "set_attrs" functionality would enable them to do so since there are no unit strings prescribed for point observations.

[georgemccabe]

Add METplus issue to test that something like this works with these changes:

FCST_VAR1_OPTIONS = set_attrs={units='K'; valid=...;}

[JohnHalleyGotway]

In the METplus meeting on 6/25/20, @CPKalb mentioned the need to use this with pcp_combine. Add a unit test which runs pcp_combine with set_attrs.set_valid on the -derive option to report the output valid time as the middle of the averaging period rather than the end which is the default behavior.

[JohnHalleyGotway]

The changes in feature_1020_set_attrs were compiling and running fine on my Mac laptop. However, testing them on kiowa and dakota both revealed a runtime error:
ERROR : StringArray::operator const -> range check error!

I ran through the debugger and also added print statements to figure out that this occurs when parsing the "is_..." flags. The parse_set_attrs_flag() function was actually parsing "set_attrs.is_precipitation" and the problem appeared to originate from vx_config when trying to parse those 2 levels from higher and higher levels of context. I was not able to identify/fix the specific problem.

In order to make things easier for both the user and the parsing code, I decided to abandon the "set_attrs" dictionary entirely. Instead, just parse bareword entries, not contained in a dictionary:

set_attr_name = "string"; ->
set_attr_long_name = "string";
set_attr_level = "string";
set_attr_units = "string";

set_attr_valid = "timestring";
set_attr_init = "timestring";
set_attr_lead = "timestring";

set_attr_grid = "path to gridded data file or string defining the grid";

is_precipitation = boolean;
is_specific_humidity = boolean;
is_u_wind = boolean;
is_v_wind = boolean;
is_grid_relative = boolean;
is_wind_speed = boolean;
is_wind_direction = boolean;
is_prob = boolean;