Update develop-ref after dtcenter/met@d11e614 hotfix to develop #3090
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
* Per #2395, add new columns to VL1L2, VAL1L2, and VCNT line types for wind direction statistics. Work still in progress. * Per #2395, write the new VCNT columns to the output and document the additions to the VL1L2, VAL1L2, and VCNT columns. * Per #2395, add the definition of new statistics to Appendix G. * Per #2395, update file version history. * Per #2395, tweak warning message about zero wind vectors and update grid-stat and point-stat to log calls to the do_vl1l2() function. * Per #2395, refine the weights for wind direction stats, ignoring the undefined directions. * Update src/tools/core/stat_analysis/aggr_stat_line.cc * Update src/tools/core/stat_analysis/parse_stat_line.cc * Update src/tools/core/stat_analysis/aggr_stat_line.cc
… broken the logic of the update_truth.yml GHA workflow. Instead of submitting a PR to merge develop into develop-ref directly, use an intermediate update_truth_for_develop branch.
* Per #2280, update to support probability threshold strings like ==8, where 8 is the number of ensemble members, to create probability bins centered on the n/8 for n = 0 ... 8. * Per #2280, update docs about probability threshold settings. * Per #2280, use a loose tolerance when checking for consistent bin widths. * Per #2280, add a new unit test for grid_stat to demonstrate processing the output from gen_ens_prod. * Per #2280, when verifying NMEP probability forecasts, smooth the obs data first. * Per #2280, only request STAT output for the PCT line type to match unit_grid_stat.xml and minimize the new output files. * Per #2280, update config option docs. * Per #2280, update config option docs.
…qube_beta3_namespace
…ullptr Feature 2673 sonarqube beta4 nullptr
…eturn Feature 2673 sonarqube beta4 return
…qube_beta3_namespace
…ata/table_files/Makefile.in
* #3012 Initial release * #3012 Added search_3d_kdtree.cc * #3012 Initial release * #3012 Added check_llh_to_ecef and test_llh_to_ecef * #3012Added search_3d_kdtree_api * #3012 Reanmed point_lonlat to points_lonlat. Added points_XYZ & points_XYZ_km * #3012 Compare points_XYZ or points_latlon * #3012 Commented out latlonalt_to_xy & xy_to_latlonalt * #3012 Reanmed point_lonlat to points_lonlat. Added points_XYZ & points_XYZ_km. Added llh_to_ecef APIs * Per #3012, update the .gitignore settings to ignore the newly created test executables in the test_util/libcode/vx_grid directory. * Per #3012, changes to the vx_grid library to address SonarQube code smells in New Code and hopefully reduce the Overall number of code smells. * Per #3012, couple more minor SonarQube tweaks --------- Co-authored-by: Howard Soh <[email protected]> Co-authored-by: MET Tools Test Account <[email protected]>
* Source code changes for multiple radii/thresholds in multivariate mode * Slight cleanup * added multiple thresh/radii functionality for the mvmode superobjects only situation * mvmode log message improvements * Mods to the mode users guide for multiple convolution thresholds and radii in multivariate mode * Added v12.1 * Minor bugfix and tweak * Per #2709, updates to fix SonarQube code smells. * Per #2709, fix compilation problem I introduced with SonarQube fixes. * Per #2709, eliminate SonarQube bug that was flagged * Per #2709, fix typo in docs --------- Co-authored-by: MET Tools Test Account <[email protected]>
* Per #3032, add data type column to all of the output tables * Per #3032, remove the first row from each output table since its info is repeated from the table name. Additional changes for consistency and accuracy in column names. * Update docs/Users_Guide/gsi-tools.rst Co-authored-by: Julie Prestopnik <[email protected]> --------- Co-authored-by: Julie Prestopnik <[email protected]>
…' based on 1/9/2025 decision at the METplus-Analysis Tools project meeting. Also update MET to link to the METplus User's Guide.
…Integers (not Doubles).
* Per #3024, roll the version number from 12.0 to 12.1 prior to adding new columns to the existing GRAD line type. * Per #3024, return bad data for the square root of a negative number * Per #3024, update Grid-Stat to write 4 new columns to the GRAD line type. * Per #3024, tweak laplace MSE computation * Per #3024, add appendixC.rst equations for the new statistics * Per #3024, escape embedded underscores in equations only for latex formatting, not html * Per #3024, add reference to paper and summary of the new gradient stats * Per #3030, update docs by moving the gradient config entry from Grid-Stat to the common area and then referencing it in both Grid-Stat and Series-Analysis. * Per #3030, update Series-Analysis config file by adding the gradient dictionary and an entry for output_stats.gradient. Update the conf_info source code to parse them. Still need to update OTHER Series-Analysis config files and also update the logic in series_analysis.cc to compute GRAD statistics. * Per #3030, add descriptions of GRAD statistics for inclusion in the long_name attribute of the Series-Analysis output files. * Per #3030, add the GRADInfo::get_stat() member function to be called by Series-Analysis * Per #3030, fix parsing logic for the gradient dictionary. * Per #3030, add gradient entries to existing Series-Analysis config files * Per #3030, add GRADInfo::set_stat() member function * Per #3030, enhance Series-Analysis to aggregate the GRAD line type incrementally across multiple runs. However, this can only be done when requesting that 'ALL' GRAD columns be written. * Per #3030, reduce the volume of SonarQube code smells. * Updated syntax * Updated syntax * Updated syntax --------- Co-authored-by: MET Tools Test Account <[email protected]> Co-authored-by: j-opatz <[email protected]>
…ve to MET_BASE (<install_loc>/share/met) and other files that are only in the MET repo are found relative to MET_TEST_BASE (MET/internal/test_unit). Also remove MET_BUILD_BASE env var (#3053)
* Per #1019, initial setup for supporting -format uscrn. It compiles but I still need to make it work for the variety of USCRN inputs. * Per #1019, add NumArray constructor using a vector of doubles. * Per #1019, saving progress after handling monthly, daily, and hourly input files. Need to complete support for other format types and handle the unit strings * Per #1019, consistent spacing. * Per #1019, tweak log messages so that the file being read is logged before it's actually read so that an error in parsing the data will indicate which file caused it. * Per #1019, update DataLine and LineDataFile classes to support parsing .csv files. Get rid of the unneeded Offsets vector. Add AllowEmptyColumns option to the DataLine class so that multiple delimiters in a row will be treated as separate columns. Since the default delim is whitespace, it makes sense that you'd want to parse multiple delims in a group. But for .csv files, each comma indicates a new column. * Per #1019, update USCRN handler code to support all 7 input variants, including .csv files. This required updates to the DataLine and LineDataFile classes to parse the .csv data properly. Still need to enhance ascii2nc to write units * Per #1019, add units to the Observation class. * Per #1019, add units string to the SummaryObs and SummaryKey classes. * Per #1019, update USCRN format to write units. Consider defining units for all the other ascii file types as well. * Per #1019, add StringArray::all_empty() member function to check for a list of all empty strings. This is used in ascii2nc to determine if observation units and descriptions should be written. * Per #1019, update library code to handle the independent writing of point observation descriptions. Previously, if units were present then descriptions (usually empty ones) were added. Now, units and descriptions and handled independently. * Per #1019, add descriptions for all USCRN observations, pulled from the USCRN website. * Per #1019, need to subtract 1900 from the year and 1 from the month to make this work. Seems like we should ADD these numbers where needed rather than subtracting them everywhere else! * Per #1019, get rid of USCRNHandler::_readHeaderInfo() since it compilicated the logic for ignoring the first line from csv files. * Per #1019, rather than always skipping the first line of USCRN csv files, just skip any lines where the station ID begins with 'WBAN'. That'll handle files being concatenated together and is simpler logic. * Per #1019, add an ascii2nc unit test for USCRN point observations. * Per #1019, doc-only change to indicate the prefix/suffix of the input USCRN files are used to the determine the specific format. * Per #1019, update USCRN code to no longer specify a default _qcOffset value since it conflicts with the initialization. While the GHA compiler is fine with it, the SonarQube one is not. These changes should enble to SonarQube build to complete. * Per #1019, for SonarQube replace ALL instances of push_back() with emplace_back() which SonarQube prefers for efficiency. * Per #1019, more changes to address SonarQube code smells and reduce the overall number of them lower than what's in the develop branch. * Per #1019, second pass through to further reduce SonarQube findings. * Per #1019, revert back to protected members in file_handler.h * Per #1019, one last round of minor SonarQube code smell remediation.
* Per #3047, modify the intel oneapi settings for compiler variable * Per #3047, modifications for ATLAS and ECKIT * Per 3047, creating a new development environment file for MET for seneca using the necessary values for the Intel oneAPI compilers on seneca * Per #3047, uploading config file used to install MET on seneca with Intel oneAPI * Per #3047, updated development file to use appropriate compiler settings * Per #3047, removing paths to compilers * Per #3047, removing paths to compilers * Per #3047, added BUFRLIB_NAME and GRIB2CLIB_NAME * Per #3047, adding paths back in for now * Per #3047, removing paths to compilers * Per #3047, commenting out source of setvars.sh and adding paths to older Intel oneAPI compilers * Per #2047, updates for compilation however a pb2nc unit test still fails * Per #3047, updating file for the 2024.2.1.100 compilers * Per #3047, updating some ADDTL_DIRs --------- Co-authored-by: Julie Prestopnik <[email protected]>
* Per #3006, add new pair_stat tool as a full copy of the point_stat tool with all instances of point_stat renamed as pair_stat. * Per #3006, add pair_stat to the list of things for which no 'make test' command is run. * Per #3006, saving work in progress prior to seneca reboot * Per #3006, revert back to using FileType instead of GrdFileType. That change was not meaningful or warranted. * Per #3006, revert back to using FileType instead of GrdFileType. That change was not meaningful or warranted. * Per #3006, committing changes since the code is compiling. Added IODADataConfig_default file to store default settings for reading IODA data. * Per #3006, starting to tweak config options. Saving progress while it's successfully compiling * Per #3006, add fcst.pairs and obs.pairs config entries. * #3007 Added vx_ioda * #3007 Added vx_ioda * #3007 Added vx_ioda * #3007 Derived from IODADataConfInfo * #3007 Reduced the code smells (SonarQube findings) * #3007 Added station_value_base_t and point_pair_t * Initial release * #3007 Changed ack the location of nc_point_obs.set_nc_out_data * Changed station_value_base_t::clear() to station_value_base_t::clear_base() * Changed bAPI names * #3007 Reduced code smells * #3007 CLeanup * #3007 Cahnged API for IODADataConfInfo * #3007 Renamed ioda_file to ioda_reader * #3007 Corrected comment * #3007 Added -lvx_statistics again * #3007 Added get_nc_data(NcVar *, unixtime) * #3007 Cleanup * #3007 Added add_to_unixtime((unixtime) * #3007 Reduced the complexiity of read_time. Added read_time_as_number * #3007 Added read_time_as_number * #3007 Added add_to_unixtime(unixtime) * #3007 Cleanup * #3007 Set bad_data_int to qc_buf * #3007 Cleanup * Per #3006, define new GrdFileType::FileType_Pairs enumerated value to be used in the pair_stat tool. * Per #3006, update pair_stat to use the newly added GrdFileType::FileType_Pairs enumerated value. * #3007 Temporarily removed pair_stat * Per #3006, rerun bootstrap on seneca to incorporate the compilation of the vx_ioda library. * Per #3006, make docs build without warning * Per #3006, saving compiling state * Per #3006, use ConcatString instead of std::string for consistency. * Per #3006, work in progress * Unrelated to #3006, but fix typo in log message. * Per #3006, default_column_union was defined in 2 spots. Renaming one of them to avoid compilation conflict. * Per #3006, move StatHdrInfo out of aggr_stat_line.h/.cc and into vx_stat_out/stat_hdr_info.h/.cc. This make it available to both Stat-Analysis and the Pair-Stat tool to track the unique STAT headers elements read. * Per #3006, remove the unused land/topo/msg_type type config options from the pair_stat tool's configuration file and code that parses it. If needed, we can add it back in the future. * Per #3006, update VarInfoPairs::set_dict() to also call VarInfo::set_magic(). * Per #3006, since python_line.h lives in src/basic/vx_util, the vx_util library now also depends on the * Per #3006, saving off version that compiles before trying changes that may not. * #3007 Deleted commented out cpde * Changed data typo (float to double) * #3007 Resio;lved SonarQube finding * Per #3006, added logic to track ck unique header input columns like Stat-Analysis does. Committing the current state of this branch prior to merging in changes from the #3007 feature branch and completing development for the beta1 cycle. * Per #3006, fix indexing for vx_opt * Per #3006, error out for -format ioda and -format python * Per #3006, make -format ioda or -format python error out, but add unit tests demonstrating those errors. Also note this in the user's guide. * Per #3006, replace -outdir with -out and remove output_prefix config option. * Per #3006, remove output_prefix from Pair-Stat config files. * Per #3006, update unit_pair_stat.xml to use the -out option. * Per #3006, working version. However, the filtering by grid is not working all that well right now because it's based on G004. * Per #3006, expand the Pair-Stat example. * Per #3006, switch from using global 0.5 degree reference grid to 0.1 degree. This still isn't good enough though. Instead, we need to get rid of the reference grid altogether and keep track of the grid information separately for each mask. * Per #3006, fix for loop typo in 3 spots * Per #3006, remove one line from bad merge * Per #3006, SonarQube updates. * Per #3006, more SonarQube fixes * Per #3006, remove desctrutor as recommended by SonarQube --------- Co-authored-by: Howard Soh <[email protected]> Co-authored-by: MET Tools Test Account <[email protected]>
* #3056 Initial release * #3056 Added UGridConfig_lfric2 * #3056 Moved some APIs (for init_time) to nc_utils.cc from nc_cf_file.cc * #3056 Get ther InitTime from the forecast_reference_time variable or filename * #3056 Added an unittest for LFric dataset, point_stat_ugrid_lfric2_config * #3056 Moved back getting init_time and valid_time from ghe filename * #3056 Corrected version * #3056 Cleanup * #3056 Resolved SonarQUbe findings * #3056 Cleanup * Per #3056, rename 'ugrid_lfric2' as 'ugrid_lfric_pressure_levels' throughout to more clearly indicate that this configuration is for lfric data that's been interpolated in the vertical to pressure levels. * Per #3056, tweak wording of UGrid config file comments * Per #3056, reconfigure the UGrid LFRic pressure levels test to make more realistic comparisons at 2 pressure levels and don't write .stat and _mpr.txt because the data is duplicated. --------- Co-authored-by: Howard Soh <[email protected]> Co-authored-by: MET Tools Test Account <[email protected]>
* Unrelated to #3037, update the Python embedding Appendix to list gen_ens_prod. * Per #3037, add 12.1.0-beta1 release notes. * Per #3037, roll the copyright date from 2024 to 2025 * Per #3037, proposing this updated structure and content for the upgrade instructions * Per #3037, add upgrade instruction details. * Per #3037, tweak wording * Per #3037, updating format of intended bulleted list --------- Co-authored-by: Julie Prestopnik <[email protected]>
* Per #3066, add Data-IO section about file lists and reference it in all tools that use them. * Per #3060, fix typo * Update docs/Users_Guide/data_io.rst Co-authored-by: Dan Adriaansen <[email protected]> * Update docs/Users_Guide/data_io.rst Co-authored-by: Dan Adriaansen <[email protected]> * Per #3066, update usage statements in the docs and application code based on PR feedback. * Per #3066, missed a handful of additional updates. --------- Co-authored-by: Dan Adriaansen <[email protected]>
* Per #3054, fix PARUSR BUFRLIB error by solving the upstream reference to temporary memory returned by c_str(). Store a copy of the temporary variable name in a string rather than a pointer to temporary memory. Note that checked all other usage of c_str() in pb2nc.cc and found these 2 instances to be only problematic ones. All others are used as arguments to functions for which a copy is made. * Unrelated to #3054, but discovered while investigating the dtcenter/METplus#2875 discussion, the PairBase::calc_obs_summary() function loops over map entries and attempts to update the mapped 'summary_val' value. However, the current version only updates it in a copy and not what's actually in the map. This changes how we loop over the map to actually udpate its contents. Note that the only impact is fixing a log file to accurately report the 'summary_val'. So this is really a logging bug. * Per #3054, correct bugfix in PairBase::calc_obs_summary() in pair_base.cc --------- Co-authored-by: MET Tools Test Account <[email protected]>
…t_hira.xml from which they were missing and that was causing a runtime warning message from python/unit.py.'
* Per #3077, update yacc files to address the Intel warning: 'POSIX yacc reserves %type to nonterminals [-Wyacc]' * Per #3077, update the color parser to allow for empty inputs in the same way we allow for them in config.tab.yy. Also use the opportunity to refine the code based on SonarQube feedback - removing commented out code and declaring variables on dedicated lines. * Per #3077, update vx_config library to back out the insufficient fix for MET #2408 with a yystate patch in Makefile.am that was causing other intermittent failures. Update config.tab.yy to allow for empty inputs in the statement_list rule. * Per #3077, changes to config.tab.yy to address some SonarQube code smells of having each variable declared on its own line. * Per #3077, consistent formatting of error messages * Per #3077, consistent formatting of error messages * Per #3077, no real change, just whitespace. * Per #3077, fix a few more SonarQube code smells that are easily fixed and are low risk.
* Per #3087, update logic in VarInfoNcMet::set_magic(...) to actuall store the requested level string to allow for discriminating between multiple U/V vertical level matches. * Unrelated to #3087, delete unneeded 'int errno;' local variable from temp_file.cc that caused an unexpected copmilation error with GCC 9.4.0 on Ubuntu as described in the dtcenter/METplus#2897 discussion. * Per #3087, tweak logic to handle '*' and fix resolve regression test differences. * Per #3087, add regrid_data_plane and grid_stat unit test to demostrate creating vector pairs at multiple levels from NetCDF input files. * Per #3087, forgot to add the Grid-Stat config file needed for the new unit test. * Per #3075, refine name and logic the new tests. * Per #3087, update ConcatString class to simplify from a pointer to a string to just a string itself. This is based on SonarQube code smells, but the implementation is much simpler and easier to maintain. * Per #3087, drive down a few more SonarQube code smells. * Per #3087, back out the ConcatString changes to switch from enum to enum class and the use of explicit since those had huge and wide-ranging impacts. Touching that many files is not worth it to reduce SonarQube code smells. * Per #3087, modify the existing point2grid_pb2nc_big_input test in unit_point2grid.xml by switching from requesting the 'Z2' level to using '*', like all the other simliar point2grid tests. Note that I DID actually test to confirm that 'Z2' and '*' produce the same result. So specifying Z2 does NOT actually filter the obs data as you'd expect it would. With this change, the diff of the output from the test should go away for PR #3088. * Unrelated to #3087, but pointed out by @j-opatz, removing an outdated line from the Ensemble-Stat chapter of the MET User's Guide referencing the 'ens' dictionary which was removed at the same time Gen-Ens-Prod was created.
… which now depends on it.
JohnHalleyGotway
changed the title
JohnHalleyGotway
changed the title
JohnHalleyGotway
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Update order in
bin/unit_test.shsince #3088 makesunit_grid_stat.xmldepend onunit_regrid.xmland the unit test script needs to be updated for the Nightly Build to run. See develop hotfix d11e614.Created by @JohnHalleyGotway