From 72f246912b4e322772173f58ee3ee4564e1316ad Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Fri, 19 Jan 2024 15:52:31 +0000 Subject: [PATCH 01/21] #2772 Added quit_msg --- scripts/python/met/logger.py | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/scripts/python/met/logger.py b/scripts/python/met/logger.py index a7296124a6..418a39b0d1 100644 --- a/scripts/python/met/logger.py +++ b/scripts/python/met/logger.py @@ -33,7 +33,11 @@ def log_msg(msg): print(f'{logger.PROMPT} {msg}') @staticmethod - def quit(msg): - logger.error_msg([msg, "Quit..."]) - sys.exit(1) + def quit(msg, do_quit=True): + logger.quit_msg(msg) + if do_quit: + sys.exit(1) + @staticmethod + def quit_msg(msg): + logger.error_msg([msg, "Quit..."]) From 27fe6a66523fc04ab5a7f2b547914cf9b6bf70f5 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 19 Jan 2024 11:35:55 -0700 Subject: [PATCH 02/21] Update develop-ref after #2790 (#2793) * Store change of sonar.login to sonar.token. * Hotfix to develop, updating aclocal.m4 and config.h.in to what is created when running bootstrap inside the dtcenter/met-base:v3.1 Docker image. Updates autoconf 2.69 output to 2.71. * Adding -lnetcdf to configure_lib_args for NetCDF-CXX compilation * Added eckit and atlas loads and paths * Added atlas and eckit loads and paths * Changing -j to "-j 5" as the recommended value * Update ensemble-stat.rst Fixed incorrect path to obs error table * Updated values for GRIB2CLIB_NAME and BUFRLIB_NAME * Updated for proj, eckit, and atlas * Feature #2761 develop seneca (#2762) * Per #2761, define runtime python version for testing rather than using the default version which no longer exists in /usr/local * Per #2761, fix setting ci-skip-all * Per #2761, patching test_util.R to use the -C command line option for ncdiff. I did test running comp_dir.sh with this change and confirmed that it now runs to completion. * #2652 Added find_var_by_standard_name and separated common codes to find_xy_vars * #2757 Get the email list from the environment variable MET_CRON_EMAIL_LIST__MET (or MET_CRON_EMAIL_LIST_) * #2757 THe SonarQube token and URL are replaced by ujsing the environment variable SONAR_TOKEN_VALUE and SONAR_SERVER_URL * #2757 The SonarQube token and URL are replaced with the pre-defined strings, SONAR_TOKEN_VALUE and SONAR_SERVER_URL * Bugfix #2670 develop --enable-python (#2768) * Update install_met_env.acorn_py3.10 * Update install_met_env.wcoss2_py3.10 * Feature #2776 cleanup (#2777) * Per #2776, delete stale prob_pair_info.h/.cc, wwmca_utils.h/.cc, and rmw_analysis_utils.h/.cc files that were not being compiled by their Makefiles. * Per #2776, update Makefile to compile the test_read_rmw utility. * Per #2776, add Makefile for the vx_python3_utils test utility directory. * add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca * only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies * Per #2776, just whitespace and capitalization --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> * Bugfix #2782 develop MASSDEN (#2783) * Per #2782, port over fixes from the bugfix_2782_main_v11.1_MASSDEN branch over to the bugfix branch for develop. Will also add new GRIB2 filtering options in this branch. * Per #2782, add support for 4 new GRIB2 filtering options for GRIB2_aerosol_type, GRIB2_aerosol_interval_type, GRIB2_aerosol_size_lower, and GRIB2_aerosol_size_upper. These are useful in filtering the MASSDEN records in the RRFS smoke output files. * Per #2782, fix aerosol_size_lower() and aersol_size_upper() inline definitions. * Per #2782, add a unit test for GRIB2 table 4.48. * Per #2782, switch from strict equality to using the is_eq() function when checking the GRIB2_aerosol_size_lower and upper values since they're doubles and not ints. * Feature #2701 ismn (#2758) * Add support for ISMN soil moisture data * Per #2701, update ascii2nc.cc with the ismn file option. * Per #2701, update ascii2nc docs * Per #2701, store the depth for precip as 0. * Per #2701, parse the ISMN observation value from the correct column. * Per #2701, every time I run ascii2nc to I see a log message for 'DEBUG 1: Number of NDBC skipped files due to no lookup 0'. This is printed by the NdbcHandler destructor. Update the logic to only print a debug level 3 log message when the number of missing locations is greater than 0, 'DEBUG 3: Skipped 5 NDBC files whose locations are not defined in 'MET_BASE/table_files/ndbc_stations.xml'. Set the MET_NDBC_STATIONS environment variable to override this file.' * Per #2701, as instructed by @anewman89, store the average of the depth values rather than the maximum value. * Per #2701, add a test of processing the ISMN data through ascii2nc. * Per #2701, fix parsing of year and month to subtract 1900 and 1, respectively. * Per #2701, set use_var_id to true for ISMN inputs * Per #2701, even though we're mapping obs data to GRIB code names and units, we want to encode them with use_var_id = true and so we need to keep track of those var_id values. * Per #2701, fix typo in the name of the ascii2nc netcdf output file. --------- Co-authored-by: MET Tools Test Account * Feature 2753 comp script config (#2771) * set dynamic library file extension to .dylib if running on MacOS and .so otherwise * Added disabling of jasper documentation for compiliation on Hera * Updated * remove extra export of compiler env vars * include full path to log file so it is easier to file the log file to examine when a command fails * send cmake output to a log file * remove redundant semi-colon * use full path to log file so it is easier to examine on failure * use run_cmd to catch if rm command fails * Modifications for compilation on hera, gaea, and orion * Updating * fixed variable name * clean up if/else statements * set TIFF_LIBRARY_RELEASE argument to use full path to dynamic library file to prevent failure installing proj library * set LDFLAGS so that LDFLAGS value set in the user's environment will also be used * Updated based on gaea, orion, and hera installs * Updated * change extension of dynamic library files only if architecture is arm64 because older Macs still use .so * added netcdf library to args to prevent error installing NetCDF-CXX when PROJ has been installed in the same run of the script -- PATH is set in the COMPILE_PROJ if block that causes this flag from being added automatically * clean up how rpath and -L are added to LDFLAGS so that each entry is separate -- prevents errors installing on Mac arm64 because multiple rpath values aren't read using :. Also use MET_PROJLIB * Updated * removed -ltiff from MET libs * only add path to rpath and -L arguments if they are not already included in LDFLAGS * changed from using LIB_TIFF (full path to tiff lib file) to use TIFF_LIB_DIR (dir containing tiff lib file). Added TIFF_INCLUDE_DIR to proj compilation and -DJAS_ENABLE_DOC to jasper compliation taken from @jprestop branch * update comments * ensure all MET_* and MET_*LIB variables are added to the rpath for consistency * remove unnecessary if block and only export LDFLAGS at the end of setting locally * Updated * Added section for adding /lib64 and rearranged placement of ADDTL_DIR * Commenting out the running of the Jasper lib tests * Updating and/or removing files * Updating and/or removing files * Latest udpates which include the addition of the tiff library for proj * Remove commented out line. Co-authored-by: John Halley Gotway * Make indentation consistent. Co-authored-by: John Halley Gotway * Make indentation consistent. Co-authored-by: John Halley Gotway * Make indentation consistent. Co-authored-by: John Halley Gotway --------- Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: John Halley Gotway * Removing ${MAKE_ARGS} in some locations Removing ${MAKE_ARGS} from "make install" and "make test" for MET. Removing "met" prefix from met.configure.log because we really need config.log for any detail. It is confusing to have met.configure.log when that does not contain useful information. * Feature #2547 Read WRF output files natively (#2790) * Per #2547, add the file_type = NETCDF_WRF configuration option. * Per #2547, rename vx_data2d_nc_interp as vx_data2d_nc_wrf and vx_data2d_nccf as vx_data2d_nc_cf. * Per #2547, rename nccf files as nc_cf for consistency throughout * Per #2547, rename Pinterp classes as Wrf * only set -L and -I arguments for atlas and eckit if the appropriate environment variable is set -- this matches how it is handled for other external library dependencies * add rpath for atlas and eckit lib dirs so dynamic libraries can be found when running on seneca * ignore directory automatically created by CLion * rename pinterp variables and classes to Wrf * more rename and fix typos * per #2547, add support for reading WRF files by adding support for different X/Y/Z dimensions, read X/Y/Z dimension names when reading a field instead of storing them per file, track if data is staggered in X/Y/Z dimension (still need to add DataPlane function to destagger data), handle pressure field that also includes time dimension * per #2547, add DataPlane function to handle staggering of grid * per #2547, change file type returned from pinterp to wrf * per #2547, call function to stagger DataPlane if necessary * per #2547, keep store if variable is on pressure levels and only read pressure field if it is * ignore cmake directory * per #2547, support U_PL, U_ZL, U, and U where is an integer in is_u_wind and the same for is_v_wind * change regex to match zero or more instances of a number after U or V instead of 1 or more to include the fields named U and V * error and exit if var with unsupported subgrid dimension is processed * clean up logging * only destagger Z dimension for bottom_top_stag because other _stag Z dimensions don't need to be destaggered, clean up logic to read X/Y/Z dimension info * Per #2547, instead of checking dimensions of all vars in file, check dimensions for the var that is being read. Only read pressure var if the z dimension matches the var to read. Set class variable DimNames instead of local gDimNames so dump function will work as expected * per #2547, support reading variable that corresponds to dimension with different name as dimension and includes a time dimension, e.g. P_PL(Time, num_press_levels_stag) * per #2547, added unit tests to process WRF out data with plot_data_plane and grid_stat * merged develop and resolved conflicts * rename lib from pinterp to wrf * rename lib * fixed incorrect merge changes * fixed broken unit test format * Per #2547, restore the mention of NAK.poly to Appendix B. * Change model to WRF Co-authored-by: John Halley Gotway * change obtype to WRF Co-authored-by: John Halley Gotway * plot higher vertical level to get non-zero output Co-authored-by: John Halley Gotway * indentation Co-authored-by: John Halley Gotway --------- Co-authored-by: John Halley Gotway --------- Co-authored-by: MET Tools Test Account Co-authored-by: root Co-authored-by: Julie Prestopnik Co-authored-by: Christina Kalb Co-authored-by: John Halley Gotway Co-authored-by: Howard Soh Co-authored-by: Howard Soh Co-authored-by: George McCabe <23407799+georgemccabe@users.noreply.github.com> Co-authored-by: metplus-bot <97135045+metplus-bot@users.noreply.github.com> --- .gitignore | 2 + Make-include | 8 +- configure | 11 +- configure.ac | 6 +- data/config/ConfigConstants | 11 +- docs/Users_Guide/appendixA.rst | 2 +- docs/Users_Guide/config_options.rst | 1 + .../scripts/installation/compile_MET_all.sh | 261 +++++++------ .../installation/config/install_met_env.gaea | 37 +- .../installation/config/install_met_env.hera | 19 +- .../installation/config/install_met_env.jet | 15 +- .../installation/config/install_met_env.orion | 29 +- .../config/install_met_env_met_only.casper | 30 -- .../config/install_met_env_met_only.cheyenne | 30 -- .../config/install_met_env_met_only.frontera | 27 -- .../config/install_met_env_met_only.gaea | 26 -- .../config/install_met_env_met_only.hera | 34 -- .../config/install_met_env_met_only.jet | 36 -- .../config/install_met_env_met_only.orion | 28 -- .../installation/modulefiles/11.1.0_orion | 45 --- .../installation/modulefiles/12.0.0_gaea | 2 +- .../installation/modulefiles/12.0.0_orion | 19 + .../test_unit/config/GridStatConfig_WRF_pres | 288 +++++++++++++++ internal/test_unit/xml/unit_grid_stat.xml | 17 + .../test_unit/xml/unit_plot_data_plane.xml | 84 +++++ internal/test_util/basic/vx_util/Makefile.am | 4 +- internal/test_util/basic/vx_util/Makefile.in | 4 +- internal/test_util/libcode/Makefile.am | 2 +- internal/test_util/libcode/Makefile.in | 2 +- .../libcode/vx_data2d_factory/Makefile.am | 4 +- .../libcode/vx_data2d_factory/Makefile.in | 4 +- .../.gitignore | 0 .../Makefile.am | 2 +- .../Makefile.in | 8 +- .../test_read_nccf.cc | 2 +- .../test_util/libcode/vx_nc_util/Makefile.am | 4 +- .../test_util/libcode/vx_nc_util/Makefile.in | 4 +- .../test_util/libcode/vx_tc_util/Makefile.am | 12 +- .../test_util/libcode/vx_tc_util/Makefile.in | 12 +- .../tools/other/mode_time_domain/Makefile.am | 4 +- .../tools/other/mode_time_domain/Makefile.in | 4 +- src/basic/vx_config/config_constants.h | 1 + src/basic/vx_config/config_util.cc | 2 +- src/basic/vx_config/data_file_type.h | 20 +- src/basic/vx_config/grdfiletype_to_string.cc | 6 +- src/basic/vx_util/data_plane.cc | 87 ++++- src/basic/vx_util/data_plane.h | 3 +- src/libcode/Makefile.am | 4 +- src/libcode/Makefile.in | 4 +- src/libcode/vx_data2d/vx_data2d.h | 1 + .../vx_data2d_factory/data2d_factory.cc | 7 +- .../vx_data2d_factory/data2d_factory_utils.cc | 1 + .../vx_data2d_factory/is_netcdf_file.cc | 38 +- .../vx_data2d_factory/is_netcdf_file.h | 2 + .../vx_data2d_factory/var_info_factory.cc | 7 +- .../.gitignore | 0 src/libcode/vx_data2d_nc_cf/Makefile.am | 20 + .../Makefile.in | 133 +++---- .../data2d_nc_cf.cc} | 2 +- .../data2d_nc_cf.h} | 4 +- .../nc_cf_file.cc} | 2 +- .../nc_cf_file.h} | 6 +- .../var_info_nc_cf.cc} | 4 +- .../var_info_nc_cf.h} | 0 .../vx_data2d_nc_cf.h} | 6 +- src/libcode/vx_data2d_nc_pinterp/Makefile.am | 20 - .../.gitignore | 0 src/libcode/vx_data2d_nc_wrf/Makefile.am | 20 + .../Makefile.in | 165 +++++---- .../data2d_nc_wrf.cc} | 74 ++-- .../data2d_nc_wrf.h} | 26 +- .../get_wrf_grid.cc} | 10 +- .../get_wrf_grid.h} | 10 +- .../var_info_nc_wrf.cc} | 61 +-- .../var_info_nc_wrf.h} | 42 ++- .../vx_data2d_nc_wrf.h} | 14 +- .../wrf_file.cc} | 347 ++++++++++++------ .../wrf_file.h} | 23 +- src/libcode/vx_data2d_nccf/Makefile.am | 20 - src/libcode/vx_nc_util/nc_var_info.cc | 41 ++- src/libcode/vx_nc_util/nc_var_info.h | 6 + src/tools/core/ensemble_stat/Makefile.am | 4 +- src/tools/core/ensemble_stat/Makefile.in | 4 +- src/tools/core/grid_stat/Makefile.am | 4 +- src/tools/core/grid_stat/Makefile.in | 4 +- src/tools/core/mode/Makefile.am | 4 +- src/tools/core/mode/Makefile.in | 4 +- src/tools/core/mode_analysis/Makefile.am | 4 +- src/tools/core/mode_analysis/Makefile.in | 4 +- src/tools/core/pcp_combine/Makefile.am | 4 +- src/tools/core/pcp_combine/Makefile.in | 4 +- src/tools/core/point_stat/Makefile.am | 4 +- src/tools/core/point_stat/Makefile.in | 4 +- src/tools/core/series_analysis/Makefile.am | 4 +- src/tools/core/series_analysis/Makefile.in | 4 +- src/tools/core/stat_analysis/Makefile.am | 4 +- src/tools/core/stat_analysis/Makefile.in | 4 +- .../core/stat_analysis/aggr_stat_line.cc | 2 +- src/tools/core/wavelet_stat/Makefile.am | 4 +- src/tools/core/wavelet_stat/Makefile.in | 4 +- src/tools/dev_utils/Makefile.am | 4 +- src/tools/dev_utils/Makefile.in | 4 +- src/tools/other/ascii2nc/Makefile.am | 4 +- src/tools/other/ascii2nc/Makefile.in | 4 +- src/tools/other/gen_ens_prod/Makefile.am | 4 +- src/tools/other/gen_ens_prod/Makefile.in | 4 +- src/tools/other/gen_vx_mask/Makefile.am | 4 +- src/tools/other/gen_vx_mask/Makefile.in | 4 +- src/tools/other/grid_diag/Makefile.am | 4 +- src/tools/other/grid_diag/Makefile.in | 4 +- src/tools/other/gsi_tools/Makefile.am | 8 +- src/tools/other/gsi_tools/Makefile.in | 8 +- src/tools/other/ioda2nc/Makefile.am | 4 +- src/tools/other/ioda2nc/Makefile.in | 4 +- src/tools/other/lidar2nc/Makefile.am | 2 +- src/tools/other/lidar2nc/Makefile.in | 2 +- src/tools/other/madis2nc/Makefile.am | 4 +- src/tools/other/madis2nc/Makefile.in | 4 +- src/tools/other/mode_time_domain/Makefile.am | 4 +- src/tools/other/mode_time_domain/Makefile.in | 4 +- src/tools/other/modis_regrid/Makefile.am | 6 +- src/tools/other/modis_regrid/Makefile.in | 6 +- src/tools/other/pb2nc/Makefile.am | 4 +- src/tools/other/pb2nc/Makefile.in | 4 +- src/tools/other/plot_data_plane/Makefile.am | 4 +- src/tools/other/plot_data_plane/Makefile.in | 4 +- .../other/plot_data_plane/plot_data_plane.cc | 2 +- src/tools/other/plot_point_obs/Makefile.am | 4 +- src/tools/other/plot_point_obs/Makefile.in | 4 +- src/tools/other/point2grid/Makefile.am | 4 +- src/tools/other/point2grid/Makefile.in | 4 +- src/tools/other/regrid_data_plane/Makefile.am | 4 +- src/tools/other/regrid_data_plane/Makefile.in | 4 +- .../regrid_data_plane/regrid_data_plane.cc | 5 +- src/tools/other/shift_data_plane/Makefile.am | 4 +- src/tools/other/shift_data_plane/Makefile.in | 4 +- .../shift_data_plane/shift_data_plane.cc | 5 +- src/tools/other/wwmca_tool/Makefile.am | 8 +- src/tools/other/wwmca_tool/Makefile.in | 8 +- src/tools/tc_utils/rmw_analysis/Makefile.am | 4 +- src/tools/tc_utils/rmw_analysis/Makefile.in | 4 +- src/tools/tc_utils/tc_diag/Makefile.am | 4 +- src/tools/tc_utils/tc_diag/Makefile.in | 4 +- src/tools/tc_utils/tc_gen/Makefile.am | 4 +- src/tools/tc_utils/tc_gen/Makefile.in | 4 +- src/tools/tc_utils/tc_pairs/Makefile.am | 4 +- src/tools/tc_utils/tc_pairs/Makefile.in | 4 +- src/tools/tc_utils/tc_rmw/Makefile.am | 4 +- src/tools/tc_utils/tc_rmw/Makefile.in | 4 +- src/tools/tc_utils/tc_stat/Makefile.am | 4 +- src/tools/tc_utils/tc_stat/Makefile.in | 4 +- 151 files changed, 1596 insertions(+), 1077 deletions(-) delete mode 100644 internal/scripts/installation/config/install_met_env_met_only.casper delete mode 100644 internal/scripts/installation/config/install_met_env_met_only.cheyenne delete mode 100644 internal/scripts/installation/config/install_met_env_met_only.frontera delete mode 100644 internal/scripts/installation/config/install_met_env_met_only.gaea delete mode 100644 internal/scripts/installation/config/install_met_env_met_only.hera delete mode 100644 internal/scripts/installation/config/install_met_env_met_only.jet delete mode 100644 internal/scripts/installation/config/install_met_env_met_only.orion delete mode 100644 internal/scripts/installation/modulefiles/11.1.0_orion create mode 100644 internal/scripts/installation/modulefiles/12.0.0_orion create mode 100644 internal/test_unit/config/GridStatConfig_WRF_pres rename internal/test_util/libcode/{vx_data2d_nccf => vx_data2d_nc_cf}/.gitignore (100%) rename internal/test_util/libcode/{vx_data2d_nccf => vx_data2d_nc_cf}/Makefile.am (93%) rename internal/test_util/libcode/{vx_data2d_nccf => vx_data2d_nc_cf}/Makefile.in (99%) rename internal/test_util/libcode/{vx_data2d_nccf => vx_data2d_nc_cf}/test_read_nccf.cc (98%) rename src/libcode/{vx_data2d_nc_pinterp => vx_data2d_nc_cf}/.gitignore (100%) create mode 100644 src/libcode/vx_data2d_nc_cf/Makefile.am rename src/libcode/{vx_data2d_nccf => vx_data2d_nc_cf}/Makefile.in (74%) rename src/libcode/{vx_data2d_nccf/data2d_nccf.cc => vx_data2d_nc_cf/data2d_nc_cf.cc} (99%) rename src/libcode/{vx_data2d_nccf/data2d_nccf.h => vx_data2d_nc_cf/data2d_nc_cf.h} (98%) rename src/libcode/{vx_data2d_nccf/nccf_file.cc => vx_data2d_nc_cf/nc_cf_file.cc} (99%) rename src/libcode/{vx_data2d_nccf/nccf_file.h => vx_data2d_nc_cf/nc_cf_file.h} (98%) rename src/libcode/{vx_data2d_nccf/var_info_nccf.cc => vx_data2d_nc_cf/var_info_nc_cf.cc} (99%) rename src/libcode/{vx_data2d_nccf/var_info_nccf.h => vx_data2d_nc_cf/var_info_nc_cf.h} (100%) rename src/libcode/{vx_data2d_nccf/vx_data2d_nccf.h => vx_data2d_nc_cf/vx_data2d_nc_cf.h} (90%) delete mode 100644 src/libcode/vx_data2d_nc_pinterp/Makefile.am rename src/libcode/{vx_data2d_nccf => vx_data2d_nc_wrf}/.gitignore (100%) create mode 100644 src/libcode/vx_data2d_nc_wrf/Makefile.am rename src/libcode/{vx_data2d_nc_pinterp => vx_data2d_nc_wrf}/Makefile.in (68%) rename src/libcode/{vx_data2d_nc_pinterp/data2d_nc_pinterp.cc => vx_data2d_nc_wrf/data2d_nc_wrf.cc} (78%) rename src/libcode/{vx_data2d_nc_pinterp/data2d_nc_pinterp.h => vx_data2d_nc_wrf/data2d_nc_wrf.h} (72%) rename src/libcode/{vx_data2d_nc_pinterp/get_pinterp_grid.cc => vx_data2d_nc_wrf/get_wrf_grid.cc} (97%) rename src/libcode/{vx_data2d_nc_pinterp/get_pinterp_grid.h => vx_data2d_nc_wrf/get_wrf_grid.h} (77%) rename src/libcode/{vx_data2d_nc_pinterp/var_info_nc_pinterp.cc => vx_data2d_nc_wrf/var_info_nc_wrf.cc} (87%) rename src/libcode/{vx_data2d_nc_pinterp/var_info_nc_pinterp.h => vx_data2d_nc_wrf/var_info_nc_wrf.h} (84%) rename src/libcode/{vx_data2d_nc_pinterp/vx_data2d_nc_pinterp.h => vx_data2d_nc_wrf/vx_data2d_nc_wrf.h} (74%) rename src/libcode/{vx_data2d_nc_pinterp/pinterp_file.cc => vx_data2d_nc_wrf/wrf_file.cc} (63%) rename src/libcode/{vx_data2d_nc_pinterp/pinterp_file.h => vx_data2d_nc_wrf/wrf_file.h} (79%) delete mode 100644 src/libcode/vx_data2d_nccf/Makefile.am diff --git a/.gitignore b/.gitignore index 3f75b8c06f..d26e2e4a32 100644 --- a/.gitignore +++ b/.gitignore @@ -17,3 +17,5 @@ share Makefile make.log make_install.log +.idea +cmake-build-debug diff --git a/Make-include b/Make-include index bdde7b71a4..6b88767ffb 100644 --- a/Make-include +++ b/Make-include @@ -11,9 +11,9 @@ MET_CPPFLAGS = -I${top_builddir}/src/basic/vx_cal \ -I${top_builddir}/src/libcode/vx_data2d_factory \ -I${top_builddir}/src/libcode/vx_data2d_grib \ -I${top_builddir}/src/libcode/vx_data2d_grib2 \ - -I${top_builddir}/src/libcode/vx_data2d_nccf \ -I${top_builddir}/src/libcode/vx_data2d_nc_met \ - -I${top_builddir}/src/libcode/vx_data2d_nc_pinterp \ + -I${top_builddir}/src/libcode/vx_data2d_nc_wrf \ + -I${top_builddir}/src/libcode/vx_data2d_nc_cf \ -I${top_builddir}/src/libcode/vx_data2d_python \ -I${top_builddir}/src/libcode/vx_data2d_ugrid \ -I${top_builddir}/src/libcode/vx_python3_utils \ @@ -58,9 +58,9 @@ MET_LDFLAGS = -L${top_builddir}/src/basic/vx_cal \ -L${top_builddir}/src/libcode/vx_data2d_factory \ -L${top_builddir}/src/libcode/vx_data2d_grib \ -L${top_builddir}/src/libcode/vx_data2d_grib2 \ - -L${top_builddir}/src/libcode/vx_data2d_nccf \ -L${top_builddir}/src/libcode/vx_data2d_nc_met \ - -L${top_builddir}/src/libcode/vx_data2d_nc_pinterp \ + -L${top_builddir}/src/libcode/vx_data2d_nc_wrf \ + -L${top_builddir}/src/libcode/vx_data2d_nc_cf \ -L${top_builddir}/src/libcode/vx_data2d_python \ -L${top_builddir}/src/libcode/vx_data2d_ugrid \ -L${top_builddir}/src/libcode/vx_python3_utils \ diff --git a/configure b/configure index b7d9098c4f..888d197d7d 100755 --- a/configure +++ b/configure @@ -10133,11 +10133,12 @@ fi # Create configured files -ac_config_files="$ac_config_files Makefile scripts/Rscripts/Makefile scripts/Rscripts/include/Makefile scripts/python/Makefile scripts/python/examples/Makefile scripts/python/met/Makefile scripts/python/pyembed/Makefile scripts/python/utility/Makefile scripts/python/tc_diag/Makefile scripts/python/tc_diag/atcf_tools/Makefile scripts/python/tc_diag/config/Makefile scripts/python/tc_diag/diag_lib/Makefile scripts/python/tc_diag/tc_diag_driver/Makefile data/Makefile data/climo/Makefile data/climo/seeps/Makefile data/colortables/Makefile data/colortables/NCL_colortables/Makefile data/config/Makefile data/map/Makefile data/map/admin_by_country/Makefile data/poly/Makefile data/poly/HMT_masks/Makefile data/poly/NCEP_masks/Makefile data/ps/Makefile data/table_files/Makefile data/tc_data/Makefile src/Makefile src/basic/Makefile src/basic/enum_to_string/Makefile src/basic/vx_cal/Makefile src/basic/vx_config/Makefile src/basic/vx_log/Makefile src/basic/vx_math/Makefile src/basic/vx_util/Makefile src/basic/vx_util_math/Makefile src/libcode/Makefile src/libcode/vx_afm/Makefile src/libcode/vx_analysis_util/Makefile src/libcode/vx_color/Makefile src/libcode/vx_data2d/Makefile src/libcode/vx_data2d_factory/Makefile src/libcode/vx_data2d_grib/Makefile src/libcode/vx_data2d_grib2/Makefile src/libcode/vx_data2d_nc_met/Makefile src/libcode/vx_data2d_nc_pinterp/Makefile src/libcode/vx_data2d_nccf/Makefile src/libcode/vx_data2d_ugrid/Makefile src/libcode/vx_geodesy/Makefile src/libcode/vx_gis/Makefile src/libcode/vx_gnomon/Makefile src/libcode/vx_grid/Makefile src/libcode/vx_gsl_prob/Makefile src/libcode/vx_nav/Makefile src/libcode/vx_solar/Makefile src/libcode/vx_nc_obs/Makefile src/libcode/vx_nc_util/Makefile src/libcode/vx_pb_util/Makefile src/libcode/vx_plot_util/Makefile src/libcode/vx_ps/Makefile src/libcode/vx_pxm/Makefile src/libcode/vx_render/Makefile src/libcode/vx_shapedata/Makefile src/libcode/vx_stat_out/Makefile src/libcode/vx_statistics/Makefile src/libcode/vx_time_series/Makefile src/libcode/vx_physics/Makefile src/libcode/vx_series_data/Makefile src/libcode/vx_regrid/Makefile src/libcode/vx_tc_util/Makefile src/libcode/vx_summary/Makefile src/libcode/vx_python3_utils/Makefile src/libcode/vx_data2d_python/Makefile src/libcode/vx_bool_calc/Makefile src/libcode/vx_pointdata_python/Makefile src/libcode/vx_seeps/Makefile src/tools/Makefile src/tools/core/Makefile src/tools/core/ensemble_stat/Makefile src/tools/core/grid_stat/Makefile src/tools/core/mode/Makefile src/tools/core/mode_analysis/Makefile src/tools/core/pcp_combine/Makefile src/tools/core/point_stat/Makefile src/tools/core/series_analysis/Makefile src/tools/core/stat_analysis/Makefile src/tools/core/wavelet_stat/Makefile src/tools/other/Makefile src/tools/other/ascii2nc/Makefile src/tools/other/lidar2nc/Makefile src/tools/other/gen_ens_prod/Makefile src/tools/other/gen_vx_mask/Makefile src/tools/other/gis_utils/Makefile src/tools/other/ioda2nc/Makefile src/tools/other/madis2nc/Makefile src/tools/other/mode_graphics/Makefile src/tools/other/modis_regrid/Makefile src/tools/other/pb2nc/Makefile src/tools/other/plot_data_plane/Makefile src/tools/other/plot_point_obs/Makefile src/tools/other/wwmca_tool/Makefile src/tools/other/gsi_tools/Makefile src/tools/other/regrid_data_plane/Makefile src/tools/other/point2grid/Makefile src/tools/other/shift_data_plane/Makefile src/tools/other/mode_time_domain/Makefile src/tools/other/grid_diag/Makefile src/tools/tc_utils/Makefile src/tools/tc_utils/tc_dland/Makefile src/tools/tc_utils/tc_pairs/Makefile src/tools/tc_utils/tc_stat/Makefile src/tools/tc_utils/tc_gen/Makefile src/tools/tc_utils/rmw_analysis/Makefile src/tools/tc_utils/tc_rmw/Makefile src/tools/tc_utils/tc_diag/Makefile" +ac_config_files="$ac_config_files Makefile scripts/Rscripts/Makefile scripts/Rscripts/include/Makefile scripts/python/Makefile scripts/python/examples/Makefile scripts/python/met/Makefile scripts/python/pyembed/Makefile scripts/python/utility/Makefile scripts/python/tc_diag/Makefile scripts/python/tc_diag/atcf_tools/Makefile scripts/python/tc_diag/config/Makefile scripts/python/tc_diag/diag_lib/Makefile scripts/python/tc_diag/tc_diag_driver/Makefile data/Makefile data/climo/Makefile data/climo/seeps/Makefile data/colortables/Makefile data/colortables/NCL_colortables/Makefile data/config/Makefile data/map/Makefile data/map/admin_by_country/Makefile data/poly/Makefile data/poly/HMT_masks/Makefile data/poly/NCEP_masks/Makefile data/ps/Makefile data/table_files/Makefile data/tc_data/Makefile src/Makefile src/basic/Makefile src/basic/enum_to_string/Makefile src/basic/vx_cal/Makefile src/basic/vx_config/Makefile src/basic/vx_log/Makefile src/basic/vx_math/Makefile src/basic/vx_util/Makefile src/basic/vx_util_math/Makefile src/libcode/Makefile src/libcode/vx_afm/Makefile src/libcode/vx_analysis_util/Makefile src/libcode/vx_color/Makefile src/libcode/vx_data2d/Makefile src/libcode/vx_data2d_factory/Makefile src/libcode/vx_data2d_grib/Makefile src/libcode/vx_data2d_grib2/Makefile src/libcode/vx_data2d_nc_met/Makefile src/libcode/vx_data2d_nc_wrf/Makefile src/libcode/vx_data2d_nc_cf/Makefile src/libcode/vx_data2d_ugrid/Makefile src/libcode/vx_geodesy/Makefile src/libcode/vx_gis/Makefile src/libcode/vx_gnomon/Makefile src/libcode/vx_grid/Makefile src/libcode/vx_gsl_prob/Makefile src/libcode/vx_nav/Makefile src/libcode/vx_solar/Makefile src/libcode/vx_nc_obs/Makefile src/libcode/vx_nc_util/Makefile src/libcode/vx_pb_util/Makefile src/libcode/vx_plot_util/Makefile src/libcode/vx_ps/Makefile src/libcode/vx_pxm/Makefile src/libcode/vx_render/Makefile src/libcode/vx_shapedata/Makefile src/libcode/vx_stat_out/Makefile src/libcode/vx_statistics/Makefile src/libcode/vx_time_series/Makefile src/libcode/vx_physics/Makefile src/libcode/vx_series_data/Makefile src/libcode/vx_regrid/Makefile src/libcode/vx_tc_util/Makefile src/libcode/vx_summary/Makefile src/libcode/vx_python3_utils/Makefile src/libcode/vx_data2d_python/Makefile src/libcode/vx_bool_calc/Makefile src/libcode/vx_pointdata_python/Makefile src/libcode/vx_seeps/Makefile src/tools/Makefile src/tools/core/Makefile src/tools/core/ensemble_stat/Makefile src/tools/core/grid_stat/Makefile src/tools/core/mode/Makefile src/tools/core/mode_analysis/Makefile src/tools/core/pcp_combine/Makefile src/tools/core/point_stat/Makefile src/tools/core/series_analysis/Makefile src/tools/core/stat_analysis/Makefile src/tools/core/wavelet_stat/Makefile src/tools/other/Makefile src/tools/other/ascii2nc/Makefile src/tools/other/lidar2nc/Makefile src/tools/other/gen_ens_prod/Makefile src/tools/other/gen_vx_mask/Makefile src/tools/other/gis_utils/Makefile src/tools/other/ioda2nc/Makefile src/tools/other/madis2nc/Makefile src/tools/other/mode_graphics/Makefile src/tools/other/modis_regrid/Makefile src/tools/other/pb2nc/Makefile src/tools/other/plot_data_plane/Makefile src/tools/other/plot_point_obs/Makefile src/tools/other/wwmca_tool/Makefile src/tools/other/gsi_tools/Makefile src/tools/other/regrid_data_plane/Makefile src/tools/other/point2grid/Makefile src/tools/other/shift_data_plane/Makefile src/tools/other/mode_time_domain/Makefile src/tools/other/grid_diag/Makefile src/tools/tc_utils/Makefile src/tools/tc_utils/tc_dland/Makefile src/tools/tc_utils/tc_pairs/Makefile src/tools/tc_utils/tc_stat/Makefile src/tools/tc_utils/tc_gen/Makefile src/tools/tc_utils/rmw_analysis/Makefile src/tools/tc_utils/tc_rmw/Makefile src/tools/tc_utils/tc_diag/Makefile" if test -n "$MET_DEVELOPMENT"; then - ac_config_files="$ac_config_files src/tools/dev_utils/Makefile src/tools/dev_utils/shapefiles/Makefile internal/test_util/Makefile internal/test_util/basic/Makefile internal/test_util/basic/vx_config/Makefile internal/test_util/basic/vx_log/Makefile internal/test_util/basic/vx_util/Makefile internal/test_util/libcode/Makefile internal/test_util/libcode/vx_data2d/Makefile internal/test_util/libcode/vx_data2d_factory/Makefile internal/test_util/libcode/vx_data2d_grib/Makefile internal/test_util/libcode/vx_data2d_nc_met/Makefile internal/test_util/libcode/vx_data2d_nccf/Makefile internal/test_util/libcode/vx_geodesy/Makefile internal/test_util/libcode/vx_grid/Makefile internal/test_util/libcode/vx_plot_util/Makefile internal/test_util/libcode/vx_ps/Makefile internal/test_util/libcode/vx_tc_util/Makefile internal/test_util/libcode/vx_nc_util/Makefile internal/test_util/libcode/vx_python3_utils/Makefile internal/test_util/libcode/vx_physics/Makefile internal/test_util/libcode/vx_series_data/Makefile internal/test_util/libcode/vx_solar/Makefile internal/test_util/tools/Makefile internal/test_util/tools/other/Makefile internal/test_util/tools/other/mode_time_domain/Makefile" + ac_config_files="$ac_config_files src/tools/dev_utils/Makefile src/tools/dev_utils/shapefiles/Makefile internal/test_util/Makefile internal/test_util/basic/Makefile internal/test_util/basic/vx_config/Makefile internal/test_util/basic/vx_log/Makefile internal/test_util/basic/vx_util/Makefile internal/test_util/libcode/Makefile internal/test_util/libcode/vx_data2d/Makefile internal/test_util/libcode/vx_data2d_factory/Makefile internal/test_util/libcode/vx_data2d_grib/Makefile internal/test_util/libcode/vx_data2d_nc_met/Makefile internal/test_util/libcode/vx_data2d_nc_cf/Makefile internal/test_util/libcode/vx_geodesy/Makefile internal/test_util/libcode/vx_grid/Makefile internal/test_util/libcode/vx_plot_util/Makefile internal/test_util/libcode/vx_ps/Makefile internal/test_util/libcode/vx_tc_util/Makefile internal/test_util/libcode/vx_nc_util/Makefile internal/test_util/libcode/vx_python3_utils/Makefile internal/test_util/libcode/vx_physics/Makefile internal/test_util/libcode/vx_series_data/Makefile internal/test_util/libcode/vx_solar/Makefile internal/test_util/tools/Makefile internal/test_util/tools/other/Makefile internal/test_util/tools/other/mode_time_domain/Makefile" + fi @@ -11065,8 +11066,8 @@ do "src/libcode/vx_data2d_grib/Makefile") CONFIG_FILES="$CONFIG_FILES src/libcode/vx_data2d_grib/Makefile" ;; "src/libcode/vx_data2d_grib2/Makefile") CONFIG_FILES="$CONFIG_FILES src/libcode/vx_data2d_grib2/Makefile" ;; "src/libcode/vx_data2d_nc_met/Makefile") CONFIG_FILES="$CONFIG_FILES src/libcode/vx_data2d_nc_met/Makefile" ;; - "src/libcode/vx_data2d_nc_pinterp/Makefile") CONFIG_FILES="$CONFIG_FILES src/libcode/vx_data2d_nc_pinterp/Makefile" ;; - "src/libcode/vx_data2d_nccf/Makefile") CONFIG_FILES="$CONFIG_FILES src/libcode/vx_data2d_nccf/Makefile" ;; + "src/libcode/vx_data2d_nc_wrf/Makefile") CONFIG_FILES="$CONFIG_FILES src/libcode/vx_data2d_nc_wrf/Makefile" ;; + "src/libcode/vx_data2d_nc_cf/Makefile") CONFIG_FILES="$CONFIG_FILES src/libcode/vx_data2d_nc_cf/Makefile" ;; "src/libcode/vx_data2d_ugrid/Makefile") CONFIG_FILES="$CONFIG_FILES src/libcode/vx_data2d_ugrid/Makefile" ;; "src/libcode/vx_geodesy/Makefile") CONFIG_FILES="$CONFIG_FILES src/libcode/vx_geodesy/Makefile" ;; "src/libcode/vx_gis/Makefile") CONFIG_FILES="$CONFIG_FILES src/libcode/vx_gis/Makefile" ;; @@ -11147,7 +11148,7 @@ do "internal/test_util/libcode/vx_data2d_factory/Makefile") CONFIG_FILES="$CONFIG_FILES internal/test_util/libcode/vx_data2d_factory/Makefile" ;; "internal/test_util/libcode/vx_data2d_grib/Makefile") CONFIG_FILES="$CONFIG_FILES internal/test_util/libcode/vx_data2d_grib/Makefile" ;; "internal/test_util/libcode/vx_data2d_nc_met/Makefile") CONFIG_FILES="$CONFIG_FILES internal/test_util/libcode/vx_data2d_nc_met/Makefile" ;; - "internal/test_util/libcode/vx_data2d_nccf/Makefile") CONFIG_FILES="$CONFIG_FILES internal/test_util/libcode/vx_data2d_nccf/Makefile" ;; + "internal/test_util/libcode/vx_data2d_nc_cf/Makefile") CONFIG_FILES="$CONFIG_FILES internal/test_util/libcode/vx_data2d_nc_cf/Makefile" ;; "internal/test_util/libcode/vx_geodesy/Makefile") CONFIG_FILES="$CONFIG_FILES internal/test_util/libcode/vx_geodesy/Makefile" ;; "internal/test_util/libcode/vx_grid/Makefile") CONFIG_FILES="$CONFIG_FILES internal/test_util/libcode/vx_grid/Makefile" ;; "internal/test_util/libcode/vx_plot_util/Makefile") CONFIG_FILES="$CONFIG_FILES internal/test_util/libcode/vx_plot_util/Makefile" ;; diff --git a/configure.ac b/configure.ac index f7b2a73453..2f8159d58c 100644 --- a/configure.ac +++ b/configure.ac @@ -1321,8 +1321,8 @@ AC_CONFIG_FILES([Makefile src/libcode/vx_data2d_grib/Makefile src/libcode/vx_data2d_grib2/Makefile src/libcode/vx_data2d_nc_met/Makefile - src/libcode/vx_data2d_nc_pinterp/Makefile - src/libcode/vx_data2d_nccf/Makefile + src/libcode/vx_data2d_nc_wrf/Makefile + src/libcode/vx_data2d_nc_cf/Makefile src/libcode/vx_data2d_ugrid/Makefile src/libcode/vx_geodesy/Makefile src/libcode/vx_gis/Makefile @@ -1405,7 +1405,7 @@ if test -n "$MET_DEVELOPMENT"; then internal/test_util/libcode/vx_data2d_factory/Makefile internal/test_util/libcode/vx_data2d_grib/Makefile internal/test_util/libcode/vx_data2d_nc_met/Makefile - internal/test_util/libcode/vx_data2d_nccf/Makefile + internal/test_util/libcode/vx_data2d_nc_cf/Makefile internal/test_util/libcode/vx_geodesy/Makefile internal/test_util/libcode/vx_grid/Makefile internal/test_util/libcode/vx_plot_util/Makefile diff --git a/data/config/ConfigConstants b/data/config/ConfigConstants index b9cf21a67a..16954f5467 100644 --- a/data/config/ConfigConstants +++ b/data/config/ConfigConstants @@ -60,11 +60,12 @@ MB_to_PA(p) = HPA_to_PA(p); GRIB1 = 1; GRIB2 = 2; NETCDF_MET = 3; -NETCDF_PINT = 4; -NETCDF_NCCF = 5; -PYTHON_NUMPY = 6; -PYTHON_XARRAY = 7; -NETCDF_UGRID = 8; +NETCDF_WRF = 4; +NETCDF_PINT = 5; +NETCDF_NCCF = 6; +PYTHON_NUMPY = 7; +PYTHON_XARRAY = 8; +NETCDF_UGRID = 9; // Set Logic NONE = 1; diff --git a/docs/Users_Guide/appendixA.rst b/docs/Users_Guide/appendixA.rst index 7d6df54511..83b10c2c32 100644 --- a/docs/Users_Guide/appendixA.rst +++ b/docs/Users_Guide/appendixA.rst @@ -1684,7 +1684,7 @@ Q. Why is the grid upside down? but the data is packed upside down. Try using the "file_type" entry. The "file_type" entry specifies the - input file type (e.g. GRIB1, GRIB2, NETCDF_MET, NETCDF_PINT, NETCDF_NCCF) + input file type (e.g. GRIB1, GRIB2, NETCDF_MET, NETCDF_WRF, NETCDF_PINT, NETCDF_NCCF) rather than letting the code determine it itself. For valid file_type values, see "File types" in the *data/config/ConfigConstants* file. This entry should be defined within the "fcst" or "obs" dictionaries. diff --git a/docs/Users_Guide/config_options.rst b/docs/Users_Guide/config_options.rst index 9ff10893df..f2dc2803ba 100644 --- a/docs/Users_Guide/config_options.rst +++ b/docs/Users_Guide/config_options.rst @@ -950,6 +950,7 @@ to be verified. This dictionary may include the following entries: file_type = GRIB1; GRIB version 1 file_type = GRIB2; GRIB version 2 file_type = NETCDF_MET; NetCDF created by another MET tool + file_type = NETCDF_WRF; NetCDF WRF output. file_type = NETCDF_PINT; NetCDF created by running the p_interp or wrf_interp utility on WRF output. May be used to read unstaggered raw WRF diff --git a/internal/scripts/installation/compile_MET_all.sh b/internal/scripts/installation/compile_MET_all.sh index f06b6c5c6c..98d9fca899 100755 --- a/internal/scripts/installation/compile_MET_all.sh +++ b/internal/scripts/installation/compile_MET_all.sh @@ -50,8 +50,8 @@ # Please supply values for the following environment variables # in the input environment configuration file (install_met_env.: # MET_GRIB2CLIB, MET_GRIB2CINC, GRIB2CLIB_NAME, MET_BUFRLIB, BUFRLIB_NAME, -# MET_HDF5, MET_NETCDF, MET_PROJ, MET_GSL, LIB_JASPER, LIB_PNG, LIB_Z, -# SQLITE_INCLUDE_DIR, SQLITE_LIB_DIR. +# MET_HDF5, MET_NETCDF, MET_PROJ, MET_GSL, LIB_JASPER, LIB_LIBPNG, LIB_Z, +# SQLITE_INCLUDE_DIR, SQLITE_LIB_DIR, TIFF_INCLUDE_DIR, TIFF_LIB_DIR. # # The optional libraries ecKit and atlas offer support for unstructured # grids. The optional libraries HDF4, HDFEOS, FREETYPE, and CAIRO are @@ -157,6 +157,13 @@ if [[ -z "$LIB_Z" ]]; then LIB_Z=${LIB_DIR}/lib fi +# if TIFF is not defined in the environment file, enable its compilation +if [[ -z ${TIFF_INCLUDE_DIR} ]] && [[ -z ${TIFF_LIB_DIR} ]]; then + COMPILE_TIFF=1 +else + COMPILE_TIFF=0 +fi + # if SQLITE is not defined in the environment file, enable its compilation if [[ -z ${SQLITE_INCLUDE_DIR} ]] && [[ -z ${SQLITE_LIB_DIR} ]]; then COMPILE_SQLITE=1 @@ -177,29 +184,13 @@ else COMPILE_G2CLIB=0 fi -if [ -z ${MET_BUFRLIB} ]; then - COMPILE_BUFRLIB=1 -else - COMPILE_BUFRLIB=0 -fi +if [ -z ${MET_BUFRLIB} ]; then COMPILE_BUFRLIB=1; else COMPILE_BUFRLIB=0; fi -if [ -z ${MET_NETCDF} ]; then - COMPILE_NETCDF=1 -else - COMPILE_NETCDF=0 -fi +if [ -z ${MET_NETCDF} ]; then COMPILE_NETCDF=1; else COMPILE_NETCDF=0; fi -if [ -z ${MET_PROJ} ]; then - COMPILE_PROJ=1 -else - COMPILE_PROJ=0 -fi +if [ -z ${MET_PROJ} ]; then COMPILE_PROJ=1; else COMPILE_PROJ=0; fi -if [ -z ${MET_GSL} ]; then - COMPILE_GSL=1 -else - COMPILE_GSL=0 -fi +if [ -z ${MET_GSL} ]; then COMPILE_GSL=1; else COMPILE_GSL=0; fi # Only set COMPILE_ECKIT and COMPILE_ATLAS if you want to compile and enable support for unstructued grids if [ ! -z "${COMPILE_ECKIT}" ]; then COMPILE_ECKIT=1; else COMPILE_ECKIT=0; fi @@ -388,12 +379,6 @@ else exit fi -#export CC -#export CXX -#export FC -#export F77 -#export F90 - echo "export CC=${CC}" echo "export CXX=${CXX}" echo "export FC=${FC}" @@ -411,12 +396,19 @@ case "${unameOut}" in *) machine="UNKNOWN:${unameOut}" esac +# change sed command and extension for dynamic library files if [[ $machine == "Mac" ]]; then sed_inline="sed -i ''" else sed_inline="sed -i''" fi +if [[ "$(uname -m)" == "arm64" ]]; then + dynamic_lib_ext="dylib" +else + dynamic_lib_ext="so" +fi + # Load Python module if [ ${USE_MODULES} = "TRUE" ]; then @@ -442,6 +434,22 @@ fi # Compile Proj if [ $COMPILE_PROJ -eq 1 ]; then + + if [ $COMPILE_TIFF -eq 1 ]; then + echo + echo "Compiling TIFF at `date`" + mkdir -p ${LIB_DIR}/tiff + rm -rf ${LIB_DIR}/tiff/tiff* + tar -xzf ${TAR_DIR}/tiff*.tar.gz -C ${LIB_DIR}/tiff + cd ${LIB_DIR}/tiff/tiff* + echo "cd `pwd`" + run_cmd "./configure --prefix=${LIB_DIR} > $(pwd)/tiff.configure.log 2>&1" + run_cmd "make ${MAKE_ARGS} > $(pwd)/tiff.make.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/tiff.make_install.log 2>&1" + export TIFF_INCLUDE_DIR=${LIB_DIR}/include + export TIFF_LIB_DIR=${LIB_DIR}/lib + fi + if [ $COMPILE_SQLITE -eq 1 ]; then echo echo "Compiling SQLITE at `date`" @@ -450,17 +458,18 @@ if [ $COMPILE_PROJ -eq 1 ]; then tar -xf ${TAR_DIR}/sqlite*.tar.gz -C ${LIB_DIR}/sqlite > /dev/null 2>&1 cd ${LIB_DIR}/sqlite/sqlite* echo "cd `pwd`" - run_cmd "./configure --enable-shared --prefix=${LIB_DIR} > sqlite.configure.log 2>&1" - run_cmd "make ${MAKE_ARGS} > sqlite.make.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > sqlite.make_install.log 2>&1" + run_cmd "./configure --enable-shared --prefix=${LIB_DIR} > $(pwd)/sqlite.configure.log 2>&1" + run_cmd "make ${MAKE_ARGS} > $(pwd)/sqlite.make.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/sqlite.make_install.log 2>&1" export SQLITE_INCLUDE_DIR=${LIB_DIR}/include export SQLITE_LIB_DIR=${LIB_DIR}/lib fi - vrs="7.1.0"; + vrs="7.1.0" echo echo "Compiling PROJ_${vrs} at `date`" + echo "cmake version `cmake --version`" mkdir -p ${LIB_DIR}/proj rm -rf ${LIB_DIR}/proj/proj* tar -xf ${TAR_DIR}/proj-${vrs}.tar.gz -C ${LIB_DIR}/proj @@ -468,13 +477,20 @@ if [ $COMPILE_PROJ -eq 1 ]; then echo "cd `pwd`" export PATH=${LIB_DIR}/bin:${PATH} run_cmd "mkdir build; cd build" - if [[ -z "$LIB_TIFF" ]]; then - run_cmd "cmake -DCMAKE_INSTALL_PREFIX=${LIB_DIR} -DSQLITE3_INCLUDE_DIR=${SQLITE_INCLUDE_DIR} -DSQLITE3_LIBRARY=${SQLITE_LIB_DIR}/libsqlite3.so .." - else - run_cmd "cmake -DCMAKE_INSTALL_PREFIX=${LIB_DIR} -DSQLITE3_INCLUDE_DIR=${SQLITE_INCLUDE_DIR} -DSQLITE3_LIBRARY=${SQLITE_LIB_DIR}/libsqlite3.so -DTIFF_LIBRARY_RELEASE=${LIB_TIFF} .." + + tiff_arg="" + # add tiff library and include arguments if necessary + if [[ ! -z "$TIFF_LIB_DIR" ]]; then + tiff_arg+="-DTIFF_LIBRARY_RELEASE=${TIFF_LIB_DIR}/libtiff.${dynamic_lib_ext}" + fi + if [[ ! -z "$TIFF_INCLUDE_DIR" ]]; then + tiff_arg+=" -DTIFF_INCLUDE_DIR=${TIFF_INCLUDE_DIR}" fi - run_cmd "cmake --build ." - run_cmd "cmake --build . --target install" + + cmd="cmake -DCMAKE_INSTALL_PREFIX=${LIB_DIR} -DSQLITE3_INCLUDE_DIR=${SQLITE_INCLUDE_DIR} -DSQLITE3_LIBRARY=${SQLITE_LIB_DIR}/libsqlite3.${dynamic_lib_ext} ${tiff_arg} .. > $(pwd)/proj.cmake.log 2>&1" + run_cmd ${cmd} + run_cmd "cmake --build . > $(pwd)/proj.cmake_build.log 2>&1" + run_cmd "cmake --build . --target install > $(pwd)/proj.cmake_install.log 2>&1" fi @@ -482,9 +498,9 @@ fi if [ $COMPILE_GSL -eq 1 ]; then if [ ${COMPILER_FAMILY} = "pgi" ]; then - vrs="1.11"; + vrs="1.11" else - vrs="2.7.1"; + vrs="2.7.1" fi echo @@ -494,15 +510,15 @@ if [ $COMPILE_GSL -eq 1 ]; then tar -xf ${TAR_DIR}/gsl-${vrs}.tar.gz -C ${LIB_DIR}/gsl cd ${LIB_DIR}/gsl/gsl* echo "cd `pwd`" - run_cmd "./configure --prefix=${LIB_DIR} > gsl.configure.log 2>&1" - run_cmd "make ${MAKE_ARGS} > gsl.make.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > gsl.make_install.log 2>&1" + run_cmd "./configure --prefix=${LIB_DIR} > $(pwd)/gsl.configure.log 2>&1" + run_cmd "make ${MAKE_ARGS} > $(pwd)/gsl.make.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/gsl.make_install.log 2>&1" fi # Compile BUFRLIB if [ $COMPILE_BUFRLIB -eq 1 ]; then - vrs="v11.6.0"; + vrs="v11.6.0" echo echo "Compiling bufr_${vrs} at `date`" @@ -514,11 +530,11 @@ if [ $COMPILE_BUFRLIB -eq 1 ]; then echo "cd `pwd`" run_cmd "mkdir build" export BUILD_DIR=${SOURCE_DIR}/build - run_cmd "cmake -H${SOURCE_DIR} -B${BUILD_DIR} -DCMAKE_INSTALL_PREFIX=${LIB_DIR} -DCMAKE_BUILD_TYPE=Debug" + run_cmd "cmake -H${SOURCE_DIR} -B${BUILD_DIR} -DCMAKE_INSTALL_PREFIX=${LIB_DIR} -DCMAKE_BUILD_TYPE=Debug > $(pwd)/bufr.cmake.log 2>&1" run_cmd "cd ${BUILD_DIR}" - run_cmd "make ${MAKE_ARGS} > bufr.make.log 2>&1" - run_cmd "ctest > bufr.ctest.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > bufr.make_install.log 2>&1" + run_cmd "make ${MAKE_ARGS} > $(pwd)/bufr.make.log 2>&1" + run_cmd "ctest > $(pwd)/bufr.ctest.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/bufr.make_install.log 2>&1" fi @@ -531,14 +547,13 @@ if [ $COMPILE_ZLIB -eq 1 ]; then tar -xzf ${TAR_DIR}/zlib*.tar.gz -C ${LIB_DIR}/zlib cd ${LIB_DIR}/zlib/zlib* echo "cd `pwd`" - run_cmd "./configure --prefix=${LIB_DIR} > zlib.configure.log 2>&1" - run_cmd "make ${MAKE_ARGS} > zlib.make.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > zlib.make_install.log 2>&1" + run_cmd "./configure --prefix=${LIB_DIR} > $(pwd)/zlib.configure.log 2>&1" + run_cmd "make ${MAKE_ARGS} > $(pwd)/zlib.make.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/zlib.make_install.log 2>&1" # GPM: why is this removed? Could we add a comment to # describe why this is needed? - echo "rm ${LIB_DIR}/lib/zlib.a" - rm ${LIB_DIR}/lib/libz.a + run_cmd "rm ${LIB_DIR}/lib/libz.a" fi # Compile LIBPNG @@ -550,15 +565,15 @@ if [[ $COMPILE_LIBPNG -eq 1 && $HOST != ys* ]]; then tar -xzf ${TAR_DIR}/libpng*.tar.gz -C ${LIB_DIR}/libpng cd ${LIB_DIR}/libpng/libpng* echo "cd `pwd`" - run_cmd "./configure --prefix=${LIB_DIR} LDFLAGS=-L${LIB_DIR}/lib CPPFLAGS=-I${LIB_DIR}/include > libpng.configure.log 2>&1" - run_cmd "make ${MAKE_ARGS} > libpng.make.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > libpng.make_install.log 2>&1" + run_cmd "./configure --prefix=${LIB_DIR} LDFLAGS=-L${LIB_DIR}/lib CPPFLAGS=-I${LIB_DIR}/include > $(pwd)/libpng.configure.log 2>&1" + run_cmd "make ${MAKE_ARGS} > $(pwd)/libpng.make.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/libpng.make_install.log 2>&1" fi # Compile JASPER if [ $COMPILE_JASPER -eq 1 ]; then - vrs="2.0.25"; + vrs="2.0.25" echo echo "Compiling JASPER at `date`" @@ -570,17 +585,19 @@ if [ $COMPILE_JASPER -eq 1 ]; then export SOURCE_DIR=${LIB_DIR}/jasper/jasper-version-${vrs} echo "cd `pwd`" export BUILD_DIR=${LIB_DIR}/jasper/jasper-version-${vrs}/build - run_cmd "cmake -G \"Unix Makefiles\" -H${SOURCE_DIR} -B${BUILD_DIR} -DCMAKE_INSTALL_PREFIX=${LIB_DIR}" + run_cmd "cmake -G \"Unix Makefiles\" -H${SOURCE_DIR} -B${BUILD_DIR} -DCMAKE_INSTALL_PREFIX=${LIB_DIR} -DJAS_ENABLE_DOC=false > $(pwd)/jasper.cmake.log 2>&1" run_cmd "cd ${BUILD_DIR}" - run_cmd "make clean all" - run_cmd "make ${MAKE_ARGS} test > jasper.make_test.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > jasper.make_install.log 2>&1" + run_cmd "make clean all > $(pwd)/jasper.make.log 2>&1" + # Commented out due to “which: no opj2_compress in …” error, which causes one of four tests to fail + # This is a known problem, so skipping tests for now: https://github.com/AAROC/CODE-RADE/issues/36#issuecomment-359744351 + #run_cmd "make ${MAKE_ARGS} test > $(pwd)/jasper.make_test.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/jasper.make_install.log 2>&1" fi # Compile G2CLIB if [ $COMPILE_G2CLIB -eq 1 ]; then - vrs="1.6.4"; + vrs="1.6.4" echo echo "Compiling G2CLIB at `date`" @@ -590,10 +607,10 @@ if [ $COMPILE_G2CLIB -eq 1 ]; then cd ${LIB_DIR}/g2clib/NCEP* echo "cd `pwd`" run_cmd "mkdir build; cd build" - run_cmd "cmake -DCMAKE_INSTALL_PREFIX=${LIB_DIR} -DCMAKE_PREFIX_PATH=${LIB_DIR} .." - run_cmd "make ${MAKE_ARGS} > g2c.make.log 2>&1" - run_cmd "make ${MAKE_ARGS} test > g2c.make_test.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > g2c.make_install.log 2>&1" + run_cmd "cmake -DCMAKE_INSTALL_PREFIX=${LIB_DIR} -DCMAKE_PREFIX_PATH=${LIB_DIR} .. > $(pwd)/g2c.cmake.log 2>&1" + run_cmd "make ${MAKE_ARGS} > $(pwd)/g2c.make.log 2>&1" + run_cmd "make ${MAKE_ARGS} test > $(pwd)/g2c.make_test.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/g2c.make_install.log 2>&1" fi # Compile ECKIT @@ -601,7 +618,7 @@ if [ $COMPILE_ECKIT -eq 1 ]; then # Need to obtain ecbuild before installing eckit - vrs="3.5.0"; + vrs="3.5.0" echo echo "Compiling ECBUILD at `date`" @@ -611,10 +628,10 @@ if [ $COMPILE_ECKIT -eq 1 ]; then cd ${LIB_DIR}/ecbuild/ecbuild* echo "cd `pwd`" run_cmd "mkdir build; cd build" - run_cmd "cmake ../ -DCMAKE_INSTALL_PREFIX=${LIB_DIR}" - run_cmd "make ${MAKE_ARGS} install > ecbuild.make_install.log 2>&1" + run_cmd "cmake ../ -DCMAKE_INSTALL_PREFIX=${LIB_DIR} > $(pwd)/ecbuild.cmake.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/ecbuild.make_install.log 2>&1" - vrs="1.20.2"; + vrs="1.20.2" echo echo "Compiling ECKIT at `date`" @@ -624,15 +641,15 @@ if [ $COMPILE_ECKIT -eq 1 ]; then cd ${LIB_DIR}/eckit/eckit* echo "cd `pwd`" run_cmd "mkdir build; cd build" - run_cmd "cmake ../ -DCMAKE_INSTALL_PREFIX=${LIB_DIR} -DCMAKE_PREFIX_PATH=${LIB_DIR}" - run_cmd "make ${MAKE_ARGS} install > eckit.make_install.log 2>&1" + run_cmd "cmake ../ -DCMAKE_INSTALL_PREFIX=${LIB_DIR} -DCMAKE_PREFIX_PATH=${LIB_DIR} > $(pwd)/eckit.cmake.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/eckit.make_install.log 2>&1" fi # Compile ATLAS if [ $COMPILE_ATLAS -eq 1 ]; then - vrs="0.30.0"; + vrs="0.30.0" echo echo "Compiling ATLAS at `date`" @@ -642,9 +659,9 @@ if [ $COMPILE_ATLAS -eq 1 ]; then cd ${LIB_DIR}/atlas/atlas* echo "cd `pwd`" run_cmd "mkdir build; cd build" - run_cmd "cmake ../ -DCMAKE_INSTALL_PREFIX=${LIB_DIR} -DCMAKE_PREFIX_PATH=${LIB_DIR}" - run_cmd "make ${MAKE_ARGS} > atlas.make.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > atlas.make_install.log 2>&1" + run_cmd "cmake ../ -DCMAKE_INSTALL_PREFIX=${LIB_DIR} -DCMAKE_PREFIX_PATH=${LIB_DIR} > $(pwd)/atlas.cmake.log 2>&1" + run_cmd "make ${MAKE_ARGS} > $(pwd)/atlas.make.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/atlas.make_install.log 2>&1" fi @@ -661,7 +678,7 @@ if [ $COMPILE_HDF -eq 1 ]; then tar -xf ${TAR_DIR}/HDF4.2*.tar.gz -C ${LIB_DIR}/hdf cd ${LIB_DIR}/hdf/HDF* echo "cd `pwd`" - run_cmd "./configure --prefix=${LIB_DIR} --disable-netcdf --with-jpeg=${LIB_DIR} --with-zlib=${LIB_DIR} CPPFLAGS=-I/usr/include/tirpc LIBS='-lm -ltirpc' > hdf4.configure.log 2>&1" + run_cmd "./configure --prefix=${LIB_DIR} --disable-netcdf --with-jpeg=${LIB_DIR} --with-zlib=${LIB_DIR} CPPFLAGS=-I/usr/include/tirpc LIBS='-lm -ltirpc' > $(pwd)/hdf4.configure.log 2>&1" if [[ ${COMPILER_MAJOR_VERSION} -ge 10 ]]; then cat hdf/src/Makefile | \ sed 's/FFLAGS = -O2/FFLAGS = -w -fallow-argument-mismatch -O2/g' \ @@ -672,8 +689,8 @@ if [ $COMPILE_HDF -eq 1 ]; then > Makefile_new fi mv Makefile_new hdf/src/Makefile - run_cmd "make ${MAKE_ARGS} > hdf4.make.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > hdf4.make_install.log 2>&1" + run_cmd "make ${MAKE_ARGS} > $(pwd)/hdf4.make.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/hdf4.make_install.log 2>&1" fi # Compile HDFEOS @@ -686,9 +703,9 @@ if [ $COMPILE_HDFEOS -eq 1 ]; then tar -xzf ${TAR_DIR}/HDF-EOS*.tar.* -C ${LIB_DIR}/hdfeos cd ${LIB_DIR}/hdfeos/hdfeos echo "cd `pwd`" - run_cmd "./configure --prefix=${LIB_DIR} --with-hdf4=${LIB_DIR} --with-jpeg=${LIB_DIR} > hdf-eos.configure.log 2>&1" - run_cmd "make ${MAKE_ARGS} > hed-eos.make.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > hsf-eos.make_install.log 2>&1" + run_cmd "./configure --prefix=${LIB_DIR} --with-hdf4=${LIB_DIR} --with-jpeg=${LIB_DIR} > $(pwd)/hdf-eos.configure.log 2>&1" + run_cmd "make ${MAKE_ARGS} > $(pwd)/hed-eos.make.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/hsf-eos.make_install.log 2>&1" cp include/*.h ${LIB_DIR}/include/ fi @@ -703,8 +720,8 @@ if [ $COMPILE_NETCDF -eq 1 ]; then tar -xzf ${TAR_DIR}/hdf5*.tar.gz -C ${LIB_DIR}/hdf5 cd ${LIB_DIR}/hdf5/hdf5* echo "cd `pwd`" - run_cmd "./configure --prefix=${LIB_DIR} --with-zlib=${LIB_Z} CFLAGS=-fPIC CXXFLAGS=-fPIC FFLAGS=-fPIC LDFLAGS=-L${LIB_DIR}/lib:${LIB_Z} CPPFLAGS=-I${LIB_DIR}/include > hdf5.configure.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > hdf5.make_install.log 2>&1" + run_cmd "./configure --prefix=${LIB_DIR} --with-zlib=${LIB_Z} CFLAGS=-fPIC CXXFLAGS=-fPIC FFLAGS=-fPIC LDFLAGS=-L${LIB_DIR}/lib:${LIB_Z} CPPFLAGS=-I${LIB_DIR}/include > $(pwd)/hdf5.configure.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/hdf5.make_install.log 2>&1" echo echo "Compiling NetCDF-C at `date`" @@ -715,8 +732,8 @@ if [ $COMPILE_NETCDF -eq 1 ]; then export FC='' export F90='' echo "cd `pwd`" - run_cmd "./configure --prefix=${LIB_DIR} CFLAGS=-fPIC CXXFLAGS=-fPIC LDFLAGS=-L${LIB_DIR}/lib CPPFLAGS=-I${LIB_DIR}/include > netcdf-c.configure.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > netcdf-c.make_install.log 2>&1" + run_cmd "./configure --prefix=${LIB_DIR} CFLAGS=-fPIC CXXFLAGS=-fPIC LDFLAGS=-L${LIB_DIR}/lib CPPFLAGS=-I${LIB_DIR}/include > $(pwd)/netcdf-c.configure.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/netcdf-c.make_install.log 2>&1" echo echo "Compiling NetCDF-CXX at `date`" @@ -727,9 +744,8 @@ if [ $COMPILE_NETCDF -eq 1 ]; then if [[ $machine == "Mac" ]]; then configure_lib_args="-lnetcdf -lhdf5_hl -lhdf5 -lz" fi - run_cmd "./configure --prefix=${LIB_DIR} LDFLAGS=-L${LIB_DIR}/lib CPPFLAGS=-I${LIB_DIR}/include LIBS=\"${LIBS} ${configure_lib_args}\" > netcdf-cxx.configure.log 2>&1" - - run_cmd "make ${MAKE_ARGS} install > netcdf-cxx.make_install.log 2>&1" + run_cmd "./configure --prefix=${LIB_DIR} LDFLAGS=-L${LIB_DIR}/lib CPPFLAGS=-I${LIB_DIR}/include LIBS=\"${LIBS} ${configure_lib_args}\" > $(pwd)/netcdf-cxx.configure.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/netcdf-cxx.make_install.log 2>&1" fi # Compile FREETYPE @@ -741,9 +757,9 @@ if [ $COMPILE_FREETYPE -eq 1 ]; then tar -xzf ${TAR_DIR}/freetype*.tar.gz -C ${LIB_DIR}/freetype cd ${LIB_DIR}/freetype/freetype* echo "cd `pwd`" - run_cmd "./configure --prefix=${LIB_DIR} --with-png=yes > freetype.configure.log 2>&1" - run_cmd "make ${MAKE_ARGS} > freetype.make.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > freetype.make_install.log 2>&1" + run_cmd "./configure --prefix=${LIB_DIR} --with-png=yes > $(pwd)/freetype.configure.log 2>&1" + run_cmd "make ${MAKE_ARGS} > $(pwd)/freetype.make.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/freetype.make_install.log 2>&1" fi @@ -759,9 +775,9 @@ if [ $COMPILE_CAIRO -eq 1 ]; then tar -xzf ${TAR_DIR}/pixman*.tar.gz -C ${LIB_DIR}/pixman cd ${LIB_DIR}/pixman/pixman* echo "cd `pwd`" - run_cmd "./configure --prefix=${LIB_DIR} > pixman.configure.log 2>&1" - run_cmd "make ${MAKE_ARGS} > pixman.make.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > pixman.make_install.log 2>&1" + run_cmd "./configure --prefix=${LIB_DIR} > $(pwd)/pixman.configure.log 2>&1" + run_cmd "make ${MAKE_ARGS} > $(pwd)/pixman.make.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/pixman.make_install.log 2>&1" fi echo @@ -775,9 +791,9 @@ if [ $COMPILE_CAIRO -eq 1 ]; then export PKG_CONFIG_PATH=${LIB_DIR}/lib/pkgconfig/ fi echo "cd `pwd`" - run_cmd "./configure --prefix=${LIB_DIR} ax_cv_c_float_words_bigendian=no LDFLAGS=-L${LIB_DIR}/lib CPPFLAGS=-I${LIB_DIR}/include > cairo.configure.log 2>&1" - run_cmd "make ${MAKE_ARGS} > cairo.make.log 2>&1" - run_cmd "make ${MAKE_ARGS} install > cairo.make_install.log 2>&1" + run_cmd "./configure --prefix=${LIB_DIR} ax_cv_c_float_words_bigendian=no LDFLAGS=-L${LIB_DIR}/lib CPPFLAGS=-I${LIB_DIR}/include > $(pwd)/cairo.configure.log 2>&1" + run_cmd "make ${MAKE_ARGS} > $(pwd)/cairo.make.log 2>&1" + run_cmd "make ${MAKE_ARGS} install > $(pwd)/cairo.make_install.log 2>&1" fi # Compile MET @@ -827,19 +843,46 @@ fi export MET_PYTHON_BIN_EXE=${MET_PYTHON_BIN_EXE:=${MET_PYTHON}/bin/python3} export MET_PYTHON_LD export MET_PYTHON_CC -export LDFLAGS="-Wl,--disable-new-dtags" -if [[ $machine == "Mac" ]]; then - export LDFLAGS="" +# add flags to user-defined LDFLAGS for MacOS +if [[ $machine != "Mac" ]]; then + LDFLAGS="${LDFLAGS} -Wl,--disable-new-dtags" fi # https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html # ${parameter:+word} # If parameter is null or unset, nothing is substituted, otherwise the expansion of word is substituted. -export LDFLAGS="${LDFLAGS} -Wl,-rpath,${LIB_DIR}/lib:${MET_PROJ:+:$MET_PROJ/lib64}:${LIB_DIR}/lib${MET_NETCDF:+:$MET_NETCDF/lib}${MET_HDF5:+:$MET_HDF5/lib}${MET_BUFRLIB:+:$MET_BUFRLIB}${MET_GRIB2CLIB:+:$MET_GRIB2CLIB}${MET_PYTHON_LIB:+:$MET_PYTHON_LIB}${MET_GSL:+:$MET_GSL/lib}${ADDTL_DIR:+:$ADDTL_DIR}" -export LDFLAGS="${LDFLAGS} -Wl,-rpath,${LIB_JASPER:+$LIB_JASPER}${LIB_LIBPNG:+:$LIB_PNG}${LIB_Z:+$LIB_Z}" -export LDFLAGS="${LDFLAGS} ${LIB_JASPER:+-L$LIB_JASPER} ${LIB_LIBPNG:+-L$LIB_LIBPNG} ${MET_HDF5:+-L$MET_HDF5/lib} ${ADDTL_DIR:+-L$ADDTL_DIR}" -export LIBS="${LIBS} -lhdf5_hl -lhdf5 -lz -ltiff" + +# add LIB_DIR/lib and LIB_DIR/lib64 to rpath and -L +LDFLAGS="${LDFLAGS} -Wl,-rpath,${LIB_DIR}/lib -L${LIB_DIR}/lib -Wl,-rpath,${LIB_DIR}/lib64 -L${LIB_DIR}/lib64" + +# if variables are set, add /lib to rpath and -L +for x in $MET_CAIRO $MET_FREETYPE $MET_GSL $MET_HDF $MET_HDF5 $MET_NETCDF; do + arg="${x:+-Wl,-rpath,$x/lib -L$x/lib}" + if [[ "$LDFLAGS" != *"$arg"* ]]; then + LDFLAGS+=" $arg" + fi +done + +# if variables are set, add /lib64 to rpath and -L +for x in $MET_ATLAS $MET_BUFR $MET_ECKIT $MET_GRIB2C $MET_PROJ $LIB_JASPER; do + arg="${x:+-Wl,-rpath,$x/lib64 -L$x/lib64}" + if [[ "$LDFLAGS" != *"$arg"* ]]; then + LDFLAGS+=" $arg" + fi +done + +# if variables are set, add to rpath and -L +for x in $MET_ATLASLIB $MET_BUFRLIB $MET_CAIROLIB $MET_ECKITLIB $MET_FREETYPELIB $MET_GRIB2CLIB $MET_GSLLIB $MET_HDF5LIB $MET_HDFLIB $MET_NETCDFLIB $MET_PROJLIB $MET_PYTHON_LIB $LIB_JASPER $LIB_LIBPNG $LIB_Z $ADDTL_DIR; do + arg="${x:+-Wl,-rpath,$x -L$x}" + if [[ "$LDFLAGS" != *"$arg"* ]]; then + LDFLAGS+=" $arg" + fi +done + +export LDFLAGS + +export LIBS="${LIBS} -lhdf5_hl -lhdf5 -lz" export MET_FONT_DIR=${TEST_BASE}/fonts @@ -874,9 +917,9 @@ fi configure_cmd="${configure_cmd} ${OPT_ARGS}" echo "cd `pwd`" -run_cmd "${configure_cmd} > met.configure.log 2>&1" -run_cmd "make ${MAKE_ARGS} > met.make.log 2>&1" -run_cmd "make ${MAKE_ARGS} install > met.make_install.log 2>&1" -run_cmd "make ${MAKE_ARGS} test > met.make_test.log 2>&1" +run_cmd "${configure_cmd} > $(pwd)/configure.log 2>&1" +run_cmd "make ${MAKE_ARGS} > $(pwd)/met.make.log 2>&1" +run_cmd "make install > $(pwd)/met.make_install.log 2>&1" +run_cmd "make test > $(pwd)/met.make_test.log 2>&1" echo "Finished compiling at `date`" diff --git a/internal/scripts/installation/config/install_met_env.gaea b/internal/scripts/installation/config/install_met_env.gaea index a47e1d609f..78052504da 100644 --- a/internal/scripts/installation/config/install_met_env.gaea +++ b/internal/scripts/installation/config/install_met_env.gaea @@ -1,12 +1,37 @@ -module load intel/2022.2.1 +module load intel-oneapi/2022.0.2 -export TEST_BASE=/usw/met/12.0.0 -export COMPILER=intel_2022.2.1 +export FC=ifx +export F77=ifx +export F90=ifx +export CC=icx +export CXX=icpx +export TEST_BASE=/usw/met/12.0.0-beta2 +export COMPILER=intel-oneapi_2022.0.2 export MET_SUBDIR=${TEST_BASE} -export MET_TARBALL=v12.0.0.tar.gz +export MET_TARBALL=v12.0.0-beta2.tar.gz export USE_MODULES=TRUE export MET_PYTHON=/lustre/f2/dev/esrl/Julie.Prestopnik/projects/miniconda/miniconda3/envs/metplus_v5.1_py3.10/ export MET_PYTHON_CC=-I${MET_PYTHON}/include/python3.10 export MET_PYTHON_LD=-L${MET_PYTHON}/lib/python3.10/config-3.10-x86_64-linux-gnu\ -L${MET_PYTHON}/lib\ -lpython3.10\ -lcrypt\ -lpthread\ -ldl\ -lutil\ -lm -export EXTERNAL_LIBS=/usw/met/12.0.0-beta1/external_libs -export LIB_Z=${EXTERNAL_LIBS}/lib +export ADDTL_DIR=/usr/lib64 +#export TIFF_INCLUDE_DIR=/lustre/f2/dev/esrl/Julie.Prestopnik/projects/miniconda/miniconda3/pkgs/libtiff-4.4.0-h82bc61c_5/include +#export LIB_TIFF=${ADDTL_DIR}/libtiff.so.5 +#export EXTERNAL_LIBS=/usw/met/12.0.0-beta2/external_libs +#export MET_PROJ=${EXTERNAL_LIBS} +#COMPILE_ATLAS=1 +#COMPILE_ECKIT=1 +#export MET_ATLAS=${EXTERNAL_LIBS} +#export MET_ECKIT=${EXTERNAL_LIBS} +#export MET_GSL=${EXTERNAL_LIBS} +#export MET_BUFRLIB=${EXTERNAL_LIBS} +#export BUFRLIB_NAME=-lbufr_4 +#export MET_HDF5=${EXTERNAL_LIBS} +#export MET_NETCDF=${EXTERNAL_LIBS} +#export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib +#export MET_GRIB2CINC=${EXTERNAL_LIBS}/include +#export GRIB2CLIB_NAME=-lg2c +#export LIB_JASPER=${EXTERNAL_LIBS}/lib +#export LIB_LIBPNG=${EXTERNAL_LIBS}/lib +#export LIB_Z=${EXTERNAL_LIBS}/lib +export MAKE_ARGS="-j 5" + diff --git a/internal/scripts/installation/config/install_met_env.hera b/internal/scripts/installation/config/install_met_env.hera index 2b7b6ed08f..cddaa00f90 100755 --- a/internal/scripts/installation/config/install_met_env.hera +++ b/internal/scripts/installation/config/install_met_env.hera @@ -2,6 +2,11 @@ module load gnu/9.2.0 module load intel/2022.1.2 module load cmake/3.26.4 +export FC=ifort +export F77=ifort +export F90=ifort +export CC=icc +export CXX=icpc export PATH=/scratch1/BMC/dtc/miniconda/miniconda3/envs/metplus_v5.1_py3.10/bin:${PATH} export TEST_BASE=/contrib/met/12.0.0 export COMPILER=intel_2022.1.2 @@ -14,21 +19,21 @@ export MET_PYTHON_LD=`python3-config --ldflags --embed` export SET_D64BIT=FALSE export ADDTL_DIR=/usr/lib64 export EXTERNAL_LIBS=/contrib/met/12.0.0/external_libs/ +export COMPILE_ECKIT=1 +export COMPILE_ATLAS=1 #export MET_PROJ=${EXTERNAL_LIBS} +#export MET_ATLAS=${EXTERNAL_LIBS} +#export MET_ECKIT=${EXTERNAL_LIBS} #export MET_NETCDF=${EXTERNAL_LIBS} #export MET_GSL=${EXTERNAL_LIBS} #export MET_BUFRLIB=${EXTERNAL_LIBS} -#export BUFRLIB_NAME=-lbufr -#export MET_HDF5=${EXTERNAL_LIBS} +#export BUFRLIB_NAME=-lbufr_4 #export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib #export MET_GRIB2CINC=${EXTERNAL_LIBS}/include -#export GRIB2CLIB_NAME=-lgrib2c +#export GRIB2CLIB_NAME=-lg2c #export LIB_JASPER=${EXTERNAL_LIBS}/lib #export LIB_LIBPNG=${EXTERNAL_LIBS}/lib #export LIB_Z=${EXTERNAL_LIBS}/lib #export SQLITE_INCLUDE_DIR=${EXTERNAL_LIBS}/include #export SQLITE_LIB_DIR=${EXTERNAL_LIBS}/lib -#export CFLAGS="-Wall -g" -#export CXXFLAGS="-Wall -g -lcurl" -export MAKE_ARGS=-j -export CXXFLAGS="-std=c++11" +export MAKE_ARGS="-j 5" diff --git a/internal/scripts/installation/config/install_met_env.jet b/internal/scripts/installation/config/install_met_env.jet index 87442a55fe..b3be597853 100644 --- a/internal/scripts/installation/config/install_met_env.jet +++ b/internal/scripts/installation/config/install_met_env.jet @@ -1,4 +1,3 @@ -module load gnu/9.2.0 module load intel/2022.1.2 module load cmake/3.26.4 @@ -17,13 +16,19 @@ export MET_PYTHON_CC=-I${MET_PYTHON}/include/python3.10 export MET_PYTHON_LD=-L${MET_PYTHON}/lib/python3.10/config-3.10-x86_64-linux-gnu\ -L${MET_PYTHON}/lib\ -lpython3.10\ -lcrypt\ -lpthread\ -ldl\ -lutil\ -lrt\ -lm\ -lm export ADDTL_DIR=/usr/lib64 export EXTERNAL_LIBS=${TEST_BASE}/external_libs/ +COMPILE_ATLAS=1 +COMPILE_ECKIT=1 #export MET_PROJ=${EXTERNAL_LIBS} +export TIFF_INCLUDE_DIR=/usr/include +export TIFF_LIB_DIR=/usr/lib64 +#export MET_ATLAS=${EXTERNAL_LIBS} +#export MET_ECKIT=${EXTERNAL_LIBS} #export MET_GSL=${EXTERNAL_LIBS} #export MET_BUFRLIB=${EXTERNAL_LIBS} -#export BUFRLIB_NAME=-lbufr +#export BUFRLIB_NAME=-lbufr_4 #export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib #export MET_GRIB2CINC=${EXTERNAL_LIBS}/include -#export GRIB2CLIB_NAME=-lgrib2c +#export GRIB2CLIB_NAME=-lg2c #export MET_HDF5=${EXTERNAL_LIBS} #export MET_NETCDF=${EXTERNAL_LIBS} #export LIB_JASPER=${EXTERNAL_LIBS}/lib @@ -31,6 +36,4 @@ export EXTERNAL_LIBS=${TEST_BASE}/external_libs/ #export LIB_Z=${EXTERNAL_LIBS}/lib #export SQLITE_INCLUDE_DIR=${EXTERNAL_LIBS}/include #export SQLITE_LIB_DIR=${EXTERNAL_LIBS}/lib -export MAKE_ARGS=-j -export SET_D64BIT=FALSE -export CXXFLAGS="-std=c++11" +export MAKE_ARGS="-j 5" diff --git a/internal/scripts/installation/config/install_met_env.orion b/internal/scripts/installation/config/install_met_env.orion index 5f8b03bafb..b951ee41f5 100644 --- a/internal/scripts/installation/config/install_met_env.orion +++ b/internal/scripts/installation/config/install_met_env.orion @@ -1,29 +1,34 @@ module load intel/2020.2 +module load cmake/3.22.1 -export TEST_BASE=/apps/contrib/MET/11.1.0 +export TEST_BASE=/apps/contrib/MET/12.0.0-beta2 export COMPILER=intel_2020 export MET_SUBDIR=${TEST_BASE}/ -export MET_TARBALL=v11.1.0.tar.gz +export MET_TARBALL=v12.0.0-beta2.tar.gz export USE_MODULES=TRUE export MET_PYTHON=/work/noaa/ovp/miniconda/miniconda3/envs/metplus_v5.1_py3.10 export MET_PYTHON_CC=-I${MET_PYTHON}/include/python3.10 export MET_PYTHON_LD=-L${MET_PYTHON}/lib/python3.10/config-3.10-x86_64-linux-gnu\ -L${MET_PYTHON}/lib\ -lpython3.10\ -lcrypt\ -lpthread\ -ldl\ -lutil\ -lrt\ -lm\ -lm +export ADDTL_DIR=/usr/lib64 +#export TIFF_INCLUDE_DIR=${ADDTL_DIR} +#export LIB_TIFF=${ADDTL_DIR}/libtiff.so.5 export EXTERNAL_LIBS=${TEST_BASE}/external_libs -export LIB_Z=${EXTERNAL_LIBS}/lib +#export SQLITE_INCLUDE_DIR=${EXTERNAL_LIBS}/include +#export SQLITE_LIB_DIR=${EXTERNAL_LIBS}/lib +#export MET_PROJ=${EXTERNAL_LIBS} +#COMPILE_ATLAS=1 +#COMPILE_ECKIT=1 +#export MET_ATLAS=${EXTERNAL_LIBS} +#export MET_ECKIT=${EXTERNAL_LIBS} #export MET_GSL=${EXTERNAL_LIBS} #export MET_BUFRLIB=${EXTERNAL_LIBS} -#export BUFRLIB_NAME=-lbufr +#export BUFRLIB_NAME=-lbufr_4 #export MET_HDF5=${EXTERNAL_LIBS} #export MET_NETCDF=${EXTERNAL_LIBS} #export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib #export MET_GRIB2CINC=${EXTERNAL_LIBS}/include -#export GRIB2CLIB_NAME=-lgrib2c +#export GRIB2CLIB_NAME=-lg2c #export LIB_JASPER=${EXTERNAL_LIBS}/lib #export LIB_LIBPNG=${EXTERNAL_LIBS}/lib -#export SET_D64BIT=FALSE -export MAKE_ARGS=-j -#export CFLAGS="-Wall -g" -#export CXXFLAGS="-Wall -g" -export CXXFLAGS="-std=c++11" - - +#export LIB_Z=${EXTERNAL_LIBS}/lib +export MAKE_ARGS="-j 5" diff --git a/internal/scripts/installation/config/install_met_env_met_only.casper b/internal/scripts/installation/config/install_met_env_met_only.casper deleted file mode 100644 index beb46d927d..0000000000 --- a/internal/scripts/installation/config/install_met_env_met_only.casper +++ /dev/null @@ -1,30 +0,0 @@ -module load ncarenv/1.3 -module load intel/2021.2 -module load netcdf/4.8.0 - -export TEST_BASE=/glade/p/ral/jntp/MET/MET_releases/casper/11.1.0 -export COMPILER=intel_2021.2 -export MET_SUBDIR=${TEST_BASE} -export MET_TARBALL=v11.1.0.tar.gz -export USE_MODULES=TRUE -export MET_PYTHON=/glade/p/ral/jntp/MET/METplus/miniconda/miniconda3/envs/metplus_v5.1_py3.10 -export MET_PYTHON_CC=-I${MET_PYTHON}/include/python3.10 -export MET_PYTHON_LD=`${MET_PYTHON}/bin/python3-config --ldflags --embed` -export MET_NETCDF=/glade/u/apps/ch/opt/netcdf/4.8.0/intel/2021.2/ -export MET_HDF5=/glade/u/apps/ch/opt/netcdf/4.8.0/intel/2021.2/ -export EXTERNAL_LIBS=${TEST_BASE}/external_libs -export MET_GSL=${EXTERNAL_LIBS} -export MET_BUFRLIB=${EXTERNAL_LIBS} -export BUFRLIB_NAME=-lbufr -export MET_HDF5=${EXTERNAL_LIBS} -export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib -export MET_GRIB2CINC=${EXTERNAL_LIBS}/include -export GRIB2CLIB_NAME=-lgrib2c -export LIB_JASPER=${EXTERNAL_LIBS}/lib -export LIB_LIBPNG=${EXTERNAL_LIBS}/lib -export LIB_Z=${EXTERNAL_LIBS}/lib -export SET_D64BIT=FALSE -#export CFLAGS="-Wall -g" -#export CXXFLAGS="-Wall -g" -export MAKE_ARGS=-j -export CXXFLAGS="-std=c++11" diff --git a/internal/scripts/installation/config/install_met_env_met_only.cheyenne b/internal/scripts/installation/config/install_met_env_met_only.cheyenne deleted file mode 100644 index 4049fc774a..0000000000 --- a/internal/scripts/installation/config/install_met_env_met_only.cheyenne +++ /dev/null @@ -1,30 +0,0 @@ -module load ncarenv/1.3 -module load intel/2021.2 -module load netcdf/4.8.0 - -export TEST_BASE=/glade/p/ral/jntp/MET/MET_releases/11.1.0 -export COMPILER=intel_2021.2 -export MET_SUBDIR=${TEST_BASE} -export MET_TARBALL=v11.1.0.tar.gz -export USE_MODULES=TRUE -export MET_PYTHON=/glade/p/ral/jntp/MET/METplus/miniconda/miniconda3/envs/metplus_v5.1_py3.10 -export MET_PYTHON_CC=-I${MET_PYTHON}/include/python3.10 -export MET_PYTHON_LD=`${MET_PYTHON}/bin/python3-config --ldflags --embed` -export MET_NETCDF=/glade/u/apps/ch/opt/netcdf/4.8.0/intel/2021.2/ -export MET_HDF5=/glade/u/apps/ch/opt/netcdf/4.8.0/intel/2021.2/ -export EXTERNAL_LIBS=${TEST_BASE}/external_libs -export MET_GSL=${EXTERNAL_LIBS} -export MET_BUFRLIB=${EXTERNAL_LIBS} -export BUFRLIB_NAME=-lbufr -export MET_HDF5=${EXTERNAL_LIBS} -export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib -export MET_GRIB2CINC=${EXTERNAL_LIBS}/include -export GRIB2CLIB_NAME=-lgrib2c -export LIB_JASPER=${EXTERNAL_LIBS}/lib -export LIB_LIBPNG=${EXTERNAL_LIBS}/lib -export LIB_Z=${EXTERNAL_LIBS}/lib -export SET_D64BIT=FALSE -##export CFLAGS="-Wall -g" -##export CXXFLAGS="-Wall -g" -export MAKE_ARGS=-j -export CXXFLAGS="-std=c++11" diff --git a/internal/scripts/installation/config/install_met_env_met_only.frontera b/internal/scripts/installation/config/install_met_env_met_only.frontera deleted file mode 100644 index 0d56c92e8a..0000000000 --- a/internal/scripts/installation/config/install_met_env_met_only.frontera +++ /dev/null @@ -1,27 +0,0 @@ -module load intel/19.1.1 -module load hdf5/1.12.0 -module load netcdf/4.7.4 - -export TEST_BASE=/work2/06612/tg859120/frontera/MET/11.0.0 -export COMPILER=intel_19.1.1 -export MET_SUBDIR=${TEST_BASE}/ -export MET_TARBALL=v11.0.0.tar.gz -export USE_MODULES=TRUE -export MET_PYTHON=/work2/06612/tg859120/frontera/miniconda/miniconda3/envs/metplus_v5.0_py3.8 -export MET_PYTHON_CC=-I${MET_PYTHON}/include/python3.8 -export MET_PYTHON_LD=-L${MET_PYTHON}/lib\ -lpython3.8\ -lcrypt\ -lpthread\ -ldl\ -lutil\ -lm -export MET_HDF5=/opt/apps/intel19/hdf5/1.12.0/x86_64/ -export MET_NETCDF=/opt/apps/intel19/netcdf/4.7.4/x86_64/ -export EXTERNAL_LIBS=${TEST_BASE}/external_libs -export MET_GSL=${EXTERNAL_LIBS} -export MET_BUFRLIB=${EXTERNAL_LIBS} -export BUFRLIB_NAME=-lbufr -export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib -export MET_GRIB2CINC=${EXTERNAL_LIBS}/include -export GRIB2CLIB_NAME=-lgrib2c -export LIB_JASPER=${EXTERNAL_LIBS}/lib -export LIB_LIBPNG=${EXTERNAL_LIBS}/lib -export LIB_Z=${EXTERNAL_LIBS}/lib -export SET_D64BIT=FALSE -#export CFLAGS="-Wall -g" -#export CXXFLAGS="-Wall -g" \ No newline at end of file diff --git a/internal/scripts/installation/config/install_met_env_met_only.gaea b/internal/scripts/installation/config/install_met_env_met_only.gaea deleted file mode 100644 index f3a72e6dbb..0000000000 --- a/internal/scripts/installation/config/install_met_env_met_only.gaea +++ /dev/null @@ -1,26 +0,0 @@ -module load intel/2022.2.1 - -export TEST_BASE=/usw/met/12.0.0 -export COMPILER=intel_2022.2.1 -export MET_SUBDIR=${TEST_BASE} -export MET_TARBALL=v12.0.0.tar.gz -export USE_MODULES=TRUE -export MET_PYTHON=/lustre/f2/dev/esrl/Julie.Prestopnik/projects/miniconda/miniconda3/envs/metplus_v5.1_py3.10/ -export MET_PYTHON_CC=-I${MET_PYTHON}/include/python3.10 -export MET_PYTHON_LD=-L${MET_PYTHON}/lib/python3.10/config-3.10-x86_64-linux-gnu\ -L${MET_PYTHON}/lib\ -lpython3.10\ -lcrypt\ -lpthread\ -ldl\ -lutil\ -lm -export EXTERNAL_LIBS=/usw/met/12.0.0/external_libs -export MET_PROJ=${EXTERNAL_LIBS} -export MET_NETCDF=${EXTERNAL_LIBS} -export MET_GSL=${EXTERNAL_LIBS} -export MET_BUFRLIB=${EXTERNAL_LIBS} -export BUFRLIB_NAME=-lbufr -export MET_HDF5=${EXTERNAL_LIBS} -export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib -export MET_GRIB2CINC=${EXTERNAL_LIBS}/include -export GRIB2CLIB_NAME=-lgrib2c -export LIB_JASPER=${EXTERNAL_LIBS}/lib -export LIB_LIBPNG=${EXTERNAL_LIBS}/lib -export LIB_Z=${EXTERNAL_LIBS}/lib -#export CFLAGS="-Wall -g" -#export CXXFLAGS="-Wall -g -lcurl" -export MAKE_ARGS=-j diff --git a/internal/scripts/installation/config/install_met_env_met_only.hera b/internal/scripts/installation/config/install_met_env_met_only.hera deleted file mode 100644 index b8ac02a3e0..0000000000 --- a/internal/scripts/installation/config/install_met_env_met_only.hera +++ /dev/null @@ -1,34 +0,0 @@ -module load gnu/9.2.0 -module load intel/2022.1.2 -module load cmake/3.26.4 - -export PATH=/scratch1/BMC/dtc/miniconda/miniconda3/envs/metplus_v5.1_py3.10/bin:${PATH} -export TEST_BASE=/contrib/met/12.0.0 -export COMPILER=intel_2022.1.2 -export MET_SUBDIR=${TEST_BASE} -export MET_TARBALL=v12.0.0.tar.gz -export USE_MODULES=TRUE -export MET_PYTHON=/scratch1/BMC/dtc/miniconda/miniconda3/envs/metplus_v5.1_py3.10 -export MET_PYTHON_CC=-I${MET_PYTHON}/include/python3.10 -export MET_PYTHON_LD=`python3-config --ldflags --embed` -export SET_D64BIT=FALSE -export ADDTL_DIR=/usr/lib64 -export EXTERNAL_LIBS=/contrib/met/12.0.0/external_libs/ -export MET_PROJ=${EXTERNAL_LIBS} -export MET_NETCDF=${EXTERNAL_LIBS} -export MET_GSL=${EXTERNAL_LIBS} -export MET_BUFRLIB=${EXTERNAL_LIBS} -export BUFRLIB_NAME=-lbufr -export MET_HDF5=${EXTERNAL_LIBS} -export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib -export MET_GRIB2CINC=${EXTERNAL_LIBS}/include -export GRIB2CLIB_NAME=-lgrib2c -export LIB_JASPER=${EXTERNAL_LIBS}/lib -export LIB_LIBPNG=${EXTERNAL_LIBS}/lib -export LIB_Z=${EXTERNAL_LIBS}/lib -export SQLITE_INCLUDE_DIR=${EXTERNAL_LIBS}/include -export SQLITE_LIB_DIR=${EXTERNAL_LIBS}/lib -#export CFLAGS="-Wall -g" -#export CXXFLAGS="-Wall -g -lcurl" -export MAKE_ARGS=-j -export CXXFLAGS="-std=c++11" diff --git a/internal/scripts/installation/config/install_met_env_met_only.jet b/internal/scripts/installation/config/install_met_env_met_only.jet deleted file mode 100644 index 00b9d689d2..0000000000 --- a/internal/scripts/installation/config/install_met_env_met_only.jet +++ /dev/null @@ -1,36 +0,0 @@ -module load gnu/9.2.0 -module load intel/2022.1.2 -module load cmake/3.26.4 - -export FC=ifort -export F77=ifort -export F90=ifort -export CC=icc -export CXX=icpc -export TEST_BASE=/contrib/met/12.0.0 -export COMPILER=intel_2022.1.2 -export MET_SUBDIR=${TEST_BASE} -export MET_TARBALL=v12.0.0.tar.gz -export USE_MODULES=TRUE -export MET_PYTHON=/mnt/lfs1/HFIP/dtc-hurr/METplus/miniconda/miniconda3/envs/metplus_v5.1_py3.10 -export MET_PYTHON_CC=-I${MET_PYTHON}/include/python3.10 -export MET_PYTHON_LD=-L${MET_PYTHON}/lib/python3.10/config-3.10-x86_64-linux-gnu\ -L${MET_PYTHON}/lib\ -lpython3.10\ -lcrypt\ -lpthread\ -ldl\ -lutil\ -lrt\ -lm\ -lm -export ADDTL_DIR=/usr/lib64 -export EXTERNAL_LIBS=${TEST_BASE}/external_libs/ -export MET_PROJ=${EXTERNAL_LIBS} -export MET_GSL=${EXTERNAL_LIBS} -export MET_BUFRLIB=${EXTERNAL_LIBS} -export BUFRLIB_NAME=-lbufr -export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib -export MET_GRIB2CINC=${EXTERNAL_LIBS}/include -export GRIB2CLIB_NAME=-lgrib2c -export MET_HDF5=${EXTERNAL_LIBS} -export MET_NETCDF=${EXTERNAL_LIBS} -export LIB_JASPER=${EXTERNAL_LIBS}/lib -export LIB_LIBPNG=${EXTERNAL_LIBS}/lib -export LIB_Z=${EXTERNAL_LIBS}/lib -export SQLITE_INCLUDE_DIR=${EXTERNAL_LIBS}/include -export SQLITE_LIB_DIR=${EXTERNAL_LIBS}/lib -export MAKE_ARGS=-j -export SET_D64BIT=FALSE -export CXXFLAGS="-std=c++11" diff --git a/internal/scripts/installation/config/install_met_env_met_only.orion b/internal/scripts/installation/config/install_met_env_met_only.orion deleted file mode 100644 index 66cea2a43c..0000000000 --- a/internal/scripts/installation/config/install_met_env_met_only.orion +++ /dev/null @@ -1,28 +0,0 @@ -module load intel/2020.2 - -export TEST_BASE=/apps/contrib/MET/11.1.0 -export COMPILER=intel_2020 -export MET_SUBDIR=${TEST_BASE}/ -export MET_TARBALL=v11.1.0.tar.gz -export USE_MODULES=TRUE -export MET_PYTHON=/work/noaa/ovp/miniconda/miniconda3/envs/metplus_v5.1_py3.10 -export MET_PYTHON_CC=-I${MET_PYTHON}/include/python3.10 -export MET_PYTHON_LD=`${MET_PYTHON}/bin/python3-config --ldflags --embed` -export EXTERNAL_LIBS=${TEST_BASE}/external_libs -export MET_GSL=${EXTERNAL_LIBS} -export MET_BUFRLIB=${EXTERNAL_LIBS} -export BUFRLIB_NAME=-lbufr -export MET_HDF5=${EXTERNAL_LIBS} -export MET_NETCDF=${EXTERNAL_LIBS} -export MET_GRIB2CLIB=${EXTERNAL_LIBS}/lib -export MET_GRIB2CINC=${EXTERNAL_LIBS}/include -export GRIB2CLIB_NAME=-lgrib2c -export LIB_JASPER=${EXTERNAL_LIBS}/lib -export LIB_LIBPNG=${EXTERNAL_LIBS}/lib -export LIB_Z=${EXTERNAL_LIBS}/lib -export SET_D64BIT=FALSE -export MAKE_ARGS=-j -#export CFLAGS="-Wall -g" -#export CXXFLAGS="-Wall -g" -export CXXFLAGS="-std=c++11" - diff --git a/internal/scripts/installation/modulefiles/11.1.0_orion b/internal/scripts/installation/modulefiles/11.1.0_orion deleted file mode 100644 index 03ebbc352a..0000000000 --- a/internal/scripts/installation/modulefiles/11.1.0_orion +++ /dev/null @@ -1,45 +0,0 @@ -#%Module###################################################################### -## -## Model Evaluation Tools -## -proc ModulesHelp { } { - puts stderr "Sets up the paths and environment variables to use the Model Evaluation Tools v11.1.0 - *** For help see the official MET webpage at http://www.dtcenter.org/met/users ***" -} - -prereq intel/2020.2 - -set base /apps/contrib/MET -set ver 11.1.0 -set share $base/$ver/share/met -set lib_base $base/11.1.0 -setenv MET_ROOT $base/$ver/MET-11.1.0 - -prepend-path PATH $base/$ver/bin:$lib_base/external_libs/bin:/work/noaa/ovp/miniconda/miniconda3/envs/metplus_v5.1_py3.10/bin - -export CC=icc -#export CXX=icc -#export F77=ifort -#module load intel/2020.2 -#module load intelpython3/2020.2 -#export libdir=/apps/contrib/MET/11.1.0/external_libs/lib -#export incdir=/apps/contrib/MET/11.1.0/external_libs/include -#export iprefix=/apps/contrib/MET/11.1.0/external_libs -#export basedir=/apps/contrib/MET/11.1.0/MET-11.1.0 -#export MET_HDF5=$iprefix -#export MET_NETCDF=$incdir -#export MET_GRIB2CINC=$incdir -#export MET_GRIB2CLIB=$libdir -#setenv MET_GSLLIB=$libdir -#export MET_GSLINC=$incdir -#export MET_BUFR=$libdir -#export MET_HDFINC=$incdir -#export MET_HDFLIB=$libdir -#export MET_HDFEOSINC=$incdir -#export MET_HDFEOSLIB=$libdir -#export MET_PYTHON=/work/noaa/ovp/miniconda/miniconda3/envs/metplus_v5.1_py3.10 -#export MET_PYTHON_CC=-I${MET_PYTHON}/include/python3.10 -#export MET_PYTHON_LD=-L${MET_PYTHON}/lib/python3.10/config-3.10-x86_64-linux-gnu\ -L${MET_PYTHON}/lib\ -lpython3.10\ -lcrypt\ -lpthread\ -ldl\ -lutil\ -lrt\ -lm\ -lm -#export MET_FONT_DIR=$basedir/fonts/ -#export LDFLAGS=-Wl,--disable-new-dtags -Wl,-rpath,${libdir}:${MET_PYTHON}/lib -#export CPPFLAGS=-I/apps/contrib/met/11.1.0/external_libs/include diff --git a/internal/scripts/installation/modulefiles/12.0.0_gaea b/internal/scripts/installation/modulefiles/12.0.0_gaea index 3afc495ab6..1ce4cd13c3 100644 --- a/internal/scripts/installation/modulefiles/12.0.0_gaea +++ b/internal/scripts/installation/modulefiles/12.0.0_gaea @@ -7,7 +7,7 @@ proc ModulesHelp { } { *** For help see the official MET webpage at http://www.dtcenter.org/met/users ***" } -module load intel/2022.2.1 +module load intel-oneapi/2022.0.2 set base /usw/met set ver 12.0.0 diff --git a/internal/scripts/installation/modulefiles/12.0.0_orion b/internal/scripts/installation/modulefiles/12.0.0_orion new file mode 100644 index 0000000000..ed0dd56805 --- /dev/null +++ b/internal/scripts/installation/modulefiles/12.0.0_orion @@ -0,0 +1,19 @@ +#%Module###################################################################### +## +## Model Evaluation Tools +## +proc ModulesHelp { } { + puts stderr "Sets up the paths and environment variables to use the Model Evaluation Tools v12.0.0 + *** For help see the official MET webpage at http://www.dtcenter.org/met/users ***" +} + +prereq intel/2020.2 + +set base /apps/contrib/MET +set ver 12.0.0-beta2 +set share $base/$ver/share/met +set lib_base $base/12.0.0-beta2 +setenv MET_ROOT $base/$ver/MET-12.0.0-beta2 + +prepend-path PATH $base/$ver/bin:$lib_base/external_libs/bin:/work/noaa/ovp/miniconda/miniconda3/envs/metplus_v5.1_py3.10/bin + diff --git a/internal/test_unit/config/GridStatConfig_WRF_pres b/internal/test_unit/config/GridStatConfig_WRF_pres new file mode 100644 index 0000000000..10d46e8f94 --- /dev/null +++ b/internal/test_unit/config/GridStatConfig_WRF_pres @@ -0,0 +1,288 @@ +//////////////////////////////////////////////////////////////////////////////// +// +// Grid-Stat configuration file. +// +// For additional information, please see the MET User's Guide. +// +//////////////////////////////////////////////////////////////////////////////// + +// +// Output model name to be written +// +model = "WRF"; + +// +// Output description to be written +// May be set separately in each "obs.field" entry +// +desc = "NA"; + +// +// Output observation type to be written +// +obtype = "WRF"; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification grid +// May be set separately in each "field" entry +// +regrid = { + to_grid = NONE; + method = NEAREST; + width = 1; + vld_thresh = 0.5; + shape = SQUARE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// May be set separately in each "field" entry +// +censor_thresh = []; +censor_val = []; +mpr_column = []; +mpr_thresh = []; +cat_thresh = []; +cnt_thresh = [ NA ]; +cnt_logic = UNION; +wind_thresh = [ NA ]; +wind_logic = UNION; +eclv_points = 0.05; +nc_pairs_var_name = ""; +nc_pairs_var_suffix = ""; +hss_ec_value = NA; +rank_corr_flag = FALSE; + +// +// Forecast and observation fields to be verified +// +fcst = { + + field = [ + { + name = "U_PL"; + level = [ "(0,2,*,*)" ]; + cat_thresh = [ >=0.0, >=2.5 ]; + } + ]; + +} +obs = { + + field = [ + { + name = "U_PL"; + level = [ "(0,@50000,*,*)" ]; + cat_thresh = [ >=0.0, >=2.5 ]; + } + ]; + +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Climatology data +// +climo_mean = { + + file_name = []; + field = []; + + regrid = { + method = NEAREST; + width = 1; + vld_thresh = 0.5; + shape = SQUARE; + } + + time_interp_method = DW_MEAN; + day_interval = 31; + hour_interval = 6; +} + +climo_stdev = climo_mean; +climo_stdev = { + file_name = []; +} + +// +// May be set separately in each "obs.field" entry +// +climo_cdf = { + cdf_bins = 1; + center_bins = FALSE; + write_bins = TRUE; + direct_prob = FALSE; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification masking regions +// May be set separately in each "obs.field" entry +// +mask = { + grid = [ "FULL" ]; + poly = []; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Confidence interval settings +// May be set separately in each "obs.field" entry +// +ci_alpha = [ 0.05 ]; + +boot = { + interval = PCTILE; + rep_prop = 1.0; + n_rep = 0; + rng = "mt19937"; + seed = ""; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Data smoothing methods +// May be set separately in each "obs.field" entry +// +interp = { + field = BOTH; + vld_thresh = 1.0; + shape = SQUARE; + + type = [ + { + method = NEAREST; + width = 1; + } + ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Neighborhood methods +// May be set separately in each "obs.field" entry +// +nbrhd = { + field = BOTH; + vld_thresh = 1.0; + shape = SQUARE; + width = [ 1 ]; + cov_thresh = [ >=0.5 ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Fourier decomposition +// May be set separately in each "obs.field" entry +// +fourier = { + wave_1d_beg = []; + wave_1d_end = []; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Gradient statistics +// May be set separately in each "obs.field" entry +// +gradient = { + dx = [ 1 ]; + dy = [ 1 ]; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Distance Map statistics +// May be set separately in each "obs.field" entry +// +distance_map = { + baddeley_p = 2; + baddeley_max_dist = NA; + fom_alpha = 0.1; + zhu_weight = 0.5; + beta_value(n) = n * n / 2.0; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Threshold for SEEPS p1 (Probability of being dry) +// +seeps_p1_thresh = >=0.1&&<=0.85; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Statistical output types +// May be set separately in each "obs.field" entry +// +output_flag = { + fho = STAT; + ctc = STAT; + cts = STAT; + mctc = STAT; + mcts = STAT; + cnt = STAT; + sl1l2 = STAT; + sal1l2 = NONE; + vl1l2 = NONE; + val1l2 = NONE; + vcnt = NONE; + pct = NONE; + pstd = NONE; + pjc = NONE; + prc = NONE; + eclv = NONE; + nbrctc = NONE; + nbrcts = NONE; + nbrcnt = NONE; + grad = NONE; + dmap = NONE; + seeps = NONE; +} + +// +// NetCDF matched pairs output file +// May be set separately in each "obs.field" entry +// +nc_pairs_flag = { + latlon = TRUE; + raw = TRUE; + diff = TRUE; + climo = TRUE; + climo_cdp = FALSE; + seeps = FALSE; + weight = FALSE; + nbrhd = FALSE; + fourier = FALSE; + gradient = FALSE; + distance_map = FALSE; + apply_mask = TRUE; +} + +//////////////////////////////////////////////////////////////////////////////// + +ugrid_dataset = ""; +ugrid_max_distance_km = 30; +ugrid_coordinates_file = ""; + +//////////////////////////////////////////////////////////////////////////////// + +grid_weight_flag = NONE; +tmp_dir = "/tmp"; +output_prefix = "${OUTPUT_PREFIX}"; +version = "V12.0.0"; + +//////////////////////////////////////////////////////////////////////////////// diff --git a/internal/test_unit/xml/unit_grid_stat.xml b/internal/test_unit/xml/unit_grid_stat.xml index 204e934230..f70f53ad22 100644 --- a/internal/test_unit/xml/unit_grid_stat.xml +++ b/internal/test_unit/xml/unit_grid_stat.xml @@ -311,4 +311,21 @@ + + &MET_BIN;/grid_stat + + OUTPUT_PREFIX WRF_pres + + \ + &DATA_DIR_MODEL;/wrf/wrfout_solarwind_d02_2019-08-31_12:00:00 \ + &DATA_DIR_MODEL;/wrf/wrfout_solarwind_d02_2019-08-31_12:00:00 \ + &CONFIG_DIR;/GridStatConfig_WRF_pres \ + -outdir &OUTPUT_DIR;/grid_stat -v 1 + + + &OUTPUT_DIR;/grid_stat/grid_stat_WRF_pres_120000L_20190831_120000V.stat + &OUTPUT_DIR;/grid_stat/grid_stat_WRF_pres_120000L_20190831_120000V_pairs.nc + + + diff --git a/internal/test_unit/xml/unit_plot_data_plane.xml b/internal/test_unit/xml/unit_plot_data_plane.xml index 86709b0621..2337c7a705 100644 --- a/internal/test_unit/xml/unit_plot_data_plane.xml +++ b/internal/test_unit/xml/unit_plot_data_plane.xml @@ -581,4 +581,88 @@ + + &MET_BIN;/plot_data_plane + \ + &DATA_DIR_MODEL;/wrf/wrfout_solarwind_d02_2019-08-31_12:00:00 \ + &OUTPUT_DIR;/plot_data_plane/wrf_west_east_stag.ps \ + 'name="U"; level="(0,0,*,*)";' \ + -title "X-wind component (staggered)" \ + -v 1 + + + &OUTPUT_DIR;/plot_data_plane/wrf_west_east_stag.ps + + + + + &MET_BIN;/plot_data_plane + \ + &DATA_DIR_MODEL;/wrf/wrfout_solarwind_d02_2019-08-31_12:00:00 \ + &OUTPUT_DIR;/plot_data_plane/wrf_south_north_stag.ps \ + 'name="V"; level="(0,0,*,*)";' \ + -title "Y-wind component (staggered)" \ + -v 1 + + + &OUTPUT_DIR;/plot_data_plane/wrf_south_north_stag.ps + + + + + &MET_BIN;/plot_data_plane + \ + &DATA_DIR_MODEL;/wrf/wrfout_solarwind_d02_2019-08-31_12:00:00 \ + &OUTPUT_DIR;/plot_data_plane/wrf_num_press_levels_stag.ps \ + 'name="RH_PL"; level="(0,@50000,*,*)";' \ + -title "Relative Humidity P50000 (num_press_levels_stag)" \ + -v 1 + + + &OUTPUT_DIR;/plot_data_plane/wrf_num_press_levels_stag.ps + + + + + &MET_BIN;/plot_data_plane + \ + &DATA_DIR_MODEL;/wrf/wrfout_solarwind_d02_2019-08-31_12:00:00 \ + &OUTPUT_DIR;/plot_data_plane/wrf_num_z_levels_stag.ps \ + 'name="RH_ZL"; level="(0,0,*,*)";' \ + -title "Relative Humidity Z-level (num_z_levels_stag)" \ + -v 1 + + + &OUTPUT_DIR;/plot_data_plane/wrf_num_z_levels_stag.ps + + + + + &MET_BIN;/plot_data_plane + \ + &DATA_DIR_MODEL;/wrf/wrfout_solarwind_d02_2019-08-31_12:00:00 \ + &OUTPUT_DIR;/plot_data_plane/wrf_bottom_top.ps \ + 'name="T"; level="(0,0,*,*)";' \ + -title "perturbation potential temperature theta-t0 (bottom_top)" \ + -v 1 + + + &OUTPUT_DIR;/plot_data_plane/wrf_bottom_top.ps + + + + + &MET_BIN;/plot_data_plane + \ + &DATA_DIR_MODEL;/wrf/wrfout_solarwind_d02_2019-08-31_12:00:00 \ + &OUTPUT_DIR;/plot_data_plane/wrf_bottom_top_stag.ps \ + 'name="PH"; level="(0,22,*,*)";' \ + -title "perturbation geopotential (bottom_top_stag)" \ + -v 1 + + + &OUTPUT_DIR;/plot_data_plane/wrf_bottom_top_stag.ps + + + diff --git a/internal/test_util/basic/vx_util/Makefile.am b/internal/test_util/basic/vx_util/Makefile.am index 8717867cda..c39a98ace1 100644 --- a/internal/test_util/basic/vx_util/Makefile.am +++ b/internal/test_util/basic/vx_util/Makefile.am @@ -93,9 +93,9 @@ test_ascii_header_LDADD = -lvx_stat_out \ -lvx_statistics \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nc_pinterp \ + -lvx_data2d_nc_wrf \ $(PYTHON_MET_LIBS) $(PYTHON_DEP_LIBS) \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ -lvx_data2d_grib $(GRIB2_MET_LIBS) $(GRIB2_DEP_LIBS) \ -lvx_data2d \ -lvx_nc_util \ diff --git a/internal/test_util/basic/vx_util/Makefile.in b/internal/test_util/basic/vx_util/Makefile.in index d904c45e7a..ebbef0522c 100644 --- a/internal/test_util/basic/vx_util/Makefile.in +++ b/internal/test_util/basic/vx_util/Makefile.in @@ -401,9 +401,9 @@ test_ascii_header_LDADD = -lvx_stat_out \ -lvx_statistics \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nc_pinterp \ + -lvx_data2d_nc_wrf \ $(PYTHON_MET_LIBS) $(PYTHON_DEP_LIBS) \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ -lvx_data2d_grib $(GRIB2_MET_LIBS) $(GRIB2_DEP_LIBS) \ -lvx_data2d \ -lvx_nc_util \ diff --git a/internal/test_util/libcode/Makefile.am b/internal/test_util/libcode/Makefile.am index e05a0595fc..38c5944d79 100644 --- a/internal/test_util/libcode/Makefile.am +++ b/internal/test_util/libcode/Makefile.am @@ -19,7 +19,7 @@ SUBDIRS = vx_data2d \ vx_data2d_grib \ vx_data2d_nc_met \ - vx_data2d_nccf \ + vx_data2d_nc_cf \ vx_data2d_factory \ vx_geodesy \ vx_grid \ diff --git a/internal/test_util/libcode/Makefile.in b/internal/test_util/libcode/Makefile.in index 2f708a52b2..0f56d40950 100644 --- a/internal/test_util/libcode/Makefile.in +++ b/internal/test_util/libcode/Makefile.in @@ -339,7 +339,7 @@ target_alias = @target_alias@ top_build_prefix = @top_build_prefix@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ -SUBDIRS = vx_data2d vx_data2d_grib vx_data2d_nc_met vx_data2d_nccf \ +SUBDIRS = vx_data2d vx_data2d_grib vx_data2d_nc_met vx_data2d_nc_cf \ vx_data2d_factory vx_geodesy vx_grid vx_ps vx_solar \ vx_plot_util vx_tc_util vx_nc_util vx_physics vx_series_data \ $(am__append_1) diff --git a/internal/test_util/libcode/vx_data2d_factory/Makefile.am b/internal/test_util/libcode/vx_data2d_factory/Makefile.am index 77bd62fff1..ad1ad279e4 100644 --- a/internal/test_util/libcode/vx_data2d_factory/Makefile.am +++ b/internal/test_util/libcode/vx_data2d_factory/Makefile.am @@ -34,8 +34,8 @@ test_factory_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/internal/test_util/libcode/vx_data2d_factory/Makefile.in b/internal/test_util/libcode/vx_data2d_factory/Makefile.in index 7964d3fd22..1d93387de5 100644 --- a/internal/test_util/libcode/vx_data2d_factory/Makefile.in +++ b/internal/test_util/libcode/vx_data2d_factory/Makefile.in @@ -355,8 +355,8 @@ test_factory_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/internal/test_util/libcode/vx_data2d_nccf/.gitignore b/internal/test_util/libcode/vx_data2d_nc_cf/.gitignore similarity index 100% rename from internal/test_util/libcode/vx_data2d_nccf/.gitignore rename to internal/test_util/libcode/vx_data2d_nc_cf/.gitignore diff --git a/internal/test_util/libcode/vx_data2d_nccf/Makefile.am b/internal/test_util/libcode/vx_data2d_nc_cf/Makefile.am similarity index 93% rename from internal/test_util/libcode/vx_data2d_nccf/Makefile.am rename to internal/test_util/libcode/vx_data2d_nc_cf/Makefile.am index 1c9bbbe8e3..ac732919dc 100644 --- a/internal/test_util/libcode/vx_data2d_nccf/Makefile.am +++ b/internal/test_util/libcode/vx_data2d_nc_cf/Makefile.am @@ -15,7 +15,7 @@ noinst_PROGRAMS = test_read_nccf test_read_nccf_SOURCES = test_read_nccf.cc test_read_nccf_CPPFLAGS = ${MET_CPPFLAGS} test_read_nccf_LDFLAGS = -L. ${MET_LDFLAGS} -test_read_nccf_LDADD = -lvx_data2d_nccf \ +test_read_nccf_LDADD = -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ -lvx_nc_util \ -lvx_data2d \ diff --git a/internal/test_util/libcode/vx_data2d_nccf/Makefile.in b/internal/test_util/libcode/vx_data2d_nc_cf/Makefile.in similarity index 99% rename from internal/test_util/libcode/vx_data2d_nccf/Makefile.in rename to internal/test_util/libcode/vx_data2d_nc_cf/Makefile.in index 26689f2e99..52429cc106 100644 --- a/internal/test_util/libcode/vx_data2d_nccf/Makefile.in +++ b/internal/test_util/libcode/vx_data2d_nc_cf/Makefile.in @@ -89,7 +89,7 @@ POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ noinst_PROGRAMS = test_read_nccf$(EXEEXT) -subdir = internal/test_util/libcode/vx_data2d_nccf +subdir = internal/test_util/libcode/vx_data2d_nc_cf ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/configure.ac am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ @@ -329,7 +329,7 @@ MAINTAINERCLEANFILES = Makefile.in test_read_nccf_SOURCES = test_read_nccf.cc test_read_nccf_CPPFLAGS = ${MET_CPPFLAGS} test_read_nccf_LDFLAGS = -L. ${MET_LDFLAGS} -test_read_nccf_LDADD = -lvx_data2d_nccf \ +test_read_nccf_LDADD = -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ -lvx_nc_util \ -lvx_data2d \ @@ -359,9 +359,9 @@ $(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) exit 1;; \ esac; \ done; \ - echo ' cd $(top_srcdir) && $(AUTOMAKE) --foreign internal/test_util/libcode/vx_data2d_nccf/Makefile'; \ + echo ' cd $(top_srcdir) && $(AUTOMAKE) --foreign internal/test_util/libcode/vx_data2d_nc_cf/Makefile'; \ $(am__cd) $(top_srcdir) && \ - $(AUTOMAKE) --foreign internal/test_util/libcode/vx_data2d_nccf/Makefile + $(AUTOMAKE) --foreign internal/test_util/libcode/vx_data2d_nc_cf/Makefile Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status @case '$?' in \ *config.status*) \ diff --git a/internal/test_util/libcode/vx_data2d_nccf/test_read_nccf.cc b/internal/test_util/libcode/vx_data2d_nc_cf/test_read_nccf.cc similarity index 98% rename from internal/test_util/libcode/vx_data2d_nccf/test_read_nccf.cc rename to internal/test_util/libcode/vx_data2d_nc_cf/test_read_nccf.cc index f5d2939b54..7b8b5aa660 100644 --- a/internal/test_util/libcode/vx_data2d_nccf/test_read_nccf.cc +++ b/internal/test_util/libcode/vx_data2d_nc_cf/test_read_nccf.cc @@ -19,7 +19,7 @@ using namespace std; #include "vx_log.h" #include "data_class.h" -#include "data2d_nccf.h" +#include "data2d_nc_cf.h" //////////////////////////////////////////////////////////////////////// diff --git a/internal/test_util/libcode/vx_nc_util/Makefile.am b/internal/test_util/libcode/vx_nc_util/Makefile.am index b691fbf392..9ccc5b3438 100644 --- a/internal/test_util/libcode/vx_nc_util/Makefile.am +++ b/internal/test_util/libcode/vx_nc_util/Makefile.am @@ -31,9 +31,9 @@ test_pressure_levels_LDADD = -lvx_stat_out \ -lvx_statistics \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nc_pinterp \ + -lvx_data2d_nc_wrf \ $(PYTHON_MET_LIBS) $(PYTHON_DEP_LIBS) \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ -lvx_data2d_grib $(GRIB2_MET_LIBS) $(GRIB2_DEP_LIBS) \ -lvx_data2d \ -lvx_nc_util \ diff --git a/internal/test_util/libcode/vx_nc_util/Makefile.in b/internal/test_util/libcode/vx_nc_util/Makefile.in index 826e385c3e..ad865d558f 100644 --- a/internal/test_util/libcode/vx_nc_util/Makefile.in +++ b/internal/test_util/libcode/vx_nc_util/Makefile.in @@ -346,9 +346,9 @@ test_pressure_levels_LDADD = -lvx_stat_out \ -lvx_statistics \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nc_pinterp \ + -lvx_data2d_nc_wrf \ $(PYTHON_MET_LIBS) $(PYTHON_DEP_LIBS) \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ -lvx_data2d_grib $(GRIB2_MET_LIBS) $(GRIB2_DEP_LIBS) \ -lvx_data2d \ -lvx_nc_util \ diff --git a/internal/test_util/libcode/vx_tc_util/Makefile.am b/internal/test_util/libcode/vx_tc_util/Makefile.am index 6c19dad86e..e24d253119 100644 --- a/internal/test_util/libcode/vx_tc_util/Makefile.am +++ b/internal/test_util/libcode/vx_tc_util/Makefile.am @@ -33,9 +33,9 @@ test_read_LDADD = -lvx_stat_out \ -lvx_statistics \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nc_pinterp \ + -lvx_data2d_nc_wrf \ $(PYTHON_MET_LIBS) $(PYTHON_DEP_LIBS) \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ -lvx_data2d_grib $(GRIB2_MET_LIBS) $(GRIB2_DEP_LIBS) \ -lvx_data2d \ -lvx_nc_util \ @@ -69,9 +69,9 @@ test_read_prob_LDADD = -lvx_stat_out \ -lvx_statistics \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nc_pinterp \ + -lvx_data2d_nc_wrf \ $(PYTHON_MET_LIBS) $(PYTHON_DEP_LIBS) \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ -lvx_data2d_grib $(GRIB2_MET_LIBS) $(GRIB2_DEP_LIBS) \ -lvx_data2d \ -lvx_nc_util \ @@ -105,9 +105,9 @@ test_read_rmw_LDADD = -lvx_stat_out \ -lvx_statistics \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nc_pinterp \ + -lvx_data2d_nc_wrf \ $(PYTHON_MET_LIBS) $(PYTHON_DEP_LIBS) \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ -lvx_data2d_grib $(GRIB2_MET_LIBS) $(GRIB2_DEP_LIBS) \ -lvx_data2d \ -lvx_nc_util \ diff --git a/internal/test_util/libcode/vx_tc_util/Makefile.in b/internal/test_util/libcode/vx_tc_util/Makefile.in index ed6e1a58d6..daf8dce673 100644 --- a/internal/test_util/libcode/vx_tc_util/Makefile.in +++ b/internal/test_util/libcode/vx_tc_util/Makefile.in @@ -364,9 +364,9 @@ test_read_LDADD = -lvx_stat_out \ -lvx_statistics \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nc_pinterp \ + -lvx_data2d_nc_wrf \ $(PYTHON_MET_LIBS) $(PYTHON_DEP_LIBS) \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ -lvx_data2d_grib $(GRIB2_MET_LIBS) $(GRIB2_DEP_LIBS) \ -lvx_data2d \ -lvx_nc_util \ @@ -400,9 +400,9 @@ test_read_prob_LDADD = -lvx_stat_out \ -lvx_statistics \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nc_pinterp \ + -lvx_data2d_nc_wrf \ $(PYTHON_MET_LIBS) $(PYTHON_DEP_LIBS) \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ -lvx_data2d_grib $(GRIB2_MET_LIBS) $(GRIB2_DEP_LIBS) \ -lvx_data2d \ -lvx_nc_util \ @@ -436,9 +436,9 @@ test_read_rmw_LDADD = -lvx_stat_out \ -lvx_statistics \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nc_pinterp \ + -lvx_data2d_nc_wrf \ $(PYTHON_MET_LIBS) $(PYTHON_DEP_LIBS) \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ -lvx_data2d_grib $(GRIB2_MET_LIBS) $(GRIB2_DEP_LIBS) \ -lvx_data2d \ -lvx_nc_util \ diff --git a/internal/test_util/tools/other/mode_time_domain/Makefile.am b/internal/test_util/tools/other/mode_time_domain/Makefile.am index eb52695969..5bbe02612e 100644 --- a/internal/test_util/tools/other/mode_time_domain/Makefile.am +++ b/internal/test_util/tools/other/mode_time_domain/Makefile.am @@ -55,8 +55,8 @@ test_velocity_LDADD = \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/internal/test_util/tools/other/mode_time_domain/Makefile.in b/internal/test_util/tools/other/mode_time_domain/Makefile.in index 265f29603c..3e70894414 100644 --- a/internal/test_util/tools/other/mode_time_domain/Makefile.in +++ b/internal/test_util/tools/other/mode_time_domain/Makefile.in @@ -394,8 +394,8 @@ test_velocity_LDADD = \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/basic/vx_config/config_constants.h b/src/basic/vx_config/config_constants.h index 5794a86336..5985e0ba8e 100644 --- a/src/basic/vx_config/config_constants.h +++ b/src/basic/vx_config/config_constants.h @@ -1245,6 +1245,7 @@ static const char conf_key_cira_diag_flag[] = "cira_diag_flag"; static const char conf_val_grib1 [] = "GRIB1"; static const char conf_val_grib2 [] = "GRIB2"; static const char conf_val_netcdf_met [] = "NETCDF_MET"; +static const char conf_val_netcdf_wrf [] = "NETCDF_WRF"; static const char conf_val_netcdf_pint [] = "NETCDF_PINT"; static const char conf_val_netcdf_nccf [] = "NETCDF_NCCF"; static const char conf_val_netcdf_ugrid [] = "NETCDF_UGRID"; diff --git a/src/basic/vx_config/config_util.cc b/src/basic/vx_config/config_util.cc index 78bff4dadc..3d02ba2aed 100644 --- a/src/basic/vx_config/config_util.cc +++ b/src/basic/vx_config/config_util.cc @@ -24,7 +24,6 @@ using namespace std; /////////////////////////////////////////////////////////////////////////////// static const double default_vld_thresh = 1.0; -//static const char conf_key_prepbufr_map_bad[] = "obs_prefbufr_map"; // for backward compatibility /////////////////////////////////////////////////////////////////////////////// @@ -354,6 +353,7 @@ GrdFileType parse_conf_file_type(Dictionary *dict) { if(v == conf_const.lookup_int(conf_val_grib1)) t = FileType_Gb1; else if(v == conf_const.lookup_int(conf_val_grib2)) t = FileType_Gb2; else if(v == conf_const.lookup_int(conf_val_netcdf_met)) t = FileType_NcMet; + else if(v == conf_const.lookup_int(conf_val_netcdf_wrf)) t = FileType_NcWrf; else if(v == conf_const.lookup_int(conf_val_netcdf_pint)) t = FileType_NcPinterp; else if(v == conf_const.lookup_int(conf_val_netcdf_nccf)) t = FileType_NcCF; else if(v == conf_const.lookup_int(conf_val_netcdf_ugrid)) t = FileType_UGrid; diff --git a/src/basic/vx_config/data_file_type.h b/src/basic/vx_config/data_file_type.h index f9f6fa7c24..376685f7a6 100644 --- a/src/basic/vx_config/data_file_type.h +++ b/src/basic/vx_config/data_file_type.h @@ -29,13 +29,13 @@ enum GrdFileType FileType_Gb2, // GRIB version 2 FileType_NcMet, // NetCDF MET format FileType_General_Netcdf, // NetCDF + FileType_NcWrf, // NetCDF output directly from WRF-ARW FileType_NcPinterp, // NetCDF output of WRF-ARW pinterp tool FileType_NcCF, // NetCDF Climate-Forecast Convention FileType_HdfEos, // Hierarchical Data Format - Earth Observing System FileType_Bufr, // Bufr or PrepBufr format - - FileType_Python_Numpy, // python script using numpy array and attributes dictionary - FileType_Python_Xarray, // python script using xarray dataplane + FileType_Python_Numpy, // Python script using numpy array and attributes dictionary + FileType_Python_Xarray, // Python script using xarray dataplane FileType_UGrid, // Unstructured grid }; @@ -44,6 +44,20 @@ enum GrdFileType /////////////////////////////////////////////////////////////////////////////// +inline bool is_netcdf_grdfiletype(const GrdFileType _t) + +{ + +return ( (_t == FileType_NcMet) || (_t == FileType_General_Netcdf) || + (_t == FileType_NcWrf) || (_t == FileType_NcPinterp) || + (_t == FileType_NcCF) ); + +} + + +/////////////////////////////////////////////////////////////////////////////// + + inline bool is_python_grdfiletype(const GrdFileType _t) { diff --git a/src/basic/vx_config/grdfiletype_to_string.cc b/src/basic/vx_config/grdfiletype_to_string.cc index 0b802b97ab..e82450caf8 100644 --- a/src/basic/vx_config/grdfiletype_to_string.cc +++ b/src/basic/vx_config/grdfiletype_to_string.cc @@ -49,12 +49,13 @@ switch ( t ) { case FileType_NcMet: s = "FileType_NcMet"; break; case FileType_General_Netcdf: s = "FileType_General_Netcdf"; break; + case FileType_NcWrf: s = "FileType_NcWrf"; break; case FileType_NcPinterp: s = "FileType_NcPinterp"; break; case FileType_NcCF: s = "FileType_NcCF"; break; case FileType_HdfEos: s = "FileType_HdfEos"; break; case FileType_Bufr: s = "FileType_Bufr"; break; - case FileType_Python_Numpy: s = "FileType_Python_Numpy"; break; + case FileType_Python_Numpy: s = "FileType_Python_Numpy"; break; case FileType_Python_Xarray: s = "FileType_Python_Xarray"; break; case FileType_UGrid: s = "FileType_UGrid"; break; @@ -83,12 +84,13 @@ else if ( strcmp(text, "FileType_Gb2" ) == 0 ) { t = FileType_Gb2; else if ( strcmp(text, "FileType_NcMet" ) == 0 ) { t = FileType_NcMet; return ( true ); } else if ( strcmp(text, "FileType_General_Netcdf") == 0 ) { t = FileType_General_Netcdf; return ( true ); } +else if ( strcmp(text, "FileType_NcWrf" ) == 0 ) { t = FileType_NcWrf; return ( true ); } else if ( strcmp(text, "FileType_NcPinterp" ) == 0 ) { t = FileType_NcPinterp; return ( true ); } else if ( strcmp(text, "FileType_NcCF" ) == 0 ) { t = FileType_NcCF; return ( true ); } else if ( strcmp(text, "FileType_HdfEos" ) == 0 ) { t = FileType_HdfEos; return ( true ); } else if ( strcmp(text, "FileType_Bufr" ) == 0 ) { t = FileType_Bufr; return ( true ); } -else if ( strcmp(text, "FileType_Python_Numpy" ) == 0 ) { t = FileType_Python_Numpy; return ( true ); } +else if ( strcmp(text, "FileType_Python_Numpy" ) == 0 ) { t = FileType_Python_Numpy; return ( true ); } else if ( strcmp(text, "FileType_Python_Xarray" ) == 0 ) { t = FileType_Python_Xarray; return ( true ); } else if ( strcmp(text, "FileType_UGrid" ) == 0 ) { t = FileType_UGrid; return ( true ); } // diff --git a/src/basic/vx_util/data_plane.cc b/src/basic/vx_util/data_plane.cc index 52d2b72f9e..a6d702f0b6 100644 --- a/src/basic/vx_util/data_plane.cc +++ b/src/basic/vx_util/data_plane.cc @@ -298,7 +298,7 @@ void DataPlane::set_size(int nx, int ny, double v) { } // - // delete exisiting data, if necessary + // delete existing data, if necessary // Nx = nx; @@ -800,6 +800,91 @@ return; } +/////////////////////////////////////////////////////////////////////////////// + + +void DataPlane::destagger(bool x_stag, bool y_stag) +{ + // don't do anything if data is not staggered in x or y dimension + if (!x_stag && !y_stag) return; + + const char *method_name = "DataPlane::destagger(bool, bool) -> "; + + if ( Data.empty() ) { + + mlog << Error << "\n\n " << method_name << " data plane is empty!\n\n"; + exit ( 1 ); + + } + + int nx_new = Nx; + int ny_new = Ny; + int nxy_new; + int weight = 0; + int x, y, index_new; + double total; + vector new_data; + + // set nx and weight based on which dimensions are staggered + + if (x_stag) { + + mlog << Debug(3) << "De-staggering dataplane in X dimension\n"; + nx_new = Nx - 1; + weight += 2; + + } + + if (y_stag) { + + mlog << Debug(3) << "De-staggering dataplane in Y dimension\n"; + ny_new = Ny - 1; + weight += 2; + + } + + // allocate vector to store output data + + nxy_new = nx_new * ny_new; + new_data.resize(nxy_new); + + for (y=0; y < ny_new; y++) { + for (x=0; x < nx_new; x++) { + + index_new = y*nx_new + x; + + // always add data from current grid point + total = Data[two_to_one(x, y)]; + + // add data from neighboring grid points based on staggered dimension + + if (x_stag) { + total += Data[two_to_one(x+1,y)]; + } + if (y_stag) { + total += Data[two_to_one(x,y+1)]; + } + + // add diagonal point if staggered in both dimensions (may not occur) + + if (x_stag && y_stag) { + total += Data[two_to_one(x+1,y+1)]; + } + + // divide the sum of the values by the weight to compute the average + + new_data[index_new] = total / weight; + + } + } + + // replace data vector and size variables + + Data = new_data; + Nx = nx_new; + Ny = ny_new; + Nxy = nxy_new; +} /////////////////////////////////////////////////////////////////////////////// diff --git a/src/basic/vx_util/data_plane.h b/src/basic/vx_util/data_plane.h index 8f203096d8..683701ada3 100644 --- a/src/basic/vx_util/data_plane.h +++ b/src/basic/vx_util/data_plane.h @@ -133,7 +133,8 @@ class DataPlane { MaskPlane mask_plane() const; - void shift_right (int n); + void shift_right (int n); + void destagger (bool x_stag, bool y_stag); void put(const double, const int __x__, const int __y__); diff --git a/src/libcode/Makefile.am b/src/libcode/Makefile.am index e20f67551a..ff98f4df20 100644 --- a/src/libcode/Makefile.am +++ b/src/libcode/Makefile.am @@ -21,8 +21,8 @@ SUBDIRS = vx_grid \ vx_nc_util \ vx_data2d_grib \ vx_data2d_nc_met \ - vx_data2d_nccf \ - vx_data2d_nc_pinterp + vx_data2d_nc_wrf \ + vx_data2d_nc_cf if ENABLE_PYTHON SUBDIRS += vx_python3_utils diff --git a/src/libcode/Makefile.in b/src/libcode/Makefile.in index 4af407ca04..5a7e7eb12d 100644 --- a/src/libcode/Makefile.in +++ b/src/libcode/Makefile.in @@ -154,7 +154,7 @@ am__define_uniq_tagged_files = \ if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ done | $(am__uniquify_input)` DIST_SUBDIRS = vx_grid vx_data2d vx_nc_util vx_data2d_grib \ - vx_data2d_nc_met vx_data2d_nccf vx_data2d_nc_pinterp \ + vx_data2d_nc_met vx_data2d_nc_wrf vx_data2d_nc_cf \ vx_python3_utils vx_data2d_python vx_pointdata_python \ vx_data2d_grib2 vx_data2d_ugrid vx_data2d_factory vx_gsl_prob \ vx_gnomon vx_geodesy vx_gis vx_nav vx_pb_util vx_afm vx_color \ @@ -348,7 +348,7 @@ top_build_prefix = @top_build_prefix@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ SUBDIRS = vx_grid vx_data2d vx_nc_util vx_data2d_grib vx_data2d_nc_met \ - vx_data2d_nccf vx_data2d_nc_pinterp $(am__append_1) \ + vx_data2d_nc_wrf vx_data2d_nc_cf $(am__append_1) \ $(am__append_2) $(am__append_3) vx_data2d_factory vx_gsl_prob \ vx_gnomon vx_geodesy vx_gis vx_nav vx_pb_util vx_afm vx_color \ vx_pxm vx_render vx_ps vx_plot_util vx_shapedata vx_statistics \ diff --git a/src/libcode/vx_data2d/vx_data2d.h b/src/libcode/vx_data2d/vx_data2d.h index cffb18e27c..1212aaaf4d 100644 --- a/src/libcode/vx_data2d/vx_data2d.h +++ b/src/libcode/vx_data2d/vx_data2d.h @@ -29,6 +29,7 @@ #include "leveltype_to_string.h" #include "var_info.h" #include "table_lookup.h" +#include "mask_filters.h" //////////////////////////////////////////////////////////////////////// diff --git a/src/libcode/vx_data2d_factory/data2d_factory.cc b/src/libcode/vx_data2d_factory/data2d_factory.cc index 729b43417e..5461892069 100644 --- a/src/libcode/vx_data2d_factory/data2d_factory.cc +++ b/src/libcode/vx_data2d_factory/data2d_factory.cc @@ -21,8 +21,8 @@ using namespace std; #include "grdfiletype_to_string.h" #include "data2d_grib.h" #include "data2d_nc_met.h" -#include "data2d_nc_pinterp.h" -#include "data2d_nccf.h" +#include "data2d_nc_wrf.h" +#include "data2d_nc_cf.h" #ifdef WITH_UGRID #include "data2d_ugrid.h" #endif @@ -81,8 +81,9 @@ MetPythonDataFile * p = nullptr; mtddf = new MetNcMetDataFile; break; + case FileType_NcWrf: case FileType_NcPinterp: - mtddf = new MetNcPinterpDataFile; + mtddf = new MetNcWrfDataFile; break; case FileType_NcCF: diff --git a/src/libcode/vx_data2d_factory/data2d_factory_utils.cc b/src/libcode/vx_data2d_factory/data2d_factory_utils.cc index 5c6d7e991e..6818aa4098 100644 --- a/src/libcode/vx_data2d_factory/data2d_factory_utils.cc +++ b/src/libcode/vx_data2d_factory/data2d_factory_utils.cc @@ -98,6 +98,7 @@ suffix_type = file_type_by_suffix(filename); if ( is_grib1_file (filename) ) data_type = FileType_Gb1; else if ( is_grib2_file (filename) ) data_type = FileType_Gb2; else if ( is_ncpinterp_file (filename) ) data_type = FileType_NcPinterp; +else if ( is_ncwrf_file (filename) ) data_type = FileType_NcWrf; else if ( is_nccf_file (filename) ) data_type = FileType_NcCF; else if ( is_ncmet_file (filename) ) data_type = FileType_NcMet; else if ( is_bufr_file (filename) ) data_type = FileType_Bufr; diff --git a/src/libcode/vx_data2d_factory/is_netcdf_file.cc b/src/libcode/vx_data2d_factory/is_netcdf_file.cc index f6f0856009..d844d76962 100644 --- a/src/libcode/vx_data2d_factory/is_netcdf_file.cc +++ b/src/libcode/vx_data2d_factory/is_netcdf_file.cc @@ -49,8 +49,9 @@ static const string ugrid_att_value = "UGRID"; static const string ncmet_att_version = "MET_version"; static const string ncmet_att_projection = "Projection"; -static const string ncpinterp_att_name = "TITLE"; +static const string nctitle_att_name = "TITLE"; static const char ncpinterp_att_value [] = "ON PRES LEVELS"; +static const char ncwrf_att_value [] = "OUTPUT FROM WRF"; static const string mesh_spec_att_name = "mesh_spec"; @@ -157,9 +158,40 @@ bool is_ncpinterp_file(const char * filename) if (!IS_INVALID_NC_P(nc_file)) { // Get the global attribute - if (get_global_att(nc_file, ncpinterp_att_name, att_val)) { + if (get_global_att(nc_file, nctitle_att_name, att_val)) { // Check the attribute value for the target string - status = (strstr(att_val.c_str(), ncpinterp_att_value)); + status = (strstr(att_val.c_str(), ncwrf_att_value) && + strstr(att_val.c_str(), ncpinterp_att_value)); + } + } + + delete nc_file; + + }catch(...) { + } + + return status; + +} + + +//////////////////////////////////////////////////////////////////////// + + +bool is_ncwrf_file(const char * filename) +{ + bool status = false; + try { + ConcatString att_val; + NcFile *nc_file = open_ncfile(filename); + + if (!IS_INVALID_NC_P(nc_file)) { + // Get the global attribute + if (get_global_att(nc_file, nctitle_att_name, att_val)) { + // Check the attribute value for the target string + // Distinguish between WRF and PInterp output + status = ( strstr(att_val.c_str(), ncwrf_att_value) && + !strstr(att_val.c_str(), ncpinterp_att_value)); } } diff --git a/src/libcode/vx_data2d_factory/is_netcdf_file.h b/src/libcode/vx_data2d_factory/is_netcdf_file.h index 170a5d4c0f..ddfdbbe1b8 100644 --- a/src/libcode/vx_data2d_factory/is_netcdf_file.h +++ b/src/libcode/vx_data2d_factory/is_netcdf_file.h @@ -29,6 +29,8 @@ extern bool is_ncmet_file(const char * filename); extern bool is_ncpinterp_file(const char * filename); +extern bool is_ncwrf_file(const char * filename); + extern bool is_ugrid_file(const char * filename); //////////////////////////////////////////////////////////////////////// diff --git a/src/libcode/vx_data2d_factory/var_info_factory.cc b/src/libcode/vx_data2d_factory/var_info_factory.cc index 3ac92b1dd1..4a308c077b 100644 --- a/src/libcode/vx_data2d_factory/var_info_factory.cc +++ b/src/libcode/vx_data2d_factory/var_info_factory.cc @@ -26,9 +26,9 @@ using namespace std; #include "var_info_factory.h" #include "grdfiletype_to_string.h" #include "var_info_grib.h" -#include "var_info_nccf.h" +#include "var_info_nc_cf.h" #include "var_info_nc_met.h" -#include "var_info_nc_pinterp.h" +#include "var_info_nc_wrf.h" #include "var_info_ugrid.h" #ifdef WITH_PYTHON @@ -85,8 +85,9 @@ VarInfo * VarInfoFactory::new_var_info(GrdFileType type) vi = new VarInfoNcMet; break; + case FileType_NcWrf: case FileType_NcPinterp: - vi = new VarInfoNcPinterp; + vi = new VarInfoNcWrf; break; case FileType_Python_Numpy: diff --git a/src/libcode/vx_data2d_nc_pinterp/.gitignore b/src/libcode/vx_data2d_nc_cf/.gitignore similarity index 100% rename from src/libcode/vx_data2d_nc_pinterp/.gitignore rename to src/libcode/vx_data2d_nc_cf/.gitignore diff --git a/src/libcode/vx_data2d_nc_cf/Makefile.am b/src/libcode/vx_data2d_nc_cf/Makefile.am new file mode 100644 index 0000000000..e74394ea78 --- /dev/null +++ b/src/libcode/vx_data2d_nc_cf/Makefile.am @@ -0,0 +1,20 @@ +## @start 1 +## Makefile.am -- Process this file with automake to produce Makefile.in +## @end 1 + +MAINTAINERCLEANFILES = Makefile.in + +# Include the project definitions + +include ${top_srcdir}/Make-include + +# The library + +noinst_LIBRARIES = libvx_data2d_nc_cf.a +libvx_data2d_nc_cf_a_SOURCES = \ + nc_cf_file.cc nc_cf_file.h \ + var_info_nc_cf.cc var_info_nc_cf.h \ + data2d_nc_cf.cc data2d_nc_cf.h \ + vx_data2d_nc_cf.h +libvx_data2d_nc_cf_a_CPPFLAGS = ${MET_CPPFLAGS} + diff --git a/src/libcode/vx_data2d_nccf/Makefile.in b/src/libcode/vx_data2d_nc_cf/Makefile.in similarity index 74% rename from src/libcode/vx_data2d_nccf/Makefile.in rename to src/libcode/vx_data2d_nc_cf/Makefile.in index 881177942a..cd972c6936 100644 --- a/src/libcode/vx_data2d_nccf/Makefile.in +++ b/src/libcode/vx_data2d_nc_cf/Makefile.in @@ -88,7 +88,7 @@ PRE_UNINSTALL = : POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ -subdir = src/libcode/vx_data2d_nccf +subdir = src/libcode/vx_data2d_nc_cf ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/configure.ac am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ @@ -105,13 +105,13 @@ AM_V_AR = $(am__v_AR_@AM_V@) am__v_AR_ = $(am__v_AR_@AM_DEFAULT_V@) am__v_AR_0 = @echo " AR " $@; am__v_AR_1 = -libvx_data2d_nccf_a_AR = $(AR) $(ARFLAGS) -libvx_data2d_nccf_a_LIBADD = -am_libvx_data2d_nccf_a_OBJECTS = \ - libvx_data2d_nccf_a-nccf_file.$(OBJEXT) \ - libvx_data2d_nccf_a-var_info_nccf.$(OBJEXT) \ - libvx_data2d_nccf_a-data2d_nccf.$(OBJEXT) -libvx_data2d_nccf_a_OBJECTS = $(am_libvx_data2d_nccf_a_OBJECTS) +libvx_data2d_nc_cf_a_AR = $(AR) $(ARFLAGS) +libvx_data2d_nc_cf_a_LIBADD = +am_libvx_data2d_nc_cf_a_OBJECTS = \ + libvx_data2d_nc_cf_a-nc_cf_file.$(OBJEXT) \ + libvx_data2d_nc_cf_a-var_info_nc_cf.$(OBJEXT) \ + libvx_data2d_nc_cf_a-data2d_nc_cf.$(OBJEXT) +libvx_data2d_nc_cf_a_OBJECTS = $(am_libvx_data2d_nc_cf_a_OBJECTS) AM_V_P = $(am__v_P_@AM_V@) am__v_P_ = $(am__v_P_@AM_DEFAULT_V@) am__v_P_0 = false @@ -127,9 +127,10 @@ am__v_at_1 = DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir) depcomp = $(SHELL) $(top_srcdir)/depcomp am__maybe_remake_depfiles = depfiles -am__depfiles_remade = ./$(DEPDIR)/libvx_data2d_nccf_a-data2d_nccf.Po \ - ./$(DEPDIR)/libvx_data2d_nccf_a-nccf_file.Po \ - ./$(DEPDIR)/libvx_data2d_nccf_a-var_info_nccf.Po +am__depfiles_remade = \ + ./$(DEPDIR)/libvx_data2d_nc_cf_a-data2d_nc_cf.Po \ + ./$(DEPDIR)/libvx_data2d_nc_cf_a-nc_cf_file.Po \ + ./$(DEPDIR)/libvx_data2d_nc_cf_a-var_info_nc_cf.Po am__mv = mv -f AM_V_lt = $(am__v_lt_@AM_V@) am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) @@ -160,8 +161,8 @@ AM_V_CCLD = $(am__v_CCLD_@AM_V@) am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@) am__v_CCLD_0 = @echo " CCLD " $@; am__v_CCLD_1 = -SOURCES = $(libvx_data2d_nccf_a_SOURCES) -DIST_SOURCES = $(libvx_data2d_nccf_a_SOURCES) +SOURCES = $(libvx_data2d_nc_cf_a_SOURCES) +DIST_SOURCES = $(libvx_data2d_nc_cf_a_SOURCES) am__can_run_installinfo = \ case $$AM_UPDATE_INFO_DIR in \ n|no|NO) false;; \ @@ -347,14 +348,14 @@ top_srcdir = @top_srcdir@ MAINTAINERCLEANFILES = Makefile.in # The library -noinst_LIBRARIES = libvx_data2d_nccf.a -libvx_data2d_nccf_a_SOURCES = \ - nccf_file.cc nccf_file.h \ - var_info_nccf.cc var_info_nccf.h \ - data2d_nccf.cc data2d_nccf.h \ - vx_data2d_nccf.h - -libvx_data2d_nccf_a_CPPFLAGS = ${MET_CPPFLAGS} +noinst_LIBRARIES = libvx_data2d_nc_cf.a +libvx_data2d_nc_cf_a_SOURCES = \ + nc_cf_file.cc nc_cf_file.h \ + var_info_nc_cf.cc var_info_nc_cf.h \ + data2d_nc_cf.cc data2d_nc_cf.h \ + vx_data2d_nc_cf.h + +libvx_data2d_nc_cf_a_CPPFLAGS = ${MET_CPPFLAGS} all: all-am .SUFFIXES: @@ -368,9 +369,9 @@ $(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) exit 1;; \ esac; \ done; \ - echo ' cd $(top_srcdir) && $(AUTOMAKE) --foreign src/libcode/vx_data2d_nccf/Makefile'; \ + echo ' cd $(top_srcdir) && $(AUTOMAKE) --foreign src/libcode/vx_data2d_nc_cf/Makefile'; \ $(am__cd) $(top_srcdir) && \ - $(AUTOMAKE) --foreign src/libcode/vx_data2d_nccf/Makefile + $(AUTOMAKE) --foreign src/libcode/vx_data2d_nc_cf/Makefile Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status @case '$?' in \ *config.status*) \ @@ -392,10 +393,10 @@ $(am__aclocal_m4_deps): clean-noinstLIBRARIES: -test -z "$(noinst_LIBRARIES)" || rm -f $(noinst_LIBRARIES) -libvx_data2d_nccf.a: $(libvx_data2d_nccf_a_OBJECTS) $(libvx_data2d_nccf_a_DEPENDENCIES) $(EXTRA_libvx_data2d_nccf_a_DEPENDENCIES) - $(AM_V_at)-rm -f libvx_data2d_nccf.a - $(AM_V_AR)$(libvx_data2d_nccf_a_AR) libvx_data2d_nccf.a $(libvx_data2d_nccf_a_OBJECTS) $(libvx_data2d_nccf_a_LIBADD) - $(AM_V_at)$(RANLIB) libvx_data2d_nccf.a +libvx_data2d_nc_cf.a: $(libvx_data2d_nc_cf_a_OBJECTS) $(libvx_data2d_nc_cf_a_DEPENDENCIES) $(EXTRA_libvx_data2d_nc_cf_a_DEPENDENCIES) + $(AM_V_at)-rm -f libvx_data2d_nc_cf.a + $(AM_V_AR)$(libvx_data2d_nc_cf_a_AR) libvx_data2d_nc_cf.a $(libvx_data2d_nc_cf_a_OBJECTS) $(libvx_data2d_nc_cf_a_LIBADD) + $(AM_V_at)$(RANLIB) libvx_data2d_nc_cf.a mostlyclean-compile: -rm -f *.$(OBJEXT) @@ -403,9 +404,9 @@ mostlyclean-compile: distclean-compile: -rm -f *.tab.c -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nccf_a-data2d_nccf.Po@am__quote@ # am--include-marker -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nccf_a-nccf_file.Po@am__quote@ # am--include-marker -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nccf_a-var_info_nccf.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nc_cf_a-data2d_nc_cf.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nc_cf_a-nc_cf_file.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nc_cf_a-var_info_nc_cf.Po@am__quote@ # am--include-marker $(am__depfiles_remade): @$(MKDIR_P) $(@D) @@ -427,47 +428,47 @@ am--depfiles: $(am__depfiles_remade) @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXXCOMPILE) -c -o $@ `$(CYGPATH_W) '$<'` -libvx_data2d_nccf_a-nccf_file.o: nccf_file.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nccf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nccf_a-nccf_file.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nccf_a-nccf_file.Tpo -c -o libvx_data2d_nccf_a-nccf_file.o `test -f 'nccf_file.cc' || echo '$(srcdir)/'`nccf_file.cc -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nccf_a-nccf_file.Tpo $(DEPDIR)/libvx_data2d_nccf_a-nccf_file.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='nccf_file.cc' object='libvx_data2d_nccf_a-nccf_file.o' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_cf_a-nc_cf_file.o: nc_cf_file.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_cf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_cf_a-nc_cf_file.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_cf_a-nc_cf_file.Tpo -c -o libvx_data2d_nc_cf_a-nc_cf_file.o `test -f 'nc_cf_file.cc' || echo '$(srcdir)/'`nc_cf_file.cc +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_cf_a-nc_cf_file.Tpo $(DEPDIR)/libvx_data2d_nc_cf_a-nc_cf_file.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='nc_cf_file.cc' object='libvx_data2d_nc_cf_a-nc_cf_file.o' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nccf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nccf_a-nccf_file.o `test -f 'nccf_file.cc' || echo '$(srcdir)/'`nccf_file.cc +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_cf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_cf_a-nc_cf_file.o `test -f 'nc_cf_file.cc' || echo '$(srcdir)/'`nc_cf_file.cc -libvx_data2d_nccf_a-nccf_file.obj: nccf_file.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nccf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nccf_a-nccf_file.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nccf_a-nccf_file.Tpo -c -o libvx_data2d_nccf_a-nccf_file.obj `if test -f 'nccf_file.cc'; then $(CYGPATH_W) 'nccf_file.cc'; else $(CYGPATH_W) '$(srcdir)/nccf_file.cc'; fi` -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nccf_a-nccf_file.Tpo $(DEPDIR)/libvx_data2d_nccf_a-nccf_file.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='nccf_file.cc' object='libvx_data2d_nccf_a-nccf_file.obj' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_cf_a-nc_cf_file.obj: nc_cf_file.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_cf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_cf_a-nc_cf_file.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_cf_a-nc_cf_file.Tpo -c -o libvx_data2d_nc_cf_a-nc_cf_file.obj `if test -f 'nc_cf_file.cc'; then $(CYGPATH_W) 'nc_cf_file.cc'; else $(CYGPATH_W) '$(srcdir)/nc_cf_file.cc'; fi` +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_cf_a-nc_cf_file.Tpo $(DEPDIR)/libvx_data2d_nc_cf_a-nc_cf_file.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='nc_cf_file.cc' object='libvx_data2d_nc_cf_a-nc_cf_file.obj' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nccf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nccf_a-nccf_file.obj `if test -f 'nccf_file.cc'; then $(CYGPATH_W) 'nccf_file.cc'; else $(CYGPATH_W) '$(srcdir)/nccf_file.cc'; fi` +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_cf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_cf_a-nc_cf_file.obj `if test -f 'nc_cf_file.cc'; then $(CYGPATH_W) 'nc_cf_file.cc'; else $(CYGPATH_W) '$(srcdir)/nc_cf_file.cc'; fi` -libvx_data2d_nccf_a-var_info_nccf.o: var_info_nccf.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nccf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nccf_a-var_info_nccf.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nccf_a-var_info_nccf.Tpo -c -o libvx_data2d_nccf_a-var_info_nccf.o `test -f 'var_info_nccf.cc' || echo '$(srcdir)/'`var_info_nccf.cc -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nccf_a-var_info_nccf.Tpo $(DEPDIR)/libvx_data2d_nccf_a-var_info_nccf.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='var_info_nccf.cc' object='libvx_data2d_nccf_a-var_info_nccf.o' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_cf_a-var_info_nc_cf.o: var_info_nc_cf.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_cf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_cf_a-var_info_nc_cf.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_cf_a-var_info_nc_cf.Tpo -c -o libvx_data2d_nc_cf_a-var_info_nc_cf.o `test -f 'var_info_nc_cf.cc' || echo '$(srcdir)/'`var_info_nc_cf.cc +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_cf_a-var_info_nc_cf.Tpo $(DEPDIR)/libvx_data2d_nc_cf_a-var_info_nc_cf.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='var_info_nc_cf.cc' object='libvx_data2d_nc_cf_a-var_info_nc_cf.o' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nccf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nccf_a-var_info_nccf.o `test -f 'var_info_nccf.cc' || echo '$(srcdir)/'`var_info_nccf.cc +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_cf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_cf_a-var_info_nc_cf.o `test -f 'var_info_nc_cf.cc' || echo '$(srcdir)/'`var_info_nc_cf.cc -libvx_data2d_nccf_a-var_info_nccf.obj: var_info_nccf.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nccf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nccf_a-var_info_nccf.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nccf_a-var_info_nccf.Tpo -c -o libvx_data2d_nccf_a-var_info_nccf.obj `if test -f 'var_info_nccf.cc'; then $(CYGPATH_W) 'var_info_nccf.cc'; else $(CYGPATH_W) '$(srcdir)/var_info_nccf.cc'; fi` -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nccf_a-var_info_nccf.Tpo $(DEPDIR)/libvx_data2d_nccf_a-var_info_nccf.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='var_info_nccf.cc' object='libvx_data2d_nccf_a-var_info_nccf.obj' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_cf_a-var_info_nc_cf.obj: var_info_nc_cf.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_cf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_cf_a-var_info_nc_cf.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_cf_a-var_info_nc_cf.Tpo -c -o libvx_data2d_nc_cf_a-var_info_nc_cf.obj `if test -f 'var_info_nc_cf.cc'; then $(CYGPATH_W) 'var_info_nc_cf.cc'; else $(CYGPATH_W) '$(srcdir)/var_info_nc_cf.cc'; fi` +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_cf_a-var_info_nc_cf.Tpo $(DEPDIR)/libvx_data2d_nc_cf_a-var_info_nc_cf.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='var_info_nc_cf.cc' object='libvx_data2d_nc_cf_a-var_info_nc_cf.obj' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nccf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nccf_a-var_info_nccf.obj `if test -f 'var_info_nccf.cc'; then $(CYGPATH_W) 'var_info_nccf.cc'; else $(CYGPATH_W) '$(srcdir)/var_info_nccf.cc'; fi` +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_cf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_cf_a-var_info_nc_cf.obj `if test -f 'var_info_nc_cf.cc'; then $(CYGPATH_W) 'var_info_nc_cf.cc'; else $(CYGPATH_W) '$(srcdir)/var_info_nc_cf.cc'; fi` -libvx_data2d_nccf_a-data2d_nccf.o: data2d_nccf.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nccf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nccf_a-data2d_nccf.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nccf_a-data2d_nccf.Tpo -c -o libvx_data2d_nccf_a-data2d_nccf.o `test -f 'data2d_nccf.cc' || echo '$(srcdir)/'`data2d_nccf.cc -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nccf_a-data2d_nccf.Tpo $(DEPDIR)/libvx_data2d_nccf_a-data2d_nccf.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='data2d_nccf.cc' object='libvx_data2d_nccf_a-data2d_nccf.o' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_cf_a-data2d_nc_cf.o: data2d_nc_cf.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_cf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_cf_a-data2d_nc_cf.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_cf_a-data2d_nc_cf.Tpo -c -o libvx_data2d_nc_cf_a-data2d_nc_cf.o `test -f 'data2d_nc_cf.cc' || echo '$(srcdir)/'`data2d_nc_cf.cc +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_cf_a-data2d_nc_cf.Tpo $(DEPDIR)/libvx_data2d_nc_cf_a-data2d_nc_cf.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='data2d_nc_cf.cc' object='libvx_data2d_nc_cf_a-data2d_nc_cf.o' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nccf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nccf_a-data2d_nccf.o `test -f 'data2d_nccf.cc' || echo '$(srcdir)/'`data2d_nccf.cc +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_cf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_cf_a-data2d_nc_cf.o `test -f 'data2d_nc_cf.cc' || echo '$(srcdir)/'`data2d_nc_cf.cc -libvx_data2d_nccf_a-data2d_nccf.obj: data2d_nccf.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nccf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nccf_a-data2d_nccf.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nccf_a-data2d_nccf.Tpo -c -o libvx_data2d_nccf_a-data2d_nccf.obj `if test -f 'data2d_nccf.cc'; then $(CYGPATH_W) 'data2d_nccf.cc'; else $(CYGPATH_W) '$(srcdir)/data2d_nccf.cc'; fi` -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nccf_a-data2d_nccf.Tpo $(DEPDIR)/libvx_data2d_nccf_a-data2d_nccf.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='data2d_nccf.cc' object='libvx_data2d_nccf_a-data2d_nccf.obj' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_cf_a-data2d_nc_cf.obj: data2d_nc_cf.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_cf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_cf_a-data2d_nc_cf.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_cf_a-data2d_nc_cf.Tpo -c -o libvx_data2d_nc_cf_a-data2d_nc_cf.obj `if test -f 'data2d_nc_cf.cc'; then $(CYGPATH_W) 'data2d_nc_cf.cc'; else $(CYGPATH_W) '$(srcdir)/data2d_nc_cf.cc'; fi` +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_cf_a-data2d_nc_cf.Tpo $(DEPDIR)/libvx_data2d_nc_cf_a-data2d_nc_cf.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='data2d_nc_cf.cc' object='libvx_data2d_nc_cf_a-data2d_nc_cf.obj' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nccf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nccf_a-data2d_nccf.obj `if test -f 'data2d_nccf.cc'; then $(CYGPATH_W) 'data2d_nccf.cc'; else $(CYGPATH_W) '$(srcdir)/data2d_nccf.cc'; fi` +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_cf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_cf_a-data2d_nc_cf.obj `if test -f 'data2d_nc_cf.cc'; then $(CYGPATH_W) 'data2d_nc_cf.cc'; else $(CYGPATH_W) '$(srcdir)/data2d_nc_cf.cc'; fi` ID: $(am__tagged_files) $(am__define_uniq_tagged_files); mkid -fID $$unique @@ -593,9 +594,9 @@ clean: clean-am clean-am: clean-generic clean-noinstLIBRARIES mostlyclean-am distclean: distclean-am - -rm -f ./$(DEPDIR)/libvx_data2d_nccf_a-data2d_nccf.Po - -rm -f ./$(DEPDIR)/libvx_data2d_nccf_a-nccf_file.Po - -rm -f ./$(DEPDIR)/libvx_data2d_nccf_a-var_info_nccf.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_cf_a-data2d_nc_cf.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_cf_a-nc_cf_file.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_cf_a-var_info_nc_cf.Po -rm -f Makefile distclean-am: clean-am distclean-compile distclean-generic \ distclean-tags @@ -641,9 +642,9 @@ install-ps-am: installcheck-am: maintainer-clean: maintainer-clean-am - -rm -f ./$(DEPDIR)/libvx_data2d_nccf_a-data2d_nccf.Po - -rm -f ./$(DEPDIR)/libvx_data2d_nccf_a-nccf_file.Po - -rm -f ./$(DEPDIR)/libvx_data2d_nccf_a-var_info_nccf.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_cf_a-data2d_nc_cf.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_cf_a-nc_cf_file.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_cf_a-var_info_nc_cf.Po -rm -f Makefile maintainer-clean-am: distclean-am maintainer-clean-generic diff --git a/src/libcode/vx_data2d_nccf/data2d_nccf.cc b/src/libcode/vx_data2d_nc_cf/data2d_nc_cf.cc similarity index 99% rename from src/libcode/vx_data2d_nccf/data2d_nccf.cc rename to src/libcode/vx_data2d_nc_cf/data2d_nc_cf.cc index d25a1062c7..6a7cfaef48 100644 --- a/src/libcode/vx_data2d_nccf/data2d_nccf.cc +++ b/src/libcode/vx_data2d_nc_cf/data2d_nc_cf.cc @@ -17,7 +17,7 @@ #include -#include "data2d_nccf.h" +#include "data2d_nc_cf.h" #include "vx_math.h" #include "vx_log.h" diff --git a/src/libcode/vx_data2d_nccf/data2d_nccf.h b/src/libcode/vx_data2d_nc_cf/data2d_nc_cf.h similarity index 98% rename from src/libcode/vx_data2d_nccf/data2d_nccf.h rename to src/libcode/vx_data2d_nc_cf/data2d_nc_cf.h index f85ae04259..a53631f44b 100644 --- a/src/libcode/vx_data2d_nccf/data2d_nccf.h +++ b/src/libcode/vx_data2d_nc_cf/data2d_nc_cf.h @@ -19,8 +19,8 @@ #include "data_plane.h" #include "data_class.h" -#include "var_info_nccf.h" -#include "nccf_file.h" +#include "var_info_nc_cf.h" +#include "nc_cf_file.h" #include "two_to_one.h" diff --git a/src/libcode/vx_data2d_nccf/nccf_file.cc b/src/libcode/vx_data2d_nc_cf/nc_cf_file.cc similarity index 99% rename from src/libcode/vx_data2d_nccf/nccf_file.cc rename to src/libcode/vx_data2d_nc_cf/nc_cf_file.cc index a1cd259ee0..3de1882c7a 100644 --- a/src/libcode/vx_data2d_nccf/nccf_file.cc +++ b/src/libcode/vx_data2d_nc_cf/nc_cf_file.cc @@ -29,7 +29,7 @@ using namespace netCDF; #include "vx_cal.h" #include "vx_log.h" -#include "nccf_file.h" +#include "nc_cf_file.h" //////////////////////////////////////////////////////////////////////// diff --git a/src/libcode/vx_data2d_nccf/nccf_file.h b/src/libcode/vx_data2d_nc_cf/nc_cf_file.h similarity index 98% rename from src/libcode/vx_data2d_nccf/nccf_file.h rename to src/libcode/vx_data2d_nc_cf/nc_cf_file.h index 02d12c6d5a..3348a09347 100644 --- a/src/libcode/vx_data2d_nccf/nccf_file.h +++ b/src/libcode/vx_data2d_nc_cf/nc_cf_file.h @@ -14,8 +14,8 @@ //////////////////////////////////////////////////////////////////////// -#ifndef __NCCF_FILE_H__ -#define __NCCF_FILE_H__ +#ifndef __NC_CF_FILE_H__ +#define __NC_CF_FILE_H__ //////////////////////////////////////////////////////////////////////// @@ -194,7 +194,7 @@ class NcCfFile { //////////////////////////////////////////////////////////////////////// -#endif /* __NCCF_FILE_H__ */ +#endif /* __NC_CF_FILE_H__ */ //////////////////////////////////////////////////////////////////////// diff --git a/src/libcode/vx_data2d_nccf/var_info_nccf.cc b/src/libcode/vx_data2d_nc_cf/var_info_nc_cf.cc similarity index 99% rename from src/libcode/vx_data2d_nccf/var_info_nccf.cc rename to src/libcode/vx_data2d_nc_cf/var_info_nc_cf.cc index a77ece6881..3144304014 100644 --- a/src/libcode/vx_data2d_nccf/var_info_nccf.cc +++ b/src/libcode/vx_data2d_nc_cf/var_info_nc_cf.cc @@ -8,7 +8,7 @@ /////////////////////////////////////////////////////////////////////////////// // -// Filename: var_info_nccf.cc +// Filename: var_info_nc_cf.cc // // Description: // @@ -25,7 +25,7 @@ using namespace std; #include #include "var_info.h" -#include "var_info_nccf.h" +#include "var_info_nc_cf.h" #include "vx_math.h" #include "util_constants.h" diff --git a/src/libcode/vx_data2d_nccf/var_info_nccf.h b/src/libcode/vx_data2d_nc_cf/var_info_nc_cf.h similarity index 100% rename from src/libcode/vx_data2d_nccf/var_info_nccf.h rename to src/libcode/vx_data2d_nc_cf/var_info_nc_cf.h diff --git a/src/libcode/vx_data2d_nccf/vx_data2d_nccf.h b/src/libcode/vx_data2d_nc_cf/vx_data2d_nc_cf.h similarity index 90% rename from src/libcode/vx_data2d_nccf/vx_data2d_nccf.h rename to src/libcode/vx_data2d_nc_cf/vx_data2d_nc_cf.h index f16349e38f..de9e7e8051 100644 --- a/src/libcode/vx_data2d_nccf/vx_data2d_nccf.h +++ b/src/libcode/vx_data2d_nc_cf/vx_data2d_nc_cf.h @@ -21,9 +21,9 @@ //////////////////////////////////////////////////////////////////////// -#include "data2d_nccf.h" -#include "var_info_nccf.h" -#include "nccf_file.h" +#include "data2d_nc_cf.h" +#include "var_info_nc_cf.h" +#include "nc_cf_file.h" //////////////////////////////////////////////////////////////////////// diff --git a/src/libcode/vx_data2d_nc_pinterp/Makefile.am b/src/libcode/vx_data2d_nc_pinterp/Makefile.am deleted file mode 100644 index 7d89511162..0000000000 --- a/src/libcode/vx_data2d_nc_pinterp/Makefile.am +++ /dev/null @@ -1,20 +0,0 @@ -## @start 1 -## Makefile.am -- Process this file with automake to produce Makefile.in -## @end 1 - -MAINTAINERCLEANFILES = Makefile.in - -# Include the project definitions - -include ${top_srcdir}/Make-include - -# The library - -noinst_LIBRARIES = libvx_data2d_nc_pinterp.a -libvx_data2d_nc_pinterp_a_SOURCES = \ - get_pinterp_grid.cc get_pinterp_grid.h \ - pinterp_file.cc pinterp_file.h \ - var_info_nc_pinterp.cc var_info_nc_pinterp.h \ - data2d_nc_pinterp.cc data2d_nc_pinterp.h \ - vx_data2d_nc_pinterp.h -libvx_data2d_nc_pinterp_a_CPPFLAGS = ${MET_CPPFLAGS} diff --git a/src/libcode/vx_data2d_nccf/.gitignore b/src/libcode/vx_data2d_nc_wrf/.gitignore similarity index 100% rename from src/libcode/vx_data2d_nccf/.gitignore rename to src/libcode/vx_data2d_nc_wrf/.gitignore diff --git a/src/libcode/vx_data2d_nc_wrf/Makefile.am b/src/libcode/vx_data2d_nc_wrf/Makefile.am new file mode 100644 index 0000000000..78c1785755 --- /dev/null +++ b/src/libcode/vx_data2d_nc_wrf/Makefile.am @@ -0,0 +1,20 @@ +## @start 1 +## Makefile.am -- Process this file with automake to produce Makefile.in +## @end 1 + +MAINTAINERCLEANFILES = Makefile.in + +# Include the project definitions + +include ${top_srcdir}/Make-include + +# The library + +noinst_LIBRARIES = libvx_data2d_nc_wrf.a +libvx_data2d_nc_wrf_a_SOURCES = \ + get_wrf_grid.cc get_wrf_grid.h \ + wrf_file.cc wrf_file.h \ + var_info_nc_wrf.cc var_info_nc_wrf.h \ + data2d_nc_wrf.cc data2d_nc_wrf.h \ + vx_data2d_nc_wrf.h +libvx_data2d_nc_wrf_a_CPPFLAGS = ${MET_CPPFLAGS} diff --git a/src/libcode/vx_data2d_nc_pinterp/Makefile.in b/src/libcode/vx_data2d_nc_wrf/Makefile.in similarity index 68% rename from src/libcode/vx_data2d_nc_pinterp/Makefile.in rename to src/libcode/vx_data2d_nc_wrf/Makefile.in index 867c42b48a..dbe34976c3 100644 --- a/src/libcode/vx_data2d_nc_pinterp/Makefile.in +++ b/src/libcode/vx_data2d_nc_wrf/Makefile.in @@ -88,7 +88,7 @@ PRE_UNINSTALL = : POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ -subdir = src/libcode/vx_data2d_nc_pinterp +subdir = src/libcode/vx_data2d_nc_wrf ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/configure.ac am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ @@ -105,15 +105,14 @@ AM_V_AR = $(am__v_AR_@AM_V@) am__v_AR_ = $(am__v_AR_@AM_DEFAULT_V@) am__v_AR_0 = @echo " AR " $@; am__v_AR_1 = -libvx_data2d_nc_pinterp_a_AR = $(AR) $(ARFLAGS) -libvx_data2d_nc_pinterp_a_LIBADD = -am_libvx_data2d_nc_pinterp_a_OBJECTS = \ - libvx_data2d_nc_pinterp_a-get_pinterp_grid.$(OBJEXT) \ - libvx_data2d_nc_pinterp_a-pinterp_file.$(OBJEXT) \ - libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.$(OBJEXT) \ - libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.$(OBJEXT) -libvx_data2d_nc_pinterp_a_OBJECTS = \ - $(am_libvx_data2d_nc_pinterp_a_OBJECTS) +libvx_data2d_nc_wrf_a_AR = $(AR) $(ARFLAGS) +libvx_data2d_nc_wrf_a_LIBADD = +am_libvx_data2d_nc_wrf_a_OBJECTS = \ + libvx_data2d_nc_wrf_a-get_wrf_grid.$(OBJEXT) \ + libvx_data2d_nc_wrf_a-wrf_file.$(OBJEXT) \ + libvx_data2d_nc_wrf_a-var_info_nc_wrf.$(OBJEXT) \ + libvx_data2d_nc_wrf_a-data2d_nc_wrf.$(OBJEXT) +libvx_data2d_nc_wrf_a_OBJECTS = $(am_libvx_data2d_nc_wrf_a_OBJECTS) AM_V_P = $(am__v_P_@AM_V@) am__v_P_ = $(am__v_P_@AM_DEFAULT_V@) am__v_P_0 = false @@ -130,10 +129,10 @@ DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir) depcomp = $(SHELL) $(top_srcdir)/depcomp am__maybe_remake_depfiles = depfiles am__depfiles_remade = \ - ./$(DEPDIR)/libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.Po \ - ./$(DEPDIR)/libvx_data2d_nc_pinterp_a-get_pinterp_grid.Po \ - ./$(DEPDIR)/libvx_data2d_nc_pinterp_a-pinterp_file.Po \ - ./$(DEPDIR)/libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.Po + ./$(DEPDIR)/libvx_data2d_nc_wrf_a-data2d_nc_wrf.Po \ + ./$(DEPDIR)/libvx_data2d_nc_wrf_a-get_wrf_grid.Po \ + ./$(DEPDIR)/libvx_data2d_nc_wrf_a-var_info_nc_wrf.Po \ + ./$(DEPDIR)/libvx_data2d_nc_wrf_a-wrf_file.Po am__mv = mv -f AM_V_lt = $(am__v_lt_@AM_V@) am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) @@ -164,8 +163,8 @@ AM_V_CCLD = $(am__v_CCLD_@AM_V@) am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@) am__v_CCLD_0 = @echo " CCLD " $@; am__v_CCLD_1 = -SOURCES = $(libvx_data2d_nc_pinterp_a_SOURCES) -DIST_SOURCES = $(libvx_data2d_nc_pinterp_a_SOURCES) +SOURCES = $(libvx_data2d_nc_wrf_a_SOURCES) +DIST_SOURCES = $(libvx_data2d_nc_wrf_a_SOURCES) am__can_run_installinfo = \ case $$AM_UPDATE_INFO_DIR in \ n|no|NO) false;; \ @@ -351,15 +350,15 @@ top_srcdir = @top_srcdir@ MAINTAINERCLEANFILES = Makefile.in # The library -noinst_LIBRARIES = libvx_data2d_nc_pinterp.a -libvx_data2d_nc_pinterp_a_SOURCES = \ - get_pinterp_grid.cc get_pinterp_grid.h \ - pinterp_file.cc pinterp_file.h \ - var_info_nc_pinterp.cc var_info_nc_pinterp.h \ - data2d_nc_pinterp.cc data2d_nc_pinterp.h \ - vx_data2d_nc_pinterp.h - -libvx_data2d_nc_pinterp_a_CPPFLAGS = ${MET_CPPFLAGS} +noinst_LIBRARIES = libvx_data2d_nc_wrf.a +libvx_data2d_nc_wrf_a_SOURCES = \ + get_wrf_grid.cc get_wrf_grid.h \ + wrf_file.cc wrf_file.h \ + var_info_nc_wrf.cc var_info_nc_wrf.h \ + data2d_nc_wrf.cc data2d_nc_wrf.h \ + vx_data2d_nc_wrf.h + +libvx_data2d_nc_wrf_a_CPPFLAGS = ${MET_CPPFLAGS} all: all-am .SUFFIXES: @@ -373,9 +372,9 @@ $(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) exit 1;; \ esac; \ done; \ - echo ' cd $(top_srcdir) && $(AUTOMAKE) --foreign src/libcode/vx_data2d_nc_pinterp/Makefile'; \ + echo ' cd $(top_srcdir) && $(AUTOMAKE) --foreign src/libcode/vx_data2d_nc_wrf/Makefile'; \ $(am__cd) $(top_srcdir) && \ - $(AUTOMAKE) --foreign src/libcode/vx_data2d_nc_pinterp/Makefile + $(AUTOMAKE) --foreign src/libcode/vx_data2d_nc_wrf/Makefile Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status @case '$?' in \ *config.status*) \ @@ -397,10 +396,10 @@ $(am__aclocal_m4_deps): clean-noinstLIBRARIES: -test -z "$(noinst_LIBRARIES)" || rm -f $(noinst_LIBRARIES) -libvx_data2d_nc_pinterp.a: $(libvx_data2d_nc_pinterp_a_OBJECTS) $(libvx_data2d_nc_pinterp_a_DEPENDENCIES) $(EXTRA_libvx_data2d_nc_pinterp_a_DEPENDENCIES) - $(AM_V_at)-rm -f libvx_data2d_nc_pinterp.a - $(AM_V_AR)$(libvx_data2d_nc_pinterp_a_AR) libvx_data2d_nc_pinterp.a $(libvx_data2d_nc_pinterp_a_OBJECTS) $(libvx_data2d_nc_pinterp_a_LIBADD) - $(AM_V_at)$(RANLIB) libvx_data2d_nc_pinterp.a +libvx_data2d_nc_wrf.a: $(libvx_data2d_nc_wrf_a_OBJECTS) $(libvx_data2d_nc_wrf_a_DEPENDENCIES) $(EXTRA_libvx_data2d_nc_wrf_a_DEPENDENCIES) + $(AM_V_at)-rm -f libvx_data2d_nc_wrf.a + $(AM_V_AR)$(libvx_data2d_nc_wrf_a_AR) libvx_data2d_nc_wrf.a $(libvx_data2d_nc_wrf_a_OBJECTS) $(libvx_data2d_nc_wrf_a_LIBADD) + $(AM_V_at)$(RANLIB) libvx_data2d_nc_wrf.a mostlyclean-compile: -rm -f *.$(OBJEXT) @@ -408,10 +407,10 @@ mostlyclean-compile: distclean-compile: -rm -f *.tab.c -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.Po@am__quote@ # am--include-marker -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nc_pinterp_a-get_pinterp_grid.Po@am__quote@ # am--include-marker -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nc_pinterp_a-pinterp_file.Po@am__quote@ # am--include-marker -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nc_wrf_a-data2d_nc_wrf.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nc_wrf_a-get_wrf_grid.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nc_wrf_a-var_info_nc_wrf.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libvx_data2d_nc_wrf_a-wrf_file.Po@am__quote@ # am--include-marker $(am__depfiles_remade): @$(MKDIR_P) $(@D) @@ -433,61 +432,61 @@ am--depfiles: $(am__depfiles_remade) @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXXCOMPILE) -c -o $@ `$(CYGPATH_W) '$<'` -libvx_data2d_nc_pinterp_a-get_pinterp_grid.o: get_pinterp_grid.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_pinterp_a-get_pinterp_grid.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_pinterp_a-get_pinterp_grid.Tpo -c -o libvx_data2d_nc_pinterp_a-get_pinterp_grid.o `test -f 'get_pinterp_grid.cc' || echo '$(srcdir)/'`get_pinterp_grid.cc -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_pinterp_a-get_pinterp_grid.Tpo $(DEPDIR)/libvx_data2d_nc_pinterp_a-get_pinterp_grid.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='get_pinterp_grid.cc' object='libvx_data2d_nc_pinterp_a-get_pinterp_grid.o' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_wrf_a-get_wrf_grid.o: get_wrf_grid.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_wrf_a-get_wrf_grid.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_wrf_a-get_wrf_grid.Tpo -c -o libvx_data2d_nc_wrf_a-get_wrf_grid.o `test -f 'get_wrf_grid.cc' || echo '$(srcdir)/'`get_wrf_grid.cc +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_wrf_a-get_wrf_grid.Tpo $(DEPDIR)/libvx_data2d_nc_wrf_a-get_wrf_grid.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='get_wrf_grid.cc' object='libvx_data2d_nc_wrf_a-get_wrf_grid.o' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_pinterp_a-get_pinterp_grid.o `test -f 'get_pinterp_grid.cc' || echo '$(srcdir)/'`get_pinterp_grid.cc +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_wrf_a-get_wrf_grid.o `test -f 'get_wrf_grid.cc' || echo '$(srcdir)/'`get_wrf_grid.cc -libvx_data2d_nc_pinterp_a-get_pinterp_grid.obj: get_pinterp_grid.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_pinterp_a-get_pinterp_grid.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_pinterp_a-get_pinterp_grid.Tpo -c -o libvx_data2d_nc_pinterp_a-get_pinterp_grid.obj `if test -f 'get_pinterp_grid.cc'; then $(CYGPATH_W) 'get_pinterp_grid.cc'; else $(CYGPATH_W) '$(srcdir)/get_pinterp_grid.cc'; fi` -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_pinterp_a-get_pinterp_grid.Tpo $(DEPDIR)/libvx_data2d_nc_pinterp_a-get_pinterp_grid.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='get_pinterp_grid.cc' object='libvx_data2d_nc_pinterp_a-get_pinterp_grid.obj' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_wrf_a-get_wrf_grid.obj: get_wrf_grid.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_wrf_a-get_wrf_grid.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_wrf_a-get_wrf_grid.Tpo -c -o libvx_data2d_nc_wrf_a-get_wrf_grid.obj `if test -f 'get_wrf_grid.cc'; then $(CYGPATH_W) 'get_wrf_grid.cc'; else $(CYGPATH_W) '$(srcdir)/get_wrf_grid.cc'; fi` +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_wrf_a-get_wrf_grid.Tpo $(DEPDIR)/libvx_data2d_nc_wrf_a-get_wrf_grid.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='get_wrf_grid.cc' object='libvx_data2d_nc_wrf_a-get_wrf_grid.obj' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_pinterp_a-get_pinterp_grid.obj `if test -f 'get_pinterp_grid.cc'; then $(CYGPATH_W) 'get_pinterp_grid.cc'; else $(CYGPATH_W) '$(srcdir)/get_pinterp_grid.cc'; fi` +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_wrf_a-get_wrf_grid.obj `if test -f 'get_wrf_grid.cc'; then $(CYGPATH_W) 'get_wrf_grid.cc'; else $(CYGPATH_W) '$(srcdir)/get_wrf_grid.cc'; fi` -libvx_data2d_nc_pinterp_a-pinterp_file.o: pinterp_file.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_pinterp_a-pinterp_file.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_pinterp_a-pinterp_file.Tpo -c -o libvx_data2d_nc_pinterp_a-pinterp_file.o `test -f 'pinterp_file.cc' || echo '$(srcdir)/'`pinterp_file.cc -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_pinterp_a-pinterp_file.Tpo $(DEPDIR)/libvx_data2d_nc_pinterp_a-pinterp_file.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='pinterp_file.cc' object='libvx_data2d_nc_pinterp_a-pinterp_file.o' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_wrf_a-wrf_file.o: wrf_file.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_wrf_a-wrf_file.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_wrf_a-wrf_file.Tpo -c -o libvx_data2d_nc_wrf_a-wrf_file.o `test -f 'wrf_file.cc' || echo '$(srcdir)/'`wrf_file.cc +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_wrf_a-wrf_file.Tpo $(DEPDIR)/libvx_data2d_nc_wrf_a-wrf_file.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='wrf_file.cc' object='libvx_data2d_nc_wrf_a-wrf_file.o' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_pinterp_a-pinterp_file.o `test -f 'pinterp_file.cc' || echo '$(srcdir)/'`pinterp_file.cc +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_wrf_a-wrf_file.o `test -f 'wrf_file.cc' || echo '$(srcdir)/'`wrf_file.cc -libvx_data2d_nc_pinterp_a-pinterp_file.obj: pinterp_file.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_pinterp_a-pinterp_file.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_pinterp_a-pinterp_file.Tpo -c -o libvx_data2d_nc_pinterp_a-pinterp_file.obj `if test -f 'pinterp_file.cc'; then $(CYGPATH_W) 'pinterp_file.cc'; else $(CYGPATH_W) '$(srcdir)/pinterp_file.cc'; fi` -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_pinterp_a-pinterp_file.Tpo $(DEPDIR)/libvx_data2d_nc_pinterp_a-pinterp_file.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='pinterp_file.cc' object='libvx_data2d_nc_pinterp_a-pinterp_file.obj' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_wrf_a-wrf_file.obj: wrf_file.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_wrf_a-wrf_file.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_wrf_a-wrf_file.Tpo -c -o libvx_data2d_nc_wrf_a-wrf_file.obj `if test -f 'wrf_file.cc'; then $(CYGPATH_W) 'wrf_file.cc'; else $(CYGPATH_W) '$(srcdir)/wrf_file.cc'; fi` +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_wrf_a-wrf_file.Tpo $(DEPDIR)/libvx_data2d_nc_wrf_a-wrf_file.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='wrf_file.cc' object='libvx_data2d_nc_wrf_a-wrf_file.obj' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_pinterp_a-pinterp_file.obj `if test -f 'pinterp_file.cc'; then $(CYGPATH_W) 'pinterp_file.cc'; else $(CYGPATH_W) '$(srcdir)/pinterp_file.cc'; fi` +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_wrf_a-wrf_file.obj `if test -f 'wrf_file.cc'; then $(CYGPATH_W) 'wrf_file.cc'; else $(CYGPATH_W) '$(srcdir)/wrf_file.cc'; fi` -libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.o: var_info_nc_pinterp.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.Tpo -c -o libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.o `test -f 'var_info_nc_pinterp.cc' || echo '$(srcdir)/'`var_info_nc_pinterp.cc -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.Tpo $(DEPDIR)/libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='var_info_nc_pinterp.cc' object='libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.o' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_wrf_a-var_info_nc_wrf.o: var_info_nc_wrf.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_wrf_a-var_info_nc_wrf.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_wrf_a-var_info_nc_wrf.Tpo -c -o libvx_data2d_nc_wrf_a-var_info_nc_wrf.o `test -f 'var_info_nc_wrf.cc' || echo '$(srcdir)/'`var_info_nc_wrf.cc +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_wrf_a-var_info_nc_wrf.Tpo $(DEPDIR)/libvx_data2d_nc_wrf_a-var_info_nc_wrf.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='var_info_nc_wrf.cc' object='libvx_data2d_nc_wrf_a-var_info_nc_wrf.o' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.o `test -f 'var_info_nc_pinterp.cc' || echo '$(srcdir)/'`var_info_nc_pinterp.cc +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_wrf_a-var_info_nc_wrf.o `test -f 'var_info_nc_wrf.cc' || echo '$(srcdir)/'`var_info_nc_wrf.cc -libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.obj: var_info_nc_pinterp.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.Tpo -c -o libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.obj `if test -f 'var_info_nc_pinterp.cc'; then $(CYGPATH_W) 'var_info_nc_pinterp.cc'; else $(CYGPATH_W) '$(srcdir)/var_info_nc_pinterp.cc'; fi` -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.Tpo $(DEPDIR)/libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='var_info_nc_pinterp.cc' object='libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.obj' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_wrf_a-var_info_nc_wrf.obj: var_info_nc_wrf.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_wrf_a-var_info_nc_wrf.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_wrf_a-var_info_nc_wrf.Tpo -c -o libvx_data2d_nc_wrf_a-var_info_nc_wrf.obj `if test -f 'var_info_nc_wrf.cc'; then $(CYGPATH_W) 'var_info_nc_wrf.cc'; else $(CYGPATH_W) '$(srcdir)/var_info_nc_wrf.cc'; fi` +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_wrf_a-var_info_nc_wrf.Tpo $(DEPDIR)/libvx_data2d_nc_wrf_a-var_info_nc_wrf.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='var_info_nc_wrf.cc' object='libvx_data2d_nc_wrf_a-var_info_nc_wrf.obj' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.obj `if test -f 'var_info_nc_pinterp.cc'; then $(CYGPATH_W) 'var_info_nc_pinterp.cc'; else $(CYGPATH_W) '$(srcdir)/var_info_nc_pinterp.cc'; fi` +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_wrf_a-var_info_nc_wrf.obj `if test -f 'var_info_nc_wrf.cc'; then $(CYGPATH_W) 'var_info_nc_wrf.cc'; else $(CYGPATH_W) '$(srcdir)/var_info_nc_wrf.cc'; fi` -libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.o: data2d_nc_pinterp.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.Tpo -c -o libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.o `test -f 'data2d_nc_pinterp.cc' || echo '$(srcdir)/'`data2d_nc_pinterp.cc -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.Tpo $(DEPDIR)/libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='data2d_nc_pinterp.cc' object='libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.o' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_wrf_a-data2d_nc_wrf.o: data2d_nc_wrf.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_wrf_a-data2d_nc_wrf.o -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_wrf_a-data2d_nc_wrf.Tpo -c -o libvx_data2d_nc_wrf_a-data2d_nc_wrf.o `test -f 'data2d_nc_wrf.cc' || echo '$(srcdir)/'`data2d_nc_wrf.cc +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_wrf_a-data2d_nc_wrf.Tpo $(DEPDIR)/libvx_data2d_nc_wrf_a-data2d_nc_wrf.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='data2d_nc_wrf.cc' object='libvx_data2d_nc_wrf_a-data2d_nc_wrf.o' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.o `test -f 'data2d_nc_pinterp.cc' || echo '$(srcdir)/'`data2d_nc_pinterp.cc +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_wrf_a-data2d_nc_wrf.o `test -f 'data2d_nc_wrf.cc' || echo '$(srcdir)/'`data2d_nc_wrf.cc -libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.obj: data2d_nc_pinterp.cc -@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.Tpo -c -o libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.obj `if test -f 'data2d_nc_pinterp.cc'; then $(CYGPATH_W) 'data2d_nc_pinterp.cc'; else $(CYGPATH_W) '$(srcdir)/data2d_nc_pinterp.cc'; fi` -@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.Tpo $(DEPDIR)/libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.Po -@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='data2d_nc_pinterp.cc' object='libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.obj' libtool=no @AMDEPBACKSLASH@ +libvx_data2d_nc_wrf_a-data2d_nc_wrf.obj: data2d_nc_wrf.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT libvx_data2d_nc_wrf_a-data2d_nc_wrf.obj -MD -MP -MF $(DEPDIR)/libvx_data2d_nc_wrf_a-data2d_nc_wrf.Tpo -c -o libvx_data2d_nc_wrf_a-data2d_nc_wrf.obj `if test -f 'data2d_nc_wrf.cc'; then $(CYGPATH_W) 'data2d_nc_wrf.cc'; else $(CYGPATH_W) '$(srcdir)/data2d_nc_wrf.cc'; fi` +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/libvx_data2d_nc_wrf_a-data2d_nc_wrf.Tpo $(DEPDIR)/libvx_data2d_nc_wrf_a-data2d_nc_wrf.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='data2d_nc_wrf.cc' object='libvx_data2d_nc_wrf_a-data2d_nc_wrf.obj' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_pinterp_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.obj `if test -f 'data2d_nc_pinterp.cc'; then $(CYGPATH_W) 'data2d_nc_pinterp.cc'; else $(CYGPATH_W) '$(srcdir)/data2d_nc_pinterp.cc'; fi` +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libvx_data2d_nc_wrf_a_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o libvx_data2d_nc_wrf_a-data2d_nc_wrf.obj `if test -f 'data2d_nc_wrf.cc'; then $(CYGPATH_W) 'data2d_nc_wrf.cc'; else $(CYGPATH_W) '$(srcdir)/data2d_nc_wrf.cc'; fi` ID: $(am__tagged_files) $(am__define_uniq_tagged_files); mkid -fID $$unique @@ -613,10 +612,10 @@ clean: clean-am clean-am: clean-generic clean-noinstLIBRARIES mostlyclean-am distclean: distclean-am - -rm -f ./$(DEPDIR)/libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.Po - -rm -f ./$(DEPDIR)/libvx_data2d_nc_pinterp_a-get_pinterp_grid.Po - -rm -f ./$(DEPDIR)/libvx_data2d_nc_pinterp_a-pinterp_file.Po - -rm -f ./$(DEPDIR)/libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_wrf_a-data2d_nc_wrf.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_wrf_a-get_wrf_grid.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_wrf_a-var_info_nc_wrf.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_wrf_a-wrf_file.Po -rm -f Makefile distclean-am: clean-am distclean-compile distclean-generic \ distclean-tags @@ -662,10 +661,10 @@ install-ps-am: installcheck-am: maintainer-clean: maintainer-clean-am - -rm -f ./$(DEPDIR)/libvx_data2d_nc_pinterp_a-data2d_nc_pinterp.Po - -rm -f ./$(DEPDIR)/libvx_data2d_nc_pinterp_a-get_pinterp_grid.Po - -rm -f ./$(DEPDIR)/libvx_data2d_nc_pinterp_a-pinterp_file.Po - -rm -f ./$(DEPDIR)/libvx_data2d_nc_pinterp_a-var_info_nc_pinterp.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_wrf_a-data2d_nc_wrf.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_wrf_a-get_wrf_grid.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_wrf_a-var_info_nc_wrf.Po + -rm -f ./$(DEPDIR)/libvx_data2d_nc_wrf_a-wrf_file.Po -rm -f Makefile maintainer-clean-am: distclean-am maintainer-clean-generic diff --git a/src/libcode/vx_data2d_nc_pinterp/data2d_nc_pinterp.cc b/src/libcode/vx_data2d_nc_wrf/data2d_nc_wrf.cc similarity index 78% rename from src/libcode/vx_data2d_nc_pinterp/data2d_nc_pinterp.cc rename to src/libcode/vx_data2d_nc_wrf/data2d_nc_wrf.cc index 83872f4614..ef51698ed1 100644 --- a/src/libcode/vx_data2d_nc_pinterp/data2d_nc_pinterp.cc +++ b/src/libcode/vx_data2d_nc_wrf/data2d_nc_wrf.cc @@ -17,54 +17,54 @@ using namespace std; #include #include -#include "data2d_nc_pinterp.h" -#include "get_pinterp_grid.h" +#include "data2d_nc_wrf.h" +#include "get_wrf_grid.h" #include "vx_nc_util.h" #include "vx_math.h" #include "vx_log.h" //////////////////////////////////////////////////////////////////////// // -// Code for class MetNcPinterpDataFile +// Code for class MetNcWrfDataFile // //////////////////////////////////////////////////////////////////////// -MetNcPinterpDataFile::MetNcPinterpDataFile() { +MetNcWrfDataFile::MetNcWrfDataFile() { - nc_pinterp_init_from_scratch(); + nc_wrf_init_from_scratch(); } //////////////////////////////////////////////////////////////////////// -MetNcPinterpDataFile::~MetNcPinterpDataFile() { +MetNcWrfDataFile::~MetNcWrfDataFile() { close(); } //////////////////////////////////////////////////////////////////////// -MetNcPinterpDataFile::MetNcPinterpDataFile(const MetNcPinterpDataFile &) { +MetNcWrfDataFile::MetNcWrfDataFile(const MetNcWrfDataFile &) { - mlog << Error << "\nMetNcPinterpDataFile::MetNcPinterpDataFile(const MetNcPinterpDataFile &) -> " + mlog << Error << "\nMetNcWrfDataFile::MetNcWrfDataFile(const MetNcWrfDataFile &) -> " << "should never be called!\n\n"; exit(1); } //////////////////////////////////////////////////////////////////////// -MetNcPinterpDataFile & MetNcPinterpDataFile::operator=(const MetNcPinterpDataFile &) { +MetNcWrfDataFile & MetNcWrfDataFile::operator=(const MetNcWrfDataFile &) { - mlog << Error << "\nMetNcPinterpDataFile::operator=(const MetNcPinterpDataFile &) -> " + mlog << Error << "\nMetNcWrfDataFile::operator=(const MetNcWrfDataFile &) -> " << "should never be called!\n\n"; exit(1); } //////////////////////////////////////////////////////////////////////// -void MetNcPinterpDataFile::nc_pinterp_init_from_scratch() { +void MetNcWrfDataFile::nc_wrf_init_from_scratch() { - PinterpNc = (PinterpFile *) 0; + WrfNc = (WrfFile *) 0; close(); @@ -73,23 +73,23 @@ void MetNcPinterpDataFile::nc_pinterp_init_from_scratch() { //////////////////////////////////////////////////////////////////////// -void MetNcPinterpDataFile::close() { +void MetNcWrfDataFile::close() { - if(PinterpNc) { delete PinterpNc; PinterpNc = (PinterpFile *) 0; } + if(WrfNc) { delete WrfNc; WrfNc = (WrfFile *) 0; } return; } //////////////////////////////////////////////////////////////////////// -bool MetNcPinterpDataFile::open(const char * _filename) { +bool MetNcWrfDataFile::open(const char * _filename) { close(); - PinterpNc = new PinterpFile; + WrfNc = new WrfFile; - if(!PinterpNc->open(_filename)) { - mlog << Error << "\nMetNcPinterpDataFile::open(const char *) -> " + if(!WrfNc->open(_filename)) { + mlog << Error << "\nMetNcWrfDataFile::open(const char *) -> " << "unable to open NetCDF file \"" << _filename << "\"\n\n"; close(); @@ -100,7 +100,7 @@ bool MetNcPinterpDataFile::open(const char * _filename) { Raw_Grid = new Grid; - (*Raw_Grid) = PinterpNc->grid; + (*Raw_Grid) = WrfNc->grid; Dest_Grid = new Grid; @@ -111,34 +111,34 @@ bool MetNcPinterpDataFile::open(const char * _filename) { //////////////////////////////////////////////////////////////////////// -void MetNcPinterpDataFile::dump(ostream & out, int depth) const { +void MetNcWrfDataFile::dump(ostream & out, int depth) const { - if(PinterpNc) PinterpNc->dump(out, depth); + if(WrfNc) WrfNc->dump(out, depth); return; } //////////////////////////////////////////////////////////////////////// -bool MetNcPinterpDataFile::data_plane(VarInfo &vinfo, DataPlane &plane) { +bool MetNcWrfDataFile::data_plane(VarInfo &vinfo, DataPlane &plane) { bool status = false; double pressure; ConcatString level_str; - VarInfoNcPinterp * vinfo_nc = (VarInfoNcPinterp *) &vinfo; + VarInfoNcWrf * vinfo_nc = (VarInfoNcWrf *) &vinfo; NcVarInfo *info = (NcVarInfo *) 0; // Initialize the data plane plane.clear(); // Read the data - PinterpNc->get_nc_var_info(vinfo_nc->req_name().c_str(), info); + WrfNc->get_nc_var_info(vinfo_nc->req_name().c_str(), info); LongArray dimension = vinfo_nc->dimension(); int dim_count = dimension.n_elements(); for (int k=0; kvar, k)); - NcVarInfo *var_info = find_var_info_by_dim_name(PinterpNc->Var, dim_name, - PinterpNc->Nvars); + NcVarInfo *var_info = find_var_info_by_dim_name(WrfNc->Var, dim_name, + WrfNc->Nvars); if (var_info) { long new_offset = get_index_at_nc_data(var_info->var, vinfo_nc->dim_value(k), @@ -148,8 +148,8 @@ bool MetNcPinterpDataFile::data_plane(VarInfo &vinfo, DataPlane &plane) { } } - status = PinterpNc->data(vinfo_nc->req_name().c_str(), - dimension, plane, pressure, info); + status = WrfNc->data(vinfo_nc->req_name().c_str(), + dimension, plane, pressure, info); // Check that the times match those requested if(status) { @@ -157,7 +157,7 @@ bool MetNcPinterpDataFile::data_plane(VarInfo &vinfo, DataPlane &plane) { // Check that the valid time matches the request if(vinfo.valid() > 0 && vinfo.valid() != plane.valid()) { - mlog << Warning << "\nMetNcPinterpDataFile::data_plane() -> " + mlog << Warning << "\nMetNcWrfDataFile::data_plane() -> " << "for \"" << vinfo.req_name() << "\" variable, the valid " << "time does not match the requested valid time: (" << unix_to_yyyymmdd_hhmmss(plane.valid()) << " != " @@ -168,7 +168,7 @@ bool MetNcPinterpDataFile::data_plane(VarInfo &vinfo, DataPlane &plane) { // Check that the lead time matches the request if(vinfo.lead() > 0 && vinfo.lead() != plane.lead()) { - mlog << Warning << "\nMetNcPinterpDataFile::data_plane() -> " + mlog << Warning << "\nMetNcWrfDataFile::data_plane() -> " << "for \"" << vinfo.req_name() << "\" variable, the lead " << "time does not match the requested lead time: (" << sec_to_hhmmss(plane.lead()) << " != " @@ -196,13 +196,13 @@ bool MetNcPinterpDataFile::data_plane(VarInfo &vinfo, DataPlane &plane) { //////////////////////////////////////////////////////////////////////// -int MetNcPinterpDataFile::data_plane_array(VarInfo &vinfo, +int MetNcWrfDataFile::data_plane_array(VarInfo &vinfo, DataPlaneArray &plane_array) { int i, i_dim, n_level, status, lower, upper; ConcatString level_str; double pressure, min_level, max_level; bool found = false; - VarInfoNcPinterp *vinfo_nc = (VarInfoNcPinterp *) &vinfo; + VarInfoNcWrf *vinfo_nc = (VarInfoNcWrf *) &vinfo; LongArray dim = vinfo_nc->dimension(); NcVarInfo *info = (NcVarInfo *) 0; @@ -241,8 +241,8 @@ int MetNcPinterpDataFile::data_plane_array(VarInfo &vinfo, cur_dim[i_dim] = lower + i; // Read data for the current level - status = PinterpNc->data(vinfo_nc->req_name().c_str(), - cur_dim, cur_plane, pressure, info); + status = WrfNc->data(vinfo_nc->req_name().c_str(), + cur_dim, cur_plane, pressure, info); // Check that the times match those requested if(status) { @@ -250,7 +250,7 @@ int MetNcPinterpDataFile::data_plane_array(VarInfo &vinfo, // Check that the valid time matches the request if(vinfo.valid() > 0 && vinfo.valid() != cur_plane.valid()) { - mlog << Warning << "\nMetNcPinterpDataFile::data_plane_array() -> " + mlog << Warning << "\nMetNcWrfDataFile::data_plane_array() -> " << "for \"" << vinfo.req_name() << "\" variable, the valid " << "time does not match the requested valid time: (" << unix_to_yyyymmdd_hhmmss(cur_plane.valid()) << " != " @@ -261,7 +261,7 @@ int MetNcPinterpDataFile::data_plane_array(VarInfo &vinfo, // Check that the lead time matches the request if(vinfo.lead() > 0 && vinfo.lead() != cur_plane.lead()) { - mlog << Warning << "\nMetNcPinterpDataFile::data_plane_array() -> " + mlog << Warning << "\nMetNcWrfDataFile::data_plane_array() -> " << "for \"" << vinfo.req_name() << "\" variable, the lead " << "time does not match the requested lead time: (" << sec_to_hhmmss(cur_plane.lead()) << " != " @@ -307,7 +307,7 @@ int MetNcPinterpDataFile::data_plane_array(VarInfo &vinfo, //////////////////////////////////////////////////////////////////////// -int MetNcPinterpDataFile::index(VarInfo &vinfo){ +int MetNcWrfDataFile::index(VarInfo &vinfo){ return -1; } diff --git a/src/libcode/vx_data2d_nc_pinterp/data2d_nc_pinterp.h b/src/libcode/vx_data2d_nc_wrf/data2d_nc_wrf.h similarity index 72% rename from src/libcode/vx_data2d_nc_pinterp/data2d_nc_pinterp.h rename to src/libcode/vx_data2d_nc_wrf/data2d_nc_wrf.h index 40ab3f88f0..0ecc8eb112 100644 --- a/src/libcode/vx_data2d_nc_pinterp/data2d_nc_pinterp.h +++ b/src/libcode/vx_data2d_nc_wrf/data2d_nc_wrf.h @@ -10,8 +10,8 @@ //////////////////////////////////////////////////////////////////////// -#ifndef __MET_VX_DATA_2D_NC_PINTERP_H__ -#define __MET_VX_DATA_2D_NC_PINTERP_H__ +#ifndef __MET_VX_DATA_2D_NC_WRF_H__ +#define __MET_VX_DATA_2D_NC_WRF_H__ //////////////////////////////////////////////////////////////////////// @@ -21,33 +21,33 @@ #include "data_plane.h" #include "data_class.h" -#include "var_info_nc_pinterp.h" -#include "pinterp_file.h" +#include "var_info_nc_wrf.h" +#include "wrf_file.h" #include "two_to_one.h" //////////////////////////////////////////////////////////////////////// -class MetNcPinterpDataFile : public Met2dDataFile { +class MetNcWrfDataFile : public Met2dDataFile { private: - void nc_pinterp_init_from_scratch(); + void nc_wrf_init_from_scratch(); - MetNcPinterpDataFile(const MetNcPinterpDataFile &); - MetNcPinterpDataFile & operator=(const MetNcPinterpDataFile &); + MetNcWrfDataFile(const MetNcWrfDataFile &); + MetNcWrfDataFile & operator=(const MetNcWrfDataFile &); // // NetCDF file // - PinterpFile * PinterpNc; // allocated + WrfFile * WrfNc; // allocated public: - MetNcPinterpDataFile(); - ~MetNcPinterpDataFile(); + MetNcWrfDataFile(); + ~MetNcWrfDataFile(); // // set stuff @@ -87,13 +87,13 @@ class MetNcPinterpDataFile : public Met2dDataFile { //////////////////////////////////////////////////////////////////////// -inline GrdFileType MetNcPinterpDataFile::file_type () const { return ( FileType_NcPinterp ); } +inline GrdFileType MetNcWrfDataFile::file_type () const { return ( FileType_NcWrf ); } //////////////////////////////////////////////////////////////////////// -#endif /* __MET_VX_DATA_2D_NC_PINTERP_H__ */ +#endif /* __MET_VX_DATA_2D_NC_WRF_H__ */ //////////////////////////////////////////////////////////////////////// diff --git a/src/libcode/vx_data2d_nc_pinterp/get_pinterp_grid.cc b/src/libcode/vx_data2d_nc_wrf/get_wrf_grid.cc similarity index 97% rename from src/libcode/vx_data2d_nc_pinterp/get_pinterp_grid.cc rename to src/libcode/vx_data2d_nc_wrf/get_wrf_grid.cc index eaed12bd01..e94fcec619 100644 --- a/src/libcode/vx_data2d_nc_pinterp/get_pinterp_grid.cc +++ b/src/libcode/vx_data2d_nc_wrf/get_wrf_grid.cc @@ -24,7 +24,7 @@ using namespace netCDF; #include "vx_math.h" #include "vx_nc_util.h" -#include "get_pinterp_grid.h" +#include "get_wrf_grid.h" //////////////////////////////////////////////////////////////////////// @@ -66,15 +66,15 @@ static double mercator_v_to_lat(double v); //////////////////////////////////////////////////////////////////////// -bool get_pinterp_grid(const char * pinterp_filename, Grid & grid) +bool get_wrf_grid(const char * wrf_filename, Grid & grid) { bool status = false; -NcFile nc(pinterp_filename, NcFile::read); +NcFile nc(wrf_filename, NcFile::read); -status = get_pinterp_grid(nc, grid); +status = get_wrf_grid(nc, grid); return ( status ); @@ -85,7 +85,7 @@ return ( status ); //////////////////////////////////////////////////////////////////////// -bool get_pinterp_grid(NcFile & nc, Grid & grid) +bool get_wrf_grid(netCDF::NcFile & nc, Grid & grid) { diff --git a/src/libcode/vx_data2d_nc_pinterp/get_pinterp_grid.h b/src/libcode/vx_data2d_nc_wrf/get_wrf_grid.h similarity index 77% rename from src/libcode/vx_data2d_nc_pinterp/get_pinterp_grid.h rename to src/libcode/vx_data2d_nc_wrf/get_wrf_grid.h index c3c1f45cb8..76f80391e7 100644 --- a/src/libcode/vx_data2d_nc_pinterp/get_pinterp_grid.h +++ b/src/libcode/vx_data2d_nc_wrf/get_wrf_grid.h @@ -14,8 +14,8 @@ //////////////////////////////////////////////////////////////////////// -#ifndef __GET_PINTERP_GRID_H__ -#define __GET_PINTERP_GRID_H__ +#ifndef __GET_WRF_GRID_H__ +#define __GET_WRF_GRID_H__ //////////////////////////////////////////////////////////////////////// @@ -29,15 +29,15 @@ //////////////////////////////////////////////////////////////////////// -extern bool get_pinterp_grid(const char * pinterp_filename, Grid &); +extern bool get_wrf_grid(const char * wrf_filename, Grid & grid); -extern bool get_pinterp_grid(netCDF::NcFile &, Grid &); +extern bool get_wrf_grid(netCDF::NcFile & nc, Grid & grid); //////////////////////////////////////////////////////////////////////// -#endif /* __GET_PINTERP_GRID_H__ */ +#endif /* __GET_WRF_GRID_H__ */ //////////////////////////////////////////////////////////////////////// diff --git a/src/libcode/vx_data2d_nc_pinterp/var_info_nc_pinterp.cc b/src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.cc similarity index 87% rename from src/libcode/vx_data2d_nc_pinterp/var_info_nc_pinterp.cc rename to src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.cc index f0e01dc6a6..eecf2270cb 100644 --- a/src/libcode/vx_data2d_nc_pinterp/var_info_nc_pinterp.cc +++ b/src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.cc @@ -8,7 +8,7 @@ /////////////////////////////////////////////////////////////////////////////// // -// Filename: var_info_nc_pinterp.cc +// Filename: var_info_nc_wrf.cc // // Description: // @@ -20,12 +20,13 @@ using namespace std; #include +#include #include #include #include #include "var_info.h" -#include "var_info_nc_pinterp.h" +#include "var_info_nc_wrf.h" #include "vx_math.h" #include "vx_util.h" @@ -33,25 +34,25 @@ using namespace std; /////////////////////////////////////////////////////////////////////////////// // -// Code for class VarInfoNcPinterp +// Code for class VarInfoNcWrf // /////////////////////////////////////////////////////////////////////////////// -VarInfoNcPinterp::VarInfoNcPinterp() { +VarInfoNcWrf::VarInfoNcWrf() { init_from_scratch(); } /////////////////////////////////////////////////////////////////////////////// -VarInfoNcPinterp::~VarInfoNcPinterp() { +VarInfoNcWrf::~VarInfoNcWrf() { clear(); } /////////////////////////////////////////////////////////////////////////////// -VarInfoNcPinterp::VarInfoNcPinterp(const VarInfoNcPinterp &f) { +VarInfoNcWrf::VarInfoNcWrf(const VarInfoNcWrf &f) { init_from_scratch(); @@ -60,7 +61,7 @@ VarInfoNcPinterp::VarInfoNcPinterp(const VarInfoNcPinterp &f) { /////////////////////////////////////////////////////////////////////////////// -VarInfoNcPinterp & VarInfoNcPinterp::operator=(const VarInfoNcPinterp &f) { +VarInfoNcWrf & VarInfoNcWrf::operator=(const VarInfoNcWrf &f) { if ( this == &f ) return ( *this ); @@ -71,7 +72,7 @@ VarInfoNcPinterp & VarInfoNcPinterp::operator=(const VarInfoNcPinterp &f) { /////////////////////////////////////////////////////////////////////////////// -void VarInfoNcPinterp::init_from_scratch() { +void VarInfoNcWrf::init_from_scratch() { // First call the parent's initialization VarInfo::init_from_scratch(); @@ -82,7 +83,7 @@ void VarInfoNcPinterp::init_from_scratch() { /////////////////////////////////////////////////////////////////////////////// -void VarInfoNcPinterp::assign(const VarInfoNcPinterp &v) { +void VarInfoNcWrf::assign(const VarInfoNcWrf &v) { int i; // First call the parent's assign @@ -98,7 +99,7 @@ void VarInfoNcPinterp::assign(const VarInfoNcPinterp &v) { /////////////////////////////////////////////////////////////////////////////// -void VarInfoNcPinterp::clear() { +void VarInfoNcWrf::clear() { // First call the parent's clear VarInfo::clear(); @@ -110,10 +111,10 @@ void VarInfoNcPinterp::clear() { /////////////////////////////////////////////////////////////////////////////// -void VarInfoNcPinterp::dump(ostream &out) const { +void VarInfoNcWrf::dump(ostream &out) const { // Dump out the contents - out << "VarInfoNcPinterp::dump():\n" + out << "VarInfoNcWrf::dump():\n" << " Dimension:\n"; Dimension.dump(out); out << " Is_offset:\n"; @@ -125,7 +126,7 @@ void VarInfoNcPinterp::dump(ostream &out) const { /////////////////////////////////////////////////////////////////////////////// -void VarInfoNcPinterp::add_dimension(int dim, bool as_offset, double dim_value) { +void VarInfoNcWrf::add_dimension(int dim, bool as_offset, double dim_value) { Dimension.add(dim); Is_offset.add(as_offset); Dim_value.add(dim_value); @@ -133,7 +134,7 @@ void VarInfoNcPinterp::add_dimension(int dim, bool as_offset, double dim_value) /////////////////////////////////////////////////////////////////////////////// -void VarInfoNcPinterp::clear_dimension() { +void VarInfoNcWrf::clear_dimension() { Dimension.clear(); Is_offset.clear(); Dim_value.clear(); @@ -141,17 +142,17 @@ void VarInfoNcPinterp::clear_dimension() { /////////////////////////////////////////////////////////////////////////////// -void VarInfoNcPinterp::set_dimension(int i_dim, int dim) { +void VarInfoNcWrf::set_dimension(int i_dim, int dim) { Dimension[i_dim] = dim; return; } /////////////////////////////////////////////////////////////////////////////// -void VarInfoNcPinterp::set_magic(const ConcatString &nstr, const ConcatString &lstr) { +void VarInfoNcWrf::set_magic(const ConcatString &nstr, const ConcatString &lstr) { ConcatString tmp_str; char *ptr = (char *) 0, *ptr2 = (char *) 0, *ptr3 = (char *) 0, *save_ptr = (char *) 0; - const char *method_name = "VarInfoNcPinterp::set_magic() -> "; + const char *method_name = "VarInfoNcWrf::set_magic() -> "; // Store the magic string VarInfo::set_magic(nstr, lstr); @@ -272,7 +273,7 @@ void VarInfoNcPinterp::set_magic(const ConcatString &nstr, const ConcatString &l /////////////////////////////////////////////////////////////////////////////// -void VarInfoNcPinterp::set_dict(Dictionary & dict) { +void VarInfoNcWrf::set_dict(Dictionary & dict) { VarInfo::set_dict(dict); @@ -285,7 +286,7 @@ void VarInfoNcPinterp::set_dict(Dictionary & dict) { /////////////////////////////////////////////////////////////////////////////// -bool VarInfoNcPinterp::is_precipitation() const { +bool VarInfoNcWrf::is_precipitation() const { // // Check set_attrs entry @@ -305,7 +306,7 @@ bool VarInfoNcPinterp::is_precipitation() const { /////////////////////////////////////////////////////////////////////////////// -bool VarInfoNcPinterp::is_specific_humidity() const { +bool VarInfoNcWrf::is_specific_humidity() const { // // Check set_attrs entry @@ -325,7 +326,7 @@ bool VarInfoNcPinterp::is_specific_humidity() const { /////////////////////////////////////////////////////////////////////////////// -bool VarInfoNcPinterp::is_u_wind() const { +bool VarInfoNcWrf::is_u_wind() const { // // Check set_attrs entry @@ -334,6 +335,11 @@ bool VarInfoNcPinterp::is_u_wind() const { return(SetAttrIsUWind != 0); } + // Check if the VarInfo name is U or U where is an integer + if( regex_match (Name.c_str(), regex("^U[0-9]*$") )) { + return true; + } + // // Check to see if the VarInfo name matches any of expected Pinterp // U-component of wind variables. @@ -345,7 +351,7 @@ bool VarInfoNcPinterp::is_u_wind() const { /////////////////////////////////////////////////////////////////////////////// -bool VarInfoNcPinterp::is_v_wind() const { +bool VarInfoNcWrf::is_v_wind() const { // // Check set_attrs entry @@ -354,6 +360,11 @@ bool VarInfoNcPinterp::is_v_wind() const { return(SetAttrIsVWind != 0); } + // Check if the VarInfo name is V or V where is an integer + if( regex_match (Name.c_str(), regex("^V[0-9]*$") )) { + return true; + } + // // Check to see if the VarInfo name matches any of expected Pinterp // V-component of wind variables. @@ -365,7 +376,7 @@ bool VarInfoNcPinterp::is_v_wind() const { /////////////////////////////////////////////////////////////////////////////// -bool VarInfoNcPinterp::is_wind_speed() const { +bool VarInfoNcWrf::is_wind_speed() const { // // Check set_attrs entry @@ -385,7 +396,7 @@ bool VarInfoNcPinterp::is_wind_speed() const { /////////////////////////////////////////////////////////////////////////////// -bool VarInfoNcPinterp::is_wind_direction() const { +bool VarInfoNcWrf::is_wind_direction() const { // // Check set_attrs entry @@ -400,7 +411,7 @@ bool VarInfoNcPinterp::is_wind_direction() const { /////////////////////////////////////////////////////////////////////////////// -bool VarInfoNcPinterp::is_grid_relative() const { +bool VarInfoNcWrf::is_grid_relative() const { // // Check to see if the VarInfo name matches any of expected Pinterp diff --git a/src/libcode/vx_data2d_nc_pinterp/var_info_nc_pinterp.h b/src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.h similarity index 84% rename from src/libcode/vx_data2d_nc_pinterp/var_info_nc_pinterp.h rename to src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.h index 61cc0204c0..182ad62d28 100644 --- a/src/libcode/vx_data2d_nc_pinterp/var_info_nc_pinterp.h +++ b/src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.h @@ -9,8 +9,8 @@ /////////////////////////////////////////////////////////////////////////////// -#ifndef __VAR_INFO_NC_PINTERP_H__ -#define __VAR_INFO_NC_PINTERP_H__ +#ifndef __VAR_INFO_NC_WRF_H__ +#define __VAR_INFO_NC_WRF_H__ /////////////////////////////////////////////////////////////////////////////// @@ -100,7 +100,9 @@ static const int n_pinterp_specific_humidity_names = static const char *pinterp_u_wind_names[] = { "UU", // x-wind component, m s-1 "TS_U", // Surface wind U-component, earth-relative - "UZ0" // U WIND COMPONENT AT ZNT, m s-1 + "UZ0", // U WIND COMPONENT AT ZNT, m s-1 + "U_PL", // x-wind on pressure levels from native WRF out + "U_ZL" // x-wind on z levels from native WRF out }; // @@ -121,7 +123,9 @@ static const int n_pinterp_u_wind_names = static const char *pinterp_v_wind_names[] = { "VV", // y-wind component, m s-1 "TS_V", // Surface wind V-component, earth-relative - "VZ0" // V WIND COMPONENT AT ZNT, m s-1 + "VZ0", // V WIND COMPONENT AT ZNT, m s-1 + "V_PL", // y-wind on pressure levels from native WRF out + "V_ZL" // y-wind on z levels from native WRF out }; // @@ -179,7 +183,7 @@ static const int n_pinterp_wind_speed_names = /////////////////////////////////////////////////////////////////////////////// -class VarInfoNcPinterp : public VarInfo +class VarInfoNcWrf : public VarInfo { private: @@ -192,14 +196,14 @@ class VarInfoNcPinterp : public VarInfo NumArray Dim_value; // Dimension values as float for extracting 2D field void init_from_scratch(); - void assign(const VarInfoNcPinterp &); + void assign(const VarInfoNcWrf &); void clear_dimension(); public: - VarInfoNcPinterp(); - ~VarInfoNcPinterp(); - VarInfoNcPinterp(const VarInfoNcPinterp &); - VarInfoNcPinterp & operator=(const VarInfoNcPinterp &); + VarInfoNcWrf(); + ~VarInfoNcWrf(); + VarInfoNcWrf(const VarInfoNcWrf &); + VarInfoNcWrf & operator=(const VarInfoNcWrf &); void dump(std::ostream &) const; void clear(); @@ -242,17 +246,17 @@ class VarInfoNcPinterp : public VarInfo /////////////////////////////////////////////////////////////////////////////// -inline GrdFileType VarInfoNcPinterp::file_type() const { return(FileType_NcPinterp); } -inline const LongArray & VarInfoNcPinterp::dimension() const { return(Dimension); } -inline int VarInfoNcPinterp::dimension(int i) const { return(Dimension[i]); } -inline int VarInfoNcPinterp::n_dimension() const { return(Dimension.n_elements()); } -inline const NumArray & VarInfoNcPinterp::dim_value() const { return(Dim_value); } -inline double VarInfoNcPinterp::dim_value(int i) const { return(Dim_value[i]); } -inline const BoolArray & VarInfoNcPinterp::is_offset() const { return(Is_offset); } -inline bool VarInfoNcPinterp::is_offset(int i) const { return(Is_offset[i]); } +inline GrdFileType VarInfoNcWrf::file_type() const { return(FileType_NcWrf); } +inline const LongArray & VarInfoNcWrf::dimension() const { return(Dimension); } +inline int VarInfoNcWrf::dimension(int i) const { return(Dimension[i]); } +inline int VarInfoNcWrf::n_dimension() const { return(Dimension.n_elements()); } +inline const NumArray & VarInfoNcWrf::dim_value() const { return(Dim_value); } +inline double VarInfoNcWrf::dim_value(int i) const { return(Dim_value[i]); } +inline const BoolArray & VarInfoNcWrf::is_offset() const { return(Is_offset); } +inline bool VarInfoNcWrf::is_offset(int i) const { return(Is_offset[i]); } /////////////////////////////////////////////////////////////////////////////// -#endif // __VAR_INFO_NC_PINTERP_H__ +#endif // __VAR_INFO_NC_WRF_H__ /////////////////////////////////////////////////////////////////////////////// diff --git a/src/libcode/vx_data2d_nc_pinterp/vx_data2d_nc_pinterp.h b/src/libcode/vx_data2d_nc_wrf/vx_data2d_nc_wrf.h similarity index 74% rename from src/libcode/vx_data2d_nc_pinterp/vx_data2d_nc_pinterp.h rename to src/libcode/vx_data2d_nc_wrf/vx_data2d_nc_wrf.h index e8094a0bba..dec1c38acd 100644 --- a/src/libcode/vx_data2d_nc_pinterp/vx_data2d_nc_pinterp.h +++ b/src/libcode/vx_data2d_nc_wrf/vx_data2d_nc_wrf.h @@ -14,23 +14,23 @@ //////////////////////////////////////////////////////////////////////// -#ifndef __VX_DATA2D_NC_PINTERP_H__ -#define __VX_DATA2D_NC_PINTERP_H__ +#ifndef __VX_DATA2D_NC_WRF_H__ +#define __VX_DATA2D_NC_WRF_H__ //////////////////////////////////////////////////////////////////////// -#include "data2d_nc_pinterp.h" -#include "var_info_nc_pinterp.h" -#include "get_pinterp_grid.h" -#include "pinterp_file.h" +#include "data2d_nc_wrf.h" +#include "var_info_nc_wrf.h" +#include "get_wrf_grid.h" +#include "wrf_file.h" //////////////////////////////////////////////////////////////////////// -#endif // __VX_DATA2D_NC_PINTERP_H__ +#endif // __VX_DATA2D_NC_WRF_H__ //////////////////////////////////////////////////////////////////////// diff --git a/src/libcode/vx_data2d_nc_pinterp/pinterp_file.cc b/src/libcode/vx_data2d_nc_wrf/wrf_file.cc similarity index 63% rename from src/libcode/vx_data2d_nc_pinterp/pinterp_file.cc rename to src/libcode/vx_data2d_nc_wrf/wrf_file.cc index b84aee3a9f..3232887525 100644 --- a/src/libcode/vx_data2d_nc_pinterp/pinterp_file.cc +++ b/src/libcode/vx_data2d_nc_wrf/wrf_file.cc @@ -27,18 +27,26 @@ using namespace netCDF; #include "vx_log.h" #include "vx_nc_util.h" -#include "pinterp_file.h" -#include "get_pinterp_grid.h" +#include "wrf_file.h" +#include "get_wrf_grid.h" //////////////////////////////////////////////////////////////////////// static const char x_dim_name [] = "west_east"; +static const char x_dim_stag_name [] = "west_east_stag"; +static const char x_dim_subgrid_name [] = "west_east_subgrid"; static const char y_dim_name [] = "south_north"; +static const char y_dim_stag_name [] = "south_north_stag"; +static const char y_dim_subgrid_name [] = "south_north_subgrid"; static const char t_dim_name [] = "time"; static const char z_dim_p_interp_name [] = "num_metgrid_levels"; static const char z_dim_wrf_interp_name[] = "vlevs"; +static const char z_dim_wrf_stag_name [] = "bottom_top_stag"; +static const char z_dim_wrf_name [] = "bottom_top"; +static const char z_dim_wrf_pres_name [] = "num_press_levels_stag"; +static const char z_dim_wrf_z_name [] = "num_z_levels_stag"; static const string strl_dim_name = "DateStrLen"; static const char times_var_name [] = "Times"; @@ -51,15 +59,16 @@ static const char second_var_name [] = "second"; static const char pressure_var_p_interp_name [] = "pressure"; static const char pressure_var_wrf_interp_name [] = "LEV"; +static const char pressure_var_wrf_name [] = "P_PL"; static const char pa_units_str [] = "Pa"; static const char hpa_units_str [] = "hPa"; static const string start_time_att_name = "START_DATE"; -static const int max_pinterp_args = 30; +static const int max_wrf_args = 30; -static const double pinterp_missing = 1.0e35; +static const double wrf_missing = 1.0e35; static const char *accum_var_names [] = { "ACGRDFLX", "CUPPT", "RAINC", "RAINNC", @@ -72,7 +81,7 @@ static const int n_accum_var_names = sizeof(accum_var_names)/sizeof(*accu static unixtime parse_init_time(const char *); -static bool is_bad_data_pinterp(double); +static bool is_bad_data_wrf(double); static bool is_accumulation(const char *); @@ -80,14 +89,14 @@ static bool is_accumulation(const char *); // - // Code for class PinterpFile + // Code for class WrfFile // //////////////////////////////////////////////////////////////////////// -PinterpFile::PinterpFile() +WrfFile::WrfFile() { @@ -99,7 +108,7 @@ init_from_scratch(); //////////////////////////////////////////////////////////////////////// -PinterpFile::~PinterpFile() +WrfFile::~WrfFile() { @@ -111,7 +120,7 @@ close(); //////////////////////////////////////////////////////////////////////// -void PinterpFile::init_from_scratch() +void WrfFile::init_from_scratch() { @@ -133,7 +142,7 @@ return; //////////////////////////////////////////////////////////////////////// -void PinterpFile::close() +void WrfFile::close() { @@ -147,7 +156,7 @@ Ndims = 0; DimNames.clear(); -Xdim = Ydim = Zdim = Tdim = (NcDim *) 0; +Tdim = (NcDim *) 0; Nvars = 0; @@ -157,10 +166,6 @@ InitTime = (unixtime) 0; Ntimes = 0; -PressureIndex = -1; - -hPaCF = 1.0; - // // done // @@ -173,7 +178,7 @@ return; //////////////////////////////////////////////////////////////////////// -bool PinterpFile::open(const char * filename) +bool WrfFile::open(const char * filename) { @@ -182,13 +187,13 @@ int month, day, year, hour, minute, second, str_len; char time_str[max_str_len]; string c; NcVar v; -const char *method_name = "PinterpFile::open() -> "; +const char *method_name = "WrfFile::open() -> "; close(); Nc = open_ncfile(filename); mlog << Debug(5) << "\n" << method_name - << "opend \"" << filename << "\".\n\n"; + << "open \"" << filename << "\".\n\n"; if ( IS_INVALID_NC_P(Nc) ) { close(); return ( false ); } @@ -196,7 +201,7 @@ if ( IS_INVALID_NC_P(Nc) ) { close(); return ( false ); } // grid // -if ( ! get_pinterp_grid(*Nc, grid) ) { close(); return ( false ); } +if ( !get_wrf_grid(*Nc, grid) ) { close(); return ( false ); } // // dimensions @@ -204,19 +209,13 @@ if ( ! get_pinterp_grid(*Nc, grid) ) { close(); return ( false ); } Ndims = get_dim_count(Nc); Dim = new NcDim*[Ndims]; -StringArray gDimNames; -get_dim_names(Nc, &gDimNames); +get_dim_names(Nc, &DimNames); for (j=0; j= Ntimes) ) { - mlog << Error << "\nPinterpFile::valid_time(int) const -> " + mlog << Error << "\nWrfFile::valid_time(int) const -> " << "range check error\n\n"; exit ( 1 ); @@ -504,13 +481,13 @@ return ( Time [n] ); //////////////////////////////////////////////////////////////////////// -int PinterpFile::lead_time(int n) const +int WrfFile::lead_time(int n) const { if ( (n < 0) || (n >= Ntimes) ) { - mlog << Error << "\nPinterpFile::lead_time(int) const -> " + mlog << Error << "\nWrfFile::lead_time(int) const -> " << "range check error\n\n"; exit ( 1 ); @@ -527,14 +504,13 @@ return ( (int) dt ); //////////////////////////////////////////////////////////////////////// -double PinterpFile::data(NcVar * var, const LongArray & a) const +double WrfFile::data(NcVar * var, const LongArray & a) const { - +const char *method_name = "WrfFile::data(NcVar *, const LongArray &) const -> "; if ( !args_ok(a) ) { - mlog << Error << "\nPinterpFile::data(NcVar *, const LongArray &) const -> " - << "bad arguments:\n"; + mlog << Error << "\n" << method_name << "bad arguments:\n"; a.dump(cerr); @@ -545,7 +521,7 @@ if ( !args_ok(a) ) { int dim_count = var->getDimCount(); if ( dim_count != a.n_elements() ) { - mlog << Error << "\nPinterpFile::data(NcVar *, const LongArray &) const -> " + mlog << Error << "\n" << method_name << "needed " << (dim_count) << " arguments for variable " << (GET_NC_NAME_P(var)) << ", got " << (a.n_elements()) << "\n\n"; @@ -553,9 +529,9 @@ if ( dim_count != a.n_elements() ) { } -if ( dim_count >= max_pinterp_args ) { +if (dim_count >= max_wrf_args ) { - mlog << Error << "\nPinterpFile::data(NcVar *, const LongArray &) const -> " + mlog << Error << "\n" << method_name << " too may arguments for variable \"" << (GET_NC_NAME_P(var)) << "\"\n\n"; exit ( 1 ); @@ -563,9 +539,6 @@ if ( dim_count >= max_pinterp_args ) { } bool status = false; -int i; -short s; -float f; double fill_value; double d = bad_data_double; double missing_value = get_var_missing_value(var); @@ -575,8 +548,7 @@ status = get_nc_data(var, &d, a); if ( !status ) { - mlog << Error << "\nPinterpFile::data(NcVar *, const LongArray &) const -> " - << " bad status for var->get()\n\n"; + mlog << Error << "\n" << method_name << " bad status for var->get()\n\n"; exit ( 1 ); } @@ -593,14 +565,13 @@ return ( d ); //////////////////////////////////////////////////////////////////////// -bool PinterpFile::data(NcVar * v, const LongArray & a, DataPlane & plane, double & pressure) const +bool WrfFile::data(NcVar * v, const LongArray & a, DataPlane & plane, double & pressure) const { - +const char *method_name = "WrfFile::data(NcVar *, const LongArray &, DataPlane &, double &) const -> "; if ( !args_ok(a) ) { - mlog << Warning << "\nPinterpFile::data(NcVar *, const LongArray &, DataPlane &, double &) const -> " - << "bad arguments:\n"; + mlog << Warning << "\n" << method_name << "bad arguments:\n"; a.dump(cerr); @@ -612,7 +583,7 @@ string var_name = GET_NC_NAME_P(v); int dim_count = v->getDimCount(); if ( dim_count != a.n_elements() ) { - mlog << Warning << "\nPinterpFile::data(NcVar *, const LongArray &, DataPlane &, double &) const -> " + mlog << Warning << "\n" << method_name << "needed " << dim_count << " arguments for variable " << (var_name) << ", got " << (a.n_elements()) << "\n\n"; @@ -620,9 +591,9 @@ if ( dim_count != a.n_elements() ) { } -if ( dim_count >= max_pinterp_args ) { +if (dim_count >= max_wrf_args ) { - mlog << Warning << "\nPinterpFile::data(NcVar *, const LongArray &, DataPlane &, double &) const -> " + mlog << Warning << "\n" << method_name << " too may arguments for variable \"" << (var_name) << "\"\n\n"; return ( false ); @@ -630,14 +601,12 @@ if ( dim_count >= max_pinterp_args ) { } -int j, count; +int j, k, count; int x, y; double value; bool found = false; NcVarInfo * var = (NcVarInfo *) 0; NcVarInfo * P = (NcVarInfo *) 0; -const int Nx = grid.nx(); -const int Ny = grid.ny(); LongArray b = a; pressure = bad_data_double; @@ -646,8 +615,6 @@ pressure = bad_data_double; // find varinfo's // -if ( PressureIndex >= 0 ) P = Var + PressureIndex; - found = false; for (j=0; j " + mlog << Warning << "\n" << method_name << "variable " << (var_name) << " not found!\n\n"; return ( false ); } + // check dimensions + StringArray varDimNames; + get_dim_names(var->var, &varDimNames); + string c, z_name; + for (k=0; k<(var->Ndims); ++k) { + c = to_lower(varDimNames[k]); + + // X dimension + if ( c.compare(x_dim_name) == 0 || + c.compare(x_dim_stag_name) == 0 || + c.compare(x_dim_subgrid_name) == 0) { + + var->x_slot = k; + + // track fields that need to be de-staggered in the X dimension + if ( c.compare(x_dim_stag_name) == 0 ) { + + var->x_stag = true; + + } + + // error if unsupported subgrid + if ( c.compare(x_dim_subgrid_name) == 0 ) { + + mlog << Error << "\n" << method_name + << "X Dimension \"" << x_dim_subgrid_name << "\" is not supported.\n\n"; + return ( false ); + + } + + } + // Y dimension + else if ( c.compare(y_dim_name) == 0 || + c.compare(y_dim_stag_name) == 0 || + c.compare(y_dim_subgrid_name) == 0 ) { + + var->y_slot = k; + + // track fields that need to be de-staggered in the Y dimension + if ( c.compare(y_dim_stag_name) == 0 ) { + + var->y_stag = true; + + } + + // error if unsupported subgrid + if ( c.compare(y_dim_subgrid_name) == 0 ) { + + mlog << Error << "\n" << method_name + << "Y Dimension \"" << y_dim_subgrid_name << "\" is not supported.\n\n"; + return ( false ); + + } + + } + // Z dimension + else if ( c.compare(z_dim_p_interp_name ) == 0 || + c.compare(z_dim_wrf_interp_name) == 0 || + c.compare(z_dim_wrf_name ) == 0 || + c.compare(z_dim_wrf_stag_name) == 0 || + c.compare(z_dim_wrf_pres_name) == 0 || + c.compare(z_dim_wrf_z_name ) == 0 ) { + + var->z_slot = k; + + // track fields that are on pressure levels + if ( c.compare(z_dim_p_interp_name) == 0 || + c.compare(z_dim_wrf_interp_name) == 0 || + c.compare(z_dim_wrf_pres_name) == 0 ) { + + var->is_pressure = true; + z_name = c; + + } + + // track fields that need to be de-staggered in the Z dimension + if ( c.compare(z_dim_wrf_stag_name) == 0 ) { + + var->z_stag = true; + + } + } + // T dimension + else if ( c.compare(t_dim_name) == 0 ) { + + var->t_slot = k; + + } + + } // end if k + + bool time_in_pressure = false; + double pressure_unit_conversion = 1.0; + + // if reading var on pressure, find pressure field that is on the same vertical (z) dimension as the var to read + if (var->is_pressure) { + + StringArray varNames; + get_var_names(Nc, &varNames); + + for (j = 0; j < Nvars; ++j) { + // + // get the pressure variable and store the hPa conversion factor + // + + if (varNames[j] == pressure_var_p_interp_name || + varNames[j] == pressure_var_wrf_interp_name || + varNames[j] == pressure_var_wrf_name) { + + varDimNames.clear(); + get_dim_names(Var[j].var, &varDimNames); + + // check that the z dimension matches the var to read + found = false; + for(k=0; k< Var[j].Ndims; k++){ + if(varDimNames[k] == z_name) { + found = true; + break; + } + } + + if(!found) { + continue; + } + + // set pressure field + P = Var + j; + + if (varNames[j] == pressure_var_wrf_name) { + + time_in_pressure = true; + + } + if (strcasecmp(Var[j].units_att.c_str(), pa_units_str) == 0) pressure_unit_conversion = 0.01; + else if (strcasecmp(Var[j].units_att.c_str(), hpa_units_str) == 0) pressure_unit_conversion = 1.0; + } + } + } + + // + // set nx and ny based on staggering of dimensions of the variable to read + // + const int Nx = var->x_stag ? grid.nx() + 1 : grid.nx(); + const int Ny = var->y_stag ? grid.ny() + 1 : grid.ny(); + // // check x_slot and y_slot // if ( var == nullptr || (var->x_slot < 0) || (var->y_slot < 0) ) { - mlog << Error - << "\nPinterpFile::data(NcVar *, const LongArray &, DataPlane &, double &) const -> " + mlog << Error << "\n" << method_name << "can't find needed dimensions(s) for variable \"" << var_name << "\" ... one of the dimensions may be staggered.\n\n"; @@ -694,8 +805,7 @@ for (j=0; j<(a.n_elements()); ++j) { if ( (j != var->x_slot) && (j != var->y_slot) ) { - mlog << Warning << "\nPinterpFile::data(NcVar *, const LongArray &, DataPlane &, double &) const -> " - << " star found in bad slot\n\n"; + mlog << Warning << "\n" << method_name << " star found in bad slot\n\n"; return ( false ); @@ -707,8 +817,7 @@ for (j=0; j<(a.n_elements()); ++j) { if ( count != 2 ) { - mlog << Warning << "\nPinterpFile::data(NcVar *, const LongArray &, DataPlane &, double &) const -> " - << " bad star count ... " << count << "\n\n"; + mlog << Warning << "\n" << method_name << " bad star count ... " << count << "\n\n"; return ( false ); @@ -724,8 +833,7 @@ const int z_slot = var->z_slot; if ( (x_slot < 0) || (y_slot < 0) ) { - mlog << Warning << "\nPinterpFile::data(NcVar *, const LongArray &, DataPlane &, double &) const -> " - << " bad x|y slot\n\n"; + mlog << Warning << "\n" << method_name << " bad x|y slot\n\n"; return ( false ); @@ -762,7 +870,7 @@ for (x=0; xx_stag, var->y_stag); + // // get the pressure // if ( P && z_slot > 0 ) { + mlog << Debug(3) << "Reading pressure field " << P->name << "\n"; + LongArray c; + if(time_in_pressure) c.add(a[var->t_slot]); + c.add(a[z_slot]); - pressure = data(P->var, c) * hPaCF; + pressure = data(P->var, c) * pressure_unit_conversion; } @@ -798,8 +913,8 @@ return ( true ); //////////////////////////////////////////////////////////////////////// -bool PinterpFile::data(const char * var_name, const LongArray & a, DataPlane & plane, - double & pressure, NcVarInfo *&info) const { +bool WrfFile::data(const char * var_name, const LongArray & a, DataPlane & plane, + double & pressure, NcVarInfo *&info) const { int time_index; bool found = false; @@ -822,7 +937,7 @@ bool PinterpFile::data(const char * var_name, const LongArray & a, DataPlane & p plane.set_lead ( lead_time(time_index) ); // - // since Pinterp files only contain WRF-ARW output, it is always a + // since Pinterp files only contain WRF-ARW output, it is always // a runtime accumulation // @@ -847,7 +962,7 @@ bool PinterpFile::data(const char * var_name, const LongArray & a, DataPlane & p //////////////////////////////////////////////////////////////////////// -bool PinterpFile::get_nc_var_info(const char *var_name, NcVarInfo *&info) const { +bool WrfFile::get_nc_var_info(const char *var_name, NcVarInfo *&info) const { bool found = false; if (nullptr == info) { @@ -911,11 +1026,11 @@ return ( t ); //////////////////////////////////////////////////////////////////////// -bool is_bad_data_pinterp(double v) +bool is_bad_data_wrf(double v) { -if ( v < pinterp_missing ) return ( false ); +if (v < wrf_missing ) return ( false ); else return ( true ); } diff --git a/src/libcode/vx_data2d_nc_pinterp/pinterp_file.h b/src/libcode/vx_data2d_nc_wrf/wrf_file.h similarity index 79% rename from src/libcode/vx_data2d_nc_pinterp/pinterp_file.h rename to src/libcode/vx_data2d_nc_wrf/wrf_file.h index dc5ff86f99..722dc26d01 100644 --- a/src/libcode/vx_data2d_nc_pinterp/pinterp_file.h +++ b/src/libcode/vx_data2d_nc_wrf/wrf_file.h @@ -14,8 +14,8 @@ //////////////////////////////////////////////////////////////////////// -#ifndef __MET_PINTERP_FILE_H__ -#define __MET_PINTERP_FILE_H__ +#ifndef __MET_WRF_FILE_H__ +#define __MET_WRF_FILE_H__ //////////////////////////////////////////////////////////////////////// @@ -34,19 +34,19 @@ //////////////////////////////////////////////////////////////////////// -class PinterpFile { +class WrfFile { private: void init_from_scratch(); - PinterpFile(const PinterpFile &); - PinterpFile & operator=(const PinterpFile &); + WrfFile(const WrfFile &); + WrfFile & operator=(const WrfFile &); public: - PinterpFile(); - ~PinterpFile(); + WrfFile(); + ~WrfFile(); bool open(const char * filename); @@ -80,9 +80,6 @@ class PinterpFile { StringArray DimNames; - netCDF::NcDim * Xdim; // not allocated - netCDF::NcDim * Ydim; // not allocated - netCDF::NcDim * Zdim; // not allocated netCDF::NcDim * Tdim; // not allocated // @@ -93,10 +90,6 @@ class PinterpFile { NcVarInfo * Var; // allocated - int PressureIndex; // index into Var array - - double hPaCF; // pressure to hPa conversion factor - // // Grid // @@ -121,7 +114,7 @@ class PinterpFile { //////////////////////////////////////////////////////////////////////// -#endif /* __MET_PINTERP_FILE_H__ */ +#endif /* __MET_WRF_FILE_H__ */ //////////////////////////////////////////////////////////////////////// diff --git a/src/libcode/vx_data2d_nccf/Makefile.am b/src/libcode/vx_data2d_nccf/Makefile.am deleted file mode 100644 index 4b606a8376..0000000000 --- a/src/libcode/vx_data2d_nccf/Makefile.am +++ /dev/null @@ -1,20 +0,0 @@ -## @start 1 -## Makefile.am -- Process this file with automake to produce Makefile.in -## @end 1 - -MAINTAINERCLEANFILES = Makefile.in - -# Include the project definitions - -include ${top_srcdir}/Make-include - -# The library - -noinst_LIBRARIES = libvx_data2d_nccf.a -libvx_data2d_nccf_a_SOURCES = \ - nccf_file.cc nccf_file.h \ - var_info_nccf.cc var_info_nccf.h \ - data2d_nccf.cc data2d_nccf.h \ - vx_data2d_nccf.h -libvx_data2d_nccf_a_CPPFLAGS = ${MET_CPPFLAGS} - diff --git a/src/libcode/vx_nc_util/nc_var_info.cc b/src/libcode/vx_nc_util/nc_var_info.cc index 834df56ac8..6159316080 100644 --- a/src/libcode/vx_nc_util/nc_var_info.cc +++ b/src/libcode/vx_nc_util/nc_var_info.cc @@ -151,6 +151,10 @@ if ( Dims ) { delete [] Dims; Dims = (NcDim **) 0; } x_slot = y_slot = z_slot = t_slot = -1; +x_stag = y_stag = z_stag = false; + +is_pressure = false; + // // done // @@ -213,10 +217,17 @@ if ( Dims ) { } -out << prefix << "x_slot = " << x_slot << "\n"; -out << prefix << "y_slot = " << y_slot << "\n"; -out << prefix << "z_slot = " << z_slot << "\n"; +out << prefix << "x_slot = " << x_slot; +if (x_stag) out << " (staggered)"; +out << "\n"; +out << prefix << "y_slot = " << y_slot; +if (y_stag) out << " (staggered)"; +out << "\n"; +out << prefix << "z_slot = " << z_slot; +if (z_stag) out << " (staggered)"; +out << "\n"; out << prefix << "t_slot = " << t_slot << "\n"; +out << prefix << "is_pressure = " << (is_pressure ? "true" : "false") << "\n"; // // done @@ -270,6 +281,12 @@ y_slot = i.y_slot; z_slot = i.z_slot; t_slot = i.t_slot; +x_stag = i.x_stag; +y_stag = i.y_stag; +z_stag = i.z_stag; + +is_pressure = i.is_pressure; + if ( i.Dims ) { Dims = new NcDim * [i.Ndims]; @@ -309,18 +326,18 @@ NcVarInfo *find_var_info_by_dim_name(NcVarInfo *vars, const string dim_name, } } + // if dimension variable is not found, find variable that has only dim_name or dim_name and time + int dim_offset = 0; if (!var) { - //StringArray dim_names; for (int i=0; i 2) continue; + dim_offset = vars[i].Ndims == 2 ? 1 : 0; + NcDim dim = get_nc_dim(vars[i].var, dim_offset); + if (IS_VALID_NC(dim) && GET_NC_NAME(dim) == dim_name) { + var = &vars[i]; + break; } + } } diff --git a/src/libcode/vx_nc_util/nc_var_info.h b/src/libcode/vx_nc_util/nc_var_info.h index 3ecc888ed3..04f422cd85 100644 --- a/src/libcode/vx_nc_util/nc_var_info.h +++ b/src/libcode/vx_nc_util/nc_var_info.h @@ -88,6 +88,12 @@ class NcVarInfo { int z_slot; // -1 if not defined int t_slot; // + bool x_stag; // are dimensions staggered + bool y_stag; + bool z_stag; + + bool is_pressure; // is variable on pressure levels + }; //////////////////////////////////////////////////////////////////////// diff --git a/src/tools/core/ensemble_stat/Makefile.am b/src/tools/core/ensemble_stat/Makefile.am index 5d87a3aa4c..d05ad256cc 100644 --- a/src/tools/core/ensemble_stat/Makefile.am +++ b/src/tools/core/ensemble_stat/Makefile.am @@ -24,8 +24,8 @@ ensemble_stat_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/core/ensemble_stat/Makefile.in b/src/tools/core/ensemble_stat/Makefile.in index d2c8c0d798..a960e21483 100644 --- a/src/tools/core/ensemble_stat/Makefile.in +++ b/src/tools/core/ensemble_stat/Makefile.in @@ -345,8 +345,8 @@ ensemble_stat_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/core/grid_stat/Makefile.am b/src/tools/core/grid_stat/Makefile.am index 4df815bed4..db4c9bba80 100644 --- a/src/tools/core/grid_stat/Makefile.am +++ b/src/tools/core/grid_stat/Makefile.am @@ -24,8 +24,8 @@ grid_stat_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/core/grid_stat/Makefile.in b/src/tools/core/grid_stat/Makefile.in index 2c8b909c39..30cc717d27 100644 --- a/src/tools/core/grid_stat/Makefile.in +++ b/src/tools/core/grid_stat/Makefile.in @@ -344,8 +344,8 @@ grid_stat_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/core/mode/Makefile.am b/src/tools/core/mode/Makefile.am index 72a35a6175..d8dc507385 100644 --- a/src/tools/core/mode/Makefile.am +++ b/src/tools/core/mode/Makefile.am @@ -43,8 +43,8 @@ mode_LDADD = -lvx_pxm \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/core/mode/Makefile.in b/src/tools/core/mode/Makefile.in index 249a93d1bb..741c5fea4f 100644 --- a/src/tools/core/mode/Makefile.in +++ b/src/tools/core/mode/Makefile.in @@ -381,8 +381,8 @@ mode_LDADD = -lvx_pxm \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/core/mode_analysis/Makefile.am b/src/tools/core/mode_analysis/Makefile.am index 1bff9fadc7..a905710fe6 100644 --- a/src/tools/core/mode_analysis/Makefile.am +++ b/src/tools/core/mode_analysis/Makefile.am @@ -26,8 +26,8 @@ mode_analysis_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/core/mode_analysis/Makefile.in b/src/tools/core/mode_analysis/Makefile.in index ce4e7bc477..334137147a 100644 --- a/src/tools/core/mode_analysis/Makefile.in +++ b/src/tools/core/mode_analysis/Makefile.in @@ -347,8 +347,8 @@ mode_analysis_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/core/pcp_combine/Makefile.am b/src/tools/core/pcp_combine/Makefile.am index 0206f497ca..189960810f 100644 --- a/src/tools/core/pcp_combine/Makefile.am +++ b/src/tools/core/pcp_combine/Makefile.am @@ -18,8 +18,8 @@ pcp_combine_LDADD = -lvx_data2d_factory \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/core/pcp_combine/Makefile.in b/src/tools/core/pcp_combine/Makefile.in index 751ecc5b8a..44b0f69c50 100644 --- a/src/tools/core/pcp_combine/Makefile.in +++ b/src/tools/core/pcp_combine/Makefile.in @@ -335,8 +335,8 @@ pcp_combine_LDADD = -lvx_data2d_factory \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/core/point_stat/Makefile.am b/src/tools/core/point_stat/Makefile.am index 39b45fcb1b..bbd9b9c142 100644 --- a/src/tools/core/point_stat/Makefile.am +++ b/src/tools/core/point_stat/Makefile.am @@ -23,8 +23,8 @@ point_stat_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/core/point_stat/Makefile.in b/src/tools/core/point_stat/Makefile.in index ccaa3f8e36..81735f1efb 100644 --- a/src/tools/core/point_stat/Makefile.in +++ b/src/tools/core/point_stat/Makefile.in @@ -343,8 +343,8 @@ point_stat_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/core/series_analysis/Makefile.am b/src/tools/core/series_analysis/Makefile.am index 24f14c90e8..7d57a6db6f 100644 --- a/src/tools/core/series_analysis/Makefile.am +++ b/src/tools/core/series_analysis/Makefile.am @@ -24,8 +24,8 @@ series_analysis_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/core/series_analysis/Makefile.in b/src/tools/core/series_analysis/Makefile.in index 9c68f8f191..2e26c49701 100644 --- a/src/tools/core/series_analysis/Makefile.in +++ b/src/tools/core/series_analysis/Makefile.in @@ -346,8 +346,8 @@ series_analysis_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/core/stat_analysis/Makefile.am b/src/tools/core/stat_analysis/Makefile.am index f73b415ca2..1f26904bf5 100644 --- a/src/tools/core/stat_analysis/Makefile.am +++ b/src/tools/core/stat_analysis/Makefile.am @@ -27,8 +27,8 @@ stat_analysis_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/core/stat_analysis/Makefile.in b/src/tools/core/stat_analysis/Makefile.in index 9d0b3c78ea..168c3da589 100644 --- a/src/tools/core/stat_analysis/Makefile.in +++ b/src/tools/core/stat_analysis/Makefile.in @@ -354,8 +354,8 @@ stat_analysis_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/core/stat_analysis/aggr_stat_line.cc b/src/tools/core/stat_analysis/aggr_stat_line.cc index 5487b1fd3f..3c8395a09c 100644 --- a/src/tools/core/stat_analysis/aggr_stat_line.cc +++ b/src/tools/core/stat_analysis/aggr_stat_line.cc @@ -52,7 +52,7 @@ using namespace std; #include "vx_log.h" #include "vx_data2d_grib.h" -#include "vx_data2d_nc_pinterp.h" +#include "vx_data2d_nc_wrf.h" #include "aggr_stat_line.h" #include "parse_stat_line.h" diff --git a/src/tools/core/wavelet_stat/Makefile.am b/src/tools/core/wavelet_stat/Makefile.am index 398ce49dcf..89ed171482 100644 --- a/src/tools/core/wavelet_stat/Makefile.am +++ b/src/tools/core/wavelet_stat/Makefile.am @@ -30,8 +30,8 @@ wavelet_stat_LDADD = -lvx_pxm \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/core/wavelet_stat/Makefile.in b/src/tools/core/wavelet_stat/Makefile.in index bb37d28807..96490833a2 100644 --- a/src/tools/core/wavelet_stat/Makefile.in +++ b/src/tools/core/wavelet_stat/Makefile.in @@ -351,8 +351,8 @@ wavelet_stat_LDADD = -lvx_pxm \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/dev_utils/Makefile.am b/src/tools/dev_utils/Makefile.am index 6aa78046a9..b4463c983f 100644 --- a/src/tools/dev_utils/Makefile.am +++ b/src/tools/dev_utils/Makefile.am @@ -126,8 +126,8 @@ gen_climo_bin_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/dev_utils/Makefile.in b/src/tools/dev_utils/Makefile.in index 5ddb1db20b..e9b745a97f 100644 --- a/src/tools/dev_utils/Makefile.in +++ b/src/tools/dev_utils/Makefile.in @@ -580,8 +580,8 @@ gen_climo_bin_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/ascii2nc/Makefile.am b/src/tools/other/ascii2nc/Makefile.am index f4cef08512..0647687561 100644 --- a/src/tools/other/ascii2nc/Makefile.am +++ b/src/tools/other/ascii2nc/Makefile.am @@ -42,8 +42,8 @@ ascii2nc_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/ascii2nc/Makefile.in b/src/tools/other/ascii2nc/Makefile.in index ea6e06e27f..4c364feb94 100644 --- a/src/tools/other/ascii2nc/Makefile.in +++ b/src/tools/other/ascii2nc/Makefile.in @@ -404,8 +404,8 @@ ascii2nc_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/gen_ens_prod/Makefile.am b/src/tools/other/gen_ens_prod/Makefile.am index 75ddde6311..b9b5e8a186 100644 --- a/src/tools/other/gen_ens_prod/Makefile.am +++ b/src/tools/other/gen_ens_prod/Makefile.am @@ -24,8 +24,8 @@ gen_ens_prod_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/gen_ens_prod/Makefile.in b/src/tools/other/gen_ens_prod/Makefile.in index 285f993f59..8586403cb6 100644 --- a/src/tools/other/gen_ens_prod/Makefile.in +++ b/src/tools/other/gen_ens_prod/Makefile.in @@ -345,8 +345,8 @@ gen_ens_prod_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/gen_vx_mask/Makefile.am b/src/tools/other/gen_vx_mask/Makefile.am index c936c2f0f5..cbc2af0337 100644 --- a/src/tools/other/gen_vx_mask/Makefile.am +++ b/src/tools/other/gen_vx_mask/Makefile.am @@ -21,8 +21,8 @@ gen_vx_mask_LDADD = -lvx_shapedata \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/gen_vx_mask/Makefile.in b/src/tools/other/gen_vx_mask/Makefile.in index 638ca649b7..0454fcc726 100644 --- a/src/tools/other/gen_vx_mask/Makefile.in +++ b/src/tools/other/gen_vx_mask/Makefile.in @@ -337,8 +337,8 @@ gen_vx_mask_LDADD = -lvx_shapedata \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/grid_diag/Makefile.am b/src/tools/other/grid_diag/Makefile.am index 8c55950475..59e735bf05 100644 --- a/src/tools/other/grid_diag/Makefile.am +++ b/src/tools/other/grid_diag/Makefile.am @@ -24,8 +24,8 @@ grid_diag_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/grid_diag/Makefile.in b/src/tools/other/grid_diag/Makefile.in index 765eb7ce9c..c93be79561 100644 --- a/src/tools/other/grid_diag/Makefile.in +++ b/src/tools/other/grid_diag/Makefile.in @@ -344,8 +344,8 @@ grid_diag_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/gsi_tools/Makefile.am b/src/tools/other/gsi_tools/Makefile.am index 54cf2cb4da..9afe2e37a1 100644 --- a/src/tools/other/gsi_tools/Makefile.am +++ b/src/tools/other/gsi_tools/Makefile.am @@ -41,8 +41,8 @@ gsid2mpr_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ @@ -92,8 +92,8 @@ gsidens2orank_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/gsi_tools/Makefile.in b/src/tools/other/gsi_tools/Makefile.in index 37a6a37ebf..c7f068cfd9 100644 --- a/src/tools/other/gsi_tools/Makefile.in +++ b/src/tools/other/gsi_tools/Makefile.in @@ -396,8 +396,8 @@ gsid2mpr_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ @@ -448,8 +448,8 @@ gsidens2orank_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/ioda2nc/Makefile.am b/src/tools/other/ioda2nc/Makefile.am index d393417c51..9694454dac 100644 --- a/src/tools/other/ioda2nc/Makefile.am +++ b/src/tools/other/ioda2nc/Makefile.am @@ -23,8 +23,8 @@ ioda2nc_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/ioda2nc/Makefile.in b/src/tools/other/ioda2nc/Makefile.in index a4b16c47aa..37f335deae 100644 --- a/src/tools/other/ioda2nc/Makefile.in +++ b/src/tools/other/ioda2nc/Makefile.in @@ -356,8 +356,8 @@ ioda2nc_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/lidar2nc/Makefile.am b/src/tools/other/lidar2nc/Makefile.am index 5692fe6373..58f35de02a 100644 --- a/src/tools/other/lidar2nc/Makefile.am +++ b/src/tools/other/lidar2nc/Makefile.am @@ -24,7 +24,7 @@ lidar2nc_LDADD = -lvx_shapedata \ -lvx_analysis_util \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/lidar2nc/Makefile.in b/src/tools/other/lidar2nc/Makefile.in index 2f6da2bc07..e5d3703780 100644 --- a/src/tools/other/lidar2nc/Makefile.in +++ b/src/tools/other/lidar2nc/Makefile.in @@ -358,7 +358,7 @@ lidar2nc_LDADD = -lvx_shapedata \ -lvx_analysis_util \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/madis2nc/Makefile.am b/src/tools/other/madis2nc/Makefile.am index 0cfef882c0..28da7d16cb 100644 --- a/src/tools/other/madis2nc/Makefile.am +++ b/src/tools/other/madis2nc/Makefile.am @@ -22,8 +22,8 @@ madis2nc_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/madis2nc/Makefile.in b/src/tools/other/madis2nc/Makefile.in index 7055ee7a03..c6eb822fc3 100644 --- a/src/tools/other/madis2nc/Makefile.in +++ b/src/tools/other/madis2nc/Makefile.in @@ -341,8 +341,8 @@ madis2nc_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/mode_time_domain/Makefile.am b/src/tools/other/mode_time_domain/Makefile.am index 93fed7d4bb..34ad4461ee 100644 --- a/src/tools/other/mode_time_domain/Makefile.am +++ b/src/tools/other/mode_time_domain/Makefile.am @@ -53,8 +53,8 @@ mtd_LDADD = -lvx_pxm \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/mode_time_domain/Makefile.in b/src/tools/other/mode_time_domain/Makefile.in index 332ca93ac2..3f3d8b446a 100644 --- a/src/tools/other/mode_time_domain/Makefile.in +++ b/src/tools/other/mode_time_domain/Makefile.in @@ -413,8 +413,8 @@ mtd_LDADD = -lvx_pxm \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/modis_regrid/Makefile.am b/src/tools/other/modis_regrid/Makefile.am index 75b2bfaccc..0bb3fc7cb5 100644 --- a/src/tools/other/modis_regrid/Makefile.am +++ b/src/tools/other/modis_regrid/Makefile.am @@ -32,11 +32,11 @@ modis_regrid_LDADD = -lvx_pxm \ -lvx_analysis_util \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/modis_regrid/Makefile.in b/src/tools/other/modis_regrid/Makefile.in index 65c244e36a..68ff194a77 100644 --- a/src/tools/other/modis_regrid/Makefile.in +++ b/src/tools/other/modis_regrid/Makefile.in @@ -361,11 +361,11 @@ modis_regrid_LDADD = -lvx_pxm \ -lvx_analysis_util \ -lvx_data2d_factory \ -lvx_data2d_nc_met \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_cf \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/pb2nc/Makefile.am b/src/tools/other/pb2nc/Makefile.am index 5de5fc4210..f898c62c60 100644 --- a/src/tools/other/pb2nc/Makefile.am +++ b/src/tools/other/pb2nc/Makefile.am @@ -40,8 +40,8 @@ pb2nc_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/pb2nc/Makefile.in b/src/tools/other/pb2nc/Makefile.in index 06d60500ae..05629adb61 100644 --- a/src/tools/other/pb2nc/Makefile.in +++ b/src/tools/other/pb2nc/Makefile.in @@ -383,8 +383,8 @@ pb2nc_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/plot_data_plane/Makefile.am b/src/tools/other/plot_data_plane/Makefile.am index 5251524faa..bbbb6a3672 100644 --- a/src/tools/other/plot_data_plane/Makefile.am +++ b/src/tools/other/plot_data_plane/Makefile.am @@ -18,8 +18,8 @@ plot_data_plane_LDADD = -lvx_data2d_factory \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/plot_data_plane/Makefile.in b/src/tools/other/plot_data_plane/Makefile.in index eb0af931e5..fcf2a525cf 100644 --- a/src/tools/other/plot_data_plane/Makefile.in +++ b/src/tools/other/plot_data_plane/Makefile.in @@ -337,8 +337,8 @@ plot_data_plane_LDADD = -lvx_data2d_factory \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/plot_data_plane/plot_data_plane.cc b/src/tools/other/plot_data_plane/plot_data_plane.cc index 646e51066f..d5a46be2ee 100644 --- a/src/tools/other/plot_data_plane/plot_data_plane.cc +++ b/src/tools/other/plot_data_plane/plot_data_plane.cc @@ -49,7 +49,7 @@ using namespace std; #include "vx_data2d_factory.h" #include "vx_data2d_grib.h" #include "vx_data2d_nc_met.h" -#include "vx_data2d_nc_pinterp.h" +#include "vx_data2d_nc_wrf.h" #include "vx_util.h" #include "vx_cal.h" #include "vx_grid.h" diff --git a/src/tools/other/plot_point_obs/Makefile.am b/src/tools/other/plot_point_obs/Makefile.am index 17dbb404e0..b4255f71a5 100644 --- a/src/tools/other/plot_point_obs/Makefile.am +++ b/src/tools/other/plot_point_obs/Makefile.am @@ -20,8 +20,8 @@ plot_point_obs_LDADD = -lvx_statistics \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/plot_point_obs/Makefile.in b/src/tools/other/plot_point_obs/Makefile.in index c5e166b45c..18b81a273c 100644 --- a/src/tools/other/plot_point_obs/Makefile.in +++ b/src/tools/other/plot_point_obs/Makefile.in @@ -341,8 +341,8 @@ plot_point_obs_LDADD = -lvx_statistics \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/point2grid/Makefile.am b/src/tools/other/point2grid/Makefile.am index 17b0987b89..4eb9f6e374 100644 --- a/src/tools/other/point2grid/Makefile.am +++ b/src/tools/other/point2grid/Makefile.am @@ -20,8 +20,8 @@ point2grid_LDADD = -lvx_statistics \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/point2grid/Makefile.in b/src/tools/other/point2grid/Makefile.in index 35e9722171..17f72484ab 100644 --- a/src/tools/other/point2grid/Makefile.in +++ b/src/tools/other/point2grid/Makefile.in @@ -352,8 +352,8 @@ point2grid_LDADD = -lvx_statistics \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/regrid_data_plane/Makefile.am b/src/tools/other/regrid_data_plane/Makefile.am index e6ed355083..8d61f70608 100644 --- a/src/tools/other/regrid_data_plane/Makefile.am +++ b/src/tools/other/regrid_data_plane/Makefile.am @@ -19,8 +19,8 @@ regrid_data_plane_LDADD = -lvx_statistics \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/regrid_data_plane/Makefile.in b/src/tools/other/regrid_data_plane/Makefile.in index 1cf17714f2..61c475b1ae 100644 --- a/src/tools/other/regrid_data_plane/Makefile.in +++ b/src/tools/other/regrid_data_plane/Makefile.in @@ -339,8 +339,8 @@ regrid_data_plane_LDADD = -lvx_statistics \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/regrid_data_plane/regrid_data_plane.cc b/src/tools/other/regrid_data_plane/regrid_data_plane.cc index 6b3076b029..a37cda1803 100644 --- a/src/tools/other/regrid_data_plane/regrid_data_plane.cc +++ b/src/tools/other/regrid_data_plane/regrid_data_plane.cc @@ -329,10 +329,7 @@ void process_data_file() { if(VarNameSA.n() == 0) { vname << cs_erase << vinfo->name_attr(); if(vinfo->level().type() != LevelType_Accum && - ftype != FileType_NcMet && - ftype != FileType_General_Netcdf && - ftype != FileType_NcPinterp && - ftype != FileType_NcCF) { + !is_netcdf_grdfiletype(ftype)) { vname << "_" << vinfo->level_attr(); } } diff --git a/src/tools/other/shift_data_plane/Makefile.am b/src/tools/other/shift_data_plane/Makefile.am index d4854a1f75..badfdb6922 100644 --- a/src/tools/other/shift_data_plane/Makefile.am +++ b/src/tools/other/shift_data_plane/Makefile.am @@ -19,8 +19,8 @@ shift_data_plane_LDADD = -lvx_statistics \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/shift_data_plane/Makefile.in b/src/tools/other/shift_data_plane/Makefile.in index 6c53dfcdc4..4eb7b7f50b 100644 --- a/src/tools/other/shift_data_plane/Makefile.in +++ b/src/tools/other/shift_data_plane/Makefile.in @@ -339,8 +339,8 @@ shift_data_plane_LDADD = -lvx_statistics \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/other/shift_data_plane/shift_data_plane.cc b/src/tools/other/shift_data_plane/shift_data_plane.cc index 8402c0d2d8..4ceeaa6a41 100644 --- a/src/tools/other/shift_data_plane/shift_data_plane.cc +++ b/src/tools/other/shift_data_plane/shift_data_plane.cc @@ -313,10 +313,7 @@ void write_netcdf(const DataPlane &dp, const Grid &grid, // Define output variable and attributes cs << cs_erase << vinfo->name_attr(); if(vinfo->level().type() != LevelType_Accum && - ftype != FileType_NcMet && - ftype != FileType_General_Netcdf && - ftype != FileType_NcPinterp && - ftype != FileType_NcCF) { + !is_netcdf_grdfiletype(ftype)) { cs << "_" << vinfo->level_attr(); } diff --git a/src/tools/other/wwmca_tool/Makefile.am b/src/tools/other/wwmca_tool/Makefile.am index 73603061ad..6731f3af4c 100644 --- a/src/tools/other/wwmca_tool/Makefile.am +++ b/src/tools/other/wwmca_tool/Makefile.am @@ -42,8 +42,8 @@ wwmca_regrid_LDADD = -lvx_pxm \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ @@ -91,8 +91,8 @@ wwmca_plot_LDADD = -lvx_pxm \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/other/wwmca_tool/Makefile.in b/src/tools/other/wwmca_tool/Makefile.in index f11d548391..972b3f9255 100644 --- a/src/tools/other/wwmca_tool/Makefile.in +++ b/src/tools/other/wwmca_tool/Makefile.in @@ -421,8 +421,8 @@ wwmca_regrid_LDADD = -lvx_pxm \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ @@ -471,8 +471,8 @@ wwmca_plot_LDADD = -lvx_pxm \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_data2d \ diff --git a/src/tools/tc_utils/rmw_analysis/Makefile.am b/src/tools/tc_utils/rmw_analysis/Makefile.am index 46dfff3df2..2ebbe9a79e 100644 --- a/src/tools/tc_utils/rmw_analysis/Makefile.am +++ b/src/tools/tc_utils/rmw_analysis/Makefile.am @@ -24,8 +24,8 @@ rmw_analysis_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/tc_utils/rmw_analysis/Makefile.in b/src/tools/tc_utils/rmw_analysis/Makefile.in index 96b646a99a..9a89e51057 100644 --- a/src/tools/tc_utils/rmw_analysis/Makefile.in +++ b/src/tools/tc_utils/rmw_analysis/Makefile.in @@ -345,8 +345,8 @@ rmw_analysis_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/tc_utils/tc_diag/Makefile.am b/src/tools/tc_utils/tc_diag/Makefile.am index 111277e4e8..4553fc0c8c 100644 --- a/src/tools/tc_utils/tc_diag/Makefile.am +++ b/src/tools/tc_utils/tc_diag/Makefile.am @@ -25,8 +25,8 @@ tc_diag_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/tc_utils/tc_diag/Makefile.in b/src/tools/tc_utils/tc_diag/Makefile.in index 89c097ddb9..b884c28f85 100644 --- a/src/tools/tc_utils/tc_diag/Makefile.in +++ b/src/tools/tc_utils/tc_diag/Makefile.in @@ -348,8 +348,8 @@ tc_diag_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/tc_utils/tc_gen/Makefile.am b/src/tools/tc_utils/tc_gen/Makefile.am index 2cc13426ff..3d9bc7003e 100644 --- a/src/tools/tc_utils/tc_gen/Makefile.am +++ b/src/tools/tc_utils/tc_gen/Makefile.am @@ -25,8 +25,8 @@ tc_gen_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/tc_utils/tc_gen/Makefile.in b/src/tools/tc_utils/tc_gen/Makefile.in index 263c36b038..97b0b5994f 100644 --- a/src/tools/tc_utils/tc_gen/Makefile.in +++ b/src/tools/tc_utils/tc_gen/Makefile.in @@ -346,8 +346,8 @@ tc_gen_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/tc_utils/tc_pairs/Makefile.am b/src/tools/tc_utils/tc_pairs/Makefile.am index c44cd9b3ca..c4ccfa7d43 100644 --- a/src/tools/tc_utils/tc_pairs/Makefile.am +++ b/src/tools/tc_utils/tc_pairs/Makefile.am @@ -27,8 +27,8 @@ tc_pairs_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/tc_utils/tc_pairs/Makefile.in b/src/tools/tc_utils/tc_pairs/Makefile.in index ef0f4a5583..736cb2afe3 100644 --- a/src/tools/tc_utils/tc_pairs/Makefile.in +++ b/src/tools/tc_utils/tc_pairs/Makefile.in @@ -372,8 +372,8 @@ tc_pairs_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/tc_utils/tc_rmw/Makefile.am b/src/tools/tc_utils/tc_rmw/Makefile.am index 4bb5c11475..d1c8d9dbd1 100644 --- a/src/tools/tc_utils/tc_rmw/Makefile.am +++ b/src/tools/tc_utils/tc_rmw/Makefile.am @@ -25,8 +25,8 @@ tc_rmw_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/tc_utils/tc_rmw/Makefile.in b/src/tools/tc_utils/tc_rmw/Makefile.in index 05d4f518e3..0c5220595a 100644 --- a/src/tools/tc_utils/tc_rmw/Makefile.in +++ b/src/tools/tc_utils/tc_rmw/Makefile.in @@ -348,8 +348,8 @@ tc_rmw_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/tc_utils/tc_stat/Makefile.am b/src/tools/tc_utils/tc_stat/Makefile.am index 8a53668656..5f1661cf5c 100644 --- a/src/tools/tc_utils/tc_stat/Makefile.am +++ b/src/tools/tc_utils/tc_stat/Makefile.am @@ -27,8 +27,8 @@ tc_stat_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ diff --git a/src/tools/tc_utils/tc_stat/Makefile.in b/src/tools/tc_utils/tc_stat/Makefile.in index 91e9944932..24bba4f72e 100644 --- a/src/tools/tc_utils/tc_stat/Makefile.in +++ b/src/tools/tc_utils/tc_stat/Makefile.in @@ -350,8 +350,8 @@ tc_stat_LDADD = -lvx_stat_out \ -lvx_data2d_nc_met \ -lvx_data2d_grib \ $(GRIB2_MET_LIBS) \ - -lvx_data2d_nc_pinterp \ - -lvx_data2d_nccf \ + -lvx_data2d_nc_wrf \ + -lvx_data2d_nc_cf \ $(UGRID_MET_LIBS) \ $(PYTHON_MET_LIBS) \ -lvx_statistics \ From d5b4550420d34083fd199afe959efa85bec688bf Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Fri, 19 Jan 2024 19:35:17 +0000 Subject: [PATCH 03/21] #2772 Use JSON for attrubutes and numpy serialization dor 2D data instead of NetCDF --- scripts/python/met/dataplane.py | 153 ++++++++++++-------------------- 1 file changed, 59 insertions(+), 94 deletions(-) diff --git a/scripts/python/met/dataplane.py b/scripts/python/met/dataplane.py index 57c9ac367b..1a8b834c73 100644 --- a/scripts/python/met/dataplane.py +++ b/scripts/python/met/dataplane.py @@ -1,7 +1,8 @@ import os import sys +import json +import re import numpy as np -import netCDF4 as nc import xarray as xr from importlib import util as import_util @@ -19,17 +20,16 @@ class dataplane(logger): @staticmethod def call_python(argv): + # argv[0] is the python wrapper script (caller) logger.log_msg(f"Module:\t{repr(argv[0])}") if 1 == len(argv): - logger.quit(f"User command is missing") + logger.quit_msg(f"User python command is missing") + sys.exit(1) + + logger.log_msg(f"User python command:\t{repr(' '.join(argv[1:]))}") - logger.log_msg("User Command:\t" + repr(' '.join(argv[1:]))) - # argv[0] is the python wrapper script (caller) # argv[1] contains the user defined python script pyembed_module_name = argv[1] - sys.argv = argv[1:] - logger.log_msg(f" sys.argv:\t{sys.argv}") - # append user script dir to system path pyembed_dir, pyembed_name = os.path.split(pyembed_module_name) if pyembed_dir: @@ -40,11 +40,18 @@ def call_python(argv): user_base = pyembed_name.replace('.py','') + argv_org = sys.argv # save sys.argv + sys.argv = argv[1:] spec = import_util.spec_from_file_location(user_base, pyembed_module_name) met_in = import_util.module_from_spec(spec) spec.loader.exec_module(met_in) + sys.argv = argv_org # restore sys.argv return met_in + @staticmethod + def get_tmp_numpy_filename(tmp_filename): + return re.sub(".json$", ".npy", tmp_filename) if tmp_filename.endswith(".json") else f'{tmp_filename}.npy' + @staticmethod def is_integer(a_data): return isinstance(a_data, int) @@ -100,76 +107,33 @@ def read_2d_text_input(input_file): return met_data @staticmethod - def read_dataplane(netcdf_filename): - # read NetCDF file - ds = nc.Dataset(netcdf_filename, 'r') - - dp = ds['met_data'] - met_data = dp[:] - attr_name = dataplane.ATTR_USER_FILL_VALUE - user_fill_value = dp.getncattr(attr_name) if hasattr(dp, attr_name) else None - - met_attrs = {} - - # grid is defined as a dictionary or string - grid = {} - for attr, attr_val in ds.__dict__.items(): - if 'grid.' in attr: - grid_attr = attr.split('.')[1] - grid[grid_attr] = attr_val - else: - met_attrs[attr] = attr_val - - if grid: - met_attrs['grid'] = grid - - met_attrs['name'] = met_attrs['name_str'] - del met_attrs['name_str'] - + def read_dataplane(tmp_filename): met_info = {} - met_info['met_data'] = met_data - if user_fill_value is not None: - met_attrs['fill_value'] = user_fill_value - met_info['attrs'] = met_attrs - + with open(tmp_filename) as json_h: + met_info['attrs'] = json.load(json_h) + # read 2D numeric data + numpy_dump_name = dataplane.get_tmp_numpy_filename(tmp_filename) + met_dp_data = np.load(numpy_dump_name) + met_info['met_data'] = met_dp_data return met_info @staticmethod - def write_dataplane(met_in, netcdf_filename): - met_info = {'met_data': met_in.met_data} + def write_dataplane(met_in, tmp_filename): if hasattr(met_in.met_data, 'attrs') and met_in.met_data.attrs: attrs = met_in.met_data.attrs else: attrs = met_in.attrs - met_info['attrs'] = attrs + with open(tmp_filename,'w') as json_h: + json.dump(attrs, json_h) - # write NetCDF file - ds = nc.Dataset(netcdf_filename, 'w') - - # create dimensions and variable - nx, ny = met_in.met_data.shape - ds.createDimension('x', nx) - ds.createDimension('y', ny) - dp = ds.createVariable('met_data', met_in.met_data.dtype, ('x', 'y'), - fill_value=dataplane.MET_FILL_VALUE) - dp[:] = met_in.met_data - - # append attributes - for attr, attr_val in met_info['attrs'].items(): - if attr_val is None: - continue - - if attr == 'name': - setattr(ds, 'name_str', attr_val) - elif attr == 'fill_value': - setattr(dp, dataplane.ATTR_USER_FILL_VALUE, attr_val) - elif type(attr_val) == dict: - for key in attr_val: - setattr(ds, attr + '.' + key, attr_val[key]) - else: - setattr(ds, attr, attr_val) - - ds.close() + if isinstance(met_in.met_data, (np.ma.MaskedArray, np.ma.core.MaskedArray)): + met_dp_data = np.ma.getdata(met_in.met_data, subok=False) + elif isinstance(met_in.met_data, np.ndarray): + met_dp_data = met_in.met_data + else: + met_dp_data = np.array(met_in.met_data) + numpy_dump_name = dataplane.get_tmp_numpy_filename(tmp_filename) + np.save(numpy_dump_name, met_dp_data) @staticmethod def validate_met_data(met_data, fill_value=None): @@ -180,33 +144,34 @@ def validate_met_data(met_data, fill_value=None): from_ndarray = False if met_data is None: logger.quit(f"{method_name} The met_data is None") + sys.exit(1) + + nx, ny = met_data.shape + + met_fill_value = dataplane.MET_FILL_VALUE + if dataplane.is_xarray_dataarray(met_data): + from_xarray = True + attrs = met_data.attrs + met_data = met_data.data + modified_met_data = True + if isinstance(met_data, np.ndarray): + from_ndarray = True + met_data = np.ma.array(met_data) + + if isinstance(met_data, np.ma.MaskedArray): + is_int_data = dataplane.is_integer(met_data[0,0]) or dataplane.is_integer(met_data[int(nx/2),int(ny/2)]) + met_data = np.ma.masked_equal(met_data, float('nan')) + met_data = np.ma.masked_equal(met_data, float('inf')) + if fill_value is not None: + met_data = np.ma.masked_equal(met_data, fill_value) + met_data = met_data.filled(int(met_fill_value) if is_int_data else met_fill_value) + else: + logger.log_msg(f"{method_name} unknown datatype {type(met_data)}") + + if dataplane.KEEP_XARRAY: + return xr.DataArray(met_data,attrs=attrs) if from_xarray else met_data else: - nx, ny = met_data.shape - - met_fill_value = dataplane.MET_FILL_VALUE - if dataplane.is_xarray_dataarray(met_data): - from_xarray = True - attrs = met_data.attrs - met_data = met_data.data - modified_met_data = True - if isinstance(met_data, np.ndarray): - from_ndarray = True - met_data = np.ma.array(met_data) - - if isinstance(met_data, np.ma.MaskedArray): - is_int_data = dataplane.is_integer(met_data[0,0]) or dataplane.is_integer(met_data[int(nx/2),int(ny/2)]) - met_data = np.ma.masked_equal(met_data, float('nan')) - met_data = np.ma.masked_equal(met_data, float('inf')) - if fill_value is not None: - met_data = np.ma.masked_equal(met_data, fill_value) - met_data = met_data.filled(int(met_fill_value) if is_int_data else met_fill_value) - else: - logger.log_msg(f"{method_name} unknown datatype {type(met_data)}") - - if dataplane.KEEP_XARRAY: - return xr.DataArray(met_data,attrs=attrs) if from_xarray else met_data - else: - return met_data + return met_data def main(argv): From 6ca151131ab743c6efac51a6645bb464473b594b Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Mon, 29 Jan 2024 22:03:33 +0000 Subject: [PATCH 04/21] #2772 Initial release, Separated from point.py --- scripts/python/met/point_nc.py | 293 +++++++++++++++++++++++++++++++++ 1 file changed, 293 insertions(+) create mode 100644 scripts/python/met/point_nc.py diff --git a/scripts/python/met/point_nc.py b/scripts/python/met/point_nc.py new file mode 100644 index 0000000000..37063bdb0d --- /dev/null +++ b/scripts/python/met/point_nc.py @@ -0,0 +1,293 @@ +''' +Created on Jan 10, 2024 + +@author: hsoh + +- This is a derived class to support a NetCDF format data. Separated from point.py +- The potential risk with the netCDF python package is the NetCDF library conflicts beteen MET and python3. + +''' + +import os + +import numpy as np +import netCDF4 as nc + +from met.point import met_point_obs, met_point_tools + +DO_PRINT_DATA = False +ARG_PRINT_DATA = "print_data" + + +def get_nc_point_obs(): + return nc_point_obs() + + +# Note: caller should import netCDF4 +# The argements nc_group(dataset) and nc_var should not be None +class met_point_nc_tools(met_point_tools): + + #met_missing = -99999999. + + @staticmethod + def get_num_array(nc_group, var_name): + nc_var = nc_group.variables.get(var_name, None) + return [] if nc_var is None else nc_var[:] + + @staticmethod + def get_nc_point_obs(): + return nc_point_obs() + + @staticmethod + def get_ncbyte_array_to_str(nc_var): + nc_str_data = nc_var[:] + if nc_var.datatype.name == 'bytes8': + nc_str_data = [ str(s.compressed(),"utf-8") for s in nc_var[:] ] + return nc_str_data + + @staticmethod + def get_string_array(nc_group, var_name): + nc_var = nc_group.variables.get(var_name, None) + return [] if nc_var is None else met_point_nc_tools.get_ncbyte_array_to_str(nc_var) + + +class nc_point_obs(met_point_obs): + + # args should be string, list, or dictionary + def get_nc_filename(self, args): + nc_filename = None + if isinstance(args, dict): + nc_filename = args.get('nc_name',None) + elif isinstance(args, list): + nc_filename = args[0] + elif args != ARG_PRINT_DATA: + nc_filename = args + + return nc_filename + + def read_data(self, nc_filename): + method_name = f"{self.__class__.__name__}.read_data()" + if nc_filename is None: + self.log_error_msg(f"{method_name} The input NetCDF filename is missing") + elif not os.path.exists(nc_filename): + self.log_error_msg(f"{method_name} input NetCDF file ({nc_filename}) does not exist") + else: + dataset = nc.Dataset(nc_filename, 'r') + + attr_name = 'use_var_id' + use_var_id_str = dataset.getncattr(attr_name) if attr_name in dataset.ncattrs() else "false" + self.use_var_id = use_var_id_str.lower() == 'true' + + # Header + self.hdr_typ = dataset['hdr_typ'][:] + self.hdr_sid = dataset['hdr_sid'][:] + self.hdr_vld = dataset['hdr_vld'][:] + self.hdr_lat = dataset['hdr_lat'][:] + self.hdr_lon = dataset['hdr_lon'][:] + self.hdr_elv = dataset['hdr_elv'][:] + self.hdr_typ_table = met_point_nc_tools.get_string_array(dataset, 'hdr_typ_table') + self.hdr_sid_table = met_point_nc_tools.get_string_array(dataset, 'hdr_sid_table') + self.hdr_vld_table = met_point_nc_tools.get_string_array(dataset, 'hdr_vld_table') + + nc_var = dataset.variables.get('obs_unit', None) + if nc_var: + self.obs_var_unit = nc_var[:] + nc_var = dataset.variables.get('obs_desc', None) + if nc_var: + self.obs_var_desc = nc_var[:] + + nc_var = dataset.variables.get('hdr_prpt_typ', None) + if nc_var: + self.hdr_prpt_typ = nc_var[:] + nc_var = dataset.variables.get('hdr_irpt_typ', None) + if nc_var: + self.hdr_irpt_typ = nc_var[:] + nc_var = dataset.variables.get('hdr_inst_typ', None) + if nc_var: + self.hdr_inst_typ =nc_var[:] + + #Observation data + self.hdr_sid = dataset['hdr_sid'][:] + self.obs_qty = np.array(dataset['obs_qty'][:]) + self.obs_hid = np.array(dataset['obs_hid'][:]) + self.obs_lvl = np.array(dataset['obs_lvl'][:]) + self.obs_hgt = np.array(dataset['obs_hgt'][:]) + self.obs_val = np.array(dataset['obs_val'][:]) + nc_var = dataset.variables.get('obs_vid', None) + if nc_var is None: + self.use_var_id = False + nc_var = dataset.variables.get('obs_gc', None) + else: + self.obs_var_table = met_point_nc_tools.get_string_array(dataset, 'obs_var') + if nc_var: + self.obs_vid = np.array(nc_var[:]) + + self.obs_qty_table = met_point_nc_tools.get_string_array(dataset, 'obs_qty_table') + + def save_ncfile(self, nc_filename): + met_data = self.get_point_data() + with nc.Dataset(nc_filename, 'w') as nc_dataset: + self.set_nc_data(nc_dataset) + return met_data + + def set_nc_data(self, nc_dataset): + return nc_point_obs.write_nc_data(nc_dataset, self) + + @staticmethod + def write_nc_file(nc_filename, point_obs): + with nc.Dataset(nc_filename, 'w') as nc_dataset: + nc_point_obs.write_nc_data(nc_dataset, point_obs) + + @staticmethod + def write_nc_data(nc_dataset, point_obs): + method_name = "point_nc.write_nc_data()" + try: + do_nothing = False + if 0 == point_obs.nhdr: + do_nothing = True + met_point_obs.info_message(f"{method_name} the header is empty") + if 0 == point_obs.nobs: + do_nothing = True + met_point_obs.info_message(f"{method_name} the observation data is empty") + if do_nothing: + print() + return + + # Set global attributes + nc_dataset.MET_Obs_version = "1.02" ; + nc_dataset.use_var_id = "true" if point_obs.use_var_id else "false" + + # Create dimensions + nc_dataset.createDimension('mxstr', 16) + nc_dataset.createDimension('mxstr2', 40) + nc_dataset.createDimension('mxstr3', 80) + nc_dataset.createDimension('nhdr', point_obs.nhdr) + nc_dataset.createDimension('nobs', point_obs.nobs) + #npbhdr = len(point_obs.hdr_prpt_typ) + if 0 < point_obs.npbhdr: + nc_dataset.createDimension('npbhdr', point_obs.npbhdr) + nc_dataset.createDimension('nhdr_typ', point_obs.nhdr_typ) + nc_dataset.createDimension('nhdr_sid', point_obs.nhdr_sid) + nc_dataset.createDimension('nhdr_vld', point_obs.nhdr_vld) + nc_dataset.createDimension('nobs_qty', point_obs.nobs_qty) + nc_dataset.createDimension('obs_var_num', point_obs.nobs_var) + + type_for_string = 'S1' # np.byte + dims_hdr = ('nhdr',) + dims_obs = ('nobs',) + + # Create header and observation variables + var_hdr_typ = nc_dataset.createVariable('hdr_typ', np.int32, dims_hdr, fill_value=-9999) + var_hdr_sid = nc_dataset.createVariable('hdr_sid', np.int32, dims_hdr, fill_value=-9999) + var_hdr_vld = nc_dataset.createVariable('hdr_vld', np.int32, dims_hdr, fill_value=-9999) + var_hdr_lat = nc_dataset.createVariable('hdr_lat', np.float32, dims_hdr, fill_value=-9999.) + var_hdr_lon = nc_dataset.createVariable('hdr_lon', np.float32, dims_hdr, fill_value=-9999.) + var_hdr_elv = nc_dataset.createVariable('hdr_elv', np.float32, dims_hdr, fill_value=-9999.) + + var_obs_qty = nc_dataset.createVariable('obs_qty', np.int32, dims_obs, fill_value=-9999) + var_obs_hid = nc_dataset.createVariable('obs_hid', np.int32, dims_obs, fill_value=-9999) + var_obs_vid = nc_dataset.createVariable('obs_vid', np.int32, dims_obs, fill_value=-9999) + var_obs_lvl = nc_dataset.createVariable('obs_lvl', np.float32, dims_obs, fill_value=-9999.) + var_obs_hgt = nc_dataset.createVariable('obs_hgt', np.float32, dims_obs, fill_value=-9999.) + var_obs_val = nc_dataset.createVariable('obs_val', np.float32, dims_obs, fill_value=-9999.) + + if 0 == point_obs.npbhdr: + var_hdr_prpt_typ = None + var_hdr_irpt_typ = None + var_hdr_inst_typ = None + else: + dims_npbhdr = ('npbhdr',) + var_hdr_prpt_typ = nc_dataset.createVariable('hdr_prpt_typ', np.int32, dims_npbhdr, fill_value=-9999.) + var_hdr_irpt_typ = nc_dataset.createVariable('hdr_irpt_typ', np.int32, dims_npbhdr, fill_value=-9999.) + var_hdr_inst_typ = nc_dataset.createVariable('hdr_inst_typ', np.int32, dims_npbhdr, fill_value=-9999.) + + var_hdr_typ_table = nc_dataset.createVariable('hdr_typ_table', type_for_string, ('nhdr_typ','mxstr2')) + var_hdr_sid_table = nc_dataset.createVariable('hdr_sid_table', type_for_string, ('nhdr_sid','mxstr2')) + var_hdr_vld_table = nc_dataset.createVariable('hdr_vld_table', type_for_string, ('nhdr_vld','mxstr')) + var_obs_qty_table = nc_dataset.createVariable('obs_qty_table', type_for_string, ('nobs_qty','mxstr')) + var_obs_var_table = nc_dataset.createVariable('obs_var', type_for_string, ('obs_var_num','mxstr2')) + var_obs_var_unit = nc_dataset.createVariable('obs_unit', type_for_string, ('obs_var_num','mxstr2')) + var_obs_var_desc = nc_dataset.createVariable('obs_desc', type_for_string, ('obs_var_num','mxstr3')) + + # Set variables + var_hdr_typ[:] = point_obs.hdr_typ[:] + var_hdr_sid[:] = point_obs.hdr_sid[:] + var_hdr_vld[:] = point_obs.hdr_vld[:] + var_hdr_lat[:] = point_obs.hdr_lat[:] + var_hdr_lon[:] = point_obs.hdr_lon[:] + var_hdr_elv[:] = point_obs.hdr_elv[:] + for i in range(0, point_obs.nhdr_typ): + for j in range(0, len(point_obs.hdr_typ_table[i])): + var_hdr_typ_table[i,j] = point_obs.hdr_typ_table[i][j] + for i in range(0, point_obs.nhdr_sid): + for j in range(0, len(point_obs.hdr_sid_table[i])): + var_hdr_sid_table[i,j] = point_obs.hdr_sid_table[i][j] + for i in range(0, point_obs.nhdr_vld): + for j in range(0, len(point_obs.hdr_vld_table[i])): + var_hdr_vld_table[i,j] = point_obs.hdr_vld_table[i][j] + if 0 < point_obs.npbhdr: + var_hdr_prpt_typ[:] = point_obs.hdr_prpt_typ[:] + var_hdr_irpt_typ[:] = point_obs.hdr_irpt_typ[:] + var_hdr_inst_typ[:] = point_obs.hdr_inst_typ[:] + + var_obs_qty[:] = point_obs.obs_qty[:] + var_obs_hid[:] = point_obs.obs_hid[:] + var_obs_vid[:] = point_obs.obs_vid[:] + var_obs_lvl[:] = point_obs.obs_lvl[:] + var_obs_hgt[:] = point_obs.obs_hgt[:] + var_obs_val[:] = point_obs.obs_val[:] + for i in range(0, point_obs.nobs_var): + for j in range(0, len(point_obs.obs_var_table[i])): + var_obs_var_table[i,j] = point_obs.obs_var_table[i][j] + var_obs_var_unit[i] = "" if i >= len(point_obs.obs_var_unit) else point_obs.obs_var_unit[i] + var_obs_var_desc[i] = "" if i >= len(point_obs.obs_var_desc) else point_obs.obs_var_desc[i] + for i in range(0, point_obs.nobs_qty): + for j in range(0, len(point_obs.obs_qty_table[i])): + var_obs_qty_table[i,j] = point_obs.obs_qty_table[i][j] + + # Set variable attributes + var_hdr_typ.long_name = "index of message type" + var_hdr_sid.long_name = "index of station identification" + var_hdr_vld.long_name = "index of valid time" + var_hdr_lat.long_name = "latitude" + var_hdr_lat.units = "degrees_north" + var_hdr_lon.long_name = "longitude" + var_hdr_lon.units = "degrees_east" + var_hdr_elv.long_name = "elevation" + var_hdr_elv.units = "meters above sea level (msl)" + + var_obs_qty.long_name = "index of quality flag" + var_obs_hid.long_name = "index of matching header data" + var_obs_vid.long_name = "index of BUFR variable corresponding to the observation type" + var_obs_lvl.long_name = "pressure level (hPa) or accumulation interval (sec)" + var_obs_hgt.long_name = "height in meters above sea level (msl)" + var_obs_val.long_name = "observation value" + var_hdr_typ_table.long_name = "message type" + var_hdr_sid_table.long_name = "station identification" + var_hdr_vld_table.long_name = "valid time" + var_hdr_vld_table.units = "YYYYMMDD_HHMMSS UTC" + var_obs_qty_table.long_name = "quality flag" + var_obs_var_table.long_name = "variable names" + var_obs_var_unit.long_name = "variable units" + var_obs_var_desc.long_name = "variable descriptions" + except: + print(f' === ERROR at {method_name} type(nc_dataset)={type(nc_dataset)} type(point_obs)={type(point_obs)}') + raise + + +def main(argv): + if len(argv) != 1 and argv[1] != ARG_PRINT_DATA: + netcdf_filename = argv[1] + tmp_nc_name = 'tmp_met_point.nc' + point_obs_data = nc_point_obs() + point_obs_data.read_data(point_obs_data.get_nc_filename(netcdf_filename)) + met_point_data = point_obs_data.save_ncfile(tmp_nc_name) + print(f'{met_point_tools.get_prompt()} saved met_point_data to {tmp_nc_name}') + met_point_data['met_point_data'] = point_obs_data + + if DO_PRINT_DATA or ARG_PRINT_DATA == argv[-1]: + point_obs_data.print_point_data(met_point_data) + +if __name__ == '__main__': + main() + print('Done python script') From 355b38477634ef614d4440239ee0a2744c45b405 Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Mon, 29 Jan 2024 22:04:10 +0000 Subject: [PATCH 05/21] #2772 Added point_nc.py --- scripts/python/met/Makefile.am | 1 + scripts/python/met/Makefile.in | 1 + 2 files changed, 2 insertions(+) diff --git a/scripts/python/met/Makefile.am b/scripts/python/met/Makefile.am index fd802449dd..c24b47247b 100644 --- a/scripts/python/met/Makefile.am +++ b/scripts/python/met/Makefile.am @@ -28,6 +28,7 @@ pythonmetscripts_DATA = \ logger.py \ dataplane.py \ mprbase.py \ + point_nc.py \ point.py EXTRA_DIST = ${pythonmetscripts_DATA} diff --git a/scripts/python/met/Makefile.in b/scripts/python/met/Makefile.in index fea84eace6..6a7570efa2 100644 --- a/scripts/python/met/Makefile.in +++ b/scripts/python/met/Makefile.in @@ -314,6 +314,7 @@ pythonmetscripts_DATA = \ logger.py \ dataplane.py \ mprbase.py \ + point_nc.py \ point.py EXTRA_DIST = ${pythonmetscripts_DATA} From 118a2becfca344d2a04466fb668cb8cd6b510435 Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Mon, 29 Jan 2024 22:26:07 +0000 Subject: [PATCH 06/21] #2772 Changed write_tmp_nc and read_tmp_nc to write_tmp_py and read_tmp_py --- .../vx_data2d_python/python_dataplane.cc | 36 +++++++++---------- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/src/libcode/vx_data2d_python/python_dataplane.cc b/src/libcode/vx_data2d_python/python_dataplane.cc index 118f2b45aa..1c5c0a1e31 100644 --- a/src/libcode/vx_data2d_python/python_dataplane.cc +++ b/src/libcode/vx_data2d_python/python_dataplane.cc @@ -31,9 +31,9 @@ extern GlobalPython GP; // this needs external linkage static const char * user_ppath = nullptr; -static const char write_tmp_nc [] = "MET_BASE/python/pyembed/write_tmp_dataplane.py"; +static const char write_tmp_py [] = "MET_BASE/python/pyembed/write_tmp_dataplane.py"; -static const char read_tmp_nc [] = "MET_BASE/python/pyembed/read_tmp_dataplane.py"; +static const char read_tmp_py [] = "MET_BASE/python/pyembed/read_tmp_dataplane.py"; static const char tmp_nc_var_name [] = "met_info"; @@ -48,10 +48,10 @@ static bool straight_python_dataplane(const char * script_name, Grid & met_grid_out, VarInfoPython &vinfo); -static bool tmp_nc_dataplane(const char * script_name, - int script_argc, char ** script_argv, - const bool use_xarray, DataPlane & met_dp_out, - Grid & met_grid_out, VarInfoPython &vinfo); +static bool tmp_dataplane(const char * script_name, + int script_argc, char ** script_argv, + const bool use_xarray, DataPlane & met_dp_out, + Grid & met_grid_out, VarInfoPython &vinfo); //////////////////////////////////////////////////////////////////////// @@ -83,10 +83,10 @@ bool status = false; if ( (user_ppath = getenv(user_python_path_env)) != 0 ) { // do_tmp_nc = true; - status = tmp_nc_dataplane(user_script_name, - user_script_argc, user_script_argv, - use_xarray, met_dp_out, - met_grid_out, vinfo); + status = tmp_dataplane(user_script_name, + user_script_argc, user_script_argv, + use_xarray, met_dp_out, + met_grid_out, vinfo); } else { @@ -309,10 +309,10 @@ return ( true ); //////////////////////////////////////////////////////////////////////// -bool tmp_nc_dataplane(const char * user_script_name, - int user_script_argc, char ** user_script_argv, - const bool use_xarray, DataPlane & met_dp_out, - Grid & met_grid_out, VarInfoPython &vinfo) +bool tmp_dataplane(const char * user_script_name, + int user_script_argc, char ** user_script_argv, + const bool use_xarray, DataPlane & met_dp_out, + Grid & met_grid_out, VarInfoPython &vinfo) { @@ -334,13 +334,13 @@ if ( ! tmp_dir ) tmp_dir = default_tmp_dir; path << cs_erase << tmp_dir << '/' - << tmp_nc_base_name; + << tmp_py_base_name; tmp_nc_path = make_temp_file_name(path.text(), 0); command << cs_erase << user_ppath << ' ' // user's path to python - << replace_path(write_tmp_nc) << ' ' // write_tmp_nc.py + << replace_path(write_tmp_py) << ' ' // write_tmp_nc.py << tmp_nc_path << ' ' // tmp_nc output filename << user_script_name; // user's script name @@ -397,7 +397,7 @@ StringArray a; a.add(validate_dataplane); -a.add(replace_path(read_tmp_nc)); +a.add(replace_path(read_tmp_py)); a.add(tmp_nc_path); @@ -412,7 +412,7 @@ mlog << Debug(4) << "Reading temporary Python dataplane file: " // import the python wrapper script as a module // -//path = get_short_name(read_tmp_nc); +//path = get_short_name(read_tmp_py); path = get_short_name(validate_dataplane); PyObject * module_obj = PyImport_ImportModule (path.text()); From 1cba5ec3e1cc09cc95fba40ce1193c1bcbfa56ff Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Mon, 29 Jan 2024 22:29:06 +0000 Subject: [PATCH 07/21] #2772 Removed python_key_point_data & python_key_point_data_list and replaced them to tmp_point_var_name and tmp_point_data --- .../vx_pointdata_python/python_pointdata.cc | 14 +++++++------- src/libcode/vx_pointdata_python/python_pointdata.h | 3 --- 2 files changed, 7 insertions(+), 10 deletions(-) diff --git a/src/libcode/vx_pointdata_python/python_pointdata.cc b/src/libcode/vx_pointdata_python/python_pointdata.cc index 1b34915bf8..acf6ce030c 100644 --- a/src/libcode/vx_pointdata_python/python_pointdata.cc +++ b/src/libcode/vx_pointdata_python/python_pointdata.cc @@ -543,18 +543,18 @@ if ( ! module_obj ) { } bool result = false; -PyObject *met_point_data = get_python_object(module_obj, python_key_point_data); +PyObject *met_point_data = get_python_object(module_obj, tmp_point_var_name); if ( met_point_data && met_point_data != &_Py_NoneStruct) { result = process_point_data(met_point_data, met_pd_out); } else { - PyObject *point_data = get_python_object(module_obj, python_key_point_data_list); + PyObject *point_data = get_python_object(module_obj, tmp_point_data); if ( point_data && point_data != &_Py_NoneStruct) result = process_point_data_list(point_data, met_pd_out, filters); else { mlog << Warning << "\n" << method_name - << "no \"" << python_key_point_data << "\" and \"" - << python_key_point_data_list << "\" from " + << "no \"" << tmp_point_var_name << "\" and \"" + << tmp_point_data << "\" from " << script_name << "\"\n\n"; } } @@ -626,7 +626,7 @@ if ( ! tmp_dir ) tmp_dir = default_tmp_dir; path << cs_erase << tmp_dir << '/' - << tmp_nc_base_name; + << tmp_py_base_name; tmp_nc_path = make_temp_file_name(path.text(), 0); @@ -722,12 +722,12 @@ if ( ! module_obj ) { // -PyObject *met_point_data = get_python_object(module_obj, python_key_point_data); +PyObject *met_point_data = get_python_object(module_obj, tmp_point_var_name); if ( met_point_data ) { process_point_data(met_point_data, met_pd_out); } else { - PyObject *point_data = get_python_object(module_obj, python_key_point_data_list); + PyObject *point_data = get_python_object(module_obj, tmp_point_data); process_point_data_list(point_data, met_pd_out, filters); } diff --git a/src/libcode/vx_pointdata_python/python_pointdata.h b/src/libcode/vx_pointdata_python/python_pointdata.h index 284421b17c..409d2ad6c0 100644 --- a/src/libcode/vx_pointdata_python/python_pointdata.h +++ b/src/libcode/vx_pointdata_python/python_pointdata.h @@ -29,9 +29,6 @@ extern "C" { //////////////////////////////////////////////////////////////////////// -static const char python_key_point_data [] = "met_point_data"; -static const char python_key_point_data_list[] = "point_data"; - static const char python_key_nhdr [] = "nhdr"; //static const char python_key_npbhdr [] = "npbhdr"; static const char python_use_var_id [] = "use_var_id"; From d64d12766ac771fae712c92afab790a8ce5acff8 Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Mon, 29 Jan 2024 22:30:33 +0000 Subject: [PATCH 08/21] #2772 Renamed tmp_nc_base_name, tmp_nc_file_var_name & tmp_nc_point_var_name to tmp_py_base_name, tmp_file_var_name, & tmp_point_var_name --- src/libcode/vx_python3_utils/python3_util.h | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/src/libcode/vx_python3_utils/python3_util.h b/src/libcode/vx_python3_utils/python3_util.h index 33a5ed63b3..9924f9f89f 100644 --- a/src/libcode/vx_python3_utils/python3_util.h +++ b/src/libcode/vx_python3_utils/python3_util.h @@ -34,11 +34,13 @@ static const char wrappers_dir [] = "MET_BASE/wrappers"; static const char python_dir [] = "MET_BASE/python"; -static const char tmp_nc_base_name [] = "tmp_met_nc"; +static const char tmp_py_base_name [] = "tmp_met_data"; -static const char tmp_nc_file_var_name [] = "tmp_nc_filename"; +static const char tmp_file_var_name [] = "tmp_py_filename"; -static const char tmp_nc_point_var_name[] = "met_point_data"; +static const char tmp_point_var_name [] = "met_point_data"; + +static const char tmp_point_data [] = "point_data"; //////////////////////////////////////////////////////////////////////// From de19d84dac3b66a0bc7357b961810f32f974c000 Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Mon, 29 Jan 2024 22:57:18 +0000 Subject: [PATCH 09/21] #2772 More log messages for error --- scripts/python/examples/read_ascii_numpy_grid.py | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/scripts/python/examples/read_ascii_numpy_grid.py b/scripts/python/examples/read_ascii_numpy_grid.py index 79e6829052..7ca2b3b6b6 100644 --- a/scripts/python/examples/read_ascii_numpy_grid.py +++ b/scripts/python/examples/read_ascii_numpy_grid.py @@ -27,8 +27,11 @@ met_data = dataplane.read_2d_text_input(input_file) print("Data Shape:\t" + repr(met_data.shape)) print("Data Type:\t" + repr(met_data.dtype)) -except NameError: - print("Can't find the input file") +except NameError as ex: + print(" === ERROR from read_ascii_numpy_grid.py") + print(f" Exception: {type(ex)} {ex}") + print(f" sys.argv: {sys.argv}") + print(" Can't find the input file") # attrs is a dictionary which contains attributes describing the dataplane. # attrs should have 9 items, each of data type string: From df63d2ad7fe0367557012f4c589a59553dff6d18 Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Mon, 29 Jan 2024 22:58:08 +0000 Subject: [PATCH 10/21] #2772 Changed API (log_msg to log_message) --- scripts/python/examples/read_ascii_numpy.py | 4 ++-- scripts/python/examples/read_ascii_xarray.py | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/python/examples/read_ascii_numpy.py b/scripts/python/examples/read_ascii_numpy.py index 3c6310cec2..5692472b10 100644 --- a/scripts/python/examples/read_ascii_numpy.py +++ b/scripts/python/examples/read_ascii_numpy.py @@ -5,7 +5,7 @@ ########################################### def log(msg): - dataplane.log_msg(msg) + dataplane.log_message(msg) def set_dataplane_attrs(): # attrs is a dictionary which contains attributes describing the dataplane. @@ -95,5 +95,5 @@ def set_dataplane_attrs(): attrs = set_dataplane_attrs() log("Attributes:\t" + repr(attrs)) -# Sets fill_value if it exists +# Sets fill_value if it exists at the dataplane #attrs['fill_value'] = 255 # for letter.txt diff --git a/scripts/python/examples/read_ascii_xarray.py b/scripts/python/examples/read_ascii_xarray.py index e4ba1f9a28..f897982f07 100644 --- a/scripts/python/examples/read_ascii_xarray.py +++ b/scripts/python/examples/read_ascii_xarray.py @@ -6,7 +6,7 @@ ########################################### def log(msg): - dataplane.log_msg(msg) + dataplane.log_message(msg) log("Python Script:\t" + repr(sys.argv[0])) From be02853fc1d827e0099bbb2ad759d4c90fd2fe09 Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Mon, 29 Jan 2024 22:59:24 +0000 Subject: [PATCH 11/21] #2772 Use met_point_nc_tools instead of met_point_tools --- scripts/python/examples/read_met_point_obs.py | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/scripts/python/examples/read_met_point_obs.py b/scripts/python/examples/read_met_point_obs.py index e16ccf2d86..b8c8cb3db3 100644 --- a/scripts/python/examples/read_met_point_obs.py +++ b/scripts/python/examples/read_met_point_obs.py @@ -21,6 +21,7 @@ from datetime import datetime from met.point import met_point_tools +from met.point_nc import met_point_nc_tools from pyembed.python_embedding import pyembed_tools ARG_PRINT_DATA = 'show_data' @@ -44,14 +45,15 @@ netcdf_filename = os.path.expandvars(input_name) args = [ netcdf_filename ] #args = { 'nc_name': netcdf_filename } - point_obs_data = met_point_tools.get_nc_point_obs() + point_obs_data = met_point_nc_tools.get_nc_point_obs() point_obs_data.read_data(point_obs_data.get_nc_filename(args)) if point_obs_data is not None: met_point_data = point_obs_data.get_point_data() met_point_data['met_point_data'] = point_obs_data - print("met_point_data: ", met_point_data) - print(met_point_data) + if os.getenv("MET_PYTHON_DEBUG", "") != "": + print("met_point_data: ", met_point_data) + print(met_point_data) if DO_PRINT_DATA: point_obs_data.dump() From 549ff3dd2aebacb56551375006df10ecf3b929d5 Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Mon, 29 Jan 2024 23:00:09 +0000 Subject: [PATCH 12/21] #2772 Changed APIs --- scripts/python/pyembed/read_tmp_point_nc.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/python/pyembed/read_tmp_point_nc.py b/scripts/python/pyembed/read_tmp_point_nc.py index 622405c520..d3e0f13cb7 100644 --- a/scripts/python/pyembed/read_tmp_point_nc.py +++ b/scripts/python/pyembed/read_tmp_point_nc.py @@ -8,7 +8,7 @@ import sys -from met.point import met_point_tools +from met.point import get_empty_point_obs, met_point_tools try: from python_embedding import pyembed_tools except: @@ -19,8 +19,8 @@ # read NetCDF file print('{p} reading {f}'.format(p=met_point_tools.get_prompt(), f=input_filename)) try: - point_obs_data = met_point_tools.get_nc_point_obs() - point_obs_data.read_data(input_filename) + point_obs_data = get_empty_point_obs() + point_obs_data.read_point_data(input_filename) met_point_data = point_obs_data.get_point_data() met_point_data['met_point_data'] = point_obs_data From 858a484a9be2afb7d5a3f12e82777494671357fd Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Mon, 29 Jan 2024 23:01:43 +0000 Subject: [PATCH 13/21] #2772 Changed API --- scripts/python/pyembed/write_tmp_point_nc.py | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/scripts/python/pyembed/write_tmp_point_nc.py b/scripts/python/pyembed/write_tmp_point_nc.py index 41f380de77..fc828e3237 100644 --- a/scripts/python/pyembed/write_tmp_point_nc.py +++ b/scripts/python/pyembed/write_tmp_point_nc.py @@ -17,20 +17,24 @@ except: from pyembed.python_embedding import pyembed_tools -from met.point import met_point_tools +try: + from point import get_empty_point_obs +except: + from met.point import get_empty_point_obs + if __name__ == '__main__': tmp_filename = sys.argv[1] met_in = pyembed_tools.call_python(sys.argv) - if hasattr(met_in, 'point_data'): + if hasattr(met_in, 'point_obs_data'): + met_in.point_obs_data.write_point_data(tmp_filename) + elif hasattr(met_in, 'point_data'): pyembed_tools.write_tmp_ascii(tmp_filename, met_in.point_data) - elif hasattr(met_in, 'point_obs_data'): - met_in.point_obs_data.save_ncfile(tmp_filename) else: if hasattr(met_in.met_point_data, 'point_obs_data'): - met_in.met_point_data['point_obs_data'].save_ncfile(tmp_filename) + met_in.met_point_data['point_obs_data'].write_point_data(tmp_filename) else: - tmp_point_obs = met_point_tools.get_nc_point_obs() + tmp_point_obs = get_empty_point_obs() tmp_point_obs.put_data(met_in.met_point_data) tmp_point_obs.save_ncfile(tmp_filename) From 6a54b1bbd66113f60e4f62953cbdb789affd3a94 Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Mon, 29 Jan 2024 23:02:58 +0000 Subject: [PATCH 14/21] #2772 Changed default temp output format to JSON ande numpoy serialization --- scripts/python/met/dataplane.py | 168 ++++++-- scripts/python/met/logger.py | 152 ++++++- scripts/python/met/point.py | 703 ++++++++++++-------------------- 3 files changed, 541 insertions(+), 482 deletions(-) diff --git a/scripts/python/met/dataplane.py b/scripts/python/met/dataplane.py index 1a8b834c73..8f3d1ac37c 100644 --- a/scripts/python/met/dataplane.py +++ b/scripts/python/met/dataplane.py @@ -1,32 +1,30 @@ import os import sys import json -import re import numpy as np import xarray as xr from importlib import util as import_util -from met.logger import logger +from met.logger import met_base, met_base_tools ########################################### -class dataplane(logger): +class dataplane(met_base): KEEP_XARRAY = True class_name = "dataplane" - MET_FILL_VALUE = -9999. ATTR_USER_FILL_VALUE = 'user_fill_value' @staticmethod def call_python(argv): # argv[0] is the python wrapper script (caller) - logger.log_msg(f"Module:\t{repr(argv[0])}") + met_base.log_message(f"Module:\t{repr(argv[0])}") if 1 == len(argv): - logger.quit_msg(f"User python command is missing") + met_base.quit_msg(f"User python command is missing") sys.exit(1) - logger.log_msg(f"User python command:\t{repr(' '.join(argv[1:]))}") + met_base.log_message(f"User python command:\t{repr(' '.join(argv[1:]))}") # argv[1] contains the user defined python script pyembed_module_name = argv[1] @@ -48,9 +46,10 @@ def call_python(argv): sys.argv = argv_org # restore sys.argv return met_in - @staticmethod - def get_tmp_numpy_filename(tmp_filename): - return re.sub(".json$", ".npy", tmp_filename) if tmp_filename.endswith(".json") else f'{tmp_filename}.npy' + #@staticmethod + #def get_numpy_filename(tmp_filename): + # return met_base_tools.replace_extension(tmp_filename, "json", "npy") if tmp_filename.endswith(".json") else \ + # met_base_tools.replace_extension(tmp_filename, "nc", "npy") if tmp_filename.endswith(".nc") else f'{tmp_filename}.npy' @staticmethod def is_integer(a_data): @@ -108,47 +107,78 @@ def read_2d_text_input(input_file): @staticmethod def read_dataplane(tmp_filename): + # Default is JSON for attributes and NUMPY serialization for 2D array + return dataplane.read_dataplane_nc(tmp_filename) if met_base_tools.use_netcdf_format() \ + else dataplane.read_dataplane_json_numpy(tmp_filename) + + @staticmethod + def read_dataplane_json_numpy(tmp_filename): + if met_base_tools.is_debug_enabled("dataplane"): + met_base.log_message(f"Read from a temporary JSON file and a temporary numpy output (dataplane)") + met_info = {} - with open(tmp_filename) as json_h: - met_info['attrs'] = json.load(json_h) + with open(tmp_filename) as json_fh: + met_info['attrs'] = json.load(json_fh) # read 2D numeric data - numpy_dump_name = dataplane.get_tmp_numpy_filename(tmp_filename) + numpy_dump_name = met_base_tools.get_numpy_filename(tmp_filename) met_dp_data = np.load(numpy_dump_name) met_info['met_data'] = met_dp_data return met_info @staticmethod - def write_dataplane(met_in, tmp_filename): - if hasattr(met_in.met_data, 'attrs') and met_in.met_data.attrs: - attrs = met_in.met_data.attrs - else: - attrs = met_in.attrs - with open(tmp_filename,'w') as json_h: - json.dump(attrs, json_h) + def read_dataplane_nc(netcdf_filename): + import netCDF4 as nc - if isinstance(met_in.met_data, (np.ma.MaskedArray, np.ma.core.MaskedArray)): - met_dp_data = np.ma.getdata(met_in.met_data, subok=False) - elif isinstance(met_in.met_data, np.ndarray): - met_dp_data = met_in.met_data - else: - met_dp_data = np.array(met_in.met_data) - numpy_dump_name = dataplane.get_tmp_numpy_filename(tmp_filename) - np.save(numpy_dump_name, met_dp_data) + if met_base_tools.is_debug_enabled("dataplane"): + met_base.log_message(f"Read from a temporary NetCDF file (dataplane)") + + # read NetCDF file + ds = nc.Dataset(netcdf_filename, 'r') + + dp = ds['met_data'] + met_data = dp[:] + attr_name = dataplane.ATTR_USER_FILL_VALUE + user_fill_value = dp.getncattr(attr_name) if hasattr(dp, attr_name) else None + + met_attrs = {} + + # grid is defined as a dictionary or string + grid = {} + for attr, attr_val in ds.__dict__.items(): + if 'grid.' in attr: + grid_attr = attr.split('.')[1] + grid[grid_attr] = attr_val + else: + met_attrs[attr] = attr_val + + if grid: + met_attrs['grid'] = grid + + met_attrs['name'] = met_attrs['name_str'] + del met_attrs['name_str'] + + met_info = {} + met_info['met_data'] = met_data + if user_fill_value is not None: + met_attrs['fill_value'] = user_fill_value + met_info['attrs'] = met_attrs + + return met_info @staticmethod def validate_met_data(met_data, fill_value=None): method_name = f"{dataplane.class_name}.validate()" - #logger.log_msg(f"{method_name} type(met_data)= {type(met_data)}") + #met_base.log_msg(f"{method_name} type(met_data)= {type(met_data)}") attrs = None from_xarray = False from_ndarray = False if met_data is None: - logger.quit(f"{method_name} The met_data is None") + met_base.quit(f"{method_name} The met_data is None") sys.exit(1) nx, ny = met_data.shape - met_fill_value = dataplane.MET_FILL_VALUE + met_fill_value = met_base.MET_FILL_VALUE if dataplane.is_xarray_dataarray(met_data): from_xarray = True attrs = met_data.attrs @@ -166,13 +196,79 @@ def validate_met_data(met_data, fill_value=None): met_data = np.ma.masked_equal(met_data, fill_value) met_data = met_data.filled(int(met_fill_value) if is_int_data else met_fill_value) else: - logger.log_msg(f"{method_name} unknown datatype {type(met_data)}") + met_base.log_message(f"{method_name} unknown datatype {type(met_data)}") if dataplane.KEEP_XARRAY: return xr.DataArray(met_data,attrs=attrs) if from_xarray else met_data else: return met_data + @staticmethod + def write_dataplane(met_in, tmp_filename): + # Default is JSON for attributes and NUMPY serialization for 2D array + if met_base_tools.use_netcdf_format(): + dataplane.write_dataplane_nc(met_in, tmp_filename) + else: + dataplane.write_dataplane_json_numpy(met_in, tmp_filename) + + @staticmethod + def write_dataplane_json_numpy(met_in, tmp_filename): + if met_base_tools.is_debug_enabled("dataplane"): + met_base.log_message(f"Save to a temporary JSON file and a temporary numpy output (dataplane)") + if hasattr(met_in.met_data, 'attrs') and met_in.met_data.attrs: + attrs = met_in.met_data.attrs + else: + attrs = met_in.attrs + with open(tmp_filename,'w') as json_fh: + json.dump(attrs, json_fh) + + met_dp_data = met_base_tools.convert_to_ndarray(met_in.met_data) + numpy_dump_name = met_base_tools.get_numpy_filename(tmp_filename) + np.save(numpy_dump_name, met_dp_data) + + @staticmethod + def write_dataplane_nc(met_in, netcdf_filename): + import netCDF4 as nc + + if met_base_tools.is_debug_enabled("dataplane"): + met_base.log_message(f"Save to a temporary NetCDF file (dataplane)") + + met_info = {'met_data': met_in.met_data} + if hasattr(met_in.met_data, 'attrs') and met_in.met_data.attrs: + attrs = met_in.met_data.attrs + else: + attrs = met_in.attrs + met_info['attrs'] = attrs + + # write NetCDF file + ds = nc.Dataset(netcdf_filename, 'w') + + # create dimensions and variable + nx, ny = met_in.met_data.shape + ds.createDimension('x', nx) + ds.createDimension('y', ny) + dp = ds.createVariable('met_data', met_in.met_data.dtype, ('x', 'y'), + fill_value=dataplane.MET_FILL_VALUE) + dp[:] = met_in.met_data + + # append attributes + for attr, attr_val in met_info['attrs'].items(): + if attr_val is None: + continue + + if attr == 'name': + setattr(ds, 'name_str', attr_val) + elif attr == 'fill_value': + setattr(dp, dataplane.ATTR_USER_FILL_VALUE, attr_val) + elif type(attr_val) == dict: + for key in attr_val: + setattr(ds, attr + '.' + key, attr_val[key]) + else: + setattr(ds, attr, attr_val) + + ds.close() + + def main(argv): global attrs, met_data, met_info @@ -196,14 +292,14 @@ def main(argv): fill_value = met_in.user_fill_value fill_value = attrs.get('fill_value', None) - dataplane.log_msg('validating the dataplane array...') + met_base.log_message('validating the dataplane array...') met_data = dataplane.validate_met_data(init_met_data, fill_value) met_info['met_data'] = met_data if os.environ.get('MET_PYTHON_DEBUG', None) is not None: - dataplane.log_msg('--- met_data after validating ---') - dataplane.log_msg(met_data) + met_base.log_message('--- met_data after validating ---') + met_base.log_message(met_data) if __name__ == '__main__' or __name__ == sys.argv[0]: main(sys.argv) - dataplane.log_msg(f'{__name__} complete') + met_base.log_message(f'{__name__} complete') diff --git a/scripts/python/met/logger.py b/scripts/python/met/logger.py index 418a39b0d1..db58286e07 100644 --- a/scripts/python/met/logger.py +++ b/scripts/python/met/logger.py @@ -1,35 +1,40 @@ ########################################### +import os import sys +import re -class logger(): +import numpy as np - PROMPT= " PYTHON:" - ERROR_PROMPT= "ERROR" +class logger(): - ## - ## create the metadata dictionary - ## + PROMPT = " PYTHON:" + ERROR_P = " ==PYTHON_ERROR==" + INFO_P = " ==PYTHON_INFO==" @staticmethod def append_error_prompt(msg): - return f'{logger.ERROR_PROMPT}: {msg}' + return f'{logger.ERROR_P}: {msg}' @staticmethod - def error_msg(msg): + def error_messageg(msg): msgs = msg if isinstance(msg, list) else [msg] msgs.insert(0, '') msgs.append('') for a_msg in msgs: - logger.log_msg(logger.append_error_prompt(a_msg)) + logger.log_message(logger.append_error_prompt(a_msg)) #@staticmethod #def get_met_fill_value(): # return logger.MET_FILL_VALUE @staticmethod - def log_msg(msg): + def info_message(msg): + print(f'{logger.PROMPT} {logger.INFO_P} {msg}') + + @staticmethod + def log_message(msg): print(f'{logger.PROMPT} {msg}') @staticmethod @@ -40,4 +45,129 @@ def quit(msg, do_quit=True): @staticmethod def quit_msg(msg): - logger.error_msg([msg, "Quit..."]) + logger.error_message([msg, "Quit..."]) + + +class met_base(logger): + + MET_FILL_VALUE = -9999. + + def convert_to_array(self, ndarray_data): + return met_base_tools.convert_to_array(ndarray_data) + + def convert_to_ndarray(self, array_data): + return met_base_tools.convert_to_ndarray(array_data) + + def get_met_fill_value(self): + return met_base.MET_FILL_VALUE + + def error_msg(self, msg): + logger.error_messageg(msg) + + def get_prompt(self): + return met_base_tools.get_prompt() + + def info_msg(self, msg): + logger.info_message(msg) + + def is_numpy_array(self, var_data): + return isinstance(var_data, np.ndarray) + + def log_msg(self, msg): + logger.log_message(msg) + + @staticmethod + def get_numpy_filename(tmp_filename): + return logger.replace_extension(tmp_filename, "json", "npy") if tmp_filename.endswith(".json") else \ + logger.replace_extension(tmp_filename, "nc", "npy") if tmp_filename.endswith(".nc") else f'{tmp_filename}.npy' + + def is_debug_enabled(self, component_name=""): + return met_base_tools.is_debug_enabled(component_name) + + def replace_extension(self, file_name, from_ext, to_ext): + return met_base_tools.replace_extension(file_name, from_ext, to_ext) + + def remove_file(self, file_name): + os.remove(file_name) + + def use_netcdf_format(self): + return met_base_tools.use_netcdf_format() + +class met_base_tools(object): + + ENV_MET_KEEP_TEMP_FILE = "MET_KEEP_TEMP_FILE" + ENV_MET_PYTHON_DEBUG = "MET_PYTHON_DEBUG" + ENV_MET_PYTHON_TMP_FORMAT = "MET_PYTHON_TMP_FORMAT" + + @staticmethod + def convert_to_array(ndarray_data): + is_byte_type = False + if 0 < len(ndarray_data): + is_byte_type = isinstance(ndarray_data[0], (bytes, np.bytes_)) + if isinstance(ndarray_data[0], np.ndarray): + if 0 < len(ndarray_data[0]): + is_byte_type = isinstance(ndarray_data[0][0], (bytes, np.bytes_)) + if is_byte_type: + array_data = [] + if isinstance(ndarray_data[0], (np.ma.MaskedArray, np.ma.core.MaskedArray)): + for byte_data in ndarray_data: + array_data.append(byte_data.tobytes(fill_value=' ').decode('utf-8').rstrip()) + else: + for byte_data in ndarray_data: + array_data.append(byte_data.decode("utf-8").rstrip()) + elif isinstance(ndarray_data, (np.ma.MaskedArray, np.ma.core.MaskedArray)): + array_data = np.ma.getdata(ndarray_data, subok=False).tolist() + elif isinstance(ndarray_data, np.ndarray): + array_data = ndarray_data.tolist() + else: + array_data = ndarray_data + return array_data + + @staticmethod + def convert_to_ndarray(array_data): + if isinstance(array_data, (np.ma.MaskedArray, np.ma.core.MaskedArray)): + ndarray_data = np.ma.getdata(array_data, subok=False) + elif isinstance(array_data, np.ndarray): + ndarray_data = array_data + else: + ndarray_data = np.array(array_data) + return ndarray_data + + @staticmethod + def get_numpy_filename(tmp_filename): + return logger.replace_extension(tmp_filename, "json", "npy") if tmp_filename.endswith(".json") else \ + logger.replace_extension(tmp_filename, "txt", "npy") if tmp_filename.endswith(".txt") else \ + logger.replace_extension(tmp_filename, "nc", "npy") if tmp_filename.endswith(".nc") else f'{tmp_filename}.npy' + + @staticmethod + def get_prompt(): + return logger.PROMPT + + @staticmethod + def is_debug_enabled(component_name=""): + env_value = os.getenv(met_base_tools.ENV_MET_PYTHON_DEBUG, "").lower() + return env_value == "all" or env_value == component_name.lower() + + @staticmethod + def keep_temp_file(): + env_value = os.getenv(met_base_tools.ENV_MET_KEEP_TEMP_FILE, "") + return env_value.lower() == "true" or env_value.lower() == "yes" + + @staticmethod + def replace_extension(file_name, from_ext, to_ext): + return re.sub(f".{from_ext}$", f".{to_ext}", file_name) + + @staticmethod + def remove_file(file_name): + if os.path.exists(file_name): + os.remove(file_name) + + @staticmethod + def remove_temp_file(file_name): + if not met_base_tools.keep_temp_file(): + met_base_tools.remove_file(file_name) + + @staticmethod + def use_netcdf_format(): + env_value = os.getenv(met_base_tools.ENV_MET_PYTHON_TMP_FORMAT, "") + return env_value.lower() == "netcdf" diff --git a/scripts/python/met/point.py b/scripts/python/met/point.py index fbfb112f51..a2d1619ec1 100644 --- a/scripts/python/met/point.py +++ b/scripts/python/met/point.py @@ -46,24 +46,24 @@ def read_data(data_filename): ''' import os +import json from abc import ABC, abstractmethod import numpy as np -import netCDF4 as nc import pandas as pd +from met.logger import met_base, met_base_tools + + COUNT_SHOW = 30 -class base_met_point_obs(object): + +class met_base_point(met_base): ''' classdocs ''' - ERROR_P = " ==PYTHON_ERROR==" - INFO_P = " ==PYTHON_INFO==" - python_prefix = 'PYTHON_POINT_USER' - - FILL_VALUE = -9999. + COMPONENT_NAME = "met_point" def __init__(self, use_var_id=True): ''' @@ -73,7 +73,7 @@ def __init__(self, use_var_id=True): self.input_name = None self.ignore_input_file = False self.use_var_id = use_var_id # True if variable index, False if GRIB code - self.error_msg = "" + self.error_msgs = "" self.has_error = False # Header @@ -114,11 +114,11 @@ def __init__(self, use_var_id=True): def add_error_msg(self, error_msg): self.has_error = True - self.log_error_msg(error_msg) - if 0 == len(self.error_msg): - self.error_msg = error_msg + self.error_msg(error_msg) + if 0 == len(self.error_msgs): + self.error_msgs = error_msg else: - self.error_msg = "{m1}\n{m2}".format(m1=self.error_msg, m2=error_msg) + self.error_msgs = "{m1}\n{m2}".format(m1=self.error_msgs, m2=error_msg) def add_error_msgs(self, error_msgs): self.has_error = True @@ -126,43 +126,40 @@ def add_error_msgs(self, error_msgs): self.add_error_msg(error_msg) def check_data_member_float(self, local_var, var_name): + method_name = f"{self.__class__.__name__}.check_data_member_float()" if 0 == len(local_var): - self.add_error_msg("{v} is empty (float)".format(v=var_name)) + self.add_error_msg(f"{method_name} {var_name} is empty") elif isinstance(local_var, list): if isinstance(local_var[0], str) and not self.is_number(local_var[0]): - self.add_error_msg("Not supported data type: {n}[0]={v}, string type, not a number (int or float only)".format( - n=var_name, v=local_var[0])) + self.add_error_msg(f"{method_name} Not supported data type: {type(local_var[0])} for {var_name}[0], string type, not a number (int or float only)") elif 0 > str(type(local_var[0])).find('numpy') and not isinstance(local_var[0], (int, float)): - self.add_error_msg("Not supported data type ({t}) for {v}[0] (int or float only)".format( - v=var_name, t=type(local_var[0]))) + self.add_error_msg(f"{method_name} Not supported data type: {type(local_var[0])} for {var_name}[0] (int or float only)") elif not self.is_numpy_array(local_var): - self.add_error_msg("Not supported data type ({t}) for {v} (list and numpy.ndarray)".format( - v=var_name, t=type(local_var))) + self.add_error_msg(f"{method_name} Not supported data type ({type(local_var)}) for {var_name} (list and numpy.ndarray)") def check_data_member_int(self, local_var, var_name): + method_name = f"{self.__class__.__name__}.check_data_member_int()" if 0 == len(local_var): - self.add_error_msg("{v} is empty (int)".format(v=var_name)) + self.add_error_msg(f"{method_name} {var_name} is empty") elif isinstance(local_var, list): if isinstance(local_var[0], str) and not self.is_number(local_var[0]): - self.add_error_msg("Not supported data type: {n}[0]={v}, string type, not a number (int only)".format( - n=var_name, v=local_var[0])) + self.add_error_msg(f"{method_name} Not supported data type: {type(local_var[0])} for {var_name}[0], string type, not a number (int only)") elif 0 > str(type(local_var[0])).find('numpy') and not isinstance(local_var[0], int): - self.add_error_msg("Not supported data type ({t}) for {v}[0] (int only)".format( - v=var_name, t=type(local_var[0]))) + self.add_error_msg(f"{method_name} Not supported data type: {type(local_var[0])} for {var_name}[0] (int only)") elif not self.is_numpy_array(local_var): - self.add_error_msg("Not supported data type ({t}) for {v} (list and numpy.ndarray)".format( - v=var_name, t=type(local_var))) + self.add_error_msg(f"{method_name} Not supported data type ({type(local_var)}) for {var_name} (list and numpy.ndarray)") def check_data_member_string(self, local_var, var_name): + method_name = f"{self.__class__.__name__}.check_data_member_string()" if 0 == len(local_var): - self.add_error_msg("{v} is empty (string)".format(v=var_name)) + self.add_error_msg(f"{method_name} {var_name} is empty") elif not isinstance(local_var, (list)): - self.add_error_msg("Not supported data type ({t}) for {v} (list)".format( - v=var_name, t=type(local_var))) + self.add_error_msg(f"{method_name} Not supported data type ({type(local_var)}) for {var_name} (list)") def check_point_data(self): + method_name = f"{self.__class__.__name__}.check_point_data()" if not self.ignore_input_file and self.input_name is not None and not os.path.exists(self.input_name): - self.add_error_msg('The netcdf input {f} does not exist'.format(f=self.input_name)) + self.add_error_msg(f'{method_name} The input {self.input_name} does not exist') else: self.check_data_member_int(self.hdr_typ,'hdr_typ') self.check_data_member_int(self.hdr_sid,'hdr_sid') @@ -184,11 +181,11 @@ def check_point_data(self): if self.use_var_id: self.check_data_member_string(self.obs_var_table,'obs_var_table') - def convert_to_numpy(self, value_list): - return np.array(value_list) + #def convert_to_numpy(self, value_list): + # return met_point_tools.convert_to_ndarray(value_list) def dump(self): - base_met_point_obs.print_point_data(self.get_point_data()) + met_base_point.print_point_data(self.get_point_data()) def get_count_string(self): return f' nobs={self.nobs} nhdr={self.nhdr} ntyp={self.nhdr_typ} nsid={self.nhdr_sid} nvld={self.nhdr_vld} nqty={self.nobs_qty} nvar={self.nobs_var}' @@ -213,53 +210,41 @@ def get_point_data(self): self.check_point_data() if not self.is_numpy_array(self.hdr_typ): - self.hdr_typ = self.convert_to_numpy(self.hdr_typ) + self.hdr_typ = self.convert_to_ndarray(self.hdr_typ) if not self.is_numpy_array(self.hdr_sid): - self.hdr_sid = self.convert_to_numpy(self.hdr_sid) + self.hdr_sid = self.convert_to_ndarray(self.hdr_sid) if not self.is_numpy_array(self.hdr_vld): - self.hdr_vld = self.convert_to_numpy(self.hdr_vld) + self.hdr_vld = self.convert_to_ndarray(self.hdr_vld) if not self.is_numpy_array(self.hdr_lat): - self.hdr_lat = self.convert_to_numpy(self.hdr_lat) + self.hdr_lat = self.convert_to_ndarray(self.hdr_lat) if not self.is_numpy_array(self.hdr_lon): - self.hdr_lon = self.convert_to_numpy(self.hdr_lon) + self.hdr_lon = self.convert_to_ndarray(self.hdr_lon) if not self.is_numpy_array(self.hdr_elv): - self.hdr_elv = self.convert_to_numpy(self.hdr_elv) + self.hdr_elv = self.convert_to_ndarray(self.hdr_elv) if not self.is_numpy_array(self.obs_qty): - self.obs_qty = self.convert_to_numpy(self.obs_qty) + self.obs_qty = self.convert_to_ndarray(self.obs_qty) if not self.is_numpy_array(self.obs_hid): - self.obs_hid = self.convert_to_numpy(self.obs_hid) + self.obs_hid = self.convert_to_ndarray(self.obs_hid) if not self.is_numpy_array(self.obs_vid): - self.obs_vid = self.convert_to_numpy(self.obs_vid) + self.obs_vid = self.convert_to_ndarray(self.obs_vid) if not self.is_numpy_array(self.obs_lvl): - self.obs_lvl = self.convert_to_numpy(self.obs_lvl) + self.obs_lvl = self.convert_to_ndarray(self.obs_lvl) if not self.is_numpy_array(self.obs_hgt): - self.obs_hgt = self.convert_to_numpy(self.obs_hgt) + self.obs_hgt = self.convert_to_ndarray(self.obs_hgt) if not self.is_numpy_array(self.obs_val): - self.obs_val = self.convert_to_numpy(self.obs_val) + self.obs_val = self.convert_to_ndarray(self.obs_val) self.count_info = self.get_count_string() self.met_point_data = self return self.__dict__ +# def get_prompt(self): +# return met_point_tools.get_prompt() + def is_number(self, num_str): return num_str.replace('-','1').replace('+','2').replace('.','3').isdigit() - def is_numpy_array(self, var): - return isinstance(var, np.ndarray) - - def log_error_msg(self, err_msg): - base_met_point_obs.error_msg(err_msg) - - def log_error(self, err_msgs): - print(self.ERROR_P) - for err_line in err_msgs.split('\n'): - self.log_error_msg(err_line) - print(self.ERROR_P) - - def log_info(self, info_msg): - base_met_point_obs.info_msg(info_msg) - def put_data(self, point_obs_dict): self.use_var_id = point_obs_dict['use_var_id'] self.hdr_typ = point_obs_dict['hdr_typ'] @@ -298,92 +283,122 @@ def put_data(self, point_obs_dict): if po_array is not None: self.hdr_inst_typ = po_array - @staticmethod - def get_prompt(): - return " python:" - - @staticmethod - def error_msg(msg): - print(f'{base_met_point_obs.get_prompt()} {base_met_point_obs.ERROR_P} {msg}') - - @staticmethod - def info_msg(msg): - print(f'{base_met_point_obs.get_prompt()} {base_met_point_obs.INFO_P} {msg}') - - @staticmethod - def get_python_script(arg_value): - return arg_value[len(met_point_obs.python_prefix)+1:] - - @staticmethod - def is_python_script(arg_value): - return arg_value.startswith(met_point_obs.python_prefix) - - @staticmethod - def print_data(key, data_array, show_count=COUNT_SHOW): - if isinstance(data_array, list): - data_len = len(data_array) - if show_count >= data_len: - print(" {k:10s}: {v}".format(k=key, v= data_array)) - else: - end_offset = int(show_count/2) - print(" {k:10s}: count={v}".format(k=key, v=data_len)) - print(" {k:10s}[0:{o}] {v}".format(k=key, v=data_array[:end_offset], o=end_offset)) - print(" {k:10s}[{s}:{e}]: {v}".format(k=key, v='...', s=end_offset+1, e=data_len-end_offset-1)) - print(" {k:10s}[{s}:{e}]: {v}".format(k=key, v= data_array[-end_offset:], s=(data_len-end_offset), e=(data_len-1))) - else: - print(" {k:10s}: {v}".format(k=key, v= data_array)) + def read_point_data(self, tmp_filename): + method_name = f"{self.__class__.__name__}.read_point_data()" + if met_base_tools.use_netcdf_format(): + from met.point_nc import nc_point_obs - @staticmethod - def print_point_data(met_point_data, print_subset=True): - print(' === MET point data by python embedding ===') - if print_subset: - met_point_obs.print_data('nhdr',met_point_data['nhdr']) - met_point_obs.print_data('nobs',met_point_data['nobs']) - met_point_obs.print_data('use_var_id',met_point_data['use_var_id']) - met_point_obs.print_data('hdr_typ',met_point_data['hdr_typ']) - met_point_obs.print_data('hdr_typ_table',met_point_data['hdr_typ_table']) - met_point_obs.print_data('hdr_sid',met_point_data['hdr_sid']) - met_point_obs.print_data('hdr_sid_table',met_point_data['hdr_sid_table']) - met_point_obs.print_data('hdr_vld',met_point_data['hdr_vld']) - met_point_obs.print_data('hdr_vld_table',met_point_data['hdr_vld_table']) - met_point_obs.print_data('hdr_lat',met_point_data['hdr_lat']) - met_point_obs.print_data('hdr_lon',met_point_data['hdr_lon']) - met_point_obs.print_data('hdr_elv',met_point_data['hdr_elv']) - met_point_obs.print_data('obs_hid',met_point_data['obs_hid']) - met_point_obs.print_data('obs_vid',met_point_data['obs_vid']) - met_point_obs.print_data('obs_var_table',met_point_data['obs_var_table']) - met_point_obs.print_data('obs_qty',met_point_data['obs_qty']) - met_point_obs.print_data('obs_qty_table',met_point_data['obs_qty_table']) - met_point_obs.print_data('obs_lvl',met_point_data['obs_lvl']) - met_point_obs.print_data('obs_hgt',met_point_data['obs_hgt']) - met_point_obs.print_data('obs_val',met_point_data['obs_val']) + met_point_data = nc_point_obs() + met_point_data.read_data(tmp_filename) + self.put_data(met_point_data.get_point_data()) + if met_base_tools.is_debug_enabled("point"): + met_base.log_message(f"{method_name} Read from a temporary NetCDF file (point)") else: - print('All',met_point_data) - print(" nhdr: ",met_point_data['nhdr']) - print(" nobs: ",met_point_data['nobs']) - print(' use_var_id: ',met_point_data['use_var_id']) - print(' hdr_typ: ',met_point_data['hdr_typ']) - print('hdr_typ_table: ',met_point_data['hdr_typ_table']) - print(' hdr_sid: ',met_point_data['hdr_sid']) - print('hdr_sid_table: ',met_point_data['hdr_sid_table']) - print(' hdr_vld: ',met_point_data['hdr_vld']) - print('hdr_vld_table: ',met_point_data['hdr_vld_table']) - print(' hdr_lat: ',met_point_data['hdr_lat']) - print(' hdr_lon: ',met_point_data['hdr_lon']) - print(' hdr_elv: ',met_point_data['hdr_elv']) - print(' obs_hid: ',met_point_data['obs_hid']) - print(' obs_vid: ',met_point_data['obs_vid']) - print('obs_var_table: ',met_point_data['obs_var_table']) - print(' obs_qty: ',met_point_data['obs_qty']) - print('obs_qty_table: ',met_point_data['obs_qty_table']) - print(' obs_lvl: ',met_point_data['obs_lvl']) - print(' obs_hgt: ',met_point_data['obs_hgt']) - print(' obs_val: ',met_point_data['obs_val']) - - print(' === MET point data by python embedding ===') + self.read_point_data_json_numpy(tmp_filename) + + def read_point_data_json_numpy(self, tmp_filename): + method_name = f"{self.__class__.__name__}.read_point_data_json_numpy()" + if met_base_tools.is_debug_enabled("point"): + met_base.log_message(f"{method_name} Read from a temporary JSON file and a temporary numpy output (point)") + + with open(tmp_filename) as json_fh: + json_dict = json.load(json_fh) + + self.use_var_id = json_dict['use_var_id'] + self.nhdr = json_dict['nhdr'] + self.nobs = json_dict['nobs'] + self.hdr_typ_table = json_dict['hdr_typ_table'] + self.hdr_sid_table = json_dict['hdr_sid_table'] + self.hdr_vld_table = json_dict['hdr_vld_table'] + self.obs_var_table = json_dict['obs_var_table'] + self.obs_qty_table = json_dict['obs_qty_table'] + self.obs_var_unit = json_dict['obs_var_unit'] + self.obs_var_desc = json_dict['obs_var_desc'] + self.hdr_prpt_typ = json_dict['hdr_prpt_typ'] + self.hdr_irpt_typ = json_dict['hdr_irpt_typ'] + self.hdr_inst_typ = json_dict['hdr_inst_typ'] + + # read 2D numeric data + numpy_dump_name = met_base_tools.get_numpy_filename(tmp_filename) + point_array_list = np.load(numpy_dump_name) + + # Header data + self.hdr_typ = point_array_list[0,:self.nhdr] + self.hdr_sid = point_array_list[1,:self.nhdr] + self.hdr_vld = point_array_list[2,:self.nhdr] + self.hdr_lat = point_array_list[3,:self.nhdr] + self.hdr_lon = point_array_list[4,:self.nhdr] + self.hdr_elv = point_array_list[5,:self.nhdr] + # Observation data + self.obs_hid = point_array_list[6] + self.obs_lvl = point_array_list[7] + self.obs_hgt = point_array_list[8] + self.obs_val = point_array_list[9] + self.obs_vid = point_array_list[10] + self.obs_qty = point_array_list[11] + + if numpy_dump_name != tmp_filename: + os.remove(numpy_dump_name) + def write_point_data(self, tmp_filename): + if met_base_tools.use_netcdf_format(): + from met.point_nc import nc_point_obs -class csv_point_obs(base_met_point_obs): + nc_point_obs.write_nc_file(tmp_filename, self) + if met_base_tools.is_debug_enabled("point"): + met_base.log_message(f"Save to a temporary NetCDF file (point)") + else: + self.write_point_data_json_numpy(tmp_filename) + + def write_point_data_json_numpy(self, tmp_filename): + method_name = f"{self.__class__.__name__}.write_point_data_json_numpy()" + if met_base_tools.is_debug_enabled("dataplane"): + met_base.log_message(f"{method_name} Save to a temporary JSON file and a temporary numpy output (point)") + + self.nhdr = len(self.hdr_sid) + self.nobs = len(self.obs_hid) + + json_dict = {} + json_dict['use_var_id'] = self.use_var_id + json_dict['nhdr'] = self.nhdr + json_dict['nobs'] = self.nobs + json_dict['hdr_typ_table'] = self.convert_to_array(self.hdr_typ_table) + json_dict['hdr_sid_table'] = self.convert_to_array(self.hdr_sid_table) + json_dict['hdr_vld_table'] = self.convert_to_array(self.hdr_vld_table) + json_dict['obs_var_table'] = self.convert_to_array(self.obs_var_table) + json_dict['obs_qty_table'] = self.convert_to_array(self.obs_qty_table) + json_dict['obs_var_unit'] = self.convert_to_array(self.obs_var_unit) + json_dict['obs_var_desc'] = self.convert_to_array(self.obs_var_desc) + json_dict['hdr_prpt_typ'] = self.convert_to_array(self.hdr_prpt_typ) + json_dict['hdr_irpt_typ'] = self.convert_to_array(self.hdr_irpt_typ) + json_dict['hdr_inst_typ'] = self.convert_to_array(self.hdr_inst_typ) + + point_array_list = np.empty([12, self.nobs]) + + # Header data + point_array_list[0,:self.nhdr] = self.convert_to_ndarray(self.hdr_typ) + point_array_list[1,:self.nhdr] = self.convert_to_ndarray(self.hdr_sid) + point_array_list[2,:self.nhdr] = self.convert_to_ndarray(self.hdr_vld) + point_array_list[3,:self.nhdr] = self.convert_to_ndarray(self.hdr_lat) + point_array_list[4,:self.nhdr] = self.convert_to_ndarray(self.hdr_lon) + point_array_list[5,:self.nhdr] = self.convert_to_ndarray(self.hdr_elv) + # Observation data + point_array_list[6] = self.convert_to_ndarray(self.obs_hid) + point_array_list[7] = self.convert_to_ndarray(self.obs_lvl) + point_array_list[8] = self.convert_to_ndarray(self.obs_hgt) + point_array_list[9] = self.convert_to_ndarray(self.obs_val) + point_array_list[10] = self.convert_to_ndarray(self.obs_vid) + point_array_list[11] = self.convert_to_ndarray(self.obs_qty) + + with open(tmp_filename,'w') as json_fh: + json.dump(json_dict, json_fh) + + numpy_dump_name = met_base_tools.get_numpy_filename(tmp_filename) + #np.save(numpy_dump_name, self.convert_to_ndarray(point_array_list)) + np.save(numpy_dump_name, point_array_list) + + +class csv_point_obs(met_base_point): def __init__(self, point_data): self.point_data = point_data @@ -400,50 +415,50 @@ def __init__(self, point_data): self.convert_point_data() def check_csv_record(self, csv_point_data, index): + method_name = f"{self.__class__.__name__}.check_csv_record()" error_msgs = [] # names=['typ', 'sid', 'vld', 'lat', 'lon', 'elv', 'var', 'lvl', 'hgt', 'qc', 'obs'] # dtype={'typ':'str', 'sid':'str', 'vld':'str', 'var':'str', 'qc':'str'} if 11 > len(csv_point_data): - error_msgs.append("{i}-th data: missing columns. should be 11 columns, not {c} columns".format( - i=index, c=len(csv_point_data))) + error_msgs.append(f"{method_name} {index}-th data: missing columns. should be 11 columns, not {len(csv_point_data)} columns") elif 11 < len(csv_point_data): - print("{i}-th data: ignore after 11-th columns out of {c} columns".format( - i=index, c=len(csv_point_data))) + print("{i}-th data: ignore after 11-th columns out of {len(csv_point_data)} columns") if not isinstance(csv_point_data[0], str): - error_msgs.append("{i}-th data: message_type is not string".format(i=index)) + error_msgs.append(f"{method_name} {index}-th data: message_type is not string") if not isinstance(csv_point_data[1], str): - error_msgs.append("{i}-th data: station_id is not string".format(i=index)) + error_msgs.append(f"{method_name} {index}-th data: station_id is not string") if not isinstance(csv_point_data[2], str): - error_msgs.append("{i}-th data: valid_time is not string".format(i=index)) + error_msgs.append(f"{method_name} {index}-th data: valid_time is not string") if isinstance(csv_point_data[3], str): - error_msgs.append("{i}-th data: latitude can not be a string".format(i=index)) + error_msgs.append(f"{method_name} {index}-th data: latitude can not be a string") elif csv_point_data[3] < -90.0 or csv_point_data[3] > 90.0: - error_msgs.append("{i}-th data: latitude ({l}) is out of range".format(i=index, l=csv_point_data[3])) + error_msgs.append(f"{method_name} {index}-th data: latitude ({csv_point_data[3]}) is out of range") if isinstance(csv_point_data[4], str): - error_msgs.append("{i}-th data: longitude can not be a string".format(i=index)) + error_msgs.append(f"{method_name} {index}-th data: longitude can not be a string") elif csv_point_data[4] < -180.0 or csv_point_data[4] > 360.0: - error_msgs.append("{i}-th data: longitude ({l}) is out of range".format(i=index, l=csv_point_data[4])) + error_msgs.append(f"{method_name} {index}-th data: longitude ({csv_point_data[4]}) is out of range") if not isinstance(csv_point_data[6], str): - error_msgs.append("{i}-th data: grib_code/var_name is not string".format(i=index)) + error_msgs.append(f"{method_name} {index}-th data: grib_code/var_name is not string") if not isinstance(csv_point_data[9], str): - error_msgs.append("{i}-th data: quality_mark is not string".format(i=index)) + error_msgs.append(f"{method_name} {index}-th data: quality_mark is not string") is_string, is_num = self.is_num_string(csv_point_data[5]) if is_string and not is_num: - error_msgs.append("{i}-th data: elevation: only NA is accepted as string".format(i=index)) + error_msgs.append(f"{method_name} {index}-th data: elevation: only NA is accepted as string") is_string, is_num = self.is_num_string(csv_point_data[7]) if is_string and not is_num: - error_msgs.append("{i}-th data: obs_level: only NA is accepted as string".format(i=index)) + error_msgs.append(f"{method_name} {index}-th data: obs_level: only NA is accepted as string") is_string, is_num = self.is_num_string(csv_point_data[8]) if is_string and not is_num: - error_msgs.append("{i}-th data: obs_height: only NA is accepted as string".format(i=index)) + error_msgs.append(f"{method_name} {index}-th data: obs_height: only NA is accepted as string") is_string, is_num = self.is_num_string(csv_point_data[10]) if is_string and not is_num: - error_msgs.append("{i}-th data: obs_value: only NA is accepted as string".format(i=index)) + error_msgs.append(f"{method_name} {index}-th data: obs_value: only NA is accepted as string") return error_msgs def check_csv_point_data(self, all_records=False): + method_name = f"{self.__class__.__name__}.check_csv_point_data()" if 0 == len(self.point_data): - self.add_error_msg("No data!") + self.add_error_msg(f"{method_name} No data!") elif all_records: data_idx = 0 for csv_point_data in self.point_data: @@ -573,6 +588,7 @@ def convert_point_data(self): self.obs_var_table[idx] = key def get_num_value(self, column_value): + method_name = f"{self.__class__.__name__}.get_num_value()" num_value = column_value if isinstance(column_value, str): if self.is_number(column_value): @@ -580,7 +596,7 @@ def get_num_value(self, column_value): else: num_value = self.FILL_VALUE if column_value.lower() != 'na' and column_value.lower() != 'n/a': - self.log_info(f'{column_value} is not a number, converted to the missing value') + self.info_msg(f'{method_name} {column_value} is not a number, converted to the missing value') return num_value def is_grib_code(self): @@ -602,9 +618,7 @@ def is_num_string(self, column_value): return is_string, is_num -class met_point_obs(ABC, base_met_point_obs): - - MET_ENV_RUN = 'MET_FORCE_TO_RUN' +class met_point_obs(ABC, met_base_point): @abstractmethod def read_data(self, args): @@ -625,7 +639,15 @@ def read_data(self, args): pass -class met_point_tools(): +class dummy_point_obs(met_point_obs): + + def read_data(self, args): + pass + + +class met_point_tools(met_base_tools): + + python_prefix = 'PYTHON_POINT_USER' @staticmethod def convert_point_data(point_data, check_all_records=False, input_type='csv'): @@ -635,24 +657,85 @@ def convert_point_data(point_data, check_all_records=False, input_type='csv'): csv_point_data.check_csv_point_data(check_all_records) tmp_point_data = csv_point_data.get_point_data() else: - base_met_point_obs.error_msg('Not supported input type: {input_type}') + met_base_point.error_message(f'met_point_tools.convert_point_data() Not supported input type: {input_type}') return tmp_point_data @staticmethod - def get_prompt(): - return " python:" + def get_sample_point_obs(): + return sample_met_point_obs() @staticmethod - def get_nc_point_obs(): - return nc_point_obs() + def get_python_script(arg_value): + return arg_value[len(met_point_tools.python_prefix)+1:] @staticmethod - def get_sample_point_obs(): - return sample_met_point_obs() + def is_python_prefix(user_cmd): + return user_cmd.startswith(met_point_tools.python_prefix) @staticmethod - def is_python_prefix(user_cmd): - return user_cmd.startswith(base_met_point_obs.python_prefix) + def print_data(key, data_array, show_count=COUNT_SHOW): + if isinstance(data_array, list): + data_len = len(data_array) + if show_count >= data_len: + print(" {k:10s}: {v}".format(k=key, v= data_array)) + else: + end_offset = int(show_count/2) + print(" {k:10s}: count={v}".format(k=key, v=data_len)) + print(" {k:10s}[0:{o}] {v}".format(k=key, v=data_array[:end_offset], o=end_offset)) + print(" {k:10s}[{s}:{e}]: {v}".format(k=key, v='...', s=end_offset+1, e=data_len-end_offset-1)) + print(" {k:10s}[{s}:{e}]: {v}".format(k=key, v= data_array[-end_offset:], s=(data_len-end_offset), e=(data_len-1))) + else: + print(" {k:10s}: {v}".format(k=key, v= data_array)) + + @staticmethod + def print_point_data(met_point_data, print_subset=True): + method_name = f"met_point_tools.print_point_data()" + print(' === MET point data by python embedding ===') + if print_subset: + met_point_tools.print_data('nhdr',met_point_data['nhdr']) + met_point_tools.print_data('nobs',met_point_data['nobs']) + met_point_tools.print_data('use_var_id',met_point_data['use_var_id']) + met_point_tools.print_data('hdr_typ',met_point_data['hdr_typ']) + met_point_tools.print_data('hdr_typ_table',met_point_data['hdr_typ_table']) + met_point_tools.print_data('hdr_sid',met_point_data['hdr_sid']) + met_point_tools.print_data('hdr_sid_table',met_point_data['hdr_sid_table']) + met_point_tools.print_data('hdr_vld',met_point_data['hdr_vld']) + met_point_tools.print_data('hdr_vld_table',met_point_data['hdr_vld_table']) + met_point_tools.print_data('hdr_lat',met_point_data['hdr_lat']) + met_point_tools.print_data('hdr_lon',met_point_data['hdr_lon']) + met_point_tools.print_data('hdr_elv',met_point_data['hdr_elv']) + met_point_tools.print_data('obs_hid',met_point_data['obs_hid']) + met_point_tools.print_data('obs_vid',met_point_data['obs_vid']) + met_point_tools.print_data('obs_var_table',met_point_data['obs_var_table']) + met_point_tools.print_data('obs_qty',met_point_data['obs_qty']) + met_point_tools.print_data('obs_qty_table',met_point_data['obs_qty_table']) + met_point_tools.print_data('obs_lvl',met_point_data['obs_lvl']) + met_point_tools.print_data('obs_hgt',met_point_data['obs_hgt']) + met_point_tools.print_data('obs_val',met_point_data['obs_val']) + else: + print(f'{method_name} All',met_point_data) + print(f" nhdr: met_point_data['nhdr']") + print(f" nobs: met_point_data['nobs']") + print(f" use_var_id: met_point_data['use_var_id']") + print(f" hdr_typ: met_point_data['hdr_typ']") + print(f"hdr_typ_table: met_point_data['hdr_typ_table']") + print(f" hdr_sid: met_point_data['hdr_sid']") + print(f"hdr_sid_table: met_point_data['hdr_sid_table']") + print(f" hdr_vld: met_point_data['hdr_vld']") + print(f"hdr_vld_table: met_point_data['hdr_vld_table']") + print(f" hdr_lat: met_point_data['hdr_lat']") + print(f" hdr_lon: met_point_data['hdr_lon']") + print(f" hdr_elv: met_point_data['hdr_elv']") + print(f" obs_hid: met_point_data['obs_hid']") + print(f" obs_vid: met_point_data['obs_vid']") + print(f"obs_var_table: met_point_data['obs_var_table']") + print(f" obs_qty: met_point_data['obs_qty']") + print(f"obs_qty_table: met_point_data['obs_qty_table']") + print(f" obs_lvl: met_point_data['obs_lvl']") + print(f" obs_hgt: met_point_data['obs_hgt']") + print(f" obs_val: met_point_data['obs_val']") + + print(' === MET point data by python embedding ===') @staticmethod # Read the input file which is 11 column text file as the first argument @@ -670,254 +753,14 @@ def read_text_point_obs(input_file, header=None, # (9) numeric: Height(msl or agl) # (10) string: QC_String # (11) numeric: Observation_Value - ascii_point_data = pd.read_csv(input_file, header=header, - delim_whitespace=delim_whitespace, - keep_default_na=keep_default_na, - names=['typ', 'sid', 'vld', 'lat', 'lon', 'elv', 'var', 'lvl', 'hgt', 'qc', 'obs'], - dtype={'typ':'string', 'sid':'string', 'vld':'string', 'var':'string', 'qc':'string'}).values.tolist() + ascii_point_data = pd.read_csv( + input_file, header=header, + delim_whitespace=delim_whitespace, + keep_default_na=keep_default_na, + names=['typ', 'sid', 'vld', 'lat', 'lon', 'elv', 'var', 'lvl', 'hgt', 'qc', 'obs'], + dtype={'typ':'string', 'sid':'string', 'vld':'string', 'var':'string', 'qc':'string'}).values.tolist() return ascii_point_data -# Note: caller should import netCDF4 -# The argements nc_group(dataset) and nc_var should not be None -class nc_tools(): - - met_missing = -99999999. - - @staticmethod - def get_num_array(nc_group, var_name): - nc_var = nc_group.variables.get(var_name, None) - return [] if nc_var is None else nc_var[:] - - @staticmethod - def get_ncbyte_array_to_str(nc_var): - nc_str_data = nc_var[:] - if nc_var.datatype.name == 'bytes8': - nc_str_data = [ str(s.compressed(),"utf-8") for s in nc_var[:] ] - return nc_str_data - - @staticmethod - def get_string_array(nc_group, var_name): - nc_var = nc_group.variables.get(var_name, None) - return [] if nc_var is None else nc_tools.get_ncbyte_array_to_str(nc_var) - - -class nc_point_obs(met_point_obs): - - # args should be string, list, or dictionary - def get_nc_filename(self, args): - nc_filename = None - if isinstance(args, dict): - nc_filename = args.get('nc_name',None) - elif isinstance(args, list): - nc_filename = args[0] - elif args != ARG_PRINT_DATA: - nc_filename = args - - return nc_filename - - def read_data(self, nc_filename): - if nc_filename is None: - self.log_error_msg("The input NetCDF filename is missing") - elif not os.path.exists(nc_filename): - self.log_error_msg(f"input NetCDF file ({nc_filename}) does not exist") - else: - dataset = nc.Dataset(nc_filename, 'r') - - attr_name = 'use_var_id' - use_var_id_str = dataset.getncattr(attr_name) if attr_name in dataset.ncattrs() else "false" - self.use_var_id = use_var_id_str.lower() == 'true' - - # Header - self.hdr_typ = dataset['hdr_typ'][:] - self.hdr_sid = dataset['hdr_sid'][:] - self.hdr_vld = dataset['hdr_vld'][:] - self.hdr_lat = dataset['hdr_lat'][:] - self.hdr_lon = dataset['hdr_lon'][:] - self.hdr_elv = dataset['hdr_elv'][:] - self.hdr_typ_table = nc_tools.get_string_array(dataset, 'hdr_typ_table') - self.hdr_sid_table = nc_tools.get_string_array(dataset, 'hdr_sid_table') - self.hdr_vld_table = nc_tools.get_string_array(dataset, 'hdr_vld_table') - - nc_var = dataset.variables.get('obs_unit', None) - if nc_var: - self.obs_var_unit = nc_var[:] - nc_var = dataset.variables.get('obs_desc', None) - if nc_var: - self.obs_var_desc = nc_var[:] - - nc_var = dataset.variables.get('hdr_prpt_typ', None) - if nc_var: - self.hdr_prpt_typ = nc_var[:] - nc_var = dataset.variables.get('hdr_irpt_typ', None) - if nc_var: - self.hdr_irpt_typ = nc_var[:] - nc_var = dataset.variables.get('hdr_inst_typ', None) - if nc_var: - self.hdr_inst_typ =nc_var[:] - - #Observation data - self.hdr_sid = dataset['hdr_sid'][:] - self.obs_qty = np.array(dataset['obs_qty'][:]) - self.obs_hid = np.array(dataset['obs_hid'][:]) - self.obs_lvl = np.array(dataset['obs_lvl'][:]) - self.obs_hgt = np.array(dataset['obs_hgt'][:]) - self.obs_val = np.array(dataset['obs_val'][:]) - nc_var = dataset.variables.get('obs_vid', None) - if nc_var is None: - self.use_var_id = False - nc_var = dataset.variables.get('obs_gc', None) - else: - self.obs_var_table = nc_tools.get_string_array(dataset, 'obs_var') - if nc_var: - self.obs_vid = np.array(nc_var[:]) - - self.obs_qty_table = nc_tools.get_string_array(dataset, 'obs_qty_table') - - def save_ncfile(self, nc_filename): - met_data = self.get_point_data() - with nc.Dataset(nc_filename, 'w') as nc_dataset: - self.set_nc_data(nc_dataset) - return met_data - - def set_nc_data(self, nc_dataset): - return nc_point_obs.write_nc_data(nc_dataset, self) - - @staticmethod - def write_nc_file(nc_filename, point_obs): - with nc.Dataset(nc_filename, 'w') as nc_dataset: - nc_point_obs.set_nc_data(nc_dataset, point_obs) - - @staticmethod - def write_nc_data(nc_dataset, point_obs): - do_nothing = False - if 0 == point_obs.nhdr: - do_nothing = True - base_met_point_obs.info_msg("the header is empty") - if 0 == point_obs.nobs: - do_nothing = True - base_met_point_obs.info_msg("the observation data is empty") - if do_nothing: - print() - return - - # Set global attributes - nc_dataset.MET_Obs_version = "1.02" ; - nc_dataset.use_var_id = "true" if point_obs.use_var_id else "false" - - # Create dimensions - nc_dataset.createDimension('mxstr', 16) - nc_dataset.createDimension('mxstr2', 40) - nc_dataset.createDimension('mxstr3', 80) - nc_dataset.createDimension('nhdr', point_obs.nhdr) - nc_dataset.createDimension('nobs', point_obs.nobs) - #npbhdr = len(point_obs.hdr_prpt_typ) - if 0 < point_obs.npbhdr: - nc_dataset.createDimension('npbhdr', point_obs.npbhdr) - nc_dataset.createDimension('nhdr_typ', point_obs.nhdr_typ) - nc_dataset.createDimension('nhdr_sid', point_obs.nhdr_sid) - nc_dataset.createDimension('nhdr_vld', point_obs.nhdr_vld) - nc_dataset.createDimension('nobs_qty', point_obs.nobs_qty) - nc_dataset.createDimension('obs_var_num', point_obs.nobs_var) - - type_for_string = 'S1' # np.byte - dims_hdr = ('nhdr',) - dims_obs = ('nobs',) - - # Create header and observation variables - var_hdr_typ = nc_dataset.createVariable('hdr_typ', np.int32, dims_hdr, fill_value=-9999) - var_hdr_sid = nc_dataset.createVariable('hdr_sid', np.int32, dims_hdr, fill_value=-9999) - var_hdr_vld = nc_dataset.createVariable('hdr_vld', np.int32, dims_hdr, fill_value=-9999) - var_hdr_lat = nc_dataset.createVariable('hdr_lat', np.float32, dims_hdr, fill_value=-9999.) - var_hdr_lon = nc_dataset.createVariable('hdr_lon', np.float32, dims_hdr, fill_value=-9999.) - var_hdr_elv = nc_dataset.createVariable('hdr_elv', np.float32, dims_hdr, fill_value=-9999.) - - var_obs_qty = nc_dataset.createVariable('obs_qty', np.int32, dims_obs, fill_value=-9999) - var_obs_hid = nc_dataset.createVariable('obs_hid', np.int32, dims_obs, fill_value=-9999) - var_obs_vid = nc_dataset.createVariable('obs_vid', np.int32, dims_obs, fill_value=-9999) - var_obs_lvl = nc_dataset.createVariable('obs_lvl', np.float32, dims_obs, fill_value=-9999.) - var_obs_hgt = nc_dataset.createVariable('obs_hgt', np.float32, dims_obs, fill_value=-9999.) - var_obs_val = nc_dataset.createVariable('obs_val', np.float32, dims_obs, fill_value=-9999.) - - if 0 == point_obs.npbhdr: - var_hdr_prpt_typ = None - var_hdr_irpt_typ = None - var_hdr_inst_typ = None - else: - dims_npbhdr = ('npbhdr',) - var_hdr_prpt_typ = nc_dataset.createVariable('hdr_prpt_typ', np.int32, dims_npbhdr, fill_value=-9999.) - var_hdr_irpt_typ = nc_dataset.createVariable('hdr_irpt_typ', np.int32, dims_npbhdr, fill_value=-9999.) - var_hdr_inst_typ = nc_dataset.createVariable('hdr_inst_typ', np.int32, dims_npbhdr, fill_value=-9999.) - - var_hdr_typ_table = nc_dataset.createVariable('hdr_typ_table', type_for_string, ('nhdr_typ','mxstr2')) - var_hdr_sid_table = nc_dataset.createVariable('hdr_sid_table', type_for_string, ('nhdr_sid','mxstr2')) - var_hdr_vld_table = nc_dataset.createVariable('hdr_vld_table', type_for_string, ('nhdr_vld','mxstr')) - var_obs_qty_table = nc_dataset.createVariable('obs_qty_table', type_for_string, ('nobs_qty','mxstr')) - var_obs_var_table = nc_dataset.createVariable('obs_var', type_for_string, ('obs_var_num','mxstr2')) - var_obs_var_unit = nc_dataset.createVariable('obs_unit', type_for_string, ('obs_var_num','mxstr2')) - var_obs_var_desc = nc_dataset.createVariable('obs_desc', type_for_string, ('obs_var_num','mxstr3')) - - # Set variables - var_hdr_typ[:] = point_obs.hdr_typ[:] - var_hdr_sid[:] = point_obs.hdr_sid[:] - var_hdr_vld[:] = point_obs.hdr_vld[:] - var_hdr_lat[:] = point_obs.hdr_lat[:] - var_hdr_lon[:] = point_obs.hdr_lon[:] - var_hdr_elv[:] = point_obs.hdr_elv[:] - for i in range(0, point_obs.nhdr_typ): - for j in range(0, len(point_obs.hdr_typ_table[i])): - var_hdr_typ_table[i,j] = point_obs.hdr_typ_table[i][j] - for i in range(0, point_obs.nhdr_sid): - for j in range(0, len(point_obs.hdr_sid_table[i])): - var_hdr_sid_table[i,j] = point_obs.hdr_sid_table[i][j] - for i in range(0, point_obs.nhdr_vld): - for j in range(0, len(point_obs.hdr_vld_table[i])): - var_hdr_vld_table[i,j] = point_obs.hdr_vld_table[i][j] - if 0 < point_obs.npbhdr: - var_hdr_prpt_typ[:] = point_obs.hdr_prpt_typ[:] - var_hdr_irpt_typ[:] = point_obs.hdr_irpt_typ[:] - var_hdr_inst_typ[:] = point_obs.hdr_inst_typ[:] - - var_obs_qty[:] = point_obs.obs_qty[:] - var_obs_hid[:] = point_obs.obs_hid[:] - var_obs_vid[:] = point_obs.obs_vid[:] - var_obs_lvl[:] = point_obs.obs_lvl[:] - var_obs_hgt[:] = point_obs.obs_hgt[:] - var_obs_val[:] = point_obs.obs_val[:] - for i in range(0, point_obs.nobs_var): - for j in range(0, len(point_obs.obs_var_table[i])): - var_obs_var_table[i,j] = point_obs.obs_var_table[i][j] - var_obs_var_unit[i] = "" if i >= len(point_obs.obs_var_unit) else point_obs.obs_var_unit[i] - var_obs_var_desc[i] = "" if i >= len(point_obs.obs_var_desc) else point_obs.obs_var_desc[i] - for i in range(0, point_obs.nobs_qty): - for j in range(0, len(point_obs.obs_qty_table[i])): - var_obs_qty_table[i,j] = point_obs.obs_qty_table[i][j] - - # Set variable attributes - var_hdr_typ.long_name = "index of message type" - var_hdr_sid.long_name = "index of station identification" - var_hdr_vld.long_name = "index of valid time" - var_hdr_lat.long_name = "latitude" - var_hdr_lat.units = "degrees_north" - var_hdr_lon.long_name = "longitude" - var_hdr_lon.units = "degrees_east" - var_hdr_elv.long_name = "elevation" - var_hdr_elv.units = "meters above sea level (msl)" - - var_obs_qty.long_name = "index of quality flag" - var_obs_hid.long_name = "index of matching header data" - var_obs_vid.long_name = "index of BUFR variable corresponding to the observation type" - var_obs_lvl.long_name = "pressure level (hPa) or accumulation interval (sec)" - var_obs_hgt.long_name = "height in meters above sea level (msl)" - var_obs_val.long_name = "observation value" - var_hdr_typ_table.long_name = "message type" - var_hdr_sid_table.long_name = "station identification" - var_hdr_vld_table.long_name = "valid time" - var_hdr_vld_table.units = "YYYYMMDD_HHMMSS UTC" - var_obs_qty_table.long_name = "quality flag" - var_obs_var_table.long_name = "variable names" - var_obs_var_unit.long_name = "variable units" - var_obs_var_desc.long_name = "variable descriptions" - # This is a sample drived class class sample_met_point_obs(met_point_obs): @@ -946,6 +789,7 @@ def read_data(self, arg_map={}): self.obs_var_table = [ "TMP", "RH" ] self.obs_qty_table = [ "NA" ] + def convert_point_data(point_data, check_all_records=False, input_type='csv'): tmp_point_data = {} if 'csv' == input_type: @@ -953,9 +797,12 @@ def convert_point_data(point_data, check_all_records=False, input_type='csv'): csv_point_data.check_csv_point_data(check_all_records) tmp_point_data = csv_point_data.get_point_data() else: - base_met_point_obs.error_msg('Not supported input type: {input_type}') + met_base.error_messageg(f'convert_point_data(() Not supported input type: {input_type}') return tmp_point_data +def get_empty_point_obs(): + return dummy_point_obs() + def main(): args = {} # or args = [] point_obs_data = sample_met_point_obs() @@ -964,20 +811,6 @@ def main(): point_obs_data.print_point_data(met_point_data, print_subset=False) -def main_nc(argv): - if len(argv) != 1 and argv[1] != ARG_PRINT_DATA: - netcdf_filename = argv[1] - tmp_nc_name = 'tmp_met_point.nc' - point_obs_data = nc_point_obs() - point_obs_data.read_data(point_obs_data.get_nc_filename(netcdf_filename)) - met_point_data = point_obs_data.save_ncfile(tmp_nc_name) - print(f'{base_met_point_obs.get_prompt()} saved met_point_data to {tmp_nc_name}') - met_point_data['met_point_data'] = point_obs_data - - if DO_PRINT_DATA or ARG_PRINT_DATA == argv[-1]: - met_point_obs.print_point_data(met_point_data) - - if __name__ == '__main__': main() print('Done python script') From 1600963e0d2e9bcd14269092e2598e046b0ce148 Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Mon, 29 Jan 2024 23:04:10 +0000 Subject: [PATCH 15/21] #2772 Allow to keep the temporary files by using the environment variable --- src/basic/vx_config/temp_file.cc | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/basic/vx_config/temp_file.cc b/src/basic/vx_config/temp_file.cc index 98683a1c5b..f1ed751190 100644 --- a/src/basic/vx_config/temp_file.cc +++ b/src/basic/vx_config/temp_file.cc @@ -70,6 +70,12 @@ void remove_temp_file(const ConcatString file_name) { // // Attempt to remove the file and print out any error message // + const char *keep_temp = getenv("MET_KEEP_TEMP_FILE"); + if (nullptr != keep_temp + && (0 == strcmp(keep_temp, "true") || 0 == strcmp(keep_temp, "yes"))) { + return; + } + if((errno = remove(file_name.c_str())) != 0) { mlog << Error << "\nremove_temp_file() -> " << "can't delete temporary file: \"" From 6264c6dc119f13ba5af8fc4146c97eb0041f1a49 Mon Sep 17 00:00:00 2001 From: Howard Soh Date: Mon, 29 Jan 2024 23:23:44 +0000 Subject: [PATCH 16/21] Added log message if the temprary fiule was not deleted --- src/basic/vx_config/temp_file.cc | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/basic/vx_config/temp_file.cc b/src/basic/vx_config/temp_file.cc index f1ed751190..85d5528a49 100644 --- a/src/basic/vx_config/temp_file.cc +++ b/src/basic/vx_config/temp_file.cc @@ -73,6 +73,8 @@ void remove_temp_file(const ConcatString file_name) { const char *keep_temp = getenv("MET_KEEP_TEMP_FILE"); if (nullptr != keep_temp && (0 == strcmp(keep_temp, "true") || 0 == strcmp(keep_temp, "yes"))) { + mlog << Debug(2) << "The temporary file (" + << file_name << ") was not deleted. Please remove it manually\n\n"; return; } From 0defee8e808256cf55d7c8a3b651a85cf3d95662 Mon Sep 17 00:00:00 2001 From: davidalbo Date: Mon, 29 Jan 2024 17:08:13 -0700 Subject: [PATCH 17/21] Feature #2745 mvmode enhancements (#2779) * Working mvmode with percentile thresholds and grid_type fixes, plus more unit tests * separated multivar and traditional mode frontend classes to simplify the mode executive class design * documentation changes to go with changes to percentiles and data_type * Cleanup and bugfix to doc * Bug fix to doc * Another bugfix attempt * synching up development.seneca with what is in the develop branch * added clone() method to all the vx_data2d var_info classes * Fix multivar_name/level definitions in UG * bug fix regarding accessing the config to create a verification grid * Update docs/Users_Guide/mode.rst * Update docs/Users_Guide/mode.rst * Per #2745, commit changes to Makefile.am/.in files after running bootstrap on this feature branch on seneca. * Per #2745, no real code changes. Just whitespace. --------- Co-authored-by: MET Tools Test Account Co-authored-by: Tracy Hertneky Co-authored-by: John Halley Gotway --- docs/Users_Guide/mode.rst | 21 +- internal/test_unit/config/MODEConfig_multivar | 274 +++++ .../test_unit/config/MODEConfig_multivar_3_2 | 275 +++++ .../config/MODEConfig_multivar_super | 271 +++++ internal/test_unit/xml/unit_mode_multivar.xml | 85 +- src/basic/vx_config/config_file.cc | 14 + src/libcode/vx_data2d/var_info.h | 5 +- src/libcode/vx_data2d_grib/var_info_grib.cc | 9 + src/libcode/vx_data2d_grib/var_info_grib.h | 1 + src/libcode/vx_data2d_grib2/var_info_grib2.cc | 9 + src/libcode/vx_data2d_grib2/var_info_grib2.h | 1 + src/libcode/vx_data2d_nc_cf/var_info_nc_cf.cc | 9 + src/libcode/vx_data2d_nc_cf/var_info_nc_cf.h | 1 + .../vx_data2d_nc_met/var_info_nc_met.cc | 9 + .../vx_data2d_nc_met/var_info_nc_met.h | 1 + .../vx_data2d_nc_wrf/var_info_nc_wrf.cc | 9 + .../vx_data2d_nc_wrf/var_info_nc_wrf.h | 1 + .../vx_data2d_python/var_info_python.cc | 9 + .../vx_data2d_python/var_info_python.h | 1 + src/libcode/vx_data2d_ugrid/var_info_ugrid.cc | 10 +- src/libcode/vx_data2d_ugrid/var_info_ugrid.h | 1 + src/libcode/vx_shapedata/Makefile.am | 1 + src/libcode/vx_shapedata/Makefile.in | 1 + src/libcode/vx_shapedata/engine.cc | 22 +- src/libcode/vx_shapedata/engine.h | 15 +- src/libcode/vx_shapedata/mode_conf_info.cc | 609 ++++++++--- src/libcode/vx_shapedata/mode_conf_info.h | 42 +- src/libcode/vx_shapedata/mode_field_info.cc | 32 +- src/libcode/vx_shapedata/mode_field_info.h | 8 +- src/libcode/vx_shapedata/mode_input_data.h | 42 + src/tools/core/mode/Makefile.am | 1 + src/tools/core/mode/Makefile.in | 25 +- src/tools/core/mode/combine_boolplanes.h | 3 +- src/tools/core/mode/mode.cc | 9 +- src/tools/core/mode/mode_exec.cc | 675 ++++++------ src/tools/core/mode/mode_exec.h | 93 +- src/tools/core/mode/mode_frontend.cc | 508 +-------- src/tools/core/mode/mode_frontend.h | 43 +- src/tools/core/mode/mode_superobject.cc | 184 ++++ src/tools/core/mode/mode_superobject.h | 51 + src/tools/core/mode/multivar_data.cc | 22 +- src/tools/core/mode/multivar_data.h | 7 +- src/tools/core/mode/multivar_frontend.cc | 965 ++++++++---------- src/tools/core/mode/multivar_frontend.h | 106 ++ 44 files changed, 2776 insertions(+), 1704 deletions(-) create mode 100644 internal/test_unit/config/MODEConfig_multivar create mode 100644 internal/test_unit/config/MODEConfig_multivar_3_2 create mode 100644 internal/test_unit/config/MODEConfig_multivar_super create mode 100644 src/libcode/vx_shapedata/mode_input_data.h create mode 100644 src/tools/core/mode/mode_superobject.cc create mode 100644 src/tools/core/mode/mode_superobject.h create mode 100644 src/tools/core/mode/multivar_frontend.h diff --git a/docs/Users_Guide/mode.rst b/docs/Users_Guide/mode.rst index 5a6503f0f5..491b452002 100644 --- a/docs/Users_Guide/mode.rst +++ b/docs/Users_Guide/mode.rst @@ -118,6 +118,23 @@ The **multivar_intensity_compare_fcst** and **multivar_intensity_compare_obs** c When regridding to the FCST or OBS field (e.g. to_grid = FCST), the first field of the field array is used from the forecast and observation field dictionaries, respectively. All regridding is then done to that grid. Other regrid options described in :ref:`regrid` can also be used as normal. +"file_type" can be set independently for each input in multivariate mode. If not set for an input, MET uses file names and file content to determine the type. + +When setting a threshold to a percentile, some choices require both an observation input and a forecast input. When this is the case, it's assumed the indices match, so for example if forecast input 1 has such a percentile setting, then observation input 1 will be used to compute the percentile. Percentiles in which this will happen are: + +* SFP in an observation input. + * The matching forecast input will be used to determine the threshold. e.g. ">SFP33.3" in the 2nd observation input means greater than 33.3-rd percentile of the 2nd forecast input will be used as the threshold for that observation input. + +* SOP in a forecast input. + * The matching observation input will be used to determine the threshold. e.g. ">SOP33.3" in the 2nd forecast input means greater than 33.3-rd percentile of the 2nd observation input will be used as the threshold for that forecast input. + +* "==FBIAS" in an observation input. + * e.g. "==FBIAS1" in an observation input to automatically de-bias the data, using a simple threshold in the matching forecast input. For example, when observation input 3 has "==FBIAS1", and forecast input 3 has ">5.0", MET applies the >5.0 threshold to the forecast and then chooses an observation threshold which results in a frequency bias of 1. The frequency bias can be any float value > 0.0. + +* "==FBIAS" in a forecast input. + * e.g. "==FBIAS1" in a forecast input to automatically de-bias the data, using a simple threshold in the matching observation input. For example, when forecast input 2 has "==FBIAS1", and observation input 2 has ">5.0", MET applies the >5.0 threshold to the observation and then chooses a forecast threshold which results in a frequency bias of 1. The frequency bias can be any float value > 0.0. + + Practical Information ===================== @@ -260,7 +277,7 @@ _____________________ multivar_name = "Super"; -The **multivar_name** entry appears only in the **MODEMultivarConfig_default** file. This option is used only when the multivar option is enabled, and only when all **multivar_intensity_flag** values are FALSE. It can be thought of as an identifier for the multivariate super object. It shows up in output files names and content. It can be set separately for forecasts and observations or as a common value for both. +The **multivar_name** entry appears only in the **MODEMultivarConfig_default** file. This option is used only when the multivar option is enabled, and only when **multivar_intensity_compare_fcst** and **multivar_intensity_compare_obs** are empty. It can be thought of as an identifier for the multivariate super object. It shows up in output files names and content. It can be set separately for forecasts and observations or as a common value for both. _____________________ @@ -268,7 +285,7 @@ _____________________ multivar_level = "LO"; -The **multivar_level** entry appears only in the **MODEMultivarConfig_default** file. This option is used only when the multivar option is enabled, and only when all **multivar_intensity_flag** values are FALSE. It is the identifier for the multivariate super object as regards level. It shows up in output files names and content. If not set the default value is "NA". It can be set separately for forecasts and observations, or as a common value for both. +The **multivar_level** entry appears only in the **MODEMultivarConfig_default** file. This option is used only when the multivar option is enabled, and only when **multivar_intensity_compare_fcst** and **multivar_intensity_compare_obs** are empty. It is the identifier for the multivariate super object as regards level. It shows up in output files names and content. If not set the default value is "NA". It can be set separately for forecasts and observations, or as a common value for both. _____________________ diff --git a/internal/test_unit/config/MODEConfig_multivar b/internal/test_unit/config/MODEConfig_multivar new file mode 100644 index 0000000000..d3fba7556d --- /dev/null +++ b/internal/test_unit/config/MODEConfig_multivar @@ -0,0 +1,274 @@ +//////////////////////////////////////////////////////////////////////////////// +// +// MODE configuration file. +// +// For additional information, see the MET_BASE/config/README file. +// +//////////////////////////////////////////////////////////////////////////////// + +// +// Output model name to be written +// +// model = +model = "HRRR"; + +// +// Output description to be written +// +// desc = + +// +// Output observation type to be written +// +// obtype = +obtype = "ANALYSIS"; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification grid +// +regrid = {to_grid = FCST;method = NEAREST;width = 1;vld_thresh = 0.5;} + +//////////////////////////////////////////////////////////////////////////////// + +// +`// Approximate grid resolution (km) +// +// grid_res = +// + +//////////////////////////////////////////////////////////////////////////////// + +// +// Run all permutations of radius and threshold +// +// quilt = +// ${METPLUS_QUILT} + +// +// MODE Multivar boolean combination logic +// +//multivar_logic = +//${METPLUS_MULTIVAR_LOGIC} +multivar_logic = "#1 && #2 && #3"; + +// keep this around to compare to older versions, the new version doesn't read this +multivar_intensity_flag = [FALSE, TRUE, TRUE]; + +multivar_intensity_compare_fcst = [ 2, 3 ]; +multivar_intensity_compare_obs = [ 2, 3 ]; + +// +// Forecast and observation fields to be verified +// +fcst = { + //${METPLUS_FCST_FIELD} + field = [{ name="CSNOW"; level="L0"; conv_radius = 0; conv_thresh = ==1; file_type = GRIB2; merge_flag = NONE; }, + { name="VIS"; level="L0"; conv_radius = 5; conv_thresh = <=804.672; merge_thresh = <=1207.008; merge_flag = THRESH; }, + { name="WIND"; level="Z10"; conv_radius = 5; conv_thresh = >=8.9408; merge_thresh = >=6.7056; merge_flag = THRESH; }]; + + //${METPLUS_FCST_CENSOR_THRESH} + //${METPLUS_FCST_CENSOR_VAL} + //${METPLUS_FCST_CONV_RADIUS} + //${METPLUS_FCST_CONV_THRESH} + //${METPLUS_FCST_VLD_THRESH} + //${METPLUS_FCST_FILTER_ATTR_NAME} + filter_attr_name = ["AREA"]; + + //${METPLUS_FCST_FILTER_ATTR_THRESH} + filter_attr_thresh = [>=25]; + + //${METPLUS_FCST_MERGE_THRESH} + //${METPLUS_FCST_MERGE_FLAG} + //${METPLUS_FCST_FILE_TYPE} + //file_type = GRIB2; + multivar_name = "Snow"; + multivar_level = "LO"; +} + +obs = { + //${METPLUS_OBS_FIELD} + field = [{ name="PrecipFlag"; level="L0"; conv_radius = 0; conv_thresh = ==3; file_type = GRIB2; merge_flag = NONE; }, + { name="VIS"; level="L0"; conv_radius = 5; conv_thresh = <=804.672; merge_thresh = <=1207.008; merge_flag = THRESH; }, + { name="WIND"; level="Z10"; conv_radius = 5; conv_thresh = >=8.9408; merge_thresh = >=6.7056; merge_flag = THRESH; } ]; + + //${METPLUS_OBS_CENSOR_THRESH} + //${METPLUS_OBS_CENSOR_VAL} + //${METPLUS_OBS_CONV_RADIUS} + //${METPLUS_OBS_CONV_THRESH} + //${METPLUS_OBS_VLD_THRESH} + + //${METPLUS_OBS_FILTER_ATTR_NAME} + filter_attr_name = ["AREA"]; + + //${METPLUS_OBS_FILTER_ATTR_THRESH} + filter_attr_thresh = [>=25]; + + // ${METPLUS_OBS_MERGE_THRESH} + // ${METPLUS_OBS_MERGE_FLAG} + // ${METPLUS_OBS_FILE_TYPE} + multivar_name = "Precip"; + multivar_level = "LO"; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Handle missing data +// +// mask_missing_flag = +// ${METPLUS_MASK_MISSING_FLAG} + +// +// Match objects between the forecast and observation fields +// +//match_flag = +//${METPLUS_MATCH_FLAG} +match_flag = MERGE_BOTH; +// +// Maximum centroid distance for objects to be compared +// +//max_centroid_dist = +//${METPLUS_MAX_CENTROID_DIST} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification masking regions +// +//mask = { +//${METPLUS_MASK_DICT} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Fuzzy engine weights +// +//weight = { +//${METPLUS_WEIGHT_DICT} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Fuzzy engine interest functions +// +interest_function = { + + //${METPLUS_INTEREST_FUNCTION_CENTROID_DIST} + + //${METPLUS_INTEREST_FUNCTION_BOUNDARY_DIST} + + //${METPLUS_INTEREST_FUNCTION_CONVEX_HULL_DIST} + + angle_diff = ( + ( 0.0, 1.0 ) + ( 30.0, 1.0 ) + ( 90.0, 0.0 ) + ); + + aspect_diff = ( + ( 0.00, 1.0 ) + ( 0.10, 1.0 ) + ( 0.75, 0.0 ) + ); + + corner = 0.8; + ratio_if = ( + ( 0.0, 0.0 ) + ( corner, 1.0 ) + ( 1.0, 1.0 ) + ); + + area_ratio = ratio_if; + + int_area_ratio = ( + ( 0.00, 0.00 ) + ( 0.10, 0.50 ) + ( 0.25, 1.00 ) + ( 1.00, 1.00 ) + ); + + curvature_ratio = ratio_if; + + complexity_ratio = ratio_if; + + inten_perc_ratio = ratio_if; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Total interest threshold for determining matches +// +//total_interest_thresh = +//${METPLUS_TOTAL_INTEREST_THRESH} + +// +// Interest threshold for printing output pair information +// +print_interest_thresh = 0.0; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Plotting information +// +met_data_dir = "MET_BASE"; + +fcst_raw_plot = { + color_table = "MET_BASE/colortables/met_default.ctable"; + plot_min = 0.0; + plot_max = 0.0; +} + +obs_raw_plot = { + color_table = "MET_BASE/colortables/met_default.ctable"; + plot_min = 0.0; + plot_max = 0.0; +} + +object_plot = { + color_table = "MET_BASE/colortables/mode_obj.ctable"; +} + +// +// Boolean for plotting on the region of valid data within the domain +// +plot_valid_flag = FALSE; + +// +// Plot polyline edges using great circle arcs instead of straight lines +// +plot_gcarc_flag = FALSE; + +//////////////////////////////////////////////////////////////////////////////// + +// +// NetCDF matched pairs, PostScript, and contingency table output files +// +//ps_plot_flag = +//${METPLUS_PS_PLOT_FLAG} + +//nc_pairs_flag = { +//${METPLUS_NC_PAIRS_FLAG_DICT} + +//ct_stats_flag = +//${METPLUS_CT_STATS_FLAG} + + +//////////////////////////////////////////////////////////////////////////////// + +shift_right = 0; // grid squares + +//////////////////////////////////////////////////////////////////////////////// + +//${METPLUS_OUTPUT_PREFIX} +//version = "V10.0"; + +//tmp_dir = "${MET_TMP_DIR}"; +tmp_dir = "/tmp"; + +//////////////////////////////////////////////////////////////////////////////// + +//${METPLUS_MET_CONFIG_OVERRIDES} diff --git a/internal/test_unit/config/MODEConfig_multivar_3_2 b/internal/test_unit/config/MODEConfig_multivar_3_2 new file mode 100644 index 0000000000..716b93c9be --- /dev/null +++ b/internal/test_unit/config/MODEConfig_multivar_3_2 @@ -0,0 +1,275 @@ +//////////////////////////////////////////////////////////////////////////////// +// +// Mode configuration file. +// +// For additional information, see the MET_BASE/config/README file. +// +//////////////////////////////////////////////////////////////////////////////// + +// +// Output model name to be written +// +// model = +model = "HRRR"; + +// +// Output description to be written +// +// desc = + +// +// Output observation type to be written +// +// obtype = +obtype = "ANALYSIS"; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification grid +// +// regrid = { +regrid = {to_grid = FCST;method = NEAREST;width = 1;vld_thresh = 0.5;} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Approximate grid resolution (km) +// +// grid_res = +// + +//////////////////////////////////////////////////////////////////////////////// + +// +// Run all permutations of radius and threshold +// +// quilt = +// ${METPLUS_QUILT} + +// +// MODE Multivar boolean combination logic +// +//multivar_logic = +//${METPLUS_MULTIVAR_LOGIC} +multivar_logic = "#1 && #2 && #3"; + +multivar_intensity_compare_fcst = [ 2, 2 ]; +multivar_intensity_compare_obs = [ 1, 2 ]; + +// +// Forecast and observation fields to be verified +// +fcst = { + //${METPLUS_FCST_FIELD} + field = [{ name="CSNOW"; level="L0"; conv_radius = 0; conv_thresh = ==1; file_type = GRIB2; merge_flag = NONE; }, + { name="VIS"; level="L0"; conv_radius = 5; conv_thresh = <=804.672; merge_thresh = <=1207.008; merge_flag = THRESH; }, + { name="WIND"; level="Z10"; conv_radius = 5; conv_thresh = >=8.9408; merge_thresh = >=6.7056; merge_flag = THRESH; } ]; + + multivar_logic = "#1 && #2 && #3"; + + //${METPLUS_FCST_CENSOR_THRESH} + //${METPLUS_FCST_CENSOR_VAL} + //${METPLUS_FCST_CONV_RADIUS} + //${METPLUS_FCST_CONV_THRESH} + //${METPLUS_FCST_VLD_THRESH} + //${METPLUS_FCST_FILTER_ATTR_NAME} + filter_attr_name = ["AREA"]; + + //${METPLUS_FCST_FILTER_ATTR_THRESH} + filter_attr_thresh = [>=25]; + + //${METPLUS_FCST_MERGE_THRESH} + //${METPLUS_FCST_MERGE_FLAG} + //${METPLUS_FCST_FILE_TYPE} + //file_type = GRIB2; + multivar_name = "Snow"; + multivar_level = "LO"; +} + +obs = { + //${METPLUS_OBS_FIELD} + field = [{ name="VIS"; level="L0"; conv_radius = 5; conv_thresh = <=804.672; merge_thresh = <=1207.008; merge_flag = THRESH; }, + { name="WIND"; level="Z10"; conv_radius = 5; conv_thresh = >=8.9408; merge_thresh = >=6.7056; merge_flag = THRESH; } ]; + + multivar_logic = "#1 && #2"; + + //${METPLUS_OBS_CENSOR_THRESH} + //${METPLUS_OBS_CENSOR_VAL} + //${METPLUS_OBS_CONV_RADIUS} + //${METPLUS_OBS_CONV_THRESH} + //${METPLUS_OBS_VLD_THRESH} + + //${METPLUS_OBS_FILTER_ATTR_NAME} + filter_attr_name = ["AREA"]; + + //${METPLUS_OBS_FILTER_ATTR_THRESH} + filter_attr_thresh = [>=25]; + + // ${METPLUS_OBS_MERGE_THRESH} + // ${METPLUS_OBS_MERGE_FLAG} + // ${METPLUS_OBS_FILE_TYPE} + multivar_name = "Precip"; + multivar_level = "LO"; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Handle missing data +// +// mask_missing_flag = +// ${METPLUS_MASK_MISSING_FLAG} + +// +// Match objects between the forecast and observation fields +// +//match_flag = +//${METPLUS_MATCH_FLAG} +match_flag = MERGE_BOTH; +// +// Maximum centroid distance for objects to be compared +// +//max_centroid_dist = +//${METPLUS_MAX_CENTROID_DIST} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification masking regions +// +//mask = { +//${METPLUS_MASK_DICT} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Fuzzy engine weights +// +//weight = { +//${METPLUS_WEIGHT_DICT} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Fuzzy engine interest functions +// +interest_function = { + + //${METPLUS_INTEREST_FUNCTION_CENTROID_DIST} + + //${METPLUS_INTEREST_FUNCTION_BOUNDARY_DIST} + + //${METPLUS_INTEREST_FUNCTION_CONVEX_HULL_DIST} + + angle_diff = ( + ( 0.0, 1.0 ) + ( 30.0, 1.0 ) + ( 90.0, 0.0 ) + ); + + aspect_diff = ( + ( 0.00, 1.0 ) + ( 0.10, 1.0 ) + ( 0.75, 0.0 ) + ); + + corner = 0.8; + ratio_if = ( + ( 0.0, 0.0 ) + ( corner, 1.0 ) + ( 1.0, 1.0 ) + ); + + area_ratio = ratio_if; + + int_area_ratio = ( + ( 0.00, 0.00 ) + ( 0.10, 0.50 ) + ( 0.25, 1.00 ) + ( 1.00, 1.00 ) + ); + + curvature_ratio = ratio_if; + + complexity_ratio = ratio_if; + + inten_perc_ratio = ratio_if; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Total interest threshold for determining matches +// +//total_interest_thresh = +//${METPLUS_TOTAL_INTEREST_THRESH} + +// +// Interest threshold for printing output pair information +// +print_interest_thresh = 0.0; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Plotting information +// +met_data_dir = "MET_BASE"; + +fcst_raw_plot = { + color_table = "MET_BASE/colortables/met_default.ctable"; + plot_min = 0.0; + plot_max = 0.0; +} + +obs_raw_plot = { + color_table = "MET_BASE/colortables/met_default.ctable"; + plot_min = 0.0; + plot_max = 0.0; +} + +object_plot = { + color_table = "MET_BASE/colortables/mode_obj.ctable"; +} + +// +// Boolean for plotting on the region of valid data within the domain +// +plot_valid_flag = FALSE; + +// +// Plot polyline edges using great circle arcs instead of straight lines +// +plot_gcarc_flag = FALSE; + +//////////////////////////////////////////////////////////////////////////////// + +// +// NetCDF matched pairs, PostScript, and contingency table output files +// +//ps_plot_flag = +//${METPLUS_PS_PLOT_FLAG} + +//nc_pairs_flag = { +//${METPLUS_NC_PAIRS_FLAG_DICT} + +//ct_stats_flag = +//${METPLUS_CT_STATS_FLAG} + + +//////////////////////////////////////////////////////////////////////////////// + +shift_right = 0; // grid squares + +//////////////////////////////////////////////////////////////////////////////// + +//${METPLUS_OUTPUT_PREFIX} +//version = "V10.0"; + +//tmp_dir = "${MET_TMP_DIR}"; +tmp_dir = "/tmp"; + +//////////////////////////////////////////////////////////////////////////////// + +//${METPLUS_MET_CONFIG_OVERRIDES} diff --git a/internal/test_unit/config/MODEConfig_multivar_super b/internal/test_unit/config/MODEConfig_multivar_super new file mode 100644 index 0000000000..e035f6ba63 --- /dev/null +++ b/internal/test_unit/config/MODEConfig_multivar_super @@ -0,0 +1,271 @@ +//////////////////////////////////////////////////////////////////////////////// +// +// MODE configuration file. +// +// For additional information, see the MET_BASE/config/README file. +// +//////////////////////////////////////////////////////////////////////////////// + +// +// Output model name to be written +// +// model = +model = "HRRR"; + +// +// Output description to be written +// +// desc = + +// +// Output observation type to be written +// +// obtype = +obtype = "ANALYSIS"; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification grid +// +regrid = {to_grid = FCST;method = NEAREST;width = 1;vld_thresh = 0.5;} + +//////////////////////////////////////////////////////////////////////////////// + +// +`// Approximate grid resolution (km) +// +// grid_res = +// + +//////////////////////////////////////////////////////////////////////////////// + +// +// Run all permutations of radius and threshold +// +// quilt = +// ${METPLUS_QUILT} + +// +// MODE Multivar boolean combination logic +// +//multivar_logic = +//${METPLUS_MULTIVAR_LOGIC} +multivar_logic = "#1 && #2 && #3"; + +multivar_intensity_compare_fcst = [ ]; +multivar_intensity_compare_obs = [ ]; + +// +// Forecast and observation fields to be verified +// +fcst = { + //${METPLUS_FCST_FIELD} + field = [{ name="CSNOW"; level="L0"; conv_radius = 0; conv_thresh = ==1; file_type = GRIB2; merge_flag = NONE; }, + { name="VIS"; level="L0"; conv_radius = 5; conv_thresh = <=804.672; merge_thresh = <=1207.008; merge_flag = THRESH; }, + { name="WIND"; level="Z10"; conv_radius = 5; conv_thresh = >=8.9408; merge_thresh = >=6.7056; merge_flag = THRESH; } ]; + + //${METPLUS_FCST_CENSOR_THRESH} + //${METPLUS_FCST_CENSOR_VAL} + //${METPLUS_FCST_CONV_RADIUS} + //${METPLUS_FCST_CONV_THRESH} + //${METPLUS_FCST_VLD_THRESH} + //${METPLUS_FCST_FILTER_ATTR_NAME} + filter_attr_name = ["AREA"]; + + //${METPLUS_FCST_FILTER_ATTR_THRESH} + filter_attr_thresh = [>=25]; + + //${METPLUS_FCST_MERGE_THRESH} + //${METPLUS_FCST_MERGE_FLAG} + //${METPLUS_FCST_FILE_TYPE} + //file_type = GRIB2; + multivar_name = "Snow"; + multivar_level = "LO"; +} + +obs = { + //${METPLUS_OBS_FIELD} + field = [{ name="PrecipFlag"; level="L0"; conv_radius = 0; conv_thresh = ==3; file_type = GRIB2; merge_flag = NONE; }, + { name="VIS"; level="L0"; conv_radius = 5; conv_thresh = <=804.672; merge_thresh = <=1207.008; merge_flag = THRESH; }, + { name="WIND"; level="Z10"; conv_radius = 5; conv_thresh = >=8.9408; merge_thresh = >=6.7056; merge_flag = THRESH; } ]; + + //${METPLUS_OBS_CENSOR_THRESH} + //${METPLUS_OBS_CENSOR_VAL} + //${METPLUS_OBS_CONV_RADIUS} + //${METPLUS_OBS_CONV_THRESH} + //${METPLUS_OBS_VLD_THRESH} + + //${METPLUS_OBS_FILTER_ATTR_NAME} + filter_attr_name = ["AREA"]; + + //${METPLUS_OBS_FILTER_ATTR_THRESH} + filter_attr_thresh = [>=25]; + + // ${METPLUS_OBS_MERGE_THRESH} + // ${METPLUS_OBS_MERGE_FLAG} + // ${METPLUS_OBS_FILE_TYPE} + multivar_name = "Precip"; + multivar_level = "LO"; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Handle missing data +// +// mask_missing_flag = +// ${METPLUS_MASK_MISSING_FLAG} + +// +// Match objects between the forecast and observation fields +// +//match_flag = +//${METPLUS_MATCH_FLAG} +match_flag = MERGE_BOTH; +// +// Maximum centroid distance for objects to be compared +// +//max_centroid_dist = +//${METPLUS_MAX_CENTROID_DIST} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Verification masking regions +// +//mask = { +//${METPLUS_MASK_DICT} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Fuzzy engine weights +// +//weight = { +//${METPLUS_WEIGHT_DICT} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Fuzzy engine interest functions +// +interest_function = { + + //${METPLUS_INTEREST_FUNCTION_CENTROID_DIST} + + //${METPLUS_INTEREST_FUNCTION_BOUNDARY_DIST} + + //${METPLUS_INTEREST_FUNCTION_CONVEX_HULL_DIST} + + angle_diff = ( + ( 0.0, 1.0 ) + ( 30.0, 1.0 ) + ( 90.0, 0.0 ) + ); + + aspect_diff = ( + ( 0.00, 1.0 ) + ( 0.10, 1.0 ) + ( 0.75, 0.0 ) + ); + + corner = 0.8; + ratio_if = ( + ( 0.0, 0.0 ) + ( corner, 1.0 ) + ( 1.0, 1.0 ) + ); + + area_ratio = ratio_if; + + int_area_ratio = ( + ( 0.00, 0.00 ) + ( 0.10, 0.50 ) + ( 0.25, 1.00 ) + ( 1.00, 1.00 ) + ); + + curvature_ratio = ratio_if; + + complexity_ratio = ratio_if; + + inten_perc_ratio = ratio_if; +} + +//////////////////////////////////////////////////////////////////////////////// + +// +// Total interest threshold for determining matches +// +//total_interest_thresh = +//${METPLUS_TOTAL_INTEREST_THRESH} + +// +// Interest threshold for printing output pair information +// +print_interest_thresh = 0.0; + +//////////////////////////////////////////////////////////////////////////////// + +// +// Plotting information +// +met_data_dir = "MET_BASE"; + +fcst_raw_plot = { + color_table = "MET_BASE/colortables/met_default.ctable"; + plot_min = 0.0; + plot_max = 0.0; +} + +obs_raw_plot = { + color_table = "MET_BASE/colortables/met_default.ctable"; + plot_min = 0.0; + plot_max = 0.0; +} + +object_plot = { + color_table = "MET_BASE/colortables/mode_obj.ctable"; +} + +// +// Boolean for plotting on the region of valid data within the domain +// +plot_valid_flag = FALSE; + +// +// Plot polyline edges using great circle arcs instead of straight lines +// +plot_gcarc_flag = FALSE; + +//////////////////////////////////////////////////////////////////////////////// + +// +// NetCDF matched pairs, PostScript, and contingency table output files +// +//ps_plot_flag = +//${METPLUS_PS_PLOT_FLAG} + +//nc_pairs_flag = { +//${METPLUS_NC_PAIRS_FLAG_DICT} + +//ct_stats_flag = +//${METPLUS_CT_STATS_FLAG} + + +//////////////////////////////////////////////////////////////////////////////// + +shift_right = 0; // grid squares + +//////////////////////////////////////////////////////////////////////////////// + +//${METPLUS_OUTPUT_PREFIX} +//version = "V10.0"; + +//tmp_dir = "${MET_TMP_DIR}"; +tmp_dir = "/tmp"; + +//////////////////////////////////////////////////////////////////////////////// + +//${METPLUS_MET_CONFIG_OVERRIDES} diff --git a/internal/test_unit/xml/unit_mode_multivar.xml b/internal/test_unit/xml/unit_mode_multivar.xml index 4aad5dc7d6..aea7f7f2a0 100644 --- a/internal/test_unit/xml/unit_mode_multivar.xml +++ b/internal/test_unit/xml/unit_mode_multivar.xml @@ -16,7 +16,90 @@ &TEST_DIR; true - + + echo "&DATA_DIR_MODEL;/mode_multivar/hrrr/2021020100/hrrr.t00z.wrfprsf21.sub.grib2 \ + &DATA_DIR_MODEL;/mode_multivar/hrrr/2021020100/hrrr.t00z.wrfprsf21.sub.grib2 \ + &DATA_DIR_MODEL;/mode_multivar/hrrr/2021020100/hrrr.t00z.wrfprsf21.sub.grib2" \ + > &OUTPUT_DIR;/mode_multivar/input_fcst_file_list; \ + echo "&DATA_DIR_OBS;/mode_multivar/mrms/20210201/PrecipFlag_00.00_20210201-210000.sub.grib2 \ + &DATA_DIR_OBS;/mode_multivar/hrrr/anl/20210201/hrrr.t21z.wrfprsf00.sub.grib2 \ + &DATA_DIR_OBS;/mode_multivar/hrrr/anl/20210201/hrrr.t21z.wrfprsf00.sub.grib2" \ + > &OUTPUT_DIR;/mode_multivar/input_obs_file_list; \ + &MET_BIN;/mode + \ + &OUTPUT_DIR;/mode_multivar/input_fcst_file_list \ + &OUTPUT_DIR;/mode_multivar/input_obs_file_list \ + &CONFIG_DIR;/MODEConfig_multivar \ + -outdir &OUTPUT_DIR;/mode_multivar \ + -v 2 + + + &OUTPUT_DIR;/mode_multivar/mode_Fcst_VIS_L0_Obs_VIS_L0_210000L_20210201_210000V_000000A_obj.nc + &OUTPUT_DIR;/mode_multivar/mode_Fcst_VIS_L0_Obs_VIS_L0_210000L_20210201_210000V_000000A_obj.txt + &OUTPUT_DIR;/mode_multivar/mode_Fcst_VIS_L0_Obs_VIS_L0_210000L_20210201_210000V_000000A_cts.txt + &OUTPUT_DIR;/mode_multivar/mode_Fcst_VIS_L0_Obs_VIS_L0_210000L_20210201_210000V_000000A.ps + &OUTPUT_DIR;/mode_multivar/mode_Fcst_WIND_Z10_Obs_WIND_Z10_210000L_20210201_210000V_000000A_obj.nc + &OUTPUT_DIR;/mode_multivar/mode_Fcst_WIND_Z10_Obs_WIND_Z10_210000L_20210201_210000V_000000A_obj.txt + &OUTPUT_DIR;/mode_multivar/mode_Fcst_WIND_Z10_Obs_WIND_Z10_210000L_20210201_210000V_000000A_cts.txt + &OUTPUT_DIR;/mode_multivar/mode_Fcst_WIND_Z10_Obs_WIND_Z10_210000L_20210201_210000V_000000A.ps + + + + + + echo "&DATA_DIR_MODEL;/mode_multivar/hrrr/2021020100/hrrr.t00z.wrfprsf21.sub.grib2 \ + &DATA_DIR_MODEL;/mode_multivar/hrrr/2021020100/hrrr.t00z.wrfprsf21.sub.grib2 \ + &DATA_DIR_MODEL;/mode_multivar/hrrr/2021020100/hrrr.t00z.wrfprsf21.sub.grib2" \ + > &OUTPUT_DIR;/mode_multivar/input_fcst_file_list; \ + echo "&DATA_DIR_OBS;/mode_multivar/hrrr/anl/20210201/hrrr.t21z.wrfprsf00.sub.grib2 \ + &DATA_DIR_OBS;/mode_multivar/hrrr/anl/20210201/hrrr.t21z.wrfprsf00.sub.grib2" \ + > &OUTPUT_DIR;/mode_multivar/input_obs_file_list; \ + &MET_BIN;/mode + \ + &OUTPUT_DIR;/mode_multivar/input_fcst_file_list \ + &OUTPUT_DIR;/mode_multivar/input_obs_file_list \ + &CONFIG_DIR;/MODEConfig_multivar_3_2 \ + -outdir &OUTPUT_DIR;/mode_multivar \ + -v 2 + + + &OUTPUT_DIR;/mode_multivar/mode_Fcst_VIS_L0_Obs_VIS_L0_210000L_20210201_210000V_000000A_obj.nc + &OUTPUT_DIR;/mode_multivar/mode_Fcst_VIS_L0_Obs_VIS_L0_210000L_20210201_210000V_000000A_obj.txt + &OUTPUT_DIR;/mode_multivar/mode_Fcst_VIS_L0_Obs_VIS_L0_210000L_20210201_210000V_000000A_cts.txt + &OUTPUT_DIR;/mode_multivar/mode_Fcst_VIS_L0_Obs_VIS_L0_210000L_20210201_210000V_000000A.ps + &OUTPUT_DIR;/mode_multivar/mode_Fcst_VIS_L0_Obs_WIND_Z10_210000L_20210201_210000V_000000A_obj.nc + &OUTPUT_DIR;/mode_multivar/mode_Fcst_VIS_L0_Obs_WIND_Z10_210000L_20210201_210000V_000000A_obj.txt + &OUTPUT_DIR;/mode_multivar/mode_Fcst_VIS_L0_Obs_WIND_Z10_210000L_20210201_210000V_000000A_cts.txt + &OUTPUT_DIR;/mode_multivar/mode_Fcst_VIS_L0_Obs_WIND_Z10_210000L_20210201_210000V_000000A.ps + + + + + echo "&DATA_DIR_MODEL;/mode_multivar/hrrr/2021020100/hrrr.t00z.wrfprsf21.sub.grib2 \ + &DATA_DIR_MODEL;/mode_multivar/hrrr/2021020100/hrrr.t00z.wrfprsf21.sub.grib2 \ + &DATA_DIR_MODEL;/mode_multivar/hrrr/2021020100/hrrr.t00z.wrfprsf21.sub.grib2" \ + > &OUTPUT_DIR;/mode_multivar/input_fcst_file_list; \ + echo "&DATA_DIR_OBS;/mode_multivar/mrms/20210201/PrecipFlag_00.00_20210201-210000.sub.grib2 \ + &DATA_DIR_OBS;/mode_multivar/hrrr/anl/20210201/hrrr.t21z.wrfprsf00.sub.grib2 \ + &DATA_DIR_OBS;/mode_multivar/hrrr/anl/20210201/hrrr.t21z.wrfprsf00.sub.grib2" \ + > &OUTPUT_DIR;/mode_multivar/input_obs_file_list; \ + &MET_BIN;/mode + \ + &OUTPUT_DIR;/mode_multivar/input_fcst_file_list \ + &OUTPUT_DIR;/mode_multivar/input_obs_file_list \ + &CONFIG_DIR;/MODEConfig_multivar_super \ + -outdir &OUTPUT_DIR;/mode_multivar \ + -v 2 + + + &OUTPUT_DIR;/mode_multivar/mode_Fcst_Snow_LO_Obs_Precip_LO_210000L_20210201_210000V_000000A_obj.nc + &OUTPUT_DIR;/mode_multivar/mode_Fcst_Snow_LO_Obs_Precip_LO_210000L_20210201_210000V_000000A_obj.txt + &OUTPUT_DIR;/mode_multivar/mode_Fcst_Snow_LO_Obs_Precip_LO_210000L_20210201_210000V_000000A_cts.txt + &OUTPUT_DIR;/mode_multivar/mode_Fcst_Snow_LO_Obs_Precip_LO_210000L_20210201_210000V_000000A.ps + + + + echo "&DATA_DIR_MODEL;/mode_multivar/alpha_fcst.nc \ &DATA_DIR_MODEL;/mode_multivar/beta_fcst.nc \ &DATA_DIR_MODEL;/mode_multivar/gamma_fcst.nc" \ diff --git a/src/basic/vx_config/config_file.cc b/src/basic/vx_config/config_file.cc index d349b2d140..0ca0bf5aa0 100644 --- a/src/basic/vx_config/config_file.cc +++ b/src/basic/vx_config/config_file.cc @@ -115,6 +115,20 @@ assign(c); } +//////////////////////////////////////////////////////////////////////// + +MetConfig & MetConfig::operator=(const MetConfig &s) +{ + if(this == &s) return(*this); + + init_from_scratch(); + + assign(s); + + return(*this); + +} + //////////////////////////////////////////////////////////////////////// diff --git a/src/libcode/vx_data2d/var_info.h b/src/libcode/vx_data2d/var_info.h index 7dcf6e386a..15fa2f90b7 100644 --- a/src/libcode/vx_data2d/var_info.h +++ b/src/libcode/vx_data2d/var_info.h @@ -91,11 +91,14 @@ class VarInfo VarInfo(const VarInfo &); VarInfo & operator=(const VarInfo &); + virtual VarInfo *clone() const = 0; + // Conversion function UserFunc_1Arg ConvertFx; void clear(); - + void clone_base() const; + virtual void dump(std::ostream &) const; // diff --git a/src/libcode/vx_data2d_grib/var_info_grib.cc b/src/libcode/vx_data2d_grib/var_info_grib.cc index c414bc94f8..e6bb74198f 100644 --- a/src/libcode/vx_data2d_grib/var_info_grib.cc +++ b/src/libcode/vx_data2d_grib/var_info_grib.cc @@ -75,6 +75,15 @@ VarInfoGrib & VarInfoGrib::operator=(const VarInfoGrib &f) { /////////////////////////////////////////////////////////////////////////////// +VarInfo *VarInfoGrib::clone() const { + + VarInfoGrib *ret = new VarInfoGrib(*this); + + return (VarInfo *)ret; +} + +/////////////////////////////////////////////////////////////////////////////// + void VarInfoGrib::init_from_scratch() { // First call the parent's initialization diff --git a/src/libcode/vx_data2d_grib/var_info_grib.h b/src/libcode/vx_data2d_grib/var_info_grib.h index cdf0947b90..d695eed007 100644 --- a/src/libcode/vx_data2d_grib/var_info_grib.h +++ b/src/libcode/vx_data2d_grib/var_info_grib.h @@ -67,6 +67,7 @@ class VarInfoGrib : public VarInfo ~VarInfoGrib(); VarInfoGrib(const VarInfoGrib &); VarInfoGrib & operator=(const VarInfoGrib &); + VarInfo *clone() const; void dump(std::ostream &) const; void clear(); diff --git a/src/libcode/vx_data2d_grib2/var_info_grib2.cc b/src/libcode/vx_data2d_grib2/var_info_grib2.cc index 6ea70eecfa..ef3a3c7db6 100644 --- a/src/libcode/vx_data2d_grib2/var_info_grib2.cc +++ b/src/libcode/vx_data2d_grib2/var_info_grib2.cc @@ -77,6 +77,15 @@ VarInfoGrib2 & VarInfoGrib2::operator=(const VarInfoGrib2 &f) { /////////////////////////////////////////////////////////////////////////////// +VarInfo *VarInfoGrib2::clone() const { + + VarInfoGrib2 *ret = new VarInfoGrib2(*this); + + return (VarInfo *)ret; +} + +/////////////////////////////////////////////////////////////////////////////// + void VarInfoGrib2::init_from_scratch() { // First call the parent's initialization diff --git a/src/libcode/vx_data2d_grib2/var_info_grib2.h b/src/libcode/vx_data2d_grib2/var_info_grib2.h index 619508358b..5f8e320339 100644 --- a/src/libcode/vx_data2d_grib2/var_info_grib2.h +++ b/src/libcode/vx_data2d_grib2/var_info_grib2.h @@ -72,6 +72,7 @@ class VarInfoGrib2 : public VarInfo ~VarInfoGrib2(); VarInfoGrib2(const VarInfoGrib2 &); VarInfoGrib2 & operator=(const VarInfoGrib2 &); + VarInfo *clone() const; void dump(std::ostream &) const; void clear(); diff --git a/src/libcode/vx_data2d_nc_cf/var_info_nc_cf.cc b/src/libcode/vx_data2d_nc_cf/var_info_nc_cf.cc index 318f4524bd..dba8bd4ec5 100644 --- a/src/libcode/vx_data2d_nc_cf/var_info_nc_cf.cc +++ b/src/libcode/vx_data2d_nc_cf/var_info_nc_cf.cc @@ -76,6 +76,15 @@ VarInfoNcCF & VarInfoNcCF::operator=(const VarInfoNcCF &f) { /////////////////////////////////////////////////////////////////////////////// +VarInfo *VarInfoNcCF::clone() const { + + VarInfoNcCF *ret = new VarInfoNcCF(*this); + + return (VarInfo *)ret; +} + +/////////////////////////////////////////////////////////////////////////////// + void VarInfoNcCF::init_from_scratch() { // First call the parent's initialization diff --git a/src/libcode/vx_data2d_nc_cf/var_info_nc_cf.h b/src/libcode/vx_data2d_nc_cf/var_info_nc_cf.h index 2463bcbde2..ce20289c2c 100644 --- a/src/libcode/vx_data2d_nc_cf/var_info_nc_cf.h +++ b/src/libcode/vx_data2d_nc_cf/var_info_nc_cf.h @@ -45,6 +45,7 @@ class VarInfoNcCF : public VarInfo ~VarInfoNcCF(); VarInfoNcCF(const VarInfoNcCF &); VarInfoNcCF & operator=(const VarInfoNcCF &); + VarInfo *clone() const; void dump(std::ostream &) const; void clear(); diff --git a/src/libcode/vx_data2d_nc_met/var_info_nc_met.cc b/src/libcode/vx_data2d_nc_met/var_info_nc_met.cc index 651832d69d..c1624d8785 100644 --- a/src/libcode/vx_data2d_nc_met/var_info_nc_met.cc +++ b/src/libcode/vx_data2d_nc_met/var_info_nc_met.cc @@ -76,6 +76,15 @@ VarInfoNcMet & VarInfoNcMet::operator=(const VarInfoNcMet &f) { /////////////////////////////////////////////////////////////////////////////// +VarInfo *VarInfoNcMet::clone() const { + + VarInfoNcMet *ret = new VarInfoNcMet(*this); + + return (VarInfo *)ret; +} + +/////////////////////////////////////////////////////////////////////////////// + void VarInfoNcMet::init_from_scratch() { // First call the parent's initialization diff --git a/src/libcode/vx_data2d_nc_met/var_info_nc_met.h b/src/libcode/vx_data2d_nc_met/var_info_nc_met.h index a9f0c66f53..f85bdfdb7d 100644 --- a/src/libcode/vx_data2d_nc_met/var_info_nc_met.h +++ b/src/libcode/vx_data2d_nc_met/var_info_nc_met.h @@ -41,6 +41,7 @@ class VarInfoNcMet : public VarInfo ~VarInfoNcMet(); VarInfoNcMet(const VarInfoNcMet &); VarInfoNcMet & operator=(const VarInfoNcMet &); + virtual VarInfo *clone() const; void dump(std::ostream &) const; void clear(); diff --git a/src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.cc b/src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.cc index eecf2270cb..351ddf302a 100644 --- a/src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.cc +++ b/src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.cc @@ -72,6 +72,15 @@ VarInfoNcWrf & VarInfoNcWrf::operator=(const VarInfoNcWrf &f) { /////////////////////////////////////////////////////////////////////////////// +VarInfo *VarInfoNcWrf::clone() const { + + VarInfoNcWrf *ret = new VarInfoNcWrf(*this); + + return (VarInfo *)ret; +} + +/////////////////////////////////////////////////////////////////////////////// + void VarInfoNcWrf::init_from_scratch() { // First call the parent's initialization diff --git a/src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.h b/src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.h index 182ad62d28..24569fe7e4 100644 --- a/src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.h +++ b/src/libcode/vx_data2d_nc_wrf/var_info_nc_wrf.h @@ -204,6 +204,7 @@ class VarInfoNcWrf : public VarInfo ~VarInfoNcWrf(); VarInfoNcWrf(const VarInfoNcWrf &); VarInfoNcWrf & operator=(const VarInfoNcWrf &); + VarInfo *clone() const; void dump(std::ostream &) const; void clear(); diff --git a/src/libcode/vx_data2d_python/var_info_python.cc b/src/libcode/vx_data2d_python/var_info_python.cc index 5c111ec28b..74d8767816 100644 --- a/src/libcode/vx_data2d_python/var_info_python.cc +++ b/src/libcode/vx_data2d_python/var_info_python.cc @@ -75,6 +75,15 @@ VarInfoPython & VarInfoPython::operator=(const VarInfoPython &f) { /////////////////////////////////////////////////////////////////////////////// +VarInfo *VarInfoPython::clone() const { + + VarInfoPython *ret = new VarInfoPython(*this); + + return (VarInfo *)ret; +} + +/////////////////////////////////////////////////////////////////////////////// + void VarInfoPython::init_from_scratch() { // First call the parent's initialization diff --git a/src/libcode/vx_data2d_python/var_info_python.h b/src/libcode/vx_data2d_python/var_info_python.h index 68309f8b4a..e38f9edae2 100644 --- a/src/libcode/vx_data2d_python/var_info_python.h +++ b/src/libcode/vx_data2d_python/var_info_python.h @@ -41,6 +41,7 @@ class VarInfoPython : public VarInfo ~VarInfoPython(); VarInfoPython(const VarInfoPython &); VarInfoPython & operator=(const VarInfoPython &); + VarInfo *clone() const; void dump(std::ostream &) const; void clear(); diff --git a/src/libcode/vx_data2d_ugrid/var_info_ugrid.cc b/src/libcode/vx_data2d_ugrid/var_info_ugrid.cc index e29fa44c95..9e74762a48 100644 --- a/src/libcode/vx_data2d_ugrid/var_info_ugrid.cc +++ b/src/libcode/vx_data2d_ugrid/var_info_ugrid.cc @@ -32,7 +32,6 @@ using namespace std; - /////////////////////////////////////////////////////////////////////////////// static bool is_grib_code_abbr_match(const ConcatString &, int); @@ -77,6 +76,15 @@ VarInfoUGrid & VarInfoUGrid::operator=(const VarInfoUGrid &f) { /////////////////////////////////////////////////////////////////////////////// +VarInfo *VarInfoUGrid::clone() const { + + VarInfoUGrid *ret = new VarInfoUGrid(*this); + + return (VarInfo *)ret; +} + +/////////////////////////////////////////////////////////////////////////////// + void VarInfoUGrid::init_from_scratch() { // First call the parent's initialization diff --git a/src/libcode/vx_data2d_ugrid/var_info_ugrid.h b/src/libcode/vx_data2d_ugrid/var_info_ugrid.h index b7c64b1014..e3bb5a1bb9 100644 --- a/src/libcode/vx_data2d_ugrid/var_info_ugrid.h +++ b/src/libcode/vx_data2d_ugrid/var_info_ugrid.h @@ -45,6 +45,7 @@ class VarInfoUGrid : public VarInfo ~VarInfoUGrid(); VarInfoUGrid(const VarInfoUGrid &); VarInfoUGrid & operator=(const VarInfoUGrid &); + VarInfo *clone() const; void dump(std::ostream &) const; void clear(); diff --git a/src/libcode/vx_shapedata/Makefile.am b/src/libcode/vx_shapedata/Makefile.am index f54e937535..ea52307df3 100644 --- a/src/libcode/vx_shapedata/Makefile.am +++ b/src/libcode/vx_shapedata/Makefile.am @@ -21,6 +21,7 @@ libvx_shapedata_a_SOURCES = \ mode_columns.h \ mode_field_info.cc mode_field_info.h \ mode_conf_info.cc mode_conf_info.h \ + mode_input_data.h \ engine.cc engine.h \ ihull.cc ihull.h \ vx_shapedata.h diff --git a/src/libcode/vx_shapedata/Makefile.in b/src/libcode/vx_shapedata/Makefile.in index df218fceda..290ee7582f 100644 --- a/src/libcode/vx_shapedata/Makefile.in +++ b/src/libcode/vx_shapedata/Makefile.in @@ -369,6 +369,7 @@ libvx_shapedata_a_SOURCES = \ mode_columns.h \ mode_field_info.cc mode_field_info.h \ mode_conf_info.cc mode_conf_info.h \ + mode_input_data.h \ engine.cc engine.h \ ihull.cc ihull.h \ vx_shapedata.h diff --git a/src/libcode/vx_shapedata/engine.cc b/src/libcode/vx_shapedata/engine.cc index e81c2d21bc..6087bf62bd 100644 --- a/src/libcode/vx_shapedata/engine.cc +++ b/src/libcode/vx_shapedata/engine.cc @@ -144,6 +144,7 @@ void ModeFuzzyEngine::init_from_scratch() { // // Reset all fcst and obs processing flags to initial state // + need_fcst_conv = true; need_fcst_thresh = true; need_fcst_filter = true; @@ -964,9 +965,7 @@ void ModeFuzzyEngine::do_obs_merging(const char *default_config, /////////////////////////////////////////////////////////////////////// -void ModeFuzzyEngine::do_fcst_merging(const char *default_config, - const char *merge_config, - const ShapeData &merge_data) +void ModeFuzzyEngine::do_fcst_merging(const ShapeData &merge_data) { if(need_fcst_thresh) do_fcst_thresholding(); @@ -985,7 +984,7 @@ void ModeFuzzyEngine::do_fcst_merging(const char *default_config, if(conf_info.Fcst->merge_flag == MergeType_Both || conf_info.Fcst->merge_flag == MergeType_Engine) - do_fcst_merge_engine(default_config, merge_config); + do_fcst_merge_engine("", ""); // // Done @@ -1000,9 +999,7 @@ void ModeFuzzyEngine::do_fcst_merging(const char *default_config, /////////////////////////////////////////////////////////////////////// -void ModeFuzzyEngine::do_obs_merging(const char *default_config, - const char *merge_config, - const ShapeData &merge_data) +void ModeFuzzyEngine::do_obs_merging(const ShapeData &merge_data) { if(need_obs_thresh) do_obs_thresholding(); @@ -1014,7 +1011,6 @@ void ModeFuzzyEngine::do_obs_merging(const char *default_config, << "inconsistent array dims\n\n"; exit(1); } - if(conf_info.Obs->merge_flag == MergeType_Both || conf_info.Obs->merge_flag == MergeType_Thresh) @@ -1022,7 +1018,7 @@ void ModeFuzzyEngine::do_obs_merging(const char *default_config, if(conf_info.Obs->merge_flag == MergeType_Both || conf_info.Obs->merge_flag == MergeType_Engine) - do_obs_merge_engine(default_config, merge_config); + do_obs_merge_engine("", ""); // // Done @@ -1724,8 +1720,8 @@ void ModeFuzzyEngine::do_fcst_merge_engine(const char *default_config, fcst_engine->ctable = ctable; if(default_config && merge_config) { fcst_engine->conf_info.read_config(default_config, merge_config); - fcst_engine->conf_info.process_config(conf_info.Fcst->var_info->file_type(), - conf_info.Obs->var_info->file_type()); + fcst_engine->conf_info.process_config_traditional(conf_info.Fcst->var_info->file_type(), + conf_info.Obs->var_info->file_type()); path = replace_path(fcst_engine->conf_info.object_pi.color_table.c_str()); fcst_engine->ctable.read(path.c_str()); } @@ -1891,8 +1887,8 @@ void ModeFuzzyEngine::do_obs_merge_engine(const char *default_config, obs_engine->ctable = ctable; if(default_config && merge_config) { obs_engine->conf_info.read_config(default_config, merge_config); - obs_engine->conf_info.process_config(conf_info.Fcst->var_info->file_type(), - conf_info.Obs->var_info->file_type()); + obs_engine->conf_info.process_config_traditional(conf_info.Fcst->var_info->file_type(), + conf_info.Obs->var_info->file_type()); path = replace_path(obs_engine->conf_info.object_pi.color_table.c_str()); obs_engine->ctable.read(path.c_str()); } diff --git a/src/libcode/vx_shapedata/engine.h b/src/libcode/vx_shapedata/engine.h index 8bc5995dfb..50d67628cf 100644 --- a/src/libcode/vx_shapedata/engine.h +++ b/src/libcode/vx_shapedata/engine.h @@ -215,16 +215,19 @@ class ModeFuzzyEngine { void do_fcst_merging(); void do_obs_merging(); + // traditional version void do_fcst_merging(const char *default_config, const char *merge_config); - void do_fcst_merging(const char *default_config, - const char *merge_config, - const ShapeData &merge_data); + + // multivar version + void do_fcst_merging(const ShapeData &merge_data); + + // traditional version void do_obs_merging(const char *default_config, const char *merge_config); - void do_obs_merging(const char *default_config, - const char *merge_config, - const ShapeData &merge_data); + + // multivar version + void do_obs_merging(const ShapeData &merge_data); void do_matching(); diff --git a/src/libcode/vx_shapedata/mode_conf_info.cc b/src/libcode/vx_shapedata/mode_conf_info.cc index a2ebee3c5b..9008152e6f 100644 --- a/src/libcode/vx_shapedata/mode_conf_info.cc +++ b/src/libcode/vx_shapedata/mode_conf_info.cc @@ -17,6 +17,7 @@ using namespace std; #include #include #include +#include #include "mode_conf_info.h" #include "configobjecttype_to_string.h" @@ -51,6 +52,16 @@ ModeConfInfo::~ModeConfInfo() clear(); } +ModeConfInfo & ModeConfInfo::operator=(const ModeConfInfo &s) +{ + if(this == &s) return(*this); + + clear(); + assign(s); + + return(*this); +} + //////////////////////////////////////////////////////////////////////// void ModeConfInfo::init_from_scratch() @@ -77,6 +88,99 @@ void ModeConfInfo::init_from_scratch() //////////////////////////////////////////////////////////////////////// +void ModeConfInfo::assign( const ModeConfInfo &m) +{ + Field_Index_f = m.Field_Index_f; + Field_Index_o = m.Field_Index_o; + N_fields_f = m.N_fields_f; + N_fields_o = m.N_fields_o; + fcst_multivar_logic = m.fcst_multivar_logic; + obs_multivar_logic = m.obs_multivar_logic; + fcst_multivar_compare_index = m.fcst_multivar_compare_index; + obs_multivar_compare_index = m.obs_multivar_compare_index; + fcst_multivar_name = m.fcst_multivar_name; + fcst_multivar_level = m.fcst_multivar_level; + obs_multivar_name = m.obs_multivar_name; + obs_multivar_level = m.obs_multivar_level; + data_type = m.data_type; + conf = m.conf; + centroid_dist_wt = m.centroid_dist_wt; + boundary_dist_wt = m.boundary_dist_wt; + convex_hull_dist_wt = m.convex_hull_dist_wt; + angle_diff_wt = m.angle_diff_wt; + aspect_diff_wt = m.aspect_diff_wt; + area_ratio_wt = m.area_ratio_wt; + int_area_ratio_wt = m.int_area_ratio_wt; + curvature_ratio_wt = m.curvature_ratio_wt; + complexity_ratio_wt = m.complexity_ratio_wt; + inten_perc_ratio_wt = m.inten_perc_ratio_wt; + + fcst_array = 0; + Fcst = 0; + if (N_fields_f > 0) { + fcst_array = new Mode_Field_Info[N_fields_f]; + for (int i=0; i 0) { + obs_array = new Mode_Field_Info[N_fields_o]; + for (int i=0; ilookup_int(conf_key_shift_right); - read_fields (fcst_array, fcst_dict, ftype, 'F'); // the order is important here - read_fields ( obs_array, obs_dict, otype, 'O'); // the order is important here - Fcst = fcst_array; // + 0 - Obs = obs_array; // + 0 + if (field_index == 0) { + read_fields_0 (fcst_array, fcst_dict, ftype, 'F'); + read_fields_1 (fcst_array, fcst_dict, ftype, 'F', 0); + read_fields_0 (obs_array, obs_dict, otype, 'O'); + read_fields_1 (obs_array, obs_dict, otype, 'O', 0); + fcst_array->raw_pi = parse_conf_plot_info(conf.lookup_dictionary(conf_key_fcst_raw_plot)); + obs_array->raw_pi = parse_conf_plot_info(conf.lookup_dictionary(conf_key_obs_raw_plot)); + + } else { + read_fields_1 (fcst_array, fcst_dict, ftype, 'F', field_index); + read_fields_1 (obs_array, obs_dict, otype, 'O', field_index); + } + + Fcst = fcst_array + field_index; + Obs = obs_array + field_index; // Dump the contents of the VarInfo objects if(mlog.verbosity_level() >= 5) { - for (j=0; jdump(cout); - } - for (j=0; jdump(cout); - } // for j - } - for (j=0; jdump(cout); + mlog << Debug(5) + << "Parsed observation field:\n"; + obs_array[field_index].var_info->dump(cout); } - for (j=0; j " - << "fcst and obs convolution radius arrays need to be the same size for traditional mode\n\n"; - exit ( 1 ); - } - if ( fcst_array[j].conv_thresh_array.n_elements() != obs_array[j].conv_thresh_array.n_elements() ) { + if ( fcst_array[field_index].conv_radius_array.n_elements() != + obs_array[field_index].conv_radius_array.n_elements() ) { + mlog << Error << "\nModeConfInfo::process_config_both() -> " + << "fcst and obs convolution radius arrays need to be the same size for traditional mode\n\n"; + exit ( 1 ); + } + if ( fcst_array[field_index].conv_thresh_array.n_elements() != + obs_array[field_index].conv_thresh_array.n_elements() ) { - mlog << Error << "\nModeConfInfo::process_config_both() -> " - << "fcst and obs convolution threshold arrays need to be the same size\n\n"; - exit ( 1 ); - } + mlog << Error << "\nModeConfInfo::process_config_both() -> " + << "fcst and obs convolution threshold arrays need to be the same size\n\n"; + exit ( 1 ); } } - // Conf: fcst_raw_plot - - fcst_array->raw_pi = parse_conf_plot_info(conf.lookup_dictionary(conf_key_fcst_raw_plot)); - - // Conf: obs_raw_plot - - obs_array->raw_pi = parse_conf_plot_info(conf.lookup_dictionary(conf_key_obs_raw_plot)); - - return; + return; } //////////////////////////////////////////////////////////////////////// -void ModeConfInfo::process_config_fcst(GrdFileType ftype) +void ModeConfInfo::process_config_fcst(GrdFileType ftype, int field_index) { int j, k, n; @@ -501,25 +613,26 @@ void ModeConfInfo::process_config_fcst(GrdFileType ftype) fcst_dict = conf.lookup_dictionary(conf_key_fcst); shift_right = fcst_dict->lookup_int(conf_key_shift_right); - read_fields (fcst_array, fcst_dict, ftype, 'F'); // the order is important here - Fcst = fcst_array; // + 0 + if (field_index == 0) { + read_fields_0 (fcst_array, fcst_dict, ftype, 'F'); + read_fields_1 (fcst_array, fcst_dict, ftype, 'F', 0); + fcst_array->raw_pi = parse_conf_plot_info(conf.lookup_dictionary(conf_key_fcst_raw_plot)); + } else { + read_fields_1 (fcst_array, fcst_dict, ftype, 'F', field_index); + } + + Fcst = fcst_array + field_index; - // Dump the contents of the VarInfo objects + // Dump the contents of the VarInfo object if(mlog.verbosity_level() >= 5) { - for (j=0; jdump(cout); - } // for j + mlog << Debug(5) + << "Parsed forecast field:\n"; + fcst_array[field_index].var_info->dump(cout); } - for (j=0; jraw_pi = parse_conf_plot_info(conf.lookup_dictionary(conf_key_fcst_raw_plot)); + evaluate_fcst_settings(field_index); return; @@ -528,7 +641,7 @@ void ModeConfInfo::process_config_fcst(GrdFileType ftype) //////////////////////////////////////////////////////////////////////// -void ModeConfInfo::process_config_obs(GrdFileType otype) +void ModeConfInfo::process_config_obs(GrdFileType otype, int field_index) { int j, k, n; @@ -537,25 +650,25 @@ void ModeConfInfo::process_config_obs(GrdFileType otype) obs_dict = conf.lookup_dictionary(conf_key_obs); shift_right = obs_dict->lookup_int(conf_key_shift_right); - read_fields ( obs_array, obs_dict, otype, 'O'); // the order is important here - Obs = obs_array; // + 0 + if (field_index == 0) { + read_fields_0 (obs_array, obs_dict, otype, 'O'); + read_fields_1 (obs_array, obs_dict, otype, 'O', 0); + obs_array->raw_pi = parse_conf_plot_info(conf.lookup_dictionary(conf_key_obs_raw_plot)); + } else { + read_fields_1 (obs_array, obs_dict, otype, 'O', field_index); + } + Obs = obs_array + field_index; - // Dump the contents of the VarInfo objects + // Dump the contents of the VarInfo object if(mlog.verbosity_level() >= 5) { - for (j=0; jdump(cout); - } // for j + mlog << Debug(5) + << "Parsed observation field:\n"; + obs_array[field_index].var_info->dump(cout); } - for (j=0; jraw_pi = parse_conf_plot_info(conf.lookup_dictionary(conf_key_obs_raw_plot)); + evaluate_obs_settings(field_index); return; @@ -563,6 +676,140 @@ void ModeConfInfo::process_config_obs(GrdFileType otype) //////////////////////////////////////////////////////////////////////// +void ModeConfInfo::config_set_all_percentile_thresholds(const std::vector &fdata, + const std::vector &odata) +{ + + // for each forecast input, it's either not a percentile threshold, + // is a simple (single input) percentile, is a frequency bias percentile threshold, or is a + // climatology percentile (we don't have climatology). + // + // As a complication we have both conv_thresh and merge_thresh, so what if they are different? + // right now I simply go with frequency bias as highest priority + + // indices of forecast and obs inputs that have frequency bias percentile thresholding + vector fcst_freq, obs_freq; + + // indices (common to forecast and obs) that require both inputs (fcst and obs) + // which is either frequency bias, or sample obs with a forecast input, or sample fcst + // with an obs input + vector indices_with_both; + + for (int j=0; j= N_fields_o) { + mlog << Error << "\nModeConfInfo::config_set_all_percentile_thresholds\n" + << " SOP Thresholding on fcst index " << j+1 + << " out of range of obs " << N_fields_o + 1 << "\n\n"; + exit ( 1 ); + } + // deal with this later + indices_with_both.push_back(j); + break; + case perc_thresh_freq_bias: + // currently expect matching index in obs and fcst, so check to make sure not both + if (j >= N_fields_o) { + mlog << Error << "\nModeConfInfo::config_set_all_percentile_thresholds\n" + << " FBIAS Thresholding on fcst index " << j+1 + << " out of range of obs " << N_fields_o + 1 << "\n\n"; + exit ( 1 ); + } + fcst_freq.push_back(j); + indices_with_both.push_back(j); + break; + default: + break; + } + } + for (int j=0; j= N_fields_f) { + mlog << Error << "\nModeConfInfo::config_set_all_percentile_thresholds\n" + << " SFP Thresholding on obs index " << j+1 + << " out of range of fcst " << N_fields_f + 1 << "\n\n"; + exit ( 1 ); + } + // deal with this later + if (find(indices_with_both.begin(), indices_with_both.end(), j) == indices_with_both.end()) { + indices_with_both.push_back(j); + } + break; + case perc_thresh_freq_bias: + // currently expect matching index in obs and fcst, so check to make sure not both + if (j >= N_fields_f) { + mlog << Error << "\nModeConfInfo::config_set_all_percentile_thresholds\n" + << " FBIAS Thresholding on obs index " << j+1 + << " out of range of fcst " << N_fields_f + 1 << "\n\n"; + exit ( 1 ); + } + // deal with this later + obs_freq.push_back(j); + if (find(indices_with_both.begin(), indices_with_both.end(), j) == indices_with_both.end()) { + indices_with_both.push_back(j); + } + break; + default: + break; + } + } + + // make sure no overlap in fcst/obs frequencies, as that's a no no + for (size_t i=0; iis_array() ) { exit ( 1 ); } + } - info_array[j].set(true, j, e->dict_value(), &conf, type, _fo, false); + return; - if ( j == 0 ) { +} // if is array - for (k=1; klookup(conf_key_field); + if ( !ee ) { + + mlog << "\nModeConfInfo::read_fields () -> \"field\" entry not found in dictionary!\n\n"; + + exit ( 1 ); + +} + +const Dictionary * field = ee->dict(); + +const int N = ( (field->is_array()) ? (field->n_entries()) : 1 ); + +if ( field->is_array() ) { + + const DictionaryEntry * e = 0; + const Dictionary & D = *field; + + e = D[field_index]; + if ( (e->type() != DictionaryType) && (e->type() != ArrayType) ) { + mlog << Error + << "\nModeConfInfo::read_fields() -> field entry # " << field_index+1 << " is not a dictionary!\n\n"; + exit ( 1 ); + + } + + info_array[field_index].set(true, field_index, e->dict_value(), &conf, type, _fo, false); return; } // if is array - // // nope, traditional mode // -N_fields_f = N; -N_fields_o = N; + if (field_index != 0) { + mlog << Error + << "\nModeConfInfo::read_fields() -> traditional mode but looking for a field array index " << field_index + << "\n\n"; + exit ( 1 ); + } + + info_array[0].set(false, 0, dict, &conf, type, _fo); // @@ -1287,6 +1582,46 @@ void ModeConfInfo::set_data_type(ModeDataType type) //////////////////////////////////////////////////////////////////////// +GrdFileType ModeConfInfo::file_type_for_field(bool isFcst, int field_index) +{ + // look at the dictionary for the obs or forecast at index, with + // parents + + Dictionary * dict = (Dictionary *) 0; + if (isFcst) { + dict = conf.lookup_dictionary(conf_key_fcst); + } else { + dict = conf.lookup_dictionary(conf_key_obs); + } + + const DictionaryEntry *e = dict->lookup(conf_key_field); + if ( !e ) { + mlog << "\nModeConfInfo::read_fields () -> \"field\" entry not found in dictionary!\n\n"; + exit ( 1 ); + } + + const Dictionary * field = e->dict(); + const int N = ( (field->is_array()) ? (field->n_entries()) : 1 ); + if (field_index < 0 || field_index >= N) { + mlog << Error + << "\nModeConfInfo::file_type_for_field() -> " + << "index " << field_index+1 << " out of range 0 to " << N + << "\n\n"; + exit (1); + } + e = (*field)[field_index]; + if ( (e->type() != DictionaryType) && (e->type() != ArrayType) ) { + mlog << Error + << "\nModeConfInfo::file_type_for_field() -> " + << "field entry # " << field_index+1 << " is not a dictionary!\n\n"; + exit ( 1 ); + } + Dictionary *dictf = e->dict_value(); + return parse_conf_file_type(dictf); +} + +//////////////////////////////////////////////////////////////////////// + int ModeConfInfo::n_runs() const @@ -1406,7 +1741,12 @@ void ModeConfInfo::check_multivar_not_implemented() << "\nModeConfInfo::check_multivar_not_implemented():\n" << " merge_flag ENGINE or BOTH not implemented for multivariate mode\n\n"; status = true; - break; + } + if (fcst_array[i].conv_thresh_array.n() > 1 || fcst_array[i].merge_thresh_array.n() > 1) { + mlog << Error + << "\nModeConfInfo::check_multivar_not_implemented():\n" + << " more than one conv_thresh or merge_thresh per input is not allowed in multivariate mode\n\n"; + status = true; } } } @@ -1419,6 +1759,12 @@ void ModeConfInfo::check_multivar_not_implemented() status = true; break; } + if (obs_array[i].conv_thresh_array.n() > 1 || obs_array[i].merge_thresh_array.n() > 1) { + mlog << Error + << "\nModeConfInfo::check_multivar_not_implemented():\n" + << " more than one conv_thresh or merge_thresh per input is not allowed in multivariate mode\n\n"; + status = true; + } } } @@ -1433,52 +1779,43 @@ void ModeConfInfo::check_multivar_not_implemented() //////////////////////////////////////////////////////////////////////// -void ModeConfInfo::check_multivar_perc_thresh(bool isSimple, bool isSimpleMerge) const +PercThreshType ModeConfInfo::perctype(const Mode_Field_Info &f) const { - if (isSimple) { - if (data_type == ModeDataType_MvMode_Fcst) { - for (int j=0; j< Fcst->conv_thresh_array.n(); ++j) { - if (Fcst->conv_thresh_array[j].get_ptype() == perc_thresh_sample_obs) { - mlog << Warning - << "\nModeConfInfo::check_multivar_perc_thresh:\n" - << " Thresholding with 'SOP' in a forecast input not well defined for multivariate mode simple object creation\n" - << " 'SFP' will be used as a replacement\n\n"; - } - } - } else if (data_type == ModeDataType_MvMode_Obs) { - for (int j=0; j< Obs->conv_thresh_array.n(); ++j) { - if (Obs->conv_thresh_array[j].get_ptype() == perc_thresh_sample_fcst) { - mlog << Warning - << "\nModeConfInfo::check_multivar_perc_thresh:\n" - << " Thresholding with 'SFP' in an obs input not well defined for multivariate mode simple object creation\n" - << " 'SOP' will be used as a replacement\n\n"; - } - } - } + PercThreshType pm=no_perc_thresh_type;; + PercThreshType pc=no_perc_thresh_type;; + if (f.merge_thresh_array.n() > 0) { + pm = f.merge_thresh_array[0].get_ptype(); } - - if (isSimpleMerge) { - if (data_type == ModeDataType_MvMode_Fcst) { - for (int j=0; j< Fcst->merge_thresh_array.n(); ++j) { - if (Fcst->merge_thresh_array[j].get_ptype() == perc_thresh_sample_obs) { - mlog << Warning - << "\nModeConfInfo::check_multivar_perc_thresh:\n" - << " Thresholding with 'SOP' in a forecast input not well defined for multivariate mode simple object creation\n" - << " 'SFP' will be used as a replacement\n\n"; - } - } - } else if (data_type == ModeDataType_MvMode_Obs) { - for (int j=0; j< Obs->merge_thresh_array.n(); ++j) { - if (Obs->merge_thresh_array[j].get_ptype() == perc_thresh_sample_fcst) { - mlog << Warning - << "\nModeConfInfo::check_multivar_perc_thresh():\n" - << " Thresholding with 'SFP' in an obs input not well defined for multivariate mode simple object creation\n" - << " 'SOP' will be used as a replacement\n\n"; - } - } - } + if (f.conv_thresh_array.n() > 0) { + pc = f.conv_thresh_array[0].get_ptype(); } -} + if (pm == perc_thresh_sample_climo || pc == perc_thresh_sample_climo) { + mlog << Error << "\nModeConfInfo::perctype()\n" + << " Thresholding with 'SCP' in an input not implemented for multivariate mode\n\n"; + exit ( 1 ); + } + if (pm == perc_thresh_climo_dist || pc == perc_thresh_climo_dist) { + mlog << Error << "\nModeConfInfo::perctype()\n" + << " Thresholding with 'CDP' in an input not implemented for multivariate mode\n\n"; + exit ( 1 ); + } + if (pm == perc_thresh_freq_bias || + pc == perc_thresh_freq_bias) { + return perc_thresh_freq_bias; + } + // put in tests if they are not the same and + // both are sample settings, so the right one + // gets returned (pass in fcst/obs boolean) + if (pm == perc_thresh_sample_obs || + pc == perc_thresh_sample_obs) { + return perc_thresh_sample_obs; + } + if (pm == perc_thresh_sample_fcst || + pc == perc_thresh_sample_fcst) { + return perc_thresh_sample_fcst; + } + return no_perc_thresh_type; +} //////////////////////////////////////////////////////////////////////// diff --git a/src/libcode/vx_shapedata/mode_conf_info.h b/src/libcode/vx_shapedata/mode_conf_info.h index 7fab368dc4..fcaca3e5a0 100644 --- a/src/libcode/vx_shapedata/mode_conf_info.h +++ b/src/libcode/vx_shapedata/mode_conf_info.h @@ -25,6 +25,7 @@ #include "mode_field_info.h" #include "mode_data_type.h" +#include "mode_input_data.h" //////////////////////////////////////////////////////////////////////// @@ -74,12 +75,17 @@ class ModeConfInfo { int N_fields_f; // for traditional mode expect N_fields_f/_o to be the same int N_fields_o; + void assign(const ModeConfInfo &); + public: ModeConfInfo(); ~ModeConfInfo(); + // attempt to implement this + ModeConfInfo & operator=(const ModeConfInfo &); + void clear(); // sets both obs and fcst field indices if traditional mode, or mvmode 'both' @@ -106,7 +112,15 @@ class ModeConfInfo { void check_multivar_not_implemented(); - void check_multivar_perc_thresh(bool isSimple, bool isSimpleMerge) const; + // check all inputs for climatology percentiles, which are not implemented + // warn about forecasts and obs set backwards + // void check_multivar_perc_thresh(const Mode_Field_Info &f, bool isFcst) const; + + + // // return index to other input if the particular input is a frequency bias percentile + // // return -1 if input is not a fequency bias percentile + // // if input is forecast, 'other' is obs, and vice versa + // int frequency_bias_other_index(bool input_is_fcst, int input_index) const; ///////////////////////////////////////////////////////////////////// @@ -122,7 +136,7 @@ class ModeConfInfo { // and to distinguish traditional mode from mvmode ModeDataType data_type; - + ///////////////////////////////////////////////////////////////////// @@ -135,9 +149,21 @@ class ModeConfInfo { void read_config (const char * default_filename, const char * user_filename); - void process_config (GrdFileType ftype, GrdFileType otype, ModeDataType dt=ModeDataType_Traditional); + void process_config_traditional(GrdFileType ftype, GrdFileType otype); - void read_fields (Mode_Field_Info * &, Dictionary * dict, GrdFileType, char _fo); + void process_config_except_fields(); + + void process_config_field (GrdFileType ftype, GrdFileType otype, ModeDataType dt, int field_index); + + void config_set_all_percentile_thresholds(const std::vector &fdata, + const std::vector &odata); + PercThreshType perctype(const Mode_Field_Info &f) const; + + // deal with zeroth field + void read_fields_0 (Mode_Field_Info * &, Dictionary * dict, GrdFileType, char _fo); + + // deal with non-zeroth field + void read_fields_1 (Mode_Field_Info * &, Dictionary * dict, GrdFileType, char _fo, int field_index); PiecewiseLinear * parse_interest_function(Dictionary * dict, const char * conf_key_if); @@ -269,12 +295,14 @@ class ModeConfInfo { void set_data_type(ModeDataType type); + GrdFileType file_type_for_field(bool isFcst, int field_index); + private: // some private methods - void process_config_both(GrdFileType ftype, GrdFileType otype); - void process_config_fcst(GrdFileType ftype); - void process_config_obs(GrdFileType otype); + void process_config_both(GrdFileType ftype, GrdFileType otype, int field_index=0); + void process_config_fcst(GrdFileType ftype, int field_index=0); + void process_config_obs(GrdFileType otype, int field_index=0); void evaluate_fcst_settings(int); void evaluate_obs_settings(int); diff --git a/src/libcode/vx_shapedata/mode_field_info.cc b/src/libcode/vx_shapedata/mode_field_info.cc index 8dd024e803..fcd652a09a 100644 --- a/src/libcode/vx_shapedata/mode_field_info.cc +++ b/src/libcode/vx_shapedata/mode_field_info.cc @@ -85,6 +85,32 @@ return ( * this ); } +//////////////////////////////////////////////////////////////////////// + +void Mode_Field_Info::clone(const Mode_Field_Info & i) +{ + dict = i.dict; + conf = i.conf; + gft = i.gft; + FO = i.FO; + Multivar = i.Multivar; + index = i.index; + conv_radius = i.conv_radius; + vld_thresh = i.vld_thresh; + + var_info = i.var_info->clone(); + + conv_radius_array = i.conv_radius_array; + conv_thresh_array = i.conv_thresh_array; + merge_thresh_array = i.merge_thresh_array; + conv_thresh = i.conv_thresh; + merge_thresh = i.merge_thresh; + merge_flag = i.merge_flag; + raw_pi = i.raw_pi; + filter_attr_map = i.filter_attr_map; + file_type = i.file_type; +} + //////////////////////////////////////////////////////////////////////// @@ -143,6 +169,8 @@ merge_thresh.clear(); merge_flag = MergeType_Engine; +file_type = FileType_None; + // raw_pi.clear(); } @@ -310,8 +338,8 @@ if ( FO == 'F' ) raw_pi = parse_conf_plot_info(conf->lookup_dictionary(conf_key else raw_pi = parse_conf_plot_info(conf->lookup_dictionary(conf_key_obs_raw_plot)); - - +file_type = parse_conf_file_type(dict); + // // done // diff --git a/src/libcode/vx_shapedata/mode_field_info.h b/src/libcode/vx_shapedata/mode_field_info.h index 132f2df4b2..bb7e08cb57 100644 --- a/src/libcode/vx_shapedata/mode_field_info.h +++ b/src/libcode/vx_shapedata/mode_field_info.h @@ -42,10 +42,9 @@ class Mode_Field_Info { void assign(const Mode_Field_Info &); + Dictionary * dict; // not allocated, based on reading the config file - Dictionary * dict; // not allocated - - MetConfig * conf; // not allocated + MetConfig * conf; // not allocated, based on reading the config file GrdFileType gft; @@ -59,6 +58,7 @@ class Mode_Field_Info { ~Mode_Field_Info(); Mode_Field_Info(const Mode_Field_Info &); Mode_Field_Info & operator=(const Mode_Field_Info &); + void clone(const Mode_Field_Info &); void clear(); @@ -85,6 +85,8 @@ class Mode_Field_Info { PlotInfo raw_pi; // Raw forecast plotting info + GrdFileType file_type; // each field can have a configured file type + // // member functions // diff --git a/src/libcode/vx_shapedata/mode_input_data.h b/src/libcode/vx_shapedata/mode_input_data.h new file mode 100644 index 0000000000..0aa9b4c36d --- /dev/null +++ b/src/libcode/vx_shapedata/mode_input_data.h @@ -0,0 +1,42 @@ +// ** Copyright UCAR (c) 1992 - 2023 +// ** University Corporation for Atmospheric Research (UCAR) +// ** National Center for Atmospheric Research (NCAR) +// ** Research Applications Lab (RAL) +// ** P.O.Box 3000, Boulder, Colorado, 80307-3000, USA +// *=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=* + + +//////////////////////////////////////////////////////////////////////// + + +#ifndef __MODE_INPUT_DATA_H__ +#define __MODE_INPUT_DATA_H__ + +#include +#include "data_plane.h" +#include "grid_base.h" + +//////////////////////////////////////////////////////////////////////// + + +class ModeInputData { + + private: + + public: + + ModeInputData(const std::string &name, const DataPlane &dp, const Grid &g) : + _name(name), _dataPlane(dp), _grid(g) {} + + ~ModeInputData() {} + + std::string _name; + DataPlane _dataPlane; + Grid _grid; +}; + + +#endif + + +///////////////////////////////////////////////////////////////////////// diff --git a/src/tools/core/mode/Makefile.am b/src/tools/core/mode/Makefile.am index d8dc507385..34bb6cc8dd 100644 --- a/src/tools/core/mode/Makefile.am +++ b/src/tools/core/mode/Makefile.am @@ -13,6 +13,7 @@ include ${top_srcdir}/Make-include bin_PROGRAMS = mode mode_SOURCES = mode_usage.cc \ mode_frontend.cc \ + mode_superobject.cc \ multivar_data.cc \ multivar_frontend.cc \ mode.cc \ diff --git a/src/tools/core/mode/Makefile.in b/src/tools/core/mode/Makefile.in index 741c5fea4f..10f09a3200 100644 --- a/src/tools/core/mode/Makefile.in +++ b/src/tools/core/mode/Makefile.in @@ -102,9 +102,9 @@ CONFIG_CLEAN_VPATH_FILES = am__installdirs = "$(DESTDIR)$(bindir)" PROGRAMS = $(bin_PROGRAMS) am_mode_OBJECTS = mode-mode_usage.$(OBJEXT) \ - mode-mode_frontend.$(OBJEXT) mode-multivar_data.$(OBJEXT) \ - mode-multivar_frontend.$(OBJEXT) mode-mode.$(OBJEXT) \ - mode-combine_boolplanes.$(OBJEXT) \ + mode-mode_frontend.$(OBJEXT) mode-mode_superobject.$(OBJEXT) \ + mode-multivar_data.$(OBJEXT) mode-multivar_frontend.$(OBJEXT) \ + mode-mode.$(OBJEXT) mode-combine_boolplanes.$(OBJEXT) \ mode-objects_from_netcdf.$(OBJEXT) mode-mode_ps_file.$(OBJEXT) \ mode-plot_engine.$(OBJEXT) mode-page_1.$(OBJEXT) \ mode-fcst_enlarge_page.$(OBJEXT) \ @@ -138,6 +138,7 @@ am__depfiles_remade = ./$(DEPDIR)/mode-cluster_page.Po \ ./$(DEPDIR)/mode-mode_exec.Po \ ./$(DEPDIR)/mode-mode_frontend.Po \ ./$(DEPDIR)/mode-mode_ps_file.Po \ + ./$(DEPDIR)/mode-mode_superobject.Po \ ./$(DEPDIR)/mode-mode_usage.Po \ ./$(DEPDIR)/mode-multivar_data.Po \ ./$(DEPDIR)/mode-multivar_frontend.Po \ @@ -350,6 +351,7 @@ top_srcdir = @top_srcdir@ MAINTAINERCLEANFILES = Makefile.in mode_SOURCES = mode_usage.cc \ mode_frontend.cc \ + mode_superobject.cc \ multivar_data.cc \ multivar_frontend.cc \ mode.cc \ @@ -503,6 +505,7 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mode-mode_exec.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mode-mode_frontend.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mode-mode_ps_file.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mode-mode_superobject.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mode-mode_usage.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mode-multivar_data.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mode-multivar_frontend.Po@am__quote@ # am--include-marker @@ -560,6 +563,20 @@ mode-mode_frontend.obj: mode_frontend.cc @AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mode_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o mode-mode_frontend.obj `if test -f 'mode_frontend.cc'; then $(CYGPATH_W) 'mode_frontend.cc'; else $(CYGPATH_W) '$(srcdir)/mode_frontend.cc'; fi` +mode-mode_superobject.o: mode_superobject.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mode_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT mode-mode_superobject.o -MD -MP -MF $(DEPDIR)/mode-mode_superobject.Tpo -c -o mode-mode_superobject.o `test -f 'mode_superobject.cc' || echo '$(srcdir)/'`mode_superobject.cc +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/mode-mode_superobject.Tpo $(DEPDIR)/mode-mode_superobject.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='mode_superobject.cc' object='mode-mode_superobject.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mode_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o mode-mode_superobject.o `test -f 'mode_superobject.cc' || echo '$(srcdir)/'`mode_superobject.cc + +mode-mode_superobject.obj: mode_superobject.cc +@am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mode_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT mode-mode_superobject.obj -MD -MP -MF $(DEPDIR)/mode-mode_superobject.Tpo -c -o mode-mode_superobject.obj `if test -f 'mode_superobject.cc'; then $(CYGPATH_W) 'mode_superobject.cc'; else $(CYGPATH_W) '$(srcdir)/mode_superobject.cc'; fi` +@am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/mode-mode_superobject.Tpo $(DEPDIR)/mode-mode_superobject.Po +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ $(AM_V_CXX)source='mode_superobject.cc' object='mode-mode_superobject.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCXX_FALSE@ DEPDIR=$(DEPDIR) $(CXXDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCXX_FALSE@ $(AM_V_CXX@am__nodep@)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mode_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -c -o mode-mode_superobject.obj `if test -f 'mode_superobject.cc'; then $(CYGPATH_W) 'mode_superobject.cc'; else $(CYGPATH_W) '$(srcdir)/mode_superobject.cc'; fi` + mode-multivar_data.o: multivar_data.cc @am__fastdepCXX_TRUE@ $(AM_V_CXX)$(CXX) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mode_CPPFLAGS) $(CPPFLAGS) $(AM_CXXFLAGS) $(CXXFLAGS) -MT mode-multivar_data.o -MD -MP -MF $(DEPDIR)/mode-multivar_data.Tpo -c -o mode-multivar_data.o `test -f 'multivar_data.cc' || echo '$(srcdir)/'`multivar_data.cc @am__fastdepCXX_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/mode-multivar_data.Tpo $(DEPDIR)/mode-multivar_data.Po @@ -876,6 +893,7 @@ distclean: distclean-am -rm -f ./$(DEPDIR)/mode-mode_exec.Po -rm -f ./$(DEPDIR)/mode-mode_frontend.Po -rm -f ./$(DEPDIR)/mode-mode_ps_file.Po + -rm -f ./$(DEPDIR)/mode-mode_superobject.Po -rm -f ./$(DEPDIR)/mode-mode_usage.Po -rm -f ./$(DEPDIR)/mode-multivar_data.Po -rm -f ./$(DEPDIR)/mode-multivar_frontend.Po @@ -936,6 +954,7 @@ maintainer-clean: maintainer-clean-am -rm -f ./$(DEPDIR)/mode-mode_exec.Po -rm -f ./$(DEPDIR)/mode-mode_frontend.Po -rm -f ./$(DEPDIR)/mode-mode_ps_file.Po + -rm -f ./$(DEPDIR)/mode-mode_superobject.Po -rm -f ./$(DEPDIR)/mode-mode_usage.Po -rm -f ./$(DEPDIR)/mode-multivar_data.Po -rm -f ./$(DEPDIR)/mode-multivar_frontend.Po diff --git a/src/tools/core/mode/combine_boolplanes.h b/src/tools/core/mode/combine_boolplanes.h index e879827740..084a31aed1 100644 --- a/src/tools/core/mode/combine_boolplanes.h +++ b/src/tools/core/mode/combine_boolplanes.h @@ -20,7 +20,8 @@ #include "two_d_array.h" #include "bool_calc.h" #include "vx_pxm.h" - +#include +#include //////////////////////////////////////////////////////////////////////// diff --git a/src/tools/core/mode/mode.cc b/src/tools/core/mode/mode.cc index b7277b0110..d71a7698b5 100644 --- a/src/tools/core/mode/mode.cc +++ b/src/tools/core/mode/mode.cc @@ -57,6 +57,7 @@ // 020 07/06/22 Howard Soh METplus-Internal #19 Rename main to met_main // 021 06/09/23 Albo Major changes for multivariate mode // 022 11/02/23 Halley Gotway MET #2724 add OpenMP to convolution +// 023 12/27/23 Albo MET #2745 more unit tests, read data one time, percentile thresholding // //////////////////////////////////////////////////////////////////////// @@ -78,6 +79,7 @@ using namespace std; #include "string_array.h" #include "mode_usage.h" #include "mode_frontend.h" +#include "multivar_frontend.h" #include "mode_conf_info.h" #ifdef WITH_PYTHON @@ -97,7 +99,6 @@ using namespace std; /////////////////////////////////////////////////////////////////////// -extern int mode_frontend(const StringArray &); extern int multivar_frontend(const StringArray &); extern const char * const program_name; @@ -186,8 +187,10 @@ int met_main(int argc, char * argv []) config.check_multivar_not_implemented(); // run the multivar version of mode - - status = multivar_frontend(Argv); + + MultivarFrontEnd *frontend = new MultivarFrontEnd(); + status = frontend->run(Argv); + if ( frontend ) { delete frontend; frontend = 0; } } else { diff --git a/src/tools/core/mode/mode_exec.cc b/src/tools/core/mode/mode_exec.cc index f381ff549b..d0c2e01b7a 100644 --- a/src/tools/core/mode/mode_exec.cc +++ b/src/tools/core/mode/mode_exec.cc @@ -35,7 +35,6 @@ static const char * cts_str[n_cts] = {"RAW", "OBJECT"}; static const char program_name [] = "mode"; static const char * default_config_filename = "MET_BASE/config/MODEConfig_default"; -static const char * default_multivar_config_filename = "MET_BASE/config/MODEMultivarConfig_default"; // took this out of the do_conv_thresh() method static int local_r_index = -1; @@ -60,12 +59,11 @@ static void replaceAll(std::string& str, const std::string& from, const std::str /////////////////////////////////////////////////////////////////////// -ModeExecutive::ModeExecutive(Processing_t p) +ModeExecutive::ModeExecutive()//Processing_t p) { init_from_scratch(); - ptype = p; } @@ -132,8 +130,6 @@ void ModeExecutive::clear() R_index = T_index = 0; - ptype = TRADITIONAL; - // // done // @@ -154,7 +150,7 @@ void ModeExecutive::init_traditional(int n_files) R_index = T_index = 0; - conf_read(default_config_filename); + conf_read(); // Get the forecast and observation file types from config, if present ftype = parse_conf_file_type(engine.conf_info.conf.lookup_dictionary(conf_key_fcst)); @@ -180,7 +176,7 @@ void ModeExecutive::init_traditional(int n_files) otype = obs_mtddf->file_type(); // Process the configuration - engine.conf_info.process_config(ftype, otype); + engine.conf_info.process_config_traditional(ftype, otype); int nf = engine.conf_info.n_fields_f(); // same as obs for traditional mode if (nf != n_files) { mlog << Error << "\nNumber of input files " << n_files << " Not equal to config number of fields " @@ -206,161 +202,19 @@ void ModeExecutive::init_traditional(int n_files) /////////////////////////////////////////////////////////////////////// - -void ModeExecutive::init_multivar_verif_grid() +void ModeExecutive::init_multivar_simple(int j, int n_files, ModeDataType dtype, + const ModeConfInfo &conf) { - - Met2dDataFileFactory mtddf_factory; - R_index = T_index = 0; - - conf_read(default_multivar_config_filename); - - // Get the forecast and observation file types from config, if present - ftype = parse_conf_file_type(engine.conf_info.conf.lookup_dictionary(conf_key_fcst)); - otype = parse_conf_file_type(engine.conf_info.conf.lookup_dictionary(conf_key_obs)); - - - // Read observation file - if(!(obs_mtddf = mtddf_factory.new_met_2d_data_file(obs_file.c_str(), otype))) { - mlog << Error << "\nTrouble reading observation file \"" - << obs_file << "\"\n\n"; - exit(1); - } - - // Read forecast file - if(!(fcst_mtddf = mtddf_factory.new_met_2d_data_file(fcst_file.c_str(), ftype))) { - mlog << Error << "\nTrouble reading forecast file \"" - << fcst_file << "\"\n\n"; - exit(1); - } - - // Store the input data file types - ftype = fcst_mtddf->file_type(); - otype = obs_mtddf->file_type(); - - // Process the configuration - engine.conf_info.process_config(ftype, otype, ModeDataType_MvMode_Both); - - // check one again for multivar problems - engine.conf_info.check_multivar_not_implemented(); - - const int shift = engine.conf_info.shift_right; - - fcst_mtddf->set_shift_right(shift); - obs_mtddf->set_shift_right(shift); - - // List the input files - mlog << Debug(1) - << "Forecast File: " << fcst_file << "\n" - << "Observation File: " << obs_file << "\n"; - - engine.conf_info.nc_info.compress_level = engine.conf_info.get_compression_level(); - - return; - -} - -/////////////////////////////////////////////////////////////////////// - -void ModeExecutive::init_multivar_simple(int n_files, ModeDataType dtype) - -{ - - Met2dDataFileFactory mtddf_factory; - - R_index = T_index = 0; - - conf_read(default_multivar_config_filename); + engine.conf_info = conf; // tell the engine which type of data it is engine.set_data_type(dtype); - // Get the forecast and observation file types from config, if present - ftype = parse_conf_file_type(engine.conf_info.conf.lookup_dictionary(conf_key_fcst)); - otype = parse_conf_file_type(engine.conf_info.conf.lookup_dictionary(conf_key_obs)); - - - // Read observation file or forecast file - if (dtype == ModeDataType_MvMode_Obs) - { - if(!(obs_mtddf = mtddf_factory.new_met_2d_data_file(obs_file.c_str(), otype))) { - mlog << Error << "\nTrouble reading observation file \"" - << obs_file << "\"\n\n"; - exit(1); - - } - // Store the input data file types - otype = obs_mtddf->file_type(); - } - else if (dtype == ModeDataType_MvMode_Fcst) - { - // Read forecast file - if(!(fcst_mtddf = mtddf_factory.new_met_2d_data_file(fcst_file.c_str(), ftype))) { - mlog << Error << "\nTrouble reading forecast file \"" - << fcst_file << "\"\n\n"; - exit(1); - } - // Store the input data file types - ftype = fcst_mtddf->file_type(); - } - else { - mlog << Error << "\nModeExecutive::init_multivar_simple()->" - << "simple object creation requires obs or forecast mode, got " - << sprintModeDataType(dtype) << "\n\n"; - exit(1); - } + engine.conf_info.set_field_index(j); - // Process the configuration - engine.conf_info.process_config(ftype, otype, dtype); - int nf; - if (dtype == ModeDataType_MvMode_Obs) { - nf = engine.conf_info.n_fields_o(); - } else if (dtype == ModeDataType_MvMode_Fcst) { - nf = engine.conf_info.n_fields_f(); - } else { - mlog << Error << "\nModeExecutive::init_multivar_simple()->" - << "simple object creation requires obs or forecast mode, got " - << sprintModeDataType(dtype) << "\n\n"; - exit(1); - } - if (nf != n_files) { - mlog << Error << "\nModeExecutive::init_multivar_simple()->" - << "Number of input files " << n_files << " Not equal to config number of fields " - << nf << "\n\n"; - exit(1); - } - - // check one again for multivar problems - engine.conf_info.check_multivar_not_implemented(); - - const int shift = engine.conf_info.shift_right; - - if (dtype != ModeDataType_MvMode_Fcst) { - obs_mtddf->set_shift_right(shift); - } - - if (dtype != ModeDataType_MvMode_Obs) { - fcst_mtddf->set_shift_right(shift); - } - - if (dtype == ModeDataType_MvMode_Obs) { - mlog << Debug(1) - << "Observation File: " << obs_file << "\n"; - - } else if (dtype == ModeDataType_MvMode_Fcst) { - mlog << Debug(1) - << "Forecast File: " << fcst_file << "\n"; - } - else { - // List the input files - mlog << Debug(1) - << "Forecast File: " << fcst_file << "\n" - << "Observation File: " << obs_file << "\n"; - } - - engine.conf_info.nc_info.compress_level = engine.conf_info.get_compression_level(); + // engine.conf_info.nc_info.compress_level = engine.conf_info.get_compression_level(); return; @@ -368,34 +222,21 @@ void ModeExecutive::init_multivar_simple(int n_files, ModeDataType dtype) /////////////////////////////////////////////////////////////////////// -void ModeExecutive::init_multivar_intensities(GrdFileType ftype, GrdFileType otype) +void ModeExecutive::init_multivar_intensities(const ModeConfInfo &conf) { R_index = T_index = 0; - conf_read(default_multivar_config_filename); - - // Get the forecast and observation file types from config, if present - GrdFileType l_ftype = parse_conf_file_type(engine.conf_info.conf.lookup_dictionary(conf_key_fcst)); - GrdFileType l_otype = parse_conf_file_type(engine.conf_info.conf.lookup_dictionary(conf_key_obs)); - - // actually use the values passed in instead, the config ones are not set for this second pass - l_ftype = ftype; - l_otype = otype; - - // Process the configuration - engine.conf_info.process_config(l_ftype, l_otype, ModeDataType_MvMode_Both); + engine.conf_info = conf; + + // tell the engine which type of data it is + engine.set_data_type(ModeDataType_MvMode_Both); // check one again for multivar problems engine.conf_info.check_multivar_not_implemented(); - // NOTE: do not do shifting, should have been done in the first pass - // const int shift = engine.conf_info.shift_right; - // fcst_mtddf->set_shift_right(shift); - // obs_mtddf->set_shift_right(shift); - - engine.conf_info.nc_info.compress_level = engine.conf_info.get_compression_level(); + // engine.conf_info.nc_info.compress_level = engine.conf_info.get_compression_level(); return; @@ -404,16 +245,8 @@ void ModeExecutive::init_multivar_intensities(GrdFileType ftype, GrdFileType oty /////////////////////////////////////////////////////////////////////// -void ModeExecutive::check_multivar_perc_thresh_settings() -{ - engine.conf_info.check_multivar_perc_thresh(ptype == ModeExecutive::MULTIVAR_SIMPLE, - ptype == ModeExecutive::MULTIVAR_SIMPLE_MERGE); -} -/////////////////////////////////////////////////////////////////////// - - -void ModeExecutive::setup_fcst_obs_data_traditional() +void ModeExecutive::setup_traditional_fcst_obs_data() { @@ -427,7 +260,7 @@ void ModeExecutive::setup_fcst_obs_data_traditional() if ( !(fcst_mtddf->data_plane(*(engine.conf_info.Fcst->var_info), Fcst_sd.data)) ) { - mlog << Error << "\nModeExecutive::setup_fcst_obs_data_traditional() -> " + mlog << Error << "\nModeExecutive::setup_traditionalfcst_obs_data() -> " << "can't get forecast data \"" << engine.conf_info.Fcst->var_info->magic_str() << "\" from file \"" << fcst_file << "\"\n\n"; @@ -438,7 +271,7 @@ void ModeExecutive::setup_fcst_obs_data_traditional() if ( !(obs_mtddf->data_plane(*(engine.conf_info.Obs->var_info), Obs_sd.data)) ) { - mlog << Error << "\nModeExecutive::setup_fcst_obs_data_traditional() -> " + mlog << Error << "\nModeExecutive::setup_traditional_fcst_obs_data() -> " << "can't get observation data \"" << engine.conf_info.Obs->var_info->magic_str() << "\" from file \"" << obs_file << "\"\n\n"; @@ -577,68 +410,39 @@ void ModeExecutive::setup_fcst_obs_data_traditional() /////////////////////////////////////////////////////////////////////// - -void ModeExecutive::setup_verification_grid() - +void ModeExecutive::setup_verification_grid(const ModeInputData &fcst, + const ModeInputData &obs, + const ModeConfInfo &conf) { - - // ShapeData fcst_sd, obs_sd; - double fmin, omin, fmax, omax; + R_index = T_index = 0; + engine.conf_info = conf; + engine.conf_info.check_multivar_not_implemented(); Fcst_sd.clear(); Obs_sd.clear(); - - // Read the gridded data from the input forecast file - - if ( !(fcst_mtddf->data_plane(*(engine.conf_info.Fcst->var_info), Fcst_sd.data)) ) { - - mlog << Error << "\nModeExecutive::setup_verification_grid() -> " - << "can't get forecast data \"" - << engine.conf_info.Fcst->var_info->magic_str() - << "\" from file \"" << fcst_file << "\"\n\n"; - exit(1); - } - - // Read the gridded data from the input observation file - - if ( !(obs_mtddf->data_plane(*(engine.conf_info.Obs->var_info), Obs_sd.data)) ) { - - mlog << Error << "\nModeExecutive::setup_verification_grid() -> " - << "can't get observation data \"" - << engine.conf_info.Obs->var_info->magic_str() - << "\" from file \"" << obs_file << "\"\n\n"; - exit(1); - } - - // Determine the verification grid - + Fcst_sd.data = fcst._dataPlane; + Obs_sd.data = obs._dataPlane; + + // set this local conf to point to forecast 0, so that that regrid info + // can be accessed and it can be decided if we are to use fcst or obs + // input + engine.conf_info.set_data_type(ModeDataType_MvMode_Fcst); + engine.conf_info.set_field_index(0); grid = parse_vx_grid(engine.conf_info.Fcst->var_info->regrid(), - &(fcst_mtddf->grid()), &(obs_mtddf->grid())); - + &fcst._grid, &obs._grid); return; } /////////////////////////////////////////////////////////////////////// - -void ModeExecutive::setup_fcst_data(const Grid &verification_grid) - +void ModeExecutive::setup_multivar_fcst_data(const Grid &verification_grid, + const ModeInputData &input) { - // ShapeData fcst_sd, obs_sd; double fmin, fmax; Fcst_sd.clear(); - - // Read the gridded data from the input forecast file - if ( !(fcst_mtddf->data_plane(*(engine.conf_info.Fcst->var_info), Fcst_sd.data)) ) { - - mlog << Error << "\nModeExecutive::setup_fcst_data() -> " - << "can't get forecast data \"" - << engine.conf_info.Fcst->var_info->magic_str() - << "\" from file \"" << fcst_file << "\"\n\n"; - exit(1); - } + Fcst_sd.data = input._dataPlane; grid = verification_grid; @@ -647,12 +451,11 @@ void ModeExecutive::setup_fcst_data(const Grid &verification_grid) engine.set_grid(&grid); // Regrid, if necessary - - if ( !(fcst_mtddf->grid() == grid) ) { + if ( !(input._grid == grid) ) { mlog << Debug(1) << "Regridding forecast " << engine.conf_info.Fcst->var_info->magic_str() << " to the verification grid.\n"; - Fcst_sd.data = met_regrid(Fcst_sd.data, fcst_mtddf->grid(), grid, + Fcst_sd.data = met_regrid(Fcst_sd.data, input._grid, grid, engine.conf_info.Fcst->var_info->regrid()); } @@ -676,10 +479,6 @@ void ModeExecutive::setup_fcst_data(const Grid &verification_grid) data_min = fmin; data_max = fmax; - // Process percentile thresholds - - engine.conf_info.set_perc_thresh(Fcst_sd.data); - // store the input data units funits = engine.conf_info.Fcst->var_info->units(); ounits = na_str; @@ -698,9 +497,8 @@ void ModeExecutive::setup_fcst_data(const Grid &verification_grid) /////////////////////////////////////////////////////////////////////// - -void ModeExecutive::setup_obs_data(const Grid &verification_grid) - +void ModeExecutive::setup_multivar_obs_data(const Grid &verification_grid, + const ModeInputData &input) { // ShapeData fcst_sd, obs_sd; @@ -709,15 +507,7 @@ void ModeExecutive::setup_obs_data(const Grid &verification_grid) Obs_sd.clear(); // Read the gridded data from the input observation file - - if ( !(obs_mtddf->data_plane(*(engine.conf_info.Obs->var_info), Obs_sd.data)) ) { - - mlog << Error << "\nModeExecutive::setup_obs_data() -> " - << "can't get observation data \"" - << engine.conf_info.Obs->var_info->magic_str() - << "\" from file \"" << obs_file << "\"\n\n"; - exit(1); - } + Obs_sd.data = input._dataPlane; grid = verification_grid; @@ -727,11 +517,11 @@ void ModeExecutive::setup_obs_data(const Grid &verification_grid) // Regrid, if necessary - if ( !(obs_mtddf->grid() == grid) ) { + if ( !(input._grid == grid) ) { mlog << Debug(1) << "Regridding observation " << engine.conf_info.Obs->var_info->magic_str() << " to the verification grid.\n"; - Obs_sd.data = met_regrid(Obs_sd.data, obs_mtddf->grid(), grid, + Obs_sd.data = met_regrid(Obs_sd.data, input._grid, grid, engine.conf_info.Obs->var_info->regrid()); } @@ -755,10 +545,6 @@ void ModeExecutive::setup_obs_data(const Grid &verification_grid) data_min = omin; data_max = omax; - // Process percentile thresholds - - engine.conf_info.set_perc_thresh(Obs_sd.data); - // store the input data units funits = na_str; ounits = engine.conf_info.Obs->var_info->units(); @@ -777,7 +563,7 @@ void ModeExecutive::setup_obs_data(const Grid &verification_grid) /////////////////////////////////////////////////////////////////////// -void ModeExecutive::setup_fcst_obs_data_multivar_intensities(const MultiVarData &mvdf, +void ModeExecutive::setup_multivar_fcst_obs_data_intensities(const MultiVarData &mvdf, const MultiVarData &mvdo) { double fmin, fmax, omin, omax; @@ -789,17 +575,11 @@ void ModeExecutive::setup_fcst_obs_data_multivar_intensities(const MultiVarData Obs_sd.debug_examine(); grid = *(mvdf._grid); - // do not need to read any data, it is stored in the mvd input - // the verification grid was created in the first pass, so we have that as well - // Store the grid engine.set_grid(&grid); - // regridding of inputs is not needed in the second pass, as regridded data outputs - // are stored to input mvd - - // Rescale probabilites from [0, 100] to [0, 1] also not neede in the second pass + // Rescale probabilites from [0, 100] to [0, 1] not needed in the second pass // Check that the valid times match @@ -855,15 +635,12 @@ void ModeExecutive::setup_fcst_obs_data_multivar_intensities(const MultiVarData // in case perc thresh was done, retrieve the values created in the // first pass, which have been stored in mvdo and mvdf + // NOTE: this might be removable with new design engine.conf_info.Obs->conv_thresh_array = mvdo._merge->_convThreshArray; engine.conf_info.Obs->merge_thresh_array = mvdo._merge->_mergeThreshArray; engine.conf_info.Fcst->conv_thresh_array = mvdf._merge->_convThreshArray; engine.conf_info.Fcst->merge_thresh_array = mvdf._merge->_mergeThreshArray; - // This doesn't work with perc_thresh, it uses values restricted to within - // superobjects, which gives different thresholds, so we do the above instead. - //engine.conf_info.set_perc_thresh(Fcst_sd.data, Obs_sd.data); - // masking of inputs not needed, as it was done in the first pass // and stored to Fcst_sd and Obs_sd @@ -887,7 +664,8 @@ void ModeExecutive::setup_fcst_obs_data_multivar_intensities(const MultiVarData /////////////////////////////////////////////////////////////////////// -void ModeExecutive::setup_fcst_obs_data_multivar_super(ShapeData &f_super, ShapeData &o_super, +void ModeExecutive::setup_multivar_fcst_obs_data_super(const ShapeData &f_super, + const ShapeData &o_super, const Grid &igrid) { double fmin, omin, fmax, omax; @@ -972,10 +750,6 @@ void ModeExecutive::setup_fcst_obs_data_multivar_super(ShapeData &f_super, Shape else if(!is_bad_data(fmax) && is_bad_data(omax)) data_max = fmax; else if( is_bad_data(fmax) && !is_bad_data(omax)) data_max = omax; - // Process percentile thresholds...Not applicable for this pass - // do this in the second pass using masked Fcst_sd and Obs_sd - // engine.conf_info.set_perc_thresh(Fcst_sd.data, Obs_sd.data); - // // done // @@ -988,7 +762,7 @@ void ModeExecutive::setup_fcst_obs_data_multivar_super(ShapeData &f_super, Shape /////////////////////////////////////////////////////////////////////// -void ModeExecutive::do_conv_thresh(const int r_index, const int t_index) +void ModeExecutive::do_conv_thresh_traditional(const int r_index, const int t_index) { ModeConfInfo & conf = engine.conf_info; @@ -1000,65 +774,162 @@ void ModeExecutive::do_conv_thresh(const int r_index, const int t_index) R_index = r_index; T_index = t_index; - if (ptype == MULTIVAR_SIMPLE_MERGE) { - conf.set_conv_thresh_by_merge_index(T_index); - } else if (ptype == MULTIVAR_SUPER) { - SingleThresh s("ne-9999"); - conf.set_conv_thresh(s); - conf.set_conv_radius(0.0); + conf.set_conv_radius_by_index(R_index); + conf.set_conv_thresh_by_index(T_index); + conf.set_merge_thresh_by_index(T_index); + + // + // Set up the engine with these raw fields + // + + string what = "forecast and observation fields"; + mlog << Debug(2) << "Identifying objects in the " << what << "...\n"; + + if ( r_index != local_r_index ) { + // need to do convolution + engine.set(Fcst_sd, Obs_sd); } else { - // this could break down if obs and fcst have different arrays - // currently not allowed in multivar mode, should work correctly - // for traditional mode - conf.set_conv_radius_by_index(R_index); - conf.set_conv_thresh_by_index(T_index); + // don't need to do convolution + engine.set_no_conv(Fcst_sd, Obs_sd); } + // + // Compute the contingency table statistics for the fields, if needed + // + if ( conf.ct_stats_flag ) compute_ct_stats(); + + // + // done + // + + local_r_index = r_index; + + return; + +} + +/////////////////////////////////////////////////////////////////////// + +void ModeExecutive::do_conv_thresh_multivar_super() +{ + + ModeConfInfo & conf = engine.conf_info; + + R_index = 0; + T_index = 0; + + SingleThresh s("ne-9999"); + conf.set_conv_thresh(s); + conf.set_conv_radius(0.0); conf.set_merge_thresh_by_index(T_index); // // Set up the engine with these raw fields // - string what = "forecast and observation fields"; - if (ptype == MULTIVAR_SIMPLE || ptype == MULTIVAR_SIMPLE_MERGE) { - if (conf.data_type == ModeDataType_MvMode_Obs) { - what = "observation field"; - } else { - what = "forecast field"; - } - } mlog << Debug(2) << "Identifying objects in the " << what << "...\n"; - if ( r_index != local_r_index ) { // need to do convolution + engine.set_only_split(Fcst_sd, Obs_sd); - if (ptype == MULTIVAR_INTENSITY || ptype == MULTIVAR_SUPER) { - engine.set_only_split(Fcst_sd, Obs_sd); - } else {// MULTIVAR_SIMPLE, MULTIVAR_SIMPLE_MERGE, TRADITIONAL - engine.set(Fcst_sd, Obs_sd); - } - } else { // don't need to do convolution + // + // Compute the contingency table statistics for the fields, if needed + // (not needed for simple or merge, only one field) + // + if ( conf.ct_stats_flag ) compute_ct_stats(); - if (ptype == MULTIVAR_INTENSITY || ptype == MULTIVAR_SUPER) { - engine.set_only_split(Fcst_sd, Obs_sd); - } else {// MULTIVAR_SIMPLE, MULTIVAR_SIMPLE_MERGE, TRADITIONAL - engine.set_no_conv(Fcst_sd, Obs_sd); - } - } + // + // done + // + + local_r_index = 0; + + return; + +} + + +/////////////////////////////////////////////////////////////////////// + + +void ModeExecutive::do_conv_thresh_multivar_intensity_compare() +{ + + ModeConfInfo & conf = engine.conf_info; + + R_index = 0; + T_index = 0; + + conf.set_conv_radius_by_index(R_index); + conf.set_conv_thresh_by_index(T_index); + conf.set_merge_thresh_by_index(T_index); + + // + // Set up the engine with these raw fields + // + + string what = "forecast and observation fields"; + mlog << Debug(2) << "Identifying objects in the " << what << "...\n"; + + engine.set_only_split(Fcst_sd, Obs_sd); // // Compute the contingency table statistics for the fields, if needed - // (not needed for simple or merge, only one field) // - if (ptype != MULTIVAR_SIMPLE && ptype != MULTIVAR_SIMPLE_MERGE) { - if ( conf.ct_stats_flag ) compute_ct_stats(); - } + if ( conf.ct_stats_flag ) compute_ct_stats(); // // done // - local_r_index = r_index; + local_r_index = 0; + + return; + +} + +/////////////////////////////////////////////////////////////////////// + +void ModeExecutive::do_conv_thresh_multivar_simple(Processing_t p) +{ + + ModeConfInfo & conf = engine.conf_info; + + R_index = 0; + T_index = 0; + + if (p == MULTIVAR_SIMPLE_MERGE) { + conf.set_conv_thresh_by_merge_index(T_index); + } else if (p == MULTIVAR_SIMPLE) { + conf.set_conv_radius_by_index(0); + conf.set_conv_thresh_by_index(0); + } else { + mlog << Error << "\nModeExecutive::do_conv_thresh_multivar_simple() -> " + << "Wrong processing type input " << stype(p) << "\"\n\n"; + exit(1); + } + + conf.set_merge_thresh_by_index(0); + + // + // Set up the engine with these raw fields + // + + string what; + if (conf.data_type == ModeDataType_MvMode_Obs) { + what = "observation field"; + } else { + what = "forecast field"; + } + mlog << Debug(2) << "Identifying objects in the " << what << "...\n"; + + engine.set(Fcst_sd, Obs_sd); + + // (ct_stats not needed for simple or merge, only one field) + // + // done + // + + local_r_index = 0; return; @@ -1073,7 +944,7 @@ void ModeExecutive::clear_internal_r_index() /////////////////////////////////////////////////////////////////////// -void ModeExecutive::do_merging() +void ModeExecutive::do_merging_traditional() { mlog << Debug(2) << "Identified: " << engine.n_fcst << " forecast objects " @@ -1105,10 +976,11 @@ void ModeExecutive::do_merging() /////////////////////////////////////////////////////////////////////// -void ModeExecutive::do_merging(ShapeData &f_merge, ShapeData &o_merge) - +void ModeExecutive::do_merging_multivar(const ShapeData &f_merge, + const ShapeData &o_merge, + Processing_t p) { - if (ptype == MULTIVAR_SUPER) { + if (p == MULTIVAR_SUPER) { // set the merge flag and merge_thresh appropriately ModeConfInfo & conf = engine.conf_info; SingleThresh s("ne-9999"); @@ -1116,14 +988,11 @@ void ModeExecutive::do_merging(ShapeData &f_merge, ShapeData &o_merge) conf.set_fcst_merge_thresh(s); conf.set_obs_merge_flag(MergeType_Thresh); conf.set_obs_merge_thresh(s); - } else { - if (ptype != MULTIVAR_INTENSITY) { - mlog << Error << "\nModeExecutive::do_merging(shapedata, shapedata) -> " - << "wrong method for processing type " << stype(ptype) << "\n\n"; - exit(1); - } + } else if (p != MULTIVAR_INTENSITY) { + mlog << Error << "\nModeExecutive::do_merging(shapedata, shapedata, p) -> " + << "wrong method for processing type " << stype(p) << "\n\n"; + exit(1); } - mlog << Debug(2) << "Identified: " << engine.n_fcst << " forecast objects " @@ -1136,8 +1005,7 @@ void ModeExecutive::do_merging(ShapeData &f_merge, ShapeData &o_merge) // Do the forecast merging - engine.do_fcst_merging(default_config_file.c_str(), merge_config_file.c_str(), - f_merge); + engine.do_fcst_merging(f_merge); mlog << Debug(2) << "Performing merging (" @@ -1146,8 +1014,7 @@ void ModeExecutive::do_merging(ShapeData &f_merge, ShapeData &o_merge) // Do the observation merging - engine.do_obs_merging(default_config_file.c_str(), merge_config_file.c_str(), - o_merge); + engine.do_obs_merging(o_merge); mlog << Debug(2) << "Remaining: " << engine.n_fcst << " forecast objects " @@ -1158,10 +1025,12 @@ void ModeExecutive::do_merging(ShapeData &f_merge, ShapeData &o_merge) /////////////////////////////////////////////////////////////////////// -void ModeExecutive::do_match_merge(ShapeData &f_merge, ShapeData &o_merge) +void ModeExecutive::do_match_merge_multivar(const ShapeData &f_merge, + const ShapeData &o_merge, + Processing_t p) { - do_merging(f_merge, o_merge); + do_merging_multivar(f_merge, o_merge, p); // Do the matching of objects between fields @@ -1174,10 +1043,10 @@ void ModeExecutive::do_match_merge(ShapeData &f_merge, ShapeData &o_merge) /////////////////////////////////////////////////////////////////////// -void ModeExecutive::do_match_merge() +void ModeExecutive::do_match_merge_traditional() { - do_merging(); + do_merging_traditional(); // Do the matching of objects between fields @@ -1348,82 +1217,104 @@ void ModeExecutive::set_raw_to_full(float *fcst_raw_data, /////////////////////////////////////////////////////////////////////// -void ModeExecutive::process_output(const MultiVarData *mvdf, - const MultiVarData *mvdo) - +void ModeExecutive::process_output_multivar_super() { + isMultivarOutput = false; + isMultivarSuperOutput = true; - // store to class member so don't have to pass it around - isMultivarOutput = (ptype == MULTIVAR_INTENSITY); - isMultivarSuperOutput = (ptype == MULTIVAR_SUPER); + // use the configured multivar name and level + fcst_magic_string = engine.conf_info.fcst_multivar_name.string() + "_" + engine.conf_info.fcst_multivar_level.string(); + obs_magic_string = engine.conf_info.obs_multivar_name.string() + "_" + engine.conf_info.obs_multivar_level.string(); - if (isMultivarOutput) { + // Create output stats files and plots - // get the magic strings, which will be used in file naming - fcst_magic_string = engine.conf_info.Fcst->var_info->magic_str().c_str(); - obs_magic_string = engine.conf_info.Obs->var_info->magic_str().c_str(); + write_obj_stats(); - // replace forward slashes with underscores to prevent new directories - replace(fcst_magic_string.begin(), fcst_magic_string.end(), '/', '_'); - replace(obs_magic_string.begin(), obs_magic_string.end(), '/', '_'); + if ( engine.conf_info.ct_stats_flag ) write_ct_stats(); + + write_obj_netcdf(engine.conf_info.nc_info); - // replace (*,*) with '_all_all_' - replaceAll(fcst_magic_string, "*", "all"); - replaceAll(obs_magic_string, "*", "all"); - replaceAll(fcst_magic_string, ",", "_"); - replaceAll(obs_magic_string, ",", "_"); - replaceAll(fcst_magic_string, "(", "_"); - replaceAll(obs_magic_string, "(", "_"); - replaceAll(fcst_magic_string, ")", ""); - replaceAll(obs_magic_string, ")", ""); - - } else if (isMultivarSuperOutput) { + if ( engine.conf_info.ps_plot_flag ) plot_engine(); - // use the configured multivar name and level - fcst_magic_string = engine.conf_info.fcst_multivar_name.string() + "_" + engine.conf_info.fcst_multivar_level.string(); - obs_magic_string = engine.conf_info.obs_multivar_name.string() + "_" + engine.conf_info.obs_multivar_level.string(); + return; - } else { +} - // just in case make these empty - fcst_magic_string = ""; - obs_magic_string = ""; +/////////////////////////////////////////////////////////////////////// - } - +void ModeExecutive::process_output_multivar_intensity_compare(const MultiVarData *mvdf, + const MultiVarData *mvdo) + +{ + isMultivarOutput = true; + isMultivarSuperOutput = false; + + // get the magic strings, which will be used in file naming + fcst_magic_string = engine.conf_info.Fcst->var_info->magic_str().c_str(); + obs_magic_string = engine.conf_info.Obs->var_info->magic_str().c_str(); + + // replace forward slashes with underscores to prevent new directories + replace(fcst_magic_string.begin(), fcst_magic_string.end(), '/', '_'); + replace(obs_magic_string.begin(), obs_magic_string.end(), '/', '_'); + + // replace (*,*) with '_all_all_' + replaceAll(fcst_magic_string, "*", "all"); + replaceAll(obs_magic_string, "*", "all"); + replaceAll(fcst_magic_string, ",", "_"); + replaceAll(obs_magic_string, ",", "_"); + replaceAll(fcst_magic_string, "(", "_"); + replaceAll(obs_magic_string, "(", "_"); + replaceAll(fcst_magic_string, ")", ""); + replaceAll(obs_magic_string, ")", ""); + // Create output stats files and plots write_obj_stats(); if ( engine.conf_info.ct_stats_flag ) write_ct_stats(); - - if (isMultivarOutput) { - if (mvdf && mvdo) { - double fmin = mvdf->_data_min; - double fmax = mvdf->_data_max; - double omin = mvdo->_data_min; - double omax = mvdo->_data_max; - double data_min, data_max; - if (!is_bad_data(fmin) && !is_bad_data(omin)) data_min = min(fmin, omin); - else if(!is_bad_data(fmin) && is_bad_data(omin)) data_min = fmin; - else if( is_bad_data(fmin) && !is_bad_data(omin)) data_min = omin; - - if (!is_bad_data(fmax) && !is_bad_data(omax)) data_max = max(fmax, omax); - else if(!is_bad_data(fmax) && is_bad_data(omax)) data_max = fmax; - else if( is_bad_data(fmax) && !is_bad_data(omax)) data_max = omax; + double fmin = mvdf->_data_min; + double fmax = mvdf->_data_max; + double omin = mvdo->_data_min; + double omax = mvdo->_data_max; + double data_min, data_max; + if (!is_bad_data(fmin) && !is_bad_data(omin)) data_min = min(fmin, omin); + else if(!is_bad_data(fmin) && is_bad_data(omin)) data_min = fmin; + else if( is_bad_data(fmin) && !is_bad_data(omin)) data_min = omin; + + if (!is_bad_data(fmax) && !is_bad_data(omax)) data_max = max(fmax, omax); + else if(!is_bad_data(fmax) && is_bad_data(omax)) data_max = fmax; + else if( is_bad_data(fmax) && !is_bad_data(omax)) data_max = omax; - set_raw_to_full(mvdf->_simple->_raw_data, - mvdo->_simple->_raw_data, - mvdf->_nx, mvdf->_ny, - data_min, data_max); - } else { - mlog << Error << "\nModeExecutive::process_output() -> " - << "no multivar data when multivar data is expected\n\n"; - exit(1); - } - } + set_raw_to_full(mvdf->_simple->_raw_data,mvdo->_simple->_raw_data, + mvdf->_nx, mvdf->_ny, data_min, data_max); + + write_obj_netcdf(engine.conf_info.nc_info); + + if ( engine.conf_info.ps_plot_flag ) plot_engine(); + + return; + +} + +/////////////////////////////////////////////////////////////////////// + +void ModeExecutive::process_output_traditional() +{ + isMultivarOutput = false; + isMultivarSuperOutput = false; + + // just in case make these empty + fcst_magic_string = ""; + obs_magic_string = ""; + + // Create output stats files and plots + + write_obj_stats(); + + if ( engine.conf_info.ct_stats_flag ) write_ct_stats(); + write_obj_netcdf(engine.conf_info.nc_info); if ( engine.conf_info.ps_plot_flag ) plot_engine(); @@ -1737,7 +1628,8 @@ MultiVarData *ModeExecutive::get_multivar_data(ModeDataType dtype) obs_magic_string = engine.conf_info.Obs->var_info->magic_str().c_str(); // replace forward slashes with underscores to prevent new directories replace(obs_magic_string.begin(), obs_magic_string.end(), '/', '_'); - mvd->init(dtype, obs_magic_string, grid, otype, ounits, olevel, data_min, data_max); + mvd->init(dtype, obs_magic_string, grid, + ounits, olevel, data_min, data_max); mvd->set_obj(engine.obs_split, simple); mvd->set_raw(engine.obs_raw, simple); mvd->set_shapedata(Obs_sd, simple); @@ -1748,7 +1640,8 @@ MultiVarData *ModeExecutive::get_multivar_data(ModeDataType dtype) fcst_magic_string = engine.conf_info.Fcst->var_info->magic_str().c_str(); // replace forward slashes with underscores to prevent new directories replace(fcst_magic_string.begin(), fcst_magic_string.end(), '/', '_'); - mvd->init(dtype, fcst_magic_string, grid, ftype, funits, flevel, data_min, data_max); + mvd->init(dtype, fcst_magic_string, grid, + funits, flevel, data_min, data_max); mvd->set_obj(engine.fcst_split, simple); mvd->set_raw(engine.fcst_raw, simple); mvd->set_shapedata(Fcst_sd, simple); @@ -2738,7 +2631,7 @@ void ModeExecutive::write_ct_stats() /////////////////////////////////////////////////////////////////////// -void ModeExecutive::conf_read(const string &default_config_filename) +void ModeExecutive::conf_read() { // Create the default config file name diff --git a/src/tools/core/mode/mode_exec.h b/src/tools/core/mode/mode_exec.h index 34c8d7fb5e..f3c9937d35 100644 --- a/src/tools/core/mode/mode_exec.h +++ b/src/tools/core/mode/mode_exec.h @@ -75,83 +75,118 @@ class ModeExecutive { // the various mode algorithm settings typedef enum {TRADITIONAL, MULTIVAR_SIMPLE, MULTIVAR_SIMPLE_MERGE, MULTIVAR_INTENSITY, MULTIVAR_SUPER} Processing_t; - ModeExecutive(Processing_t p=TRADITIONAL); + ModeExecutive(); ~ModeExecutive(); void clear(); void init_traditional(int n_files); - void init_multivar_verif_grid(); - void init_multivar_simple(int n_files, ModeDataType dtype); - void init_multivar_intensities(GrdFileType ftype, GrdFileType otype); - void check_multivar_perc_thresh_settings(); - + void init_multivar_simple(int j, int n_files, ModeDataType dtype, const ModeConfInfo &conf); + void init_multivar_intensities(const ModeConfInfo &conf); int n_conv_radii () const; int n_conv_threshs () const; - int n_runs() const; + // these are used only for traditional mode int R_index; // indices into the convolution radius and threshold arrays int T_index; // for the current run // - // Input configuration files + // Input configuration files, all 3 used only for traditional mode + // Multivar mode handles the configs outside of the exec // - + // the hardwired default traditional mode config ConcatString default_config_file; - ConcatString match_config_file; + + // set for both trad and multivar, this is the default file on the command line + // used only for traditional + ConcatString match_config_file; + + // the extra one that can be set only for traditional mode ConcatString merge_config_file; // - // Input files + // Input filenames, set for both multivar and trad mode + // but used only for trad mode // - ConcatString fcst_file; ConcatString obs_file; + + // set and used only for trad mode Met2dDataFile * fcst_mtddf; Met2dDataFile * obs_mtddf; + // used for both trad and multivar mode TTContingencyTable cts[n_cts]; + // used for both trad and multivar mode ModeFuzzyEngine engine; // verification grid + // used for both trad and multivar mode, set by both Grid grid; Box xy_bb; ConcatString out_dir; + + // set for both trad and multivar mode, used for plotting limits double data_min, data_max; + // set for trad and multivar, used in the engine mode algorithm ShapeData Fcst_sd, Obs_sd; + // not used by multivar GrdFileType ftype, otype; + + // set into execs's conf varInfo object, only for multivar intensity comparisons + // for trad it's read in from the config string funits, ounits; + + // set into execs's conf varInfo object, only for multivar intensity comparisons + // for trad it's read in from the config string flevel, olevel; + // used in multivar only to customize outputs correctly bool isMultivarOutput; bool isMultivarSuperOutput; - Processing_t ptype; - + void setup_verification_grid(const ModeInputData &fcst, + const ModeInputData &obs, + const ModeConfInfo &conf); + void clear_internal_r_index(); - void setup_verification_grid(); - void setup_fcst_obs_data_traditional(); - void setup_fcst_data(const Grid &verification_grid); - void setup_obs_data(const Grid &verification_grid); - void setup_fcst_obs_data_multivar_intensities(const MultiVarData &mvdf, const MultiVarData &mvdo); - void setup_fcst_obs_data_multivar_super(ShapeData &f_super, ShapeData &o_super, const Grid &igrid); - void do_conv_thresh(const int r_index, const int t_index); - void do_merging(); - void do_merging(ShapeData &f_merge, ShapeData &o_merge); - void do_match_merge(); - void do_match_merge(ShapeData &f_merge, ShapeData &o_merge); + + void setup_traditional_fcst_obs_data(); + void setup_multivar_fcst_data(const Grid &verification_grid, const ModeInputData &input); + void setup_multivar_obs_data(const Grid &verification_grid, const ModeInputData &input); + void setup_multivar_fcst_obs_data_intensities(const MultiVarData &mvdf, + const MultiVarData &mvdo); + void setup_multivar_fcst_obs_data_super(const ShapeData &f_super, + const ShapeData &o_super, + const Grid &igrid); + + void do_conv_thresh_traditional(const int r_index, const int t_index); + void do_conv_thresh_multivar_super(); + void do_conv_thresh_multivar_intensity_compare(); + void do_conv_thresh_multivar_simple(Processing_t p); + + void do_merging_traditional(); + void do_merging_multivar(const ShapeData &f_merge, const ShapeData &o_merge, + Processing_t p); + + void do_match_merge_traditional(); + void do_match_merge_multivar(const ShapeData &f_merge, const ShapeData &o_merge, + Processing_t p); void process_masks(ShapeData &, ShapeData &); void process_fcst_masks(ShapeData &); void process_obs_masks(ShapeData &); - void process_output(const MultiVarData *mvdf=NULL, - const MultiVarData *mvdo=NULL); + + void process_output_traditional(); + void process_output_multivar_intensity_compare(const MultiVarData *mvdf, + const MultiVarData *mvdo); + void process_output_multivar_super(); void set_raw_to_full(float *fcst_raw_data, float *obs_raw_data, @@ -178,7 +213,9 @@ class ModeExecutive { void write_poly_netcdf(netCDF::NcFile *, const ObjPolyType); void write_ct_stats(); - void conf_read(const string &default_config_filename); + // traditional only, multivar reads outside of the exec + void conf_read(); + static string stype(Processing_t t); }; diff --git a/src/tools/core/mode/mode_frontend.cc b/src/tools/core/mode/mode_frontend.cc index 456ae71f7a..4831cb59d2 100644 --- a/src/tools/core/mode/mode_frontend.cc +++ b/src/tools/core/mode/mode_frontend.cc @@ -35,9 +35,10 @@ using namespace std; extern const char * const program_name; static ModeExecutive *mode_exec = 0; -static ModeExecutive::Processing_t ptype = ModeExecutive::TRADITIONAL; -static int compress_level = -1; +// used only for traditional mode, multivar sets it into config previous +// to the frontend creation +static int compress_level = -1; /////////////////////////////////////////////////////////////////////// @@ -61,160 +62,13 @@ ModeFrontEnd::~ModeFrontEnd() } -/////////////////////////////////////////////////////////////////////// - - -Grid ModeFrontEnd::create_verification_grid(const StringArray & Argv) - -{ - if ( mode_exec ) { delete mode_exec; mode_exec = 0; } - mode_exec = new ModeExecutive; - compress_level = -1; - - // - // Process the command line arguments - // - - process_command_line(Argv, false); - - mode_exec->init_multivar_verif_grid(); - - ModeConfInfo & conf = mode_exec->engine.conf_info; - conf.set_field_index(0); - if (compress_level >= 0) conf.nc_info.set_compress_level(compress_level); - - - // - // read in data (Note multiple reads of same data) - // - mode_exec->setup_verification_grid(); - Grid g = mode_exec->grid; - delete mode_exec; mode_exec = 0; - return g; -} - - -/////////////////////////////////////////////////////////////////////// - -int ModeFrontEnd::create_multivar_simple_objects(const StringArray & Argv, ModeDataType dtype, - const Grid &verification_grid, int field_index, int n_files) - -{ - init(ModeExecutive::MULTIVAR_SIMPLE); - - // - // Process the command line arguments - // - - process_command_line_for_simple_objects(Argv, dtype); - - mode_exec->init_multivar_simple(n_files, dtype); - - ModeConfInfo & conf = mode_exec->engine.conf_info; - if ( field_index >= 0 ) conf.set_field_index(field_index); - if (compress_level >= 0) conf.nc_info.set_compress_level(compress_level); - - // need to do this after setting field index above - mode_exec->check_multivar_perc_thresh_settings(); - - // - // read in data (Note multiple reads of data) - // - - if (dtype == ModeDataType_MvMode_Fcst) { - mode_exec->setup_fcst_data(verification_grid); - } else { - mode_exec->setup_obs_data(verification_grid); - } - - // - // mode algorithm - // - if ( conf.quilt ) { - - do_quilt(); - - } else { - - do_straight(); - - } - - // - // done - // - -#ifdef WITH_PYTHON - GP.finalize(); -#endif - return (0); -} - -/////////////////////////////////////////////////////////////////////// - -int ModeFrontEnd::create_multivar_merge_objects(const StringArray & Argv, ModeDataType dtype, - const Grid &verification_grid, int field_index, - int n_files) - -{ - init(ModeExecutive::MULTIVAR_SIMPLE_MERGE); - - // - // Process the command line arguments - // - - process_command_line_for_simple_objects(Argv, dtype); - - mode_exec->init_multivar_simple(n_files, dtype); - - ModeConfInfo & conf = mode_exec->engine.conf_info; - if ( field_index >= 0 ) conf.set_field_index(field_index); - if (compress_level >= 0) conf.nc_info.set_compress_level(compress_level); - - // need to do this after setting field index above - mode_exec->check_multivar_perc_thresh_settings(); - - - // - // read in data (Note multiple reads not desired) - // - - if (dtype == ModeDataType_MvMode_Fcst) { - mode_exec->setup_fcst_data(verification_grid); - } else { - mode_exec->setup_obs_data(verification_grid); - } - - - // - // mode algorithm - // - if ( conf.quilt ) { - - do_quilt(); - - } else { - - do_straight(); - - } - - // - // done - // - -#ifdef WITH_PYTHON - GP.finalize(); -#endif - return (0); -} /////////////////////////////////////////////////////////////////////// int ModeFrontEnd::run_traditional(const StringArray & Argv) { - init(ModeExecutive::TRADITIONAL); + init(); int field_index = -1; int n_files = 1; @@ -223,7 +77,7 @@ int ModeFrontEnd::run_traditional(const StringArray & Argv) // Process the command line arguments // - process_command_line(Argv, false); + process_command_line(Argv); mode_exec->init_traditional(n_files); @@ -236,7 +90,7 @@ int ModeFrontEnd::run_traditional(const StringArray & Argv) // read in data // - mode_exec->setup_fcst_obs_data_traditional(); + mode_exec->setup_traditional_fcst_obs_data(); // // mode algorithm @@ -263,129 +117,6 @@ int ModeFrontEnd::run_traditional(const StringArray & Argv) /////////////////////////////////////////////////////////////////////// -int ModeFrontEnd::multivar_intensity_comparisons(const StringArray & Argv, const MultiVarData &mvdf, - const MultiVarData &mvdo, bool has_union_f, - bool has_union_o, ShapeData &merge_f, - ShapeData &merge_o, int field_index_f, int field_index_o) -{ - init(ModeExecutive::MULTIVAR_INTENSITY); - - // - // Process the command line arguments - // - - process_command_line(Argv, false); - - mode_exec->init_multivar_intensities(mvdf._type, mvdo._type); - - ModeConfInfo & conf = mode_exec->engine.conf_info; - if (compress_level >= 0) conf.nc_info.set_compress_level(compress_level); - conf.set_field_index(field_index_f, field_index_o); - - // for multivar intensities, explicity set the level and units using stored values - // from pass1 - conf.Fcst->var_info->set_level_name(mvdf._level.c_str()); - conf.Fcst->var_info->set_units(mvdf._units.c_str()); - if (has_union_f && conf.Fcst->merge_flag == MergeType_Thresh) { - mlog << Warning << "\nModeFrontEnd::multivar_intensity_comparisons() -> " - << "Logic includes union '||' along with 'merge_flag=THRESH' " - << ". This can lead to bad results\n\n"; - } - conf.Obs->var_info->set_level_name(mvdo._level.c_str()); - conf.Obs->var_info->set_units(mvdo._units.c_str()); - if (has_union_o && conf.Obs->merge_flag == MergeType_Thresh) { - mlog << Warning << "\nModeFrontEnd::multivar_intensity_comparisons() -> " - << "Logic includes union '||' along with 'merge_flag=THRESH' " - << ". This can lead to bad results\n\n"; - } - - // - // set up data access using inputs - // - mode_exec->setup_fcst_obs_data_multivar_intensities(mvdf, mvdo); - - // - // run the mode algorithm - // - - if ( conf.quilt ) { - - do_quilt(); - - } else { - - do_straight_multivar_intensity(mvdf, mvdo, merge_f, merge_o); - - } - - // - // done - // - -#ifdef WITH_PYTHON - GP.finalize(); -#endif - return (0); -} - -/////////////////////////////////////////////////////////////////////// - -int ModeFrontEnd::run_super(const StringArray & Argv, - ShapeData &f_super, ShapeData &o_super, - ShapeData &f_merge, ShapeData &o_merge, - GrdFileType ftype, GrdFileType otype, const Grid &grid, - bool has_union) -{ - init(ModeExecutive::MULTIVAR_SUPER); - - // - // Process the command line arguments - // - - process_command_line(Argv, true); - - mode_exec->init_multivar_intensities(ftype, otype); - - ModeConfInfo & conf = mode_exec->engine.conf_info; - if (compress_level >= 0) conf.nc_info.set_compress_level(compress_level); - if (has_union && (conf.Fcst->merge_flag == MergeType_Thresh || - conf.Obs->merge_flag == MergeType_Thresh)) { - mlog << Warning << "\nModeFrontEnd::run_super() -> " - << "Logic includes union '||' along with 'merge_flag=THRESH' " - << ". This can lead to bad results\n\n"; - } - - // - // set up data access using inputs - // - mode_exec->setup_fcst_obs_data_multivar_super(f_super, o_super, grid); - - // - // run the mode algorithm - // - - if ( conf.quilt ) { - - do_quilt(); - - } else { - - do_straight_multivar_super(f_merge, o_merge); - - } - - // - // done - // - -#ifdef WITH_PYTHON - GP.finalize(); -#endif - return (0); -} - -/////////////////////////////////////////////////////////////////////// - void ModeFrontEnd::do_straight() @@ -398,78 +129,9 @@ void ModeFrontEnd::do_straight() for (int index=0; indexdo_conv_thresh(index, index); - if (ptype == ModeExecutive::TRADITIONAL) { - - mode_exec->do_match_merge(); - mode_exec->process_output(); - } - } - - mode_exec->clear_internal_r_index(); - - // - // done - // - - return; - -} - -/////////////////////////////////////////////////////////////////////// - - -void ModeFrontEnd::do_straight_multivar_intensity(const MultiVarData &mvdf, - const MultiVarData &mvdo, - ShapeData &f_merge, - ShapeData &o_merge) - -{ - int NCT, NCR; - - do_straight_init(NCT, NCR); - - mode_exec->clear_internal_r_index(); - - for (int index=0; indexdo_conv_thresh(index, index); - mode_exec->do_match_merge(f_merge, o_merge); - - // here replace raw data and min/max for plotting - - mode_exec->process_output(&mvdf, &mvdo); - } - - mode_exec->clear_internal_r_index(); - - // - // done - // - - return; - -} - - - -/////////////////////////////////////////////////////////////////////// - - -void ModeFrontEnd::do_straight_multivar_super(ShapeData &f_merge, ShapeData &o_merge) - -{ - int NCT, NCR; - - do_straight_init(NCT, NCR); - - mode_exec->clear_internal_r_index(); - - for (int index=0; indexdo_conv_thresh(index, index); - mode_exec->do_match_merge(f_merge, o_merge); - mode_exec->process_output(); + mode_exec->do_conv_thresh_traditional(index, index); + mode_exec->do_match_merge_traditional(); + mode_exec->process_output_traditional(); } mode_exec->clear_internal_r_index(); @@ -482,19 +144,12 @@ void ModeFrontEnd::do_straight_multivar_super(ShapeData &f_merge, ShapeData &o_m } - /////////////////////////////////////////////////////////////////////// void ModeFrontEnd::do_quilt() { - if (ptype != ModeExecutive::TRADITIONAL) { - mlog << Error << "\nModeFrontend::do_quilt() -> quilting not yet implemented for multivar mode \n\n"; - exit ( 1 ); - } - - int t_index, r_index; // indices into the convolution threshold and radius arrays @@ -503,14 +158,9 @@ void ModeFrontEnd::do_quilt() for (r_index=0; r_index<(mode_exec->n_conv_radii()); ++r_index) { for (t_index=0; t_index<(mode_exec->n_conv_threshs()); ++t_index) { - - mode_exec->do_conv_thresh(r_index, t_index); - - mode_exec->do_match_merge(); - - if (ptype == ModeExecutive::TRADITIONAL) { - mode_exec->process_output(); - } + mode_exec->do_conv_thresh_traditional(r_index, t_index); + mode_exec->do_match_merge_traditional(); + mode_exec->process_output_traditional(); } } @@ -526,29 +176,13 @@ void ModeFrontEnd::do_quilt() /////////////////////////////////////////////////////////////////////// -MultiVarData *ModeFrontEnd::get_multivar_data(ModeDataType dtype) -{ - return mode_exec->get_multivar_data(dtype); -} - - -/////////////////////////////////////////////////////////////////////// - -void ModeFrontEnd::add_multivar_merge_data(MultiVarData *mvdi, ModeDataType dtype) -{ - return mode_exec->add_multivar_merge_data(mvdi, dtype); -} - -/////////////////////////////////////////////////////////////////////// - -void ModeFrontEnd::init(ModeExecutive::Processing_t p) +void ModeFrontEnd::init() { - ptype = p; - mlog << Debug(1) << "Running multivar front end for " << ModeExecutive::stype(ptype) << "\n"; + mlog << Debug(1) << "Running traditional mode front end\n"; if ( mode_exec ) { delete mode_exec; mode_exec = 0; } - mode_exec = new ModeExecutive(ptype); + mode_exec = new ModeExecutive();//ModeExecutive::TRADITIONAL); compress_level = -1; } @@ -569,20 +203,12 @@ void ModeFrontEnd::do_straight_init(int &NCT, int &NCR) const exit ( 1 ); } - - if (NCT > 1 && ptype != ModeExecutive::TRADITIONAL) { - - mlog << Error << "\nModeFrontEnd::do_straight_init() ->" - << ": multiple convolution radii and thresholds not implemented in multivar mode\n\n"; - - exit ( 1 ); - } } /////////////////////////////////////////////////////////////////////// -void ModeFrontEnd::process_command_line(const StringArray & argv, bool ismultivar) +void ModeFrontEnd::process_command_line(const StringArray & argv) { CommandLine cline; ConcatString s; @@ -595,7 +221,7 @@ void ModeFrontEnd::process_command_line(const StringArray & argv, bool ismultiva mode_exec->out_dir = replace_path(default_out_dir); // - // Check for zero arguments (note not correct for multivar mode, want to show multivar_usage + // Check for zero arguments // if(argc == 1) singlevar_usage(); @@ -607,7 +233,7 @@ void ModeFrontEnd::process_command_line(const StringArray & argv, bool ismultiva cline.set(argv); // - // Set the usage function NOTE wrong for multivar, want multivar_usage + // Set the usage function // cline.set_usage(singlevar_usage); @@ -628,103 +254,19 @@ void ModeFrontEnd::process_command_line(const StringArray & argv, bool ismultiva cline.parse(); - if (ismultivar) { - // - // Check for error. There should be 1 argument left: - // config filename - // - if(cline.n() != 1) singlevar_usage(); // wrong need multivar usage - - // - // Store the input forecast and observation file names, placeholders - // - mode_exec->fcst_file = "not set"; - mode_exec->obs_file = "not set"; - mode_exec->match_config_file = cline[0]; - - } else { - // - // Check for error. There should be three arguments left: - // forecast, observation, and config filenames - // - if(cline.n() != 3) singlevar_usage(); - - // - // Store the input forecast and observation file names - // - mode_exec->fcst_file = cline[0]; - mode_exec->obs_file = cline[1]; - mode_exec->match_config_file = cline[2]; - - } -} - - -/////////////////////////////////////////////////////////////////////// - -void ModeFrontEnd::process_command_line_for_simple_objects(const StringArray &argv, ModeDataType dtype) -{ - CommandLine cline; - ConcatString s; - const int argc = argv.n(); - - // - // Set the default output directory - // - - mode_exec->out_dir = replace_path(default_out_dir); - - // - // Check for zero arguments (note not correct for multivar mode, want to show multivar_usage - // - - if(argc == 1) singlevar_usage(); - - // - // Parse the command line into tokens - // - - cline.set(argv); - - // - // Set the usage function NOTE wrong for multivar, want multivar_usage - // - - cline.set_usage(singlevar_usage); - - // - // Add the options function calls - // - - cline.add(set_config_merge_file, "-config_merge", 1); - cline.add(set_outdir, "-outdir", 1); - cline.add(set_logfile, "-log", 1); - cline.add(set_verbosity, "-v", 1); - cline.add(set_compress, "-compress", 1); - // - // Parse the command line + // Check for error. There should be three arguments left: + // forecast, observation, and config filenames // - - cline.parse(); + if(cline.n() != 3) singlevar_usage(); // - // Check for error. There should be two arguments left: - // data and config filenames + // Store the input forecast and observation file names // - if(cline.n() != 2) singlevar_usage(); + mode_exec->fcst_file = cline[0]; + mode_exec->obs_file = cline[1]; + mode_exec->match_config_file = cline[2]; - // - // Store the file name - // - if (dtype == ModeDataType_MvMode_Fcst) { - mode_exec->fcst_file = cline[0]; - mode_exec->obs_file = "None"; - } else { - mode_exec->obs_file = cline[0]; - mode_exec->fcst_file = "None"; - } - mode_exec->match_config_file = cline[1]; } /////////////////////////////////////////////////////////////////////// diff --git a/src/tools/core/mode/mode_frontend.h b/src/tools/core/mode/mode_frontend.h index 55ecdf7f50..658f90ff4c 100644 --- a/src/tools/core/mode/mode_frontend.h +++ b/src/tools/core/mode/mode_frontend.h @@ -19,8 +19,6 @@ #include #include "mode_exec.h" #include "string_array.h" -#include "multivar_data.h" -#include "mode_data_type.h" class ModeFrontEnd { @@ -34,54 +32,21 @@ class ModeFrontEnd { string default_out_dir; - Grid create_verification_grid(const StringArray & Argv); - - // run the multivar simple object, where there is only one input data, either forecast or obs - int create_multivar_simple_objects(const StringArray & Argv, ModeDataType dtype, const Grid &verification_grid, - int field_index=-1, int n_files=1); - - // run the multivar simple object merge algorithm, with one input data, either forecast or obs - int create_multivar_merge_objects(const StringArray & Argv, ModeDataType dtype, const Grid &verification_grid, - int field_index=-1, int n_files=1); // run the default single var mode interface (traditional mode) int run_traditional(const StringArray & Argv); - // run the multivar intensity algorithm, where one forecast and one obs are restricted to be within superobjects - // and the traditional mode algorithm compares them - int multivar_intensity_comparisons(const StringArray & Argv, const MultiVarData &mvdf, const MultiVarData &mvdo, - bool has_union_f, bool has_union_o, ShapeData &merge_f, - ShapeData &merge_o, int field_index_f, int field_index_o); - - // multivar superobject interface, with no intensities - int run_super(const StringArray & Argv, ShapeData &f_super, ShapeData &o_super, - ShapeData &f_merge, ShapeData &o_merge, - GrdFileType ftype, GrdFileType otype, const Grid &grid, bool has_union); - + void init(); + // so far only implemented for traditional mode void do_quilt (); - // MODE algorithm for traditional, multivar simple, or multivar merge cases + // MODE algorithm for traditional mode void do_straight (); - // MODE algorithm when doing multivar intensities - void do_straight_multivar_intensity (const MultiVarData &mvdf, - const MultiVarData &mvdo, ShapeData &mergef, - ShapeData &mergeo); - - // MODE algorithm when doing multivar super with no intensities - void do_straight_multivar_super (ShapeData &f_merge, ShapeData &o_merge); - - - MultiVarData *get_multivar_data(ModeDataType dtype); - - void add_multivar_merge_data(MultiVarData *mvdi, ModeDataType dtype); - - void init(ModeExecutive::Processing_t p); void do_straight_init(int &NCT, int &NCR) const; - void process_command_line_for_simple_objects(const StringArray &, ModeDataType dtype); - void process_command_line(const StringArray &, bool is_multivar); + void process_command_line(const StringArray &); static void set_config_merge_file (const StringArray &); static void set_outdir (const StringArray &); diff --git a/src/tools/core/mode/mode_superobject.cc b/src/tools/core/mode/mode_superobject.cc new file mode 100644 index 0000000000..27d1c3092f --- /dev/null +++ b/src/tools/core/mode/mode_superobject.cc @@ -0,0 +1,184 @@ +using namespace std; +#include "mode_superobject.h" +#include "multivar_data.h" + +//////////////////////////////////////////////////////////////////////// +static void _mask_super(const string &name, int nx, int ny, DataPlane &data) +{ + + if (nx != data.nx() || ny != data.ny()) { + mlog << Error << "\nModeSuperObject::mask_data_super() -> " << name + << " :dimensions don't match " << nx << " " << ny + << " " << data.nx() << " " << data.ny() << "\n\n"; + + exit( 1 ); + } + + int nmasked=0, nkeep=0; + + for (int x=0; x " << name + << " :dimensions don't match " << nx << " " << ny + << " " << data.nx() << " " << data.ny() << "\n\n"; + + exit( 1 ); + } + + int nmasked=0, nkeep=0; + + for (int x=0; x values; + vector count; + for (int x=0; x::iterator vi; + vi = find(values.begin(), values.end(), v); + if (vi == values.end()) { + values.push_back(v); + count.push_back(1); + } else { + int ii = vi - values.begin(); + count[ii] = count[ii] + 1; + } + } + } + for (size_t i=0; i &mvd, + BoolCalc &calc) +{ + _hasUnion = calc.has_union(); + + // + // set the BoolPlane values using the mvd content + // + + BoolPlane * simple_plane = new BoolPlane [n_files]; + BoolPlane * merge_plane = new BoolPlane [n_files]; + + for (int j=0; jobjects_from_arrays(do_clusters, true, simple_plane[j]); + mvd[j]->objects_from_arrays(do_clusters, false, merge_plane[j]); + } + + // + // combine the objects into super-objects + // + const int nx = simple_plane[0].nx(); + const int ny = simple_plane[0].ny(); + + BoolPlane merge_result; // local, not used + _simple_result.set_size(nx, ny); + merge_result.set_size(nx, ny); + + string simple_name, merge_name; + + if (isFcst) { + simple_name = "Fcst_Simple"; + merge_name = "Fcst_Merge"; + } else { + simple_name = "Obs_Simple"; + merge_name = "Obs_Merge"; + } + combine_boolplanes(simple_name, simple_plane, n_files, calc, _simple_result); + combine_boolplanes(merge_name, merge_plane, n_files, calc, merge_result); + + // create ShapeData objects using something from mvd as a template + // (shape data has 1's or bad) + + _simple_sd = ShapeData(*(mvd[0]->_simple->_sd)); + for (int x=0; x_simple->_sd)); + for (int x=0; x_sd->data); +} + + +void ModeSuperObject::mask_data_super(const string &name, const MultiVarData &mvd) +{ + int nx = mvd._nx; + int ny = mvd._ny; + _mask_super(name, nx, ny, _simple_sd.data); +} diff --git a/src/tools/core/mode/mode_superobject.h b/src/tools/core/mode/mode_superobject.h new file mode 100644 index 0000000000..83278b20a6 --- /dev/null +++ b/src/tools/core/mode/mode_superobject.h @@ -0,0 +1,51 @@ +// ** Copyright UCAR (c) 1992 - 2023 +// ** University Corporation for Atmospheric Research (UCAR) +// ** National Center for Atmospheric Research (NCAR) +// ** Research Applications Lab (RAL) +// ** P.O.Box 3000, Boulder, Colorado, 80307-3000, USA +// *=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=* + + +//////////////////////////////////////////////////////////////////////// + + +#ifndef __MODE_SUPEROBJECT_H__ +#define __MODE_SUPEROBJECT_H__ + + +//////////////////////////////////////////////////////////////////////// + + +#include "combine_boolplanes.h" +#include "multivar_data.h" +#include "shapedata.h" +#include +#include + +class MultiVarData; + +class ModeSuperObject { + + private: + + public: + + ModeSuperObject(bool isFcst, int n_files, bool do_clusters, + const vector &mvd, + BoolCalc &calc); + inline ~ModeSuperObject() {} + + void mask_data_simple(const string &name, MultiVarData &mvd) const; + void mask_data_super(const string &name, const MultiVarData &mvd); + + bool _isFcst; + bool _hasUnion; + BoolPlane _simple_result; + ShapeData _simple_sd; + ShapeData _merge_sd_split; +}; + + +#endif /* __MODE_SUPEROBJECT_H__ */ + +///////////////////////////////////////////////////////////////////////// diff --git a/src/tools/core/mode/multivar_data.cc b/src/tools/core/mode/multivar_data.cc index 1d38d270ba..3cde9ccf83 100644 --- a/src/tools/core/mode/multivar_data.cc +++ b/src/tools/core/mode/multivar_data.cc @@ -142,8 +142,7 @@ MultiVarData::MultiVarData() : _merge(0), _name("notset"), _nx(0), _ny(0), - _grid(0), - _type(FileType_None) + _grid(0) { } @@ -152,23 +151,10 @@ MultiVarData::~MultiVarData() _clear(); } -void MultiVarData::checkFileTypeConsistency(const MultiVarData &mvdi, int j) -{ - bool err = false; - if (_type != mvdi._type) { - mlog << Error << "MultivarData::checkFileTypeConsistgency() -> " - << "inputs of different file types not supported " - << "Input 0:" << grdfiletype_to_string(_type).c_str() - << "Input " << j << ":" << grdfiletype_to_string(mvdi._type).c_str() - << "\n\n"; - exit ( 1 ); - } -} - void MultiVarData::init(ModeDataType dataType, const string &name, - const Grid &grid, GrdFileType type, - const string &units, + const Grid &grid, + const string &units, const string &level, double data_min, double data_max) { @@ -179,7 +165,6 @@ void MultiVarData::init(ModeDataType dataType, _nx = grid.nx(); _ny = grid.ny(); _grid = new Grid(grid); - _type = type; _units = units; _level = level; _data_min = data_min; @@ -286,7 +271,6 @@ void MultiVarData::_clear() _grid = 0; } _nx = _ny = 0; - _type = FileType_None; } diff --git a/src/tools/core/mode/multivar_data.h b/src/tools/core/mode/multivar_data.h index 142e1aa65b..f93b7f9ac5 100644 --- a/src/tools/core/mode/multivar_data.h +++ b/src/tools/core/mode/multivar_data.h @@ -85,13 +85,11 @@ class MultiVarData { MultiVarData(); ~MultiVarData(); - void checkFileTypeConsistency(const MultiVarData &mvdi, int j); - void init(ModeDataType dataType, const string &name, - const Grid &grid, GrdFileType type, + const Grid &grid, const string &units, - const string &level, + const string &level, double data_min, double data_max); void set_obj(ShapeData *sd, bool simple); @@ -109,7 +107,6 @@ class MultiVarData { string _name; int _nx, _ny; Grid *_grid; - GrdFileType _type; string _units; string _level; double _data_min, _data_max; diff --git a/src/tools/core/mode/multivar_frontend.cc b/src/tools/core/mode/multivar_frontend.cc index 2205fc4329..6bd3a5aa57 100644 --- a/src/tools/core/mode/multivar_frontend.cc +++ b/src/tools/core/mode/multivar_frontend.cc @@ -6,170 +6,88 @@ // ** P.O.Box 3000, Boulder, Colorado, 80307-3000, USA // *=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=* - -//////////////////////////////////////////////////////////////////////// - - -// for multivar mode, this is the default file -static const char mode_default_config [] = "MET_BASE/config/MODEMultivarConfig_default"; - -static const int dir_creation_mode = 0755; - //////////////////////////////////////////////////////////////////////// using namespace std; -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include -#include - -#include - -#include "vx_util.h" -#include "file_exists.h" -#include "two_d_array.h" -#include "get_filenames.h" -#include "mode_conf_info.h" -#include "shapedata.h" -#include "interest.h" -#include "met_file.h" -#include "mode_usage.h" -#include "mode_exec.h" - -#include "combine_boolplanes.h" -#include "objects_from_netcdf.h" -#include "parse_file_list.h" -#include "mode_frontend.h" -#include "multivar_data.h" -#include "mode_data_type.h" +#include "multivar_frontend.h" -using namespace netCDF; +#include "mode_usage.h" +#ifdef WITH_PYTHON +#include "global_python.h" +#endif //////////////////////////////////////////////////////////////////////// extern const char * const program_name; - static const char sep [] = "===================================================="; -static const char tab [] = " "; - -// this is hardwired for the multivar case, at least for now - -static const bool do_clusters = false; +static string outdir; +static int compress_level = -1; -static string default_out_dir = "."; +// for multivar mode, this is the default file +static const char mode_default_config [] = "MET_BASE/config/MODEMultivarConfig_default"; -static ModeConfInfo config; +static const int dir_creation_mode = 0755; -static string mode_path; -static string fcst_fof; -static string obs_fof; -static string config_file; -static string outdir; +static ModeExecutive *mode_exec = 0; -static Grid verification_grid; //////////////////////////////////////////////////////////////////////// - -static void set_outdir (const StringArray &); -static void set_logfile (const StringArray &); -static void set_verbosity (const StringArray &); - -static void multivar_consistency_checks(StringArray &fcst_filenames, StringArray &obs_filenames, - BoolCalc &f_calc, BoolCalc &o_calc, int &n_fcst_files, - int &n_obs_files); - -static ConcatString set_multivar_dir(); - -static void create_verification_grid(const string &fcst_filename, - const string &obs_filename, - const ConcatString &dir); - -static MultiVarData *create_simple_objects(ModeDataType dtype, int j, int n_files, - const string &filename, - const ConcatString &dir); - -static void create_superobjects(int n_fcst_files, const vector &mvdFcst, - int n_obs_files, const vector &mvdObs, - BoolCalc &f_calc, BoolCalc &o_calc, - BoolPlane &f_simple_result, BoolPlane &o_simple_result, - ShapeData &f_simple_sd, ShapeData &o_simple_sd, - ShapeData &f_merge_sd_split, ShapeData &o_merge_sd_split); - -static void create_intensity_comparisons(int findex, int oindex, const BoolPlane &f_result, - BoolPlane &o_result, - const ConcatString &dir, - MultiVarData &mvdf, MultiVarData &mvdo, - bool has_union_f, bool has_union_o, - const string &fcst_filename, const string &obs_filename, - ShapeData &merge_f, ShapeData &merge_o); - -static void process_superobjects(ShapeData &f_result, ShapeData &o_result, - ShapeData &f_merge, ShapeData &o_merge, - int nx, int ny, const ConcatString &dir, - GrdFileType ftype, GrdFileType otype, const Grid &grid, bool has_union); - -static void mask_data(const string &name, int nx, int ny, const BoolPlane &mask, DataPlane &data); -static void mask_data_super(const string &name, int nx, int ny, DataPlane &data); - -static void read_config(const string & filename); - -static void process_command_line(const StringArray &); - -static int _mkdir(const char *dir); - -static void _debug_shape_examine(string &name, const ShapeData &sd, int nx, int ny); - +MultivarFrontEnd::MultivarFrontEnd() +{ + // this is hardwired for the multivar case, at least for now + do_clusters = false; + default_out_dir = "."; + compress_level = -1; + mode_exec = 0; +} //////////////////////////////////////////////////////////////////////// - -int multivar_frontend(const StringArray & Argv) +int MultivarFrontEnd::run(const StringArray & Argv) { - const int Argc = Argv.n(); + // initialize - if ( Argc < 4 ) multivar_usage(); - - int j, n_fcst_files, n_obs_files; - StringArray fcst_filenames; - StringArray obs_filenames; - BoolCalc f_calc, o_calc ; + init(Argv); - process_command_line(Argv); + mlog << Debug(2) << "\n" << sep << "\n"; - read_config(config_file); + // read in all the data - multivar_consistency_checks(fcst_filenames, obs_filenames, f_calc, o_calc, - n_fcst_files, n_obs_files); + // in the conf object, shift *can* be set independently for obs and fcst + int shift = config.shift_right; - bool f_has_union = f_calc.has_union(); - bool o_has_union = o_calc.has_union(); + for (int i=0; i mvdObs, mvdFcst; - - for (j=0; j 0) { - mvdFcst[0]->checkFileTypeConsistency(*mvdi, j); - } + << "\n" << sep << "\ncreating simple forecast objects from forecast " + << (j + 1) << " of " << n_fcst_files << "\n" << sep << "\n"; + MultiVarData *mvdi = create_simple_objects(ModeDataType_MvMode_Fcst, j, + n_fcst_files, fcst_filenames[j], + fcstInput[j]); mvdFcst.push_back(mvdi); mvdi->print(); } // for j - for (j=0; j 0) { - mvdObs[0]->checkFileTypeConsistency(*mvdi, j); - } + << "\n" << sep << "\ncreating simple obs objects from obs " + << (j + 1) << " of " << n_obs_files << "\n" << sep << "\n"; + MultiVarData *mvdi = create_simple_objects(ModeDataType_MvMode_Obs, j, + n_obs_files, obs_filenames[j], + obsInput[j]); mvdObs.push_back(mvdi); mvdi->print(); } // for j @@ -207,72 +123,86 @@ int multivar_frontend(const StringArray & Argv) // now create forecast and obs superobjects - BoolPlane f_simple_result, o_simple_result; - ShapeData f_simple_sd, o_simple_sd, f_merge_sd_split, o_merge_sd_split; - - create_superobjects(n_fcst_files, mvdFcst, n_obs_files, mvdObs, - f_calc, o_calc, f_simple_result, o_simple_result, - f_simple_sd, o_simple_sd, - f_merge_sd_split, o_merge_sd_split); - + ModeSuperObject fsuper(true, n_fcst_files, do_clusters, mvdFcst, f_calc); + ModeSuperObject osuper(true, n_obs_files, do_clusters, mvdObs, o_calc); + // - // Filter the data to within the superobjects only and do statistics by invoking mode algorithm again - // on the masked data pairs + // Filter the data to within the superobjects only and do statistics by invoking mode + // algorithm again on the masked data pairs // + for (int k=0; k_nx; - int ny = mvdFcst[0]->_ny; - GrdFileType ftype = mvdFcst[0]->_type; - GrdFileType otype = mvdObs[0]->_type; - Grid grid = *(mvdFcst[0]->_grid); + if (config.fcst_multivar_compare_index.n() <= 0) { - // here run one more time using superobjects as input + process_superobjects(fsuper, osuper, *mvdFcst[0], *mvdObs[0]); + } + + // + // done + // + return (0); +} - bool has_union = f_calc.has_union() || o_calc.has_union(); +//////////////////////////////////////////////////////////////////////// - process_superobjects(f_simple_sd, o_simple_sd, f_merge_sd_split, o_merge_sd_split, - nx, ny, dir, ftype, otype, grid, has_union); +MultivarFrontEnd::~MultivarFrontEnd() +{ + if ( mode_exec ) { + delete mode_exec; mode_exec = 0; } - - // free up memory - for (j=0; jgrid(); + GrdFileType ft = f->file_type(); + + //? + f->set_shift_right(shift); + + // update config now that we know file type (this sets Fcst to index i) + DataPlane dp; + + if (type == ModeDataType_MvMode_Fcst) { + config.process_config_field(ft, other_t, type, index); + f->data_plane(*(config.Fcst->var_info), dp); + fcstInput.push_back(ModeInputData(name, dp, g)); + } else { + config.process_config_field(other_t, ft, type, index); + f->data_plane(*(config.Obs->var_info), dp); + obsInput.push_back(ModeInputData(name, dp, g)); + } + + delete f; +} + +//////////////////////////////////////////////////////////////////////// - ConcatString path; +void MultivarFrontEnd::create_verif_grid() +{ + mlog << Debug(2) << "\n creating the verification grid \n" << sep << "\n"; - path = replace_path(mode_default_config); + _init_exec(ModeExecutive::TRADITIONAL, "None", "None"); + mode_exec->setup_verification_grid(fcstInput[0], obsInput[0], config); + verification_grid = mode_exec->grid; + delete mode_exec; mode_exec = 0; +} - config.read_config(path.c_str(), filename.c_str()); +//////////////////////////////////////////////////////////////////////// - return; +MultiVarData *MultivarFrontEnd::create_simple_objects(ModeDataType dtype, int j, + int n_files, + const string &filename, + const ModeInputData &input) +{ + // + // create simple non merged objects + // + _simple_objects(ModeExecutive::MULTIVAR_SIMPLE, dtype, j, n_files, + filename, input); + MultiVarData *mvdi = mode_exec->get_multivar_data(dtype); + delete mode_exec; mode_exec = 0; + // + // create simple merged objects + // + _simple_objects(ModeExecutive::MULTIVAR_SIMPLE_MERGE, dtype, j, n_files, + filename, input); + mode_exec->add_multivar_merge_data(mvdi, dtype); + delete mode_exec; mode_exec = 0; + return mvdi; } +//////////////////////////////////////////////////////////////////////// + +void +MultivarFrontEnd::create_intensity_comparisons(int findex, int oindex, + const ModeSuperObject &fsuper, + const ModeSuperObject &osuper, + MultiVarData &mvdf, MultiVarData &mvdo, + const string &fcst_filename, + const string &obs_filename) +{ + + // mask the input data to be valid only inside the simple super objects + fsuper.mask_data_simple("Fcst", mvdf); + osuper.mask_data_simple("Obs", mvdo); + + mlog << Debug(1) << "Running mvmode intensity comparisions \n\n"; + + _init_exec(ModeExecutive::MULTIVAR_INTENSITY, fcst_filename, obs_filename); + mode_exec->init_multivar_intensities(config); + + ModeConfInfo & conf = mode_exec->engine.conf_info; + conf.set_field_index(findex, oindex); + + // for multivar intensities, explicity set the level and units using stored values + // from pass1 + conf.Fcst->var_info->set_level_name(mvdf._level.c_str()); + conf.Fcst->var_info->set_units(mvdf._units.c_str()); + if (fsuper._hasUnion && conf.Fcst->merge_flag == MergeType_Thresh) { + mlog << Warning << "\nModeFrontEnd::multivar_intensity_comparisons() -> " + << "Logic includes union '||' along with 'merge_flag=THRESH' " + << ". This can lead to bad results\n\n"; + } + conf.Obs->var_info->set_level_name(mvdo._level.c_str()); + conf.Obs->var_info->set_units(mvdo._units.c_str()); + if (osuper._hasUnion && conf.Obs->merge_flag == MergeType_Thresh) { + mlog << Warning << "\nModeFrontEnd::multivar_intensity_comparisons() -> " + << "Logic includes union '||' along with 'merge_flag=THRESH' " + << ". This can lead to bad results\n\n"; + } + + // + // set up data access using inputs + // + mode_exec->setup_multivar_fcst_obs_data_intensities(mvdf, mvdo); + + // + // run the mode algorithm for multivar intensities + // + _intensity_compare_mode_algorithm(mvdf, mvdo, fsuper, osuper); + + delete mode_exec; mode_exec = 0; +} //////////////////////////////////////////////////////////////////////// +void MultivarFrontEnd::process_superobjects(ModeSuperObject &fsuper, + ModeSuperObject &osuper, + const MultiVarData &mvdf, + const MultiVarData &mvdo) +{ + mlog << Debug(1) << "Running superobject mode \n\n"; + + // set the data to 0 inside superobjects and missing everywhere else + + fsuper.mask_data_super("FcstSimple", mvdf); + osuper.mask_data_super("ObsSimple", mvdo); + + _init_exec(ModeExecutive::MULTIVAR_SUPER, "None", "None"); + mode_exec->init_multivar_intensities(config); + + ModeConfInfo & conf = mode_exec->engine.conf_info; + if ((fsuper._hasUnion || osuper._hasUnion) && + (conf.Fcst->merge_flag == MergeType_Thresh || + conf.Obs->merge_flag == MergeType_Thresh)) { + mlog << Warning << "\nModeFrontEnd::run_super() -> " + << "Logic includes union '||' along with 'merge_flag=THRESH' " + << ". This can lead to bad results\n\n"; + } + + // + // set up data access using inputs + // + mode_exec->setup_multivar_fcst_obs_data_super(fsuper._simple_sd, osuper._simple_sd, + *mvdf._grid); + + // run the mode algorithm + _superobject_mode_algorithm(fsuper, osuper); + + delete mode_exec; mode_exec = 0; +} + +//////////////////////////////////////////////////////////////////////// -void process_command_line(const StringArray & argv) +void MultivarFrontEnd::_process_command_line(const StringArray & argv) { @@ -357,6 +436,7 @@ void process_command_line(const StringArray & argv) cline.add(set_outdir, "-outdir", 1); cline.add(set_logfile, "-log", 1); cline.add(set_verbosity, "-v", 1); + cline.add(set_compress, "-compress", 1); cline.parse(); @@ -364,11 +444,37 @@ void process_command_line(const StringArray & argv) // should be 3 arguments left // - fcst_fof = cline[0]; obs_fof = cline[1]; config_file = cline[2]; + return; + +} + +//////////////////////////////////////////////////////////////////////// + + +void MultivarFrontEnd::_read_config(const string & filename) + +{ + + ConcatString path; + + path = replace_path(mode_default_config); + + config.read_config(path.c_str(), filename.c_str()); + + // process the config except for the fields + config.process_config_except_fields(); + + // done once here, used for all data + // what is this, command line overrides config? look deeper.. remove from exec + // except traditional mode + if (compress_level >= 0) config.nc_info.set_compress_level(compress_level); + // from within mode_exec: + // engine.conf_info.nc_info.compress_level = engine.conf_info.get_compression_level(); + return; @@ -376,9 +482,7 @@ void process_command_line(const StringArray & argv) //////////////////////////////////////////////////////////////////////// -void multivar_consistency_checks(StringArray &fcst_filenames, StringArray &obs_filenames, - BoolCalc &f_calc, BoolCalc &o_calc, int &n_fcst_files, - int &n_obs_files) +void MultivarFrontEnd::_setup_inputs() { // // make sure the multivar logic programs are in the config file @@ -464,450 +568,191 @@ void multivar_consistency_checks(StringArray &fcst_filenames, StringArray &obs_f //////////////////////////////////////////////////////////////////////// -ConcatString set_multivar_dir() +void MultivarFrontEnd::_set_output_path() { - ConcatString dir; int status; - dir.clear(); + output_path.clear(); // no longer want numbered subdirectories - if ( outdir.length() > 0 ) dir << outdir; + if ( outdir.length() > 0 ) output_path << outdir; // // test to see of the output directory for this // mode runs exists, and if not, create it // - if ( ! directory_exists(dir.c_str()) ) { + if ( ! directory_exists(output_path.c_str()) ) { mlog << Debug(2) << program_name << ": creating output directory \"" - << dir << "\"\n\n"; + << output_path << "\"\n\n"; - status = _mkdir(dir.c_str()); + status = _mkdir(output_path.c_str()); if ( status < 0 ) { mlog << Error << "\nset_multivar_dir() ->" << " unable to create output directory \"" - << dir << "\"\n\n"; + << output_path << "\"\n\n"; exit ( 1 ); } } - return dir; } //////////////////////////////////////////////////////////////////////// -void create_verification_grid(const string &fcst_filename, const string &obs_filename, - const ConcatString &dir) +int MultivarFrontEnd::_mkdir(const char *dir) { - ConcatString command; - StringArray a, mode_argv; - - // - // build the command for running mode frontend - // - - mode_argv.clear(); - mode_argv.add(mode_path); - mode_argv.add(fcst_filename); - mode_argv.add(obs_filename); - mode_argv.add(config_file); - - command << cs_erase - << mode_path << ' ' - << fcst_filename << ' ' - << obs_filename << ' ' - << config_file; - - mode_argv.add("-v"); - char junk [256]; - snprintf(junk, sizeof(junk), "%d", mlog.verbosity_level()); - mode_argv.add(junk); - - mode_argv.add("-outdir"); - mode_argv.add(dir); - - command << " -v " << mlog.verbosity_level(); - command << " -outdir " << dir; - - // - // run the pass1 portions of mode, which creates simple objects - // - - mlog << Debug(3) << "Running mode command: \"" << command << "\"\n\n"; - ModeFrontEnd *frontend = new ModeFrontEnd; - verification_grid = frontend->create_verification_grid(mode_argv); - delete frontend; + char tmp[256]; + char *p = NULL; + size_t len; + snprintf(tmp, sizeof(tmp),"%s",dir); + len = strlen(tmp); + if (tmp[len - 1] == '/') + tmp[len - 1] = 0; + for (p = tmp + 1; *p; p++) + if (*p == '/') { + *p = 0; + string s = tmp; + if (s != ".") { + if (mkdir(tmp, dir_creation_mode) < 0) { + mlog << Error << "\n_mkdir() -> Error making " << tmp << "\n"; + return -1; + } + } + *p = '/'; + } + return (mkdir(tmp, dir_creation_mode)); } //////////////////////////////////////////////////////////////////////// -MultiVarData *create_simple_objects(ModeDataType dtype, int j, int n_files, - const string &filename, - const ConcatString &dir) +void MultivarFrontEnd::_simple_objects(ModeExecutive::Processing_t p, + ModeDataType dtype, + int j, int n_files, const string &filename, + const ModeInputData &input) { - ConcatString command; - StringArray a, mode_argv; - - // - // build the command for running mode frontend - // - - mode_argv.clear(); - mode_argv.add(mode_path); - mode_argv.add(filename); - mode_argv.add(config_file); - - command << cs_erase - << mode_path << ' ' - << filename << ' ' - << config_file; - - mode_argv.add("-v"); - char junk [256]; - snprintf(junk, sizeof(junk), "%d", mlog.verbosity_level()); - mode_argv.add(junk); - - mode_argv.add("-outdir"); - mode_argv.add(dir); - - command << " -v " << mlog.verbosity_level(); - command << " -outdir " << dir; - - // - // create the simple objects, forecast or obs, from this input data file - // - - mlog << Debug(3) << "Running mode command: \"" << command << "\"\n\n"; - ModeFrontEnd *frontend = new ModeFrontEnd; - int status = frontend->create_multivar_simple_objects(mode_argv, dtype, verification_grid, j, n_files); - MultiVarData *mvdi = frontend->get_multivar_data(dtype); - delete frontend; - - // - // create simple merge objects - // - - frontend = new ModeFrontEnd; - status = frontend->create_multivar_merge_objects(mode_argv, dtype, verification_grid, j, n_files); - - // add the merge results to the mvdi object - frontend->add_multivar_merge_data(mvdi, dtype); - delete frontend; - - return mvdi; -} + if (dtype == ModeDataType_MvMode_Fcst) { + _init_exec(p, filename, "None"); + mode_exec->init_multivar_simple(j, n_files, dtype, config); + mode_exec->setup_multivar_fcst_data(verification_grid, input); + } else { + _init_exec(p, "None", filename); + mode_exec->init_multivar_simple(j, n_files, dtype, config); + mode_exec->setup_multivar_obs_data(verification_grid, input); + } + + _simple_mode_algorithm(p); +} //////////////////////////////////////////////////////////////////////// -void create_superobjects(int n_fcst_files, const vector &mvdFcst, - int n_obs_files, const vector &mvdObs, - BoolCalc &f_calc, BoolCalc &o_calc, - BoolPlane &f_simple_result, BoolPlane &o_simple_result, - ShapeData &f_simple_sd, ShapeData &o_simple_sd, - ShapeData &f_merge_sd_split, ShapeData &o_merge_sd_split) +void MultivarFrontEnd::_init_exec(ModeExecutive::Processing_t p, + const string &ffile, + const string &ofile) { - // - // set the BoolPlane values using the mvd content - // + mlog << Debug(1) << "Running multivar front end for " << ModeExecutive::stype(p) << "\n"; - BoolPlane * f_simple_plane = new BoolPlane [n_fcst_files]; - BoolPlane * o_simple_plane = new BoolPlane [n_obs_files]; - BoolPlane * f_merge_plane = new BoolPlane [n_fcst_files]; - BoolPlane * o_merge_plane = new BoolPlane [n_obs_files]; + if ( mode_exec ) { delete mode_exec; mode_exec = 0; } - for (int j=0; jobjects_from_arrays(do_clusters, true, f_simple_plane[j]); - mvdFcst[j]->objects_from_arrays(do_clusters, false, f_merge_plane[j]); - } - - for (int j=0; jobjects_from_arrays(do_clusters, true, o_simple_plane[j]); - mvdObs[j]->objects_from_arrays(do_clusters, false, o_merge_plane[j]); - } + mode_exec = new ModeExecutive(); + // compress_level = -1; + mode_exec->fcst_file = ffile; + mode_exec->obs_file = ofile; - // - // combine the objects into super-objects - // - const int nx = f_simple_plane[0].nx(); - const int ny = f_simple_plane[0].ny(); - - BoolPlane f_merge_result, o_merge_result; - f_simple_result.set_size(nx, ny); - o_simple_result.set_size(nx, ny); - f_merge_result.set_size(nx, ny); - o_merge_result.set_size(nx, ny); - - combine_boolplanes("Fcst_Simple", f_simple_plane, n_fcst_files, f_calc, f_simple_result); - combine_boolplanes("Obs_Simple", o_simple_plane, n_obs_files, o_calc, o_simple_result); - combine_boolplanes("Fcst_Merge", f_merge_plane, n_fcst_files, f_calc, f_merge_result); - combine_boolplanes("Obs_Merge", o_merge_plane, n_obs_files, o_calc, o_merge_result); - - - // create ShapeData objects using something from mvd as a template - // (shape data has 1's or bad) - - f_simple_sd = ShapeData(*(mvdFcst[0]->_simple->_sd)); - for (int x=0; x_simple->_sd)); - for (int x=0; x_simple->_sd)); - for (int x=0; x_simple->_sd)); - for (int x=0; xmatch_config_file = config_file; // this is never used + mode_exec->out_dir = output_path; } - //////////////////////////////////////////////////////////////////////// void -create_intensity_comparisons(int findex, int oindex, const BoolPlane &f_result, - BoolPlane &o_result, const ConcatString &dir, - MultiVarData &mvdf, MultiVarData &mvdo, - bool has_union_f, bool has_union_o, - const string &fcst_filename, const string &obs_filename, - ShapeData &merge_f, ShapeData &merge_o) +MultivarFrontEnd::_superobject_mode_algorithm(const ModeSuperObject &fsuper, + const ModeSuperObject &osuper) { - - // mask the input data to be valid only inside the simple super objects - int nx = mvdf._nx; - int ny = mvdf._ny; - - mask_data("Fcst", nx, ny, f_result, mvdf._simple->_sd->data); - mask_data("Obs", nx, ny, o_result, mvdo._simple->_sd->data); - - // - // build the command for running mode frontend - // - StringArray mode_argv; - char junk [256]; - - mode_argv.clear(); - mode_argv.add(mode_path); - mode_argv.add(fcst_filename); - mode_argv.add(obs_filename); - mode_argv.add(config_file); - mode_argv.add("-v"); - snprintf(junk, sizeof(junk), "%d", mlog.verbosity_level()); - mode_argv.add(junk); - mode_argv.add("-outdir"); - mode_argv.add(dir); - - mlog << Debug(1) << "Running mvmode intensity comparisions \n\n"; - - ModeFrontEnd *frontend = new ModeFrontEnd; - int status = frontend->multivar_intensity_comparisons(mode_argv, mvdf, mvdo, has_union_f, - has_union_o, merge_f, merge_o, findex, - oindex); - delete frontend; + _mode_algorithm_init(); + mode_exec->clear_internal_r_index(); + mode_exec->do_conv_thresh_multivar_super(); + mode_exec->do_match_merge_multivar(fsuper._merge_sd_split, osuper._merge_sd_split, + ModeExecutive::MULTIVAR_SUPER); + mode_exec->process_output_multivar_super(); + mode_exec->clear_internal_r_index(); +#ifdef WITH_PYTHON + GP.finalize(); + #endif } //////////////////////////////////////////////////////////////////////// -void process_superobjects(ShapeData &f_result, ShapeData &o_result, - ShapeData &f_merge, ShapeData &o_merge, - int nx, int ny, const ConcatString &dir, - GrdFileType ftype, GrdFileType otype, const Grid &grid, - bool has_union) +void +MultivarFrontEnd::_intensity_compare_mode_algorithm(const MultiVarData &mvdf, + const MultiVarData &mvdo, + const ModeSuperObject &fsuper, + const ModeSuperObject &osuper) { - StringArray mode_argv; - char junk [256]; + _mode_algorithm_init(); + mode_exec->do_conv_thresh_multivar_intensity_compare(); + mode_exec->do_match_merge_multivar(fsuper._merge_sd_split, osuper._merge_sd_split, + ModeExecutive::MULTIVAR_INTENSITY); + // here replace raw data and min/max for plotting + mode_exec->process_output_multivar_intensity_compare(&mvdf, &mvdo); + mode_exec->clear_internal_r_index(); +#ifdef WITH_PYTHON + GP.finalize(); + #endif +} - // - // build the command for running mode frontend - // - mode_argv.clear(); - mode_argv.add(mode_path); - mode_argv.add(config_file); - mode_argv.add("-v"); - snprintf(junk, sizeof(junk), "%d", mlog.verbosity_level()); - mode_argv.add(junk); - mode_argv.add("-outdir"); - mode_argv.add(dir); - - mlog << Debug(1) << "Running superobject mode \n\n"; - - // set the data to 0 inside superobjects and missing everywhere else - mask_data_super("FcstSimple", nx, ny, f_result.data); - mask_data_super("ObsSimple", nx, ny, o_result.data); - - - ModeFrontEnd *frontend = new ModeFrontEnd; - int status = frontend->run_super(mode_argv, f_result, o_result, - f_merge, o_merge, ftype, otype, grid, has_union); - delete frontend; -} - //////////////////////////////////////////////////////////////////////// -void mask_data(const string &name, int nx, int ny, const BoolPlane &bp, DataPlane &data) +void MultivarFrontEnd::_simple_mode_algorithm(ModeExecutive::Processing_t p) { - - if (nx != data.nx() || ny != data.ny()) { - mlog << Error << "\nmask_data() -> " << name - << " :dimensions don't match " << nx << " " << ny - << " " << data.nx() << " " << data.ny() << "\n\n"; - - exit( 1 ); - } - - int nmasked=0, nkeep=0; + _mode_algorithm_init(); + mode_exec->clear_internal_r_index(); + mode_exec->do_conv_thresh_multivar_simple(p); + mode_exec->clear_internal_r_index(); - for (int x=0; xengine.conf_info; + if ( conf.quilt ) { + mlog << Error << "\nMultiVarFontend::mode_algorithm() -> " + << "quilting not yet implemented for multivar mode \n\n"; + exit ( 1 ); } - - mlog << Debug(1) << name << " superobject masking.." - << nkeep << " points of " - << nmasked + nkeep << " in superobjects\n"; -} + int NCT = conf.n_conv_threshs(); + int NCR = conf.n_conv_radii(); + if ( NCT != NCR ) { -//////////////////////////////////////////////////////////////////////// -void mask_data_super(const string &name, int nx, int ny, DataPlane &data) -{ + mlog << Error << "\nMultivarFrontEnd::_mode_algorithm_init() ->" + << "all convolution radius and threshold arrays must have the same number of elements\n\n"; - if (nx != data.nx() || ny != data.ny()) { - mlog << Error << "\nmask_data_super() -> " << name - << " :dimensions don't match " << nx << " " << ny - << " " << data.nx() << " " << data.ny() << "\n\n"; + exit ( 1 ); - exit( 1 ); } - int nmasked=0, nkeep=0; - - for (int x=0; x 1) { - for (int y=0; y" + << ": multiple convolution radii and thresholds not implemented in multivar mode\n\n"; - if(is_bad_data(data.get(x,y))) { - nmasked ++; - } else { - data.set(0.0, x, y); - nkeep ++; - } - } + exit ( 1 ); } - - mlog << Debug(1) << name << " superobject masking.." - << nkeep << " points of " - << nmasked + nkeep << " in superobjects\n"; } -//////////////////////////////////////////////////////////////////////// - -int _mkdir(const char *dir) -{ - char tmp[256]; - char *p = NULL; - size_t len; - - snprintf(tmp, sizeof(tmp),"%s",dir); - len = strlen(tmp); - if (tmp[len - 1] == '/') - tmp[len - 1] = 0; - for (p = tmp + 1; *p; p++) - if (*p == '/') { - *p = 0; - string s = tmp; - if (s != ".") { - if (mkdir(tmp, dir_creation_mode) < 0) { - mlog << Error << "\n_mkdir() -> Error making " << tmp << "\n"; - return -1; - } - } - *p = '/'; - } - - return (mkdir(tmp, dir_creation_mode)); -} -//////////////////////////////////////////////////////////////////////// -void _debug_shape_examine(string &name, const ShapeData &sd, int nx, int ny) -{ - vector values; - vector count; - for (int x=0; x::iterator vi; - vi = find(values.begin(), values.end(), v); - if (vi == values.end()) { - values.push_back(v); - count.push_back(1); - } else { - int ii = vi - values.begin(); - count[ii] = count[ii] + 1; - } - } - } - for (size_t i=0; i +#include "mode_conf_info.h" +#include "two_d_array.h" +#include "bool_calc.h" +#include "multivar_data.h" +#include "mode_superobject.h" +#include "mode_input_data.h" +#include "mode_exec.h" + +class MultivarFrontEnd { + +private: + + int n_fcst_files, n_obs_files; + StringArray fcst_filenames; + StringArray obs_filenames; + BoolCalc f_calc, o_calc ; + vector fcstInput, obsInput; + vector mvdFcst, mvdObs; + string fcst_fof; + string obs_fof; + + void _process_command_line(const StringArray &); + void _read_config(const string & filename); + void _setup_inputs(); + void _set_output_path(); + int _mkdir(const char *dir); + void _simple_objects(ModeExecutive::Processing_t p, ModeDataType dtype, + int j, int n_files, const string &filename, + const ModeInputData &input); + void _init_exec(ModeExecutive::Processing_t p, const string &ffile, const string &ofile); + void _superobject_mode_algorithm(const ModeSuperObject &fsuper, const ModeSuperObject &osuper); + void _intensity_compare_mode_algorithm(const MultiVarData &mvdf, const MultiVarData &mvdo, + const ModeSuperObject &fsuper, const ModeSuperObject &osuper); + void _simple_mode_algorithm(ModeExecutive::Processing_t p); + void _mode_algorithm_init() const; + +public: + + bool do_clusters; + string default_out_dir; + ModeConfInfo config; + ConcatString output_path; + string mode_path; + string config_file; + Grid verification_grid; + + MultivarFrontEnd(); + + ~MultivarFrontEnd(); + + + int run(const StringArray & Argv); + void init(const StringArray & Argv); + + static void set_outdir (const StringArray &); + static void set_logfile (const StringArray &); + static void set_verbosity (const StringArray &); + static void set_compress (const StringArray &); + + void read_input(const string &name, int index, ModeDataType type, + GrdFileType f_t, GrdFileType other_t, int shift); + + + void create_verif_grid(void); + + MultiVarData *create_simple_objects(ModeDataType dtype, int j, int n_files, + const string &filename, + const ModeInputData &input); + + void create_intensity_comparisons(int findex, int oindex, + const ModeSuperObject &fsuper, + const ModeSuperObject &osuper, + MultiVarData &mvdf, MultiVarData &mvdo, + const string &fcst_filename, + const string &obs_filename); + + void process_superobjects(ModeSuperObject &fsuper, + ModeSuperObject &osuper, + const MultiVarData &mvdf, + const MultiVarData &mvdo); + +}; + + +#endif /* __MULTIVAR_FRONT_END_H__ */ + + +///////////////////////////////////////////////////////////////////////// From c1532ddc223bf3f684a5a9ccfda2008d1f8adc5a Mon Sep 17 00:00:00 2001 From: Julie Prestopnik Date: Wed, 31 Jan 2024 15:11:18 -0700 Subject: [PATCH 18/21] Per #2772, added MET_PYTHON_EXE to various test cases and removed what would have been a duplicate after adding MET_PYTHON_EXE --- internal/test_unit/xml/unit_python.xml | 25 ++++++++++--------------- 1 file changed, 10 insertions(+), 15 deletions(-) diff --git a/internal/test_unit/xml/unit_python.xml b/internal/test_unit/xml/unit_python.xml index 051f709a62..f6e011dc0d 100644 --- a/internal/test_unit/xml/unit_python.xml +++ b/internal/test_unit/xml/unit_python.xml @@ -206,6 +206,7 @@ &MET_BIN;/point_stat + MET_PYTHON_EXE &MET_PYTHON_EXE; FCST_COMMAND &MET_BASE;/python/examples/read_ascii_numpy.py &DATA_DIR_PYTHON;/fcst.txt FCST OBS_COMMAND &MET_BASE;/python/examples/read_ascii_numpy.py &DATA_DIR_PYTHON;/obs.txt OBS @@ -480,21 +481,6 @@ - - &MET_BIN;/point2grid - \ - 'PYTHON_NUMPY=&MET_BASE;/python/examples/read_met_point_obs.py &OUTPUT_DIR;/pb2nc/ndas.20120409.t12z.prepbufr.tm00.nc' \ - G212 \ - &OUTPUT_DIR;/python/pb2nc_TMP.nc \ - -field 'name="TMP"; level="*"; valid_time="20120409_120000"; censor_thresh=[ <0 ]; censor_val=[0];' \ - -name TEMP \ - -v 1 - - - &OUTPUT_DIR;/python/pb2nc_TMP.nc - - - &MET_BIN;/point2grid @@ -535,6 +521,7 @@ &MET_BIN;/plot_point_obs + MET_PYTHON_EXE &MET_PYTHON_EXE; TO_GRID NONE \ @@ -561,6 +548,7 @@ > &OUTPUT_DIR;/python/ensemble_stat/input_file_list; \ &MET_BIN;/ensemble_stat + MET_PYTHON_EXE &MET_PYTHON_EXE; DESC NA OBS_ERROR_FLAG FALSE SKIP_CONST FALSE @@ -587,6 +575,7 @@ &MET_BIN;/point_stat + MET_PYTHON_EXE &MET_PYTHON_EXE; BEG_DS -1800 END_DS 1800 OUTPUT_PREFIX GRIB1_NAM_GDAS_WINDS @@ -605,6 +594,9 @@ + + MET_PYTHON_EXE &MET_PYTHON_EXE; + &MET_BIN;/plot_data_plane \ PYTHON_NUMPY \ @@ -619,6 +611,9 @@ + + MET_PYTHON_EXE &MET_PYTHON_EXE; + &MET_BIN;/pcp_combine \ -add PYTHON_NUMPY \ From 06498d7c7e7f11147195f73a06e547bb5f3114e2 Mon Sep 17 00:00:00 2001 From: John Halley Gotway Date: Wed, 31 Jan 2024 16:18:04 -0700 Subject: [PATCH 19/21] Per #2772, add documentation about 3 new environment variables. --- docs/Users_Guide/config_options.rst | 42 +++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/docs/Users_Guide/config_options.rst b/docs/Users_Guide/config_options.rst index f2dc2803ba..f59c8f9a87 100644 --- a/docs/Users_Guide/config_options.rst +++ b/docs/Users_Guide/config_options.rst @@ -494,6 +494,48 @@ Where code is running in a production context, it is worth being familiar with the binding / affinitization method on the particular system and building it into any relevant scripting. +.. _met_keep_temp_file: + +MET_KEEP_TEMP_FILE +------------------ + +The MET_KEEP_TEMP_FILE environment variable can be set to control the runtime +behavior of the MET tools. The MET tools write temporary files in several places +in the application and library code. By default, those temporary files are deleted +when they are no longer needed. However it can be useful for development, testing, +and debugging to keep them for further inspection. Setting this environment variable +to a value of :code:`yes` or :code:`true` instructs the MET tools to retain temporary +files instead of deleting them. + +Note that doing so may fill up the temporary directory. It is the responsiblity of +the user to monitor the temporary directory usage and remove temporary files that +are no longer needed. + +When running with this option, users are advised to refer to section +:numref:`config_tmp_dir` and write temporary files to a personal location rather than +the default shared :code:`/tmp` directory. + +.. _met_python_debug: + +MET_PYTHON_DEBUG +---------------- + +The MET_PYTHON_DEBUG environment variable can be set to turn one debugging log messages +related to Python embedding. By default, these log messages are disabled. It can be set +to a value of :code:`all` for all log messages, :code:`dataplane` for log messages when +reading gridded data, or :code:`point` for log messages when reading point data. + +.. _met_python_tmp_format: + +MET_PYTHON_TMP_FORMAT +--------------------- + +The MET_PYTHON_TMP_FORMAT environment variable defines whether temporary files for +Python embedding should be written as NetCDF files or using JSON and NumPy serialization. +By default, they are written as JSON (for attributes) and NumPy serialization (for data) +to avoid NetCDF library conflicts between MET and Python. Setting this environment +variable to :code:`netcdf` enables the use of temporary NetCDF files instead. + Settings Common to Multiple Tools ================================= From c0c3d69bf7d55eff20a6ef42af73feb1eb66d4ec Mon Sep 17 00:00:00 2001 From: John Halley Gotway Date: Wed, 31 Jan 2024 16:24:11 -0700 Subject: [PATCH 20/21] Per #2772, tweak the wording. --- docs/Users_Guide/config_options.rst | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/Users_Guide/config_options.rst b/docs/Users_Guide/config_options.rst index f59c8f9a87..e3bf7ccd6e 100644 --- a/docs/Users_Guide/config_options.rst +++ b/docs/Users_Guide/config_options.rst @@ -520,10 +520,11 @@ the default shared :code:`/tmp` directory. MET_PYTHON_DEBUG ---------------- -The MET_PYTHON_DEBUG environment variable can be set to turn one debugging log messages -related to Python embedding. By default, these log messages are disabled. It can be set -to a value of :code:`all` for all log messages, :code:`dataplane` for log messages when -reading gridded data, or :code:`point` for log messages when reading point data. +The MET_PYTHON_DEBUG environment variable can be set to enable debugging log messages +related to Python embedding. These log messages are disabled by default. The environment +variable cab be set to a value of :code:`all` for all log messages, :code:`dataplane` +for log messages when reading gridded data, or :code:`point` for log messages when +reading point data. .. _met_python_tmp_format: @@ -531,8 +532,8 @@ MET_PYTHON_TMP_FORMAT --------------------- The MET_PYTHON_TMP_FORMAT environment variable defines whether temporary files for -Python embedding should be written as NetCDF files or using JSON and NumPy serialization. -By default, they are written as JSON (for attributes) and NumPy serialization (for data) +Python embedding should be written as NetCDF files or using JSON/NumPy serialization. +By default, they are written using JSON for attributes and NumPy serialization for data to avoid NetCDF library conflicts between MET and Python. Setting this environment variable to :code:`netcdf` enables the use of temporary NetCDF files instead. From e3343f77297c71b6930d866aa2802c4cdf0c7817 Mon Sep 17 00:00:00 2001 From: Julie Prestopnik Date: Thu, 1 Feb 2024 09:29:15 -0700 Subject: [PATCH 21/21] Fixing typo --- docs/Users_Guide/config_options.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Users_Guide/config_options.rst b/docs/Users_Guide/config_options.rst index e3bf7ccd6e..5b533a1bb1 100644 --- a/docs/Users_Guide/config_options.rst +++ b/docs/Users_Guide/config_options.rst @@ -522,7 +522,7 @@ MET_PYTHON_DEBUG The MET_PYTHON_DEBUG environment variable can be set to enable debugging log messages related to Python embedding. These log messages are disabled by default. The environment -variable cab be set to a value of :code:`all` for all log messages, :code:`dataplane` +variable can be set to a value of :code:`all` for all log messages, :code:`dataplane` for log messages when reading gridded data, or :code:`point` for log messages when reading point data.