Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bugfix: Fix the TC-Stat RIRW runtime error when computing CTS statistics from an empty contingency table #2542

Closed
8 of 22 tasks
JohnHalleyGotway opened this issue May 15, 2023 · 1 comment · Fixed by #2543
Closed
8 of 22 tasks
Assignees
Labels
MET: Tropical Cyclone Tools priority: medium Medium Priority requestor: NOAA/EMC NOAA Environmental Modeling Center type: bug Fix something that is not working
Milestone

Comments

@JohnHalleyGotway
Copy link
Collaborator

JohnHalleyGotway commented May 15, 2023

Describe the Problem

This issue was identified based on feedback from the NOAA METplus Telecon on 5/15/2023 from Olivia Ostwald. TC-Stat produces a runtime error when attempting to compute CTS statistics from an empty contingency table. A short-term solution of only requesting CTC output with -out_line_type CTC will work for the time being. Recommend fixing this in the develop branch for MET version 11.1.0 but not providing a patch in the main_v11.0 branch.

Expected Behavior

Computing CTS stats for an empty contingency table produces a runtime error from ContingencyTable::gheidke_ec(double). Recommend removing the empty table check and just return bad data instead. The error goes away if you only request -out_line_type CTC output, and the contingency table counts for that empty table are indeed written:

COL_NAME: ATIME  BTIME  AEXACT BEXACT ATHRESH BTHRESH WINDOW_BEG WINDOW_END AMODEL TOTAL FY_OY FY_ON FN_OY FN_ON EC_VALUE
RIRW_CTC: 240000 240000 true   true   >=30.0  >=30.0  000000     000000     BCLP       0     0     0     0     0      0.5

Testing this same situation using MET version 10.1.0, prior to the addition of the HSS_EC column, an empty contingency table from the RIRW job just produced a CTC line with all stats computed as NA. Recommend fixing the code to do the same... do not error out because of an empty table.

Environment

Can be reproduced on seneca.

To Reproduce

Run the following on seneca:

cd /d1/projects/MET/MET_regression/develop/NB20230515/MET-develop/test_output
# This runs fine
 ../bin/tc_stat -lookin tc_pairs/alal2010.tcst -job rirw -out_line_type CTC,CTS
# Adding -by amodel errors out
 ../bin/tc_stat -lookin tc_pairs/alal2010.tcst -job rirw -out_line_type CTC,CTS -by amodel
ERROR  : 
ERROR  : ContingencyTable::gheidke_ec(double) -> table empty!
ERROR  : 

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Select component(s)
  • Select priority
  • Select requestor(s)

Projects and Milestone

  • Select Organization level Project for support of the current coordinated release
  • Select Repository level Project for development toward the next official release or add alert: NEED CYCLE ASSIGNMENT label
  • Select Milestone as the next bugfix version

Define Related Issue(s)

Consider the impact to the other METplus components.

Bugfix Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of main_<Version>.
    Branch name: bugfix_<Issue Number>_main_<Version>_<Description>
  • Fix the bug and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into main_<Version>.
    Pull request: bugfix <Issue Number> main_<Version> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Organization level software support Project for the current coordinated release
    Select: Milestone as the next bugfix version
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Complete the steps above to fix the bug on the develop branch.
    Branch name: bugfix_<Issue Number>_develop_<Description>
    Pull request: bugfix <Issue Number> develop <Description>
    Select: Reviewer(s) and Development issue
    Select: Repository level development cycle Project for the next official release
    Select: Milestone as the next official version
  • Close this issue.
@JohnHalleyGotway JohnHalleyGotway added type: bug Fix something that is not working requestor: NOAA/EMC NOAA Environmental Modeling Center priority: medium Medium Priority MET: Tropical Cyclone Tools labels May 15, 2023
@JohnHalleyGotway JohnHalleyGotway added this to the MET 11.1.0 milestone May 15, 2023
@JohnHalleyGotway JohnHalleyGotway self-assigned this May 15, 2023
@JohnHalleyGotway JohnHalleyGotway changed the title Bugfix: Fix runtime TC-Stat RIRW runtime error when computing CTS statistics Bugfix: Fix TC-Stat RIRW runtime error when computing CTS statistics from an empty contingency table May 15, 2023
@JohnHalleyGotway
Copy link
Collaborator Author

Note that for the HSS statistic in the CTS line type, MET calls TTContingencyTable::hss() which does NOT check for an empty contingency table. An empty table results in bad data to avoid dividing by 0.

For the HSS_EC statistic in the CTS line type, MET calls ContingencyTable::gheidke_ec(double ec_value) which does current error out for empty contingency tables. The same function is used for both the CTS and MCTS output line types.

So that explains why this issue didn't show up for the HSS statistic but does show up for HSS_EC.

JohnHalleyGotway added a commit that referenced this issue May 15, 2023
…c(), gkuiper(), and gerrity() to return bad data for empty contingency tables rather than erroring out.
JohnHalleyGotway added a commit that referenced this issue May 15, 2023
…2425 by adding the  option. That produces output for both the AHWI and BCLP model names. The BCLP contingency table is empty which triggers this runtime error. Tweaking this newly added tests and confirming that it now runs without error demonstrate the fix.
@JohnHalleyGotway JohnHalleyGotway moved this from 📋 Backlog to 🏗 In progress in MET-11.1.0 Development May 15, 2023
@JohnHalleyGotway JohnHalleyGotway linked a pull request May 15, 2023 that will close this issue
15 tasks
sethlinden pushed a commit that referenced this issue May 15, 2023
* Per #2542, in the ContingencyTable class, update gheidke(), gheidke_ec(), gkuiper(), and gerrity() to return bad data for empty contingency tables rather than erroring out.

* Per #2542, modify TC-Stat config file job number 8 that was added for #2425 by adding the  option. That produces output for both the AHWI and BCLP model names. The BCLP contingency table is empty which triggers this runtime error. Tweaking this newly added tests and confirming that it now runs without error demonstrate the fix.
@JohnHalleyGotway JohnHalleyGotway moved this from 🏗 In progress to ✅ Done in MET-11.1.0 Development May 15, 2023
@JohnHalleyGotway JohnHalleyGotway changed the title Bugfix: Fix TC-Stat RIRW runtime error when computing CTS statistics from an empty contingency table Bugfix: Fix the TC-Stat RIRW runtime error when computing CTS statistics from an empty contingency table Jun 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
MET: Tropical Cyclone Tools priority: medium Medium Priority requestor: NOAA/EMC NOAA Environmental Modeling Center type: bug Fix something that is not working
Projects
No open projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant