the normalized (bias-corrected) RMSE in Taylor Diagram

[Ho-ChunHuang-NOAA]

I try to find out how the the normalized (bias-corrected) RMSE calculated from RMSE. I can not find the answer from the
Metviewer users guide https://metviewer.readthedocs.io/en/main_v4.1/Users_Guide/taylordiag.html.

The example of Taylor diagram for day 2 fcst is the fcst hours 25-48 in RMSE mean versus lead time. Using red dot (line) as an example, how does the day 2 RMSE (peak around 10-11 near fcst hour 36 ) became the red dot near 1.5 RMSE curve in Taylor's diagrams?

[CPKalb]

Hi:

The points on the Taylor Diagram plot are not generated using RMSE. Rather, OSTDEV, FSTDEV, and PR_CORR (observation and forecast standard deviation and pearson correlation) are used to generate points on the plot. There are arcs that emanate from the bottom axis that represent the RMSE. I believe @bikegeek did some of the work creating the Taylor diagrams so maybe she has additional information).

Christina

[Ho-ChunHuang-NOAA]

Hi, Christina: Thanks for the reply. I have a hard time believing normalized RMSE is not used, in addition to OSTDEV, FSTDEV, and PR_CORR, to determine the location of a point on the taylor diagram. One should be able to get the relative RMSE magnitude difference from the diagram to determine the performance of two models (dots, points). Assuming OSTDEV, FSTDEV, and PR_CORR are the same between two models. Ho-Chun Huang, Ph.D. Physical Scientist III, Contractor with Lynker in Support of NOAA/NWS/NCEP/EMC, U.S. Department of Commerce 5830 University Research Ct., Rm. 2792 College Park, MD 20740 ***@***.*** ***@***.***> 301-683-3958

…

On Fri, Nov 18, 2022 at 2:06 PM Christina Kalb ***@***.***> wrote: Hi: The points on the Taylor Diagram plot are not generated using RMSE. Rather, OSTDEV, FSTDEV, and PR_CORR (observation and forecast standard deviation and pearson correlation) are used to generate points on the plot. There are arcs that emanate from the bottom axis that represent the RMSE. I believe @bikegeek <https://github.com/bikegeek> did some of the work creating the Taylor diagrams so maybe she has additional information). Christina — Reply to this email directly, view it on GitHub <#1949 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALPHE3CUF4MYOKAO6J5MZ4LWI7HT5ANCNFSM6AAAAAASCVKNBM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bikegeek]

Hi @Ho-ChunHuang-NOAA ,

In all the implementations of the Taylor diagram I have seen, all of them use the stdevs and correlation values to plot. It was my understanding that the location of the point relative to the RMSE curve was where you determined the RMSE value. Here is an example of an R implementation of the Taylor diagram, which was originally used by METviewer:

https://rdrr.io/cran/plotrix/src/R/taylor.diagram.R

and here is the what the Python version in METplotpy is based on:
https://gist.github.com/ycopin/3342888

(which is what METviewer is currently using)

If you know any implementation that directly uses the RMSE, we'd be very interested :)

Regards,
Minna

[Ho-ChunHuang-NOAA]

Hi, Thanks. Can you tell me the relationship between normalized RMSE determined from Taylor diagrams (by STDEV and CORR )and non-normalized RMSE. Ho-Chun Huang, Ph.D. Physical Scientist III, Contractor with Lynker in Support of NOAA/NWS/NCEP/EMC, U.S. Department of Commerce 5830 University Research Ct., Rm. 2792 College Park, MD 20740 ***@***.*** ***@***.***> 301-683-3958

…

On Fri, Nov 18, 2022 at 4:46 PM bikegeek ***@***.***> wrote: Hi @Ho-ChunHuang-NOAA <https://github.com/Ho-ChunHuang-NOAA> , In all the implementations of the Taylor diagram I have seen, all of them use the stdevs and correlation values to plot. It was my understanding that the location of the point relative to the RMSE curve was where you determined the RMSE value. Here is an example of an R implementation of the Taylor diagram, which was originally used by METviewer: https://rdrr.io/cran/plotrix/src/R/taylor.diagram.R and here is the what the Python version in METplotpy is based on: https://gist.github.com/ycopin/3342888 (which is what METviewer is currently using) If you know any implementation that directly uses the RMSE, we'd be very interested :) Regards, Minna — Reply to this email directly, view it on GitHub <#1949 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALPHE3FLIYA72FXUZLP3BLLWI72LJANCNFSM6AAAAAASCVKNBM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[bikegeek]

Hi @Ho-ChunHuang-NOAA ,
In the code, we normalize the fstdev by dividing it by the ostdev so we can use 1.0 for the reference standard deviation and 0 for the minimum std deviation. Perhaps I misunderstood your question...

[Ho-ChunHuang-NOAA]

Let me asked different question, Is normalized RMSE derived from a function of (ostdev, fcstdev, correlation)? If yes, does METPlus have the equation ? I really do not know how to ask this question properly. If you have no idea of the question asked above, please close this ticket, I will try to find the answer from somewhere else. Ho-Chun Huang, Ph.D. Physical Scientist III, Contractor with Lynker in Support of NOAA/NWS/NCEP/EMC, U.S. Department of Commerce 5830 University Research Ct., Rm. 2792 College Park, MD 20740 ***@***.*** ***@***.***> 301-683-3958

…

On Fri, Nov 18, 2022 at 5:34 PM bikegeek ***@***.***> wrote: Hi @Ho-ChunHuang-NOAA <https://github.com/Ho-ChunHuang-NOAA> , In the code, we normalize the fstdev by dividing it by the ostdev so we can use 1.0 for the reference standard deviation and 0 for the minimum std deviation. Perhaps I misunderstood your question... — Reply to this email directly, view it on GitHub <#1949 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALPHE3AA3TGNIXEL2ARCSLLWJAAAZANCNFSM6AAAAAASCVKNBM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[bikegeek]

Hi @Ho-ChunHuang-NOAA,

Thanks for reframing the question. There isn't an equation in METplus for directly calculating the RMSE from the stdevs and correlation.

Regards,
Minna

[CPKalb]

@Ho-ChunHuang-NOAA, I believe there is an equation that relates the forecast and observed standard deviation to the normalized RMSE. However, it is not needed to create the graphic and therefore not in METplus.

Christina

[Ho-ChunHuang-NOAA]

Thanks, please close the ticket. Ho-Chun Huang, Ph.D. Physical Scientist III, Contractor with Lynker in Support of NOAA/NWS/NCEP/EMC, U.S. Department of Commerce 5830 University Research Ct., Rm. 2792 College Park, MD 20740 ***@***.*** ***@***.***> 301-683-3958

…

On Mon, Nov 21, 2022 at 4:17 PM bikegeek ***@***.***> wrote: Hi @Ho-ChunHuang-NOAA <https://github.com/Ho-ChunHuang-NOAA>, Thanks for reframing the question. There isn't an equation in METplus for directly calculating the RMSE from the stdevs and correlation. Regards, Minna — Reply to this email directly, view it on GitHub <#1949 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALPHE3FBRU5MUZB4QM7JBHLWJPRFZANCNFSM6AAAAAASCVKNBM> . You are receiving this because you were mentioned.Message ID: ***@***.***>