Add .plot() to all modular metrics

[SkafteNicki]

🚀 Feature

We are in the process of adding native plotting to torchmetrics such that our users can easily create simple plots of metrics, which is especially useful for metrics such as confusion matrix and pr-curves but even single-scalar metrics such as accuracy it can be beneficial to quickly create a plot of computed values.

The first proof of concept PR has just been merged: #1328

We are inviting anyone interested in contributing to torchmetrics to help add this feature to all remaining metrics. The tasks are fairly simple and an example of how to implement the plot metric can be seen here: https://github.com/Lightning-AI/metrics/blob/d8caeeb28efc0afb86ae68cf66b06412749a4e46/src/torchmetrics/classification/accuracy.py#L110-L156

We would like the work divided into PRs tackling each domain, however if any are interested in only contributing to a few metrics within a particular domain that is also more than welcome.

• Aggregation ->Plot method for aggregation + refactor tests #1485 @SkafteNicki
• Audio ->Add plot() functionality to audio metrics #1434 @shhs29
• Classification -> Add plotting 13/n #1624 Add plotting 15/n #1638 @srishti-git1110
• Image ->Adding support for plot() in image metrics #1480 @venomouscyanide
• Nominal -> Add plot to nominal #1581 @SkafteNicki
• Multimodal -> Add plotting 16/n #1639 @SkafteNicki
• Detection -> Add plot to detection #1585 @SkafteNicki
• Regression -> Add plotting 9/n #1609 Add plotting 11/n #1621 @srishti-git1110
• Retrieval -> Add plotting 10/n #1610 Add plotting 12/n #1623 @ufukhurriyetoglu
• Text -> Add plotting 14/n #1631 @ufukhurriyetoglu
• Wrappers -> Add plotting 16/n #1639 @SkafteNicki

[ufukhurriyetoglu]

I would be happy to work on this issue

[srishti-git1110]

Interested in working on the Regression and Classification subcategory.

[Borda]

I would be happy to work on this issue

@ufukhurriyetoglu cool, which category you would like to take?

Interested in working on the Regression and Classification subcategory.

@srishti-git1110 great, noting it 🐰

[venomouscyanide]

I would love to work on any subtask as well.

[shhs29]

I am also interested in contributing to any of the subtasks.

[ufukhurriyetoglu]

@Borda text and retrieval, and maybe audio if needed at the time we finish the first two.

[Borda]

I am also interested in contributing to any of the subtasks.

@shhs29 cool, so how about audio?

[shhs29]

I am also interested in contributing to any of the subtasks.

@shhs29 cool, so how about audio?

@Borda That sounds good.

[venomouscyanide]

I would love to work on any subtask as well.

I was wondering if there was anything left for me to pickup as well?

[Borda]

I would love to work on any subtask as well.

I was wondering if there was anything left for me to pickup as well?

sure, there are still Image, Detection and Aggregation 🎁

[venomouscyanide]

I would love to work on any subtask as well.

I was wondering if there was anything left for me to pickup as well?

sure, there are still Image, Detection and Aggregation 🎁

I guess I'll go with Image.

[Borda]

hi, everyone how was your Xmass break? did you have a chance to look at your tasks? 🦦
cc: @shhs29, @srishti-git1110, @venomouscyanide, @ufukhurriyetoglu

[venomouscyanide]

@Borda Sorry for the lack of communication on this. I will have a draft PR ready latest by Sunday.

[shhs29]

@Borda I am also planning to raise a draft PR by this weekend. Sorry for the delay.

[srishti-git1110]

@Borda Hey, yes! Have been looking into it. This is also my first time contributing to an open source project so a bit overwhelmed at the moment, but I'm on it 💪. Please expect an update within a week from now.

[rschireman]

Hi @Borda, I can work on plotting Detection metrics, do you have any preferences on what should be included?

[ufukhurriyetoglu]

hi @Borda targeting a draft PR for text this weekend. And will also kickstart information retrieval in parallel.

[ufukhurriyetoglu]

hi @Borda targeting a draft PR for text this weekend. And will also kickstart information retrieval in parallel.

I kickstarted the text branch work locally, will be working on this and information retrieval daily and keep you posted.

[shhs29]

@Borda @SkafteNicki I have created a draft PR for adding plot to pesq metric. The link to the PR is: #1434. I am looking for some feedback before I move to the next set of metrics. I would appreciate it if you can point me to the contact person for this.

[venomouscyanide]

Just a quick update. I have started working on this locally for image. I will try to have a draft PR open soon. I am also getting help as reference from @shhs29 's #1434 to help me kick-start things.

[Borda]

Hi @shhs29, @srishti-git1110, @venomouscyanide, @ufukhurriyetoglu, how is it going? we would like to get this to the upcoming 0.12 release...
btw, if you need some assistance, just create a draft PR and ping us 🦦
in case you won't have time to make it, it is completely fine, just let us know, and we offer it to someone else from the community 🐿️

[shhs29]

Hi @shhs29, @srishti-git1110, @venomouscyanide, @ufukhurriyetoglu, how is it going? we would like to get this to the upcoming 0.12 release... btw, if you need some assistance, just create a draft PR and ping us 🦦 in case you won't have time to make it, it is completely fine, just let us know, and we offer it to someone else from the community 🐿️

Hi @Borda,
I have finished adding plot to one of the audio metrics. The CI workflow is however failing and I would like some feedback. The link to the PR is: #1434. I have pinged you regarding the issue on Slack as well. I would appreciate it if you could provide feedback on my current approach.

[venomouscyanide]

@Borda I will definitely have a draft PR open by the weekend. If not, I will resign from this activity to avoid further delays for your team 😄

[alexkrz]

Hi @Borda @SkafteNicki @srishti-git1110: I feel like the classification category is a rather large task. Maybe we can divide it into subcategories? I would be happy to draft a PR for plot support for AUROC and ROC metrics.

[srishti-git1110]

Hi @alexkrz. Great! I'm working on other classification and regression metrics - you can take up AUROC and ROC.