Metric Accuracy doesn't work as previous version

[byte-sourcerer]

versions

pytorch 1.8.1
pytorch_lightning 1.2.10

reproduce

import torch
import pytorch_lightning as pl
from pytorch_lightning.metrics import Accuracy

acc = Accuracy()
preds = torch.rand(4,4)
target = torch.tensor([0, 1, 2, 3]).long()
print(acc(preds, target))

The code above produces errors like:

~/anaconda3/lib/python3.8/site-packages/pytorch_lightning/metrics/classification/helpers.py in _check_classification_inputs(preds, target, threshold, num_classes, is_multiclass, top_k)
    302     if case in (DataType.MULTICLASS, DataType.MULTIDIM_MULTICLASS) and preds.is_floating_point():
    303         if not torch.isclose(preds.sum(dim=1), torch.ones_like(preds.sum(dim=1))).all():
--> 304             raise ValueError("Probabilities in `preds` must sum up to 1 accross the `C` dimension.")
    305 
    306     # Check consistency with the `C` dimension in case of multi-class data

ValueError: Probabilities in `preds` must sum up to 1 accross the `C` dimension.

I believe🤔 that in the previous version, the above code works well, and I don't have to normalize C dimension of preds to 1.

Workaround

import torch
import pytorch_lightning as pl
from pytorch_lightning.metrics import Accuracy

acc = Accuracy()
preds = torch.rand(4,4)
target = torch.tensor([0, 1, 2, 3]).long()
print(acc(preds.argmax(dim=1), target))

[carmocca]

Transferring the issue to torchmetrics since we rely on their implementation

[SkafteNicki]

It is correct that we at some point in the past supported raw model input (logits) for various classification metrics.
We then started to only to support probability input when the metrics gained a few extra features (like top_k calculations).
That said, we have acknowledged that we should support logit input and have another issue open already #74 and already working on a PR #200.
I am going to close this, feel free to add additional comments in #74 .