MultiTask models are not handling evaluation metrics correctly

[britojr]

Describe the bug(问题描述)

When dealing with MultiTask, i.e. when there is more than one target variable, the evaluation metrics are not being correctly applied. As we can see on this line the evaluation metric is applied as if y was a single column.

To Reproduce(复现步骤)
Steps to reproduce the behavior:

1. Copy the data and example code on the documentation page for MultiTask model MMOE
2. Execute the code as is and see the warning: UserWarning: The y_pred values do not sum to one. Make sure to pass probabilities.
3. The warning indicates that that the columns in y_pred are being interpreted as a single probability distribution instead of two distinct distributions for each of the binary tasks.

Operating environment(运行环境):

• python version: 3.12.4
• torch version: 2.5.1
• deepctr-torch version: 0.2.9

[iristengtrx]

It seems the warning comes from package sklearn. I execute the code, but didn't see the same warning.

my operating environment:

• python 3.6.13
• PyTorch 1.7.1
• deepctr-torch 0.2.9
• scikit-learn 0.24.2

[britojr]

The warning does come from sklearn, and it was how I found out the bug.
It could be that for certain cases and values the warning does not trigger, but that does not change the fact that the code is not handling the metrics computation for multi task properly.

For reference, the code that I referred to

                        if verbose > 0:
                            for name, metric_fun in self.metrics.items():
                                if name not in train_result:
                                    train_result[name] = []
                                train_result[name].append(metric_fun(
                                    y.cpu().data.numpy(), y_pred.cpu().data.numpy().astype("float64")))

When doing Multi Task y is not a 1D array, it is actually a (n_samples, n_tasks) matrix. But the code is not handling that, it is passing y and y_pred directly to a sklearn metric function (metric_fun, check here to see the possible values).
You can check the sklearn documentation to see how those metric functions interpret the input.