Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

R2 score metric #1274

Merged
merged 27 commits into from
Sep 27, 2021
Merged

R2 score metric #1274

merged 27 commits into from
Sep 27, 2021

Conversation

asteyo
Copy link
Contributor

@asteyo asteyo commented Aug 6, 2021 •
edited
Loading

Before submitting (checklist)

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contribution guide?
  • Did you check the code style? catalyst-make-codestyle && catalyst-check-codestyle (pip install -U catalyst-codestyle).
  • Did you make sure to update the docs? We use Google format for all the methods and classes.
  • Did you check the docs with make check-docs?
  • Did you write any new necessary tests?
  • Did you check that your code passes the unit tests pytest . ?
  • Did you add your new functionality to the docs?
  • Did you update the CHANGELOG?
  • Did you run colab minimal CI/CD with latest and minimal requirements?

Description

Related Issue

Type of Change

  • Examples / docs / tutorials / contributors update
  • Bug fix (non-breaking change which fixes an issue)
  • Improvement (non-breaking change which improves an existing feature)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

FAQ

Please review the FAQ before submitting an issue:

@pep8speaks
Copy link

pep8speaks commented Aug 6, 2021 •
edited
Loading

Hello @asteyo! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2021-09-27 04:34:57 UTC

Scitator
Scitator reviewed Aug 9, 2021
View reviewed changes
Copy link
Member

@Scitator Scitator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nevertheless, we need to add a few things:

  1. correct code style with catalyst-make-codestyle
  2. tests :)
  3. Metric API extension?
  4. Callback API extension?

Podidiving
Podidiving reviewed Aug 27, 2021
View reviewed changes
catalyst/metrics/functional/_r2_score.py Outdated
torch.square(targets.float() - torch.mean(targets.float()))
).view(-1)
residual_sum_of_squares = torch.sum(torch.square(targets.float() - outputs.float())).view(-1)
output = 1 - residual_sum_of_squares / total_sum_of_squares
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe, you should do something like max(total_sum_of_squares, eps) to avoid zero division

catalyst/metrics/functional/_r2_score.py Outdated
import torch


def r2_score(outputs: torch.Tensor, targets: torch.Tensor,) -> Sequence[torch.Tensor]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mb it should be done like sklearn implementation https://scikit-learn.org/stable/modules/generated/sklearn.metrics.r2_score.html ?
I mean, it seems cool to have possibility to calculate r2 score of tensors with shape [bs, num_outputs] right in torch
plus, sample_weight might be useful

Scitator
Scitator previously requested changes Sep 13, 2021
View reviewed changes
Copy link
Member

@Scitator Scitator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw, could you please add all of them to the required __init__s?

tests/catalyst/metrics/functional/test_r2_squared.py Outdated
"""
y_true = torch.tensor([2.5, 0.0, 2, 8])
y_pred = torch.tensor([3, -0.5, 2, 7])
val = r2_squared(y_true, y_pred)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you have r2_squared(outputs: torch.Tensor, targets: torch.Tensor), rather than targets, outputs

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops

catalyst/metrics/_r2_squared.py Show resolved Hide resolved
@mergify mergify bot dismissed Scitator’s stale review September 20, 2021 08:42

Pull request has been modified.

Scitator
Scitator previously approved these changes Sep 20, 2021
View reviewed changes
@mergify mergify bot dismissed Scitator’s stale review September 21, 2021 21:16

Pull request has been modified.

@Scitator Scitator merged commit 4c8e685 into catalyst-team:master Sep 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Podidiving Podidiving Podidiving left review comments

@Scitator Scitator Scitator left review comments

@bagxi bagxi Awaiting requested review from bagxi bagxi is a code owner

@ditwoo ditwoo Awaiting requested review from ditwoo

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@asteyo @pep8speaks @Scitator @Podidiving