Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Add] Add EvalHook inherited from MMCV EvalHook #90

Merged
merged 40 commits into from
Feb 11, 2022
Merged

[Add] Add EvalHook inherited from MMCV EvalHook #90

merged 40 commits into from
Feb 11, 2022

Conversation

ttxskk
Copy link
Collaborator

@ttxskk ttxskk commented Jan 24, 2022

  • Add a eval_hook script that reimplements EvalHook and DistEvalHook based on mmcv.runner.EvalHook.
  • Seperate joint_error into mpjpe and p-mpjpe.
  • Add pve, 3dpck, p-3dpck, 3dauc, p-3dauc metrics.
  • Refactor evaluate function in human_image_dataset.py and human_hybrik_dataset.py.
  • Add _parse_result function in human_image_dataset.py and human_hybrik_dataset.py for parsing prediction results and GT.

@ttxskk ttxskk requested review from yl-1993 and caizhongang January 24, 2022 07:44
@codecov
Copy link

codecov bot commented Jan 24, 2022

Codecov Report

Merging #90 (7cb5793) into main (cdd395a) will increase coverage by 0.36%.
The diff coverage is 84.75%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main      #90      +/-   ##
==========================================
+ Coverage   84.39%   84.76%   +0.36%     
==========================================
  Files         163      164       +1     
  Lines       12348    12554     +206     
==========================================
+ Hits        10421    10641     +220     
+ Misses       1927     1913      -14     
Flag Coverage Δ
unittests 84.76% <84.75%> (+0.36%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
mmhuman3d/models/__init__.py 100.00% <ø> (ø)
mmhuman3d/models/architectures/hybrik.py 15.06% <0.00%> (-0.43%) ⬇️
mmhuman3d/models/heads/hybrik_head.py 38.67% <0.00%> (ø)
mmhuman3d/models/body_models/smpl.py 58.63% <50.00%> (+0.90%) ⬆️
mmhuman3d/core/evaluation/eval_utils.py 74.24% <74.24%> (ø)
mmhuman3d/data/datasets/human_hybrik_dataset.py 80.83% <76.03%> (-6.00%) ⬇️
mmhuman3d/data/datasets/human_image_dataset.py 91.53% <93.83%> (+12.66%) ⬆️
mmhuman3d/apis/test.py 12.74% <100.00%> (+0.45%) ⬆️
mmhuman3d/apis/train.py 20.63% <100.00%> (+0.32%) ⬆️
mmhuman3d/core/evaluation/__init__.py 100.00% <100.00%> (ø)
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cdd395a...7cb5793. Read the comment docs.

@ttxskk ttxskk changed the title Add a eval hook [Add] Add EvalHook inherited from MMCV EvalHook Jan 27, 2022
Copy link
Collaborator

@caizhongang caizhongang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!

@yl-1993 yl-1993 merged commit 26f821b into main Feb 11, 2022
@ttxskk ttxskk deleted the upgrade_eval branch February 11, 2022 16:05
ttxskk added a commit that referenced this pull request Jun 17, 2022
* upgrade eval

* fix lint

* fix lint

* fix lint

* add a unit test: test eval hook

* add unit test

* fix unit test

* fix unit test: remove the requirement for cuda

* use kwargs to receive EvalHook args

* remove useless comments

* create the folder if it does not exist

* add new metric

* fix some bugs

* fix unit test

* remove joint_error metric

* fix unit test

* fix pck thresholds

* fix import error

* fix import error

* remove unused paramter

* add more unit test

* add unit test

* rename p-mpjpe to pa-mpjpe

* fix unit test

* remove `mpjpe` in `__all__`

* fix comments

* add more unit tests

* fix

* rename

* fix docsting

* fix typo

* update `getting_started.md`

* fix docstring

* add evaluation config

* fix unit test

* use mmhuman3d greater/less key
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants