Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[MXNET-235] add axis support and gradient for L2norm #9740

Merged
merged 16 commits into from
Apr 3, 2018
Merged

[MXNET-235] add axis support and gradient for L2norm #9740

merged 16 commits into from
Apr 3, 2018

Conversation

zheng-da
Copy link
Contributor

@zheng-da zheng-da commented Feb 8, 2018

Description

This PR adds axis support and compute gradient for L2norm.

Checklist

Essentials

  • Passed code style checking (make lint)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

  • add axis support, which adds new parameters to norm.
  • compute gradient of L2 norm.

@piiswrong
Copy link
Contributor

also need ord

@zheng-da
Copy link
Contributor Author

@piiswrong how many ord should we support?

We can support ord=1, inf and None.
I guess ord=0 isn't differentiable.
What is the definition of ord=2 for an NDArray? When the array is a vector, it's easy. When the array is a matrix, it's the largest singular value. What is the gradient? What is the definition of L2 for an array with more than 2 dimensions?

@CodingCat
Copy link
Contributor

Hi, the community has passed to vote about associating the code changes with JIRA (https://lists.apache.org/thread.html/ab22cf0e35f1bce2c3bf3bec2bc5b85a9583a3fe7fd56ba1bbade55f@%3Cdev.mxnet.apache.org%3E)

We have updated the guidelines for contributors in https://cwiki.apache.org/confluence/display/MXNET/Development+Process, please ensure that you have created a JIRA at https://issues.apache.org/jira/projects/MXNET/issues/ to describe your work in this pull request and include the JIRA title in your PR as [MXNET-xxxx] your title where MXNET-xxxx is the JIRA id

Thanks!

@piiswrong
Copy link
Contributor

@zheng-da ping
only supporting ord=2 is fine for now. We just need to keep the placeholder

@zheng-da
Copy link
Contributor Author

I almost forgot this one. I'll look into it tomorrow.

@zheng-da zheng-da force-pushed the l2norm branch 2 times, most recently from e0554ab to 32a7f06 Compare March 27, 2018 02:36
sxjscience
sxjscience reviewed Mar 27, 2018
View reviewed changes
src/operator/tensor/broadcast_reduce_op.h Outdated
MSHADOW_XINLINE static DType Map(DType a) {
return a * a;
}
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think square is supported in mshadow_op::square.

cjolivier01
cjolivier01 suggested changes Mar 27, 2018
View reviewed changes
Copy link
Member

@cjolivier01 cjolivier01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needs a JIRA

@zheng-da zheng-da changed the title add axis support and gradient for L2norm [MXNET-235] add axis support and gradient for L2norm Mar 27, 2018
@zheng-da
Copy link
Contributor Author

@cjolivier01 I created a JIRA issue

cjolivier01
cjolivier01 approved these changes Mar 28, 2018
View reviewed changes
Copy link
Member

@cjolivier01 cjolivier01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@piiswrong piiswrong merged commit 62a615d into apache:master Apr 3, 2018
@zheng-da zheng-da deleted the l2norm branch April 5, 2018 22:38
@eric-haibin-lin eric-haibin-lin mentioned this pull request May 12, 2018
Merged
7 tasks
rahul003 pushed a commit to rahul003/mxnet that referenced this pull request Jun 4, 2018
@zheng-da @piiswrong
[MXNET-235] add axis support and gradient for L2norm (apache#9740)
2b92fde 
* Add axis support for norm.

* Add tests on norm in diff cases.

* Fix a compilation error.

* Fix a lint complain.

* Fix a compilation error.

* Implement norm backward.

* Fix a bug on computing norm on entire sparse arrays.

* Fix tests.

* Fix a bug in L2Norm on dense arrays.

* Fix a lint error.

* Remove square.

* Add ord in Norm.

* Fix a compile error.

* Use NormParam.

* Fix tests.

* Update broadcast_reduce_op_value.cc
zheng-da added a commit to zheng-da/incubator-mxnet that referenced this pull request Jun 28, 2018
@zheng-da
[MXNET-235] add axis support and gradient for L2norm (apache#9740)
3cc0b16 
* Add axis support for norm.

* Add tests on norm in diff cases.

* Fix a compilation error.

* Fix a lint complain.

* Fix a compilation error.

* Implement norm backward.

* Fix a bug on computing norm on entire sparse arrays.

* Fix tests.

* Fix a bug in L2Norm on dense arrays.

* Fix a lint error.

* Remove square.

* Add ord in Norm.

* Fix a compile error.

* Use NormParam.

* Fix tests.

* Update broadcast_reduce_op_value.cc
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@sxjscience sxjscience sxjscience left review comments

@cjolivier01 cjolivier01 cjolivier01 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@zheng-da @piiswrong @CodingCat @sxjscience @cjolivier01