-
Notifications
You must be signed in to change notification settings - Fork 105
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add AvgPool1D layer #97
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove dataFormat and change the params in constructor
api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/core/layer/pooling/AvgPool1D.kt
Outdated
Show resolved
Hide resolved
*/ | ||
public class AvgPool1D( | ||
public val poolSize: Int = 2, | ||
public val strides: Int? = null, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we add here the default value?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since in pooling layers it's very common for strides to be the same as pool size value, the benefit of using null
as default value is that the user won't need to set it explicitly and we just default to pool size value instead. This is the same behavior in Keras and PyTorch.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Kotlin and Python users have a different experience with nullability, and Kotlin is trying to be a place without nulls.
As I commented in the MaxPool1D layer PR, please change it to the default value 1,2,1.
We need to keep control of the parameters that could be a hyper-parameter for AutoML tasks and show it to the user.
As a variant - to make a warning if stride and pool are not equal with reminder that it's a common practice
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. I understand your point about removing nullability and hidden dependencies. However, I don't get the part about losing control of the parameter during hyper-parameter tuning scenarios or architecture search. Anyways, since I am new to Kotlin I think you have a much better understanding of what would be the best decision here and I would follow it gladly :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I meant that we train the user to use only one parameter and leave the second null to change together. Perhaps, looking for a new architecture, a method a la NAS, something with this approach will be missed, at least I think so.
api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/core/layer/pooling/AvgPool1D.kt
Show resolved
Hide resolved
api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/core/layer/pooling/AvgPool1D.kt
Outdated
Show resolved
Hide resolved
api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/core/layer/pooling/AvgPool1D.kt
Outdated
Show resolved
Hide resolved
@zaleslaw I just merged it with master branch and updated it with recent changes. |
api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/core/layer/pooling/AvgPool1D.kt
Outdated
Show resolved
Hide resolved
api/src/main/kotlin/org/jetbrains/kotlinx/dl/api/core/layer/pooling/AvgPool1D.kt
Outdated
Show resolved
Hide resolved
@zaleslaw I changed the strides type and default value per your request. |
Resolves #61. This PR adds support for AvgPool1D layer.