Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SimpleChannelPool "POOL_KEY" attribute name is easy to get conflict from user code #9542

Closed
nizarm opened this issue Sep 5, 2019 · 5 comments
Milestone

Comments

@nizarm
Copy link
Contributor

nizarm commented Sep 5, 2019

It is noticed that SimpleChannelPool's POOL_KEY attribute name "channelPool" is easy to get conflict with user code and throws an exception 'channelPool' is already in use. Being a generic framework - it would be great if we can name the attribute something unique - may be use UUID for the name since the name is not required later?

Create a channel attribute in user code with name 'channelPool' and when SimpleChannelPool is instantiated - 'channelPool' is already in use exception is thrown. This issue surfaced when we utilized Azure SDK.

private static final AttributeKey<SimpleChannelPool> POOL_KEY = AttributeKey.newInstance("channelPool");

Netty version:4.1.39

Suggested Fix: Use UUID to avoid user code conflict as shown below

private static final AttributeKey<SimpleChannelPool> POOL_KEY = AttributeKey.newInstance("channelPool_" + UUID.randomUUID().toString());
@normanmaurer
Copy link
Member

normanmaurer commented Sep 5, 2019

@nizarm fancy to do a pr ?

I think we may even use something like this:

private final AttributeKey<SimpleChannelPool> poolKey = AttributeKey.newInstance("channelPool." +
            System.identityHashCode(this));

Just in case there are multiple SimpleChannelPool instances.

@nizarm
Copy link
Contributor Author

nizarm commented Sep 5, 2019

@nizarm fancy to do a pr ?

I think we may even use something like this:

private final AttributeKey<SimpleChannelPool> poolKey = AttributeKey.newInstance("channelPool." +
            System.identityHashCode(this));

Just in case there are multiple SimpleChannelPool instances.

Yes - on it !

@normanmaurer
Copy link
Member

@nizarm ok cool... let me know if you get to busy and so can not finish it as I want to release the next version of netty mid next week and it would be nice to have this as well.

@normanmaurer normanmaurer added this to the 4.1.40.Final milestone Sep 6, 2019
nizarm pushed a commit to nizarm/netty that referenced this issue Sep 9, 2019
…om user code. Avoid such conflicts by introducing a UUID in the name (netty#9542)

    Motivation:

    It is noticed that SimpleChannelPool's POOL_KEY attribute name channelPool is easy to get conflict with user code and throws an exception 'channelPool' is already in use. Being a generic framework - it would be great if we can name the attribute something unique - may be use UUID for the name since the name is not required later.

    Modifications:

    This change make sure that the POOL_KEY used inside SimpleChannelPool is unique by appending the object hashcode in the name.

    Result:

    No unwanted channel attribute name conflict with user code.
normanmaurer pushed a commit that referenced this issue Sep 9, 2019
…om user code (#9542) (#9548)

Motivation:

    It is noticed that SimpleChannelPool's POOL_KEY attribute name channelPool is easy to get conflict with user code and throws an exception 'channelPool' is already in use. Being a generic framework - it would be great if we can name the attribute something unique - may be use UUID for the name since the name is not required later.

    Modifications:

    This change make sure that the POOL_KEY used inside SimpleChannelPool is unique by appending the object hashcode in the name.

    Result:

    No unwanted channel attribute name conflict with user code.
violetagg added a commit to reactor/reactor-netty that referenced this issue Sep 12, 2019
Adapt to a change in Netty connection pool
netty/netty#9542
violetagg added a commit to reactor/reactor-netty that referenced this issue Sep 12, 2019
Adapt to a change in Netty connection pool
netty/netty#9542
smaldini pushed a commit to reactor/reactor-netty that referenced this issue Oct 24, 2019
Adapt to a change in Netty connection pool
netty/netty#9542
@zzmao
Copy link

zzmao commented Jun 18, 2020

HI @nizarm @normanmaurer ,

We are using Netty 4.1.42.Final, but we see the following exception twice in a month:

java.lang.IllegalArgumentException: 'channelPool.781942502' is already in use
        at io.netty.util.ConstantPool.createOrThrow(ConstantPool.java:113) ~[netty-all-4.1.42.Final.jar:4.1.42.Final]
        at io.netty.util.ConstantPool.newInstance(ConstantPool.java:95) ~[netty-all-4.1.42.Final.jar:4.1.42.Final]
        at io.netty.util.AttributeKey.newInstance(AttributeKey.java:55) ~[netty-all-4.1.42.Final.jar:4.1.42.Final]
        at io.netty.channel.pool.SimpleChannelPool.<init>(SimpleChannelPool.java:42) ~[netty-all-4.1.42.Final.jar:4.1.42.Final]
        at io.netty.channel.pool.SimpleChannelPool.<init>(SimpleChannelPool.java:85) ~[netty-all-4.1.42.Final.jar:4.1.42.Final]
        at io.netty.channel.pool.SimpleChannelPool.<init>(SimpleChannelPool.java:70) ~[netty-all-4.1.42.Final.jar:4.1.42.Final]
        at io.netty.channel.pool.SimpleChannelPool.<init>(SimpleChannelPool.java:58) ~[netty-all-4.1.42.Final.jar:4.1.42.Final]
        at com.github.ambry.network.http2.Http2MultiplexedChannelPool.<init>(Http2MultiplexedChannelPool.java:102) ~[ambry-network-0.3.266.jar:?]
        at com.github.ambry.network.http2.Http2ChannelPoolMap.newPool(Http2ChannelPoolMap.java:53) ~[ambry-network-0.3.266.jar:?]
        at com.github.ambry.network.http2.Http2ChannelPoolMap.newPool(Http2ChannelPoolMap.java:32) ~[ambry-network-0.3.266.jar:?]
        at io.netty.channel.pool.AbstractChannelPoolMap.get(AbstractChannelPoolMap.java:40) ~[netty-all-4.1.42.Final.jar:4.1.42.Final]
        at com.github.ambry.network.http2.Http2NetworkClient.sendAndPoll(Http2NetworkClient.java:93) ~[ambry-network-0.3.266.jar:?]
        at com.github.ambry.router.NonBlockingRouter$OperationController.run(NonBlockingRouter.java:1005) [ambry-router-0.3.266.jar:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]

The related code is:

public class SimpleChannelPool implements ChannelPool {
    private final AttributeKey<SimpleChannelPool> poolKey = AttributeKey.newInstance("channelPool." +
        System.identityHashCode(this))

Since our code creates brand new SimpleChannelPool, Is it possible that that a SimpleChannelPool was garbage collected, then a new SimpleChannelPool is created at the same memory space and hit this already in use exception?

@cgtz
Copy link

cgtz commented Jun 18, 2020

I see that this change is rolled back with #9760 and the AttributeKey is static once again in netty 4.1.44+. I guess the assumption with the static key is that there is exactly one SimpleChannelPool per channel but the member variable version would have allowed multiple pool instances to be attached to one channel (with the downside of the potential memory leak described in #9760)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants