Failed to decompress snappy

[edenhill]

Snappy compressed messages sometimes fail decompression in the consumer

[edenhill]

This is due to the snappy-java framing that will sometimes create multiple frames, which rdkafka currently doesn't support.

[winbatch]

The problem I saw was on the publisher side though, where messages simply
have local timeout when using snappy

On Wednesday, January 8, 2014, Magnus Edenhill wrote:

This is due to the snappy-java framing that will sometimes create multiple
frames, which rdkafka currently doesn't support.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/55#issuecomment-31882548
.

[edenhill]

Can you reproduce with rdkafka_performance or .._example?
Or run with "debug" set to "broker,topic,msg"?

[winbatch]

Reproduced with example.

On Wednesday, January 8, 2014, Magnus Edenhill wrote:

Can you reproduce with rdkafka_performance or .._example?
Or run with "debug" set to "broker,topic,msg"?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/55#issuecomment-31884205
.

[edenhill]

Perfect, can you provide command lines?

[winbatch]

./rdkafka_example -P -b kafkadevcluster1-1.masked.com:5757,
kafkadevcluster1-2.masked.com:5757,kafkadevcluster1-3.masked.com:5757 -p 8
-t LaraReplicator_kafkacluster4 -z snappy

Error on publisher with debug on:

1389224014.657 RDKAFKA-7-MSGSET: rdkafka#producer-0:
d145931-004.masked.com:5757/2: MessageSet with 1 message(s) delivered^M

1389224014.657 RDKAFKA-7-MSGSET: rdkafka#producer-0:
d145931-004.masked.com:5757/2: MessageSet with 1 message(s) encountered
error: Err-65535?^M

On Wednesday, January 8, 2014, Magnus Edenhill wrote:

Perfect, can you provide command lines?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/55#issuecomment-31887813
.

[edenhill]

Is there any exception on the broker when this happens?

Can you run with debug enabled?: ... -d all and see if it says anything more specific about that unknown error.

[winbatch]

I can reproduce it against an 0.8.0 (beta) cluster and not 0.8.1 so maybe
it was a cluster/broker thing that was fixed?

On Wednesday, January 8, 2014, Magnus Edenhill wrote:

Is there any exception on the broker when this happens?

Can you run with debug enabled?: ... -d all and see if it says anything
more specific about that unknown error.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/55#issuecomment-31888482
.

[edenhill]

Maybe, but Ive been running most of my tests on 0.8.0 beta.
Would be very interesting with the debug output.

[winbatch]

Unfortunately I don't have access to the cluster hosts/logs. I guess try
and reproduce it on your side and if you can't don't worry about it.

On Thursday, January 9, 2014, Magnus Edenhill wrote:

Maybe, but Ive been running most of my tests on 0.8.0 beta.
Would be very interesting with the debug output.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/55#issuecomment-31908762
.

[edenhill]

I cant reproduce it and would need more to go on.
Let me know if you find more information.

2014/1/9 winbatch [email protected]

Unfortunately I don't have access to the cluster hosts/logs. I guess try
and reproduce it on your side and if you can't don't worry about it.

On Thursday, January 9, 2014, Magnus Edenhill wrote:

Maybe, but Ive been running most of my tests on 0.8.0 beta.
Would be very interesting with the debug output.

—
Reply to this email directly or view it on GitHub<
https://github.com/edenhill/librdkafka/issues/55#issuecomment-31908762>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/55#issuecomment-31922582
.

[winbatch]

Got the broker log. Looks like snappy wasn't in the classpath/noclassdeffounderror

[edenhill]

Okay, good news.
Thank you.