fix: Improvements to encoding ranges (int vs long) #439

astubbs · 2022-10-12T16:01:02Z

Replace integer offset references with long - use Long everywhere we deal with offsets, and where we truncate down, do it exactly, detect and handle truncation issues.

Checklist

Changelog

Bumps `testcontainers.version` from 1.17.1 to 1.17.2. Updates `testcontainers` from 1.17.1 to 1.17.2 - [Release notes](https://github.com/testcontainers/testcontainers-java/releases) - [Changelog](https://github.com/testcontainers/testcontainers-java/blob/master/CHANGELOG.md) - [Commits](testcontainers/testcontainers-java@1.17.1...1.17.2) Updates `kafka` from 1.17.1 to 1.17.2 - [Release notes](https://github.com/testcontainers/testcontainers-java/releases) - [Changelog](https://github.com/testcontainers/testcontainers-java/blob/master/CHANGELOG.md) - [Commits](testcontainers/testcontainers-java@1.17.1...1.17.2) Updates `junit-jupiter` from 1.17.1 to 1.17.2 - [Release notes](https://github.com/testcontainers/testcontainers-java/releases) - [Changelog](https://github.com/testcontainers/testcontainers-java/blob/master/CHANGELOG.md) - [Commits](testcontainers/testcontainers-java@1.17.1...1.17.2) Updates `postgresql` from 1.17.1 to 1.17.2 - [Release notes](https://github.com/testcontainers/testcontainers-java/releases) - [Changelog](https://github.com/testcontainers/testcontainers-java/blob/master/CHANGELOG.md) - [Commits](testcontainers/testcontainers-java@1.17.1...1.17.2) --- updated-dependencies: - dependency-name: org.testcontainers:testcontainers dependency-type: direct:production update-type: version-update:semver-patch - dependency-name: org.testcontainers:kafka dependency-type: direct:production update-type: version-update:semver-patch - dependency-name: org.testcontainers:junit-jupiter dependency-type: direct:production update-type: version-update:semver-patch - dependency-name: org.testcontainers:postgresql dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>

Bumps [versions-maven-plugin](https://github.com/mojohaus/versions-maven-plugin) from 2.10.0 to 2.11.0. - [Release notes](https://github.com/mojohaus/versions-maven-plugin/releases) - [Changelog](https://github.com/mojohaus/versions-maven-plugin/blob/master/ReleaseNotes.md) - [Commits](mojohaus/versions@versions-maven-plugin-2.10.0...versions-maven-plugin-2.11.0) --- updated-dependencies: - dependency-name: org.codehaus.mojo:versions-maven-plugin dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>

… topics

https://ossindex.sonatype.org/component/pkg:maven/com.google.guava/[email protected]?utm_source=ossindex-client&utm_medium=integration&utm_content=1.8.1

Bumps `vertx.version` from 4.3.0 to 4.3.1. Updates `vertx-web-client` from 4.3.0 to 4.3.1 Updates `vertx-junit5` from 4.3.0 to 4.3.1 --- updated-dependencies: - dependency-name: io.vertx:vertx-web-client dependency-type: direct:production update-type: version-update:semver-patch - dependency-name: io.vertx:vertx-junit5 dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]>

Bumps [postgresql](https://github.com/pgjdbc/pgjdbc) from 42.3.5 to 42.4.0. - [Release notes](https://github.com/pgjdbc/pgjdbc/releases) - [Changelog](https://github.com/pgjdbc/pgjdbc/blob/master/CHANGELOG.md) - [Commits](pgjdbc/pgjdbc@REL42.3.5...REL42.4.0) --- updated-dependencies: - dependency-name: org.postgresql:postgresql dependency-type: direct:development update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]>

- CVE-2020-8908: Files::createTempDir local information disclosure vulnerability #4011 - Only used transitively from tests, and is a deprecated function - google/guava#4011

# Conflicts: # pom.xml

…rs.version-1.17.2' into improvements/version-bumps

…ql-postgresql-42.4.0' into improvements/version-bumps

….mojo-versions-maven-plugin-2.11.0' into improvements/version-bumps

New issues with WireMocks dep on Jetty BOM ~v9

…that Especially given we also use dependabot. If anyone is concerned to that level, they can use the plugin in their end user applications.

…o-long

rkolesnev

One comment - BitSet constructor still accepts Integer.
otherwise LGTM.

parallel-consumer-core/src/main/java/io/confluent/parallelconsumer/offsets/BitSetEncoder.java

astubbs

...

astubbs

lgtm

rkolesnev

LGTM, couple of comments, but nothing blocking.

parallel-consumer-core/src/main/java/io/confluent/csid/utils/Range.java

parallel-consumer-core/src/main/java/io/confluent/parallelconsumer/offsets/BitSetEncoder.java

Co-authored-by: Roman Kolesnev <[email protected]>

astubbs

lgtm

astubbs

lgtm

dependabot bot and others added 30 commits May 20, 2022 16:33

START: Try to reproduce issue confluentinc#184 - run against multiple…

ec21ad0

… topics

test compiles

1b9d2ad

failling test

51ef075

fixes test - introduce ShardKey

7e32d4d

step

0a558f6

step

9628bee

step

b1fd89e

turn off ossindex for guava 31.1

d237158

https://ossindex.sonatype.org/component/pkg:maven/com.google.guava/[email protected]?utm_source=ossindex-client&utm_medium=integration&utm_content=1.8.1

step

d8ea983

step - extract

b883ba0

step - tg demo, keep consumer open

ae21fb7

test for null keys

ad6c89e

allow truth snapshot

23e5389

add missing annotations

7fdb06a

step

03ff349

build: Exclude Guava 31.1 from OSSIndex sec scan

72ac264

- CVE-2020-8908: Files::createTempDir local information disclosure vulnerability #4011 - Only used transitively from tests, and is a deprecated function - google/guava#4011

...

f905cdb

build: Update to Truth-Generator 0.1.1 for Subject discovery

12dc7cf

Merge branch 'exclude-guava-oss-index' into bugs/issue-184-reproduce

bf37740

# Conflicts: # pom.xml

Merge remote-tracking branch 'confluent/dependabot/maven/testcontaine…

080ecd3

…rs.version-1.17.2' into improvements/version-bumps

Merge remote-tracking branch 'confluent/dependabot/maven/org.postgres…

71a48dc

…ql-postgresql-42.4.0' into improvements/version-bumps

Merge remote-tracking branch 'confluent/dependabot/maven/org.codehaus…

a087efb

….mojo-versions-maven-plugin-2.11.0' into improvements/version-bumps

build: Only run OSS Index audit on runtime dependencies, not tests

3e468a7

New issues with WireMocks dep on Jetty BOM ~v9

build: Only run OSS Index audit on runtime dependencies, not tests

10ac83a

New issues with WireMocks dep on Jetty BOM ~v9

build: Don't fail for OSSIndex scan - will set up another CI job for …

7efc8de

…that Especially given we also use dependabot. If anyone is concerned to that level, they can use the plugin in their end user applications.

build: Don't fail for OSSIndex scan - will set up another CI job for …

70a37cd

…that Especially given we also use dependabot. If anyone is concerned to that level, they can use the plugin in their end user applications.

Merge remote-tracking branch 'astubbs/fixes/logging' into fixes/int-t…

b5ef682

…o-long

astubbs mentioned this pull request Oct 21, 2022

feature: Add new encoding runlength v3 using Longs instead of Integers #408

Closed

2 tasks

rkolesnev approved these changes Oct 21, 2022

View reviewed changes

parallel-consumer-core/src/main/java/io/confluent/parallelconsumer/offsets/BitSetEncoder.java Show resolved Hide resolved

astubbs added 4 commits October 21, 2022 13:12

remove unused constructor

e65b7b8

use exact casting for sanity

7eaee4f

Change encoding interface to long as well

3272332

review

e2a3d05

astubbs commented Oct 21, 2022

View reviewed changes

review

af8f119

astubbs commented Oct 21, 2022

View reviewed changes

astubbs added 4 commits October 21, 2022 14:27

review

8dc65db

Merge branch 'master' into fixes/int-to-long

ab1ba7d

review

49ab1ef

review

bc6d592

astubbs requested a review from rkolesnev October 21, 2022 14:57

rkolesnev approved these changes Oct 21, 2022

View reviewed changes

astubbs and others added 6 commits October 26, 2022 12:16

Apply suggestions from code review

920ec04

Co-authored-by: Roman Kolesnev <[email protected]>

step: compiles, encode steps potentially throw encode exception

9558e71

test update

d7bfc79

review, pull version up

0782ae9

review

e8e0d40

Merge remote-tracking branch 'origin/master' into fixes/int-to-long

4aa295b

astubbs commented Nov 1, 2022

View reviewed changes

minor: remove comment - uses SortedSet not Set

6a8cbf4

astubbs mentioned this pull request Nov 1, 2022

refactor: SortedSet's all the way down (Set->SortedSet) #463

Merged

2 tasks

Changelog entry

c89d9e7

astubbs mentioned this pull request Nov 1, 2022

fixes #71: Feature: Health check and metrics #464

Closed

2 tasks

astubbs commented Nov 2, 2022

View reviewed changes

astubbs merged commit 66b0158 into confluentinc:master Nov 2, 2022

astubbs deleted the fixes/int-to-long branch November 2, 2022 13:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Improvements to encoding ranges (int vs long) #439

fix: Improvements to encoding ranges (int vs long) #439

astubbs commented Oct 12, 2022 •

edited

Loading

rkolesnev left a comment

astubbs left a comment

astubbs left a comment

rkolesnev left a comment

astubbs left a comment

astubbs left a comment

fix: Improvements to encoding ranges (int vs long) #439

fix: Improvements to encoding ranges (int vs long) #439

Conversation

astubbs commented Oct 12, 2022 • edited Loading

Checklist

rkolesnev left a comment

Choose a reason for hiding this comment

astubbs left a comment

Choose a reason for hiding this comment

astubbs left a comment

Choose a reason for hiding this comment

rkolesnev left a comment

Choose a reason for hiding this comment

astubbs left a comment

Choose a reason for hiding this comment

astubbs left a comment

Choose a reason for hiding this comment

astubbs commented Oct 12, 2022 •

edited

Loading