Skip to content

Commit

Permalink
Implement Iterators on Storage, arithmetic and comparisons use the St…
Browse files Browse the repository at this point in the history
…orageIterators (#12274)

The first step to get back some of the performance from the storage refactor.
- New `ColumnLongStorageWithArray` and `ColumnDoubleStorageWithArray` interfaces allowing low-level iterators to iterate over the array directly.
- Removed the `DoubleArrayAdapter`, `BigIntegerArrayAdapter` and `BigDecimalArrayAdapter` in favour of having `ColumnStorage` based facades allowing for some storage to present as others.
- Moved the iterations in `NumericBinaryOpImplementation` into `StorageIterators`. Currently a lot of duplication to the `NumericComparison`. Will look to make more common later.

The intention is to keep working on the common iteration layer as build up the operator space.

Various tests on the performance impact of various parts (tested on my PC, over 1,000,000 additions, only provided for reference):
- Using a builder versus populating array and bitset adds about 0.1ms.
- Using the iterator over a tight inline loop appeared to be about 0.2ms
- Using the array loops returns about 0.4ms over the getItemAsLong calls.
  • Loading branch information
jdunkerley authored Feb 17, 2025
1 parent 06bb294 commit b786a33
Show file tree
Hide file tree
Showing 35 changed files with 1,934 additions and 1,343 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ public void appendBulkStorage(Storage<?> storage) {
if (storage instanceof DoubleStorage doubleStorage) {
int n = (int) doubleStorage.getSize();
ensureFreeSpaceFor(n);
System.arraycopy(doubleStorage.getRawData(), 0, data, currentSize, n);
System.arraycopy(doubleStorage.getArray(), 0, data, currentSize, n);
BitSets.copy(doubleStorage.getIsNothingMap(), isNothing, currentSize, n);
currentSize += n;
} else if (storage instanceof ColumnDoubleStorage doubleStorage) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ public void appendBulkStorage(Storage<?> storage) {
// A fast path for the same type - no conversions/checks needed.
int n = (int) longStorage.getSize();
ensureFreeSpaceFor(n);
System.arraycopy(longStorage.getRawData(), 0, data, currentSize, n);
System.arraycopy(longStorage.getArray(), 0, data, currentSize, n);
BitSets.copy(longStorage.getIsNothingMap(), isNothing, currentSize, n);
currentSize += n;
} else if (storage.getType() instanceof IntegerType otherType && getType().fits(otherType)) {
Expand Down

Large diffs are not rendered by default.

Loading

0 comments on commit b786a33

Please sign in to comment.