Skip to content

Commit

Permalink
ORC-1752: Fix NumberFormatException when reading json timestamp type …
Browse files Browse the repository at this point in the history
…in benchmark

### What changes were proposed in this pull request?
This PR aims to fix NumberFormatException when reading json timestamp type in benchmark.

### Why are the changes needed?
Because the source data of the benchmark has two timestamp formats, we need to be compatible with both Long and String timestamp formats.

```java
Exception in thread "main" java.lang.NumberFormatException: For input string: "2015-11-04T12:00:00Z"
	at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:67)
	at java.base/java.lang.Long.parseLong(Long.java:711)
	at java.base/java.lang.Long.parseLong(Long.java:836)
	at com.google.gson.JsonPrimitive.getAsLong(JsonPrimitive.java:206)
	at org.apache.orc.bench.core.convert.json.JsonReader$TimestampColumnConverter.convert(JsonReader.java:175)
	at org.apache.orc.bench.core.convert.json.JsonReader.nextBatch(JsonReader.java:86)
	at org.apache.orc.bench.core.convert.GenerateVariants$RecursiveReader.nextBatch(GenerateVariants.java:195)
	at org.apache.orc.bench.core.convert.GenerateVariants.run(GenerateVariants.java:154)
	at org.apache.orc.bench.core.Driver.main(Driver.java:64)
```

### How was this patch tested?
```bash
java -jar core/target/orc-benchmarks-core-*-uber.jar generate data
java -jar core/target/orc-benchmarks-core-*-uber.jar scan data
```

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #1995 from cxzl25/ORC-1752.

Authored-by: sychen <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
  • Loading branch information
cxzl25 authored and dongjoon-hyun committed Aug 5, 2024
1 parent d09dbf3 commit 954ba0c
Showing 1 changed file with 6 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,12 @@ public void convert(JsonElement value, ColumnVector vect, int row) {
vect.isNull[row] = true;
} else {
TimestampColumnVector vector = (TimestampColumnVector) vect;
vector.set(row, new Timestamp(value.getAsLong()));
try {
vector.set(row, new Timestamp(value.getAsLong()));
} catch (NumberFormatException e) {
vector.set(row, Timestamp.valueOf(value.getAsString()
.replaceAll("[TZ]", " ")));
}
}
}
}
Expand Down

0 comments on commit 954ba0c

Please sign in to comment.