Skip to content

Commit

Permalink
PARQUET-1387: Nanosecond precision time and timestamp - parquet-format (
Browse files Browse the repository at this point in the history
  • Loading branch information
nandorKollar authored and gszadovszky committed Aug 28, 2018
1 parent 677ed8e commit b879065
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 3 deletions.
14 changes: 12 additions & 2 deletions LogicalTypes.md
Original file line number Diff line number Diff line change
Expand Up @@ -256,7 +256,7 @@ The sort order used for `DATE` is signed.

`TIME` is used for a logical time type without a date with millisecond or microsecond precision.
The type has two type parameters: UTC adjustment (`true` or `false`)
and precision (`MILLIS` or `MICROS`).
and precision (`MILLIS` or `MICROS`, `NANOS`).

`TIME` with precision `MILLIS` is used for millisecond precision.
It must annotate an `int32` that stores the number of
Expand All @@ -266,6 +266,10 @@ milliseconds after midnight.
It must annotate an `int64` that stores the number of
microseconds after midnight.

`TIME` with precision `NANOS` is used for nanosecond precision.
It must annotate an `int64` that stores the number of
nanoseconds after midnight.

The sort order used for `TIME` is signed.

#### Deprecated time ConvertedType
Expand Down Expand Up @@ -305,7 +309,7 @@ with precision `MICROS`. Like the logical type counterpart, it must annotate an

`TIMESTAMP` is used for a combined logical date and time type, with
millisecond or microsecond precision. The type has two type parameters:
UTC adjustment (`true` or `false`) and precision (`MILLIS` or `MICROS`).
UTC adjustment (`true` or `false`) and precision (`MILLIS` or `MICROS`, `NANOS`).

`TIMESTAMP` with precision `MILLIS` is used for millisecond precision.
It must annotate an `int64` that stores the number of
Expand All @@ -315,6 +319,12 @@ milliseconds from the Unix epoch, 00:00:00.000 on 1 January 1970, UTC.
It must annotate an `int64` that stores the number of
microseconds from the Unix epoch, 00:00:00.000000 on 1 January 1970, UTC.

`TIMESTAMP` with precision `NANOS` is used for nanosecond precision.
It must annotate an `int64` that stores the number of
nanoseconds from the Unix epoch, 00:00:00.000000000 on 1 January 1970, UTC.
Valid values for nanosecond precision are between
00:12:43 21 September 1677 UTC and 23:47:16 11 April 2262 UTC.

The sort order used for `TIMESTAMP` is signed.

#### Deprecated timestamp ConvertedType
Expand Down
4 changes: 3 additions & 1 deletion src/main/thrift/parquet.thrift
Original file line number Diff line number Diff line change
Expand Up @@ -257,9 +257,11 @@ struct DecimalType {
/** Time units for logical types */
struct MilliSeconds {}
struct MicroSeconds {}
struct NanoSeconds {}
union TimeUnit {
1: MilliSeconds MILLIS
2: MicroSeconds MICROS
3: NanoSeconds NANOS
}

/**
Expand All @@ -275,7 +277,7 @@ struct TimestampType {
/**
* Time logical type annotation
*
* Allowed for physical types: INT32 (millis), INT64 (micros)
* Allowed for physical types: INT32 (millis), INT64 (micros, nanos)
*/
struct TimeType {
1: required bool isAdjustedToUTC
Expand Down

0 comments on commit b879065

Please sign in to comment.