Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

minor: Update data type support documentation #4984

Merged
merged 2 commits into from
Jan 20, 2023
Merged

minor: Update data type support documentation #4984

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b19f7ae
minor: Update data type support documentation
alamb Jan 19, 2023
dc96264
add arrow_typeof docs
alamb Jan 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/dev.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,8 +42,7 @@ jobs:
node-version: "14"
- name: Prettier check
run: |
# if you encounter error, try rerun the command below with --write instead of --check
# and commit the changes
# if you encounter error, rerun the command below and commit the changes
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

driveby cleanup (the command below actually uses --write rather than --check already

#
# ignore subproject CHANGELOG.md because they are machine generated
npx [email protected] --write \
Expand Down
26 changes: 19 additions & 7 deletions docs/source/user-guide/sql/data_types.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,26 @@ execution. The SQL types from
are mapped to [Arrow data types](https://docs.rs/arrow/latest/arrow/datatypes/enum.DataType.html) according to the following table.
This mapping occurs when defining the schema in a `CREATE EXTERNAL TABLE` command or when performing a SQL `CAST` operation.

You can see the corresponding Arrow type for any SQL expression using
the `arrow_typeof` function. For example:

```sql
❯ select arrow_typeof(interval '1 month');
+-------------------------------------+
| arrowtypeof(IntervalYearMonth("1")) |
+-------------------------------------+
| Interval(YearMonth) |
+-------------------------------------+
```

## Character Types

| SQL DataType | Arrow DataType |
| ------------ | -------------- |
| `CHAR` | `Utf8` |
| `VARCHAR` | `Utf8` |
| `TEXT` | `Utf8` |
| `STRING` | `Utf8` |
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

❯ create table foo (x string) as select 'foo';
0 rows in set. Query took 0.003 seconds.


## Numeric Types

Expand All @@ -52,11 +65,12 @@ This mapping occurs when defining the schema in a `CREATE EXTERNAL TABLE` comman

## Date/Time Types

| SQL DataType | Arrow DataType |
| ------------ | :-------------------------------------- |
| `DATE` | `Date32` |
| `TIME` | `Time64(TimeUnit::Nanosecond)` |
| `TIMESTAMP` | `Timestamp(TimeUnit::Nanosecond, None)` |
| SQL DataType | Arrow DataType |
| ------------ | :----------------------------------------------------------------------- |
| `DATE` | `Date32` |
| `TIME` | `Time64(TimeUnit::Nanosecond)` |
| `TIMESTAMP` | `Timestamp(TimeUnit::Nanosecond, None)` |
| `INTERVAL` | `Interval(IntervalUnit::YearMonth)` or `Interval(IntervalUnit::DayTime)` |
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intervals work great:

❯ select now() + interval '1 minute';
+----------------------------------+
| now() + IntervalDayTime("60000") |
+----------------------------------+
| 2023-01-19T14:30:03.656542+00:00 |
+----------------------------------+

❯ select arrow_typeof(interval '1 month');
+-------------------------------------+
| arrowtypeof(IntervalYearMonth("1")) |
+-------------------------------------+
| Interval(YearMonth)                 |
+-------------------------------------+
1 row in set. Query took 0.006 seconds.
❯ select arrow_typeof(interval '1 second');
+--------------------------------------+
| arrowtypeof(IntervalDayTime("1000")) |
+--------------------------------------+
| Interval(DayTime)                    |
+--------------------------------------+


## Boolean Types

Expand All @@ -81,10 +95,8 @@ This mapping occurs when defining the schema in a `CREATE EXTERNAL TABLE` comman
| `VARBINARY` | _Not yet supported_ |
| `REGCLASS` | _Not yet supported_ |
| `NVARCHAR` | _Not yet supported_ |
| `STRING` | _Not yet supported_ |
| `CUSTOM` | _Not yet supported_ |
| `ARRAY` | _Not yet supported_ |
| `ENUM` | _Not yet supported_ |
| `SET` | _Not yet supported_ |
| `INTERVAL` | _Not yet supported_ |
| `DATETIME` | _Not yet supported_ |
13 changes: 13 additions & 0 deletions docs/source/user-guide/sql/scalar_functions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,19 @@ wherever it appears in the statement, using a value chosen at planning time.

### `array`

### `arrow_typeof`

Returns the underlying Arrow type of the the expression:

```sql
❯ select arrow_typeof(4 + 4.3);
+--------------------------------------+
| arrowtypeof(Int64(4) + Float64(4.3)) |
+--------------------------------------+
| Float64 |
+--------------------------------------+
```

### `in_list`

### `random`
Expand Down