Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Table names with periods are not handled correctly #4513

Closed
alamb opened this issue Dec 5, 2022 · 6 comments · Fixed by #4530
Closed

Table names with periods are not handled correctly #4513

alamb opened this issue Dec 5, 2022 · 6 comments · Fixed by #4530
Assignees
Labels
bug Something isn't working

Comments

@alamb
Copy link
Contributor

alamb commented Dec 5, 2022

Describe the bug
We have a customer with a table whose name contains a period

In SQL you can refer to such tables using double quotes "foo.bar". Note this is different than foo.bar which means table named bar in the schema named foo.

You can see how this works in postgres:

postgres=# create table "foo.bar" as values (1), (2);
SELECT 2
postgres=# select * from foo.bar;
ERROR:  relation "foo.bar" does not exist
LINE 1: select * from foo.bar;
                      ^
postgres=# select * from "foo.bar";
 column1 
---------
       1
       2
(2 rows)

postgres=# \d foo.bar
Did not find any relation named "foo.bar".
postgres=# \d "foo.bar"
               Table "public.foo.bar"
 Column  |  Type   | Collation | Nullable | Default 
---------+---------+-----------+----------+---------
 column1 | integer |           |          | 


**To Reproduce**
Try to create (or query) tables with double quotes:

From the error message it is clear that DataFusion is treating `"foo.bar"`  as schema named `foo` table named `bar` 

```sql
❯ select * from "foo.bar";
Plan("failed to resolve schema: foo")
❯ select * from foo.bar;
Plan("failed to resolve schema: foo")
❯ create table "foo.bar" as values (1), (2);
Plan("failed to resolve schema: \"foo")

Expected behavior
DataFusion should treat "foo.bar" as a table named foo.bar like SQL / postgres

Additional context
Add any other context about the problem here.

@alamb alamb added the bug Something isn't working label Dec 5, 2022
@alamb alamb changed the title Table names with periods are not supported correctly Table names with periods are not handled correctly Dec 5, 2022
@alamb
Copy link
Contributor Author

alamb commented Dec 5, 2022

@alamb alamb self-assigned this Dec 6, 2022
@alamb
Copy link
Contributor Author

alamb commented Dec 6, 2022

I am actively working on this issue

@waitingkuo
Copy link
Contributor

waitingkuo commented Dec 6, 2022

hi @alamb

i tried to add this

        if s.len() >= 2 && s.chars().nth(0) == Some('"') && s.chars().last() == Some('"') {
            return Self::Bare {table: s}
        }

to return the whole double quoted string before let parts: Vec<&str> = s.split('.').collect();

and it works while creating a new table

❯ create table "foo.bar" as values (1), (2);
0 rows in set. Query took 0.001 seconds.

❯ show tables;
+---------------+--------------------+-------------+------------+
| table_catalog | table_schema       | table_name  | table_type |
+---------------+--------------------+-------------+------------+
| datafusion    | public             | "foo.bar"   | BASE TABLE |
| datafusion    | information_schema | tables      | VIEW       |
| datafusion    | information_schema | views       | VIEW       |
| datafusion    | information_schema | columns     | VIEW       |
| datafusion    | information_schema | df_settings | VIEW       |
+---------------+--------------------+-------------+------------+
6 rows in set. Query took 0.003 seconds.

but failed to select it

select * from "foo.bar";
Plan("failed to resolve schema: foo")

@waitingkuo
Copy link
Contributor

@alamb
i also tested the single quoted string

❯ create table 'foo' as values (1), (2);
0 rows in set. Query took 0.001 seconds.

i'm not sure whether this is expected

❯ show tables;
s information_schema.tables
+---------------+--------------------+-------------+------------+
| table_catalog | table_schema       | table_name  | table_type |
+---------------+--------------------+-------------+------------+
| datafusion    | public             | 'foo'       | BASE TABLE |
| datafusion    | information_schema | tables      | VIEW       |
| datafusion    | information_schema | views       | VIEW       |
| datafusion    | information_schema | columns     | VIEW       |
| datafusion    | information_schema | df_settings | VIEW       |
+---------------+--------------------+-------------+------------+

as postgresql doesn't support it

willy=# create table 'foo' as values (1), (2);
ERROR:  syntax error at or near "'foo'"
LINE 1: create table 'foo' as values (1), (2);

@alamb
Copy link
Contributor Author

alamb commented Dec 6, 2022

I have a fix for this partially coded -- I'll get it up shortly

@alamb
Copy link
Contributor Author

alamb commented Dec 6, 2022

Proposed PR to fix: #4530

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants