feat(sql): implement basic duckdb dialect

[machow]

Implements a duckdb sql dialect, that inherits from the postgres one. It supports the same range of methods (e.g. including str.contains), with the exception of str.title.

There are 4 failing tests, due to an issue of the duckdb sql alchemy -> pandas process, that seems to go as follows:

• when a verb has a literal value (e.g. mutate(x = 1))
• the resulting pandas column is "object" type
• this seems related to the use of sqlalchemy literals in queries

Below is short code to replicate the problem:

import pandas as pd
from sqlalchemy import sql, create_engine

engine = create_engine("duckdb:///:memory:")

res_expr = pd.read_sql(
    sql.select(sql.literal(1).label('x')),  engine
)

res_str = pd.read_sql(
    """SELECT 1 AS x""",
    engine
)

res_expr.x.dtype     # object :/
res_str.x.dtype      # int

[Alex-Monahan]

Thank you for all of your work to integrate DuckDB! Super cool.

Before today, I have only used SQLAlchemy for its parameter binding / DB connection strings, so please forgive me if this is not a valid approach!

It looks like using literal_column instead of literal will preserve the type that you expect. Although oddly enough it requires a string as input instead of an integer!

import pandas as pd
from sqlalchemy import sql, create_engine

engine = create_engine("duckdb:///:memory:")

test = sql.select(sql.literal_column('1').label('x'))
print(test) # SELECT 1 as x 

res_expr = pd.read_sql(
    sql.select(sql.literal_column('1').label('x')),  engine
)

res_str = pd.read_sql(
    """SELECT 1 AS x""",
    engine
)

print(res_expr.x.dtype)     # int64 
print(res_str.x.dtype)      # int64

[machow]

Ah thanks--I think literal passes in its values as parameters. Whenever duckdb receives parameters, it turns them into strings?

import duckdb
con = duckdb.connect(database=':memory:')

# int32
con.execute("SELECT 1").fetch_df().info()

# object
con.execute("SELECT ?", (1, )).fetch_df().info()

# type in SQL is varchar
con.execute("SELECT pg_typeof(?)", (1, )).fetch_df()

[machow]

I wonder if this is related to duckdb/duckdb#2128?

[machow]

@Alex-Monahan -- re-implemented in a new PR (#422), and it's finally passing all tests! It requires the changes on the master branch of duckdb_engine, but once those are released, should be good to merge it..

[Alex-Monahan]

Fantastic!! I'm planning on having a siuba quick start how-to guide in the DuckDB docs. If you want to take a first pass at one, you are welcome to!

[machow]

I'm planning on having a siuba quick start how-to guide in the DuckDB docs. If you want to take a first pass at one, you are welcome to!

Ah, thanks--I don't mind taking a pass at a quick start section! Is it easiest for me to open a PR against a doc page, or send another way?

(I'm realizing that executing on duckdb is a real game changer, so might also do some screencasts / blog posts..!)

[Alex-Monahan]

A PR against the docs repo would be perfect! Let us know when/if you plan on posting any content and we can gladly upvote on Hacker News, etc... :-)

Docs repo: https://github.com/duckdb/duckdb-web

Here is an example of the metadata needed for a how to guide to get it to show up in the right menus and all:
https://github.com/duckdb/duckdb-web/pull/192/files