Add way to specify date format in read_csv

[MarcoGorelli]

Problem description

Taking this forwards from #8168

Originally the report was about performance, but then turned into a conversation about whether to be able to specify the date format in read_csv

There do seem to be potential perf gains:

# t.py
from datetime import datetime
import polars as pl

df = pl.DataFrame({
    'ts': pl.date_range(datetime(1000, 1, 1), datetime(9999, 1, 1), eager=True),
    })
df.write_csv('tmp.csv')

In [1]: %timeit pl.read_csv('tmp.csv', try_parse_dates=True)
28.3 ms ± 402 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [2]: %timeit pl.read_csv('tmp.csv').with_columns(pl.col('ts').str.to_datetime(format='%Y-%m-%dT%H:%M:%S.%6f'))
668 ms ± 41.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

[fernandocast]

Hello I'm new to polars and I would like to contribute on this project ,
could you please assign this issue to me?

[MarcoGorelli]

sure, go ahead, and please check https://github.com/pola-rs/polars/blob/main/CONTRIBUTING.md

[MarcoGorelli]

from discussion, this should either be a single string, or a mapping from column names to formats

[Julian-J-S]

would love to have this as well!

Is there any progress on this? 🤔 😄

[MarcoGorelli]

hey - i'm not working on this at the moment, there's other higher-priority things (rolling functions, interpolate), would be happy to review a pr though

[deanm0000]

Should this be an optimization to scan_csv? As in having pl.scan_csv('tmp.csv').with_columns(pl.col('ts').str.to_datetime(format='%Y-%m-%dT%H:%M:%S.%6f')).collect() do the same thing as pl.read_csv('tmp.csv', try_parse_dates=True)