You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
What happened:
When delta-rs tries to write a timestamp partition on Windows it fails with the following error.
test timestamp
thread 'test_files_scanned' panicked at 'called `Result::unwrap()` on an `Err` value: ObjectStore { source: Generic { store: "LocalFileSystem", source: UnableToCreateFile { path: "C:\\Users\\runneradmin\\AppData\\Local\\Temp\\.tmpKm0Y2T\\timestamp=1970-01-01 00:00:00\\part-00000-537f4856-39e3-4058-895a-a726fcf769ae-c000.snappy.parquet", err: Os { code: 123, kind: InvalidFilename, message: "The filename, directory name, or volume label syntax is incorrect." } } } }', rust\tests\datafusion_test.rs:89:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
This occurs since the following characters are illegal for a path in windows. < > ? [ ] : | *
This likely is also an issue with the string column too
What you expected to happen:
Writing timestamp columns on Windows should work. We also need to handle any other illegal value in the path.
The solution should conform to Spark's behavior and end users should only use the partition value from the Delta log
The text was updated successfully, but these errors were encountered:
Environment
Delta-rs version: Rust
Binding: main
Environment:
Bug
What happened:
When delta-rs tries to write a timestamp partition on Windows it fails with the following error.
This occurs since the following characters are illegal for a path in windows. < > ? [ ] : | *
This likely is also an issue with the string column too
What you expected to happen:
Writing timestamp columns on Windows should work. We also need to handle any other illegal value in the path.
The solution should conform to Spark's behavior and end users should only use the partition value from the Delta log
The text was updated successfully, but these errors were encountered: