Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Expression Ranges #21

Closed
wants to merge 497 commits into from
Closed

Support Expression Ranges #21

wants to merge 497 commits into from

Conversation

berkaysynnada
Copy link
Collaborator

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

gruuya and others added 30 commits April 5, 2024 07:13
…e deprecated function (apache#9923)

* Minor: Improve documentation for FilterPushdown and remove deprecated method

* Remove deprecated method

* Improve docs

* more

* Apply suggestions from code review

Co-authored-by: Phillip LeBlanc <[email protected]>

---------

Co-authored-by: Phillip LeBlanc <[email protected]>
…ccumulatorArgs` (apache#9920)

* Minor: Improve documentation for AggregateUDFImpl::accumulator and `AccumulatorArgs`

* Add test and helper functions

* Improve docs and examples

* Fix CI

* Remove checks for ORDER BY and IGNORE NULLS
* Minor: improve TableReference docs

* Clarify normalization
…dy exists. (apache#9912)

* prevent panic

* initial version, bad code

* some error handling

* Some slt tests

* docs and minor refactors

* cleaning up

* fix tests

* clear err message for single-file partitioned tables

* typo

* test invalid/mixed partitions on disk

* ensure order in error msg for testing
…e#9962)

* Extract local variables in physical join planning to function

* Comments
…pache#9958)

* for debug

finish

remove print

add space

* fix clippy

* finish

* fix clippy
…hould be rewritten in OperatorToFunction` (apache#9968)

* Minor: Return InternalError rather than panic for 'NamedStructField should be rewritten in OperatorToFunction'

* clippy
…-functions` (apache#9960)

* backup

Signed-off-by: jayzhan211 <[email protected]>

* move PhysicalExpr

Signed-off-by: jayzhan211 <[email protected]>

* cleanup

Signed-off-by: jayzhan211 <[email protected]>

* move physical sort

Signed-off-by: jayzhan211 <[email protected]>

* cleanup dependencies

Signed-off-by: jayzhan211 <[email protected]>

* add readme

Signed-off-by: jayzhan211 <[email protected]>

* disable doc test

Signed-off-by: jayzhan211 <[email protected]>

* move column

Signed-off-by: jayzhan211 <[email protected]>

* fmt

Signed-off-by: jayzhan211 <[email protected]>

* move aggregatexp

Signed-off-by: jayzhan211 <[email protected]>

* move other two utils

Signed-off-by: jayzhan211 <[email protected]>

* license

Signed-off-by: jayzhan211 <[email protected]>

* switch to ignore

Signed-off-by: jayzhan211 <[email protected]>

* move reverse order

Signed-off-by: jayzhan211 <[email protected]>

* rename to common

Signed-off-by: jayzhan211 <[email protected]>

* cleanup

Signed-off-by: jayzhan211 <[email protected]>

* backup

Signed-off-by: jayzhan211 <[email protected]>

* move acc to first value

Signed-off-by: jayzhan211 <[email protected]>

* move builtin expr too

Signed-off-by: jayzhan211 <[email protected]>

* use macro

Signed-off-by: jayzhan211 <[email protected]>

* fmt

Signed-off-by: jayzhan211 <[email protected]>

* fix doc

Signed-off-by: jayzhan211 <[email protected]>

* add todo

Signed-off-by: jayzhan211 <[email protected]>

* rm comments

Signed-off-by: jayzhan211 <[email protected]>

* rm unused

Signed-off-by: jayzhan211 <[email protected]>

* rm unused code

Signed-off-by: jayzhan211 <[email protected]>

* change to private

Signed-off-by: jayzhan211 <[email protected]>

* fix lock

Signed-off-by: jayzhan211 <[email protected]>

* cleanup

Signed-off-by: jayzhan211 <[email protected]>

* cleanup

Signed-off-by: jayzhan211 <[email protected]>

* support roundtrip

Signed-off-by: jayzhan211 <[email protected]>

* remmove old format state

Signed-off-by: jayzhan211 <[email protected]>

* move aggregate related things to aggr crate

Signed-off-by: jayzhan211 <[email protected]>

* move back to common

Signed-off-by: jayzhan211 <[email protected]>

* taplo

Signed-off-by: jayzhan211 <[email protected]>

* rm comment

Signed-off-by: jayzhan211 <[email protected]>

* cleanup

Signed-off-by: jayzhan211 <[email protected]>

* lock

Signed-off-by: jayzhan211 <[email protected]>

---------

Signed-off-by: jayzhan211 <[email protected]>
* Minor: Improve documentation about optimizer

* fix unused commit
* Update documentation for COPY command

* Fix example

* prettier
* Minor: fix bug in pruning predicate doc

* formatting
* fix: improve `unnest_generic_list` handling of null list

* fix clippy

* fix comment
…ode::visit (apache#9913)

* fix

* clippy

* remove accidental extra apply

* minor fixes

* fix `LogicalPlan::apply_expressions()` and `LogicalPlan::map_subqueries()`

* fix `LogicalPlan::visit_with_subqueries()`

* Add deprecated LogicalPlan::inspect_expressions

---------

Co-authored-by: Andrew Lamb <[email protected]>
* Removing broken tests

* Simplifying tests / removing support for failed tests

* Revert "Simplifying tests / removing support for failed tests"

This reverts commit 6e50a80.

* Fixing tests for real

* Apply suggestions from code review

Thanks @alamb !

Co-authored-by: Andrew Lamb <[email protected]>

---------

Co-authored-by: Andrew Lamb <[email protected]>
* move Floor, Gcd, Lcm, Pi to datafusion-functions
* move the Log, Power functions to datafusion-functions

* match type instead of name

* fix formatting errors
…9985)

* Revert "Add test for reading back file created with FORMAT options (apache#9753)"

This reverts commit b50f3aa.

* Revert "support format in options of COPY command (apache#9744)"

This reverts commit 40fb1b8.

* update docs and example to remove old syntax
…che#10000)

* move Floor, Gcd, Lcm, Pi to datafusion-functions

* remove floor fn

* move Trunc, Cot, Round, iszero functions to datafusion-functions

* Make mod iszero public, minor ordering change to keep the alphabetical ordering theme.

---------

Co-authored-by: Andrew Lamb <[email protected]>
alamb and others added 19 commits May 9, 2024 12:57
…e#10434)

* Move ordering discovery to equality check

* Minor changes

* Minor changes

* Apply suggestions from code review

Co-authored-by: Mehmet Ozan Kabak <[email protected]>

---------

Co-authored-by: Mehmet Ozan Kabak <[email protected]>
…10436)

* Revert "make common expression alias human-readable (apache#10333)"

This reverts commit 2a15614.

* keep some ok parts
* Implement get_flight_info_tables

* remove unimplemented methods

* Simplify FlightInfo construction
…r planning) (apache#10405)

* Add `LogicalPlan::recompute_schema` for handling rewrite passes

* Stop copying LogicalPlan and Exprs in  `OptimizeProjections`
* move bit_and_or_xor unit tests to slt

Signed-off-by: NoeB <[email protected]>

* Apply suggestions from code review

---------

Signed-off-by: NoeB <[email protected]>
* introduce new sig

Signed-off-by: jayzhan211 <[email protected]>

* add udfimpl

Signed-off-by: jayzhan211 <[email protected]>

* replace fun

Signed-off-by: jayzhan211 <[email protected]>

* replace array

Signed-off-by: jayzhan211 <[email protected]>

* coalesce

Signed-off-by: jayzhan211 <[email protected]>

* nvl2

Signed-off-by: jayzhan211 <[email protected]>

* rm variadic equal

Signed-off-by: jayzhan211 <[email protected]>

* fix test

Signed-off-by: jayzhan211 <[email protected]>

* rm err msg to fix ci

Signed-off-by: jayzhan211 <[email protected]>

* user defined sig

Signed-off-by: jayzhan211 <[email protected]>

* add err msg

Signed-off-by: jayzhan211 <[email protected]>

* fmt

Signed-off-by: jayzhan211 <[email protected]>

* cleanup

Signed-off-by: jayzhan211 <[email protected]>

* fix ci

Signed-off-by: jayzhan211 <[email protected]>

* fix ci

Signed-off-by: jayzhan211 <[email protected]>

* upd comment

Signed-off-by: jayzhan211 <[email protected]>

---------

Signed-off-by: jayzhan211 <[email protected]>
apache#10430)

* Remove some Expr clones in `EliminateCrossJoin`

* Apply suggestions from code review

Co-authored-by: comphead <[email protected]>

* fix

---------

Co-authored-by: comphead <[email protected]>
…instead of format!) (apache#10454)

* refactor: use Write instead of format! to implement display_name

* Use static dispatch for write

* remove more allocations

---------

Co-authored-by: Andrew Lamb <[email protected]>
…l syntax for `HEADER ROW`, `DELIMITER` and `COMPRESSION` (apache#10404)

* Simplify format options

* Keep PG copy from tests same

* Update datafusion/common/src/config.rs

Co-authored-by: Andrew Lamb <[email protected]>

* Update datafusion/core/src/datasource/file_format/csv.rs

Co-authored-by: Andrew Lamb <[email protected]>

* Remove WITH HEADER ROW

* Review Part 1

* .

* Fix failing tests

* Revert "Fix failing tests"

This reverts commit 9d81601.

* Final commit

* Minor

* Review

* Update avro.slt

* Apply suggestions

* Fix imports

---------

Co-authored-by: Andrew Lamb <[email protected]>
Co-authored-by: Mehmet Ozan Kabak <[email protected]>
ozankabak pushed a commit that referenced this pull request Jul 24, 2024
* Configurable date field extraction style for unparsing (#21)

* Add support for IntervalStyle::MySQL (#18)

* Support alternate format for Int64 unparsing (SIGNED for MySQL) (#22)

* Alternate format support for Timestamp casting (DATETIME for MySQL) (#23)

* Improve

* Fix clippy and docs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.