Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: deprecate Expr.head, Expr.tail, Expr.sort, Expr.gather_every, Expr.sample (but keep them in stable.v1) #1791

Merged

Conversation

MarcoGorelli
Copy link
Member

@MarcoGorelli MarcoGorelli commented Jan 11, 2025

One step closer to narwhals.stable.v2

These are some operations which we won't be able to support in the lazy layer at all, and I think it would be confusing to have some Expr methods which aren't available at all for lazyframes (which would be different to Expr.unique, which could be supported if followed by an aggregation, or Expr.cum_sum, which could be supported if order_by is passed)

Further, I just don't think any of these expressions are very useful...people call head/tail/sort/sample on dataframes, not typically on individual expressions..

For the cases where people need them, they can use DataFrame or Series operations

What type of PR is this? (check all applicable)

  • 💾 Refactor
  • ✨ Feature
  • 🐛 Bug Fix
  • 🔧 Optimization
  • 📝 Documentation
  • ✅ Test
  • 🐳 Other

Related issues

  • Related issue #<issue number>
  • Closes #<issue number>

Checklist

  • Code follows style guide (ruff)
  • Tests added
  • Documented the changes

If you have comments or can explain your changes, please do so below

@MarcoGorelli

This comment was marked as outdated.

@MarcoGorelli MarcoGorelli force-pushed the remove-non-agg-shape-changes branch 2 times, most recently from 5b92ccb to 20ef637 Compare January 11, 2025 16:05
Comment on lines -11 to +12
def test_head(
def test_tail(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol

@MarcoGorelli MarcoGorelli marked this pull request as ready for review January 11, 2025 16:21
@MarcoGorelli MarcoGorelli added the enhancement New feature or request label Jan 11, 2025
@MarcoGorelli MarcoGorelli marked this pull request as draft January 12, 2025 13:55
@MarcoGorelli MarcoGorelli changed the title feat: remove partial reductions from Expr feat: remove Expr.head, Expr.tail, Expr.sort, Expr.sample Jan 15, 2025
@MarcoGorelli MarcoGorelli force-pushed the remove-non-agg-shape-changes branch from 20ef637 to 655616e Compare January 15, 2025 13:19
@MarcoGorelli MarcoGorelli changed the title feat: remove Expr.head, Expr.tail, Expr.sort, Expr.sample feat: remove Expr.head, Expr.tail, Expr.sort, Expr.gather_every Jan 15, 2025
@MarcoGorelli MarcoGorelli changed the title feat: remove Expr.head, Expr.tail, Expr.sort, Expr.gather_every feat: remove Expr.head, Expr.tail, Expr.sort, Expr.gather_every, Expr.sort Jan 15, 2025
@MarcoGorelli MarcoGorelli changed the title feat: remove Expr.head, Expr.tail, Expr.sort, Expr.gather_every, Expr.sort feat: remove Expr.head, Expr.tail, Expr.sort, Expr.gather_every, Expr.sort (but keep them in stable.v1) Jan 15, 2025
@MarcoGorelli MarcoGorelli marked this pull request as ready for review January 15, 2025 13:25
Copy link
Collaborator

@EdAbati EdAbati left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes a lot of sense! It looks 👌

Copy link
Member

@FBruzzesi FBruzzesi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am all in favor for deprecating head, tail and gather every. I still think that sort and sample might have valid use cases, although hard/impossible to implement in some backends

@@ -21,6 +22,9 @@ def test_gather_every_expr(

assert_equal_data(result, expected)

with pytest.deprecated_call():
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL!

@FBruzzesi FBruzzesi changed the title feat: remove Expr.head, Expr.tail, Expr.sort, Expr.gather_every, Expr.sort (but keep them in stable.v1) feat: remove Expr.head, Expr.tail, Expr.sort, Expr.gather_every, Expr.sample (but keep them in stable.v1) Jan 16, 2025
@MarcoGorelli
Copy link
Member Author

thanks all for feedback and for taking a look! i think more than implementation difficulties they're just not relational, e.g.

In [16]: df
Out[16]:
shape: (3, 2)
┌───────┬─────┐
│ nameage │
│ ------ │
│ stri64 │
╞═══════╪═════╡
│ Mario31  │
│ Luigi32  │
│ Wario45  │
└───────┴─────┘

In [17]: df.select('name', pl.col('age').sort(descending=True))
Out[17]:
shape: (3, 2)
┌───────┬─────┐
│ nameage │
│ ------ │
│ stri64 │
╞═══════╪═════╡
│ Mario45  │
│ Luigi32  │
│ Wario31  │
└───────┴─────┘

you end up with tuples which aren't part of the original object - it's fine for Polars to allow it, but I'm not sure we should be allowing this kind of operation in Narwhals

In any case, happy to bring these back if use cases do come along!

@MarcoGorelli MarcoGorelli changed the title feat: remove Expr.head, Expr.tail, Expr.sort, Expr.gather_every, Expr.sample (but keep them in stable.v1) feat: deprecate Expr.head, Expr.tail, Expr.sort, Expr.gather_every, Expr.sample (but keep them in stable.v1) Jan 16, 2025
@MarcoGorelli MarcoGorelli merged commit 21df661 into narwhals-dev:main Jan 16, 2025
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants