Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do DataFrame operations under the hood using Polars for q.cells() #2259

Open
luke-quadratic opened this issue Feb 6, 2025 · 0 comments
Open

Comments

@luke-quadratic
Copy link
Contributor

luke-quadratic commented Feb 6, 2025

Pyodide added Polars support recently, should enable a huge performance uptick for Python if we decide to use it: Pyodide PR for Polars

A few possibilities:

  • q.cells() - use Polars under the hood for operations and convert to DataFrame in last step by default
  • have an option on q.cells() to allow Polars instead of Pandas to give users a huge performance uptick (enabling the option would enforce Polars DataFrames to that user's initial return type)

Regardless of whether we add the Polars instead of Pandas option as a parameter in q.cells() we can use Polars in the intermediate steps.

To be explored during upcoming perf work.

@luke-quadratic luke-quadratic added the type: enhancement new feature or request label Feb 6, 2025
@luke-quadratic luke-quadratic added type: perf and removed type: enhancement new feature or request labels Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

1 participant