Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(python): support DataFrame init from pydantic model data #8178

Merged
merged 1 commit into from
Apr 12, 2023

Conversation

alexander-beedie
Copy link
Collaborator

@alexander-beedie alexander-beedie commented Apr 12, 2023

Closes #8154.

This was a simple addition; pydantic model data is largely equivalent to @dataclass or NamedTuple data, which we already support. Only needed a few minor tweaks to leverage the existing code.

  • No additional dependencies (except for unit tests; integration is optional/lazy).
  • Works with instantiated sequences (list/tuple) of models and/or generators of such.

Example

from pydantic import BaseModel
import polars as pl

class Shipment( BaseModel ):
    exporter: str
    importer: str
    tonnes: int
    
shipments = [
    Shipment( exporter="Sri Lanka", importer="USA", tonnes=10 ),
    Shipment( exporter="India", importer="UK", tonnes=25 ),
    Shipment( exporter="China", importer="UK", tonnes=40 ),
]

pl.DataFrame( shipments )

# shape: (3, 3)
# ┌───────────┬──────────┬────────┐
# │ exporter  ┆ importer ┆ tonnes │
# │ ---       ┆ ---      ┆ ---    │
# │ str       ┆ str      ┆ i64    │
# ╞═══════════╪══════════╪════════╡
# │ Sri Lanka ┆ USA      ┆ 10     │
# │ India     ┆ UK       ┆ 25     │
# │ China     ┆ UK       ┆ 40     │
# └───────────┴──────────┴────────┘

@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars labels Apr 12, 2023
@alexander-beedie alexander-beedie changed the title feat(python): support DataFrame init from pydantic model data feat(python): support DataFrame init from pydantic model data Apr 12, 2023
Copy link
Contributor

@stinodego stinodego left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! My colleague will be very happy with this 😄

@ritchie46
Copy link
Member

@samuelcolvin FYI. :)

@ritchie46 ritchie46 merged commit 4110656 into pola-rs:main Apr 12, 2023
@samuelcolvin
Copy link

amazing!

As it happened I've started using pola.rs today, and it's been an incredible experience - I'm really impressed. Thank you.

@alexander-beedie alexander-beedie deleted the init-from-pydantic-models branch April 12, 2023 11:35
@alexander-beedie
Copy link
Collaborator Author

As it happened I've started using pola.rs today

@samuelcolvin: Well now you've made me look into the pydantic source to work out what the optimal loading path is and make sure we're doing this as fast as possible: #8181 🤣

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature highlight Highlight this PR in the changelog python Related to Python Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support from_pydantic constructor
4 participants