diff --git a/README.md b/README.md index 6133a0edc..beaa789a2 100644 --- a/README.md +++ b/README.md @@ -37,28 +37,21 @@ Get started! - [Join our community call](https://calendar.google.com/calendar/embed?src=27ff6dc5f598c1d94c1f6e627a1aaae680e2fac88f848bda1f2c7946ae74d5ab%40group.calendar.google.com) - [Read the contributing guide](https://github.com/narwhals-dev/narwhals/blob/main/CONTRIBUTING.md) -## Used by - -Join the party! - -- [altair](https://github.com/vega/altair/) -- [hierarchicalforecast](https://github.com/Nixtla/hierarchicalforecast) -- [marimo](https://github.com/marimo-team/marimo) -- [panel-graphic-walker](https://github.com/panel-extensions/panel-graphic-walker) -- [plotly](https://plotly.com) -- [pymarginaleffects](https://github.com/vincentarelbundock/pymarginaleffects) -- [py-shiny](https://github.com/posit-dev/py-shiny) -- [rio](https://github.com/rio-labs/rio) -- [scikit-lego](https://github.com/koaning/scikit-lego) -- [scikit-playtime](https://github.com/koaning/scikit-playtime) -- [tabmat](https://github.com/Quantco/tabmat) -- [timebasedcv](https://github.com/FBruzzesi/timebasedcv) -- [tubular](https://github.com/lvgig/tubular) -- [vegafusion](https://github.com/vega/vegafusion) -- [wimsey](https://github.com/benrutter/wimsey) - -Feel free to add your project to the list if it's missing, and/or -[chat with us on Discord](https://discord.gg/V3PqtB4VA4) if you'd like any support. +
+Table of contents + +- [Narwhals](#narwhals) + - [Installation](#installation) + - [Usage](#usage) + - [Example](#example) + - [Scope](#scope) + - [Roadmap](#roadmap) + - [Used by](#used-by) + - [Sponsors and institutional partners](#sponsors-and-institutional-partners) + - [Appears on](#appears-on) + - [Why "Narwhals"?](#why-narwhals) + +
## Installation @@ -106,6 +99,34 @@ See the [tutorial](https://narwhals-dev.github.io/narwhals/basics/dataframe/) fo If you said yes to both, we'd love to hear from you! +## Roadmap + +See [roadmap discussion on GitHub](https://github.com/narwhals-dev/narwhals/discussions/1370) +for an up-to-date plan of future work. + +## Used by + +Join the party! + +- [altair](https://github.com/vega/altair/) +- [hierarchicalforecast](https://github.com/Nixtla/hierarchicalforecast) +- [marimo](https://github.com/marimo-team/marimo) +- [panel-graphic-walker](https://github.com/panel-extensions/panel-graphic-walker) +- [plotly](https://plotly.com) +- [pymarginaleffects](https://github.com/vincentarelbundock/pymarginaleffects) +- [py-shiny](https://github.com/posit-dev/py-shiny) +- [rio](https://github.com/rio-labs/rio) +- [scikit-lego](https://github.com/koaning/scikit-lego) +- [scikit-playtime](https://github.com/koaning/scikit-playtime) +- [tabmat](https://github.com/Quantco/tabmat) +- [timebasedcv](https://github.com/FBruzzesi/timebasedcv) +- [tubular](https://github.com/lvgig/tubular) +- [vegafusion](https://github.com/vega/vegafusion) +- [wimsey](https://github.com/benrutter/wimsey) + +Feel free to add your project to the list if it's missing, and/or +[chat with us on Discord](https://discord.gg/V3PqtB4VA4) if you'd like any support. + ## Sponsors and institutional partners Narwhals is 100% independent, community-driven, and community-owned. diff --git a/docs/css/code_select.css b/docs/css/code_select.css deleted file mode 100644 index 259cf11c5..000000000 --- a/docs/css/code_select.css +++ /dev/null @@ -1,7 +0,0 @@ -.highlight .gp, .highlight .go { /* Generic.Prompt, Generic.Output */ - user-select: none; - -webkit-user-select: none; /* Safari */ - -moz-user-select: none; /* Firefox */ - -ms-user-select: none; /* Internet Explorer/Edge */ - color: red; -} \ No newline at end of file diff --git a/docs/css/extra.css b/docs/css/extra.css new file mode 100644 index 000000000..2d2769e09 --- /dev/null +++ b/docs/css/extra.css @@ -0,0 +1,4 @@ +.md-typeset ol li, +.md-typeset ul li { + margin-bottom: 0.1em !important; +} \ No newline at end of file diff --git a/docs/roadmap_and_related.md b/docs/ecosystem.md similarity index 50% rename from docs/roadmap_and_related.md rename to docs/ecosystem.md index e3f59c443..75e0f3921 100644 --- a/docs/roadmap_and_related.md +++ b/docs/ecosystem.md @@ -1,9 +1,29 @@ -# Roadmap and related projects +# Ecosystem -## Roadmap +## Used by -See [roadmap discussion on GitHub](https://github.com/narwhals-dev/narwhals/discussions/1370) -for an up-to-date plan of future work. +The following is a non-exhaustive list of libraries and tools that choose to use Narwhals +for their dataframe interoperability needs: + +* [altair](https://github.com/vega/altair/) +* [hierarchicalforecast](https://github.com/Nixtla/hierarchicalforecast) +* [marimo](https://github.com/marimo-team/marimo) +* [panel-graphic-walker](https://github.com/panel-extensions/panel-graphic-walker) +* [plotly](https://github.com/plotly/plotly.py) +* [pymarginaleffects](https://github.com/vincentarelbundock/pymarginaleffects) +* [py-shiny](https://github.com/posit-dev/py-shiny) +* [rio](https://github.com/rio-labs/rio) +* [scikit-lego](https://github.com/koaning/scikit-lego) +* [scikit-playtime](https://github.com/koaning/scikit-playtime) +* [tabmat](https://github.com/Quantco/tabmat) +* [timebasedcv](https://github.com/FBruzzesi/timebasedcv) +* [tubular](https://github.com/lvgig/tubular) +* [vegafusion](https://github.com/vega/vegafusion) +* [wimsey](https://github.com/benrutter/wimsey) + +If your project is missing from the list, feel free to open a PR to add it. + +If you would like to chat with us, or if you need any support, please [join our Discord server](https://discord.gg/V3PqtB4VA4). ## Related projects @@ -21,7 +41,10 @@ Array counterpart to the DataFrame API, see [here](https://data-apis.org/array-a ### PyCapsule Interface -Allows C extension modules to safely share pointers to C data structures with Python code and other C modules, encapsulating the pointer with a name and optional destructor to manage resources and ensure safe access, see [here](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html) for details. +Allows C extension modules to safely share pointers to C data structures with Python code +and other C modules, encapsulating the pointer with a name and optional destructor to +manage resources and ensure safe access, +see [here](https://arrow.apache.org/docs/format/CDataInterface/PyCapsuleInterface.html) for details. Narwhals supports exporting a DataFrame via the Arrow PyCapsule Interface. @@ -30,30 +53,29 @@ Narwhals supports exporting a DataFrame via the Arrow PyCapsule Interface. Pitched as "The portable Dataframe library", Ibis provides a Pythonic frontend to various SQL (as well as Polars LazyFrame) engines. Some differences with Narwhals are: -- Narwhals' main use case is for library maintainers wanting to support +* Narwhals' main use case is for library maintainers wanting to support different dataframe libraries without depending on any whilst keeping things as lightweight as possible. Ibis is more targeted at end users and aims to be thought of as a Dataframe library akin to pandas / Polars / etc. -- Narwhals allows you to write a "Dataframe X in, Dataframe X out" function. +* Narwhals allows you to write a "Dataframe X in, Dataframe X out" function. Ibis allows materialising to pandas, Polars (eager), and PyArrow, but has no way to get back to the input type exactly (e.g. there's no way to start with a Polars LazyFrame and get back a Polars LazyFrame) -- Narwhals respects input data types as much as possible, Ibis doesn't +* Narwhals respects input data types as much as possible, Ibis doesn't support Categorical (nor does it distinguish between fixed-size-list and list) -- Narwhals separates between lazy and eager APIs, with the eager API +* Narwhals separates between lazy and eager APIs, with the eager API provide very fine control over dataframe operations (slicing rows and columns, iterating over rows, getting values out of the dataframe as Python scalars). Ibis is more focused on lazy execution -- Ibis supports SQL engines (and can translate to SQL), +* Ibis supports SQL engines (and can translate to SQL), Narwhals is more focused traditional dataframes where row-order is defined (although we are brainstorming a lazy-only level of support) -- Narwhals is extremely lightweight and comes with zero required dependencies, +* Narwhals is extremely lightweight and comes with zero required dependencies, Ibis requires pandas and PyArrow for all backends -- Narwhals supports Dask, whereas Ibis has deprecated support for it +* Narwhals supports Dask, whereas Ibis has deprecated support for it Although people often ask about the two tools, we consider them to be very different and not in competition. Further efforts to clarify the distinction are welcome 🙏! - diff --git a/docs/extending.md b/docs/extending.md index f05a733e4..82729856b 100644 --- a/docs/extending.md +++ b/docs/extending.md @@ -1,4 +1,4 @@ -# Extending Narwhals, levels of support +# Supported libraries and extending Narwhals ## List of supported libraries (and how to add yours!) @@ -15,7 +15,7 @@ Currently, Narwhals has full API support for the following libraries: It also has lazy-only support for [Dask](https://github.com/dask/dask), and interchange-only support for [DuckDB](https://github.com/duckdb/duckdb) and [Ibis](https://github.com/ibis-project/ibis). -### Levels +### Levels of support Narwhals comes with two levels of support ("full" and "interchange"), and we are working on defining a "lazy-only" level too. @@ -93,7 +93,7 @@ converting to Narwhals DataFrame: - `.select(names)` (Ibis and DuckDB), where `names` is a list of (string) column names. This is useful for selecting columns before converting to another library. -### Extending Narwhals +## Extending Narwhals If you want your own library to be recognised too, you're welcome open a PR (with tests)!. Alternatively, if you can't do that (for example, if you library is closed-source), see diff --git a/docs/index.md b/docs/index.md index e9fe02170..7f88568a4 100644 --- a/docs/index.md +++ b/docs/index.md @@ -28,3 +28,8 @@ Anyone wishing to write a library/application/service which consumes dataframes, completely dataframe-agnostic. Let's get started! + +## Roadmap + +See [roadmap discussion on GitHub](https://github.com/narwhals-dev/narwhals/discussions/1370) +for an up-to-date plan of future work. diff --git a/docs/installation.md b/docs/installation.md index 25c014ba6..1ff9d5db4 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -9,7 +9,7 @@ If you haven't, you can follow our [_setting up your environment_](https://github.com/narwhals-dev/narwhals/blob/main/CONTRIBUTING.md#option-1-use-uv-recommended) guide. Then, run: - ```console + ```terminal uv pip install narwhals ``` @@ -19,18 +19,20 @@ Then, run: - ```console + ```terminal python -m pip install narwhals ``` ### Verifying the Installation To verify the installation, start the Python REPL and execute: + ```python >>> import narwhals >>> narwhals.__version__ '1.14.3' ``` + If you see the version number, then the installation was successful! ## Quick start @@ -60,7 +62,7 @@ import narwhals as nw from narwhals.typing import IntoFrame -def my_function(df_native: IntoFrame) -> list[str]: +def agnostic_get_columns(df_native: IntoFrame) -> list[str]: df = nw.from_native(df_native) column_names = df.columns return column_names @@ -72,17 +74,22 @@ df_polars = pl.DataFrame(data) table_pa = pa.table(data) print("pandas output") -print(my_function(df_pandas)) +print(agnostic_get_columns(df_pandas)) print("Polars output") -print(my_function(df_polars)) +print(agnostic_get_columns(df_polars)) print("PyArrow output") -print(my_function(table_pa)) +print(agnostic_get_columns(table_pa)) ``` -If you run `python t.py` then your output should look like the above. This is the simplest possible example of a dataframe-agnostic -function - as we'll soon see, we can do much more advanced things. +If you run `python t.py` then your output should look like the above. +This is the simplest possible example of a dataframe-agnostic function - as we'll soon +see, we can do much more advanced things. + Let's learn about what you just did, and what Narwhals can do for you! -Note: these examples are only using pandas, Polars and PyArrow. Please see the following to find the [supported libraries](extending.md). +!!! info + + These examples are using pandas, Polars and PyArrow, however Narwhals supports + other dataframe libraries (See [supported libraries](extending.md)). diff --git a/docs/resources.md b/docs/resources.md new file mode 100644 index 000000000..36369bf2f --- /dev/null +++ b/docs/resources.md @@ -0,0 +1,47 @@ +# Resources + +## Sponsors and institutional partners + +Narwhals is 100% independent, community-driven, and community-owned. +We are extremely grateful to the following organisations for having +provided some funding / development time: + +- [Quansight Labs](https://labs.quansight.org) +- [Quansight Futures](https://www.qi.ventures) +- [OpenTeams](https://www.openteams.com) +- [POSSEE initiative](https://possee.org) +- [BYU-Idaho](https://www.byui.edu) + +If you contribute to Narwhals on your organization's time, please let us know. We'd be happy to add your employer +to this list! + +## Appears on + +Narwhals has been featured in several talks, podcasts, and blog posts: + +- [Talk Python to me Podcast](https://youtu.be/FSH7BZ0tuE0) + Ahoy, Narwhals are bridging the data science APIs + +- [Python Bytes Podcast](https://www.youtube.com/live/N7w_ESVW40I?si=y-wN1uCsAuJOKlOT&t=382) + Episode 402, topic #2 + +- [Super Data Science: ML & AI Podcast](https://www.youtube.com/watch?v=TeG4U8R0U8U) + Narwhals: For Pandas-to-Polars DataFrame Compatibility + +- [Sample Space Podcast | probabl](https://youtu.be/8hYdq4sWbbQ?si=WG0QP1CZ6gkFf18b) + How Narwhals has many end users ... that never use it directly. - Marco Gorelli + +- [The Real Python Podcast](https://www.youtube.com/watch?v=w5DFZbFYzCM) + Narwhals: Expanding DataFrame Compatibility Between Libraries + +- [Pycon Lithuania](https://www.youtube.com/watch?v=-mdx7Cn6_6E) + Marco Gorelli - DataFrame interoperatiblity - what's been achieved, and what comes next? + +- [Pycon Italy](https://www.youtube.com/watch?v=3IqUli9XsmQ) + How you can write a dataframe-agnostic library - Marco Gorelli + +- [Polars Blog Post](https://pola.rs/posts/lightweight_plotting/) + Polars has a new lightweight plotting backend + +- [Quansight Labs blog post (w/ Scikit-Lego)](https://labs.quansight.org/blog/scikit-lego-narwhals) + How Narwhals and scikit-lego came together to achieve dataframe-agnosticism diff --git a/docs/why.md b/docs/why.md index 4ec605d16..3a0959e20 100644 --- a/docs/why.md +++ b/docs/why.md @@ -12,7 +12,9 @@ print(3 in pd.Series([1, 2, 3])) print(3 in pl.Series([1, 2, 3])) ``` -Try it out and see ;) Spoiler alert: they don't. pandas checks if `3` is in the index, +Try it out and see 😉 + +Spoiler alert: they don't. pandas checks if `3` is in the index, Polars checks if it's in the values. For another example, try running the code below - note how the outputs have different column names after the join! @@ -30,10 +32,11 @@ print(pd_left_merge.columns) print(pl_left_merge.columns) ``` -There are several such subtle difference between the libraries. Writing dataframe-agnostic code is hard! +There are several such subtle difference between the libraries. **Writing dataframe-agnostic code is hard!** + But by having a unified, simple, and predictable API, you can focus on behaviour rather than on subtle implementation differences. Furthermore, both pandas and Polars frequently deprecate behaviour. Narwhals handles this for you by -testing against nightly builds of both libraries and handling backwards compatibility internally +testing against nightly builds of both libraries and handling backwards compatibility internally (so you don't have to!). diff --git a/mkdocs.yml b/mkdocs.yml index d7f422f5e..abbbd5fce 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -16,11 +16,12 @@ nav: - pandas_like_concepts/user_warning.md - pandas_like_concepts/column_names.md - pandas_like_concepts/boolean.md - - overhead.md - - backcompat.md - - extending.md - - how_it_works.md - - Roadmap and related projects: roadmap_and_related.md + - Overhead: overhead.md + - Perfect backwards compatibility policy: backcompat.md + - Supported libraries and extending Narwhals: extending.md + - How it works: how_it_works.md + - Ecosystem: ecosystem.md + - Resources: resources.md - API Completeness: - api-completeness/index.md - Supported DataFrame methods: api-completeness/dataframe.md @@ -127,3 +128,4 @@ extra_javascript: extra_css: - https://unpkg.com/katex@0/dist/katex.min.css + - css/extra.css \ No newline at end of file