diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md index 58633dec5..8296cda81 100644 --- a/docs/CONTRIBUTING.md +++ b/docs/CONTRIBUTING.md @@ -205,7 +205,7 @@ Type hints allow us to spend less time reading documentation. Public modules are All public modules in the SDK are checked for the presence of docstrings in classes and functions. We follow the [Google Style convention](https://www.sphinx-doc.org/en/master/usage/extensions/example_google.html) for Python docstrings so functions are required to have a description of every argument and the return value, if applicable. -### What is Poetry and why do we need it? +### What is uv and why do we need it? -For more info on `Poetry` and `Pipx`, please see the topic in our +For more info on `uv`, please see the topic in our [python tips](./python_tips.md) guide. diff --git a/docs/cli_commands.md b/docs/cli_commands.md index e5d37f551..d50cb9862 100644 --- a/docs/cli_commands.md +++ b/docs/cli_commands.md @@ -7,86 +7,219 @@ prefix `poetry run`. - Note: CLI mapping is performed in `pyproject.toml` and shims are recreated during `poetry install`: + ````{tab} Poetry ```toml ... [tool.poetry.scripts] tap-mysource = 'singer_sdk.tests.sample_tap_parquet.parquet_tap:cli' ``` + ```` + + ````{tab} uv + ```toml + ... + [project.scripts] + tap-mysource = 'singer_sdk.tests.sample_tap_parquet.parquet_tap:cli' + ``` + ```` The CLI commands defined here will be configured automatically when the python library is installed by a user. ## For example, to run `--help` +````{tab} Poetry ```bash poetry install && \ poetry run tap-mysource --help ``` +```` + +````{tab} uv +```bash +uv sync && \ +uv run tap-mysource --help +``` +```` ## Run in sync mode with auto-discovery +````{tab} Poetry ```bash poetry install && \ poetry run tap-mysource \ --config singer_sdk/samples/sample_tap_parquet/parquet-config.sample.json ``` +```` + +````{tab} uv +```bash +uv sync && \ +uv run tap-mysource \ + --config singer_sdk/samples/sample_tap_parquet/parquet-config.sample.json +```` ## Run in sync mode with a catalog file input +````{tab} Poetry ```bash poetry install && \ poetry run tap-mysource \ --config singer_sdk/samples/sample_tap_parquet/parquet-config.sample.json --catalog singer_sdk/samples/sample_tap_parquet/parquet-catalog.sample.json ``` +```` + +````{tab} uv +```bash +uv sync && \ +uv run tap-mysource \ + --config singer_sdk/samples/sample_tap_parquet/parquet-config.sample.json + --catalog singer_sdk/samples/sample_tap_parquet/parquet-catalog.sample.json +```` ## Run in discovery mode +````{tab} Poetry ```bash poetry install && \ poetry run tap-mysource --discover \ --config singer_sdk/samples/sample_tap_parquet/parquet-config.sample.json ``` +```` + +````{tab} uv +```bash +uv sync && \ +uv run tap-mysource --discover \ + --config singer_sdk/samples/sample_tap_parquet/parquet-config.sample.json +```` ## Run in discovery mode with a passed catalog file +````{tab} Poetry ```bash poetry install && \ poetry run tap-mysource --discover \ --config singer_sdk/samples/sample_tap_parquet/parquet-config.sample.json \ --catalog singer_sdk/samples/sample_tap_parquet/parquet-catalog.sample.json ``` +```` + +````{tab} uv +```bash +uv sync && \ +uv run tap-mysource --discover \ + --config singer_sdk/samples/sample_tap_parquet/parquet-config.sample.json \ + --catalog singer_sdk/samples/sample_tap_parquet/parquet-catalog.sample.json +```` ## Test connectivity The `--test` option allows the user to validate configuration and assess connectivity. +````{tab} Poetry ```bash poetry install && \ poetry run tap-mysource --test \ --config singer_sdk/samples/sample_tap_parquet/parquet-config.sample.json ``` +```` + +````{tab} uv +```bash +uv sync && \ +uv run tap-mysource --test \ + --config singer_sdk/samples/sample_tap_parquet/parquet-config.sample.json +```` ## Package Information The `--about` option displays metadata about the package. +````{tab} Poetry ```console $ poetry run sdk-tap-countries-sample --about Name: sample-tap-countries +Description: Sample tap for Countries GraphQL API. +Version: [could not be detected] +SDK Version: 0.44.2 +Capabilities: + - catalog + - state + - discover + - about + - stream-maps + - schema-flattening + - batch +Settings: + - Name: stream_maps + Type: ['object', 'null'] + Environment Variable: SAMPLE_TAP_COUNTRIES_STREAM_MAPS + - Name: stream_map_config + Type: ['object', 'null'] + Environment Variable: SAMPLE_TAP_COUNTRIES_STREAM_MAP_CONFIG + - Name: faker_config + Type: ['object', 'null'] + Environment Variable: SAMPLE_TAP_COUNTRIES_FAKER_CONFIG + - Name: flattening_enabled + Type: ['boolean', 'null'] + Environment Variable: SAMPLE_TAP_COUNTRIES_FLATTENING_ENABLED + - Name: flattening_max_depth + Type: ['integer', 'null'] + Environment Variable: SAMPLE_TAP_COUNTRIES_FLATTENING_MAX_DEPTH + - Name: batch_config + Type: ['object', 'null'] + Environment Variable: SAMPLE_TAP_COUNTRIES_BATCH_CONFIG +``` +```` + +````{tab} uv +```console +$ uv run sdk-tap-countries-sample --about +Name: sample-tap-countries +Description: Sample tap for Countries GraphQL API. Version: [could not be detected] -Sdk_Version: 0.3.5 -Capabilities: ['sync', 'catalog', 'state', 'discover'] -Settings: {'type': 'object', 'properties': {}} +SDK Version: 0.44.2 +Capabilities: + - catalog + - state + - discover + - about + - stream-maps + - schema-flattening + - batch +Settings: + - Name: stream_maps + Type: ['object', 'null'] + Environment Variable: SAMPLE_TAP_COUNTRIES_STREAM_MAPS + - Name: stream_map_config + Type: ['object', 'null'] + Environment Variable: SAMPLE_TAP_COUNTRIES_STREAM_MAP_CONFIG + - Name: faker_config + Type: ['object', 'null'] + Environment Variable: SAMPLE_TAP_COUNTRIES_FAKER_CONFIG + - Name: flattening_enabled + Type: ['boolean', 'null'] + Environment Variable: SAMPLE_TAP_COUNTRIES_FLATTENING_ENABLED + - Name: flattening_max_depth + Type: ['integer', 'null'] + Environment Variable: SAMPLE_TAP_COUNTRIES_FLATTENING_MAX_DEPTH + - Name: batch_config + Type: ['object', 'null'] + Environment Variable: SAMPLE_TAP_COUNTRIES_BATCH_CONFIG ``` +```` This information can also be printed in JSON format for consumption by other applications +````{tab} Poetry ```console $ poetry run sdk-tap-countries-sample --about --format json { "name": "sample-tap-countries", + "description": "Sample tap for Countries GraphQL API.", "version": "[could not be detected]", - "sdk_version": "0.3.5", + "sdk_version": "0.44.2", "capabilities": [ "sync", "catalog", @@ -99,6 +232,29 @@ $ poetry run sdk-tap-countries-sample --about --format json } } ``` +```` + +````{tab} uv +```console +$ uv run sdk-tap-countries-sample --about --format json +{ + "name": "sample-tap-countries", + "description": "Sample tap for Countries GraphQL API.", + "version": "[could not be detected]", + "sdk_version": "0.44.2", + "capabilities": [ + "sync", + "catalog", + "state", + "discover" + ], + "settings": { + "type": "object", + "properties": {} + } +} +``` +```` ## Invocation options @@ -108,47 +264,9 @@ There a are few options available to invoke your connector. Activate the Poetry environment with `poetry shell` or prefix all your commands with `poetry run`. The commands you then run are the same ones you'd use if you installed your package with `pip`. -### Shell Script - -Following are some simple shell scripts (e.g. `my-tap.sh`) that can save you from typing `poetry install` and `poetry run my-tap` too many times. +### uv -#### Taps - -```bash -#!/bin/sh - -# This simple script allows you to test your tap from any directory, while still taking -# advantage of the poetry-managed virtual environment. -# Adapted from: https://github.com/python-poetry/poetry/issues/2179#issuecomment-668815276 - -unset VIRTUAL_ENV - -STARTDIR=$(pwd) -TOML_DIR=$(dirname "$0") - -cd "$TOML_DIR" || exit -poetry install 1>&2 -poetry run my-tap $* -``` - -#### Targets - -```bash -#!/bin/sh - -# This simple script allows you to test your target from any directory, while still taking -# advantage of the poetry-managed virtual environment. -# Adapted from: https://github.com/python-poetry/poetry/issues/2179#issuecomment-668815276 - -unset VIRTUAL_ENV - -STARTDIR=$(pwd) -TOML_DIR=$(dirname "$0") - -cd "$TOML_DIR" || exit -poetry install 1>&2 -poetry run my-target $* < /dev/stdin -``` +Prefix all your commands with `uv run`. The commands you then run are the same ones you'd use if you installed your package with `pip` or `uv pip`. ### Meltano @@ -175,7 +293,7 @@ plugins: ### Comparison -| | native/shell/`poetry` | `meltano` | +| | native/shell/`poetry`/`uv` | `meltano` | | ------------------- | :-----------------------------------------------------------------------------------------: | :------------------------------------------------------------------: | | Configuration store | Config JSON file (`--config=path/to/config.json`) or environment variables (`--config=ENV`) | `meltano.yml`, `.env`, environment variables, or Meltano's system db | | Simple invocation | `my-tap --config=...` | `meltano invoke my-tap` | diff --git a/docs/dev_guide.md b/docs/dev_guide.md index 43563aa3f..e8bc68fae 100644 --- a/docs/dev_guide.md +++ b/docs/dev_guide.md @@ -39,20 +39,13 @@ for more information on differences between a target's `Sink` class versus a tap ## Building a New Tap or Target -First, install [cookiecutter](https://cookiecutter.readthedocs.io), -[Poetry](https://python-poetry.org/docs/), and optionally [Tox](https://tox.wiki/): +First, install [uv](https://docs.astral.sh/uv/), [cookiecutter](https://cookiecutter.readthedocs.io), and optionally [Tox](https://tox.wiki/): ```bash -# Install pipx if you haven't already -pip install pipx -pipx ensurepath - -# Restart your terminal here, if needed, to get the updated PATH -pipx install cookiecutter -pipx install poetry +uv tool install cookiecutter # Optional: Install Tox if you want to use it to run auto-formatters, linters, tests, etc. -pipx install tox +uv tool install tox ``` :::{tip} @@ -222,7 +215,7 @@ Also using breakpoints is a great way to become familiar with the internals of t #### VSCode Debugging -Ensure the interpreter you're using in VSCode is set to use poetry. +Ensure the interpreter you're using in VSCode is set to use the one in the project's virtual environment (usually `.venv` in the project root). You can change this by using the command palette to go to interpreter settings. Doing this will also help with autocompletion. @@ -274,22 +267,47 @@ We've had success using [`viztracer`](https://github.com/gaogaotiantian/viztrace You can start doing the same in your package. Start by installing `viztracer`. +````{tab} Poetry ```console $ poetry add --group dev viztracer ``` +```` + +````{tab} uv +```console +$ uv add --group dev viztracer +``` +```` Then simply run your package's CLI as normal, preceded by the `viztracer` command +````{tab} Poetry ```console $ poetry run viztracer my-tap $ poetry run viztracer -- my-target --config=config.json --input=messages.json ``` +```` + +````{tab} uv +```console +$ uv run viztracer my-tap +$ uv run viztracer -- my-target --config=config.json --input=messages.json +``` +```` That command will produce a `result.json` file which you can explore with the `vizviewer` tool. +````{tab} Poetry ```console $ poetry run vizviewer result.json ``` +```` + +````{tab} uv +```console +$ uv run vizviewer result.json +``` +```` The output should look like this diff --git a/docs/faq.md b/docs/faq.md index cded23bbc..93a87f0fe 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -8,7 +8,7 @@ Most likely you should delete the project and start over. The property types are documented in the [JSON Schema helper classes](./reference.rst). However, if you're using an IDE such as VSCode, you should be able to set up the environment to give you autocompletion prompts or hints. -Ensure your interpreter is set to poetry if you've followed the [Dev Guide](./dev_guide.md). +Ensure your interpreter is set to the local virtual environment if you've followed the [Dev Guide](./dev_guide.md). Checkout this [gif](https://visualstudiomagazine.com/articles/2021/04/20/~/media/ECG/visualstudiomagazine/Images/2021/04/poetry.ashx) for how to change your interpreter. ### Handling credentials and other secrets in config diff --git a/docs/guides/custom-clis.md b/docs/guides/custom-clis.md index f17cabb5b..0dcb1a003 100644 --- a/docs/guides/custom-clis.md +++ b/docs/guides/custom-clis.md @@ -6,7 +6,7 @@ By default, packages created with the Singer SDK will have a single command, e.g ## Adding a custom command -To add a custom command, you will need to add a new method to your plugin class that returns an instance of [`click.Command`](https://click.palletsprojects.com/en/8.1.x/api/#commands) (or a subclass of it) and decorate it with the `singer_sdk.cli.plugin_cli` decorator. Then you will need to add the command to the `[tool.poetry.scripts]` section of your `pyproject.toml` file. +To add a custom command, you will need to add a new method to your plugin class that returns an instance of [`click.Command`](https://click.palletsprojects.com/en/8.1.x/api/#commands) (or a subclass of it) and decorate it with the `singer_sdk.cli.plugin_cli` decorator. Then you will need to add the command to the `[tool.poetry.scripts]` or `[project.scripts]` section of your `pyproject.toml` file. ```python # tap_shortcut/tap.py @@ -29,6 +29,7 @@ class ShortcutTap(Tap): return update ``` +````{tab} Poetry ```toml # pyproject.toml @@ -36,3 +37,14 @@ class ShortcutTap(Tap): tap-shortcut = "tap_shortcut.tap:ShortcutTap.cli" tap-shortcut-update-schema = "tap_shortcut.tap:ShortcutTap.update_schema" ``` +```` + +````{tab} uv +```toml +# pyproject.toml + +[project.scripts] +tap-shortcut = "tap_shortcut.tap:ShortcutTap.cli" +tap-shortcut-update-schema = "tap_shortcut.tap:ShortcutTap.update_schema" +``` +```` diff --git a/docs/guides/index.md b/docs/guides/index.md index f754908a2..1c3a0e5ab 100644 --- a/docs/guides/index.md +++ b/docs/guides/index.md @@ -11,4 +11,6 @@ custom-clis config-schema sql-tap sql-target + +migrate-to-uv ``` diff --git a/docs/guides/migrate-to-uv.md b/docs/guides/migrate-to-uv.md new file mode 100644 index 000000000..356396080 --- /dev/null +++ b/docs/guides/migrate-to-uv.md @@ -0,0 +1,29 @@ +# Migrating a plugin from Poetry to uv + +This guide will help you migrate your Singer plugin from Poetry to uv. + +## Prerequisites + +- [uv](https://docs.astral.sh/uv/) + +## Steps + +1. Use the [migrate-to-uv](https://github.com/mkniewallner/migrate-to-uv) tool to migrate your project from Poetry to uv. Run the following command in your project directory: + +```bash +uvx migrate-to-uv +``` + +2. Update the `build-system` section in your `pyproject.toml` file to use `hatchling`: + +```toml +[build-system] +build-backend = "hatchling.build" +requires = ["hatchling>=1,<2"] +``` + +3. Update your CI/CD to use `uv` commands instead of `poetry` commands. For example, replace `poetry install` with `uv sync`. You may also want to use the [official `setup-uv`](https://github.com/astral-sh/setup-uv/) GitHub Action to install `uv` in your CI/CD workflow. + +## Example + +For an example of a migration to `uv` for a Singer tap, see [how it was done for tap-stackexchange](https://github.com/MeltanoLabs/tap-stackexchange/pull/507). diff --git a/docs/guides/porting.md b/docs/guides/porting.md index aec02ea7b..6af537126 100644 --- a/docs/guides/porting.md +++ b/docs/guides/porting.md @@ -55,19 +55,31 @@ Once you have a single stream defined, with 3-6 properties, you're ready to cont Check for required libraries in `archive/requirements.txt` and/or `archive/setup.py`. For each library you find: +````{tab} Poetry ```console -#Add a library: +# Add a library: poetry add my-library -#Or add the same library with version constraints: +# Or add the same library with version constraints: poetry add my-library==1.0.2 ``` +```` + +````{tab} uv +```console +# Add a library: +uv add my-library + +# Or add the same library with version constraints: +uv add my-library==1.0.2 +``` +```` Note: - You can probably skip any libraries related to `requests`, `backoff`, or `singer-*` - as these functions are already managed in the SDK. -Once you have the necessary dependencies added, run `poetry install` to make sure everything is ready to go. +Once you have the necessary dependencies added, run `poetry install`/`uv sync` to make sure everything is ready to go. ## Perform `TODO` items in `tap.py` and `client.py` @@ -81,17 +93,31 @@ Note: You _do not_ have to resolve TODOs everywhere in the project, but if there This is the stage where you'll finally see data streaming from the tap. 🙌 -If you have not already done so, run `poetry install` to make sure your project and its dependencies are properly installed in your virtual environment. +If you have not already done so, run `poetry install`/`uv` to make sure your project and its dependencies are properly installed in your virtual environment. Repeat the following steps until you see a help message: +````{tab} Poetry 1. Run `poetry run tap-mysource --help` to confirm the program can run. 2. Find and fix any errors that occur. +```` + +````{tab} uv +1. Run `uv run tap-mysource --help` to confirm the program can run. +2. Find and fix any errors that occur. +```` Now, repeat the following steps until you get data coming through your tap: +````{tab} Poetry 1. Run `poetry run tap-mysource` to attempt your first data sync. 2. Find and fix any errors that occur. +```` + +````{tab} uv +1. Run `uv run tap-mysource` to attempt your first data sync. +2. Find and fix any errors that occur. +```` If you run into error, go back and debug, and especially double check your authentication process and input credentials. @@ -105,7 +131,7 @@ Pagination is generally unique for almost every API. There's no single method th Most likely you will use [get_new_paginator](singer_sdk.RESTStream.get_new_paginator) to instantiate a [pagination class](./../classes/singer_sdk.pagination.BaseAPIPaginator.rst) for your source, and you'll use `get_url_params` to define how to pass the "next page" token back to the API when asking for subsequent pages. -When you think you have it right, run `poetry run tap-mysource` again, and debug until you are confident the result is including multiple pages back from the API. +When you think you have it right, run `poetry run tap-mysource`/`uv run tap-mysource` again, and debug until you are confident the result is including multiple pages back from the API. You can also add unit tests for your pagination implementation for additional confidence: @@ -185,9 +211,17 @@ Note: Depending on how well the API is designed, this could take 5 minutes or mu Now is a good time to test that the built-in tests are working as expected: +````{tab} Poetry ```console poetry run pytest ``` +```` + +````{tab} uv +```console +uv run pytest +``` +```` ## Create the remaining streams @@ -204,9 +238,17 @@ Notes: Now that all streams are defined, run `pytest` again. +````{tab} Poetry ```console poetry run pytest ``` +```` + +````{tab} uv +```console +uv run pytest +``` +```` 1. If pytest is successful, add properties missing from your prior iteration. 2. Run pytest again. @@ -224,8 +266,16 @@ To handle the conversion operation, you'll override [`Tap.load_state()`](singer_ The SDK provides autogenerated markdown you can paste into your README: +````{tab} Poetry ```console poetry run tap-mysource --about --format=markdown ``` +```` + +````{tab} uv +```console +uv run tap-mysource --about --format=markdown +``` +```` This text will automatically document all settings, including setting descriptions. Optionally, paste this into your existing `README.md` file. diff --git a/docs/implementation/logging.md b/docs/implementation/logging.md index adf6f2182..c0536e924 100644 --- a/docs/implementation/logging.md +++ b/docs/implementation/logging.md @@ -85,8 +85,8 @@ If you're developing a tap or target package and would like to customize its log ``` . ├── README.md -├── poetry.lock ├── pyproject.toml +├── uv.lock └── tap_example     ├── __init__.py     ├── __main__.py diff --git a/docs/index.md_ b/docs/index.md_ new file mode 100644 index 000000000..d9dabf6b5 --- /dev/null +++ b/docs/index.md_ @@ -0,0 +1,59 @@ +# Meltano Singer SDK + +The [Meltano] Singer SDK for Taps and Targets is the fastest way to build custom data extractors and loaders! Taps and targets built on the SDK are automatically compliant with the [Singer Spec], the de-facto open source standard for extract and load pipelines, and therefore Meltano. + +If you're looking to add support to Meltano for a new data tool that would be listed on the [Meltano Hub] as a utility, check out the [Meltano EDK] (Extension Development Kit) instead. + +## Future-proof extractors and loaders, with less code + +On average, developers tell us that they write about 70% less code by using the SDK, which makes learning the SDK a great investment. Furthermore, as new features and capabilities are added to the SDK, your taps and targets can always take advantage of the latest capabilities and bug fixes, simply by updating your SDK dependency to the latest version. + +## Built by Meltano and the Singer Community + +The SDK is built with love by the [Meltano] core team and contributors, with contributions from developers across the [Singer] open source community. SDK-based plugins can easily be [integrated with Meltano](https://docs.meltano.com/tutorials/custom-extractor#add-the-plugin-to-your-meltano-project), but they can also be used in any data pipeline tool that supports the [Singer Spec](https://hub.meltano.com/singer/spec). + +The SDK project is 100% open source, licensed under the [Apache 2.0](https://en.wikipedia.org/wiki/Apache_License) permissive license. Please visit our code repo [here in GitHub](https://github.com/meltano/sdk), where you can read the [source code](https://github.com/meltano/sdk), [log an issue or feature request](https://github.com/meltano/sdk/-/issues), and [contribute back](https://github.com/meltano/sdk/issues?q=is%3Aopen+is%3Aissue+label%3A%22accepting+merge+requests%22) to the SDK. We also regularly discuss SDK topics in our [Meltano Slack](https://meltano.com/slack) community, within the [#singer-tap-development](https://meltano.slack.com/archives/C01PKLU5D1R) and [#singer-target-development](https://meltano.slack.com/archives/C01RKUVUG4S) Slack channels. + +## Developer Resources + +- [Developer Guide](dev_guide.md) +- [Guides](guides/index.md) +- [Python Tips](python_tips.md) +- [Code Samples](code_samples.md) +- [CLI Commands](cli_commands.md) +- [FAQ](faq.md) + +## Reference + +- [Reference](reference.md) +- [Implementation](implementation/index.md) +- [Typing](typing.md) +- [Capabilities](capabilities.md) + +## Advanced Concepts + +- [Incremental Replication](incremental_replication.md) +- [Parent Streams](parent_streams.md) +- [Partitioning](partitioning.md) +- [Context Object](context_object.md) +- [Stream Maps](stream_maps.md) +- [Batch](batch.md) +- [Sinks](sinks.md) +- [Testing](testing.md) + +## SDK Development + +- [Contributing](CONTRIBUTING.md) +- [Release Process](release_process.md) +- [Deprecation](deprecation.md) + +## Index and Search + +- [Index](genindex.md) +- [Search](search.md) + +[Meltano]: https://www.meltano.com +[Meltano EDK]: https://edk.meltano.com +[Meltano Hub]: https://hub.meltano.com/utilities/ +[Singer]: https://singer.io +[Singer Spec]: https://hub.meltano.com/singer/spec diff --git a/docs/python_tips.md b/docs/python_tips.md index b26cae6d3..2bac4a92e 100644 --- a/docs/python_tips.md +++ b/docs/python_tips.md @@ -1,6 +1,6 @@ # Python Tips for SDK Developers -## Tip #1: Intro to Virtual Environments, Poetry, and Pipx +## Tip #1: Intro to Virtual Environments and uv Everyone comes from a different perspective - please select the scenario that you most identify with. @@ -9,44 +9,26 @@ Everyone comes from a different perspective - please select the scenario that yo If you are completely new to the concept of virtual environments, that's great! You have nothing to "unlearn". -Poetry and Pipx will make your life easy. They basically make it so you -never have to worry about virtual environments. Pipx and Poetry take care of virtual +The uv package manager will make your life easy. It basically makes it so you +never have to worry about virtual environments. It will take care of virtual environments for you so that you don't have to worry about dependency conflicts. -- **Pipx**: Use this _instead of pip_ whenever you are installing a python - program (versus a python library). Pipx automatically creates a virtual - environment for you and automatically makes sure that the executables - contained in the python package get added to your path. -- **Poetry**: Use this when you are developing in Python. Use pipx to install poetry with - `pipx install poetry`. The SDK cookiecutter - template already sets you up for poetry. When you are - running a command with `poetry run ...`, poetry is doing the work to make - sure your command runs in the correct virtual environment behind the scenes. - This means you will automatically be running with whatever library versions you - have specified in `pyproject.toml` and/or with `poetry add ...`. If it ever feels like - your environment may be stale, you can run `poetry install` or `poetry update`. +Use `uv tool install `whenever you are installing a python +program (versus a python library). It automatically creates a virtual +environment for you and automatically makes sure that the executables +contained in the python package get added to your path. + +The SDK cookiecutter template already sets you up for uv. When you are +running a command with `uv run ...`, uv is doing the work to make +sure your command runs in the correct virtual environment behind the scenes. +This means you will automatically be running with whatever library versions you +have specified in `pyproject.toml` and/or with `uv add ...`. If it ever feels like +your environment may be stale, you can run `uv sync`. ### If you already know about virtual environments -If you are used to working with virtual environments, the challenge with pipx and poetry -is just to learn how to let these new tools do the work for you. Instead of manually -creating and managing virtual environments, these two tools automate the process for you. - -- **poetry**: Handles package management processes during **development**. - - Admittedly, there's a learning curve. Instead of `setup.py` and `requirements.txt`, - everything is managed in `pyproject.toml`. - - Adding new dependencies is performed with `poetry add ` or - `poetry add -D `. - - If version conflicts occur, relax the version constraints in `pyproject.toml` for the - libraries where the conflict is reported, then try again. - - Poetry can also publish your libraries to PyPI. -- **pipx**: Install pipx once, and then use pipx _instead of_ pip. - - If you are using poetry for development, then all other pip-installables should be - executable tools and programs, which is what pipx is designed for. - - You don't need to create a virtual environment, and you don't need to remember to - activate/deactivate the environment. For instance, you can just run `pipx install meltano` - and then directly execute `meltano` with no virtual env reference, with no - prefix to remember, and with no activation. +Congratulations! You are ahead of the game. You probably already know enough to +start using uv. ### What is an virtual environment anyway? @@ -115,9 +97,8 @@ instead of as a callable function (as in `pk = stream.primary_key()`). ### Troubleshooting -- If you are working on a SDK tap/target that uses a `poetry-core` version before v1.0.8, -you may have trouble specifying a `pip_url` in Meltano with "editable mode" (`-e path/to/package`) enabled +- If you are working on a SDK tap/target that uses a `poetry-core` version before v1.0.8 in the `build-system` table of `pyproject.toml` you may have trouble specifying a `pip_url` in Meltano with "editable mode" (`-e path/to/package`) enabled (as per [#238](https://gitlab.com/meltano/sdk/-/issues/238)). This can be resolved by upgrading -the version of `poetry-core>=1.0.8` in your `pyproject.toml`. +the version of `poetry-core>=1.0.8`. For more examples, please see the [Code Samples](./code_samples.md) page.