v0.44.2

v0.44.2 (2025-01-31)

🐛 Fixes

• #2862 Include package license metadata conditionally in templates -- Thanks @ReubenFrankel!
• #2859 Use uv in tap, target and mapper templates

📚 Documentation Improvements

• #2861 Document plugin development with uv and how to migrate existing ones

📦 Packaging changes

• #2854 Use PEP 639
• #2852 Remove urllib3 constraint
• #2851 Use uv to manage this project

v0.44.1

v0.44.1 (2025-01-29)

🐛 Fixes

• #2847 Update Cookiecutter templates
• #2844 Avoid writing an empty state -- Thanks @joaopamaral!
• #2843 Use SQLAlchemy to generate an insert statement

v0.44.0

v0.44.0 (2025-01-23)

✨ New

• #2830 Allow developers to mark stream schema and settings fields as deprecated
• #2829 Support a x-sql-datatype JSON Schema annotation to let targets customize SQL type handling
• #2819 Add SHA256 encryption method to inline stream maps -- Thanks @ben-schulz-mh!

📦 Packaging changes

• #2407 Use Poetry support for PEP 621
• #2822 Make paramiko and transitive SSH dependencies optional
• #2821 Require urllib3 < 2 on Python < 3.10

v0.43.1

v0.43.1 (2024-12-10)

🐛 Fixes

• #2807 Allow developers to set RESTStream.http_method

v0.43.0

v0.43.0 (2024-12-10)

Highlights

This release introduces a few improvements and changes to the SDK that should make it easier to work with SQL taps, REST streams, and targets.

• A new SQL tap class attribute exclude_streams allows developers to exclude certain database schemas by default from discovery. This can speed up discovery and avoid issues when mapping types from database-internal schemas.

• Discovery for SQL taps should be about 35% faster now, thanks to using SQLAlchemy 2.0+ reflection features.

• A new target built-in setting process_activate_version_messages lets users disable processing of ACTIVATE_VERSION messages.

• Support for other content-types in REST streams with the new RESTStream.payload_as_json attribute. This should make it easier to work with APIs that expect non-JSON payloads, such as XML.

• The RESTStream.rest_method attribute is deprecated in favor of RESTStream.http_method.

Important

This release drops support for Python 3.8. Please make sure to upgrade your tap or target to support Python 3.9+.

Note

SQL taps now require SQLAlchemy 2.0+.

✨ New

• #2482 Allow SQL tap developers to auto-skip certain schemas from discovery
• #2784 Added a new built-in setting activate_version for targets to optionally disable processing of ACTIVATE_VERSION messages
• #2780 Numeric values are now parsed as decimal.Decimal in REST and GraphQL stream responses
• #2775 Log a stream's bookmark (if it's avaiable) when its sync starts
• #2703 Targets now emit record count from the built-in batch file processor
• #2774 Accept a maxLength limit for VARCHARs
• #2769 Add versioning-strategy to dependabot config of Cookiecutter templates
• #2765 The last received Singer message is now logged when the target fails
• #2762 Support other content-types in REST streams

🐛 Fixes

• #2790 Ensure the required global folder tap settings are merged into the concrete implementation settings
• #2785 Use FS-specific listdir in folder tap
• #2778 The path of the offending field is now printed for config validation errors
• #2770 Respect standard Singer stream metadata table-key-properties, replication-key and forced-replication-method
• #2755 Safely compare UUID replication keys with state bookmarks -- Thanks @nikzavada!

⚙️ Under the Hood

• #2805 Rename setting activate_version to process_activate_version_messages
• #2788 Fail early if input files to --catalog or --state do not exist
• #2781 Added a class method to instantiate SQLToJSONSchema from the tap configuration
• #2566 Standardize on JSON Schema Draft 2020-12 to validate stream schemas
• #2751 Dropped support for Python 3.8

⚡ Performance Improvements

• #2793 Improved discovery performance for SQL taps

📚 Documentation Improvements

• #2796 Document how to configure nested stream maps values with environment variables in Meltano

📦 Packaging changes

• #2797 SQL taps now require SQLAlchemy 2.0+

v0.42.1

v0.42.1 (2024-11-11)

🐛 Fixes

• #2756 Safely compare UUID replication keys with state bookmarks -- Thanks @nikzavada!

Full Changelog: v0.42.0...v0.42.1

v0.42.0

v0.42.0 (2024-11-11)

Highlights

• For SQL target developers, there is now a public and official API for overriding how JSON schema types are mapped into SQL types. Let us know if there's any improvements we can make to it to better suit your use case!
• The name of the stream is now available in stream maps as __stream_name__, including in the context of __alias__ expressions.
• This is last minor version to support Python 3.8, which reached its EOL on 2024-10-07, the next release will only support Python 3.9+.

✨ New

• #2742 Update dependencies in templates
• #2732 SQL target developers can now more easily override the mapping from JSON schema to SQL column type
• #2730 Added SQLConnector.prepare_primary_key for target to implement for custom table primary key adaptation
• #2488 Nested schema properties can now be defined as nullable
• #2518 Python 3.13 is officially supported
• #2637 Environment variables are now parsed for boolean, integer, array and object setting values
• #2699 Stream name can now be accessed in stream maps -- Thanks @holly-evans!
• #2712 JSON schema title is now supported in configuration and stream properties
• #2707 Bumped simpleeval to 1.0
• #2701 Stream name can now be accessed in __alias__ context of stream maps -- Thanks @holly-evans!

🐛 Fixes

• #2741 datetime.datetime instances in stream maps are now correctly mapped to date-time JSON schema strings
• #2727 Object and array JSON types are now handled before primitive types when converting them to SQL types
• #2723 JSON schema union types are no longer conformed into boolean values

⚙️ Under the Hood

• #2743 Deprecate passing file paths to plugin and stream initialization

📚 Documentation Improvements

• #2745 Document the current release process
• #2717 Update Meltano commands in examples

📦 Packaging changes

• #2736 Skip simpleeval 1.0.1
• #2716 Stopped testing with SQLAlchemy 1.4
• #2714 Remove constraint on urllib3

v0.41.0

v0.41.0 (2024-10-02)

Highlights

• It's easier now for SQL tap developers to customize the mapping from SQL column types to JSON schema. See the guide for details.

✨ New

• #2667 Support stream aliasing of BATCH messages via stream maps -- Thanks @ReubenFrankel!
• #2651 SQL taps now emit schemas with maxLength when applicable
• #2618 Developers can now more easily override the mapping from SQL column type to JSON schema

🐛 Fixes

• #2697 All HTTP timeout exceptions are now retried in REST and GraphQL streams
• #2683 A clear error message is now emitted when flattening is enabled but flattening_max_depth is not set
• #2665 Mapped datetime values are now typed as date-time strings in the schema message -- Thanks @gregkoutsimp!
• #2663 Properties dropped using None or __NULL__ in stream maps are now also removed from the schema required array

⚙️ Under the Hood

• #2696 Use tox without installing Poetry explicitly in workflows
• #2654 Added a generic FileStream (still in active development!)
• #2695 Update dependencies in templates
• #2661 Drop support for Python 3.8 in templates
• #2670 Deprecated Faker class in stream maps
• #2666 Remove non-functional record-flattening capability -- Thanks @ReubenFrankel!
• #2652 Renamed SQLConnector.type_mapping to SQLConnector.sql_to_jsonschema
• #2647 Use future warnings.deprecated decorator

📚 Documentation Improvements

• #2658 Added more versions when stuff changed or was added

📦 Packaging changes

• #2694 Removed unused backport importlib_resources dependency in tap template
• #2664 Added a constraint on setuptools <= 70.3.0 to fix incompatibility with some dependencies

v0.40.0

v0.40.0 (2024-09-02)

Highlights

• Targets now emit basic metrics. There's nothing to do on your end, they're enabled by default!
• The Faker class is available in stream maps context. Note that like the faker instance added in v0.35.0, this requires installing the faker library in the tap/target environment. For example, with tap-example[faker].

✨ New

• #2486 Emit target metrics
• #2567 A new schema_is_valid built-in tap test validates stream schemas against the JSON Schema specification
• #2598 Stream map expressions now have access to the Faker class, rather than just a faker instance
• #2549 Added a default user agent for REST and GraphQL taps

🐛 Fixes

• #2613 Mismatch between timezone-aware and naive datetimes in start date and bookmarks is now correctly handled

⚙️ Under the Hood

• #2628 Use context manager to read gzip batch files
• #2619 Default to UTC when parsing dates without a known timezone
• #2603 Backwards-compatible identifier quoting in fully qualified names
• #2601 Improved SQL identifier (de)normalization
• #2599 Remove pytest-durations dependency from testing and use native pytest option --durations
• #2597 Mark pagination classes with @override decorator
• #2596 Made auth_headers and auth_params of APIAuthenticatorBase simple instance attributes instead of decorated properties

📚 Documentation Improvements

• #2639 Documented versions where fake and Faker objects were added to the stream maps context
• #2629 Reference get_starting_timestamp in incremental replication guide
• #2604 Update project sample links
• #2595 Documented examples of stream glob expressions and property aliasing

📦 Packaging changes

• #2640 Remove upper constraint on faker extra

v0.39.1

v0.39.1 (2024-08-07)

• PyPI 🐍

🐛 Fixes

• #2589 Make sink assertion compatible with stream maps -- Thanks @JCotton1123!
• #2592 Fixed typos in --about plain text output
• #2580 Date fields are now properly serialized as ISO dates, i.e. "YYYY-MM-DD"
• #2583 Quote add-column-ddl with column starting with _ (underscore) based on engine -- Thanks @haleemur!
• #2582 DDL for adding a column now uses a SQLAlchemy-compiled clause to apply proper quoting -- Thanks @haleemur!
• #2418 Check replication method instead of key to determine if a SQL stream is sorted

⚙️ Under the Hood

• #2520 Remove unused dependencies pendulum and python-dateutil

📚 Documentation Improvements

• #2579 Documented support for packaging semantic type in contributing guide