Skip to content

Commit

Permalink
Bumping version numbers for 6.4.1 patch release.
Browse files Browse the repository at this point in the history
  • Loading branch information
bsharifi committed Jan 6, 2025
1 parent 2b36847 commit f2a6443
Show file tree
Hide file tree
Showing 3 changed files with 13 additions and 6 deletions.
7 changes: 7 additions & 0 deletions CHANGELOG
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,13 @@

# spark-redshift Changelog

## 6.4.1 (2025-01-07)
- Corrects an attribute nullability bug when using set operators: UNION, EXCEPT, or INTERSECT [Hari Kishore Chaparala, Beaux Sharifi]
- Adds a missing validation rule for using the Redshift Data API [Beaux Sharifi]
- Modifies the visibility of RedshiftRelation to be public for external access [Armin Najafi]
- Upgrades the connector to work with the latest JDBC driver version 2.1.0.32 [Beaux Sharifi]
- Validates connector tests pass with the latest Spark patch release 3.5.4 [Beaux Sharifi]

## 6.4.0 (2024-12-02)
- Supports utilizing the Redshift Data API as an alternative mechanism for communicating with Redshift (see parameters "data_api_*" in the README). [Beaux Sharifi]
- Adds DML pushdown support for four Spark SQL operators: INSERT, DELETE, UPDATE, and MERGE. [Hari Kishore Chaparala, Ruei Yang Huang, Xiaoxuan Li, Beaux Sharifi, Brooke White]
Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ You may use this library in your applications with the following dependency info
spark-submit \
--deploy-mode cluster \
--master yarn \
--packages com.amazon.redshift:redshift-jdbc42:2.1.0.32,org.apache.spark:spark-avro_2.12:3.5.4,io.github.spark-redshift-community:spark-redshift_2.12:6.4.0-spark_3.5 \
--packages com.amazon.redshift:redshift-jdbc42:2.1.0.32,org.apache.spark:spark-avro_2.12:3.5.4,io.github.spark-redshift-community:spark-redshift_2.12:6.3.0-spark_3.5 \
my_script.py
```

Expand All @@ -115,14 +115,14 @@ You may use this library in your applications with the following dependency info
<dependency>
<groupId>io.github.spark-redshift-community</groupId>
<artifactId>spark-redshift_2.12</artifactId>
<version>6.4.0-spark_3.5</version>
<version>6.3.0-spark_3.5</version>
</dependency>
```

- **In SBT**:

```SBT
libraryDependencies += "io.github.spark-redshift-community" %% "spark-redshift_2.12" % "6.4.0-spark_3.5"
libraryDependencies += "io.github.spark-redshift-community" %% "spark-redshift_2.12" % "6.3.0-spark_3.5"
```


Expand Down Expand Up @@ -887,7 +887,7 @@ for more information.</p>
<td>""</td>
<td>
An identifier to include in the query group set when running queries with the connector. Should be 100 or fewer characters and all characters must be valid unicodeIdentifierParts. Characters in excess of 100 will be trimmed.
When running a query with the connector a json formatted string will be set as the query group (for example `{"spark-redshift-connector":{"svc":"","ver":"6.4.0-spark_3.5","op":"Read","lbl":"","tid":""}}`).
When running a query with the connector a json formatted string will be set as the query group (for example `{"spark-redshift-connector":{"svc":"","ver":"6.4.1-spark_3.5","op":"Read","lbl":"","tid":""}}`).
This option will be substituted for the value of the `lbl` key.
</td>
</tr>
Expand Down Expand Up @@ -1044,7 +1044,7 @@ SET spark.datasource.redshift.community.autopushdown.lazyMode=false
### trace_id
A new tracing identifier field that is added to the existing `label` parameter. When set, the provided string value will be used as part of label. Otherwise, it will default to the Spark application identifier. For example:
`{"spark-redshift-connector":{"svc":"","ver":"6.4.0-spark_3.5","op":"Read","lbl":"","tid":"..."}}`)
`{"spark-redshift-connector":{"svc":"","ver":"6.4.1-spark_3.5","op":"Read","lbl":"","tid":"..."}}`)
To set the value, run the following command:
```sparksql
Expand Down
2 changes: 1 addition & 1 deletion version.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@
* limitations under the License.
*/

ThisBuild / version := "6.4.0"
ThisBuild / version := "6.4.1"

0 comments on commit f2a6443

Please sign in to comment.