Skip to content

Commit

Permalink
[DBZ-PGYB] Get the latest custom JDBC sink image while building docke…
Browse files Browse the repository at this point in the history
…r image (#129)

This PR only changes the link in the `Dockerfile` to fetch the latest
custom sink connector jar from GitHub.

According to PR yugabyte/kafka-connect-jdbc#3,
changes include the following:
1. Addition of 3 new configuration properties 
    * `log.table.balance`:
i. Default is `false` but when set to `true`, the sink connector will
execute a query to get the table balance from the target table.
ii. Note that this is only applicable for consistency related tests
where the given query is applicable - it will fail if set in any other
tests.
    * `expected.total.balance`
i. Default is `1000000` (1M) which can be changed to whatever value we
are expecting the total balance to be in the target table.
    * `tables.for.balance`
i. This takes a comma separated string value for all the table names
from which the sink connector is supposed to extract balances from.
ii. This property will only be valid when `log.table.balance` is
specified as `true`
iii. There is no default for this property so if `log.table.balance` is
set to `true` and `tables.for.balance` is not specified then we will
throw a `RuntimeException`
2. Log additions to aid debugging.
  • Loading branch information
vaibhav-yb authored Jun 3, 2024
1 parent 17aa8a3 commit d720158
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ RUN curl -sLo logredactor-metrics-1.0.12.jar https://repo1.maven.org/maven2/io/c
WORKDIR /

# Add the required jar files to be packaged with the base connector
RUN cd $KAFKA_CONNECT_PLUGINS_DIR/debezium-connector-postgres && curl -sLo kafka-connect-jdbc-10.6.5.jar https://github.com/yugabyte/kafka-connect-jdbc/releases/download/10.6.5-CUSTOM.4/kafka-connect-jdbc-10.6.5-CUSTOM.4.jar
RUN cd $KAFKA_CONNECT_PLUGINS_DIR/debezium-connector-postgres && curl -sLo kafka-connect-jdbc-10.6.5-CUSTOM.6.jar https://github.com/yugabyte/kafka-connect-jdbc/releases/download/10.6.5-CUSTOM.6/kafka-connect-jdbc-10.6.5-CUSTOM.6.jar
RUN cd $KAFKA_CONNECT_PLUGINS_DIR/debezium-connector-postgres && curl -sLo jdbc-yugabytedb-42.3.5-yb-1.jar https://repo1.maven.org/maven2/com/yugabyte/jdbc-yugabytedb/42.3.5-yb-1/jdbc-yugabytedb-42.3.5-yb-1.jar
RUN cd $KAFKA_CONNECT_PLUGINS_DIR/debezium-connector-postgres && curl -sLo transforms-for-apache-kafka-connect-1.5.0.zip https://github.com/Aiven-Open/transforms-for-apache-kafka-connect/releases/download/v1.5.0/transforms-for-apache-kafka-connect-1.5.0.zip
RUN cd $KAFKA_CONNECT_PLUGINS_DIR/debezium-connector-postgres && unzip transforms-for-apache-kafka-connect-1.5.0.zip
Expand Down

0 comments on commit d720158

Please sign in to comment.