Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RDI in the Cloud: Add steps to set up AWS PrivateLink #1116

Open
wants to merge 4 commits into
base: preview-rc-rdi
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion content/operate/rc/databases/rdi/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ RDI encrypts all network connections with TLS. The pipeline will process data fr
Before you can create a data pipeline, you must have:

- A [Redis Cloud Pro database]({{< relref "/operate/rc/databases/create-database/create-pro-database-new" >}}) hosted on Amazon Web Services (AWS). This will be the target database.
- One supported source database, also hosted on AWS and connected to [AWS PrivateLink](https://aws.amazon.com/privatelink/):
- One supported source database, hosted on an AWS EC2 instance:
- MySQL
- Oracle
- SQL Server
Expand Down
66 changes: 54 additions & 12 deletions content/operate/rc/databases/rdi/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,60 @@ See [Prepare source databases]({{<relref "/integrate/redis-data-integration/data

See the [RDI architecture overview]({{< relref "/integrate/redis-data-integration/architecture#overview" >}}) for more information about CDC.

## Set up connectivity

To ensure that you can connect your Redis Cloud database to the source database hosted an AWS EC2 instance, you need to set up an endpoint service through AWS PrivateLink. To do this:
cmilesb marked this conversation as resolved.
Show resolved Hide resolved

1. [Create a network load balancer](#create-network-load-balancer) that will route incoming HTTP requests to your database.
1. [Create an endpoint service](#create-endpoint-service) through AWS PrivateLink.

### Create network load balancer

In the [AWS Management Console](https://console.aws.amazon.com/), use the **Services** menu to locate and select **Compute** > **EC2**. [Create a network load balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/create-network-load-balancer.html#configure-load-balancer) with the following settings:

1. In **Basic configuration**:
- **Scheme**: Select **Internet-facing**.
- **Load balancer IP address type**: Select **IPv4**.
1. In **Network mapping**, select the VPC and availability zone associated with your source database.
1. In **Security groups**, select the security group associated with your source database.
1. In **Listeners and routing**:
1. Select **Create target group** to [create a target group](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-target-group.html) with the following settings:
1. In **Specify group details**:
- **Target type**: Select **Instances**.
- **Protocol : Port**: Select **TCP**, and then enter the port number where your database is exposed.
- The **IP address type** and **VPC** should be selected already and match the VPC you selected earlier.
1. In **Register targets**, select the EC2 instance that runs your source database, enter the port, and select **Include as pending below**. Then, select **Create target group** to create your target group. Return **Listeners and routing** in the Network Load Balancer setup.
1. Set the following **Listener** properties:
- **Protocol**: Select **TCP**.
- **Port**: Enter **80**.
- **Default action**: Select the target group you created in the previous step.
1. Review the network load balancer settings, and then select **Create load balancer** to continue.
1. After the network load balancer is active, select **Security**, and then select the security group ID to open the Security group settings.
1. Select **Edit inbound rules**, then **Add rule** to add a rule with the following settings:
- **Type**: Select **HTTP**.
- **Source**: Select **Anywhere - IPv4**.
Select **Save rules** to save your changes.

### Create endpoint service

In the [AWS Management Console](https://console.aws.amazon.com/), use the **Services** menu to locate and select **Networking & Content Delivery** > **VPC**. There, select **PrivateLink and Lattice** > **Endpoint services**. [Create an endpoint service](https://docs.aws.amazon.com/vpc/latest/privatelink/create-endpoint-service.html) with the following settings:

1. In **Available load balancers**, select the [network load balancer](#create-network-load-balancer) you created.
1. In **Additional settings**, choose the following settings:
- **Require acceptance for endpoint**: Select **Acceptance required**.
- **Supported IP address types**: Select **IPv4**.
1. Select **Create** to create the endpoint service.

After you create the endpoint service, you need to add Redis Cloud as an Allowed Principal on your [endpoint service VPC permissions](https://docs.aws.amazon.com/vpc/latest/privatelink/configure-endpoint-service.html#add-remove-permissions).

1. In the Redis Cloud Console, copy the Amazon Resource Name (ARN) provided in the **Setup connectivity** section.
1. Return to the endpoint service list on the [Amazon VPC console](https://console.aws.amazon.com/vpc/). Select the endpoint service you just created.
1. Navigate to **Allow principals** tab.
1. Add the Redis Cloud ARN you copied and choose **Allow principals**.
1. Save the service name for later.

For more details on AWS PrivateLink, see [Share your services through AWS PrivateLink](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html).

## Share source database credentials

You need to share your source database credentials and certificates in an Amazon secret with Redis Cloud so that the pipeline can connect to your database.
Expand Down Expand Up @@ -105,18 +159,6 @@ If your source database has TLS or mTLS enabled, we recommend that you enter the

After you store this secret, you can view and copy the [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/secretsmanager/latest/userguide/reference_iam-permissions.html#iam-resources) of your secret on the secret details page.

## Set up connectivity

To expose your source database to Redis, you need to add Redis Cloud as an Allowed Principal on the [AWS PrivateLink VPC permissions](https://docs.aws.amazon.com/vpc/latest/privatelink/configure-endpoint-service.html#add-remove-permissions) for the PrivateLink connected to your source database.

1. Copy the Amazon Resource Name (ARN) provided in the **Setup connectivity** section.
1. Open the [Amazon VPC console](https://console.aws.amazon.com/vpc/) and select **Endpoint services**.
1. Navigate to **Allow principals** tab.
1. Add the Redis Cloud ARN and choose **Allow principals**.
1. Copy your PrivateLink service name for later.

For more details on AWS PrivateLink, see [Share your services through AWS PrivateLink](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html).


## Next steps

Expand Down