Introduce SOCKS5 proxy support #1180

vbabanin · 2023-08-16T22:49:36Z

Summary:

The MongoDB community has expressed the need for SOCKS5 proxy support. This PR introduces support for SOCKS5 proxy configuration for the synchronous version of the driver.

Changes:

Added support for configuring SOCKS5 proxy settings in MongoClientSettings, AutoEncryptionSettings and ClientEncryptionSettings.
Added SocksSocket which implements RFC19128 and RFC1929

Impact:

This feature enhances the driver's versatility and expands its usability in diverse networking scenarios. With this addition, users facing network restrictions, such as those accessing MongoDB deployments behind firewalls, can now seamlessly connect and interact with their databases.

Related links:

Specification
JAVA-4347

…oDB instances behind firewalls. JAVA-4347

vbabanin · 2023-08-16T22:50:18Z

driver-core/src/main/com/mongodb/internal/connection/SslHelper.java

@@ -51,6 +58,27 @@ public static void enableSni(final String host, final SSLParameters sslParameter
        }
    }

+    public static void configureSslSocket(final Socket socket, final SslSettings sslSettings, final InetSocketAddress inetSocketAddress) throws


The code of this method has been relocated without any alterations from its original location in SocketStreamHelper.java#L9

JAVA-4347

stIncMale

Static checks fail. This can be addressed before the whole PR is reviewed.

JAVA-4347

jyemin · 2023-08-17T21:43:21Z

All checks are now passing now. @vbabanin can you manually reconfigure to run the new socks tasks?

vbabanin · 2023-08-17T22:32:04Z

All checks are now passing now. @vbabanin can you manually reconfigure to run the new socks tasks?

Done, I have added test-socks5 task with two build variants included.

stIncMale

I have not yet reviewed changes in the following files:

.evergreen/.evg.yml
.evergreen/run-socks5-tests.sh
driver-core/src/main/com/mongodb/internal/connection/SocketStream.java
driver-sync/src/test/functional/com/mongodb/client/Socks5ProseTest.java

driver-core/src/main/com/mongodb/ConnectionString.java

config/spotbugs/exclude.xml

driver-reactive-streams/src/main/com/mongodb/reactivestreams/client/MongoClients.java

driver-core/src/main/com/mongodb/connection/SocketSettings.java

driver-core/src/main/com/mongodb/internal/connection/SocksSocket.java

.evergreen/run-socks5-tests.sh

driver-core/src/main/com/mongodb/AutoEncryptionSettings.java

driver-core/src/main/com/mongodb/internal/connection/SocketStream.java

stIncMale · 2023-08-21T17:46:50Z

driver-core/src/main/com/mongodb/internal/connection/SocksSocket.java

+            if (socket != null && !socket.isConnected()) {
+                socket.connect(proxyAddress, remainingMillis(timeout));
+                inputStream = socket.getInputStream();
+                outputStream = socket.getOutputStream();
+            } else {
+                super.connect(proxyAddress, remainingMillis(timeout));
+                inputStream = getInputStream();
+                outputStream = getOutputStream();
+            }


When we call Socket.connect from SocketStream.initializeSocket for a scenario without a proxy, we try in a loop all InetSocketAddresses to which ServerAddress is resolved.

When we call socket.connect/super.connect here while connecting to a proxy, we don't do the same. Instead, we pick a single InetSocketAddress to which ProxySettings.getHost&getPort may be resolved, and do only one attempt.

It seems to me that we should try all IP addresses corresponding to the proxy host when connecting to a proxy, similarly to what we do when we connect to MongoDB directly.

It is been decided to handle this in a separate DRIVERS ticket because it requires spec changes. I am going to create one.

UPD: DRIVERS-2719

driver-core/src/main/com/mongodb/internal/connection/SocketStream.java

driver-sync/src/test/functional/com/mongodb/client/Socks5ProseTest.java

stIncMale · 2023-08-21T18:57:01Z

I have now reviewed all the changes in the PR.

Co-authored-by: Valentin Kovalenko <[email protected]>

…et.java Co-authored-by: Valentin Kovalenko <[email protected]>

PR fixes. JAVA-4347

jyemin

Sorry for the late review, but I found some things that may need addressing.

.evergreen/.evg.yml

driver-sync/src/test/functional/com/mongodb/client/Socks5ProseTest.java

.evergreen/run-socks5-tests.sh

driver-core/src/main/com/mongodb/ConnectionString.java

driver-core/src/test/functional/com/mongodb/client/model/AggregatesTest.java

driver-sync/src/test/functional/com/mongodb/client/Socks5ProseTest.java

JAVA-4347

- Eliminate KMIP server configuration from SOCKS5 tests. - Organize SOCKS5 proxy tests into separate tasks for authenticated and non-authenticated scenarios. - Enhance unit test coverage. JAVA-4347

JAVA-4347

driver-core/src/main/com/mongodb/ConnectionString.java

driver-reactive-streams/src/main/com/mongodb/reactivestreams/client/MongoClients.java

driver-core/src/main/com/mongodb/connection/ProxySettings.java

driver-scala/src/main/scala/org/mongodb/scala/connection/ProxySettings.scala

driver-scala/src/main/scala/org/mongodb/scala/connection/package.scala

driver-core/src/main/com/mongodb/connection/SocketSettings.java

driver-core/src/main/com/mongodb/connection/ProxySettings.java

driver-core/src/main/com/mongodb/connection/SocketSettings.java

…lient/MongoClients.java Co-authored-by: Valentin Kovalenko <[email protected]>

Co-authored-by: Valentin Kovalenko <[email protected]>

JAVA-4347

…sult report. JAVA-4347

jyemin

LGTM for all my change requests.

jyemin · 2023-09-05T21:33:59Z

.evergreen/.evg.yml

-  matrix_spec: { os: "linux", ssl: ["nossl", "ssl"], version: [ "latest" ], topology: ["replicaset"] }
-  display_name: "Socks5: ${version} ${topology} ${ssl} ${jdk} ${os}"
+  matrix_spec: { os: "linux", ssl: ["nossl", "ssl"], version: [ "latest" ], topology: ["replicaset"], socks_auth: ["auth", "noauth"] }
+  display_name: "${socks_auth} SOCKS5 proxy: ${version} ${topology} ${ssl} ${jdk} ${os}"


Let's move socks_auth after SOCKS5 proxy so that they sort together in the display

driver-sync/src/test/functional/com/mongodb/client/Socks5ProseTest.java

driver-core/src/main/com/mongodb/internal/connection/SocksSocket.java

Co-authored-by: Valentin Kovalenko <[email protected]>

JAVA-4347

Introduce SOCKS5 proxy support, allowing seamless connections to Mong…

311515e

…oDB instances behind firewalls. JAVA-4347

vbabanin self-assigned this Aug 16, 2023

vbabanin commented Aug 16, 2023

View reviewed changes

vbabanin changed the title ~~Introduce SOCKS5 proxy support, allowing seamless connections to Mong…~~ Introduce SOCKS5 proxy support Aug 16, 2023

Include CSFLE test suites to run via Socks5 proxy.

07a567d

JAVA-4347

vbabanin marked this pull request as ready for review August 17, 2023 01:35

vbabanin requested review from stIncMale and jyemin August 17, 2023 01:36

stIncMale requested changes Aug 17, 2023

View reviewed changes

vbabanin added 2 commits August 17, 2023 08:14

Remove extra MONGODB_URI variable.

0231ae6

JAVA-4347

Remove unused imports.

53c7973

JAVA-4347

vbabanin requested a review from stIncMale August 17, 2023 21:15

stIncMale requested changes Aug 19, 2023

View reviewed changes

jyemin reviewed Aug 21, 2023

View reviewed changes

.evergreen/run-socks5-tests.sh Outdated Show resolved Hide resolved

driver-core/src/main/com/mongodb/AutoEncryptionSettings.java Outdated Show resolved Hide resolved

stIncMale requested changes Aug 21, 2023

View reviewed changes

vbabanin and others added 7 commits August 24, 2023 12:03

Update driver-core/src/main/com/mongodb/connection/SocketSettings.java

81020fb

Co-authored-by: Valentin Kovalenko <[email protected]>

Update driver-core/src/main/com/mongodb/ClientEncryptionSettings.java

1313753

Co-authored-by: Valentin Kovalenko <[email protected]>

Update driver-core/src/main/com/mongodb/AutoEncryptionSettings.java

37e0874

Co-authored-by: Valentin Kovalenko <[email protected]>

Update driver-core/src/main/com/mongodb/connection/ProxySettings.java

3790094

Co-authored-by: Valentin Kovalenko <[email protected]>

Apply suggestions from code review

494e293

Co-authored-by: Valentin Kovalenko <[email protected]>

Update driver-core/src/main/com/mongodb/internal/connection/SocksSock…

de7697d

…et.java Co-authored-by: Valentin Kovalenko <[email protected]>

Revert proxy settings for key Management Service.

3e84f5c

PR fixes. JAVA-4347

vbabanin requested a review from stIncMale August 31, 2023 06:21

jyemin requested changes Sep 1, 2023

View reviewed changes

vbabanin added 3 commits September 1, 2023 14:30

Remove KMIP config.

08d3e30

JAVA-4347

Restructuring testing:

ba111ca

- Eliminate KMIP server configuration from SOCKS5 tests. - Organize SOCKS5 proxy tests into separate tasks for authenticated and non-authenticated scenarios. - Enhance unit test coverage. JAVA-4347

Fix checkstyle errors.

db8d91b

JAVA-4347

stIncMale reviewed Sep 5, 2023

View reviewed changes

driver-core/src/main/com/mongodb/ConnectionString.java Outdated Show resolved Hide resolved

driver-core/src/main/com/mongodb/ConnectionString.java Show resolved Hide resolved

stIncMale reviewed Sep 5, 2023

View reviewed changes

vbabanin and others added 5 commits September 5, 2023 11:35

Update driver-reactive-streams/src/main/com/mongodb/reactivestreams/c…

ea1060c

…lient/MongoClients.java Co-authored-by: Valentin Kovalenko <[email protected]>

Update driver-core/src/main/com/mongodb/connection/ProxySettings.java

7b93565

Co-authored-by: Valentin Kovalenko <[email protected]>

Update driver-core/src/main/com/mongodb/connection/ProxySettings.java

e7218a8

Co-authored-by: Valentin Kovalenko <[email protected]>

Update driver-core/src/main/com/mongodb/connection/ProxySettings.java

a56d56c

Co-authored-by: Valentin Kovalenko <[email protected]>

Apply suggestions from code review

dcac4f6

Co-authored-by: Valentin Kovalenko <[email protected]>

vbabanin requested a review from stIncMale September 5, 2023 19:52

vbabanin added 4 commits September 5, 2023 12:53

- Change javadoc.

7eb1c11

JAVA-4347

Add javadoc.

19f2dca

JAVA-4347

Remove redundant code.

735e81e

JAVA-4347

Add @EnabledIf annotation for tests to remove skipped tests from a re…

34a26ce

…sult report. JAVA-4347

vbabanin requested a review from jyemin September 5, 2023 21:27

jyemin approved these changes Sep 5, 2023

View reviewed changes

stIncMale reviewed Sep 6, 2023

View reviewed changes

driver-core/src/main/com/mongodb/internal/connection/SocksSocket.java Outdated Show resolved Hide resolved

driver-core/src/main/com/mongodb/internal/connection/SocksSocket.java Show resolved Hide resolved

vbabanin and others added 3 commits September 6, 2023 11:40

Apply suggestions from code review

ede85de

Co-authored-by: Valentin Kovalenko <[email protected]>

Remove redundant code.

840d12b

JAVA-4347

Add size check in domain regex.

7d461a0

JAVA-4347

vbabanin requested a review from stIncMale September 6, 2023 21:44

Address checkstyle warnings.

42aca98

JAVA-4347

stIncMale approved these changes Sep 7, 2023

View reviewed changes

vbabanin merged commit d3008cc into mongodb:master Sep 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce SOCKS5 proxy support #1180

Introduce SOCKS5 proxy support #1180

vbabanin commented Aug 16, 2023 •

edited

Loading

vbabanin Aug 16, 2023

stIncMale left a comment

jyemin commented Aug 17, 2023

vbabanin commented Aug 17, 2023

stIncMale left a comment

stIncMale Aug 21, 2023

vbabanin Aug 31, 2023 •

edited

Loading

stIncMale commented Aug 21, 2023

jyemin left a comment

jyemin left a comment

jyemin Sep 5, 2023

Introduce SOCKS5 proxy support #1180

Introduce SOCKS5 proxy support #1180

Conversation

vbabanin commented Aug 16, 2023 • edited Loading

Summary:

Changes:

Impact:

Related links:

vbabanin Aug 16, 2023

Choose a reason for hiding this comment

stIncMale left a comment

Choose a reason for hiding this comment

jyemin commented Aug 17, 2023

vbabanin commented Aug 17, 2023

stIncMale left a comment

Choose a reason for hiding this comment

stIncMale Aug 21, 2023

Choose a reason for hiding this comment

vbabanin Aug 31, 2023 • edited Loading

Choose a reason for hiding this comment

stIncMale commented Aug 21, 2023

jyemin left a comment

Choose a reason for hiding this comment

jyemin left a comment

Choose a reason for hiding this comment

jyemin Sep 5, 2023

Choose a reason for hiding this comment

vbabanin commented Aug 16, 2023 •

edited

Loading

vbabanin Aug 31, 2023 •

edited

Loading