Skip to content

Commit

Permalink
Add Snowflake to native-image
Browse files Browse the repository at this point in the history
This change adds Snowflake's jdbc library packaged in a fat jar.

It's not a straightword inclusion mostly due to the complicated way how
the jar itself is constructed:
- it's a fat jar so includes other dependencies which we sometimes
  include
- they are patching some of the dependencies and putting them under
  their subpackage making it impossible to use the `snowflake-jdbc-thin`
dependency
- a number of configs are spurious and need to be disabled
- `FileTypeDetector` provider leads to runtime-/build-time
  initialization confict that is impossible to workaround without
disabling it
- need to open `java.base` module, similarly to #9664, due to Arrow's
  usage
- `java.nio.DirectByteBuffer` constructor changed its signature in JDK21  from
  `(long, int)` to `(long, long)` which caused problems for the
constructed NI and needed to be reflected in the config
  • Loading branch information
hubertp committed Jan 30, 2025
1 parent 5c07590 commit cb6fa59
Show file tree
Hide file tree
Showing 3 changed files with 1,298 additions and 3 deletions.
19 changes: 16 additions & 3 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -3790,6 +3790,9 @@ lazy val `engine-runner` = project
`database-polyglot-root`.listFiles("*.jar").map(_.getAbsolutePath()) ++
`std-aws-polyglot-root`.listFiles("*.jar").map(_.getAbsolutePath()) ++
`std-microsoft-polyglot-root`
.listFiles("*.jar")
.map(_.getAbsolutePath()) ++
`std-snowflake-polyglot-root`
.listFiles("*.jar")
.map(_.getAbsolutePath())

Expand Down Expand Up @@ -3870,13 +3873,17 @@ lazy val `engine-runner` = project
// which breaks all our class loading. We still want to run `SqliteJdbcFeature` which extracts a proper
// native library from the jar.
excludeConfigs = Seq(
s".*sqlite-jdbc-.*\\.jar,META-INF/native-image/org\\.xerial/sqlite-jdbc/native-image\\.properties"
s".*sqlite-jdbc-.*\\.jar,META-INF/native-image/org\\.xerial/sqlite-jdbc/native-image\\.properties",
s".*snowflake-jdbc-.*\\.jar,META-INF/native-image/.*"
),
additionalOptions = Seq(
"-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.NoOpLog",
"-H:IncludeResources=.*Main.enso$",
"-H:+AddAllCharsets",
"-H:+IncludeAllLocales",
// Workaround a problem with build-/runtime-initialization conflict
// by disabling this service provider
"-H:ServiceLoaderFeatureExcludeServiceProviders=net.snowflake.client.core.FileTypeDetector",
// useful perf & debug switches:
// "-g",
// "-H:+SourceLevelDebug",
Expand All @@ -3887,7 +3894,9 @@ lazy val `engine-runner` = project
// "--verbose",
"-Dnic=nic",
"-Dorg.enso.feature.native.lib.output=" + (engineDistributionRoot.value / "bin"),
"-Dorg.sqlite.lib.exportPath=" + (engineDistributionRoot.value / "bin")
"-Dorg.sqlite.lib.exportPath=" + (engineDistributionRoot.value / "bin"),
// Snowflake uses Apache Arrow (equivalent of #9664 in native-image setup)
"--add-opens=java.base/java.nio=ALL-UNNAMED"
),
mainClass = Some("org.enso.runner.Main"),
initializeAtRuntime = Seq(
Expand All @@ -3911,7 +3920,11 @@ lazy val `engine-runner` = project
"org.enso.table",
"org.enso.database",
"org.eclipse.jgit",
"com.amazonaws"
"com.amazonaws",
"com.google",
"io.grpc",
"io.opencensus",
"net.snowflake.client"
)
)
}
Expand Down
Loading

0 comments on commit cb6fa59

Please sign in to comment.