feat(rust): add the driver manager

[alexandreyc]

Hey!

Here is the penultimate PR containing the driver manager for Rust.

The last PR will contain all the integration tests.

[alexandreyc]

CC @mbrobbel

[alexandreyc]

CI is broken because we need the SQLite driver dynamic library in path to successfully run doctests... I have no idea how we can do that. Is there any similar situation elsewhere?

[lidavidm]

CI is broken because we need the SQLite driver dynamic library in path to successfully run doctests... I have no idea how we can do that. Is there any similar situation elsewhere?

You could consider DuckDB too? It supports ADBC in the DuckDB library itself so you can just download and point to it standalone without having to build anything else from this repo.

[alexandreyc]

You could consider DuckDB too? It supports ADBC in the DuckDB library itself so you can just download and point to it standalone without having to build anything else from this repo.

Yes I could do that but I also have integration tests against SQLite and PostgreSQL drivers so I ideally would prefer to build those as part of the CI.

Related question: shouldn't we release compiled library for drivers too?

[lidavidm]

Release them where? That's always the problem :)

There is native-unix.yml which is set up to build drivers in one portion of the pipeline then share them across several builds.

[alexandreyc]

Release them where? That's always the problem :)

To be honest, I haven't given the question too much thought 😃

There is native-unix.yml which is set up to build drivers in one portion of the pipeline then share them across several builds.

Nice, I'll look into it. Thanks!

[mbrobbel]

Looks good!

[lidavidm]

Sorry, this missed the last release...it'll be in the next one

[lidavidm]

@mbrobbel are we all good here?

[lidavidm]

CI LGTM, though, maybe we should think about refactoring things at some point (since not all the jobs in native-unix/native-windows need to run for Rust-only changes). Though, it's hard to do this without a unified build dependency graph, and at some point I assume Rust will feed back into the other builds...

[alexandreyc]

CI LGTM, though, maybe we should think about refactoring things at some point (since not all the jobs in native-unix/native-windows need to run for Rust-only changes). Though, it's hard to do this without a unified build dependency graph, and at some point I assume Rust will feed back into the other builds...

CI still needs some work, I haven't managed to make it work yet. I should be able to work on it in the next few days.

Yeah I agree with you that the current approach is a bit brute-force and will need refactoring at some point.

[alexandreyc]

CI is now working on macOS and Ubuntu. Unfortunately I'm not at all familiar with Windows and would need some help to figure it out.

Basically, we need to set the equivalent of LD_LIBRARY_PATH (or put the DLL to a specific location?) for the dynamic linker to find DLLs built during native-windows workflow. See: https://github.com/apache/arrow-adbc/actions/runs/9063466338/job/24900436623