HDFS-Native

A rust wrapper over libhdfs3 developed by Apache HAWQ

Setup

1. Install libhdfs3

You can either install it via Conda

conda install -c conda-forge libhdfs3

or build it from source

# A specific version that could be compiled on osx
git clone https://github.com/ClickHouse-Extras/libhdfs3.git
cd libhdfs3
git checkout 24b058c356794ef6cc2d31323dc9adf0386652ff

# then build it
mkdir build && cd build
../bootstrap --prefix=/usr/local
make
make install

Configuration

# client conf to use, env LIBHDFS3_CONF or hdfs-client.xml in working directory
export LIBHDFS3_CONF=/path/to/libhdfs3-hdfs-client.xml