Skip to content

Process data from postgres database to datalake with sqlx and datafusion

Notifications You must be signed in to change notification settings

cj-zhukov/demodb-to-datalake

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

35 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

demodb-to-datalake

demodb-to-datalake is a Rust library for processing data from postgres database into Data Lake.

Description

demodb-to-datalake reads data from postgres database table into datafusion dataframe and saves it as parquet file. Demonstration Database demo can be downloaded from here https://postgrespro.com/community/demodb

Installation

Use the package manager cargo to install demodb-to-datalake.

About

Process data from postgres database to datalake with sqlx and datafusion

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages