v1.6

Major features include:

vdk-oracle database plugin

A new oracle plugin can be used to execute queries against Oracle DB in both thick and thin mode.
Ingesting data is now supported including with automatic shema inference.

To see more information check the vdk-oracle plugin documentation

vdk-structlog

Various enhancements in VDK-Structlog, including syslog handler support, log level parsing, and configuration updates

Check out more about vdk-structlog in its documentation

VDK Ingestion into Vector Database for RAG initiative started

With the rise in popularity of LLMs and RAG we see VDK as a core component to getting the data where we need it to be. VDK's strengths are ETL tasks. We see that its very well suited to populating the databases needed for RAG.

For more information check out the VEP

What's Changed

• control-service: Add support for vdk version by @doks5 in #2943
• control-service: base job image - update README.md by @mivanov1988 in #2971
• control-service: fix job builder secure by @mivanov1988 in #2976
• control-service: getting all jobs should now be consistent by @mrMoZ1 in #2978
• control-service: oracle support for vdk-oracle plugin by @mivanov1988 in #3042
• control-service: write resources to db by @mrMoZ1 in #2897
• documentation: Update to READMEs: Add a monthly download badge by @chrfoyer in #2983
• examples: add Confluence data retrieval example Data Job by @yonitoo in #3040
• frontend: [SC-2632] Save Executions grid state by @hzhristova in #2991
• specs: update Vector DB Ingestion VEP by @antoniivanov in #3033
• vdk-control-cli: Modify how data is written to job config.ini by @doks5 in #2982
• vdk-core: Short job summary by @gageorgiev in #3038
• vdk-core: replace deprecated imp module with importlib by @yonitoo in #2981
• vdk-core: replace report_and_rethrow by @DeltaMichael in #3056
• vdk-impala: handle decorate operation errors by @DeltaMichael in #2975
• vdk-jupyter: e2e tests fix by @duyguHsnHsn in #2966
• vdk-jupyter: update npm packages by @antoniivanov in #3043
• vdk-kerberos-auth: fix dependency issue by @murphp15 in #2969
• vdk-license: change year by @duyguHsnHsn in #2987
• vdk-lineage: pin sqllineage version by @duyguHsnHsn in #3005
• vdk-oracle: add new config options (host,port,sid,thick_mode_lib_dir) by @antoniivanov in #3032
• vdk-oracle: escape special chars in column names by @DeltaMichael in #3045
• vdk-oracle: fixes in tests and passing secrets by @antoniivanov in #3031
• vdk-oracle: set thik mode default to true by @antoniivanov in #3037
• vdk-oracle: support secrets to connect to database by @DeltaMichael in #2961
• vdk-oracle: support thick mode by default by @DeltaMichael in #2970
• vdk-plugins: replace report_and_rethrow by @DeltaMichael in #3057
• vdk-specs: Ingestion into vector db for RAG use by @murphp15 in #3004
• vdk-structlog: Document log level module config var by @gageorgiev in #2984
• vdk-structlog: add syslog handler by @duyguHsnHsn in #2985
• vdk-structlog: filter vdk_step_name and vdk_step_type correctly by @DeltaMichael in #2968
• vdk-structlog: rename structlog configs by @DeltaMichael in #3002
• vdk-structlog: support parse log level module and logger config granularity by @yonitoo in #2980
• vdk-structlog: update README.md by @DeltaMichael in #3047
• vdk-structlog: update VEP-2448 to reflect current state of development by @DeltaMichael in #3035

New Contributors

• @chrfoyer made their first contribution in #2983

Full Changelog: v1.5...v1.6