Skip to content

Commit

Permalink
vdk-specs: add architecture.md
Browse files Browse the repository at this point in the history
The goal of the architecture file is to provide a comprehensive overview
of the VDK's architecture, components, and their relationships, serving
as a reference document for developers, contributors, stakeholders, and
users to understand the system's structure and make informed decisions.

The diagrams are based on C4 model https://c4model.com

The architecture is based on OnePage documents and we (Yoan and I) used
the following architecture overviews as an example of good (data
related) open source project arch documents:

- https://argoproj.github.io/argo-workflows/architecture/
- https://docs.getdbt.com/docs/cloud/about-cloud/architecture
- https://airflow.apache.org/
docs/apache-airflow/stable/core-concepts/overview.html
- https://docs.airbyte.com/understanding-airbyte/high-level-view/
- https://docs.metaflow.org/internals/technical-overview
- https://docs.dagster.io/deployment/overview
- https://docs.flyte.org/en/latest/concepts/architecture.html

Closes #858

Signed-off-by: Antoni Ivanov <[email protected]>
  • Loading branch information
antoniivanov committed Jun 15, 2023
1 parent afbf518 commit 4904278
Show file tree
Hide file tree
Showing 2 changed files with 2,941 additions and 2,985 deletions.
2 changes: 1 addition & 1 deletion specs/architecture/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ It follows plugin-oriented architecture where all functionality is built through

## VDK Control Service Container Diagram

![VDK Control Service Container Diagram](https://github.com/vmware/versatile-data-kit/assets/2536458/89312655-7171-432a-8147-551dd2d711ea)
![VDK Control Service Container Diagram](https://github.com/vmware/versatile-data-kit/assets/2536458/94538ffb-e9a8-4a61-86ff-0a5b5d8a374a)

Diving deeper into the VDK Control Service, the entry point from data practitioner's perspective is
* VDK Frontend UI which is used for operating and monitoring of data jobs.
Expand Down
Loading

0 comments on commit 4904278

Please sign in to comment.