feat: Kubernetes materialization engine written based on bytewax (#4087)

* feat: Kubernetes materialization engine written based on bytewax Signed-off-by: Harry <[email protected]> * fix: Resolve incorrect path Signed-off-by: Harry <[email protected]> * fix: Simplify engine name Signed-off-by: Harry <[email protected]> --------- Signed-off-by: Harry <[email protected]>
feast-dev · Apr 11, 2024 · 7617bdb · 7617bdb
1 parent 318a2b8
commit 7617bdb
Show file tree

Hide file tree

Showing 11 changed files with 707 additions and 4 deletions.
diff --git a/sdk/python/feast/infra/materialization/kubernetes/Dockerfile b/sdk/python/feast/infra/materialization/kubernetes/Dockerfile
@@ -0,0 +1,22 @@
+FROM python:3.9-slim-bullseye AS build
+
+RUN apt-get update && \
+    apt-get install --no-install-suggests --no-install-recommends --yes git
+
+WORKDIR /app
+
+COPY sdk/python/feast/infra/materialization/kuberentes/main.py /app
+
+# Copy necessary parts of the Feast codebase
+COPY sdk/python sdk/python
+COPY protos protos
+COPY go go
+COPY setup.py setup.py
+COPY pyproject.toml pyproject.toml
+COPY README.md README.md
+
+# We need this mount thingy because setuptools_scm needs access to the
+# git dir to infer the version of feast we're installing.
+# https://github.com/pypa/setuptools_scm#usage-from-docker
+# I think it also assumes that this dockerfile is being built from the root of the directory.
+RUN --mount=source=.git,target=.git,type=bind pip3 install --no-cache-dir '.[aws,gcp,k8s,snowflake,postgres]'
diff --git a/sdk/python/feast/infra/materialization/kubernetes/__init__.py b/sdk/python/feast/infra/materialization/kubernetes/__init__.py