Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubernetes support v0 #2

Merged
merged 145 commits into from
Jun 13, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
145 commits
Select commit Hold shift + click to select a range
422489d
add k8s structure
DominicBurkart Sep 29, 2020
a7c208c
make declare async
DominicBurkart Sep 29, 2020
fcd2f51
add docker test framework for k8s
DominicBurkart Sep 29, 2020
5d8cfc0
use docker suite to run example tests, use rust suite for linting
DominicBurkart Sep 30, 2020
5bc2506
lints: make sure that subprojects are formatted correctly
DominicBurkart Sep 30, 2020
4544620
bug fix: make sure that distributed feature is actually being triggered
DominicBurkart Sep 30, 2020
a770b71
add k8s example structure
DominicBurkart Sep 30, 2020
daa3870
remove unimplemented service feature
DominicBurkart Sep 30, 2020
e2fbf29
bug fix: make sure that we only return cachable values from cached fu…
DominicBurkart Sep 30, 2020
ba9ce1b
don't make services use the distributed feature flag
DominicBurkart Sep 30, 2020
dffe67a
bug fix: add missing import
DominicBurkart Sep 30, 2020
d5e844e
update kubernetes tests
DominicBurkart Sep 30, 2020
912e01b
debug cached import error
DominicBurkart Oct 1, 2020
d5ebdf0
use has_declared instead of memoizing the declaration of a function o…
DominicBurkart Oct 5, 2020
281f732
lock the distribution_platform mutex once instead of thrice while dis…
DominicBurkart Oct 5, 2020
9508585
make platform mutable
DominicBurkart Oct 6, 2020
61a103c
implement setup_repo for k8s
DominicBurkart Oct 6, 2020
9af4be2
implement make_image for k8s
DominicBurkart Oct 7, 2020
8e7ded3
set up autoscale
DominicBurkart Oct 7, 2020
cdae75d
take en0 or eth0 interface (whichever comes first in interface list)
DominicBurkart Oct 7, 2020
b5c5ba1
update readme
DominicBurkart Oct 8, 2020
a839e35
impl drop for k8s
DominicBurkart Oct 8, 2020
d87a33d
get socket assigned from os instead of hoping 5000 is open
DominicBurkart Oct 8, 2020
e6badf9
add minimal docs to k8s
DominicBurkart Oct 8, 2020
b85e4e5
build project in container
DominicBurkart Oct 8, 2020
789c573
use cmd syntax instead of entrypoint syntax in dockerfile
DominicBurkart Oct 8, 2020
989be90
push image to repo in make_image instead of in a separate function
DominicBurkart Oct 8, 2020
d07ca01
implement dispatch
DominicBurkart Oct 8, 2020
6476b27
update names
DominicBurkart Oct 8, 2020
730ba0d
use relative path for local import definitions instead of absolute path
DominicBurkart Oct 8, 2020
05178af
fix misplaced comment
DominicBurkart Oct 8, 2020
c173215
update readme
DominicBurkart Oct 8, 2020
bf8b108
always build project and services with procmacro2_semver_exempt
DominicBurkart Oct 9, 2020
b63f691
don't hardcode dependency path while building
DominicBurkart Oct 9, 2020
7355acb
update docs
DominicBurkart Oct 14, 2020
7e15b49
correctly point to kind in k8s example script
DominicBurkart Oct 14, 2020
b80bbee
add more hooks to precommit
DominicBurkart Oct 14, 2020
7f77614
bug fix: tag images to localhost instead of using the network address
DominicBurkart Oct 16, 2020
a7c645e
use tokio runtime in examples because async-std seems not to work
DominicBurkart Oct 21, 2020
f631ef5
make examples exit with error codes if the output is not expected
DominicBurkart Oct 21, 2020
bab40ef
switch from surf to reqwest, and set up a reusable client for request…
DominicBurkart Oct 21, 2020
8d25e0d
do not export async-std, we use tokio now
DominicBurkart Oct 21, 2020
1a423f3
temp bugfix: for some reason, async traits don't seem to handle futur…
DominicBurkart Oct 21, 2020
38237d6
always use tokio 0.2 instead of 0.3
DominicBurkart Oct 21, 2020
7db9b2a
try using a handle to avoid the hanging issue from https://github.com…
DominicBurkart Oct 22, 2020
8746ed9
remove attempts at fixing hanging issue & inline dispatch GET request
DominicBurkart Oct 22, 2020
ee66f63
use patched kube dependency
DominicBurkart Oct 23, 2020
3eb4a4e
print before and after presumed error site
DominicBurkart Oct 23, 2020
9d06cdb
do not use patched kube, do use patched reqwest
DominicBurkart Oct 23, 2020
6296dad
remove unused import
DominicBurkart Oct 23, 2020
cbd8129
use tokio 0.2 instead of 0.3.1
DominicBurkart Oct 23, 2020
8e42ec0
use tokio delay_for fn in local_queue_example after registering a new…
DominicBurkart Oct 24, 2020
c3167d9
add debugging comments
DominicBurkart Oct 24, 2020
bc493c4
switch from tokio 0.2 to 0.3 with compat layer for 0.2 deps (kube and…
DominicBurkart Oct 26, 2020
49dffba
don't capture stdout in tests
DominicBurkart Oct 26, 2020
d6cc7a3
add debug comments
DominicBurkart Oct 26, 2020
93a895e
use tokio mutex because std sync mutex hangs on lock in our use case
DominicBurkart Oct 27, 2020
ea44eec
allow lint errors
DominicBurkart Oct 27, 2020
0f43e5b
make kind wait until the control pane reaches a ready status (or time…
DominicBurkart Oct 27, 2020
5f64b7a
update comment in kubernetes example
DominicBurkart Oct 27, 2020
b8538ce
alias kind
DominicBurkart Oct 28, 2020
af966ca
k8s: use default namespace
DominicBurkart Oct 28, 2020
7c0a5ea
change variable name
DominicBurkart Oct 28, 2020
8516277
add debug statements to kubernetes example
DominicBurkart Oct 28, 2020
6091eeb
install kubectl in k8s example environment
DominicBurkart Oct 28, 2020
5b6c3a7
add debug statements to kubernetes example
DominicBurkart Oct 28, 2020
663e56f
group installation commands in k8s dockerfile
DominicBurkart Oct 28, 2020
e712808
refactor CI and testing schema
DominicBurkart Oct 29, 2020
e5132b4
use a valid deployment name (only alphanumerics + '-')
DominicBurkart Oct 29, 2020
9b4d31a
use valid container name (only alphanumerics + '-')
DominicBurkart Oct 29, 2020
ca7573d
bug fix: ports accepts an array, not a map
DominicBurkart Oct 29, 2020
570cf57
dont specify invalid port protocol
DominicBurkart Oct 29, 2020
811c16a
use nodeport to expose service to network
DominicBurkart Oct 29, 2020
886ed0d
expose service to the device running each node
DominicBurkart Oct 30, 2020
8d5cb30
add script for running k8s tests locally
DominicBurkart Oct 30, 2020
306d117
refactor: don't try to set up a local registry
DominicBurkart Nov 2, 2020
7ddb8df
update CI to use new k8s testing scheme
DominicBurkart Nov 2, 2020
81989c1
switch to using microk8s in test env
DominicBurkart Nov 3, 2020
36d4605
switch from using alpine to ubuntu
DominicBurkart Nov 3, 2020
30f1593
consolidate rustup commands
DominicBurkart Nov 3, 2020
1476a67
include rust compiler deps
DominicBurkart Nov 3, 2020
476058a
set timezone in container to UTC
DominicBurkart Nov 4, 2020
59c55ed
add comments to generated dockerfile
DominicBurkart Nov 4, 2020
84264e9
build k8s version as debug when project is built in debug, and releas…
DominicBurkart Nov 4, 2020
c62847c
attempt to fix microk8s ci
DominicBurkart Nov 4, 2020
23c0bc8
compact rust nightly install command
DominicBurkart Nov 20, 2020
c48813e
add tracing
DominicBurkart Nov 24, 2020
b012cb8
don't include big binaries in traces
DominicBurkart Nov 25, 2020
7afab27
debug weird file DNE error
DominicBurkart Nov 24, 2020
c69638d
copy instead of symlinking a local dependency that is also an ancesto…
DominicBurkart Nov 30, 2020
03a7a48
fix service not exposing correctly
DominicBurkart Nov 30, 2020
d594601
wip: use ingress
DominicBurkart Feb 25, 2021
05ad5bc
debug bad gateway pt 1
DominicBurkart May 24, 2021
22b52b6
remove debug code causing service to return source path
DominicBurkart May 27, 2021
65f10cd
remove duplicate service creation command
DominicBurkart May 27, 2021
de757c7
check if changing ingress api version fixes ingress issue in ci
DominicBurkart May 28, 2021
c8a84d3
debug bad gateway using pod instead of deployment
DominicBurkart Jun 5, 2021
4dada05
fix binding to the wrong port in the app container
DominicBurkart Jun 5, 2021
f17e11b
use deployment instead of pod
DominicBurkart Jun 5, 2021
4c7f4a8
use stable API
DominicBurkart Jun 5, 2021
c39fcaa
don't send .turbolift files in project jars
DominicBurkart Jun 5, 2021
c7aaf75
Revert "use stable API"
DominicBurkart Jun 5, 2021
c2d2470
explicitly set pathType
DominicBurkart Jun 5, 2021
76f1305
at runtime, turbolift should generate a unique id per run
DominicBurkart Jun 5, 2021
94cc7c7
use networking.k8s.io/v1
DominicBurkart Jun 5, 2021
c08556b
extract ignored directories into a constant
DominicBurkart Jun 5, 2021
da9a9e9
try to do multi-stage builds
DominicBurkart Jun 6, 2021
9fcba7b
use app name when we need a unique identifier and use function name w…
DominicBurkart Jun 6, 2021
aab557b
install compiler tools for targeting a specific architecture
DominicBurkart Jun 6, 2021
cf98731
remove extraneous build
DominicBurkart Jun 6, 2021
c670126
don't follow target architecture path
DominicBurkart Jun 6, 2021
580dc1c
pass debug flag to install
DominicBurkart Jun 6, 2021
912d7bd
point ingress to service
DominicBurkart Jun 6, 2021
7df3a6e
add debug prints
DominicBurkart Jun 6, 2021
4b55ef3
remove unused service
DominicBurkart Jun 6, 2021
8fa5483
change path to services to line up with the path called by the client
DominicBurkart Jun 7, 2021
8aea326
delete debug print
DominicBurkart Jun 7, 2021
d392739
re-add service name into path
DominicBurkart Jun 7, 2021
785678f
add additional debug statements
DominicBurkart Jun 9, 2021
e326048
fix in-cluster url
DominicBurkart Jun 9, 2021
b246746
reformat service url to pass run id as a parameter
DominicBurkart Jun 11, 2021
71d3245
add function to make the container accessible from the cluster into t…
DominicBurkart Jun 12, 2021
3661832
refactor kubernetes test suite
DominicBurkart Jun 12, 2021
dade1e3
setup ingress in ci
DominicBurkart Jun 12, 2021
edadbc7
try increasing timeout for ingress
DominicBurkart Jun 12, 2021
a9608c3
Revert "try increasing timeout for ingress"
DominicBurkart Jun 12, 2021
b2ee4b0
add debug statements to debug ci ingress issue
DominicBurkart Jun 12, 2021
a0adf3f
add cluster instantiation and rename file from setup_ingress to setup…
DominicBurkart Jun 12, 2021
c11b79c
Revert "add debug statements to debug ci ingress issue"
DominicBurkart Jun 12, 2021
7626051
remove debug prints
DominicBurkart Jun 12, 2021
f665ce4
re-enable autoscale code
DominicBurkart Jun 12, 2021
b590ab7
consolidate related todo comments
DominicBurkart Jun 12, 2021
6fcd8ee
don't call ingress ips service ips
DominicBurkart Jun 12, 2021
f2b2aae
remove irrelevant, commented out code
DominicBurkart Jun 12, 2021
7f6d6a8
fix nightly version in local queue test
DominicBurkart Jun 13, 2021
cb19688
remove double import
DominicBurkart Jun 13, 2021
55c3502
reorder imports for clarity
DominicBurkart Jun 13, 2021
e1f1946
upgrade to version 0.1.1
DominicBurkart Jun 13, 2021
987eaf0
update readme
DominicBurkart Jun 13, 2021
34cd2da
cleanup all run resources when global state handler is dropped
DominicBurkart Jun 13, 2021
90c43ec
add todo to readme
DominicBurkart Jun 13, 2021
6a63d09
update example text
DominicBurkart Jun 13, 2021
1238ab7
update todos
DominicBurkart Jun 13, 2021
9163494
add keywords and categories
DominicBurkart Jun 13, 2021
1d418e7
add website badge to readme
DominicBurkart Jun 13, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add docker test framework for k8s
  • Loading branch information
DominicBurkart committed Nov 12, 2020
commit fcd2f511a799c43a86ae3171c0acf10c170a1bd1
10 changes: 9 additions & 1 deletion .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,12 @@ jobs:
steps:
- uses: actions/checkout@v2
- name: build local queue example
run: docker build -f examples/local_queue_example/Dockerfile .
run: docker build -f examples/local_queue_example/Dockerfile .
kubernetes_example:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: build k8s example
run: docker build -f examples/kubernetes_example/Dockerfile -t kub .
- name: run k8s example
run: docker run -v /var/run/docker.sock:/var/run/docker.sock kub
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
/target
target/
Cargo.lock
.turbolift/
.turbolift/
.idea
11 changes: 11 additions & 0 deletions examples/kubernetes_example/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[package]
name = "kubernetes_example"
version = "0.1.0"
authors = ["Dominic Burkart <[email protected]>"]
edition = "2018"

[features]
"distributed" = ["turbolift/distributed"]

[dependencies]
turbolift = { path="../../" }
13 changes: 13 additions & 0 deletions examples/kubernetes_example/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# docker build -f examples/kubernetes_example/Dockerfile -t kub .
# docker run -v /var/run/docker.sock:/var/run/docker.sock kub

# set up kind (k8s in docker)
FROM rustlang/rust:nightly
RUN apt-get update \
&& apt-get install -y wget \
&& apt-get install -y docker.io \
&& rm -rf /var/lib/apt/lists/*
RUN wget -O ./kind --show-progress "https://kind.sigs.k8s.io/dl/v0.9.0/kind-$(uname)-amd64"
RUN chmod +x ./kind
COPY ./ ./
CMD sh examples/kubernetes_example/example_runtime.sh
15 changes: 15 additions & 0 deletions examples/kubernetes_example/example_runtime.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# error if any command fails
set -e

# setup cluster (will be used in all tests & runs)
./kind create cluster

cd examples/kubernetes_example

# test
RUSTFLAGS='--cfg procmacro2_semver_exempt' cargo +nightly test
RUSTFLAGS='--cfg procmacro2_semver_exempt' cargo +nightly test --features "distributed"

# run
RUSTFLAGS='--cfg procmacro2_semver_exempt' cargo +nightly run
RUSTFLAGS='--cfg procmacro2_semver_exempt' cargo +nightly run --features "distributed"
3 changes: 3 additions & 0 deletions examples/kubernetes_example/src/main.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
fn main() {
println!("Hello, world!");
}
2 changes: 0 additions & 2 deletions turbolift_internals/src/kubernetes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,6 @@ use crate::distributed_platform::{
const K8S_NAMESPACE: &str = "turbolift";
type ImageTag = String;

// pub type K8sConfig = kube::config::Config; todo

pub struct K8s {
pods: Vec<Pod>,
}
Expand Down