Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a workspace-only workflow for Kedro projects on Databricks #2628

Merged
merged 61 commits into from
Jun 14, 2023
Merged

Add a workspace-only workflow for Kedro projects on Databricks #2628

merged 61 commits into from
Jun 14, 2023

Conversation

jmholzer
Copy link
Contributor

@jmholzer jmholzer commented Jun 1, 2023 •
edited
Loading

Description

Resolves #2443

Important: use the iris-databricks starter to test manually

Checklist

  • Read the contributing guidelines
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the RELEASE.md file
  • Added tests to cover my changes

jmholzer and others added 26 commits June 1, 2023 15:00
@jmholzer
Rename databricks IDE development workflow
a392318 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Add new docs pages to index
f0c0394 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Add first draft of workspace-only workflow
20d880a 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Finish summary
87b2f11 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge remote-tracking branch 'origin/main' into docs/rework-deprecate…
123a811 
…d-databricks-documentation

Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Linting
7b784c7 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Linter
bb96f18 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge branch 'main' into docs/rework-deprecated-databricks-documentation
0b731f3
@jmholzer
Fix image links
e0c48c6 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge branch 'docs/rework-deprecated-databricks-documentation' of git…
a4a07e3 
…hub.com:kedro-org/kedro into docs/rework-deprecated-databricks-documentation

Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Fix broken link
b4fb72b 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge branch 'main' into docs/rework-deprecated-databricks-documentation
b170348
@jmholzer
Merge branch 'main' into docs/rework-deprecated-databricks-documentation
05dcf41
@jmholzer
Change wording
2b31c92 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge branch 'docs/rework-deprecated-databricks-documentation' of git…
81afd33 
…hub.com:kedro-org/kedro into docs/rework-deprecated-databricks-documentation

Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Add Git provider note to beginning of guide
9156c0c 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Add Repo creation guide
c9f8d11 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Fix image links
fa890af 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge branch 'main' into docs/rework-deprecated-databricks-documentation
ac3398c
@jmholzer
Remove Databricks host section
4c0a94a 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Add main steps to creating GH personal token
c10012c 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge branch 'docs/rework-deprecated-databricks-documentation' of git…
a108446 
…hub.com:kedro-org/kedro into docs/rework-deprecated-databricks-documentation

Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Validate and complete workflow
0a08915 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge branch 'main' into docs/rework-deprecated-databricks-documentation
13512b1
@jmholzer
Modify output snippets to show console logging instead of rich
3130729 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge branch 'docs/rework-deprecated-databricks-documentation' of git…
b4722b4 
…hub.com:kedro-org/kedro into docs/rework-deprecated-databricks-documentation

Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer jmholzer changed the title Docs/rework deprecated databricks documentation Add a workspace-only workflow for Kedro projects on Databricks Jun 6, 2023
jmholzer and others added 9 commits June 8, 2023 14:04
@jmholzer
Merge branch 'docs/rework-deprecated-databricks-documentation' of git…
2c3d9f4 
…hub.com:kedro-org/kedro into docs/rework-deprecated-databricks-documentation

Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge branch 'docs/update-databricks-docs-new-logging' into docs/rewo…
3b63673 
…rk-deprecated-databricks-documentation

Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge branch 'main' into docs/rework-deprecated-databricks-documentation
03f579d
@jmholzer
Fix syntax and spurious whitespace
c3f6eb2 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Remove repo scope note
c76f526 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Add overview of Git steps to sync repos
2fa5901 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Fix visualisation docs
20a2bb4 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge branch 'docs/rework-deprecated-databricks-documentation' of git…
15b1006 
…hub.com:kedro-org/kedro into docs/rework-deprecated-databricks-documentation

Signed-off-by: Jannic Holzer <[email protected]>
@stichbury
Merge branch 'main' into docs/rework-deprecated-databricks-documentation
ac6ff7b
jmholzer added 2 commits June 9, 2023 16:31
@jmholzer
Fix deployment guide title
cd61bf4 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge branch 'docs/rework-deprecated-databricks-documentation' of git…
27da7ed 
…hub.com:kedro-org/kedro into docs/rework-deprecated-databricks-documentation

Signed-off-by: Jannic Holzer <[email protected]>
astrojuanlu
astrojuanlu approved these changes Jun 12, 2023
View reviewed changes
Copy link
Member

@astrojuanlu astrojuanlu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finally tested this with the right starter, and everything works! 🦭

@jmholzer
Copy link
Contributor Author

Finally tested this with the right starter, and everything works! 🦭

Thanks @astrojuanlu! Appreciate it :)

@astrojuanlu
Copy link
Member

Pro MyST tip: to avoid breaking the xrefs when the heading changes, you can use this:

(heading-target)=
### Heading

...

This is a reference to [reference1](#heading-target).

https://myst-parser.readthedocs.io/en/latest/syntax/cross-referencing.html#creating-explicit-targets

jmholzer added 2 commits June 13, 2023 15:42
@jmholzer
Add release note
6e9e834 
Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer
Merge branch 'docs/rework-deprecated-databricks-documentation' of git…
99aa7a7 
…hub.com:kedro-org/kedro into docs/rework-deprecated-databricks-documentation

Signed-off-by: Jannic Holzer <[email protected]>
@jmholzer jmholzer requested a review from idanov as a code owner June 13, 2023 14:42
@stichbury stichbury mentioned this pull request Jun 14, 2023
Closed
@jmholzer jmholzer merged commit 5278d56 into main Jun 14, 2023
@jmholzer jmholzer deleted the docs/rework-deprecated-databricks-documentation branch June 14, 2023 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@astrojuanlu astrojuanlu astrojuanlu approved these changes

@stichbury stichbury stichbury approved these changes

@idanov idanov Awaiting requested review from idanov

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Rework the existing 'deployment to a Databricks cluster' documentation
3 participants
@jmholzer @astrojuanlu @stichbury