Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jupyter-vep: Expand details on extensions design #1304

Merged
merged 7 commits into from
Nov 11, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 21 additions & 3 deletions specs/vep-994-jupyter-notebook-integration/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,9 @@ Legend (terms and table are based on Pragmatic Marketing recommendations):
JupyterLab is the next-generation user interface for Project Jupyter offering all the familiar building blocks of the classic Jupyter Notebook (notebook, terminal, text editor, file browser, rich outputs, etc.) in a flexible and powerful user interface.
Fundamentally, JupyterLab is designed as an extensible environment. JupyterLab extensions can customize or enhance any part of JupyterLab.

The proposed design describes the solution for creating a Jupyter UI for VDK. For this purpose, a JupyterLab extension will be implemented, and it will give us the chance to create graphical elements like buttons and widgets – alternatives of currently used CLI commands. An iPython package will also be implemented to export VDK variables. This package will be imported to JupyterLab, and they will be accessible from there.
The proposed design describes the solution for creating a Jupyter UI for VDK. For this purpose, a JupyterLab extension will be implemented, and it will give us the chance to create graphical elements like buttons and widgets – alternatives of currently used CLI commands.

Since JupyterLab works with notebook files the proposed design should support getting the job steps from them. To the purpose, a new VDK plugin will be introduced which will allow vdk to run steps which came from notebook files.

| Operation | Flow | Covered use cases | Priority |
|:----------------------:|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:------------------:|:--------:|
Expand All @@ -95,8 +97,6 @@ The proposed design describes the solution for creating a Jupyter UI for VDK. Fo
| List command | Similar to execute deploy download. After the needed information is filled in the pop up, by clicking a button a window with the result information will be introduced. | 1 | low |
| Show command | Similar to execute deploy download. After the needed information is filled in the pop up, by clicking a button a window with the result information will be introduced. | 1 | low |



## API design

<!--
Expand All @@ -112,6 +112,24 @@ Explain how does the system handle API violations.


## Detailed design
### VDK Notebook plugin
This VDK plugin will provide the functionality to run Jobs which will retrieve Job steps from notebook files instead of .py and .sql files. This plugin can be used alone without the JupyterLab extension.

### VDK JupyterLab extension
This extension will be both front-end and server side extension for JupyterLab.
The front-end side will be introducing the graphical elements such as menus, buttons, etc. and will be responsible with sending
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed saying that since it can be done at follow-up PR when you have better idea about these things

But we need to be very explicit here.

What are the exact graphical elements that would be introduced. Having mockup picture would be a huge bonus .

What are the exact request that would be sent and what commands would be supported?

Also there's a lot of assumption about what the reader knows and not everyone reading this would know how VDK works and what is file step plugin and etc , so you need either describe or better link to existing documentation that explain concepts you are using.

requests to the server side extension. The server side extension will be responsible with executing the vdk commands and functions according to the
requests sent by the front-end side. Only the server extension will have direct connection to VDK and will send the needed response according to the results from VDK to the front-end extension. The extension will be using VDK Notebook plugin to run VDK Jobs and steps
,since the standard file based step run is not working with notebook files. For the remaining operations such as deploy, log in, log out, etc. it will be using the corresponding VDK plugin/project.

From the diagram below you can see what the extension will consist of:
![jupyterlab-extension](jlab-extension.png)

Example use case:

![example-use-case](example.png)


<!--
Dig deeper into each component. The section can be as long or as short as necessary.
Consider at least the below topics but you do not need to cover those that are not applicable.
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.