Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vdk-jupyter: introduce Task Runner, a polling mechanism that runs tasks in the background and tracks their status #2869

Merged
merged 39 commits into from
Nov 16, 2023
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
17c967b
vdk-jupyter: introduce task runner class and update handlers
yonitoo Oct 24, 2023
b4d48a4
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Oct 27, 2023
d9174ce
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Oct 30, 2023
c4d9765
vdk-jupyter: adapt the polling mechanism to the frontend
yonitoo Oct 31, 2023
1bee478
vdk-jupyter: fix docstring of convert job
yonitoo Oct 31, 2023
3430f1c
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Oct 31, 2023
27de36a
vdk-jupyter: fix Task Runner API-related problems
yonitoo Nov 1, 2023
9e25e32
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 1, 2023
657fea0
vdk-jupyter:
yonitoo Nov 3, 2023
3ed8caa
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 3, 2023
23c167d
vdk-jupyter: remove Delete Job
yonitoo Nov 6, 2023
e50e46c
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 6, 2023
03bfb2a
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 7, 2023
56a6a69
vdk-jupyter: add Task Runner server tests
yonitoo Nov 7, 2023
d824f9e
vdk-jupyter: fix Convert Job To Notebook case
yonitoo Nov 8, 2023
7fe2234
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 8, 2023
ff67092
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 9, 2023
978aedd
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 9, 2023
8b6e5d2
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 9, 2023
4bb6b40
build(deps): Bump org.mockito:mockito-core
dependabot[bot] Nov 3, 2023
5913607
control-service: integration test for async job deploy (#2829)
Nov 9, 2023
01fa502
build(deps): Bump org.junit.jupiter:junit-jupiter-engine from 5.9.3 t…
dependabot[bot] Nov 10, 2023
a8d9cf6
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 13, 2023
eb00f46
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 13, 2023
5ad5401
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 13, 2023
a7b3c63
vdk-jupyter: add Task Runner docstring and address review comments
yonitoo Nov 13, 2023
3c6c81b
vdk-jupyter: add a comment about the polling default values
yonitoo Nov 13, 2023
a119d57
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 13, 2023
5642ff6
vdk-jupyter: modify failing tests accordingly
yonitoo Nov 14, 2023
3046f1b
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 14, 2023
b499fc9
vdk-jupyter: make task ids unique and remove polling error threshold
yonitoo Nov 14, 2023
945869e
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 14, 2023
26ffefa
vdk-jupyter: pin react-icons dependency version to 4.11.0
yonitoo Nov 14, 2023
0141c9b
vdk-jupyter: extract duplicating task start JSON response code to func
yonitoo Nov 14, 2023
30c07fa
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 14, 2023
1726d2b
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 15, 2023
5016250
vdk-jupyter: restructure a bit the task start request responses
yonitoo Nov 15, 2023
5260eca
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 15, 2023
428a3c3
Merge branch 'main' into topic/ysalambashev/vdk-jupyter-task-runner
yonitoo Nov 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion projects/vdk-plugins/vdk-ipython/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ See more about magic commands: https://ipython.readthedocs.io/en/stable/interact

To use the extension it must be firstly installed with pip as a python package.
```
pip install vkd-ipython
pip install vdk-ipython
```

## Usage
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ import { IDocumentManager } from '@jupyterlab/docmanager';
import { FileBrowser } from '@jupyterlab/filebrowser';
import { INotebookTracker } from '@jupyterlab/notebook';
import { StatusButton } from './components/StatusButton';
import { ERROR_LABEL } from './utils';
import { ERROR_LABEL } from './utils';
import { showErrorDialog } from './components/props';
import {showLogin2Dialog} from "./components/Login2";
import { showLogin2Dialog } from './components/Login2';

export var runningVdkOperation = '';

Expand All @@ -25,11 +25,11 @@ export function updateVDKMenu(
notebookTracker: INotebookTracker,
statusButton: StatusButton
) {

commands.addCommand('jp-vdk:menu-login', {
label: 'Login',
caption: 'Execute VDK Login Command',
execute: showLogin2Dialog });
execute: showLogin2Dialog
});

// Add Run job command
add_command(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,13 +81,14 @@ export async function showConvertJobToNotebookDialog(
});
if (confirmation.button.accept) {
statusButton?.show('Convert', jobData.get(VdkOption.PATH)!);
let { message, status } = await jobConvertToNotebookRequest();
if (status) {
const transformjobResult = JSON.parse(message);
notebookContent = initializeNotebookContent(
transformjobResult['code_structure'],
transformjobResult['removed_files']
);
let { message, isSuccessful } = await jobConvertToNotebookRequest();
if (isSuccessful) {
if (typeof message !== 'string') {
notebookContent = initializeNotebookContent(
message.code_structure,
message.removed_files
);
}
await createTranformedNotebook(commands, fileBrowser, notebookTracker);
await showDialog({
title: CONVERT_JOB_TO_NOTEBOOK_BUTTON_LABEL,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import { Dialog, showDialog } from '@jupyterlab/apputils';
import { jobRequest } from '../serverRequests';
import { IJobFullProps } from './props';
import { CREATE_JOB_BUTTON_LABEL } from '../utils';
import {StatusButton} from "./StatusButton";
import { StatusButton } from './StatusButton';

export default class CreateJobDialog extends Component<IJobFullProps> {
/**
Expand Down Expand Up @@ -41,7 +41,7 @@ export default class CreateJobDialog extends Component<IJobFullProps> {
value={this.props.jobPath}
label="Path to job directory:"
tooltip="Specify the directory for the new job folder, e.g., 'x/y' with job name 'foo' becomes 'x/y/foo'. If left blank, it defaults to the Jupyter's main directory."
></VDKTextInput>
></VDKTextInput>
</>
);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@ import VDKTextInput from './VdkTextInput';
import { Dialog, showDialog, showErrorMessage } from '@jupyterlab/apputils';
import { jobRequest, jobRunRequest } from '../serverRequests';
import { IJobFullProps } from './props';
import { CREATE_DEP_BUTTON_LABEL, RUN_JOB_BUTTON_LABEL, RUN_LOG_FILE } from '../utils';
import {
CREATE_DEP_BUTTON_LABEL,
RUN_JOB_BUTTON_LABEL,
RUN_LOG_FILE
} from '../utils';
import { VdkErrorMessage } from './VdkErrorMessage';
import { VDKCheckbox } from './VdkCheckbox';
import { StatusButton } from './StatusButton';
Expand Down Expand Up @@ -54,7 +58,10 @@ export default class DeployJobDialog extends Component<IJobFullProps> {
}
}

export async function showCreateDeploymentDialog(docManager?: IDocumentManager, statusButton?: StatusButton) {
export async function showCreateDeploymentDialog(
docManager?: IDocumentManager,
statusButton?: StatusButton
) {
let runBeforeDeploy = true;

const result = await showDialog({
Expand Down
Original file line number Diff line number Diff line change
@@ -1,23 +1,23 @@
import React, { Component } from 'react';
import {requestAPI} from "../handler";
import {Dialog, showDialog} from "@jupyterlab/apputils";
import { requestAPI } from '../handler';
import { Dialog, showDialog } from '@jupyterlab/apputils';
import {
LOGIN2_BUTTON_CLASS,
LOGIN2_DIALOG_CLASS,
LOGIN2_INPUT_CLASS,
LOGIN_BUTTON_LABEL,
} from "../utils";
LOGIN2_BUTTON_CLASS,
LOGIN2_DIALOG_CLASS,
LOGIN2_INPUT_CLASS,
LOGIN_BUTTON_LABEL
} from '../utils';

// Enum representing different states of authentication
enum AuthStatus {
NONE,
WINDOW_OPENED,
SUCCESS,
FAILURE
NONE,
WINDOW_OPENED,
SUCCESS,
FAILURE
}

interface ManualOAuth2FlowState {
authStatus: AuthStatus;
authStatus: AuthStatus;
}

/**
Expand All @@ -44,112 +44,136 @@ interface ManualOAuth2FlowState {
* 5. The component's state is updated based on the success or failure of the authentication.
*
*/
export default class ManualOAuth2Flow extends Component<{}, ManualOAuth2FlowState> {
private authWindow: Window | null = null;
export default class ManualOAuth2Flow extends Component<
{},
ManualOAuth2FlowState
> {
private authWindow: Window | null = null;

constructor(props: {}) {
super(props);
this.state = {
authStatus: AuthStatus.NONE
};
}


openAuthWindow = async () => {
try {
const redirect_url = await requestAPI<any>(
'login',
{
method: 'GET'
}
);
console.debug("Authorization redirect URL is" + redirect_url);
constructor(props: {}) {
super(props);
this.state = {
authStatus: AuthStatus.NONE
};
}

this.authWindow = window.open(redirect_url, 'authWindow', 'width=600,height=400');
this.setState({authStatus: AuthStatus.WINDOW_OPENED});
} catch(error) {
console.error("Error during start of authentication:", error);
this.setState({ authStatus: AuthStatus.FAILURE });
}
openAuthWindow = async () => {
try {
const redirect_url = await requestAPI<any>('login', {
method: 'GET'
});
console.debug('Authorization redirect URL is' + redirect_url);

this.authWindow = window.open(
redirect_url,
'authWindow',
'width=600,height=400'
);
this.setState({ authStatus: AuthStatus.WINDOW_OPENED });
} catch (error) {
console.error('Error during start of authentication:', error);
this.setState({ authStatus: AuthStatus.FAILURE });
}
};

closeAuthWindow = () => {
if (this.authWindow && !this.authWindow.closed) {
this.authWindow.close();
}
closeAuthWindow = () => {
if (this.authWindow && !this.authWindow.closed) {
this.authWindow.close();
}
};

handleURLInput = async (e: { target: { value: any; }; }) => {
const redirectURI = e.target.value;
const code = new URL(redirectURI).searchParams.get('code');
if (code) {
try {
await requestAPI<any>(
'login?code=' + code,
{
method: 'GET'
}
);
this.closeAuthWindow();
this.setState({ authStatus: AuthStatus.SUCCESS });
} catch (error) {
this.closeAuthWindow();
console.error("Error during authentication:", error);
this.setState({ authStatus: AuthStatus.FAILURE });
}
}
handleURLInput = async (e: { target: { value: any } }) => {
const redirectURI = e.target.value;
const code = new URL(redirectURI).searchParams.get('code');
if (code) {
try {
await requestAPI<any>('login?code=' + code, {
method: 'GET'
});
this.closeAuthWindow();
this.setState({ authStatus: AuthStatus.SUCCESS });
} catch (error) {
this.closeAuthWindow();
console.error('Error during authentication:', error);
this.setState({ authStatus: AuthStatus.FAILURE });
}
}
};

render() {
return (
<div className={LOGIN2_DIALOG_CLASS}>
{this.state.authStatus === AuthStatus.NONE &&
<div>
<p>Authenticate process will start. After you clik a new window will be opened.</p>
<ul>
<li>Please <b>wait</b> for the authentication window to redirect you back to 127.0.0.1.</li>
<li><b>Ignore</b> the failed connection error message.</li>
<li>And then <b>copy and paste</b> the full URL you are redirected to.</li>
</ul>
render() {
return (
<div className={LOGIN2_DIALOG_CLASS}>
{this.state.authStatus === AuthStatus.NONE && (
<div>
<p>
Authenticate process will start. After you clik a new window will
be opened.
</p>
<ul>
<li>
Please <b>wait</b> for the authentication window to redirect you
back to 127.0.0.1.
</li>
<li>
<b>Ignore</b> the failed connection error message.
</li>
<li>
And then <b>copy and paste</b> the full URL you are redirected
to.
</li>
</ul>

<button className={LOGIN2_BUTTON_CLASS} onClick={this.openAuthWindow}>Click here to start.</button>
</div>
}
{this.state.authStatus === AuthStatus.WINDOW_OPENED &&
<div>
<p>A new authentication window has been opened.</p>
<ul>
<li>Please <b>wait</b> for the authentication window to redirect you back to 127.0.0.1.</li>
<li><b>Ignore</b> the failed connection error message.</li>
<li>And then <b>copy and paste</b> the full URL you are redirected to in here:</li>
</ul>
<input type="text" className={LOGIN2_INPUT_CLASS} onChange={this.handleURLInput} />
</div>
}
{this.state.authStatus === AuthStatus.SUCCESS &&
<div>
<p>Authentication successful!</p>
</div>
}
{this.state.authStatus === AuthStatus.FAILURE &&
<div>
<p>Authentication failed!</p>
</div>
}
</div>
);
}
<button
className={LOGIN2_BUTTON_CLASS}
onClick={this.openAuthWindow}
>
Click here to start.
</button>
</div>
)}
{this.state.authStatus === AuthStatus.WINDOW_OPENED && (
<div>
<p>A new authentication window has been opened.</p>
<ul>
<li>
Please <b>wait</b> for the authentication window to redirect you
back to 127.0.0.1.
</li>
<li>
<b>Ignore</b> the failed connection error message.
</li>
<li>
And then <b>copy and paste</b> the full URL you are redirected
to in here:
</li>
</ul>
<input
type="text"
className={LOGIN2_INPUT_CLASS}
onChange={this.handleURLInput}
/>
</div>
)}
{this.state.authStatus === AuthStatus.SUCCESS && (
<div>
<p>Authentication successful!</p>
</div>
)}
{this.state.authStatus === AuthStatus.FAILURE && (
<div>
<p>Authentication failed!</p>
</div>
)}
</div>
);
}
}


export async function showLogin2Dialog() {
await showDialog({
title: LOGIN_BUTTON_LABEL,
body: (
<ManualOAuth2Flow></ManualOAuth2Flow>
),
buttons: [Dialog.cancelButton({label: "Close"})],
hasClose: false
});
await showDialog({
title: LOGIN_BUTTON_LABEL,
body: <ManualOAuth2Flow></ManualOAuth2Flow>,
buttons: [Dialog.cancelButton({ label: 'Close' })],
hasClose: false
});
}
Loading