Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vdk-jupyter: use vdksql for SQL cells and steps #2729

Merged
merged 1 commit into from
Oct 2, 2023
Merged

vdk-jupyter: use vdksql for SQL cells and steps #2729

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .../vdk-plugins/vdk-jupyter/vdk-jupyterlab-extension/src/components/ConvertJobToNotebook.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -267,8 +267,7 @@ export const populateNotebook = async (notebookTracker: INotebookTracker) => {
' <li> The below cells are automatically generated corresponding to a step (.sql file or .py file with VDK run(job_input) function) in your original job.</li>\n',
' <li> When you see a title saying <b>"Step generated from: sample.py"</b> before some blocks of code, \n',
' it means that the code below that title was created from the "sample.py" file.</li>\n',
' <li> Similarly, if you come across code cells that have the format <b>"job_input.execute_query(query_string)"</b> ,\n',
' it means that those cells contain code generated from ".sql" files.</li>\n',
' <li> Similarly, if you come across code cells that start <b>%%vdksql</b> contain code generated from ".sql" steps.</li>\n',
' <li> On the other hand, code cells originating from ".py" files remain unchanged.\n',
' However, an additional cell is included that calls the "run" function using the command <b>"run(job_input)"</b> . \n',
' This cell executes the "run" function from the code generated from the ".py" file.</li>\n',
Expand Down
2 changes: 1 addition & 1 deletion .../vdk-plugins/vdk-jupyter/vdk-jupyterlab-extension/vdk_jupyterlab_extension/convert_job.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ def _process_python_files(self, file):
def _process_sql_files(self, file):
with open(file) as f:
content = f.read()
self._code_structure.append(f'job_input.execute_query("""{content}""")')
self._code_structure.append(f"%%vdksql\n{content}")
self._removed_files.append(os.path.basename(file))
os.remove(file)

Expand Down
15 changes: 7 additions & 8 deletions ...er/vdk-jupyterlab-extension/vdk_jupyterlab_extension/jupyter_sample_job/10_notebook.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,16 +38,15 @@
},
"outputs": [],
"source": [
"\"\"\"\n",
"This and the subsequent cells are marked as VDK cells.\n",
"%%vdksql\n",
"/*\n",
"Cells with '%%vdksql' magic are VDK SQL Cell (step).\n",
"\n",
"Cells can be marked or unmarked as VDK cells through the gearwheel menu\n",
"to the right.\n",
"\"\"\"\n",
"Note that this and the subsequent cells are marked as VDK cells.\n",
"Cells can be marked or unmarked as VDK cells through the gearwheel menu to the right.\n",
"*/\n",
"\n",
"job_input.execute_query(\n",
" \"CREATE TABLE IF NOT EXISTS hello_world (id NVARCHAR);\"\n",
")"
"CREATE TABLE IF NOT EXISTS hello_world (id NVARCHAR)"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion ...pyterlab-extension/vdk_jupyterlab_extension/tests/test_convert_job_directory_processor.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ def test_cleanup(self):
def test_get_code_structure(self):
self.processor.process_files()
expected_code_structure = [
f'job_input.execute_query("""{self.sql_content}""")',
f"%%vdksql\n{self.sql_content}",
self.py_content_run,
self.py_content_run_multiline,
self.py_content_run_spaces,
Expand Down