Skip to content

Commit

Permalink
vdk-jupyter: Sample job notebook step (#2364)
Browse files Browse the repository at this point in the history
This change adds a specific notebook step to the Jupyter sample job
which contains the necessary initialization cell.

![image](https://github.com/vmware/versatile-data-kit/assets/45939426/6c1eab3d-7a04-4fc3-bf44-3d330820a841)



Testing done: local

---------

Signed-off-by: Gabriel Georgiev <[email protected]>
  • Loading branch information
gabrielgeorgiev1 authored Jul 6, 2023
1 parent 70f007a commit 0beefa0
Showing 1 changed file with 74 additions and 17 deletions.
Original file line number Diff line number Diff line change
@@ -1,35 +1,92 @@
{
"cells": [],
"cells": [
{
"cell_type": "code",
"execution_count": null,
"id": "16a6ea87-7142-44dd-94b6-e95fe2d57d06",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"\"\"\"\n",
"vdk_ipython extension introduces a magic command for Jupyter.\n",
"The command enables the user to load VDK for the current notebook.\n",
"VDK provides the job_input API, which has methods for:\n",
" * executing queries to an OLAP database;\n",
" * ingesting data into a database;\n",
" * processing data into a database.\n",
"See the IJobInput documentation for more details.\n",
"https://github.com/vmware/versatile-data-kit/blob/main/projects/vdk-core/src/vdk/api/job_input.py\n",
"\n",
"Please refrain from tagging this cell with VDK as it is not an actual part of the data job\n",
"and is only used for development purposes.\n",
"\"\"\"\n",
"%reload_ext vdk_ipython\n",
"%reload_VDK\n",
"job_input = VDK.get_initialized_job_input()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "712a949e-8e9e-4c31-972a-6fab1b274c95",
"metadata": {
"tags": [
"vdk"
]
},
"outputs": [],
"source": [
"\"\"\"\n",
"This and the subsequent cells are marked as VDK cells.\n",
"\n",
"Cells can be marked or unmarked as VDK cells through the gearwheel menu\n",
"to the right.\n",
"\"\"\"\n",
"\n",
"job_input.execute_query(\n",
" \"CREATE TABLE IF NOT EXISTS hello_world (id NVARCHAR);\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "36137500-9136-4083-8b04-073ed84e2d4c",
"metadata": {
"tags": [
"vdk"
]
},
"outputs": [],
"source": [
"job_input.send_object_for_ingestion(\n",
" payload=dict(id=\"Hello World!\"),\n",
" destination_table=\"hello_world\"\n",
")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
},
"pycharm": {
"stem_cell": {
"cell_type": "raw",
"source": [
"\n"
],
"metadata": {
"collapsed": false
}
}
"pygments_lexer": "ipython3",
"version": "3.7.16"
}
},
"nbformat": 4,
"nbformat_minor": 0
"nbformat_minor": 5
}

0 comments on commit 0beefa0

Please sign in to comment.