diff --git a/sdk/python/foundation-models/system/import/import_model_into_registry.ipynb b/sdk/python/foundation-models/system/import/import_model_into_registry.ipynb index 753fb5c7b05..66134935c13 100644 --- a/sdk/python/foundation-models/system/import/import_model_into_registry.ipynb +++ b/sdk/python/foundation-models/system/import/import_model_into_registry.ipynb @@ -1,7 +1,6 @@ { "cells": [ { - "attachments": {}, "cell_type": "markdown", "id": "db3fa6e6", "metadata": { @@ -65,7 +64,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "id": "88ba6a51", "metadata": {}, @@ -94,7 +92,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "id": "8911a778", "metadata": { @@ -128,7 +125,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "id": "6cab6857", "metadata": { @@ -171,7 +167,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "id": "d6c2cede", "metadata": {}, @@ -215,7 +210,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "id": "54e53eb8", "metadata": {}, @@ -244,7 +238,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "id": "f9c1c389", "metadata": {}, @@ -272,11 +265,15 @@ "Browse models on [HuggingFace hub](https://huggingface.co/models) and identify a model to import. Make sure the task type of the model is among the supported tasks as explained in the introduction section. Copy the model id which is available in the URI of the page or can be copied using the copy icon next to the model name and assign it to the variable `MODEL_ID`.\n", "\n", "\n", - "![final-image.png](attachment:final-image.png)" + "![final-image.png](attachment:final-image.png)\n", + "\n", + "\n", + "#### token\n", + "\n", + "There are a few gated models hosted in HuggingFace, for example the [tiiuae/falcon-180B](https://huggingface.co/tiiuae/falcon-180B) model, which requires the user to accept certain terms and conditions, without which the import component cannot access it directly. To do so, the user would have to create a personal account in HuggingFace and accept terms and conditions. To import such models the user would have to create an Access Token with their HuggingFace account and pass it to the model-import component, after which the model can be downloaded to their workspace or registry. To know the more details , please visit [this link](https://huggingface.co/docs/hub/security-tokens)." ] }, { - "attachments": {}, "cell_type": "markdown", "id": "9fb68542", "metadata": {}, @@ -297,7 +294,8 @@ "outputs": [], "source": [ "MODEL_ID = \"bert-base-uncased\"\n", - "TASK_NAME = \"fill-mask\"" + "TASK_NAME = \"fill-mask\"\n", + "TOKEN = None" ] }, { @@ -311,7 +309,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "id": "d7178a91", "metadata": {}, @@ -346,7 +343,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "id": "ecd925e0", "metadata": {}, @@ -362,7 +358,7 @@ "outputs": [], "source": [ "@pipeline\n", - "def model_import_pipeline(model_id, compute, task_name):\n", + "def model_import_pipeline(model_id, compute, task_name, token=None):\n", " \"\"\"\n", " Create model import pipeline using pipeline component.\n", "\n", @@ -370,13 +366,15 @@ " ----------\n", " model_id : str\n", " compute : str\n", + " task_name : str\n", + " token : str\n", "\n", " Returns\n", " -------\n", " model_registration_details : dict\n", " \"\"\"\n", " import_model_job = import_model(\n", - " model_id=model_id, compute=compute, task_name=task_name\n", + " model_id=model_id, compute=compute, task_name=task_name, token=token\n", " )\n", "\n", " # Set job to not continue on failure\n", @@ -388,7 +386,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "id": "c94c3616", "metadata": {}, @@ -406,9 +403,7 @@ "outputs": [], "source": [ "pipeline_object = model_import_pipeline(\n", - " model_id=MODEL_ID,\n", - " compute=COMPUTE,\n", - " task_name=TASK_NAME,\n", + " model_id=MODEL_ID, compute=COMPUTE, task_name=TASK_NAME, token=TOKEN\n", ")\n", "pipeline_object.identity = UserIdentityConfiguration()\n", "\n", @@ -419,7 +414,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "id": "03f5265f", "metadata": {}, @@ -462,7 +456,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "id": "f33d5c80", "metadata": {}, @@ -471,7 +464,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "id": "dda1f809", "metadata": {}, @@ -512,7 +504,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "id": "e5504e6f", "metadata": {},