Documentation updates for notebook mode

Updates website and github documentation, and adds an example notebook.

PiperOrigin-RevId: 349459201

README.md

@@ -3,7 +3,8 @@
 <!--* freshness: { owner: 'lit-dev' reviewed: '2020-08-04' } *-->
 
 The Language Interpretability Tool (LIT) is a visual, interactive
-model-understanding tool for NLP models.
+model-understanding tool for NLP models. It can be run as a standalone server,
+or inside of notebook environments such as Colab and Jupyter.
 
 LIT is built to answer questions such as:
 
@@ -92,6 +93,10 @@ running the demos.
 Explore a collection of hosted demos on the
 [LIT website demos page](https://pair-code.github.io/lit/demos).
 
+Colab notebooks showing the use of LIT inside of notebooks can be found at
+google3/third_party/py/lit_nlp/example/notebooks. A simple example can be viewed
+[here](https://colab.research.google.com/github/pair-code/lit/blob/main/examples/notebooks/LIT_sentiment_classifier.ipynb).
+
 ### Quick-start: classification and regression
 
 To explore classification and regression models tasks from the popular [GLUE benchmark](https://gluebenchmark.com/):

documentation/python_api.md

@@ -421,6 +421,27 @@ GOOGLE_APPLICATION_CREDENTIALS environment variable to point to that file.
 With that environment variable set to the correct path, LIT can make use of the
 backtranlator generator if you pass it as a generator in the Server constructor.
 
+## Using LIT in Notebook Environmenbts
+
+As an alternative to running a LIT server and connecting to it through a
+web browser, LIT can be used directly inside of python notebook environments,
+such as [colab](https://colab.research.google.com/) and
+[jupyter](https://jupyter.org/).
+
+After installing LIT through pip, create a `lit_nlp.notebook.LitWidget` object,
+passing in a dict of models and a dict of datasets, similar to the
+`lit_nlp.dev_server.Server` constructor. You can optionally provide a height
+parameter that specifies the height in pixels to render the LIT UI.
+
+Then, in its own output cell, call the `render` method on the widget object
+to render the LIT UI. The LIT UI can be rendered in multiple cells if desired.
+
+The widget has a `stop` method which shuts down the widget's server. This
+can be important for freeing up resources if you plan to create multiple LIT
+widget instances in a single notebook.
+
+Check out an [example notebook](https://colab.research.google.com/github/pair-code/lit/blob/main/examples/notebooks/LIT_sentiment_classifier.ipynb).
+
 ## LIT Application & Serving
 
 TODO: write this

lit_nlp/examples/notebooks/LIT_sentiment_classifier.ipynb

@@ -0,0 +1,91 @@
+{
+  "nbformat": 4,
+  "nbformat_minor": 0,
+  "metadata": {
+    "colab": {
+      "name": "LIT in Notebooks",
+      "provenance": [],
+      "collapsed_sections": []
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    }
+  },
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "h2c6PyqQaNiA"
+      },
+      "source": [
+        "# Using the Language Interpretability Tool in Notebooks\n",
+        "\n",
+        "This notebook shows use of the [Language Interpretability Tool](https://pair-code.github.io/lit) on a binary classifier for labelling statement sentiment (0 for negative, 1 for positive).\n",
+        "\n",
+        "The LitWidget object constructor takes a dict mapping model names to model objects, and a dict mapping dataset names to dataset objects. Those will be the datasets and models displayed in LIT. It also optionally takes in a `height` parameter for how tall to render the LIT UI in pixels (it defaults to 1000 pixels). Running the constructor will cause the LIT server to be started in the background, loading the models and datasets and enabling the UI to be served.\n",
+        "\n",
+        "Render the LIT UI in an output cell by calling the `render` method on the LitWidget object. The LIT UI can be rendered multiple times in separate cells if desired. The widget also contains a `stop` method to shut down the LIT server.\n",
+        "\n",
+        "Copyright 2020 Google LLC.\n",
+        "SPDX-License-Identifier: Apache-2.0"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "ukXamAB_FBM8"
+      },
+      "source": [
+        "# Install LIT and transformers packages. The transformers package is needed by the model and dataset we are using.\n",
+        "!pip install lit_nlp transformers==2.11.0"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "30l9ZyTjxJjf"
+      },
+      "source": [
+        "# Fetch the files needed by the model\n",
+        "!wget https://storage.googleapis.com/what-if-tool-resources/lit-models/sst2_tiny.tar.gz\n",
+        "!tar -xvf sst2_tiny.tar.gz"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "AWhbAZg57RpB"
+      },
+      "source": [
+        "# Create the LIT widget with the model and dataset to analyze.\n",
+        "from lit_nlp import notebook\n",
+        "from lit_nlp.examples.datasets import glue\n",
+        "from lit_nlp.examples.models import glue_models\n",
+        "\n",
+        "datasets = {'sst_dev': glue.SST2Data('validation')}\n",
+        "models = {'sst_tiny': glue_models.SST2Model('./')}\n",
+        "\n",
+        "widget = notebook.LitWidget(models, datasets, height=800)"
+      ],
+      "execution_count": null,
+      "outputs": []
+    },
+    {
+      "cell_type": "code",
+      "metadata": {
+        "id": "9GSfs1waBdLd"
+      },
+      "source": [
+        "# Render the widget\n",
+        "widget.render()"
+      ],
+      "execution_count": null,
+      "outputs": []
+    }
+  ]
+}

website/src/demos.md

@@ -25,4 +25,7 @@ color: "#49596c"
 
   {% include partials/demo-card c-title: "Text generation", link: "/demos/t5.html",
   c-data-source: "CNN / Daily Mail", c-copy: "Use a T5 model to summarize text. For any example of interest, quickly find similar examples from the training set, using an approximate nearest-neighbors index.", tags: "T5, generation", external:"true" %}
+
+  {% include partials/demo-card c-title: "Using LIT in notebooks", link: "https://colab.research.google.com/github/pair-code/lit/blob/main/examples/notebooks/LIT_sentiment_classifier.ipynb",
+  c-data-source: "CNN / Daily Mail", c-copy: "Use LIT directly inside a colab notebook. Explore binary classification for sentiment analysis from the General Language Understanding Evaluation (GLUE) benchmark suite.", tags: "colab, notebooks, BERT, binary classification", external:"true" %}
   </div>

website/src/index.md

@@ -31,6 +31,7 @@ LIT contains many built-in capabilities but is also customizable, with the abili
 
 For a similar tool to explore general-purpose machine learning models, check out the [What-If Tool](https://whatif-tool.dev).
 
+LIT can be run as a standalone server, or inside of python notebook environments such as colab and jupyter.
 </div>
 
 {% include partials/spacer height:50 %}
@@ -99,7 +100,7 @@ And more...
   {% include partials/home-card image: '/assets/images/LIT_Contribute.png', action: 'CODE', 
   title: 'Contribute to LIT', desc: 'LIT is open to anyone who wants to help develop and improve it!', 
   cta-text:"View developer guide", link: 'https://github.com/PAIR-code/lit/blob/main/documentation/development.md', external:"true" %}
-  
+
   {% include partials/home-card image: '/assets/images/LIT_Updates.png', action: 'UPDATES', 
   title: 'Latest updates', desc: 'New features, updates, and improvements to LIT.', 
   cta-text:"See release notes", link: 'https://github.com/PAIR-code/lit/blob/main/RELEASE.md' external:"true" %}

website/src/setup.md

@@ -257,4 +257,28 @@ accordingly.
 Note: there are a few additional methods in the model API - see
 [`Model`](https://github.com/PAIR-code/lit/tree/main/lit_nlp/api/model.py) for details.
 
+# Run LIT inside python notebooks
+
+It's very easy to use LIT inside of colab and jupyter notebooks. Just install
+the pip package and use the `LitWidget` object with your models and datasets.
+
+```
+from lit_nlp import notebook
+
+# MulitiNLIData implements the Dataset API
+datasets = {
+    'mnli_matched': MultiNLIData('/path/to/dev_matched.tsv'),
+    'mnli_mismatched': MultiNLIData('/path/to/dev_mismatched.tsv'),
+}
+
+# NLIModel implements the Model API
+models = {
+    'model_foo': NLIModel('/path/to/model/foo/files'),
+    'model_bar': NLIModel('/path/to/model/bar/files'),
+}
+
+widget = notebook.LitWidget(models, datasets)
+widget.render()
+```
+
 </div>