From 2deb887860e094ab0a3e063c828f81ebb0c0c79b Mon Sep 17 00:00:00 2001 From: James Wexler Date: Tue, 29 Dec 2020 12:57:29 -0800 Subject: [PATCH] Documentation updates for notebook mode Updates website and github documentation, and adds an example notebook. PiperOrigin-RevId: 349459201 --- README.md | 7 +- documentation/python_api.md | 21 +++++ .../notebooks/LIT_sentiment_classifier.ipynb | 91 +++++++++++++++++++ website/src/demos.md | 3 + website/src/index.md | 3 +- website/src/setup.md | 24 +++++ 6 files changed, 147 insertions(+), 2 deletions(-) create mode 100644 lit_nlp/examples/notebooks/LIT_sentiment_classifier.ipynb diff --git a/README.md b/README.md index 4b44f039..6d6c3603 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,8 @@ The Language Interpretability Tool (LIT) is a visual, interactive -model-understanding tool for NLP models. +model-understanding tool for NLP models. It can be run as a standalone server, +or inside of notebook environments such as Colab and Jupyter. LIT is built to answer questions such as: @@ -92,6 +93,10 @@ running the demos. Explore a collection of hosted demos on the [LIT website demos page](https://pair-code.github.io/lit/demos). +Colab notebooks showing the use of LIT inside of notebooks can be found at +google3/third_party/py/lit_nlp/example/notebooks. A simple example can be viewed +[here](https://colab.research.google.com/github/pair-code/lit/blob/main/examples/notebooks/LIT_sentiment_classifier.ipynb). + ### Quick-start: classification and regression To explore classification and regression models tasks from the popular [GLUE benchmark](https://gluebenchmark.com/): diff --git a/documentation/python_api.md b/documentation/python_api.md index f4dd973a..1be6941a 100644 --- a/documentation/python_api.md +++ b/documentation/python_api.md @@ -421,6 +421,27 @@ GOOGLE_APPLICATION_CREDENTIALS environment variable to point to that file. With that environment variable set to the correct path, LIT can make use of the backtranlator generator if you pass it as a generator in the Server constructor. +## Using LIT in Notebook Environmenbts + +As an alternative to running a LIT server and connecting to it through a +web browser, LIT can be used directly inside of python notebook environments, +such as [colab](https://colab.research.google.com/) and +[jupyter](https://jupyter.org/). + +After installing LIT through pip, create a `lit_nlp.notebook.LitWidget` object, +passing in a dict of models and a dict of datasets, similar to the +`lit_nlp.dev_server.Server` constructor. You can optionally provide a height +parameter that specifies the height in pixels to render the LIT UI. + +Then, in its own output cell, call the `render` method on the widget object +to render the LIT UI. The LIT UI can be rendered in multiple cells if desired. + +The widget has a `stop` method which shuts down the widget's server. This +can be important for freeing up resources if you plan to create multiple LIT +widget instances in a single notebook. + +Check out an [example notebook](https://colab.research.google.com/github/pair-code/lit/blob/main/examples/notebooks/LIT_sentiment_classifier.ipynb). + ## LIT Application & Serving TODO: write this diff --git a/lit_nlp/examples/notebooks/LIT_sentiment_classifier.ipynb b/lit_nlp/examples/notebooks/LIT_sentiment_classifier.ipynb new file mode 100644 index 00000000..871f7dc8 --- /dev/null +++ b/lit_nlp/examples/notebooks/LIT_sentiment_classifier.ipynb @@ -0,0 +1,91 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "colab": { + "name": "LIT in Notebooks", + "provenance": [], + "collapsed_sections": [] + }, + "kernelspec": { + "name": "python3", + "display_name": "Python 3" + } + }, + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "h2c6PyqQaNiA" + }, + "source": [ + "# Using the Language Interpretability Tool in Notebooks\n", + "\n", + "This notebook shows use of the [Language Interpretability Tool](https://pair-code.github.io/lit) on a binary classifier for labelling statement sentiment (0 for negative, 1 for positive).\n", + "\n", + "The LitWidget object constructor takes a dict mapping model names to model objects, and a dict mapping dataset names to dataset objects. Those will be the datasets and models displayed in LIT. It also optionally takes in a `height` parameter for how tall to render the LIT UI in pixels (it defaults to 1000 pixels). Running the constructor will cause the LIT server to be started in the background, loading the models and datasets and enabling the UI to be served.\n", + "\n", + "Render the LIT UI in an output cell by calling the `render` method on the LitWidget object. The LIT UI can be rendered multiple times in separate cells if desired. The widget also contains a `stop` method to shut down the LIT server.\n", + "\n", + "Copyright 2020 Google LLC.\n", + "SPDX-License-Identifier: Apache-2.0" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "ukXamAB_FBM8" + }, + "source": [ + "# Install LIT and transformers packages. The transformers package is needed by the model and dataset we are using.\n", + "!pip install lit_nlp transformers==2.11.0" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "30l9ZyTjxJjf" + }, + "source": [ + "# Fetch the files needed by the model\n", + "!wget https://storage.googleapis.com/what-if-tool-resources/lit-models/sst2_tiny.tar.gz\n", + "!tar -xvf sst2_tiny.tar.gz" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "AWhbAZg57RpB" + }, + "source": [ + "# Create the LIT widget with the model and dataset to analyze.\n", + "from lit_nlp import notebook\n", + "from lit_nlp.examples.datasets import glue\n", + "from lit_nlp.examples.models import glue_models\n", + "\n", + "datasets = {'sst_dev': glue.SST2Data('validation')}\n", + "models = {'sst_tiny': glue_models.SST2Model('./')}\n", + "\n", + "widget = notebook.LitWidget(models, datasets, height=800)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "9GSfs1waBdLd" + }, + "source": [ + "# Render the widget\n", + "widget.render()" + ], + "execution_count": null, + "outputs": [] + } + ] +} \ No newline at end of file diff --git a/website/src/demos.md b/website/src/demos.md index 97291411..e7ff4bff 100644 --- a/website/src/demos.md +++ b/website/src/demos.md @@ -25,4 +25,7 @@ color: "#49596c" {% include partials/demo-card c-title: "Text generation", link: "/demos/t5.html", c-data-source: "CNN / Daily Mail", c-copy: "Use a T5 model to summarize text. For any example of interest, quickly find similar examples from the training set, using an approximate nearest-neighbors index.", tags: "T5, generation", external:"true" %} + + {% include partials/demo-card c-title: "Using LIT in notebooks", link: "https://colab.research.google.com/github/pair-code/lit/blob/main/examples/notebooks/LIT_sentiment_classifier.ipynb", + c-data-source: "CNN / Daily Mail", c-copy: "Use LIT directly inside a colab notebook. Explore binary classification for sentiment analysis from the General Language Understanding Evaluation (GLUE) benchmark suite.", tags: "colab, notebooks, BERT, binary classification", external:"true" %} diff --git a/website/src/index.md b/website/src/index.md index 9ef82f93..3bd844a1 100644 --- a/website/src/index.md +++ b/website/src/index.md @@ -31,6 +31,7 @@ LIT contains many built-in capabilities but is also customizable, with the abili For a similar tool to explore general-purpose machine learning models, check out the [What-If Tool](https://whatif-tool.dev). +LIT can be run as a standalone server, or inside of python notebook environments such as colab and jupyter. {% include partials/spacer height:50 %} @@ -99,7 +100,7 @@ And more... {% include partials/home-card image: '/assets/images/LIT_Contribute.png', action: 'CODE', title: 'Contribute to LIT', desc: 'LIT is open to anyone who wants to help develop and improve it!', cta-text:"View developer guide", link: 'https://github.com/PAIR-code/lit/blob/main/documentation/development.md', external:"true" %} - + {% include partials/home-card image: '/assets/images/LIT_Updates.png', action: 'UPDATES', title: 'Latest updates', desc: 'New features, updates, and improvements to LIT.', cta-text:"See release notes", link: 'https://github.com/PAIR-code/lit/blob/main/RELEASE.md' external:"true" %} diff --git a/website/src/setup.md b/website/src/setup.md index 0902581f..c442d86a 100644 --- a/website/src/setup.md +++ b/website/src/setup.md @@ -257,4 +257,28 @@ accordingly. Note: there are a few additional methods in the model API - see [`Model`](https://github.com/PAIR-code/lit/tree/main/lit_nlp/api/model.py) for details. +# Run LIT inside python notebooks + +It's very easy to use LIT inside of colab and jupyter notebooks. Just install +the pip package and use the `LitWidget` object with your models and datasets. + +``` +from lit_nlp import notebook + +# MulitiNLIData implements the Dataset API +datasets = { + 'mnli_matched': MultiNLIData('/path/to/dev_matched.tsv'), + 'mnli_mismatched': MultiNLIData('/path/to/dev_mismatched.tsv'), +} + +# NLIModel implements the Model API +models = { + 'model_foo': NLIModel('/path/to/model/foo/files'), + 'model_bar': NLIModel('/path/to/model/bar/files'), +} + +widget = notebook.LitWidget(models, datasets) +widget.render() +``` +