Add tutorials and guides for LangChain #10770
Conversation
Signed-off-by: Ben Wilson <[email protected]>
Signed-off-by: Ben Wilson <[email protected]>
Signed-off-by: Ben Wilson <[email protected]>
Signed-off-by: Ben Wilson <[email protected]>
|
Documentation preview for 229d775 will be available here when this CircleCI job completes successfully. More info
|
github-actions
bot
added
area/docs
BenWilson2
requested review from
liangz1,
serena-ruan,
B-Step62 and
chenmoneygithub
| bottom portion shows the same queries being submitted to a LangChain chain that incorporates a conversation history state such that the entire conversation's history | ||
| is included with each subsequent input. Preserving conversational context in this manner is key to creating a "chat bot". | ||
|
|
||
| .. figure:: ../../_static/images/tutorials/llms/stateful-chains.png |
There was a problem hiding this comment.
not a comment, just curious - did you make these pictures? they are super cool
docs/source/llms/langchain/index.rst
Outdated
|
|
||
| - **Handling Dependency Installation in LangChain and MLflow**: LangChain and MLflow do not automatically install all dependencies. | ||
|
|
||
| Other packages that might be required for specific agents, retrievers, or tools may need to be explicitly defined when saving or logging your model. |
There was a problem hiding this comment.
nit: indent to render this paragraph under the bullet point.
docs/source/llms/langchain/index.rst
Outdated
|
|
||
| - **Declaring Extra Dependencies**: Use the ``extra_pip_requirements`` parameter when saving and logging. | ||
|
|
||
| When saving or logging your model that contains external dependencies that are not part of the core langchain installation, you will need these additional |
| ----------------------- | ||
|
|
||
| - **Simplified Logging and Loading**: MLflow's `langchain` flavor provides functions like `log_model()` and `load_model()`, enabling easy logging and retrieval of LangChain models within the MLflow ecosystem. | ||
| - **Python Function Flavor**: LangChain models logged in MLflow can be interpreted as generic Python functions, simplifying their deployment and use in diverse applications. |
There was a problem hiding this comment.
"Python Function Flavor" sounds a bit strange, maybe something like "Simplified deployment"?
|
|
||
|
|
||
| The ``langchain`` model flavor enables logging of `LangChain models <https://github.com/hwchase17/langchain>`_ in MLflow format via | ||
| the :py:func:`mlflow.langchain.save_model()` and :py:func:`mlflow.langchain.log_model()` functions. Use of these |
There was a problem hiding this comment.
I am curious if we want to promote the usage of save_model. I thought log_model is the one we hope users to use, and save_model is more like a private helper function of log_model.
There was a problem hiding this comment.
I'm still on the fence about this... it's a public API which is why I've been mentioning it in other flavors. Logging is definitely the preferred method, but saving does have its uses when declaring custom pyfuncs :)
| loaded_model = mlflow.pyfunc.load_model(logged_model.model_uri) | ||
|
|
||
| # Generate an inference result using the loaded model |
There was a problem hiding this comment.
the current rendering is a bit odd because we have a few layers inside print. Can we define a variable instead?
There was a problem hiding this comment.
# Generate an inference result using the loaded model
question = "What was the high temperature in SF yesterday in Fahrenheit? What is that number raised to the .023 power?"
print(loaded_model.predict([{"input": question}]))
There was a problem hiding this comment.
good call. Refactored the existing example
|
|
||
| Key Components to Specify in MLflow | ||
| """"""""""""""""""""""""""""""""""" | ||
| 1. **persist_dir**: The directory where the retriever object is stored. |
There was a problem hiding this comment.
As a reader I would like to have this explained better, basically I am seeking answers to the following questions.
- Is the vectorDB logged to Mlflow?
- if not, does that mean I have to maintain the same vectordb in my runtime?
There was a problem hiding this comment.
excellent ideas! Added sections highlighting these topics and adjusted the existing text to focus more on the fact that the vectorDB instance is a separate entity
docs/source/index.rst
Outdated
| <div class="header"> | ||
| Guide to the MLflow Prompt Engineering UI | ||
| Guide to using 🤗 Transformers in MLflow |
There was a problem hiding this comment.
nit: I think we can remove "Guide to" prefix from all cards, as this section only lists up guides anyway.
There was a problem hiding this comment.
great point! I think these will all be replaced soon with your ticket this sprint ;)
Removed the "Guide to using" from those card titles in the interim :)
| ------------------------------------------ | ||
| - **Chains**: Sequences of actions or steps hardcoded in code. Chains in LangChain combine various components like prompts, models, stateful memory, and output parsers to create a flow of processing steps. | ||
|
|
||
| The figure below shows an example of interfacing directly with a SaaS LLM via API calls with no context to the history of the conversation in the top portion. The |
There was a problem hiding this comment.
Hmm I'm not sure if statefull/stateless thing is relevant to Chain concept. Langchain has capability for it as well but it's in Memory not Chain itself. Since most SaaS LLM providers have that chat capability without doing any context management on client side, I don't think we need to highlight it here?
Their Chain abstraction should be rather for orchestrating different components in the system - in their words;
Using an LLM in isolation is fine for simple applications, but more complex applications require chaining LLMs - either with each other or with other components.
There was a problem hiding this comment.
great catch! I wonder if we could use another tutorial that shows how to use a ConversationalBuffer (in a follow-up PR later)... for now, removed that reference.
| :width: 90% | ||
| :align: center | ||
|
|
||
| - **Agents**: Dynamic constructs that use language models to choose a sequence of actions. Unlike chains, agents decide the order of actions based on inputs, tools available, and intermediate outcomes. |
There was a problem hiding this comment.
I really like the static vs dynamic comparison for Chain vs Agent here (the most effective way to describe the two). Can we show a single side-by-side picture illustrates the difference (like the right side would be the current langchain-agents.png)?
docs/source/llms/langchain/index.rst
Outdated
| </article> | ||
| </section> | ||
|
|
||
| Download the Advanced Tutorial Notebooks |
There was a problem hiding this comment.
nit: can we just put the link without section similar to above introductory tutorial?
docs/source/llms/langchain/index.rst
Outdated
| - **LLMChain Logging**: Integrate and log language model chains that interface with both SaaS and self-managed LLM models, for streamlined management and deployment. | ||
| - **Agent Logging**: Log complex LangChain agents, allowing language models to dynamically determine action sequences. | ||
| - **RetrievalQA Chain Logging**: Seamlessly manage RetrievalQA chains (RAG), combining retrieval capabilities with question-answering. | ||
| - **Retriever Logging**: Independently log and evaluate retrievers, assessing the quality of documents retrieved without LLM processing. |
There was a problem hiding this comment.
| - **LLMChain Logging**: Integrate and log language model chains that interface with both SaaS and self-managed LLM models, for streamlined management and deployment. | |
| - **Agent Logging**: Log complex LangChain agents, allowing language models to dynamically determine action sequences. | |
| - **RetrievalQA Chain Logging**: Seamlessly manage RetrievalQA chains (RAG), combining retrieval capabilities with question-answering. | |
| - **Retriever Logging**: Independently log and evaluate retrievers, assessing the quality of documents retrieved without LLM processing. | |
| MLflow support logging following LangChain objects: | |
| - `LLMChain <https://python.langchain.com/docs/modules/chains/foundational/llm_chain>`_ | |
| - `Agent Logging <https://python.langchain.com/docs/modules/agents/>`_ | |
| - `RetrievalQA <https://js.langchain.com/docs/modules/chains/popular/vector_db_qa>`_ | |
| - `Retriever <https://python.langchain.com/docs/modules/data_connection/>`_ |
I think links are more visually standing out here and concise by off-loading description to their words.
There was a problem hiding this comment.
100% agreed. Converted to links :)
| "source": [ | ||
| "## Introduction to RAG with MLflow and LangChain\n", | ||
| "\n", | ||
| "### Tutorial Overview\n", |
There was a problem hiding this comment.
Could we make this overview section shorter? It takes 2 page scroll now which feels a bit too long. I think we can do
- Reduce image size like 70%
- Consolidate some subsections. It takes huge blank space so small subsections with 2-3 sentences make the page too sparse imo.
| "#### Installation Requirement\n", | ||
| "Before proceeding with the tutorial, ensure that you have FAISS installed. For most users, especially those running this tutorial on personal or standard computing environments, `faiss-cpu` will suffice. You can install it via `pip`. In addition to `faiss`, you'll also need [Beautiful Soup](https://pypi.org/project/beautifulsoup4/), which can also be installed via `pip`.\n" |
There was a problem hiding this comment.
I think pip install command alone explains enough and more concise.
pip install faiss-cpu bs4
| "#### Understanding FAISS\n", | ||
| "For this tutorial, we will be utilizing [FAISS](https://github.com/facebookresearch/faiss/wiki) (Facebook AI Similarity Search, developed and maintained by the [Meta AI research group](https://ai.meta.com/tools/faiss/)), an efficient similarity search and clustering library. It's a highly useful library that easily handles large datasets and is capable of performing operations such as nearest neighbor search, which are critical in Retrieval Augmented Generation (RAG) systems. There are numerous other vector database solutions that can perform similar functionality; we are using FAISS in this tutorial due to its simplicity, ease of use, and fantastic performance.\n", | ||
| "\n", | ||
| "#### FAISS CPU vs. GPU\n", |
There was a problem hiding this comment.
Can we replace this section to one sentence like "if you want to run this tutorial with GPU, use faiss-gpu instead"? I think it's obvious what -cpu/-gpu postfix means.
| "source": [ | ||
| "### Understanding the RetrievalQA Response\n", | ||
| "\n", | ||
| "In this tutorial, we explore an advanced use case of MLflow and LangChain: Retrieval-Augmented Generation (RAG) using the RetrievalQA model. This approach combines text retrieval from a database with language model generation to answer specific queries.\n", |
There was a problem hiding this comment.
| "In this tutorial, we explore an advanced use case of MLflow and LangChain: Retrieval-Augmented Generation (RAG) using the RetrievalQA model. This approach combines text retrieval from a database with language model generation to answer specific queries.\n", | |
| "In this section of the tutorial, we explore an advanced use case of MLflow and LangChain: Retrieval-Augmented Generation (RAG) using the RetrievalQA model. This approach combines text retrieval from a database with language model generation to answer specific queries.\n", |
nit, same for a few other sections
| - **RetrievalQA Chain Logging**: Seamlessly manage RetrievalQA chains (RAG), combining retrieval capabilities with question-answering. | ||
| - **Retriever Logging**: Independently log and evaluate retrievers, assessing the quality of documents retrieved without LLM processing. | ||
|
|
||
| Capabilities of LangChain and MLflow |
There was a problem hiding this comment.
Another big benefit is Langchain version management. Langchain API changes very frequently and it's a popular headache for users. By logging chain as MLflow Model, user can associate the specific version to the chain and ensure the reproducibility.
There was a problem hiding this comment.
definitely! Rewrote a section that highlights this as a compelling reason to use MLflow for LangChain management
| """"""""""""""""" | ||
|
|
||
| Agents in LangChain leverage language models to dynamically determine and execute a sequence of actions, contrasting with the hardcoded sequences in chains. | ||
| To learn more about Agents and see additional examples within LangChain, you can [read the LangChain docs on Agents](https://python.langchain.com/docs/modules/agents/). |
There was a problem hiding this comment.
| To learn more about Agents and see additional examples within LangChain, you can [read the LangChain docs on Agents](https://python.langchain.com/docs/modules/agents/). | |
| To learn more about Agents and see additional examples within LangChain, you can `read the LangChain docs on Agents <https://python.langchain.com/docs/modules/agents/>`_. |
There was a problem hiding this comment.
oooooh awesome catch! That's the danger of doing a bunch of links in markdown and then adding an rst link. Thanks!
| "To install the dependent packages simply run:\n", | ||
| "\n", | ||
| "```bash\n", | ||
| "pip install 'openai<1' tiktoken langchain\n", |
There was a problem hiding this comment.
Should we add mlflow here?
| "\n", | ||
| "#### Installing SerpAPI\n", | ||
| "\n", | ||
| "To use SerpAPI, you'll need to install it via pip. It's a straightforward process, and the package is named `google-search-results`.\n", |
There was a problem hiding this comment.
Could we directly add code pip install google-search-results?
| "- **API Key**: After signing up, you will be provided with an API Key. This key is vital as it allows you to authenticate your requests to SerpAPI.\n", | ||
| "- **Export API Key**: For security and accessibility, it's recommended to export your SerpAPI API Key into your environment. This ensures that your key is both safe and readily available for use in your application.\n", | ||
| "\n", | ||
| "#### Why SerpAPI?\n", |
There was a problem hiding this comment.
Feel like Why SerpAPI should be placed before Installing SerpAPI
| "llm = OpenAI(temperature=0)\n", | ||
| "\n", | ||
| "# Define the two tools that the agent will use: a web search engine interface and a math execution engine.\n", | ||
| "tools = load_tools([\"serpapi\", \"llm-math\"], llm=llm)\n", | ||
| "\n", | ||
| "# Create the agent\n", | ||
| "agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)" |
There was a problem hiding this comment.
I would prefer moving each line to its corresponding section. "llm = OpenAI(temperature=0)\n" to Language Model Configuration section, and "tools = load_tools(["serpapi", "llm-math"], llm=llm)\n" to Defining Tools for the Agent section
| "with mlflow.start_run():\n", | ||
| " model_info = mlflow.langchain.log_model(agent, \"search-calculation\")\n", | ||
| "\n", | ||
| "# Load our agent model for use\n", | ||
| "loaded_model = mlflow.pyfunc.load_model(model_info.model_uri)" |
There was a problem hiding this comment.
Similar here, might want to split it into each section.
Signed-off-by: Ben Wilson <[email protected]>
| "\n", | ||
| "1. **Initialize a LangChain Agent**: Set up an agent with specific tools and a language model, ready to tackle our chosen task.\n", | ||
| "2. **Log and Load the Agent Model with MLflow**: Demonstrate the process of logging the agent in MLflow and then loading it for execution.\n", | ||
| "3. **Run a Real-World Prediction**: Use the agent to determine how much snow would result from the latest year's rainfall in Raleigh, NC, and conceptualize the size of a snowman that could be built with it.\n", |
| "outputs": [], | ||
| "source": [ | ||
| "# NOTE: The LangChain OpenAI wrapper uses `gpt-3.5-turbo-instruct` as the default model type as of version 0.0.331\n", | ||
| "llm = OpenAI(temperature=0)" |
There was a problem hiding this comment.
Maybe we should explicitly set the model? comments will go stale soon. model_name="gpt-3.5-turbo-instruct"
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "### FAISS Installation for the Tutorial\n", |
There was a problem hiding this comment.
Can we add a title before these sections to call out that we are building the retrieval part?
| " persist_dir=persist_dir,\n", | ||
| " )\n", | ||
| "\n", | ||
| "shutil.rmtree(temporary_directory)" |
There was a problem hiding this comment.
I am a little confused here - after we delete the temporary_directory, the persist_dir will not exist any more, does it mean that the vectordb is destroyed? then how do we load back the model from mlflow server?
There was a problem hiding this comment.
uhhhhh good catch. yikes. Notebook state, I tell ya!
Moved this cleanup to the last code execution stage of the notebook.
| } | ||
| ], | ||
| "source": [ | ||
| "answer1 = loaded_model.predict([{\"query\": \"What does the document say about trespassers?\"}])\n", |
There was a problem hiding this comment.
shall we move up this code block before "Understanding the RetrievalQA Response"?
There was a problem hiding this comment.
good call. It reads better with this change :)
| "answer4 = loaded_model.predict(\n", | ||
| " [\n", | ||
| " {\n", | ||
| " \"query\": \"Can I lease a small parcel of land from the Federal Government for a small buffalo-themed day spa for visitors to the park?\"\n", |
There was a problem hiding this comment.
nit: this renders pretty long on our website, can we break this string into multiple lines?
There was a problem hiding this comment.
adjusted a few of the longer lines in the queries
Signed-off-by: Ben Wilson <[email protected]>
Signed-off-by: Ben Wilson <[email protected]>
🛠 DevTools 🛠
Install mlflow from this PR
Checkout with GitHub CLI
Related Issues/PRs
#xxxWhat changes are proposed in this pull request?
Converting existing LangChain documentation to new format, adding new tutorials, and adjusting some navigation cards.
How is this PR tested?
Does this PR require documentation update?
Release Notes
Is this a user-facing change?
What component(s), interfaces, languages, and integrations does this PR affect?
Components
area/artifacts: Artifact stores and artifact loggingarea/build: Build and test infrastructure for MLflowarea/deployments: MLflow Deployments client APIs, server, and third-party Deployments integrationsarea/docs: MLflow documentation pagesarea/examples: Example codearea/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registryarea/models: MLmodel format, model serialization/deserialization, flavorsarea/recipes: Recipes, Recipe APIs, Recipe configs, Recipe Templatesarea/projects: MLproject format, project running backendsarea/scoring: MLflow Model server, model deployment tools, Spark UDFsarea/server-infra: MLflow Tracking server backendarea/tracking: Tracking Service, tracking client APIs, autologgingInterface
area/uiux: Front-end, user experience, plotting, JavaScript, JavaScript dev serverarea/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Modelsarea/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registryarea/windows: Windows supportLanguage
language/r: R APIs and clientslanguage/java: Java APIs and clientslanguage/new: Proposals for new client languagesIntegrations
integrations/azure: Azure and Azure ML integrationsintegrations/sagemaker: SageMaker integrationsintegrations/databricks: Databricks integrationsHow should the PR be classified in the release notes? Choose one:
rn/none- No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" sectionrn/breaking-change- The PR will be mentioned in the "Breaking Changes" sectionrn/feature- A new user-facing feature worth mentioning in the release notesrn/bug-fix- A user-facing bug fix worth mentioning in the release notesrn/documentation- A user-facing documentation change worth mentioning in the release notes