Enhanced InferenceClient

.github/workflows/python-tests.yml

@@ -75,11 +75,15 @@ jobs:
 
           esac
 
+      # Easy debugging if CI is broken
+      - name: Pip freeze
+        run: pip freeze
+
       # Run tests
       - name: Run tests
         working-directory: ./src # For code coverage to work
         run: |
-          PYTEST="python -m pytest --cov=./huggingface_hub --cov-report=xml:../coverage.xml"
+          PYTEST="python -m pytest --cov=./huggingface_hub --cov-report=xml:../coverage.xml --vcr-record=none"
 
           case "${{ matrix.test_name }}" in
 
@@ -150,10 +154,14 @@ jobs:
           pip install .[testing]
         # sudo apt install -y libsndfile1-dev
 
+      # Easy debugging if CI is broken
+      - name: Pip freeze
+        run: pip freeze
+
       # Run tests
       - name: Run tests
         working-directory: ./src # For code coverage to work
-        run: python -m pytest --cov=./huggingface_hub --cov-report=xml:../coverage.xml ../tests
+        run: python -m pytest --cov=./huggingface_hub --cov-report=xml:../coverage.xml --vcr-record=none ../tests
 
       # Upload code coverage
       - name: Upload coverage reports to Codecov with GitHub Action

docs/source/_redirects.yml

@@ -7,4 +7,5 @@ how-to-manage: guides/repository
 how-to-model-cards: guides/model-cards
 how-to-upstream: guides/upload
 search-the-hub: guides/search
-guides/manage_spaces: guides/manage-spaces
+guides/manage_spaces: guides/manage-spaces
+package_reference/inference_api: package_reference/inference_client

docs/source/_toctree.yml

@@ -54,8 +54,8 @@
       title: Downloading files
     - local: package_reference/mixins
       title: Mixins & serialization methods
-    - local: package_reference/inference_api
-      title: Inference API
+    - local: package_reference/inference_client
+      title: Inference Client
     - local: package_reference/hf_file_system
       title: HfFileSystem
     - local: package_reference/utilities

docs/source/package_reference/inference_client.mdx

@@ -0,0 +1,48 @@
+# Inference
+
+Inference is the process of using a trained model to make predictions on new data. This process can be compute-intensive
+which makes it interesting to run on a remote machine. The `huggingface_hub` library provides an easy way to call a
+service that runs fast inference for hosted models. There are several services you can connect to:
+- [Inference API](https://huggingface.co/docs/api-inference/index): a service that allows you to run accelerated inference
+on Hugging Face's infrastructure and for free. This service is a fast way to get started, tests different models and
+prototype AI products.
+- [Inference Endpoints](https://huggingface.co/inference-endpoints): a product to easily deploy models to production.
+Inference is run by Hugging Face in a dedicated, fully managed infrastructure on the Cloud of your choice.
+- [SageMaker Hugging Face Inference Toolkit](https://github.com/aws/sagemaker-huggingface-inference-toolkit)
+
+All of these services can be called with the `InferenceClient` client. Please refer to [this guide](../guides/inference)
+for more information on how to use it.
+
+## Inference Client
+
+[[autodoc]] InferenceClient
+
+### InferenceTimeoutError
+
+[[autodoc]] InferenceTimeoutError
+
+### Return types
+
+For most tasks, the return value has a built-in type (string, list, image...). Here is a list for the more complex types.
+
+[[autodoc]] huggingface_hub._inference_types.ClassificationOutput
+
+[[autodoc]] huggingface_hub._inference_types.ConversationalOutputConversation
+
+[[autodoc]] huggingface_hub._inference_types.ConversationalOutput
+
+[[autodoc]] huggingface_hub._inference_types.ImageSegmentationOutput
+
+## InferenceAPI
+
+[`InferenceAPI`] is the legacy way to call the Inference API. The interface is more simplistic and requires to know
+the input parameters and output format for each task. It also lacks the ability to connect to other services like
+Inference Endpoints or AWS SageMaker. It is still supported but we recommend using [`InferenceClient`] whenever possible.
+Check out [this guide](../guides/inference#legacy-inferenceapi-client) to learn how to switch from [`InferenceAPI`] to
+[`InferenceClient`] in your scripts.
+
+[[autodoc]] InferenceApi
+    - __init__
+    - __call__
+    - all
+

setup.py

@@ -49,9 +49,12 @@ def get_version() -> str:
     "pytest-cov",
     "pytest-env",
     "pytest-xdist",
+    "pytest-vcr",  # to mock Inference
+    "urllib3<2.0",  # VCR.py broken with urllib3 2.0 (see https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html)
     "soundfile",
     "Pillow",
     "gradio",  # to test webhooks
+    "numpy",  # for embeddings
 ]
 
 # Typing extra dependencies list is duplicated in `.pre-commit-config.yaml`

src/huggingface_hub/__init__.py

@@ -52,6 +52,10 @@
 # WARNING: any comment added in this dictionary definition will be lost when
 # re-generating the file !
 _SUBMOD_ATTRS = {
+    "_inference": [
+        "InferenceClient",
+        "InferenceTimeoutError",
+    ],
     "_login": [
         "interpreter_login",
         "login",
@@ -342,6 +346,10 @@ def __dir__():
 # make style
 # ```
 if TYPE_CHECKING:  # pragma: no cover
+    from ._inference import (
+        InferenceClient,  # noqa: F401
+        InferenceTimeoutError,  # noqa: F401
+    )
     from ._login import (
         interpreter_login,  # noqa: F401
         login,  # noqa: F401