Skip to content

Commit

Permalink
[Feat] Webhook server (#1410)
Browse files Browse the repository at this point in the history 
* better default ui

* Add docstrings and tests

* add webhooks extra

* raise if gradio not installed

* fix setup.py

* fix windows tests

* Renamed WebhookApp to WebhooksServer

* rename tests as well

* make test more robust

* FIX midddleware issue

* make stlye

* flag WebhookServer as experimental + run server in notebook

* doc for experimental feature

* docstring

* rename to 'as_webhook_endpoint'

* doc

* fix docs?

* make style

* doc

* doc

* doc

* odc

* doc

* Rename to webhook_endpoint

* Add pinned attribute to WebhookPayloadDiscussion

* fix tests

* remove middleware

* Apply suggestions from code review

Co-authored-by: Steven Liu <[email protected]>

* add a webhooks guide

* typos

* syntxa

* add tip

---------

Co-authored-by: Steven Liu <[email protected]>
  • Loading branch information
@Wauplin @stevhliu
Wauplin and stevhliu authored Apr 14, 2023
1 parent 043e37e commit 25e20ef
Show file tree
Hide file tree
Showing 17 changed files with 1,131 additions and 2 deletions.
4 changes: 4 additions & 0 deletions docs/source/_toctree.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@
title: Manage your Space
- local: guides/integrations
title: Integrate a library
- local: guides/webhooks_server
title: Webhooks server
- title: "Conceptual guides"
sections:
- local: concepts/git_vs_http
Expand Down Expand Up @@ -66,3 +68,5 @@
title: Repo Cards and Repo Card Data
- local: package_reference/space_runtime
title: Space runtime
- local: package_reference/webhooks_server
title: Webhooks server
9 changes: 9 additions & 0 deletions docs/source/guides/overview.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -105,5 +105,14 @@ Take a look at these guides to learn how to use huggingface_hub to solve real-wo
</p>
</a>

<a class="!no-underline border dark:border-gray-700 p-5 rounded-lg shadow hover:shadow-lg"
href="./webhooks_server">
<div class="w-full text-center bg-gradient-to-br from-indigo-400 to-indigo-500 rounded-lg py-1.5 font-semibold mb-5 text-white text-lg leading-relaxed">
Webhooks server
</div><p class="text-gray-700">
How to create a server to receive Webhooks and deploy it as a Space?
</p>
</a>

</div>
</div>
195 changes: 195 additions & 0 deletions docs/source/guides/webhooks_server.mdx
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
# Webhooks Server

Webhooks are a foundation for MLOps-related features. They allow you to listen for new changes on specific repos or to
all repos belonging to particular users/organizations you're interested in following. This guide will explain how to
leverage `huggingface_hub` to create a server listening to webhooks and deploy it to a Space. It assumes you are
familiar with the concept of webhooks on the Huggingface Hub. To learn more about webhooks themselves, you can read
this [guide](https://huggingface.co/docs/hub/webhooks) first.

The base class that we will use in this guide is [`WebhooksServer`]. It is a class for easily configuring a server that
can receive webhooks from the Huggingface Hub. The server is based on a [Gradio](https://gradio.app/) app. It has a UI
to display instructions for you or your users and an API to listen to webhooks.

<Tip>

To see a running example of a webhook server, check out the [Spaces CI Bot](https://huggingface.co/spaces/spaces-ci-bot/webhook)
one. It is a Space that launches ephemeral environments when a PR is opened on a Space.

</Tip>

<Tip warning={true}>

This is an [experimental feature](../package_reference/environment_variables#hfhubdisableexperimentalwarning). This
means that we are still working on improving the API. Breaking changes might be introduced in the future without prior
notice. Make sure to pin the version of `huggingface_hub` in your requirements.

</Tip>


## Create an endpoint

Implementing a webhook endpoint is as simple as decorating a function. Let's see a first example to explain the main
concepts:

```python
# app.py
from huggingface_hub import webhook_endpoint, WebhookPayload

@webhook_endpoint
async def trigger_training(payload: WebhookPayload) -> None:
if payload.repo.type == "dataset" and payload.event.action == "update":
# Trigger a training job if a dataset is updated
...
```

Save this snippet in a file called `'app.py'` and run it with `'python app.py'`. You should see a message like this:

```text
Webhook secret is not defined. This means your webhook endpoints will be open to everyone.
To add a secret, set `WEBHOOK_SECRET` as environment variable or pass it at initialization:
`app = WebhooksServer(webhook_secret='my_secret', ...)`
For more details about webhook secrets, please refer to https://huggingface.co/docs/hub/webhooks#webhook-secret.
Running on local URL: http://127.0.0.1:7860
Running on public URL: https://1fadb0f52d8bf825fc.gradio.live
This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces
Webhooks are correctly setup and ready to use:
- POST https://1fadb0f52d8bf825fc.gradio.live/webhooks/trigger_training
Go to https://huggingface.co/settings/webhooks to setup your webhooks.
```

Good job! You just launched a webhook server! Let's break down what happened exactly:

1. By decorating a function with [`webhook_endpoint`], a [`WebhooksServer`] object has been created in the background.
As you can see, this server is a Gradio app running on http://127.0.0.1:7860. If you open this URL in your browser, you
will see a landing page with instructions about the registered webhooks.
2. A Gradio app is a FastAPI server under the hood. A new POST route `/webhooks/trigger_training` has been added to it.
This is the route that will listen to webhooks and run the `trigger_training` function when triggered. FastAPI will
automatically parse the payload and pass it to the function as a [`WebhookPayload`] object. This is a `pydantic` object
that contains all the information about the event that triggered the webhook.
3. The Gradio app also opened a tunnel to receive requests from the internet. This is the interesting part: you can
configure a Webhook on https://huggingface.co/settings/webhooks pointing to your local machine. This is useful for
debugging your webhook server and quickly iterating before deploying it to a Space.
4. Finally, the logs also tell you that your server is currently not secured by a secret. This is not problematic for
local debugging but is to keep in mind for later.

<Tip warning={true}>

By default, the server is started at the end of your script. If you are running it in a notebook, you can start the
server manually by calling `decorated_function.run()`. Since a unique server is used, you only have to start the server
once even if you have multiple endpoints.

</Tip>


## Configure a Webhook

Now that you have a webhook server running, you want to configure a Webhook to start receiving messages.
Go to https://huggingface.co/settings/webhooks, click on "Add a new webhook" and configure your Webhook. Set the target
repositories you want to watch and the Webhook URL, here `https://1fadb0f52d8bf825fc.gradio.live/webhooks/trigger_training`.

<div class="flex justify-center">
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/configure_webhook.png"/>
</div>

And that's it! You can now trigger that webhook by updating the target repository (e.g. push a commit). Check the
Activity tab of your Webhook to see the events that have been triggered. Now that you have a working setup, you can
test it and quickly iterate. If you modify your code and restart the server, your public URL might change. Make sure
to update the webhook configuration on the Hub if needed.

## Deploy to a Space

Now that you have a working webhook server, the goal is to deploy it to a Space. Go to https://huggingface.co/new-space
to create a Space. Give it a name, select the Gradio SDK and click on "Create Space". Upload your code to the Space
in a file called `app.py`. Your Space will start automatically! For more details about Spaces, please refer to this
[guide](https://huggingface.co/docs/hub/spaces-overview).

Your webhook server is now running on a public Space. If most cases, you will want to secure it with a secret. Go to
your Space settings > Section "Repository secrets" > "Add a secret". Set the `WEBHOOK_SECRET` environment variable to
the value of your choice. Go back to the [Webhooks settings](https://huggingface.co/settings/webhooks) and set the
secret in the webhook configuration. Now, only requests with the correct secret will be accepted by your server.

And this is it! Your Space is now ready to receive webhooks from the Hub. Please keep in mind that if you run the Space
on a free 'cpu-basic' hardware, it will be shut down after 48 hours of inactivity. If you need a permanent Space, you
should consider setting to an [upgraded hardware](https://huggingface.co/docs/hub/spaces-gpus#hardware-specs).

## Advanced usage

The guide above explained the quickest way to setup a [`WebhooksServer`]. In this section, we will see how to customize
it further.

### Multiple endpoints

You can register multiple endpoints on the same server. For example, you might want to have one endpoint to trigger
a training job and another one to trigger a model evaluation. You can do this by adding multiple `@webhook_endpoint`
decorators:

```python
# app.py
from huggingface_hub import webhook_endpoint, WebhookPayload

@webhook_endpoint
async def trigger_training(payload: WebhookPayload) -> None:
if payload.repo.type == "dataset" and payload.event.action == "update":
# Trigger a training job if a dataset is updated
...

@webhook_endpoint
async def trigger_evaluation(payload: WebhookPayload) -> None:
if payload.repo.type == "model" and payload.event.action == "update":
# Trigger an evaluation job if a model is updated
...
```

Which will create two endpoints:

```text
(...)
Webhooks are correctly setup and ready to use:
- POST https://1fadb0f52d8bf825fc.gradio.live/webhooks/trigger_training
- POST https://1fadb0f52d8bf825fc.gradio.live/webhooks/trigger_evaluation
```

### Custom server

To get more flexibility, you can also create a [`WebhooksServer`] object directly. This is useful if you want to
customize the landing page of your server. You can do this by passing a [Gradio UI](https://gradio.app/docs/#blocks)
that will overwrite the default one. For example, you can add instructions for your users or add a form to manually
trigger the webhooks. When creating a [`WebhooksServer`], you can register new webhooks using the
[`~WebhooksServer.add_webhook`] decorator.

Here is a complete example:

```python
import gradio as gr
from fastapi import Request
from huggingface_hub import WebhooksServer, WebhookPayload

# 1. Define UI
with gr.Blocks() as ui:
...

# 2. Create WebhooksServer with custom UI and secret
app = WebhooksServer(ui=ui, webhook_secret="my_secret_key")

# 3. Register webhook with explicit name
@app.add_webhook("/say_hello")
async def hello(payload: WebhookPayload):
return {"message": "hello"}

# 4. Register webhook with implicit name
@app.add_webhook
async def goodbye(payload: WebhookPayload):
return {"message": "goodbye"}

# 5. Start server (optional)
app.run()
```

1. We define a custom UI using Gradio blocks. This UI will be displayed on the landing page of the server.
2. We create a [`WebhooksServer`] object with a custom UI and a secret. The secret is optional and can be set with
the `WEBHOOK_SECRET` environment variable.
3. We register a webhook with an explicit name. This will create an endpoint at `/webhooks/say_hello`.
4. We register a webhook with an implicit name. This will create an endpoint at `/webhooks/goodbye`.
5. We start the server. This is optional as your server will automatically be started at the end of the script.
8 changes: 8 additions & 0 deletions docs/source/package_reference/environment_variables.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,14 @@ to disable this warning.

For more details, see [cache limitations](../guides/manage-cache#limitations).

### HF_HUB_DISABLE_EXPERIMENTAL_WARNING

Some features of `huggingface_hub` are experimental. This means you can use them but we do not guarantee they will be
maintained in the future. In particular, we might update the API or behavior of such features without any deprecation
cycle. A warning message is triggered when using an experimental feature to warn you about it. If you're comfortable debugging any potential issues using an experimental feature, you can set `HF_HUB_DISABLE_EXPERIMENTAL_WARNING=1` to disable the warning.

If you are using an experimental feature, please let us know! Your feedback can help us design and improve it.

### HF_HUB_DISABLE_TELEMETRY

By default, some data is collected by HF libraries (`transformers`, `datasets`, `gradio`,..) to monitor usage, debug issues and help prioritize features.
Expand Down
80 changes: 80 additions & 0 deletions docs/source/package_reference/webhooks_server.mdx
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Webhooks Server

Webhooks are a foundation for MLOps-related features. They allow you to listen for new changes on specific repos or to
all repos belonging to particular users/organizations you're interested in following. To learn
more about webhooks on the Huggingface Hub, you can read the Webhooks [guide](https://huggingface.co/docs/hub/webhooks).

<Tip>

Check out this [guide](../guides/webhooks_server) for a step-by-step tutorial on how to setup your webhooks server and
deploy it as a Space.

</Tip>

<Tip warning={true}>

This is an experimental feature. This means that we are still working on improving the API. Breaking changes might be
introduced in the future without prior notice. Make sure to pin the version of `huggingface_hub` in your requirements.
A warning is triggered when you use an experimental feature. You can disable it by setting `HF_HUB_DISABLE_EXPERIMENTAL_WARNING=1` as an environment variable.

</Tip>

## Server

The server is a [Gradio](https://gradio.app/) app. It has a UI to display instructions for you or your users and an API
to listen to webhooks. Implementing a webhook endpoint is as simple as decorating a function. You can then debug it
by redirecting the Webhooks to your machine (using a Gradio tunnel) before deploying it to a Space.

### WebhooksServer

[[autodoc]] huggingface_hub.WebhooksServer

### @webhook_endpoint

[[autodoc]] huggingface_hub.webhook_endpoint

## Payload

[`WebhookPayload`] is the main data structure that contains the payload from Webhooks. This is
a `pydantic` class which makes it very easy to use with FastAPI. If you pass it as a parameter to a webhook endpoint, it
will be automatically validated and parsed as a Python object.

For more information about webhooks payload, you can refer to the Webhooks Payload [guide](https://huggingface.co/docs/hub/webhooks#webhook-payloads).

[[autodoc]] huggingface_hub.WebhookPayload

### WebhookPayload

[[autodoc]] huggingface_hub.WebhookPayload

### WebhookPayloadComment

[[autodoc]] huggingface_hub.WebhookPayloadComment

### WebhookPayloadDiscussion

[[autodoc]] huggingface_hub.WebhookPayloadDiscussion

### WebhookPayloadDiscussionChanges

[[autodoc]] huggingface_hub.WebhookPayloadDiscussionChanges

### WebhookPayloadEvent

[[autodoc]] huggingface_hub.WebhookPayloadEvent

### WebhookPayloadMovedTo

[[autodoc]] huggingface_hub.WebhookPayloadMovedTo

### WebhookPayloadRepo

[[autodoc]] huggingface_hub.WebhookPayloadRepo

### WebhookPayloadUrl

[[autodoc]] huggingface_hub.WebhookPayloadUrl

### WebhookPayloadWebhook

[[autodoc]] huggingface_hub.WebhookPayloadWebhook
2 changes: 2 additions & 0 deletions setup.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ def get_version() -> str:

extras["tensorflow"] = ["tensorflow", "pydot", "graphviz"]


extras["testing"] = extras["cli"] + [
"jedi",
"Jinja2",
Expand All @@ -50,6 +51,7 @@ def get_version() -> str:
"pytest-xdist",
"soundfile",
"Pillow",
"gradio", # to test webhooks
]

# Typing extra dependencies list is duplicated in `.pre-commit-config.yaml`
Expand Down
30 changes: 30 additions & 0 deletions src/huggingface_hub/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,21 @@
"SpaceRuntime",
"SpaceStage",
],
"_webhooks_payload": [
"WebhookPayload",
"WebhookPayloadComment",
"WebhookPayloadDiscussion",
"WebhookPayloadDiscussionChanges",
"WebhookPayloadEvent",
"WebhookPayloadMovedTo",
"WebhookPayloadRepo",
"WebhookPayloadUrl",
"WebhookPayloadWebhook",
],
"_webhooks_server": [
"WebhooksServer",
"webhook_endpoint",
],
"community": [
"Discussion",
"DiscussionComment",
Expand Down Expand Up @@ -330,6 +345,21 @@ def __dir__():
SpaceRuntime, # noqa: F401
SpaceStage, # noqa: F401
)
from ._webhooks_payload import (
WebhookPayload, # noqa: F401
WebhookPayloadComment, # noqa: F401
WebhookPayloadDiscussion, # noqa: F401
WebhookPayloadDiscussionChanges, # noqa: F401
WebhookPayloadEvent, # noqa: F401
WebhookPayloadMovedTo, # noqa: F401
WebhookPayloadRepo, # noqa: F401
WebhookPayloadUrl, # noqa: F401
WebhookPayloadWebhook, # noqa: F401
)
from ._webhooks_server import (
WebhooksServer, # noqa: F401
webhook_endpoint, # noqa: F401
)
from .community import (
Discussion, # noqa: F401
DiscussionComment, # noqa: F401
Expand Down
Loading

0 comments on commit 25e20ef

Please sign in to comment.