Supporting pydantic<3

[jamesbraza]

Closes #1726

I manually and locally tested with pydantic v2 using Python 3.11.4:

> pip install -e .[testing]
> pip list | grep pydantic
pydantic                  2.4.2
pydantic_core             2.10.1
> pytest tests/test_inference*.py
...
83 passed, 6 skipped, 16 warnings in 5.73s

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[jamesbraza]

Damn it didn't just work 🥲

Edit: linked two other PRs that should enable compatibility

[Wauplin]

Both PRs are now merged and I updated this PR to latest changes. Let's see if CI is happy now! 😄

[Wauplin]

Looks good! Made some local tests as well and both versions seem to be fully supported! Thanks for taking care of it @jamesbraza 🚀