Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internals: add identifying library-plus-version string to HEADERS #1221

Merged
merged 5 commits into from
Aug 26, 2024
Merged

internals: add identifying library-plus-version string to HEADERS #1221

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
c0e3636
internals: add identifying library-plus-version string to `HEADERS`
jayaddison Aug 15, 2024
605b7d9
Use a native recipe-scrapers user-agent header
jayaddison Aug 26, 2024
ff17d9a
Fixup: add missing `__version__` import
jayaddison Aug 26, 2024
e96afee
Add `compatible` user-agent keyword
jayaddison Aug 26, 2024
0c71a1a
Revert "Fixup: add missing `__version__` import"
jayaddison Aug 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion recipe_scrapers/_abstract.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

from bs4 import BeautifulSoup

from recipe_scrapers.__version__ import __version__
from recipe_scrapers.settings import settings

from ._exceptions import ElementNotFoundInHtml
Expand All @@ -14,7 +15,7 @@

# Some sites close their content for 'bots', so user-agent must be supplied
HEADERS = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"
"User-Agent": f"Mozilla/5.0 (compatible; Windows NT 10.0; Win64; x64; rv:{__version__}) recipe-scrapers/{__version__}"
}


Expand Down
Loading