Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove unreachable source for FreeBeacon #521

Merged
merged 2 commits into from
May 20, 2024
Merged

Conversation

MaxDall
Copy link
Collaborator

@MaxDall MaxDall commented May 17, 2024

No description provided.

@MaxDall MaxDall requested a review from addie9800 May 17, 2024 06:59
Copy link
Collaborator

@addie9800 addie9800 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating this 👍

@@ -121,7 +121,7 @@ class US(PublisherEnum):
FreeBeacon = PublisherSpec(
name="The Washington Free Beacon",
domain="https://freebeacon.com/",
sources=[NewsMap("https://freebeacon.com/post_google_news.xml")],
sources=[],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a RSS Feed we could use as an altenative. The time I checked, they were having server issues (status 500), but it still seems to be valid and perhaps we can use it as soon as they're back online: https://freebeacon.com/feed/

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also found the working sitemap: https://freebeacon.com/wp-sitemap.xml
The most recent post in there is from yesterday, so I think it's still up-to-date. It needs to be crawled in reverse though with an inverse filter of posts-post

@MaxDall MaxDall requested a review from addie9800 May 18, 2024 12:50
@MaxDall MaxDall merged commit 1fa8910 into master May 20, 2024
5 checks passed
@MaxDall MaxDall deleted the remove-unreachable-source branch May 20, 2024 19:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants