New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

一个无限循环下载的Bug #7

Open

ldcr002 opened this issue Jan 20, 2020 · 0 comments

ldcr002 commented Jan 20, 2020 •

edited

Loading

在使用这个脚本爬取一个目标网站的时候。发现脚本会无限循环爬取已经爬取过的目标。看了一下源代码。第40行是不是应该做如下修改？

if url in self.processed_url:  
    continue

The text was updated successfully, but these errors were encountered:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment