Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error from VMWare Windows Instance #58

Open
kwarmansen opened this issue Mar 12, 2020 · 1 comment
Open

Error from VMWare Windows Instance #58

kwarmansen opened this issue Mar 12, 2020 · 1 comment

Comments

@kwarmansen
Copy link

I am running fbcrawl from VMWare Windows 10 image. I have python 3.7, and I am seeing following error when I run the command:

2020-03-11 19:31:57 [fb] INFO: Going through the "save-device" checkpoint
2020-03-11 19:32:03 [fb] INFO: Scraping facebook page https://mbasic.facebook.com/cnn
2020-03-11 19:32:07 [scrapy.core.scraper] ERROR: Spider error processing <GET https://mbasic.facebook.com/cnn> (referer: https://mbasic.facebook.com/?_rdr)
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\twisted\internet\defer.py", line 1418, in _inlineCallbacks
result = g.send(result)
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\core\downloader\middleware.py", line 42, in process_request
defer.returnValue((yield download_func(request=request, spider=spider)))
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\twisted\internet\defer.py", line 1362, in returnValue
raise _DefGen_Return(val)
twisted.internet.defer._DefGen_Return: <200 https://mbasic.facebook.com/cnn>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\utils\defer.py", line 55, in mustbe_deferred
result = f(*args, **kw)
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\core\spidermw.py", line 60, in process_spider_input
return scrape_func(response, request, spider)
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\core\scraper.py", line 148, in call_spider
warn_on_generator_with_return_value(spider, callback)
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\utils\misc.py", line 202, in warn_on_generator_with_return_value
if is_generator_with_return_value(callable):
File "c:\users\user\appdata\local\programs\python\python37\lib\site-packages\scrapy\utils\misc.py", line 187, in is_generator_with_return_value
tree = ast.parse(dedent(inspect.getsource(callable)))
File "c:\users\user\appdata\local\programs\python\python37\lib\ast.py", line 35, in parse
return compile(source, filename, mode, PyCF_ONLY_AST)
File "", line 1
def parse_page(self, response):
^
IndentationError: unexpected indent
2020-03-11 19:32:07 [scrapy.core.engine] INFO: Closing spider (finished)

@georgevak
Copy link

same problem,
let me ask you, my machine is 32 bit, i saw in terminal it says
'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 '
'(KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36'}

does the fbcrawl runs on 32bit machines?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants