Error on scraping followers

[Thicool]

Works fine but scraping followers gives me:

Scraping followers...
Traceback (most recent call last):
File "instagramcrawler.py", line 302, in
caption='None')
File "instagramcrawler.py", line 96, in crawl
self.scrape_followers_or_following(crawl_type, query, number)
File "instagramcrawler.py", line 214, in scrape_followers_or_following
title = self._driver.find_element_by_xpath(FOLLOW_PATH)
File "...\selenium\webdriver\remote\webdriver.py", line 313, in find_element_by_xpath
return self.find_element(by=By.XPATH, value=xpath)
File "...\selenium\webdriver\remote\webdriver.py", line 791, in find_element
'value': value})['value']
File "...\selenium\webdriver\remote\webdriver.py", line 256, in execute
self.error_handler.check_response(response)
File "...\selenium\webdriver\remote\errorhandler.py", line 194, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: //div[contains(text(), 'Followers')]

Any idea? Thanks for your help.

[tzuhsial]

Can you give me more information & environment on your query?
(E.g. the command you used to crawl followers)
Thanks!

[Thicool]

Hey Antonie, thanks for your reply. Here are the commands that i use and the response i get. I have no problem scraping pictures. After executing the code, firefox starts and i enter my login (in case of followers), which then opens the instagram-account page and open the follower-list. However, script stops at that. I hope you can help me! Here is the command line in and output:

…

________________________________________________________________________________________________________________ C:\Users\Jankl\PycharmProjects\InstagramCrawler-master>python instagramcrawler.py -q instagram -t followers -n 30 dir_prefix: ./data/, query: instagram, crawl_type: followers, number: 30, caption: False You will need to login to crawl followers Scraping followers... Traceback (most recent call last): File "instagramcrawler.py", line 297, in <module> main() File "instagramcrawler.py", line 293, in main caption=args.caption) File "instagramcrawler.py", line 96, in crawl self.scrape_followers_or_following(crawl_type, query, number) File "instagramcrawler.py", line 214, in scrape_followers_or_following title = self._driver.find_element_by_xpath(FOLLOW_PATH) File "C:\Users\Jankl\Anaconda2\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 313, in find_element_by_xpath return self.find_element(by=By.XPATH, value=xpath) File "C:\Users\Jankl\Anaconda2\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 791, in find_element 'value': value})['value'] File "C:\Users\Jankl\Anaconda2\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 256, in execute self.error_handler.check_response(response) File "C:\Users\Jankl\Anaconda2\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 194, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: //div[contains(text(), 'Followers')]

_________________________________________________________________________________________________________________ When i add paranteces on followers, i get another error message:

_________________________________________________________________________________________________________________ C:\Users\Jankl\PycharmProjects\InstagramCrawler-master>python instagramcrawler.py -q instagram -t 'followers' -n 30 dir_prefix: ./data/, query: instagram, crawl_type: 'followers', number: 30, caption: False Saving... Saving to directory..../data/instagram Quitting driver... C:\Users\Jankl\PycharmProjects\InstagramCrawler-master>

_________________________________________________________________________________________________________________ Thanks for your help :) 2017-06-30 14:22 GMT+02:00 Antonie Lin <[email protected]>:

Can you give me more information & environment on your query? (E.g. the command you used to crawl followers) Thanks! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#7 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AVm_4jEV85BqAgbrUue8mIGbsaaeX9n1ks5sJOiUgaJpZM4OJSl2> .

[tzuhsial]

@Thicool
Did you login your instagram account?
(The webdriver will open up the browser for you to login)

[Thicool]

Yes it opens firefox and i login, then the driver continues to visit the follower Page. Everything works fine until here. Maybe i try another computer later. Regards, Jan Am 03.07.2017 6:11 vorm. schrieb "Antonie Lin" <[email protected]>:

…

Did you login your instagram account? (The webdriver will open up the browser for you to login) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#7 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AVm_4ufCdBxcOeHSb2AihsgLXtdfjRBJks5sKGnUgaJpZM4OJSl2> .

[tzuhsial]

@Thicool
I think its because after clicking on the element,
my code only waits for 1 second before scrolling the followers list
I'll fix and push it later

Antonie

[tzuhsial]

@Thicool
Pushed, please check if you have time...
And also, thanks for notifying!

Regards,
Antonie Lin

[Thicool]

Thanks for your work, but the problem still exists in my case. I Attached Error Message and Gecklog.

Hope that helps!

error.txt
geckodriver.txt

[tzuhsial]

@Thicool
It works for me when crawling followers/ following at the moment
The only solution I know have is to set the timeout value a bit larger(make it wait a little longer)...

[mrpatricque]

Hi there! I am encountering the same issue here! How can I set the timeout value a bit larger? I can crawl for photos, but get the same error for followers/following. When crawling photos, there is no login done, and when I include crawling reactions, I get a different, more complicated error. Thanks!