Chrome Webdriver error: DevToolsActivePort file doesn't exist

[DerLeole]

Im trying to crawl Immobilienscout and have it all setup on the nwest version, but constantly run into this error:

[2022/08/06 10:35:58| |DEBUG ]: ====== WebDriver manager ======
[2022/08/06 10:35:58| |DEBUG ]: Get LATEST chromedriver version for google-chrome 104.0.5112
[2022/08/06 10:35:58| |DEBUG ]: Driver [/home/flathunter/.wdm/drivers/chromedriver/linux64/104.0.5112/chromedriver] found in cache
Traceback (most recent call last):
File "flathunt.py", line 105, in
main()
File "flathunt.py", line 76, in main
config.init_searchers()
File "/opt/flathunter/flathunter/config.py", line 44, in init_searchers
CrawlImmobilienscout(self),
File "/opt/flathunter/flathunter/crawl_immobilienscout.py", line 38, in init
self.driver = self.configure_driver(driver_arguments)
File "/opt/flathunter/flathunter/abstract_crawler.py", line 62, in configure_driver
options=chrome_options
File "/home/flathunter/.local/share/virtualenvs/flathunter--s35lxKo/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 72, in init
service_log_path, service, keep_alive)
File "/home/flathunter/.local/share/virtualenvs/flathunter--s35lxKo/lib/python3.7/site-packages/selenium/webdriver/chromium/webdriver.py", line 97, in init
options=options)
File "/home/flathunter/.local/share/virtualenvs/flathunter--s35lxKo/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 277, in init
self.start_session(capabilities, browser_profile)
File "/home/flathunter/.local/share/virtualenvs/flathunter--s35lxKo/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 370, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/home/flathunter/.local/share/virtualenvs/flathunter--s35lxKo/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 435, in execute
self.error_handler.check_response(response)
File "/home/flathunter/.local/share/virtualenvs/flathunter--s35lxKo/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: DevToolsActivePort file doesn't exist
Stacktrace:

My driver arguments so far are:
driver_arguments:
- "--headless"
- "--disable-dev-shm-usage"
- "--no-sandbox"
- "--disable-extensions"

[DerLeole]

To add on, I reinstalled google-chrome-stable already, run on Debian10, and did NOT use root to run the bot.

[alexanderroidl]

Hi Leolele99. 👋 Please provide context to the following questions:

• Are you running a 64-bit-based Debian?
• Does Google Chrome start normally by itself?
• Is Flathunter working as expected when not crawling ImmoScout24?

Thank you! 😊

[DerLeole]

Hi,

• Yes it is 64-bit Debian 10.
• Kind of? As its a server without any Desktop environment I can only start it in headless mode which seems to work without error.
• Yes, everything else works fine and is in fact working without problem for days now.

Thansk for taking a look at this.

[abuchmueller]

I have the same issue when running Docker (macOS Intel).

[2022/08/20 13:19:09|abstract_crawler.py     |INFO    ]: Initializing Chrome WebDriver for crawler "CrawlImmobilienscout"...
Traceback (most recent call last):
  File "flathunt.py", line 105, in <module>
    main()
  File "flathunt.py", line 76, in main
    config.init_searchers()
  File "/usr/src/app/flathunter/config.py", line 44, in init_searchers
    CrawlImmobilienscout(self),
  File "/usr/src/app/flathunter/crawl_immobilienscout.py", line 38, in __init__
    self.driver = self.configure_driver(driver_arguments)
  File "/usr/src/app/flathunter/abstract_crawler.py", line 61, in configure_driver
    options=chrome_options
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 72, in __init__
    service_log_path, service, keep_alive)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/chromium/webdriver.py", line 97, in __init__
    options=options)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 277, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 370, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 435, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x55b801568403 <unknown>
#1 0x55b80136e778 <unknown>
#2 0x55b801392fa9 <unknown>
#3 0x55b80138e12b <unknown>
#4 0x55b8013c983a <unknown>
#5 0x55b8013c38f3 <unknown>
#6 0x55b8013990d8 <unknown>
#7 0x55b80139a205 <unknown>
#8 0x55b8015afe3d <unknown>
#9 0x55b8015b2db6 <unknown>
#10 0x55b80159913e <unknown>
#11 0x55b8015b39b5 <unknown>
#12 0x55b80158d970 <unknown>
#13 0x55b8015d0228 <unknown>
#14 0x55b8015d03bf <unknown>
#15 0x55b8015eaabe <unknown>
#16 0x7ff24ea0dea7 <unknown>```

[alexanderroidl]

@Leolele99 and @abuchmueller, could you try out my solution suggested in this issue and tell me whether it worked? Thank you!

[codders]

@Leolele99 @abuchmueller did you have any additional feedback here? Otherwise I would close this.

[abuchmueller]

@Leolele99 @abuchmueller did you have any additional feedback here? Otherwise I would close this.

I think this is fixed, docker runs stable for quite some time now with the right driver args.