Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] dash_duo does not set remote thus fails when get_webdriver is called #829

Closed
Oronis opened this issue Jul 19, 2019 · 8 comments · Fixed by #894
Closed

[BUG] dash_duo does not set remote thus fails when get_webdriver is called #829

Oronis opened this issue Jul 19, 2019 · 8 comments · Fixed by #894
Assignees
@byronz
Milestone
Dash v1.3.0

Comments

@Oronis
Copy link

Oronis commented Jul 19, 2019

Describe your context

dash                                           1.0.2
dash-core-components                           1.0.0
dash-html-components                           1.0.0
dash-renderer                                  1.0.0
dash-table                                     4.0.2
  • OS: Windows 10
  • Browser Chrome
  • Version 75.0.3770.142

Describe the bug

Trying to use new dash.testing. Running the example code copied from the documentation (https://dash.plot.ly/testing) in a headless mode (with a Chrome webdriver running).

An error gets thrown in

dash\testing\browser.py:47: in __init__
    self._driver = until(lambda: self.get_webdriver(remote), timeout=1)

It appears that the variable remote is never set in the call stack of dash_duo and thus the call to get_webdriver always fails.

Expected behavior

It should work. I don't know what else to say beyond the basic example doesn't appear to work. If there is some setup requirement that is missing, then a clarification on the documentation needs to be added, otherwise at some point in the setup of dash_duo the value of remote needs to be set such that the get_webdriver call doesn't fail and it correctly fetches the webdriver when initialising the browser.
@byronz
Copy link
Contributor

byronz commented Jul 19, 2019 •
edited
Loading

@Oronis thank you for reporting the dash.testing issue in details. It looks like an extra option --disable-gpu is needed on Windows OS.

can you give it a quick try before I can find a windows machine for verification?

  1. upgrade to latest chrome and webdriver

  2. configure the option as follow if Refactor #1 doesn't work

define this in your top conftest.py

from selenium.webdriver.chrome.options import Options

def pytest_setup_options():
    """called before webdriver is initialized"""
    options = Options()
    options.add_argument('--disable-gpu')
    return options

I will also update the document once the fix is confirmed.

@Oronis
Copy link
Author

Oronis commented Jul 22, 2019

@byronz I'm afraid I'm on a managed PC and haven't been able to force an update to version 76. I've done everything else and I'm afraid this doesn't seem to affect the issue. It still fails when calling get_webdriver as the value of remote is never set in the code.

As a quick check, how or when is this value supposed to be set? I've tried to sense check the code and, as far as I can see from debugging it, the value is always None when called from the dash_duo initialisation and thus will always cause the call to until to fail.

@byronz
Copy link
Contributor

byronz commented Jul 22, 2019

@Oronis the remote is like a reserved parameter in case we might need selenium grid, it's normal that it's always None in our current usage. it calls (in this case) the private getattr(self, "_get_{}".format(self._browser))() to initialize the corresponding browser per browser name. so the actual call for Chrome case is _get_chrome.

    def get_webdriver(self, remote):
        try:
            return (
                getattr(self, "_get_{}".format(self._browser))()
                if remote is None
                else webdriver.Remote(
                    command_executor=remote,
                    desired_capabilities=getattr(
                        DesiredCapabilities, self._browser.upper()
                    ),
                )
            )
        except WebDriverException:
            logger.exception("<<<Webdriver not initialized correctly>>>")
            return None

@byronz
Copy link
Contributor

byronz commented Jul 22, 2019

@Oronis
can you paste the structure of your folder? the location of the conftest and tests
can you increase the log level by pytest --headless --log-cli-level DEBUG and past the log here?

thanks

@byronz
Copy link
Contributor

byronz commented Jul 24, 2019

@Oronis I cannot reproduce the problem using

Windows 10 Home
Chrome version 75.0.3770.100 (Official Build) (64-bit)

pip list | grep dash
dash                          1.0.2
dash-core-components          1.0.0
dash-cytoscape                0.1.1
dash-html-components          1.0.0
dash-renderer                 1.0.0
dash-table                    4.0.2
WARNING: You are using pip version 19.1.1, however version 19.2.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.

python -m pytest --log-cli-level DEBUG --headless test_x.py
Test session starts (platform: win32, Python 2.7.15, pytest 4.6.4, pytest-sugar 0.9.2)
cachedir: .pytest_cache
rootdir: c:\Users\xingh\code\dash, inifile: pytest.ini
plugins: dash-1.0.2, mock-1.10.4, sugar-0.9.2
collecting ...
------------------------------- live log setup --------------------------------
11:47:37 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session {"capabilities": {"alwaysMatch": {"goog:chromeOptions": {"prefs": {"safebrowsing.enabled": false, "download.default_directory": "c:\\users\\xingh\\appdata\\local\\temp\\pytest-of-xingh\\pytest-3\\test_bsly001_falsy_child0\\download", "download.directory_upgrade": true, "safebrowsing.disable_download_protection": true, "download.prompt_for_download": false}, "extensions": [], "args": ["--headless", "--disable-gpu"]}, "browserName": "chrome", "platformName": "any"}, "firstMatch": [{}]}, "desiredCapabilities": {"goog:chromeOptions": {"prefs": {"safebrowsing.enabled": false, "download.default_directory": "c:\\users\\xingh\\appdata\\local\\temp\\pytest-of-xingh\\pytest-3\\test_bsly001_falsy_child0\\download", "download.directory_upgrade": true, "safebrowsing.disable_download_protection": true, "download.prompt_for_download": false}, "extensions": [], "args": ["--headless", "--disable-gpu"]}, "platform": "ANY", "browserName": "chrome", "version": "", "loggingPrefs": {"browser": "SEVERE"}}}
11:47:37 | DEBUG | urllib3.connectionpool:206 | Starting new HTTP connection (1): 127.0.0.1:52395

DevTools listening on ws://127.0.0.1:52401/devtools/browser/33c2c4e8-d246-4020-955a-d712b2eb7164
11:47:39 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session HTTP/1.1" 200 678
11:47:39 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:39 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/chromium/send_command {"cmd": "Page.setDownloadBehavior", "params": {"downloadPath": "c:\\users\\xingh\\appdata\\local\\temp\\pytest-of-xingh\\pytest-3\\test_bsly001_falsy_child0\\download", "behavior": "allow"}}
11:47:39 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/chromium/send_command HTTP/1.1" 200 14
11:47:39 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:39 | DEBUG | dash.testing.browser:321 | enabled headless download returns {u'value': None}
11:47:39 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/window/rect {"y": 0, "x": 0, "width": null, "height": null}
11:47:40 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/window/rect HTTP/1.1" 200 48
11:47:40 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:40 | DEBUG | dash.testing.wait:23 | start wait.until with method, timeout, poll => <function <lambda> at 0x0000000018793748> 1 0.1
11:47:40 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/timeouts {"implicit": 2000}
11:47:40 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/timeouts HTTP/1.1" 200 14
11:47:40 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:40 | DEBUG | dash.testing.browser:65 | initialize browser with arguments
11:47:40 | DEBUG | dash.testing.browser:66 |   headless => True
11:47:40 | DEBUG | dash.testing.browser:67 |   download_path => c:\users\xingh\appdata\local\temp\pytest-of-xingh\pytest-3\test_bsly001_falsy_child0\download
-------------------------------- live log call --------------------------------
11:47:40 | DEBUG | urllib3.connectionpool:206 | Starting new HTTP connection (1): localhost:8050
11:47:40 | INFO | werkzeug:122 |  * Running on http://127.0.0.1:8050/ (Press CTRL+C to quit)
11:47:41 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:41] "GET / HTTP/1.1" 200 -
11:47:41 | DEBUG | urllib3.connectionpool:396 | http://localhost:8050 "GET / HTTP/1.1" 200 515
11:47:41 | DEBUG | dash.testing.wait:23 | start wait.until with method, timeout, poll => <function accessible at 0x0000000018793D68> 1 0.1
11:47:41 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/url {"url": "http://localhost:8050"}
11:47:41 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:41] "GET / HTTP/1.1" 200 -
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-component-suites/dash_renderer/[email protected]?v=1.0.0&m=1563993858 HTTP/1.1" 200 -
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-component-suites/dash_renderer/[email protected]?v=1.0.0&m=1563993858 HTTP/1.1" 200 -
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-component-suites/dash_renderer/[email protected]?v=1.0.0&m=1563993858 HTTP/1.1" 200 -
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-component-suites/dash_html_components/dash_html_components.min.js?v=1.0.0&m=1563993862 HTTP/1.1" 200 -
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-component-suites/dash_renderer/dash_renderer.min.js?v=1.0.0&m=1563993858 HTTP/1.1" 200 -
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-layout HTTP/1.1" 200 -
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/url HTTP/1.1" 200 14
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | dash.testing.browser:137 | method, timeout, poll => <class 'selenium.webdriver.support.expected_conditions.presence_of_element_located'> 10 0.5
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/element {"using": "css selector", "value": "#react-entry-point"}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/element HTTP/1.1" 200 88
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | dash.testing.browser:137 | method, timeout, poll => <class 'dash.testing.wait.text_to_equal'> 4 0.5
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/element {"using": "css selector", "value": "#nully-wrapper"}
11:47:42 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:42] "GET /_dash-dependencies HTTP/1.1" 200 -
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/element HTTP/1.1" 200 88
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | GET http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/element/2b90207c-f7ac-4bd2-8734-d4fac4facfc6/text {"id": "2b90207c-f7ac-4bd2-8734-d4fac4facfc6"}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "GET /session/a1880316ff15cef70ba554f94f88d50c/element/2b90207c-f7ac-4bd2-8734-d4fac4facfc6/text HTTP/1.1" 200 13
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | dash.testing.wait:85 | text to equal {0} => expected 0
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | GET http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/element/2b90207c-f7ac-4bd2-8734-d4fac4facfc6/text {"id": "2b90207c-f7ac-4bd2-8734-d4fac4facfc6"}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "GET /session/a1880316ff15cef70ba554f94f88d50c/element/2b90207c-f7ac-4bd2-8734-d4fac4facfc6/text HTTP/1.1" 200 13
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/element {"using": "css selector", "value": "#nully-wrapper"}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/element HTTP/1.1" 200 88
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | GET http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/element/2b90207c-f7ac-4bd2-8734-d4fac4facfc6/text {"id": "2b90207c-f7ac-4bd2-8734-d4fac4facfc6"}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "GET /session/a1880316ff15cef70ba554f94f88d50c/element/2b90207c-f7ac-4bd2-8734-d4fac4facfc6/text HTTP/1.1" 200 13
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | POST http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c/log {"type": "browser"}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "POST /session/a1880316ff15cef70ba554f94f88d50c/log HTTP/1.1" 200 12
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:42 | INFO | dash.testing.browser:88 | taking snapshot name => bsly001-layout - py2.7


------------------------------ live log teardown ------------------------------
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:388 | DELETE http://127.0.0.1:52395/session/a1880316ff15cef70ba554f94f88d50c {}
11:47:42 | DEBUG | urllib3.connectionpool:396 | http://127.0.0.1:52395 "DELETE /session/a1880316ff15cef70ba554f94f88d50c HTTP/1.1" 200 14
11:47:42 | DEBUG | selenium.webdriver.remote.remote_connection:440 | Finished Request
11:47:44 | INFO | dash.testing.application_runners:80 | killing the app runner
11:47:44 | DEBUG | urllib3.connectionpool:206 | Starting new HTTP connection (1): localhost:8050
11:47:45 | INFO | werkzeug:122 | 127.0.0.1 - - [24/Jul/2019 11:47:45] "GET /_stop-267a1537e9a4404ba2dbbc5c9840cc11 HTTP/1.1" 200 -
11:47:45 | DEBUG | urllib3.connectionpool:396 | http://localhost:8050 "GET /_stop-267a1537e9a4404ba2dbbc5c9840cc11 HTTP/1.1" 200 29
11:47:45 | DEBUG | dash.testing.wait:44 | start wait.until_not method, timeout, poll => <bound method Thread.isAlive of <Thread(Thread-1, started daemon 7788)>> 3 0.1
11:47:45 | DEBUG | dash.testing.wait:52 | poll => 1563994066.0
11:47:46 | DEBUG | dash.testing.wait:52 | poll => 1563994066.11
11:47:46 | DEBUG | dash.testing.wait:52 | poll => 1563994066.22
11:47:46 | DEBUG | dash.testing.wait:52 | poll => 1563994066.33
11:47:46 | DEBUG | dash.testing.wait:52 | poll => 1563994066.43
 tests\test_x.py::test_bsly001_falsy_child \r \x1b[36mtests\\test_x.py\x1b[0m::test_bsly001_falsy_child\x1b[0m \x1b[32m\u2713\x1b[0m                     \x1b[32m100% \x1b[0m\x1b[40m\x1b[32m\u2588\x1b[0m\x1b[40m\x1b[32m\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\x1b[0m

@Oronis
Copy link
Author

Oronis commented Jul 24, 2019 via email

@byronz
Copy link
Contributor

byronz commented Jul 24, 2019

no worries, get well soon.

@cormacc
Copy link

cormacc commented Aug 7, 2019

I've run into what appears to be the same issue -- not on windows, but trying to setup a CI build using gitlab-ci / docker.
The same sequence of build steps runs fine on my own linux installation and on another test machine running OSX.
I've tried using a variety of base docker images (python:3.7, some variants with chromedriver installed), all to no effect.

See below -- may be helpful in replicating the issue without a windows box?
Two variants included. The simplest installs chromium and chromedriver. The second (commented out) uses pytest-selenium and a separate selenium/chromedriver docker image as a service. Both exhibit the same issue described by @Oronis

# Official framework image. Look for the different tagged releases at:
# https://hub.docker.com/r/library/python
image: python:3.7
# Alternate image with added google-chrome/chromedriver
# image: joyzoursky/python-chromedriver:3.7

#See useful docs here:
#https://devconfusa18.readthedocs.io/en/latest/gitlab.html

# variables:
#   PERCY_ENABLE: 0

# This folder is cached between builds
# http://docs.gitlab.com/ce/ci/yaml/README.html#cache
cache:
  paths:
  - ~/.cache/pip/


stages:
  - test

Unit tests:
  stage: test
  before_script:
    - pip install pipenv
  script:
    - cp Pipfile.lock.linux Pipfile.lock
    - pipenv install --dev
    - pipenv run coverage run -m pytest -vvv -m "not webtest"

# This installs chromedriver locally....
UI tests:
  stage: test
  before_script:
    #Comment out these next two lines if using a base with embedded chromedriver
    - apt-get -y update
    - apt-get -y install chromium chromium-driver
    - pip install pipenv
  script:
    - cp Pipfile.lock.linux Pipfile.lock
    - pipenv install --dev
    - pipenv run coverage run -m pytest -vvv -m webtest --headless

# This uses a separate docker image for chrome
# UI tests:
#   stage: test
#   services:
#     - name: selenium/standalone-chrome
#       alias: selenium
#   before_script:
#     - pip install pipenv
#   script:
#     - cp Pipfile.lock.linux Pipfile.lock
#     - pipenv install --dev
#     - pipenv run coverage run -m pytest -vvv --driver Remote --capability browserName chrome --host selenium --port 4444 -m webtest
#     #N.B. the --driver Remote --capability... bit requires pytest-selenium to be installed

@cormacc cormacc mentioned this issue Aug 9, 2019
Open
@byronz byronz added this to the Dash v1.3.0 milestone Aug 9, 2019
@byronz byronz self-assigned this Aug 26, 2019
@byronz byronz mentioned this issue Aug 29, 2019
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@byronz byronz

Labels
None yet
Projects
None yet
Milestone
Dash v1.3.0
Development

Successfully merging a pull request may close this issue.

Remote grid plotly/dash
4 participants
@byronz @cormacc @Oronis @Marc-Andre-Rivet