test_join test fails #121

Valodim · 2020-01-03T14:26:15Z

Hey folks. Thanks for your work on this project!

I wanted to package furl for NixOS, and the build throws this error for me:

.........................F..................................................
======================================================================
FAIL: test_join (test_furl.TestFurl)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/furl-2.1.0/tests/test_furl.py", line 2042, in test_join
    assert f.url == 'wss://slrp.com/foo:1'
AssertionError

----------------------------------------------------------------------
Ran 76 tests in 9.423s

It seems only this single test fails, all others are fine? That leads me to believe it's not a peculiarity of my packaging that's responsible.

Anyways, for context here's the build nix expression I used:

{ lib, python3, radicale2 }:
with python3.pkgs;

buildPythonPackage rec {
    pname = "furl";
    version = "2.1.0";

    src = fetchPypi {
      inherit pname version;
      sha256 = "08dnw3bs1mk0f1ccn466a5a7fi1ivwrp0jspav9arqpf3wd27q60";
    };

    propagatedBuildInputs = [
      orderedmultidict six
    ];

    checkInputs = [
      flake8
    ];
}

The text was updated successfully, but these errors were encountered:

gruns · 2020-01-04T00:39:06Z

Strange. I can't reproduce this error.

$ detox
  ...
  codestyle: commands succeeded
  py27: commands succeeded
  py34: commands succeeded
  py35: commands succeeded
  py36: commands succeeded
  py37: commands succeeded
  pypy: commands succeeded
  pypy3: commands succeeded
  congratulations :)

All tests pass on Travis, too: https://travis-ci.org/gruns/furl.

Does this error persist if you check out furl into a clean directory and run tox?

Valodim · 2020-01-06T12:52:36Z

clean checkout running tox. bunch of other errors about missing interpreters, but the test still failed. I inserted a line print(f.url) right before the failing test, here's the interesting bit:

py37 installed: atomicwrites==1.3.0,attrs==19.3.0,entrypoints==0.3,filelock==3.0.12,flake8==3.7.9,furl==2.1.0,importlib-metadata==1.3.0,mccabe==0.6.1,more-itertools==8.0.2,nose==1.3.7,orderedmultidict==1.0.1,packaging==19.2,pluggy==0.13.1,py==1.8.1,pycodestyle==2.5.0,pyflakes==2.1.1,pyparsing==2.4.6,pytest==5.3.2,six==1.13.0,toml==0.10.0,tox==3.14.3,virtualenv==16.7.9,wcwidth==0.1.7,zipp==0.6.0
py37 run-test-pre: PYTHONHASHSEED='1216359375'
py37 run-test: commands[0] | nosetests --exe
.........................F...................................................
======================================================================
FAIL: test_join (test_furl.TestFurl)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/valodim/nixpkgs/furl/tests/test_furl.py", line 2043, in test_join
    assert f.url == 'wss://slrp.com/foo:1'
AssertionError: 
-------------------- >> begin captured stdout << ---------------------
foo:1

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 77 tests in 9.443s

that's for python version 3.7.6. hope that helps?

felixonmars · 2020-11-12T09:39:53Z

I got exactly the same issue when packaging for Arch with Python 3.9.0. It was not present in 3.8.6 though.

foutrelis · 2020-11-12T10:00:10Z

The test failure with Python 3.9 might be related to urlparse changes like in mozilla/bleach#565.

dvzrv · 2020-11-19T18:37:29Z

@gruns what would be your plans/ suggestion to fix this?

I will disable the test for these edge-case urls for now to be able to do further rebuilds against python3.9 on Arch Linux. However, it would be great to get this in working condition again for python >= 3.8.0

inakimalerba · 2020-12-06T17:25:18Z

Same problem while packaging for Debian on Python 3.9.

======================================================================
FAIL: test_join (tests.test_furl.TestFurl)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autopkgtest-lxc.y0kq_g1k/downtmp/build.NyP/src/tests/test_furl.py", line 2042, in test_join
    assert f.url == 'wss://slrp.com/foo:1'
AssertionError

gruns · 2020-12-11T13:05:34Z

strange. tests pass on my local dev box running Ubuntu with Python 3.8.0 and Python 3.9.1

$ python3.8 --version
Python 3.8.0
$ python3.9 --version
Python 3.9.1

  codestyle: commands succeeded
  py27: commands succeeded
  py34: commands succeeded
  py35: commands succeeded
  py36: commands succeeded
  py37: commands succeeded
  py38: commands succeeded
  py39: commands succeeded
  pypy: commands succeeded
  pypy3: commands succeeded
  congratulations :)

some manual digging required. any further information you guys to reproduce here would be super helpful

dvzrv · 2020-12-11T13:11:58Z

@gruns in your example you are only running flake8. The error is triggered by the nosetests call.

bnavigator · 2021-03-29T21:35:47Z

abuild@skylab:~/rpmbuild/BUILD/furl-2.1.0> pytest-3.9 --ignore=_build.python36 --ignore=_build.python39 --ignore=_build.python38 -v --pdb
====================================================== test session starts =======================================================
platform linux -- Python 3.9.2, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3.9
cachedir: .pytest_cache
rootdir: /home/abuild/rpmbuild/BUILD/furl-2.1.0
collected 76 items         

...

tests/test_furl.py::TestFurl::test_join FAILED                                                                             [ 73%]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

self = <test_furl.TestFurl testMethod=test_join>

    def test_join(self):

...
 
        # In edge cases (e.g. URLs without an authority/netloc), behave
        # identically to urllib.parse.urljoin().
        f = furl.furl('wss://slrp.com/').join('foo:1')
>       assert f.url == 'wss://slrp.com/foo:1'
E       AssertionError: assert 'foo:1' == 'wss://slrp.com/foo:1'
E         - wss://slrp.com/foo:1
E         + foo:1

tests/test_furl.py:2042: AssertionError
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /home/abuild/rpmbuild/BUILD/furl-2.1.0/tests/test_furl.py(2042)test_join()
-> assert f.url == 'wss://slrp.com/foo:1'
(Pdb) myurl = furl.furl('wss://slrp.com/')
(Pdb) debug myurl.join('foo:1')
ENTERING RECURSIVE DEBUGGER
> <string>(1)<module>()
((Pdb)) s
--Call--
> /home/abuild/rpmbuild/BUILDROOT/python-furl-2.1.0-0.x86_64/usr/lib/python3.9/site-packages/furl/furl.py(1808)join()
-> def join(self, *urls):
((Pdb)) n
> /home/abuild/rpmbuild/BUILDROOT/python-furl-2.1.0-0.x86_64/usr/lib/python3.9/site-packages/furl/furl.py(1809)join()

...

> /home/abuild/rpmbuild/BUILDROOT/python-furl-2.1.0-0.x86_64/usr/lib/python3.9/site-packages/furl/furl.py(321)urljoin()
-> root = set_scheme(base, 'http')
((Pdb)) n
> /home/abuild/rpmbuild/BUILDROOT/python-furl-2.1.0-0.x86_64/usr/lib/python3.9/site-packages/furl/furl.py(325)urljoin()
-> joined = urllib.parse.urljoin(root, url)
((Pdb)) p root
'http://slrp.com/'
((Pdb)) p url
'foo:1'
((Pdb)) n
> /home/abuild/rpmbuild/BUILDROOT/python-furl-2.1.0-0.x86_64/usr/lib/python3.9/site-packages/furl/furl.py(327)urljoin()
-> new_scheme = url_scheme if url_scheme is not None else base_scheme
((Pdb)) p joined
'foo:1'

"behave identically to urllib.parse.urljoin()". urllib.parse.urljoin() behaves differently in Python 3.9:

abuild@skylab:~/rpmbuild/BUILD/furl-2.1.0> python3.9 
Python 3.9.2 (default, Feb 19 2021, 23:09:16) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.parse
>>> urllib.__path__
['/usr/lib64/python3.9/urllib']
>>> urllib.parse.urljoin('http://slrp.com', 'foo:1')   
'foo:1'
>>> 
abuild@skylab:~/rpmbuild/BUILD/furl-2.1.0> python3.8 
Python 3.8.8 (default, Feb 19 2021, 16:53:21) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.parse
>>> urllib.__path__
['/usr/lib64/python3.8/urllib']
>>> urllib.parse.urljoin('http://slrp.com', 'foo:1')
'http://slrp.com/foo:1'
>>> 
abuild@skylab:~/rpmbuild/BUILD/furl-2.1.0> python3.6
Python 3.6.13 (default, Feb 19 2021, 18:59:35) [GCC] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.parse
>>> urllib.__path__
['/usr/lib64/python3.6/urllib']
>>> urllib.parse.urljoin('http://slrp.com', 'foo:1')
'http://slrp.com/foo:1'
>>>

Valodim · 2021-03-29T21:39:35Z

@bnavigator that specific instance of this issue was also addressed in #135, there's also a patch but it hasn't been merged yet

bnavigator · 2021-03-29T21:40:36Z

@Valodim: No it hasn't. The patch in #135 is about the other failures mentioned there. The above log is with the patch applied.

Valodim · 2021-03-29T21:44:24Z

Oh. Dang :)

Those tests are causing headaches for packaging downstreams. :( we ended up disabling this particular test for NixOS for now, and applied the patch from #135.

\\ edit

Whoops, that didn't come out right - of course it makes sense to have tests that make sure the behavior matches python's builtin behavior where documented as such. Not sure if there is a good solution that doesn't involve manually adapting to new python versions as they are released.

bnavigator · 2021-03-29T21:50:31Z

@Valodim, totally agree. Stumbled over this because it is one of the blockers before we can introduce Python 3.9 packages to openSUSE Tumbleweed. furl is in the "minimal X" ring and without a passing testsuite there is no chance of having a Python 3.9 ecosystem there.

gruns · 2021-08-17T19:19:21Z

@Valodim @bnavigator this issue should be resolved in furl v2.1.2.

i can no longer produce the test error in py39

$ tox -p
GLOB sdist-make: /mnt/fett/code/furl/setup.py
✔ OK codestyle in 6.383 seconds
✔ OK pypy in 14.491 seconds
✔ OK py39 in 17.957 seconds
✔ OK py38 in 18.093 seconds
✔ OK pypy3 in 19.528 seconds
✔ OK py36 in 20.002 seconds
✔ OK py37 in 20.303 seconds
✔ OK py27 in 20.545 seconds
_____________________________________ summary ______________________________________
  codestyle: commands succeeded
  py27: commands succeeded
  py36: commands succeeded
  py37: commands succeeded
  py38: commands succeeded
  py39: commands succeeded
  pypy: commands succeeded
  pypy3: commands succeeded
  congratulations :)

can you produce it? if not, ill close this issue! 🙌

felixonmars · 2021-08-17T19:31:00Z

I can confirm that on Arch / Python 3.9.6 the failures are present in v2.1.1 but fixed in v2.1.2.

bnavigator · 2021-08-17T20:59:39Z

Yes it has been fixed by #139 (ffe45d6) a long time ago.

gruns · 2021-08-18T19:01:34Z

@bnavigator @felixonmars thank you for confirming

thank you to everyone in this issue for your help debugging! 🙌

Valodim mentioned this issue Jan 3, 2020

package etesync-dav NixOS/nixpkgs#76886

Merged

10 tasks

shadchin mentioned this issue Feb 21, 2021

New bugs with Python 3.9.2+ #135

Closed

bnavigator mentioned this issue Mar 29, 2021

urllib changed behavior in Py 3.9 #139

Closed

gruns closed this as completed Aug 18, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test_join test fails #121

test_join test fails #121

Valodim commented Jan 3, 2020 •

edited

Loading

gruns commented Jan 4, 2020 •

edited

Loading

Valodim commented Jan 6, 2020 •

edited

Loading

felixonmars commented Nov 12, 2020

foutrelis commented Nov 12, 2020

dvzrv commented Nov 19, 2020

inakimalerba commented Dec 6, 2020

gruns commented Dec 11, 2020 •

edited

Loading

dvzrv commented Dec 11, 2020

bnavigator commented Mar 29, 2021 •

edited

Loading

Valodim commented Mar 29, 2021

bnavigator commented Mar 29, 2021

Valodim commented Mar 29, 2021 •

edited

Loading

bnavigator commented Mar 29, 2021

gruns commented Aug 17, 2021

felixonmars commented Aug 17, 2021

bnavigator commented Aug 17, 2021

gruns commented Aug 18, 2021

test_join test fails #121

test_join test fails #121

Comments

Valodim commented Jan 3, 2020 • edited Loading

gruns commented Jan 4, 2020 • edited Loading

Valodim commented Jan 6, 2020 • edited Loading

felixonmars commented Nov 12, 2020

foutrelis commented Nov 12, 2020

dvzrv commented Nov 19, 2020

inakimalerba commented Dec 6, 2020

gruns commented Dec 11, 2020 • edited Loading

dvzrv commented Dec 11, 2020

bnavigator commented Mar 29, 2021 • edited Loading

Valodim commented Mar 29, 2021

bnavigator commented Mar 29, 2021

Valodim commented Mar 29, 2021 • edited Loading

bnavigator commented Mar 29, 2021

gruns commented Aug 17, 2021

felixonmars commented Aug 17, 2021

bnavigator commented Aug 17, 2021

gruns commented Aug 18, 2021

Valodim commented Jan 3, 2020 •

edited

Loading

gruns commented Jan 4, 2020 •

edited

Loading

Valodim commented Jan 6, 2020 •

edited

Loading

gruns commented Dec 11, 2020 •

edited

Loading

bnavigator commented Mar 29, 2021 •

edited

Loading

Valodim commented Mar 29, 2021 •

edited

Loading