Unit test maintenance re legacy disparity #2243

phillxnet · 2021-01-04T17:14:46Z

Prior to embarking on our Rockstor 4 release candidates the existing unit tests on our prior CentOS base were modified to have zero failures/errors in the following pull request:
"remark out all currently failing unit tests. Fixes #2137" #2153
In the mean time we have added additional unit tests and still have some existing failures in prior tests on the non legacy 'Built on openSUSE' endeavour. This issue is to address this disparity by returning our unit test status to that of our prior OS base with the intention of establishing this state prior to our final Stable 4 release.

phillxnet · 2021-01-04T17:15:18Z

I am currently working on this issue.

phillxnet · 2021-01-04T17:30:23Z

Current state as of 4.0.5-0 tag (Release Candidate 6)

Ran 212 tests in 34.752s
FAILED (failures=6, errors=1)

phillxnet · 2021-01-04T20:11:49Z

We have 2 failures in the user section which seem to relate only to differing default users:
./bin/test -v 2 -p test_user*

Ran 8 tests in 1.578s
FAILED (failures=2)

details:

======================================================================
FAIL: test_delete_requests (storageadmin.tests.test_user.UserTests)

Traceback (most recent call last):
File "/opt/rockstor-dev/src/rockstor/storageadmin/tests/test_user.py", line 399, in test_delete_requests
status.HTTP_200_OK, msg=response.data)
AssertionError: ['User (games) does not exist.', 'None\n']

======================================================================
FAIL: test_post_requests (storageadmin.tests.test_user.UserTests)

Traceback (most recent call last):
File "/opt/rockstor-dev/src/rockstor/storageadmin/tests/test_user.py", line 175, in test_post_requests
msg=response.data)
AssertionError: {'username': u'newUser', 'public_key': None, 'shell': u'/bin/bash', 'group': 2, 'pincard_allowed': 'no', 'admin': True, 'managed_user': True, 'homedir': u'/home/newUser', 'email': None, 'groupname': u'admin', 'gid': 5, 'user': 8, 'uid': 3, 'smb_shares': [], u'id': 1, 'has_pincard': False}

phillxnet · 2021-01-04T20:21:01Z

2 failed in commands likely due to missing files associated with yum auto update:
./bin/test -v 2 -p test_commands*

Ran 17 tests in 2.976s
FAILED (failures=2)

details

======================================================================
FAIL: test_disable_auto_update_command (storageadmin.tests.test_commands.CommandTests)

Traceback (most recent call last):
File "/opt/rockstor-dev/src/rockstor/storageadmin/tests/test_commands.py", line 152, in test_disable_auto_update_command
status.HTTP_200_OK, msg=response.data)
AssertionError: ["Failed to disable auto update due to this exception: ([Errno 2] No such file or directory: '/etc/yum/yum-cron.conf').", 'Traceback (most recent call last):\n File "/opt/rockstor-dev/src/rockstor/storageadmin/views/command.py", line 392, in post\n auto_update(enable=False)\n File "/opt/rockstor-dev/src/rockstor/system/pkg_mgmt.py", line 62, in auto_update\n with open(YCFILE) as ifo, open(npath, "w") as tfo:\nIOError: [Errno 2] No such file or directory: '/etc/yum/yum-cron.conf'\n']

======================================================================
FAIL: test_enable_auto_update_command (storageadmin.tests.test_commands.CommandTests)

Traceback (most recent call last):
File "/opt/rockstor-dev/src/rockstor/storageadmin/tests/test_commands.py", line 144, in test_enable_auto_update_command
status.HTTP_200_OK, msg=response.data)
AssertionError: ["Failed to enable auto update due to this exception: ([Errno 2] No such file or directory: '/etc/yum/yum-cron.conf').", 'Traceback (most recent call last):\n File "/opt/rockstor-dev/src/rockstor/storageadmin/views/command.py", line 382, in post\n auto_update(enable=True)\n File "/opt/rockstor-dev/src/rockstor/system/pkg_mgmt.py", line 62, in auto_update\n with open(YCFILE) as ifo, open(npath, "w") as tfo:\nIOError: [Errno 2] No such file or directory: '/etc/yum/yum-cron.conf'\n']

phillxnet · 2021-01-04T20:25:51Z

1 failing in user group:
./bin/test -v 2 -p test_group*

Ran 3 tests in 0.687s
FAILED (failures=1)

details

======================================================================
FAIL: test_post_requests (storageadmin.tests.test_group.GroupTests)

Traceback (most recent call last):
File "/opt/rockstor-dev/src/rockstor/storageadmin/tests/test_group.py", line 99, in test_post_requests
msg=response.data)
AssertionError: {'admin': True, 'groupname': u'ngroup2', 'gid': 1, u'id': 1}

Don't see how this passed previously but haven't really looked yet.

phillxnet · 2021-01-04T20:31:18Z

The single error looks to be from changes we've made in Samba code maybe:
./bin/test -v 2 -p test_samba*

Ran 13 tests in 1.481s
FAILED (errors=1)

details:

======================================================================
ERROR: test_post_requests_2 (storageadmin.tests.test_samba.SambaTests)

Traceback (most recent call last):
File "/opt/rockstor-dev/eggs/mock-1.0.1-py2.7.egg/mock.py", line 1201, in patched
return func(*args, **keywargs)
File "/opt/rockstor-dev/src/rockstor/storageadmin/tests/test_samba.py", line 421, in test_post_requests_2
response = self.client.post(self.BASE_URL, data=data)
File "/opt/rockstor-dev/eggs/djangorestframework-3.1.1-py2.7.egg/rest_framework/test.py", line 168, in post
path, data=data, format=format, content_type=content_type, **extra)
File "/opt/rockstor-dev/eggs/djangorestframework-3.1.1-py2.7.egg/rest_framework/test.py", line 90, in post
return self.generic('POST', path, data, content_type, **extra)
File "/opt/rockstor-dev/eggs/djangorestframework-3.1.1-py2.7.egg/rest_framework/compat.py", line 222, in generic
return self.request(**r)
File "/opt/rockstor-dev/eggs/djangorestframework-3.1.1-py2.7.egg/rest_framework/test.py", line 157, in request
return super(APIClient, self).request(**kwargs)
File "/opt/rockstor-dev/eggs/djangorestframework-3.1.1-py2.7.egg/rest_framework/test.py", line 109, in request
request = super(APIRequestFactory, self).request(**kwargs)
File "/opt/rockstor-dev/eggs/Django-1.8.16-py2.7.egg/django/test/client.py", line 466, in request
six.reraise(*exc_info)
File "/opt/rockstor-dev/eggs/Django-1.8.16-py2.7.egg/django/core/handlers/base.py", line 132, in get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/rockstor-dev/eggs/Django-1.8.16-py2.7.egg/django/views/decorators/csrf.py", line 58, in wrapped_view
return view_func(*args, **kwargs)
File "/opt/rockstor-dev/eggs/Django-1.8.16-py2.7.egg/django/views/generic/base.py", line 71, in view
return self.dispatch(request, *args, **kwargs)
File "/opt/rockstor-dev/eggs/djangorestframework-3.1.1-py2.7.egg/rest_framework/views.py", line 452, in dispatch
response = self.handle_exception(exc)
File "/opt/rockstor-dev/eggs/djangorestframework-3.1.1-py2.7.egg/rest_framework/views.py", line 449, in dispatch
response = handler(request, *args, **kwargs)
File "/opt/rockstor-dev/eggs/Django-1.8.16-py2.7.egg/django/utils/decorators.py", line 145, in inner
return func(*args, **kwargs)
File "/opt/rockstor-dev/src/rockstor/storageadmin/views/samba.py", line 157, in post
return Response(SambaShareSerializer(smb_share).data)
File "/opt/rockstor-dev/eggs/djangorestframework-3.1.1-py2.7.egg/rest_framework/serializers.py", line 466, in data
ret = super(Serializer, self).data
File "/opt/rockstor-dev/eggs/djangorestframework-3.1.1-py2.7.egg/rest_framework/serializers.py", line 213, in data
self._data = self.to_representation(self.instance)
File "/opt/rockstor-dev/eggs/djangorestframework-3.1.1-py2.7.egg/rest_framework/serializers.py", line 435, in to_representation
ret[field.field_name] = field.to_representation(attribute)
File "/opt/rockstor-dev/eggs/djangorestframework-3.1.1-py2.7.egg/rest_framework/serializers.py", line 568, in to_representation
self.child.to_representation(item) for item in iterable
File "/opt/rockstor-dev/eggs/djangorestframework-3.1.1-py2.7.egg/rest_framework/serializers.py", line 426, in to_representation
attribute = field.get_attribute(instance)
File "/opt/rockstor-dev/eggs/djangorestframework-3.1.1-py2.7.egg/rest_framework/fields.py", line 316, in get_attribute
raise type(exc)(msg)
KeyError: u"Got KeyError when attempting to get a value for field groupname on serializer SUserSerializer.\nThe serializer field might be named incorrectly and not match any attribute or key on the User instance.\nOriginal exception text was: 'getgrgid(): gid not found: 1'."

phillxnet · 2021-01-04T20:42:44Z

And the last failure is in a new networking test that is suspected as having incomplete mocking re docker, or a reliance on the test system running docker or the like.
./bin/test -v 2 -p test_system_network*

Ran 6 tests in 0.045s
FAILED (failures=1)

details:

======================================================================
FAIL: test_get_con_config (system.tests.test_system_network.SystemNetworkTests)

Traceback (most recent call last):
File "/opt/rockstor-dev/src/rockstor/system/tests/test_system_network.py", line 690, in test_get_con_config
"expected = {}.\n ".format(returned, expected),
AssertionError: Un-expected get_con_config() result:
returned = {'7ffa9c37-4559-4608-80de-ea77a27876cd': {'bridge': {'aux_address': None, 'subnet': None, 'internal': False, 'host_binding': None, 'docker_name': None, 'icc': False, 'ip_masquerade': False, 'dgateway': None, 'ip_range': None}, 'ctype': 'bridge', 'ipv6_addresses': None, 'ipv4_method': 'manual', 'ipv6_method': None, 'ipv6_dns': None, 'name': 'docker0', 'ipv4_addresses': '172.17.0.1/16', 'ipv6_gw': None, 'ipv4_dns': None, 'state': 'activated', 'ipv6_dns_search': None, 'ipv4_gw': None, 'ipv4_dns_search': None}}.
expected = {'7ffa9c37-4559-4608-80de-ea77a27876cd': {'bridge': {'aux_address': None, 'subnet': u'172.17.0.0/16', 'internal': False, 'host_binding': u'0.0.0.0', 'docker_name': 'docker0', 'icc': u'true', 'ip_masquerade': False, 'dgateway': u'172.17.0.1', 'ip_range': None}, 'ctype': 'bridge', 'ipv6_addresses': None, 'ipv4_method': 'manual', 'ipv6_method': None, 'ipv6_dns': None, 'name': 'docker0', 'ipv4_addresses': '172.17.0.1/16', 'ipv6_gw': None, 'ipv4_dns': None, 'state': 'activated', 'ipv6_dns_search': None, 'ipv4_gw': None, 'ipv4_dns_search': None}}.

phillxnet · 2021-01-05T14:51:54Z

Getting there ...
./bin/test -v 2 -p test_*

Ran 212 tests in 36.043s
FAILED (failures=1, errors=1)

phillxnet · 2021-01-05T18:44:50Z

Only the recent added network failure one to go now:
./bin/test -v 2 -p test_*

Ran 212 tests in 37.620s
FAILED (failures=1)

Details:

FAIL: test_get_con_config (system.tests.test_system_network.SystemNetworkTests)

Traceback (most recent call last):
  File "/opt/rockstor-dev/src/rockstor/system/tests/test_system_network.py", line 690, in test_get_con_config
    "expected = {}.\n ".format(returned, expected),
AssertionError: Un-expected get_con_config() result:
 returned = {'7ffa9c37-4559-4608-80de-ea77a27876cd': {'bridge': {'aux_address': None, 'subnet': None, 'internal': False, 'host_binding': None, 'docker_name': None, 'icc': False, 'ip_masquerade': False, 'dgateway': None, 'ip_range': None}, 'ctype': 'bridge', 'ipv6_addresses': None, 'ipv4_method': 'manual', 'ipv6_method': None, 'ipv6_dns': None, 'name': 'docker0', 'ipv4_addresses': '172.17.0.1/16', 'ipv6_gw': None, 'ipv4_dns': None, 'state': 'activated', 'ipv6_dns_search': None, 'ipv4_gw': None, 'ipv4_dns_search': None}}.
 expected = {'7ffa9c37-4559-4608-80de-ea77a27876cd': {'bridge': {'aux_address': None, 'subnet': u'172.17.0.0/16', 'internal': False, 'host_binding': u'0.0.0.0', 'docker_name': 'docker0', 'icc': u'true', 'ip_masquerade': False, 'dgateway': u'172.17.0.1', 'ip_range': None}, 'ctype': 'bridge', 'ipv6_addresses': None, 'ipv4_method': 'manual', 'ipv6_method': None, 'ipv6_dns': None, 'name': 'docker0', 'ipv4_addresses': '172.17.0.1/16', 'ipv6_gw': None, 'ipv4_dns': None, 'state': 'activated', 'ipv6_dns_search': None, 'ipv4_gw': None, 'ipv4_dns_search': None}}.

Re-establish prior CentOS baseline of all existing tests passing. Summary of changes - Delete of existing, non excluded user 'games' fails as non existent in openSUSE base. Replace with non excluded 'lp' user that does exist.

for nobody (CentOS) is now available. Change to 65534 which is openSUSE uid for user nobody. rockstor#2243

based. Disable/Enable tests failed with "No such file or directory: '/etc/yum/yum-cron.conf'" but are intended as API level tests. Mock low level auto_update() and add TODO to test_pkg_mgmt.py for the appropriate low level test once we migrate this functionality to a zypper equivalent. This way we preserve & test the presence of the existing API. rockstor#2243

actually exist. Substitute with gid of users (100) which does exist in our new openSUSE base. rockstor#2243

user gid existing. This same gid does not exist in the new openSUSE base, resulting in a an exception. Fix by mocking a fake group name return via getgrgid. rockstor#2243

phillxnet · 2021-01-05T19:03:30Z

@FroggyFlox I'll squash these commits just prior to pr submission as all fairly small changes but pushed for now for backup, just in case.

FroggyFlox · 2021-01-06T12:49:44Z

@phillxnet, I believe I've figured out why the test_get_con_config() for the docker0 connection was failing: I forgot to mock the docker_status() to make the test believe the docker daemon was running.
The following works for me after a quick test on a machine where test_get_con_config() was failing as you reported:

    def test_get_con_config(self):
        """
        This tests for correct parsing of nmcli connection config by get_con_config(),
        which should return a dict with detailed config for each network connection detected.
        """
        # Mock and patch docker-specific calls
        self.patch_docker_status = patch("system.network.docker_status")
        self.mock_docker_status = self.patch_docker_status.start()
        self.mock_docker_status.return_value = True
        self.patch_dnets = patch("system.network.dnets")
        self.mock_dnets = self.patch_dnets.start()
        self.patch_dnet_inspect = patch("system.network.dnet_inspect")
        self.mock_dnet_inspect = self.patch_dnet_inspect.start()

Note the first 3 lines patching and mocking docker_status(), referring to the following if condition that was not met before:

rockstor-core/src/rockstor/system/network.py

Line 206 in 1223360

if docker_status():

I now get:

# ./bin/test -v 3 --settings=test-settings -p test_system_network*
test_get_con_config (system.tests.test_system_network.SystemNetworkTests) ... ok
test_get_con_config_con_not_found (system.tests.test_system_network.SystemNetworkTests) ... ok
test_get_con_config_exception (system.tests.test_system_network.SystemNetworkTests) ... ok
test_get_dev_config (system.tests.test_system_network.SystemNetworkTests) ... ok
test_get_dev_config_dev_not_found (system.tests.test_system_network.SystemNetworkTests) ... ok
test_get_dev_config_exception (system.tests.test_system_network.SystemNetworkTests) ... ok

----------------------------------------------------------------------
Ran 6 tests in 0.016s

OK

phillxnet · 2021-01-06T16:13:34Z

@FroggyFlox Nice. Thanks for that; Ill do the same here and get the pr prepared.

@FroggyFlox

Re-establish prior CentOS baseline of all existing tests passing. Summary of changes - Delete of existing, non excluded user 'games' fails as non existent in openSUSE base. Replace with non excluded 'lp' user that does exist. - Creation of new user with existing uid test failed as prior uid of 99 for nobody (CentOS) is now available. Change to 65534 which is openSUSE uid for user nobody. - Auto_update (legacy yum-based) is now defunct: we are now zypper based. Disable/Enable tests failed with "No such file or directory: '/etc/yum/yum-cron.conf'" but are intended as API level tests. Mock low level auto_update() and add TODO to test_pkg_mgmt.py for the appropriate low level test once we migrate this functionality to a zypper equivalent. This way we preserve & test the presence of the existing API. - Group create with pre-existing group id fails as group id used did not actually exist. Substitute with gid of users (100) which does exist in our new openSUSE base. - Prior samba test created a temporary user that relied on a real system user gid existing. This same gid does not exist in the new openSUSE base, resulting in a an exception. Fix by mocking a fake group name return via getgrgid. - Add missing docker_status() mocking that was failing a more recently added low level network test: test_get_con_config(), due to system docker state. Thanks to @FroggyFlox on GitHub for this fix

phillxnet · 2021-01-06T17:00:52Z

Looks like we are there now:
./bin/test -v 2 -p test_*

Ran 212 tests in 39.875s
OK

This is good to see; thanks.

I've removed the remote (GitHub) branch, squashed all commits, and re-pushed the branch ready for pull request preparation.

…_legacy_disparity Unit test maintenance re legacy disparity. Fixes #2243

phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Jan 5, 2021

- Creation of new user with existing uid test failed as prior uid of 99

95dacda

for nobody (CentOS) is now available. Change to 65534 which is openSUSE uid for user nobody. rockstor#2243

phillxnet added a commit to phillxnet/rockstor-core that referenced this issue Jan 5, 2021

- group create with pre-existing group id fails as group id used did not

02fb528

actually exist. Substitute with gid of users (100) which does exist in our new openSUSE base. rockstor#2243

phillxnet mentioned this issue Jan 6, 2021

Unit test maintenance re legacy disparity. Fixes #2243 #2245

Merged

phillxnet closed this as completed in #2245 Jan 7, 2021

phillxnet added a commit that referenced this issue Jan 7, 2021

Merge pull request #2245 from phillxnet/2243_Unit_test_maintenance_re…

052ad79

…_legacy_disparity Unit test maintenance re legacy disparity. Fixes #2243

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unit test maintenance re legacy disparity #2243

Unit test maintenance re legacy disparity #2243

phillxnet commented Jan 4, 2021

phillxnet commented Jan 4, 2021

phillxnet commented Jan 4, 2021

phillxnet commented Jan 4, 2021

======================================================================
FAIL: test_delete_requests (storageadmin.tests.test_user.UserTests)

======================================================================
FAIL: test_post_requests (storageadmin.tests.test_user.UserTests)

phillxnet commented Jan 4, 2021

======================================================================
FAIL: test_disable_auto_update_command (storageadmin.tests.test_commands.CommandTests)

======================================================================
FAIL: test_enable_auto_update_command (storageadmin.tests.test_commands.CommandTests)

phillxnet commented Jan 4, 2021

======================================================================
FAIL: test_post_requests (storageadmin.tests.test_group.GroupTests)

phillxnet commented Jan 4, 2021 •

edited

Loading

======================================================================
ERROR: test_post_requests_2 (storageadmin.tests.test_samba.SambaTests)

phillxnet commented Jan 4, 2021

======================================================================
FAIL: test_get_con_config (system.tests.test_system_network.SystemNetworkTests)

phillxnet commented Jan 5, 2021

phillxnet commented Jan 5, 2021

phillxnet commented Jan 5, 2021

FroggyFlox commented Jan 6, 2021

phillxnet commented Jan 6, 2021

phillxnet commented Jan 6, 2021

Unit test maintenance re legacy disparity #2243

Unit test maintenance re legacy disparity #2243

Comments

phillxnet commented Jan 4, 2021

phillxnet commented Jan 4, 2021

phillxnet commented Jan 4, 2021

phillxnet commented Jan 4, 2021

====================================================================== FAIL: test_delete_requests (storageadmin.tests.test_user.UserTests)

====================================================================== FAIL: test_post_requests (storageadmin.tests.test_user.UserTests)

phillxnet commented Jan 4, 2021

====================================================================== FAIL: test_disable_auto_update_command (storageadmin.tests.test_commands.CommandTests)

====================================================================== FAIL: test_enable_auto_update_command (storageadmin.tests.test_commands.CommandTests)

phillxnet commented Jan 4, 2021

====================================================================== FAIL: test_post_requests (storageadmin.tests.test_group.GroupTests)

phillxnet commented Jan 4, 2021 • edited Loading

====================================================================== ERROR: test_post_requests_2 (storageadmin.tests.test_samba.SambaTests)

phillxnet commented Jan 4, 2021

====================================================================== FAIL: test_get_con_config (system.tests.test_system_network.SystemNetworkTests)

phillxnet commented Jan 5, 2021

phillxnet commented Jan 5, 2021

phillxnet commented Jan 5, 2021

FroggyFlox commented Jan 6, 2021

phillxnet commented Jan 6, 2021

phillxnet commented Jan 6, 2021

======================================================================
FAIL: test_delete_requests (storageadmin.tests.test_user.UserTests)

======================================================================
FAIL: test_post_requests (storageadmin.tests.test_user.UserTests)

======================================================================
FAIL: test_disable_auto_update_command (storageadmin.tests.test_commands.CommandTests)

======================================================================
FAIL: test_enable_auto_update_command (storageadmin.tests.test_commands.CommandTests)

======================================================================
FAIL: test_post_requests (storageadmin.tests.test_group.GroupTests)

phillxnet commented Jan 4, 2021 •

edited

Loading

======================================================================
ERROR: test_post_requests_2 (storageadmin.tests.test_samba.SambaTests)

======================================================================
FAIL: test_get_con_config (system.tests.test_system_network.SystemNetworkTests)