KeyError: 'PYTEST_CURRENT_TEST' on _post_log_batch

[rplevka]

I really don't know what is going on here, i spotted this in a wild, while running py.test with xdist.
according the logs, the [gw5] worker wasn't stuck but successfully executed other tests, however the "crashed" test wasn't properly stopped on RP. Now the whole py.test seems to be hanging.

[gw5] [  7%] FAILED tests/foreman/api/test_contentviewversion.py::ContentViewVersionDeleteTestCase::test_positive_delete_with_puppet_content Traceback (most recent call last):
  File "/home/jenkins/shiningpanda/jobs/9ce8c4b2/virtualenvs/d41d8cd9/lib/python3.6/site-packages/reportportal_client/service_async.py", line 228, in process_item
    self.process_log(**kwargs)
  File "/home/jenkins/shiningpanda/jobs/9ce8c4b2/virtualenvs/d41d8cd9/lib/python3.6/site-packages/reportportal_client/service_async.py", line 212, in process_log
    self._post_log_batch()
  File "/home/jenkins/shiningpanda/jobs/9ce8c4b2/virtualenvs/d41d8cd9/lib/python3.6/site-packages/reportportal_client/service_async.py", line 200, in _post_log_batch
    self.rp_client.log_batch(self.log_batch)
  File "/home/jenkins/shiningpanda/jobs/9ce8c4b2/virtualenvs/d41d8cd9/lib/python3.6/site-packages/reportportal_client/service.py", line 294, in log_batch
    r = self.session.post(url=url, files=files, verify=self.verify_ssl)
  File "/home/jenkins/shiningpanda/jobs/9ce8c4b2/virtualenvs/d41d8cd9/lib/python3.6/site-packages/requests/sessions.py", line 559, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/home/jenkins/shiningpanda/jobs/9ce8c4b2/virtualenvs/d41d8cd9/lib/python3.6/site-packages/requests/sessions.py", line 503, in request
    prep.url, proxies, stream, verify, cert
  File "/home/jenkins/shiningpanda/jobs/9ce8c4b2/virtualenvs/d41d8cd9/lib/python3.6/site-packages/requests/sessions.py", line 676, in merge_environment_settings
    env_proxies = get_environ_proxies(url, no_proxy=no_proxy)
  File "/home/jenkins/shiningpanda/jobs/9ce8c4b2/virtualenvs/d41d8cd9/lib/python3.6/site-packages/requests/utils.py", line 763, in get_environ_proxies
    return getproxies()
  File "/usr/lib64/python3.6/urllib/request.py", line 2480, in getproxies_environment
    for name, value in os.environ.items():
  File "/home/jenkins/shiningpanda/jobs/9ce8c4b2/virtualenvs/d41d8cd9/lib64/python3.6/_collections_abc.py", line 744, in __iter__
    yield (key, self._mapping[key])
  File "/home/jenkins/shiningpanda/jobs/9ce8c4b2/virtualenvs/d41d8cd9/lib64/python3.6/os.py", line 669, in __getitem__
    raise KeyError(key) from None
KeyError: 'PYTEST_CURRENT_TEST'

this also happened on some other workers during the same run later - same symptoms.

[rplevka]

ok, my crazy theory is the following:

In [17]: environ['FOO'] = 'bar'

In [18]: for n, v in environ.items():
    ...:     if environ.get('FOO'):
    ...:         del environ['FOO']
    ...:         
    ...:     
    ...:     
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-18-f05ba9429fa2> in <module>()
----> 1 for n, v in environ.items():
      2     if environ.get('FOO'):
      3         del environ['FOO']
      4 
      5 

/usr/lib64/python3.6/_collections_abc.py in __iter__(self)
    742     def __iter__(self):
    743         for key in self._mapping:
--> 744             yield (key, self._mapping[key])
    745 
    746 ItemsView.register(dict_items)

/usr/lib64/python3.6/os.py in __getitem__(self, key)
    667         except KeyError:
    668             # raise KeyError with the original key value
--> 669             raise KeyError(key) from None
    670         return self.decodevalue(value)
    671 

KeyError: 'FOO'

.. so i think the requests lib calls urllib's getproxies_environment method (before it sends out the request to RP) which checks the ENV for a presence of the proxy env vars.
However, PYTEST manages to unset PYTEST_CURRENT_TEST while urllib is in the iteration loop on the fly (since it finished the test) and this crashes the loop since it tries to access formerly set variable, which is now unset.
..bang.

i'm wondering whether there is a way to handle this from reportportal_client level.
A dirty way could be to explicitly catch the KeyError and simply try again

[DzmitryHumianiuk]

@reportportal/python-contributors fixed?

[rplevka]

@DzmitryHumianiuk well it certainly fixed it for me. There have been no other KeyErrors since merging this.

[DzmitryHumianiuk]

then closed.