Verification callback sometimes not called on OS X

[hynek]

We get a both scary and occasional test flake coming from verification callbacks not being called:

=================================== FAILURES ===================================
_______________ ContextTests.test_set_verify_callback_exception ________________
self = <tests.test_ssl.ContextTests testMethod=test_set_verify_callback_exception>
    def test_set_verify_callback_exception(self):
        """
            If the verify callback passed to :py:obj:`Context.set_verify` raises an
            exception, verification fails and the exception is propagated to the
            caller of :py:obj:`Connection.do_handshake`.
            """
        serverContext = Context(TLSv1_METHOD)
        serverContext.use_privatekey(
            load_privatekey(FILETYPE_PEM, cleartextPrivateKeyPEM))
        serverContext.use_certificate(
            load_certificate(FILETYPE_PEM, cleartextCertificatePEM))

        clientContext = Context(TLSv1_METHOD)

        def verify_callback(*args):
            raise Exception("silly verify failure")
        clientContext.set_verify(VERIFY_PEER, verify_callback)

        exc = self.assertRaises(
>           Exception, self._handshake_test, serverContext, clientContext)
tests/test_ssl.py:1214: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
self = <tests.test_ssl.ContextTests testMethod=test_set_verify_callback_exception>
exception = <type 'exceptions.Exception'>
f = <bound method ContextTests._handshake_test of <tests.test_ssl.ContextTests testMethod=test_set_verify_callback_exception>>
args = (<OpenSSL.SSL.Context object at 0x10a33e4d0>, <OpenSSL.SSL.Context object at 0x10a338350>)
kwargs = {}
cm = <[AttributeError("'ExceptionInfo' object has no attribute 'typename'") raised in repr()] SafeRepr object at 0x10a35a290>
    def failUnlessRaises(self, exception, f, *args, **kwargs):
        """
            Fail the test unless calling the function :py:data:`f` with the given
            :py:data:`args` and :py:data:`kwargs` raises :py:data:`exception`. The
            failure will report the traceback and call stack of the unexpected
            exception.

            :param exception: exception type that is to be expected
            :param f: the function to call

            :return: The raised exception instance, if it is of the given type.
            :raise self.failureException: Raised if the function call does
                not raise an exception or if it raises an exception of a
                different type.
            """
        with pytest.raises(exception) as cm:
>           f(*args, **kwargs)
E           Failed: DID NOT RAISE
tests/util.py:283: Failed

We should at least understand what’s going on here, otherwise we may just as well ask for a CVE ID now.

Not sure how cm = <[AttributeError("'ExceptionInfo' object has no attribute 'typename'") raised in repr()] SafeRepr is relevant here.

cc @reaperhulk

[reaperhulk]

I haven't been able to reproduce this locally :(

[hynek]

I have more details:

• pip install -e git+https://github.com/dropbox/pytest-flakefinder#egg=pytest-flakefinder is really useful. :)
• I test running py.test --flake-finder -k test_set_verify_callback_exception; I get about 6/50 failures.
• It only happens if I compile myself and don’t use a pre-built wheel (i.e. pip install --no-use-wheel cryptography)

So um, looks like a nasty bug in cryptography? :-/

[hynek]

Appears to be more of a cffi issue, removing milestone since we can’t do much about it…