Missing dependency causes strange error during unit tests

[sjoro]

Code Sample, a minimal, complete, and verifiable piece of code

python setup.py test

Problem description

Trying to run unit tests from command line with the command above produces an error

AttributeError: 'module' object has no attribute 'tests'

Versions of Python, package at hand and relevant dependencies

python 3.4.6
satpy master

Behaviour verified by me and @adybbroe. Python versions 2.7.5 and 3.6.4 work.

Thank you for reporting an issue !

[djhoese]

What about with satpy's develop branch?

[djhoese]

@sjoro @adybbroe Another question, are there errors further up about certain dependencies not being built or satpy failing to install? The module not having tests makes me think that it wasn't installed all the way by setup.py.

[adybbroe]

I think we have hit the problem with an unresolved import. The message is just cryptic!
https://stackoverflow.com/questions/25575073/attributeerror-module-object-has-no-attribute-tests
We will have a look later and see if we can solve what it is/was!

[djhoese]

Interesting. What is the import? If that is something other than the required deps in setup.py then it should be moved in to a test function.

[adybbroe]

It was netCDF4. Fixed with pip install netCDF4

[djhoese]

@adybbroe Any idea what test file imports netCDF4 directly? Imports for optional dependencies should only happen inside the test functions.

[sjoro]

@davidh-ssec it*s from satpy.readers.netcdf_utils

[djhoese]

Ah looks like hdf5_utils and netcdf_utils tests import those file handlers outside of the tests to make a fake handler class. Hmmm not sure the best way to handle that right now.

[adybbroe]

@djhoese Maybe make a new issue, and close this one then?

[djhoese]

I'll just make this issue that issue.

[djhoese]

I think the best (not a ton of work) workaround is to use object if NetCDF4 isn't available during unittests and then skip the tests if NetCDF4 isn't available. The first part is needed to import the tests, the second part is needed because the tests won't run properly without NetCDF4.

[sjoro]

Now I have a similar issue with pyhdf. I'm trying to run the tests for native_msg.

from satpy.tests.reader_tests.test_native_msg import TestNativeMSGFileHandler

ImportError                               Traceback (most recent call last)
<ipython-input-1-b93bf9da6654> in <module>()
----> 1 from satpy.tests.reader_tests.test_native_msg import TestNativeMSGFileHandler

/home/joro/git/satpy/satpy/tests/__init__.py in <module>()
     26 import sys
     27 
---> 28 from satpy.tests import (reader_tests, test_dataset, test_file_handlers,
     29                          test_readers, test_resample,
     30                          test_scene, test_utils, test_writers,

/home/joro/git/satpy/satpy/tests/reader_tests/__init__.py in <module>()
     25 import sys
     26 
---> 27 from satpy.tests.reader_tests import (test_abi_l1b, test_hrit_base,
     28                                       test_viirs_sdr, test_viirs_l1b,
     29                                       test_native_msg, test_msg_base,

/home/joro/git/satpy/satpy/tests/reader_tests/test_hdf4_utils.py in <module>()
      8 import numpy as np
      9 import xarray as xr
---> 10 from satpy.readers.hdf4_utils import HDF4FileHandler
     11 if sys.version_info < (2, 7):
     12     import unittest2 as unittest

/home/joro/git/satpy/satpy/readers/hdf4_utils.py in <module>()
     27 import logging
     28 
---> 29 from pyhdf.SD import SD, SDC, SDS
     30 import dask.array as da
     31 import xarray as xr

ImportError: No module named 'pyhdf'

[djhoese]

At least you got a logical error this time. I think I've fixed this in my grib PR or at least "fixed" it. My solution would cause failed tests for the missing import. I suppose I should add all of these dependencies to the test requires in setup.py.

[djhoese]

@sjoro How about I add the test requires to my branch, wait for tests to pass, merge to develop, and you test that out?

[sjoro]

@djhoese sure, sounds good!