AttributeError: module 'h2oaicore.models' has no attribute 'arithmetic_mean'

[pseudotensor]

class LinearBlenderEx(MetaModel):
    def __init__(self, experiment_id=None, context=None, num_classes=None, score_f=None,
                 maximize=False, classification=False, labels=None, precision=1, repeats=5,
                 seed=1, verbose=0, n_jobs=-1, optimizer='hc'):
    def arithmetic_mean(self, X, w, pdim=None):
        pdim = self.pdim if pdim is None else pdim
        return (np.nan_to_num(X * np.repeat(w, pdim))).reshape(-1, self.n_models, pdim).sum(axis=1).reshape(-1, pdim)

Traceback (most recent call last):
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 269, in _getattribute
    obj = getattr(obj, subpath)
AttributeError: module 'h2oaicore.models' has no attribute 'arithmetic_mean'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 918, in save_global
    obj2, parent = _getattribute(module, name)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 272, in _getattribute
    .format(name, obj))
AttributeError: Can't get attribute 'arithmetic_mean' on <module 'h2oaicore.models' from '/home/jon/.pyenv/versions/3.6.4/lib/python3.6/site-packages/h2oaicore/models.cpython-36m-x86_64-linux-gnu.so'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "h2oaicore/systemutils.py", line 2998, in h2oaicore.systemutils.traced_func
  File "h2oaicore/auto_dl_support.py", line 517, in h2oaicore.auto_dl_support.make_final_scoring_pipeline
  File "h2oaicore/auto_dl_support.py", line 511, in h2oaicore.auto_dl_support.make_final_scoring_pipeline
  File "h2oaicore/systemutils.py", line 886, in h2oaicore.systemutils.save_obj_gz
  File "h2oaicore/systemutils.py", line 891, in h2oaicore.systemutils.save_obj
  File "h2oaicore/systemutils.py", line 904, in h2oaicore.systemutils.save_obj_unsafe
  File "h2oaicore/systemutils.py", line 905, in h2oaicore.systemutils.save_obj_unsafe
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/site-packages/dill/_dill.py", line 286, in dump
    pik.dump(obj)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 409, in dump
    self.save(obj)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 521, in save
    self.save_reduce(obj=obj, *rv)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 634, in save_reduce
    save(state)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/site-packages/dill/_dill.py", line 893, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 521, in save
    self.save_reduce(obj=obj, *rv)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 634, in save_reduce
    save(state)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/site-packages/dill/_dill.py", line 893, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 821, in save_dict
    self._batch_setitems(obj.items())
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 847, in _batch_setitems
    save(v)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/site-packages/dill/_dill.py", line 1069, in save_instancemethod0
    pickler.save_reduce(MethodType, (obj.__func__, obj.__self__), obj=obj)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 610, in save_reduce
    save(args)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 476, in save
    f(self, obj) # Call unbound method with explicit self
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 736, in save_tuple
    save(element)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 507, in save
    self.save_global(obj, rv)
  File "/home/jon/.pyenv/versions/3.6.4/lib/python3.6/pickle.py", line 922, in save_global
    (obj, module_name, name))
_pickle.PicklingError: Can't pickle <cyfunction LinearBlenderEx.arithmetic_mean at 0x7fe4a47dc270>: it's not found as h2oaicore.models.arithmetic_mean

This makes dill entirely unusable, since we required cythonized code.

Nothing like this happens with pickle. Seems dill can't handle the fact that we cythonized the code. Without cythonized code nothing like this happens.

[pseudotensor]

It seems the flaw is that "arithmetic_mean" is being looked for in h2oaicore.models but that's not where it is located. Instead it's located inside the class. Very strange dill is looking in wrong place for the function.

[mmckerns]

Same as #208. It's a known issue that requires better support for qualname than provided by pickle (which dill leverages).

[mmckerns]

Upon further review, I believe this is actually at root the same issue as #56.