passes cache_spec=False to nwb_io.write() to avoid spec writer shape …

[MatthewAitken]

Resolve #404 …error

[t-b]

Is it really required to not cache the spec? Caching the specification should always be done in order to ensure that the files can be validated later on.

[MatthewAitken]

Is it really required to not cache the spec? Caching the specification should always be done in order to ensure that the files can be validated later on.

@t-b When we write the file after adding the spike times, during the spec update in H5SpecWriter (https://github.com/hdmf-dev/hdmf/blob/dev/src/hdmf/backends/hdf5/h5_utils.py#L276) H5py's self.__group.require_dataset(name, shape=tuple(), data=data, dtype=self.__str_type) is called with the shape=tuple() argument. But the preexisting dataset has shape (1,), so it errors out instead keeping the old spec. This is just a temporary fix right now, and we do want to update the spec when we write nwb files. I'm opening an issue on pynwb.

[wbwakeman]

Will this change be used by ipfx.attach_metadata?
(i.e. when I do feature extraction with bin/run_pipeline_from_nwb_file.py it works, but when I try to attach metadata I still get the "Shapes do not match" error)

[sgratiy]

@wbwakeman have skipping spec update in the attach_metadata module fixed the issue with metadata attachment?

[wbwakeman]

@sgratiy Yes, I can attach metadata now

[sgratiy]

@MatthewAitken I could not see the issue open on pynwb as mentioned above. I wanted to link it to this issue.
Are we ready to merge this PR?

[wbwakeman]

@MatthewAitken I discussed this with @t-b and he had some good thoughts on the issue. The NWB2 files that I am working with for the June release are being written by MIES and include a cached_specification already. When ipfx attach_metadata and append_spike_times write the NWB2 file, they are also attempting to cache the spec again. This causes the "Shapes do not match" error.

This PR solves that problem by not caching the spec again, which leaves the one written by MIES. Another approach (thanks @t-b ) would be to remove the MIES one. That can be done with something like

import h5py
with h5py.File("HardwareTests-V2.nwb", mode="r+") as f:
    for spec in ['/specifications/core', '/specifications/hdmf-common']:
        if spec in f:
            del f[spec]

What do you think about following this strategy to correct the problem?

[wbwakeman]

This may be unnecessary once this PR is merged?
hdmf-dev/hdmf#346