Unusual index error in cryoCARE_extract_training_data.py

[jmdobbs]

Hello,

Thanks for all the great work on cryoCARE. I have been getting the below error recently while preparing the data for denoising (ie. while running cryoCARE_extract_training_data.py).

Does anyone know a cause, or if there's something that maybe I should be doing differently?

Thanks!!

42%|████▏ | 12596/30000 [03:32<03:34, 80.99it/s]
42%|████▏ | 12607/30000 [03:32<03:27, 83.99it/s]
42%|████▏ | 12618/30000 [03:32<03:13, 89.96it/s]
42%|████▏ | 12634/30000 [03:32<02:48, 102.97it/s]
42%|████▏ | 12646/30000 [03:32<02:42, 106.61it/s]
42%|████▏ | 12657/30000 [03:32<02:52, 100.46it/s]
42%|████▏ | 12668/30000 [03:32<02:57, 97.81it/s]
42%|████▏ | 12678/30000 [03:32<03:05, 93.48it/s]
42%|████▏ | 12688/30000 [03:33<03:18, 87.04it/s]
42%|████▏ | 12699/30000 [03:33<03:07, 92.35it/s]
42%|████▏ | 12709/30000 [03:33<03:23, 84.92it/s]
42%|████▏ | 12720/30000 [03:33<03:19, 86.79it/s]
42%|████▏ | 12733/30000 [03:33<02:57, 97.12it/s]
42%|████▏ | 12743/30000 [03:33<03:13, 89.39it/s]
42%|████▎ | 12750/30000 [03:33<04:49, 59.64it/s]
Traceback (most recent call last):
File "/g/easybuild/x86_64/Rocky/8/rome/software/cryoCARE/0.2.1-foss-2021a-CUDA-11.3.1/bin/cryoCARE_extract_train_data.py", line 45, in
main()
File "/g/easybuild/x86_64/Rocky/8/rome/software/cryoCARE/0.2.1-foss-2021a-CUDA-11.3.1/bin/cryoCARE_extract_train_data.py", line 27, in main
dm.setup(config['odd'], config['even'], n_samples_per_tomo=config['num_slices'],
File "/g/easybuild/x86_64/Rocky/8/rome/software/cryoCARE/0.2.1-foss-2021a-CUDA-11.3.1/lib/python3.9/site-packages/cryocare/internals/CryoCAREDataModule.py", line 200, in setup
self.train_dataset = CryoCARE_Dataset(tomo_paths_odd=tomo_paths_odd,
File "/g/easybuild/x86_64/Rocky/8/rome/software/cryoCARE/0.2.1-foss-2021a-CUDA-11.3.1/lib/python3.9/site-packages/cryocare/internals/CryoCAREDataModule.py", line 49, in init
self.compute_mean_std(n_samples=n_normalization_samples)
File "/g/easybuild/x86_64/Rocky/8/rome/software/cryoCARE/0.2.1-foss-2021a-CUDA-11.3.1/lib/python3.9/site-packages/cryocare/internals/CryoCAREDataModule.py", line 97, in compute_mean_std
x, _ = self.getitem(i)
File "/g/easybuild/x86_64/Rocky/8/rome/software/cryoCARE/0.2.1-foss-2021a-CUDA-11.3.1/lib/python3.9/site-packages/cryocare/internals/CryoCAREDataModule.py", line 158, in getitem
z, y, x = self.coords[tomo_index][coord_index]
IndexError: index 5 is out of bounds for axis 0 with size 5

[bionanopatterning]

Hi,

I was about to start a thread about this same error.

I think the values for 'num_slices' and 'n_normalization_samples' aren't independent. When the program is trying to pick normalization samples, it looks as though it picks a subset of slices from a previously generated pool of slices, or something like that. So when n_normalization_samples is larger than num_slices, it is trying to grab an element that is not in that pool and you end up with that error.

Setting n_normalization_samples to a value lower than num_slices seems to be a workaround.

edit: actually the maximum number seems to be num_slices * split, so n_normalization_samples depends on both num_slices and the fraction for the validation split...