Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assigning f8 2d data to a table can lead to corrupt output #280

Open
braingram opened this issue Mar 6, 2024 · 2 comments
Open

Assigning f8 2d data to a table can lead to corrupt output #280

braingram opened this issue Mar 6, 2024 · 2 comments
Labels
api-refactor part of major API changes bug Something isn't working

Comments

@braingram
Copy link
Collaborator

braingram commented Mar 6, 2024

Let's assume we have a schema for TableModel that contains

      one_column_table:
        fits_hdu: ONECOL
        datatype:
          - name: col
            datatype: float64

If we assign data to the model as follows, the produced file is corrupt:

fn = tmp_path / "test.fits"

write_model = TableModel()
arr = np.arange(16 * 32, dtype='f8').reshape((16, 32))
write_model.one_column_table = arr
np.testing.assert_equal(arr, write_model.one_column_table['col'])
write_model.save(fn)

with TableModel(fn) as read_model:
    np.testing.assert_equal(arr, read_model.one_column_table['col'])

The above code fails on TableModel(fn) with:

E           astropy.utils.exceptions.AstropyUserWarning: ERROR loading embedded ASDF: non-ASCII characters are present in the FITS file header and have been replaced by "?" characters

This can be further reduced to just astropy:

from astropy.io import fits
import numpy as np

hdulist = fits.HDUList()
hdulist.append(fits.PrimaryHDU())

arr = np.arange(32 * 16).reshape((32, 16)).astype(dtype=[('col', 'f8')])

hdulist.append(fits.BinTableHDU())
hdulist[-1].data = arr

hdulist.writeto('foo.fits', overwrite=True)

with fits.open('foo.fits') as ff:
    ff.info()

Running the above code results in:

UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position 2: ordinal not in range(128)
and
OSError: Header missing END card.

Note that in all cases the error doesn't occur until the data is read. No errors are issued on write.

@braingram
Copy link
Collaborator Author

Using f4 does not produce the same error but does produce a file with an incorrectly shaped array ((32) instead of (32, 16)).

@braingram
Copy link
Collaborator Author

This may be related to #215

@emolter emolter added bug Something isn't working api-refactor part of major API changes labels Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-refactor part of major API changes bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants