SJICube Refactor

[DanRyanIrish]

SJICube was the first working data object in IRISpy and played an important role in proving that Python could be used to handle IRIS data well. Since then, however, there has been a lot of work done and SJICube is now due a refactor.

SJICube should be rewritten to inherit from ndcube.NDCube and many of the attributes should be pruned. In order to start the discussion on this topic I will outline some suggestion for the refactor.

• SJICube should be a subclass of ndcube.NDCube, thereby inheriting its slicing, plotting, WCS, extra coordinate, etc. capabilities.
• SJIMap should be removed as it will no longer be needed if SJICube inherits from ndcube.NDCube.
• SJICube's initialization should not depend on FITS but rather take SJI data already put into Python objects like numpy arrays. A separate FITS SJI reader function should then be written to take data from the file and put it into an SJICube. This would make SJICube independent of the file type and make it more flexible for users and testing.
• The majority of the properties for accessing the metadata should be removed. The metadata can then simply be stored in the .meta dictionary. However, some methods should remain, e.g. .draw_slit(). Further discussion is required to decide which should go and which should stay.

Since opening this issue, @BaptistePellorceAstro has been working to refactor SJICube has has/will pursue the following workflow:

Completed

• Write an SJI FITS reader function to put SJI data into an NDCube object. [Done in Reading FITS function added to sji.py #67]
• Create new SJI module, new_sji.py which will eventually replace sji.py [Done in Adding SJICube class using NDCube #68]
• Create new SJICube class, subclassed from NDCube. [Done in Adding SJICube class using NDCube #68]
• Move FITS reader function into new module and make it return an new SJICube rather than an NDCube. [Done in Adding SJICube class using NDCube #68]
• Add the option to use memory mapping to the SJI FITS reader function just like the __init__ function of the old SJICube did. This will also require the identification of the bad pixels to be done/bad pixel value defined based on whether memory mapping is turned on or off. [Done in Adding SJICube class using NDCube #68 except that mask and uncertainty are disabled when memmap is True.]
• Fix bug in NDCube.crop_by_coords when data is rotated. See crop_by_coords is broken when data is rotated ndcube#100 [Done in Solving crop_by_coord problem when data are rotated ndcube#113]
• Add an apply_exposure_time_correction method to SJICube based on IRISSpectrogram.apply_time_correction (https://github.com/sunpy/irispy/blob/master/irispy/spectrograph.py#L383) [Done in Adding a time correction function to SJICube #71]
• Decide on a new naming convention for SJI classes. See Issue Proposed Changed to naming convention of SJI Classes #73 [Done in New naming convention from SJICube and IRISSpectrogram #77]
• Develop a SJICubeSequence/IRISMapCubeSequence class that can combine data from different files of the same OBS number. See Issue New SJICubeSequence Class #72. [Done in Creation of IRISMapCubeSequence class #83]
• Add an explode_along_axis method to NDCube to return an NDCubeSequence of N-1DCubes. Similar to the method that exists in NDCubeSequence already. [Done in Adding a new explode_along_axis function inside NDCube ndcube#118]
• Add apply_exposure_time_correction method to IRISMapCubeSequence. [Done in PR Add apply exposure time correction method to IRISMapCubeSequence #88]

To Do

• Investigate how to include SJI dust locations in the SJI mask. This can be a file (or files as a function of time) storing a dust mask. Or it could be a function that determines the dust mask based on the data in the SJICube instance. See PR Creation of dust mask function #74
• Add documentation for IRISpy classes. See PR Documentation on IRISpy #89
• Investigate how NDCube can inherit the write method in astropy.nddata. See Add Save Method to NDCube ndcube#111

For extra optional credit (challenging):

• Alter/enhance SJICube plotting routines to enable the slit to be drawn on the images/animations.
• Fix issue Time-Dependent IRIS Effective Areas #27: Time-Dependent Response Function Fitting

[DanRyanIrish]

@BaptistePellorceAstro: I think the first stage of this effort is to write a reader function to read in the SJI data into an NDCube object. This could be put into the sji.py file. Then once that's working we can discuss the methods and metadata we want attached to the object (e.g. determining and storing the correct colour table based on the SJI bandpass) and that'll form the basis for the SJICube object.

[DanRyanIrish]

@BaptistePellorceAstro, well done for writing the new SJI reader function in PR #67.

I think the next step should be create a new SJICube class. For now let's call it _SJICube so we can develop, play, and merge it with it without overwriting the existing SJICube. We can do that once we've got _SJICube to a usable state.

For the first version of _SJICube, it should inherit all the attributes of NDCube and add/overwrite just a few methods. To inherit from NDCube, define _SJICube like so:

class _SJICube(NDCube):
    etc. etc.

The methods that you should include explicitly include in _SJICube at the moment are:

• __repr__: which defines what is printed to the screen when you simply enter the class name on the command line. This can be closely adapted from the current SJICube.__repr__ except that the metadata is now all accessed via the meta dictionary attribute rather than their own specific attributes. https://github.com/sunpy/irispy/blob/master/irispy/sji.py#L219
• draw_slit: See https://github.com/sunpy/irispy/blob/master/irispy/sji.py#L98

Then I think it would be a good time to have a conversation about what else is needed of _SJICube and how to achieve that.

[tiagopereira]

@DanRyanIrish, I had a chat with Baptiste about draw_slit. We should better define the purpose of this method and when it should be called.

If I understand correctly, under the old SJIMap only one timestep was shown, and therefore draw_slit made more sense. However, in the new SJICube we have a time sequence. The plot object is also no longer a simple plot but a WCS animation. One can call it with a boolean argument (draw_slit(True)) and then the .plot() method would have to know how to plot the slit when called, updating the location with timestep. This would avoid the ambiguity of what to do if draw_slit is called and there is no text.

A related question: is there an axvline equivalent that works with Quantity instead of pixels for WCSAxes?

[DanRyanIrish]

Hi @tiagopereira. Yes you're right. draw_slit will need to be folded into the SJICube.plot method. I think it's fine to leave it out of the next PR and just focus on the __repr__. I think a bigger priority for SJICube is to fix the bug in NDCube.crop_by_coods. But we should also have a call and discuss what needs to go into SJICube.

Regarding axvline for Quantities, I'm not sure. Perhaps WCSAxes does this by default? But I believe axvline works on the axis values, rather than the indices. So if WCSAxes doesn't automatically recognise equivalencies, it's an easy matter to convert the Quantity you want to use in axvline to the same unit as the axes.

[Cadair]

There isn't an equivalent of ax[h/v]line but there is http://docs.astropy.org/en/stable/api/astropy.visualization.wcsaxes.WCSAxes.html#astropy.visualization.wcsaxes.WCSAxes.plot_coord, which is probably your best bet.

[DanRyanIrish]

Now that @BaptistePellorceAstro is making great progress, I'd like to suggest revisit the roadmap agreed upon at the top of this issue. I have thought of two very important features that I think are very important for handling SJI and IRIS data at large. Depending on how much time @BaptistePellorceAstro has to go in his internship, we should consider working them into our roadmap.

First, SJICube cannot handle data from multiple files. Nor can it break up the frame in one SJICube into multiple SJICubes, each with its own WCS. This is important if fine-scale pointing corrections need to be made on a frame by frame basis. To add this capability, we need to develop a new class, SJICubeSequence, which inherits from NDCubeSequence. I will raise a separate issue outlining the details of how it should work.

Second, in order to facilitate going from an SJICube to an SJICubeSequence, with each image as its own 2D SJICube, we will need to develop an explode_along_axis method for NDCube, similar to the one that already exists in NDCubeSequence.

Thirdly, IRISpy desperately needs the ability to return the time-dependent response function. It can currently get the pre-launch response function but cannot do the necessary fitting/interpolation to get it based on a date. People at LMSAL said they would do this but there has been no progress for months. I'll look into whether work is in fact still going on. If not, we should consider adding this to our roadmap.

I believe that these three actions items are more important than those still on our roadmap with the exception of the dust mask.

[BaptistePellorceAstro]

Nice ! My internship will end at the end of June so I did the first half of my internship. Tell me on what you want I work and I will doing it ! I will take a view on spectrograph.py to see how NDCubeSequence is used.

[DanRyanIrish]

Sounds like you're enthusiastic for this way forwards, @BaptistePellorceAstro! Great. I'll alter the roadmap at the top of this issue accordingly and also inquire about what if any work has been done on the response function fitting.

[DanRyanIrish]

Roadmap updated.