SNARTomoHeatwave
Heatwave is a GUI for screening and curating data generated by SNARTomo.
Starting Heatwave from the command line has the form:
snartomo-heatwave.py
With no additional arguments, Heatwave will assume that the JSON metadata file “heatwave.json” exists. Recent versions of SNARTomo create this JSON file. If you don’t have this JSON file for whatever reason, see the next section below.
Heatwave builds metadata starting from one or more SerialEM MDOC files and/or PACE-tomo target files. If you’re not sure which to use, check the commands.txt from your SNARTomo run, and look for either a --mdoc_files or --target_files flag.
To start a new session, include the --new flag.
Using MDOC files
Here is an example of starting a new session using MDOC files:
snartomo-heatwave.py --mdoc_files "SNARTomo/5-Tomo/P*/*mdoc" --new
It is assumed that each tilt series (in SNARTomo/5-Tomo/P*/*mdoc by default) has its own MDOC file. The exact directory structure can be overridden (see below), but each tilt-series directory must have its own MDOC file.
If more than one MDOC file is provided, they must be enclosed in quotes. Wild cards are allowed.
Using target files
Here is an example of starting a new session using PACE-tomo target files:
snartomo-heatwave.py --target_files "PN17_ctrl_*_tgts*.txt" --new
If more than one target file is provided, they must be enclosed in quotes. Wild cards are allowed.
Resuming a session
If you have already run Heatwave in a given working directory, you can resume simply by typing:
snartomo-heatwave.py
The metadata will have been saved in a JSON file, named “heatwave.json” by default. Without the --new flag, this JSON file will be read.
Adding new data
You can add new tilt series by providing MDOC or target files without the --new flag.
Heatwave does NOT currently add data on-the-fly yet.
Overview
Heatwave groups micrographs by tilt series, and tilt series by target file. (If not presented with target files, a virtual target file labelled “All tilt series” will be used.) This hierarchy can be expanded or collapsed with a tree-view (like in an operating system’s file manager).
Opening metadata
Right-clicking on Heatwave will allow you to open the corresponding linked data types. Currently, these data types include the following:
- tilt series
- power-spectra stacks
- MRC files
- plots: CTF scatter plots and dose-fitting
MRC files will be opened by IMOD’s 3dmod, and plots will be opened by default by ImageMagick’s display program.
Discarding micrographs
SNARTomo and AreTomo can already remove bad micrographs. In Heatwave, you can further discard micrographs. To do so, deselect micrographs using the checkbox. When you’re finished, click the button at the top “Restack micrographs”. It will create a new stack file and a new MDOC file.
The selection/deselection state of micrographs will be saved to the JSON file if you click “Save JSON”. This button will also save the editable text boxes (see screenshot below) associated with each tilt series. If you change the selection/deselection state (or edit text boxes), and try to exit without saving to JSON, Heatwave will remind you.
Discarding tilt series
Discarding entire tilt series, and the data associated with them, can save a lot of disk space, and network bandwidth when transferring data elsewhere. To deselect tilt series, like with micrographs, uncheck the checkbox associated with that MDOC file. Once you have finished, click “Incincerate tilt series”. This button will, first, move the tomogram-reconstruction directory (by default SNARTomo/5-Tomo), second, the data associated with that tilt series (for example, movies and motion-corrected micrographs), and third, the data displayed associated with those tilt series from the Heatwave GUI. The deleted files will be moved to a directory called (by default) INCINERATE.
If you erroneously delete tilt series, you can restore them easily ONLY DURING THE SAME SESSION with the “Unincinerate files” button. If you decide after you’ve closed Heatwave that you want to resurrect those files, you’ll need to manually move them. The directory structure in the INCINERATE directory should be the same as in your input directory, with the exception of the frames directory, which may be somewhere else.
Once you’re certain that you don’t want to keep the incinerated data, you can remove that directory, from the terminal or from a graphical file manager.
For the most current settings, enter:
snartomo-heatwave.py --help
You can alternatively enter snartomo-heatwave.py -h
Any of these parameters can be overridden on the command with the use of the appropriate flag, using the form:
snartomo-heatwave.py --flag_to-override=your_new_value
The “=” is optional, and can be replaced by one or more spaces.
General options
If starting a new session, the --new flag is required.
| Flag | Type | Default | Description |
|---|---|---|---|
--new |
BOOL | False | Flag to start a new session |
--json |
ANY | heatwave.json | Input/output JSON file, may be modified |
Input files
These files are required to help Heatwave find the files associated with each tilt series.
| Flag | Type | Default | Description |
|---|---|---|---|
--target_files |
ANY | None | PACE-tomo target files, surrounded by quotes if more than one |
--mdoc_files |
ANY | None | SerialEM MDOC files, surrounded by quotes if more than one |
Wild cards are allowed. Arguments containing wild cards must also be enclosed in quotes.
General parameters
| Flag | Type | Default | Description |
|---|---|---|---|
--no_imgs |
BOOL | False | Displays metadata without images (fast) |
--imgsize |
INT | 160 | Image size, pixels |
--expand |
BOOL | False | Flag to expand treeview to show micrographs upon startup |
--verbosity |
INT | 3 | Verbosity level (0..9) |
--no_rotate |
BOOL | False | Flag to display volumes in 3dmod with default orientation |
--no_gui |
BOOL | False | Flag to not open GUI, used for preparing JSON file |
--debug |
BOOL | False | Shows debugging info |
File patterns
Filename conventions expected by Heatwave. Defaults are established by SNARTomo.
| Flag | Type | Default | Description |
|---|---|---|---|
--in_dir |
ANY | SNARTomo | Top-level directory for inputs |
--movie_dir |
ANY | frames | Movie directory (e.g., EER, TIF, MRC) |
--micthumb_dir |
ANY | Thumbnails | Directory for micrograph and power-spectrum thumbnails relative to MDOC file |
--tif_dir |
ANY | $IN_DIR/1-Compressed | Relative path of compressed-movie directory |
--mic_dir |
ANY | $IN_DIR/2-MotionCor2 | Relative path of motion-corrected micrograph directory |
--mic_pattern |
ANY | _mic.mrc | Suffix for motion-corrected micrograph, including extension |
--micthumb_suffix |
ANY | _newstack | Suffix appended to MDOC basename in micrograph thumbnail images (and pattern in micrograph stack ending in ‘.mrcs’ or ‘.st’) |
--ctfthumb_suffix |
ANY | _ctfstack_center | Suffix appended to MDOC basename in CTF thumbnail images (and pattern in CTF stack) |
--recon_pattern |
ANY | *_rec*mrc *_aretomo*.mrc |
Pattern for reconstuctions (if more than one, separated by spaces and in quotes) |
--ctf_summary |
ANY | SUMMARY_CTF.txt | CTFFIND4 summary file, in same directory as MDOC file |
--slice_jpg |
ANY | _slice_norm.jpg | Suffix for central-slice image in MDOC directory, including extension |
--ctfbyts_tgts |
ANY | $IN_DIR/Images/ctfbyts*.png | File pattern for target-file CTF scatter plot |
--ctfbyts_1ts |
ANY | ctfbyts.png | Single-tilt-series CTF scatter plot, in same directory as MDOC file |
--denoise_dir |
ANY | $IN_DIR/4-Denoise | Relative path of denoised-micrograph directory |
--ts_dir |
ANY | $IN_DIR/5-Tomo/$MDOC_STEM |
Relative path of tilt-series data directory |
--dosefit_plot |
ANY | *_dose_fit.png | Pattern for dose-fitting plot, in same directory as MDOC file |
--stack_suffix |
ANY | _restack | Suffix for restacking text output, without extension |
--thumb_format |
ANY | jpg | Image format for micrographs and power-spectrum thumbnails |
--incinerate_dir |
ANY | $IN_DIR/INCINERATE | Relative path of directory where incinerated files will be moved |
In the defaults above, $IN_DIR and $MDOC_STEM will be replaced by the input directory and MDOC prefix, respectively.
Executables
The following programs are used for display from the GUI.
| Flag | Type | Default | Description |
|---|---|---|---|
--imod_bin |
ANY | set in snartomo.bashrc | IMOD binary directory (only needed if not in $PATH) |
--img_viewer |
ANY | display | Image-viewer executable |
libGL error: MESA-LOADER: failed to open iris
If you get a warning of the form libGL error: MESA-LOADER: failed to open iris, follow this suggestion from StackOverflow.
First, find the file “libstdc++.so.6” with the following command:
find / -name libstdc++.so.6 2>/dev/null
Then, assuming this file is found at “/usr/lib/x86_64-linux-gnu/libstdc++.so.6”, enter the following:
export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6
If this solution makes the warning go away, add this line to your “snartomo.bashrc” so that you don’t have to enter it every time.