-
Notifications
You must be signed in to change notification settings - Fork 0
SNARTomoClassic
SNARTomoClassic is the non-PACE version of SNARTomo. Since the images are collected at a lower rate, a single GPU is sufficient to keep up.
An example of usage is:
snartomo-classic --eer_dir frames/ --gain_file gain_reference --frame_file motioncor_frame_file --mdoc_file MDOC_file --outdir output_directory/ &
More detailed usage will be described below.
The following requirement is specific to SNARTomoClassic. This is in addition to the requirements of a gain file and a frame file. Also it needs to know the pixel size, either from an MDOC file or with the --apix flag.
Filenames
SNARTomo expects EER filenames with the following structure:
arbitraryTiltSeriesName_tiltSeriesIndex_angle_date.extension
-
arbitraryTiltSeriesName- It must be the same for each micrograph in the tilt series.
- Each tomogram will be written to a subdirectory with this name (see Outputs section below)
- This is the only part of the filename that can have underscores other than those shown in the template above.
-
tiltSeriesIndex– Each image in the tilt series is expected to have a unique (integer) index number. -
angle– SNARTomo will use that value as the tilt angle when computing the 3D reconstruction. -
date– This string can be anything, not necessarily the date. -
extension– SNARTomo can accept EER, MRC, or TIF movies.
Most settings have a default value (everything except the gain reference), so in principle, you could simply type snartomo-classic --gain_file /gain_reference/ without any further options specified. This is not recommended. Instead, I recommend a “dry run”, as described below.
Even better, enter snartomo-classic --help (or snartomo-classic -h) to see the full list of settings.
More information about the specific options can be found below.
I strongly recommend performing a dry run, with the --testing option:
snartomo-classic --eer_dir frames/ --gain_file gain_reference --frame_file motioncor_frame_file --mdoc_file MDOC_file --outdir output_directory/ --testing
The information written to the screen before SNARTomo starts looking for individual files includes summaries of how to use SNARTomo, and the settings used. Also note the Validation section. If required inputs are missing, they will be noted here. Some sanity checks will be performed here, to make sure your parameters are sensible.
If you are still collecting data, SNARTomo will continue to look for new files, and will not stop on its own. Enter ctrl-c to exit from a dry run.
If the dry run produces too much screen output, a useful parameter to change is --verbosity. The default value is 4. A verbosity of 6 will print the full command lines for each operation. A verbosity of 3 will print a summary for each tomogram. A verbosity of 2 will show notable warnings. For example:
snartomo-classic --eer_dir frames/ --gain_file gain_reference --frame_file motioncor_frame_file --mdoc_file MDOC_file --outdir output_directory/ --testing --verbosity=3
In testing mode, I recommend a verbosity level of 3 or (for big data sets) 2.
As of 2022 March 14, the pixel size is a required parameter. Previously, we specified the pixel size with the --apix command-line option. You can instead provide an example SerialEM MDOC file, in which case SNARTomo will extract the pixel size (and other information) from there.
snartomo-classic --eer_dir frames/ --gain_file gain_reference --frame_file motioncor_frame_file *--mdoc_file MDOC_file* --outdir output_directory/ --testing --verbosity=3
In addition to the pixel size, SNARTomo will check the defocus values and the number of frames in each micrograph. If you got outside of the defocus range (which CTFFINFD4 will use), it will print a warning. If you go outside of the number of frames, there may be something weird going on with your data collection. PAY ATTENTION TO THE WARNINGS!!
I will probably add other sanity checks in the future. I recommend using an MDOC file instead of supplying only the pixel size, and have updated the examples on this page accordingly.
Once you are satisfied with the results of the dry run, remove the --testing flag. A typical command to start a run might look like:
snartomo-classic --eer_dir frames/ --gain_file gain_reference --frame_file motioncor_frame_file --mdoc_file MDOC_file --outdir output_directory/ &
A log file – by default log-snartomo.txt and overridden by --log_file – will contain everything printed to the screen. If something goes wrong, the log file will be among the first things that we ask for.
You might want to use tmux to work on a remote machine. If you would like to exit the session while SNARTomo is still running, you might want to start a screen session. Or you can simply leave that console open. I recommend adding an ampersand (&) at the end of the command; SNARTomoClassic seems to work more reliably after you’ve detached a screen session.
SNARTomo would be content to wait for new data forever. There are two ways for SNARTomo to exit automatically:
- a dummy file
- a predetermined time limit
Dummy file: After your data collection has finished, create a dummy file in your EER directory that breaks the file-naming convention above, such as done.eer. (SNARTomo will also recognize such a file in testing mode. An example command to create a dummy file would be
touch frames/done.eer
Time limit: Alternatively, SNARTomo will stop looking for files after a predetermined limit, set by the command-line option --max_minutes. The default is 600 minutes, i.e., 10 hours.
NOTE : The last 3D reconstruction will not be computed until SNARTomo thinks that data-collection has ended.
UNDER CONSTRUCTION
- General difficulty – Perform a dry run, and pay close attention to the errors and warnings.
- Program finds only one tilt series or one micrograph – make sure your filenames follow the standard convention
For the most current settings, enter:
snartomo-classic --help
You can alternatively enter snartomo-classic -h
Any of these values can be overridden on the command with the use of the appropriate flag, using the form:
snartomo-classic --flag_to-override=your_new value
The “=” sign is optional, and can be replaced by one or more spaces.
Data type below is as defined in Markus Stabrin’s argumentparser_dynamic.sh.
Movie settings
One of these three flags is required.
| Flag | Type | Default | Description |
|---|---|---|---|
--eer_dir |
DIR | frames | Input EER directory |
--mrc_dir |
DIR | frames | Input MRC directory |
--tif_dir |
DIR | frames | Input TIFF directory |
Global
These settings affect multiple steps.
| Flag | Type | Default | Description |
|---|---|---|---|
--outdir |
ANY | SNARTomo | Output directory |
--mdoc_dir |
ANY | None | Input MDOC directory, will be copied to reconstruction directory |
--mdoc_file |
ANY | None | Single input example MDOC file, to extract pixel size, etc. |
--apix |
FLOAT | -1.0 | Pixel size, Å/px |
--verbosity |
INT | 4 (3 or 2 recommended for testing) | Verbosity level (0..9) |
--restore_movies |
BOOL | false | Move already-processed movies from output directory to input directory |
--testing |
BOOL | false | Testing mode |
--overwrite |
BOOL | false | Overwrite output directory (only if no EERs) |
--log |
ANY | log-snartomo.txt | Output log file |
--max_minutes |
INT | 600 | Maximum run time, minutes |
--kv |
FLOAT | set in snartomo.bashrc | Voltage, kV |
--gpus |
ANY | 0 | GPU to use (only single GPU supported in Classic mode) |
Advanced
These settings will rarely need to be changed.
| Flag | Type | Default | Description |
|---|---|---|---|
--search_interval |
INT | set in snartomo.bashrc | Interval to check for new micrographs, seconds |
--min_frames |
INT | set in snartomo.bashrc | Minimum number of EER frames before warning |
--max_frames |
INT | set in snartomo.bashrc | Maximum number of EER frames before warning |
--eer_local |
BOOL | set in snartomo.bashrc | Flag to temporarily copy EER files locally |
--eer_latency |
FLOAT | set in snartomo.bashrc | Maximum header read time (seconds) before EER is copied locally |
--debug |
BOOL | false | Flag for debugging |
MotionCor2
| Flag | Type | Default | Description |
|---|---|---|---|
--frame_file |
ANY | motioncor-frame.txt | Input MotionCor2 frame file |
--gain_file |
FILE | None | Input gain file |
--no_gain |
BOOL | false | Flag when no gain file is to be used |
--motioncor_exe |
ANY | set in snartomo.bashrc | MotionCor2 executable |
--do_dosewt |
BOOL | False | Flag to perform dose-weighting |
--mcor_patches |
ANY | set in snartomo.bashrc | Number of patches in x y, delimited by spaces and in quotes |
--do_outstack |
BOOL | False | Write aligned stacks |
--do_splitsum |
BOOL | False | Split frames into half-sets |
--split_sum |
INT | 0 | (Deprecated) Split frames into half-sets (0: no, 1: yes) |
--reffrm |
INT | 1 | Reference frame (0: first, 1: middle) |
--mc2_wait |
FLOAT | set in snartomo.bashrc | Maximum wait (seconds) for other user’s MotionCor2_FreeGpus.txt file to be deleted |
CTFFIND4
| Flag | Type | Default | Description |
|---|---|---|---|
--ctffind_dir |
ANY | set in snartomo.bashrc | CTFFIND executable directory |
--cs |
FLOAT | set in snartomo.bashrc | Spherical aberration constant |
--ac |
FLOAT | set in snartomo.bashrc | Amplitude contrast |
--box |
INT | set in snartomo.bashrc | Tile size for power-spectrum calculation |
--res_lo |
FLOAT | set in snartomo.bashrc | Low-resolution limit for CTF fitting, Å |
--res_hi |
FLOAT | set in snartomo.bashrc | High-resolution limit for CTF fitting, Å |
--df_lo |
FLOAT | set in snartomo.bashrc | Minimum defocus value, Å |
--df_hi |
FLOAT | set in snartomo.bashrc | Maximum defocus value, Å |
--df_step |
FLOAT | set in snartomo.bashrc | Defocus search step during fitting, Å |
--ast_step |
FLOAT | set in snartomo.bashrc | Astigmatism search step during fitting, Å |
--ctf1d_dpi |
INT | 250 | Resolution (dpi) of PNG of 1D CTF profile |
--ctfplot_first |
INT | -1 | Plots first CTF data for first N images in tilt series (-1 -> all) |
JANNI
| Flag | Type | Default | Description |
|---|---|---|---|
--do_janni |
BOOL | false | Flag to denoise using JANNI |
--janni_batch |
INT | set in snartomo.bashrc | Number of patches predicted in parallel |
--janni_overlap |
INT | set in snartomo.bashrc | Overlap between patches, pixels |
--janni_env |
ANY | set in snartomo.bashrc | JANNI conda environment |
Topaz
| Flag | Type | Default | Description |
|---|---|---|---|
--do_topaz |
BOOL | false | Flag to denoise |
--topaz_patch |
INT | set in snartomo.bashrc | Patch size |
--topaz_env |
ANY | set in snartomo.bashrc | Topaz conda environment |
IMOD
| Flag | Type | Default | Description |
|---|---|---|---|
--imod_dir |
ANY | set in snartomo.bashrc | IMOD executable directory |
--batch_directive |
ANY | batchDirective.adoc | IMOD batch directive file |
AreTomo
| Flag | Type | Default | Description |
|---|---|---|---|
--aretomo_exe |
ANY | set in snartomo.bashrc | AreTomo executable |
--are_bin |
INT | set in snartomo.bashrc | Binning factor for reconstruction |
--vol_zdim |
INT | set in snartomo.bashrc | z-dimension for volume |
--rec_zdim |
INT | set in snartomo.bashrc | z-dimension for reconstruction |
--tilt_axis |
FLOAT | set in snartomo.bashrc | Estimate for tilt-axis direction, degrees |
--dark_tol |
FLOAT | set in snartomo.bashrc | Tolerance for dark images (0.0-1.0) |
--tilt_cor |
INT | set in snartomo.bashrc | Tilt-correction flag (1: yes, 0: no) |
--bp_method |
INT | set in snartomo.bashrc | Reconstruction method (1: weighted backprojection, 0: SART) |
--flip_vol |
INT | set in snartomo.bashrc | Flag to flip coordinates axes (1: yes, 0: no) |
--transfile |
INT | set in snartomo.bashrc | Flag to generate IMOD XF files (1: yes, 0: no) |
--are_patches |
ANY | set in snartomo.bashrc | Number of patches in x & y (delimited by spaces) |
--are_time |
ANY | set in snartomo.bashrc | Maximum duration (AreTomo sometimes hangs) |