Skip to content

Latest commit

 

History

History
162 lines (106 loc) · 9.07 KB

ImageGraphicalStuffinUnix.md

File metadata and controls

162 lines (106 loc) · 9.07 KB

Image/graphical utilities in unix

Useful tips and utilities for working with image and vector files in commandline.

Timestamps and other metadata from TIFF, JPEG

JPEG metadata can be inspected using the exif unix utility. Standard exif tags can be looked up in a table -- they are standardised with the file format. A sample JPEG file can be looked up using your OS (properties in Windows or info on MacOS) to determine what values are stored. For example, the original date and time the image data were created is stored here:

0x9003 36867 Image Exif.Image.DateTimeOriginal Ascii The date and time when the original image data was generated.

So if the metadata are present, exif -t 0x9003 Test.jpg will output the following:

Tag: 0x9003 ('DateTimeOriginal')
  Format: 2 ('ASCII')
  Components: 20
  Size: 20
  Value: 2018:09:02 03:10:55

Of this you need the Value, which you can get by grep: exif -t 0x9003 Test.jpg | grep 'Value:'. This trick can be used to obtain frame rates from time lapse microscopy images saved individually. Enter the directory with the time lapse JPEGs and run:

for f in *.jpg ; do exif -t 0x9003 $f | grep 'Value:' ; done

This will generate a list like this:

  Value: 2016:07:14 20:03:05
  Value: 2016:07:14 20:03:06
  Value: 2016:07:14 20:03:06
  Value: 2016:07:14 20:03:06
  Value: 2016:07:14 20:03:06
  Value: 2016:07:14 20:03:06
  Value: 2016:07:14 20:03:06
  Value: 2016:07:14 20:03:06
  Value: 2016:07:14 20:03:06
  Value: 2016:07:14 20:03:06
  Value: 2016:07:14 20:03:07
  Value: 2016:07:14 20:03:07
  Value: 2016:07:14 20:03:07
  Value: 2016:07:14 20:03:07
  Value: 2016:07:14 20:03:07
  Value: 2016:07:14 20:03:07
  Value: 2016:07:14 20:03:07
  Value: 2016:07:14 20:03:07
  Value: 2016:07:14 20:03:07
  Value: 2016:07:14 20:03:08
  Value: 2016:07:14 20:03:08
  Value: 2016:07:14 20:03:09

Pick a second that is between either end of the sequence (note that this is at the end of the time lapse, and the final image was recorded with a significant delay), use grep to obtain all the values matching it and simply count the lines to get the frame rate for that second. Repeat for a couple other timepoints to ensure the value is consistent. With the example above,

for f in *.jpg ; do exif -t 0x9003 $f | grep 'Value:' ; done | grep '20:03:07' | wc -l

will yield a count of 9 (as will grep 20:03:06). This matches the max framerate of the camera used to record this particular image sequence. A better approach, of course, is to record the stated frame rate shortly after image collection.

A similar approach can be used for TIFF metadata. tiffinfo obtains the metadata (note: without the filename), and DateTime, if present, can be grepped out of it. To obtain a table with the filename as well as the DateTime, the following command can be used:

for f in *.TIF ; do printf "$f\t" ; tiffinfo "$f" | grep DateTime | sed 's/^.* //'; done

In addition to time information, these tools can be used to extract other properties from JPEG and TIFF metadata in batch.

Converting SVG to PNG and JPEG

cairosvg (source) converts SVG to PNG, PDF, PS (and is also a Python 3.6+ library). It seems to do a better job at SVG --> PNG conversion than ImageMagick (below).

The following command worked for converting SVGs (generated by Illustrator) to decent quality PNGs:

$ cairosvg --width 1000 -s 5 -d 300 -o Test.png 'Test.svg'

-s denotes scaling factor -- in this example, 5x. Setting width without scaling will not increase (or decrease) image resolution. width (in pixels) might be unnecessary here, I haven't tested without it. -d is DPI, which AFAIK is meaningless in this context but sometimes required a certain way by publishers. However, cairosvg lacks JPEG conversion. There also does not seem to be a function to set background colour or transparency in the PNG export, although apparently there are additional functions available via Python script, e.g. for transparent backgrounds.

ImageMagick (source) converts between image types, but also has a variety of other tools. The conversion tool is called as convert (without ImageMagick).

warning: the command mogrify modifies the original file!

warning #2: excess memory use crashes can lead to the original file being corrupted -- make copies before working with files.

I used the following script on the PNG generated by cairosvg to convert to JPEG:

$ convert 'Test.png' -background White -alpha background 'Test.jpg'

-alpha background modified the transparent background of the PNG to -background White (default is black since JPEG does not support transparency). Note that this method does not save any exif metadata with the JPEG file.

This was converted to a batch script run inside the directory with SVG files to be converted:

for file in *.svg; do
    cairosvg --width 1000 -s 5 -d 300 -o "${file%.*}.png" "${file}"
    convert "${file%.*}.png" -background White -alpha background "${file%.*}.jpg" 
done

This seemed to not have issues with whitespace in filenames even though the individual commands above would fail. Inspect generated JPEGs manually as SVGs of different dimensions might need different --width and --scale. Note that fonts may be changed in the process.

Other options (untested by author):

Converting 12-bit TIFFs saved as 16-bit to actually 16-bit

Some microscopy software by default saves 12-bit monochrome data in 16-bit TIFFs...but does not scale the values, so only 1/16th of the range is used*. When opening files like this in most software, they appear black except for the brightest spots that can be very faint dark grey. So far, I've only found ImageJ to correctly read these non-standard 12-bit TIFFs, so good on them -- but this was a program written specifically to work with awkward microscopy software outputs. A suboptimal work-around is to use ImageJ to save them as 8-bit TIFFs or, sigh, JPEGs. But you paid for a 12-bit camera, so it would be nice to have all the...bits.

Thankfully, these files can be converted to real 16-bit using the -evaluate Multiply function in ImageMagick (see above). Using instructions discussed here: Saving 12-bit tiff images · ImageMagick/ImageMagick · Discussion #4525 · GitHub

convert seq8712.tif -evaluate Multiply 16 seq8712.16.tif

(alternatively, you can use bit shift: -evaluate LeftShift 4 ... that might be safer)

With images generated by our Nikon software, we get the following warnings:

convert-im6.q16: Unknown field with tag 40961 (0xa001) encountered. `TIFFReadDirectory' @ warning/tiff.c/TIFFWarnings/949.
convert-im6.q16: Unknown field with tag 65325 (0xff2d) encountered. `TIFFReadDirectory' @ warning/tiff.c/TIFFWarnings/949.

Running tiffinfo on the original file reveals some tags that look like this:

  Tag 65332: 4,0,0,0,12,0,0,0,65,0,112,0,112,0,73,0,110,0,102,0,111,0,95,0,86,0,49,0,95,0,48,0,220,3,0,0,40,1,0,0,14,0,0,0,67,0,117,0,115,0,116,0,111,0,109,0,68,0,97,0,116,0,97,0,86,0,50,0,95,0,48,0,40,3,0,0,4,5,0,0,25,

(with many more numbers). Since they're unlabelled, we can only guess what these metadata are for and whether they're worth keeping. ImageMagick culls them, and ImageJ often throws out image metadata (especially when you're lazy like me and use duplicate with the rectangle instead of cropping images...) So it should be fine, but putting this out as a warning.

I added the following function to my bash profile:

tiff12bit_to_16bit () {
  convert "$1" -evaluate LeftShift 4 "${1%.*}.16.tif"
}

PS: I noticed the original file contains the tag Max Sample Value: 4095 -- it is possible this is what ImageJ uses to display the image correctly -- this tag is lost upon conversion it gets better: the tag apparently remains, unseen by: ImageMagick, tiffexif, piexif/PIL, IrfanView, Windows Explorer, and, hilariously enough, not shown by ImageJ. Opening the converted file in ImageJ shows an all white image: presumably, it still expects the max value to be 4095 and most values will be above that in a true 16-bit TIFF. However, this can be fixed by simply starting to adjust Brightness and Contrast -- moving the max value fixes the viewing window somehow. (I'm suspicious of hidden tags but this could still be a separate, unrelated issue)

* I tested this: the file sizes are nearly identical; in fact, the converted (corrected?) file is slightly smaller because of stripped metadata explained above.