Skip to content

Commit

Permalink
fix(heic): Don't auto-transform camera-rotated images
Browse files Browse the repository at this point in the history
It seems that unlike most image formats where we merely note the
camera rotation but don't rotate the pixels, the heic reader
automatically rotated.

The trick is that using the C++ API wrapper of libheif, there is no
way to pass the decoder options struct that is where you need to say
not to auto-transform.  Needed to drop down to the lower level C API
for this one spot.

We want this format to behave like the others -- the orientation is
advisory, and it's up to the app how to deal with it. But we have back
compatibility to deal with.

If a rotated image is ancountered, OIIO 2.5 and earlier will by
default still auto-rotate to preserve compatibility, but OIIO 2.6+
will not auto-rotate.

When not rotating, the Orientation metadata will reflect the desired
display orientation of the image, according to what was in the file.

When auto-rotating, Orientation will be 1 (canonical display
orientation), and a new attribute "heif:Orientation" will reflect what
was originally in the file.

When opening a file for input, the special configuration metadata hint
"heif:reorient" can express a preference for autorotation, overriding
the default. Thus, setting this hint to 0 for OIIO 2.5 will turn off
auto-rotation, and setting it to 1 for OIIO 2.6+ will make it
auto-rotate like it used to do.

I also added some additional tests for heif files.

Fixes 4123

Signed-off-by: Larry Gritz <[email protected]>
  • Loading branch information
lgritz committed Feb 6, 2024
1 parent c6d4138 commit a4e1e48
Show file tree
Hide file tree
Showing 9 changed files with 526 additions and 31 deletions.
6 changes: 4 additions & 2 deletions src/cmake/testing.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -259,8 +259,10 @@ macro (oiio_add_all_tests)
IMAGEDIR openexr-images
URL http://github.com/AcademySoftwareFoundation/openexr-images)
oiio_add_tests (heif
FOUNDVAR Libheif_FOUND ENABLEVAR ENABLE_Libheif
URL https://github.com/nokiatech/heif/tree/gh-pages/content)
FOUNDVAR Libheif_FOUND
ENABLEVAR ENABLE_Libheif
IMAGEDIR oiio-images/heif
URL http://github.com/AcademySoftwareFoundation/openexr-images)
oiio_add_tests (ico
ENABLEVAR ENABLE_ICO
IMAGEDIR oiio-images URL "Recent checkout of oiio-images")
Expand Down
11 changes: 11 additions & 0 deletions src/doc/builtinplugins.rst
Original file line number Diff line number Diff line change
Expand Up @@ -685,6 +685,11 @@ preferred except when legacy file access is required.
* - ``oiio:Gamma``
- float
- the gamma correction specified in the RGBE header (if it's gamma corrected).
* - ``heif:Orientation``
- int
- If the configuration option ``heif:reorient`` is nonzero and
reorientation was performed, this will be set to the original
orientation in the file.


**Configuration settings for HDR input**
Expand Down Expand Up @@ -769,6 +774,12 @@ attributes are supported:
cause the reader to leave alpha unassociated (versus the default of
premultiplying color channels by alpha if the alpha channel is
unassociated).
* - ``heif:reorient``
- int
- If nonzero, asks libheif to reorient any images (and report them as
having Orientation 1). If zero, then libheif will not reorient the
image and the Orientation metadata will be set to reflect the camera
orientation.

**Configuration settings for HEIF output**

Expand Down
26 changes: 13 additions & 13 deletions src/doc/stdmetadata.rst
Original file line number Diff line number Diff line change
Expand Up @@ -77,21 +77,21 @@ Display hints

.. option:: "Orientation" : int

y default, image pixels are ordered from the top of the display to the
ottom, and within each scanline, from left to right (i.e., the same
rdering as English text and scan progression on a CRT). But the
"Orientation"` field can suggest that it should be displayed with
different orientation, according to the TIFF/EXIF conventions:
By default, image pixels are ordered from the top of the display to the
bottom, and within each scanline, from left to right (i.e., the same
ordering as English text and scan progression on a CRT). But the
`"Orientation"` field can suggest that it should be displayed with
a different orientation, according to the TIFF/EXIF conventions:

=== ==========================================================================
0 normal (top to bottom, left to right)
1 flipped horizontally (top to botom, right to left)
2 rotated :math:`180^\circ` (bottom to top, right to left)
3 flipped vertically (bottom to top, left to right)
4 transposed (left to right, top to bottom)
5 rotated :math:`90^\circ` clockwise (right to left, top to bottom)
6 transverse (right to left, bottom to top)
7 rotated :math:`90^\circ` counter-clockwise (left to right, bottom to top)
1 normal (top to bottom, left to right)
2 flipped horizontally (top to bottom, right to left)
3 rotated :math:`180^\circ` (bottom to top, right to left)
4 flipped vertically (bottom to top, left to right)
5 transposed (left to right, top to bottom)
6 rotated :math:`90^\circ` clockwise (right to left, top to bottom)
7 transverse (right to left, bottom to top)
8 rotated :math:`90^\circ` counter-clockwise (left to right, bottom to top)
=== ==========================================================================

.. option:: "PixelAspectRatio" : float
Expand Down
76 changes: 60 additions & 16 deletions src/heif.imageio/heifinput.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,15 @@
// https://github.com/nokiatech/heif/tree/gh-pages/content


// OIIO < 2.6 defaults to letting libheif reorient images automatically.
#if OIIO_VERSION < OIIO_MAKE_VERSION(2, 6, 0)
# define REORIENT_DEFAULT true
#else
# define REORIENT_DEFAULT false
#endif



OIIO_PLUGIN_NAMESPACE_BEGIN

class HeifInput final : public ImageInput {
Expand Down Expand Up @@ -50,6 +59,7 @@ class HeifInput final : public ImageInput {
bool m_associated_alpha = true;
bool m_keep_unassociated_alpha = false;
bool m_do_associate = false;
bool m_reorient = REORIENT_DEFAULT;
std::unique_ptr<heif::Context> m_ctx;
heif_item_id m_primary_id; // id of primary image
std::vector<heif_item_id> m_item_ids; // ids of all other images
Expand Down Expand Up @@ -137,6 +147,7 @@ HeifInput::open(const std::string& name, ImageSpec& newspec,

m_keep_unassociated_alpha
= (config.get_int_attribute("oiio:UnassociatedAlpha") != 0);
m_reorient = config.get_int_attribute("heif:reorient", REORIENT_DEFAULT);

try {
m_ctx->read_from_file(name);
Expand Down Expand Up @@ -200,14 +211,14 @@ HeifInput::seek_subimage(int subimage, int miplevel)
return false;
}

auto id = (subimage == 0) ? m_primary_id : m_item_ids[subimage - 1];
m_ihandle = m_ctx->get_image_handle(id);
m_has_alpha = m_ihandle.has_alpha_channel();
auto chroma = m_has_alpha ? heif_chroma_interleaved_RGBA
: heif_chroma_interleaved_RGB;
#if 0
try {
auto id = (subimage == 0) ? m_primary_id : m_item_ids[subimage - 1];
m_ihandle = m_ctx->get_image_handle(id);
m_has_alpha = m_ihandle.has_alpha_channel();
auto chroma = m_has_alpha ? heif_chroma_interleaved_RGBA
: heif_chroma_interleaved_RGB;
m_himage = m_ihandle.decode_image(heif_colorspace_RGB, chroma);

m_himage = m_ihandle.decode_image(heif_colorspace_RGB, chroma);
} catch (const heif::Error& err) {
std::string e = err.get_message();
errorf("%s", e.empty() ? "unknown exception" : e.c_str());
Expand All @@ -217,6 +228,23 @@ HeifInput::seek_subimage(int subimage, int miplevel)
errorf("%s", e.empty() ? "unknown exception" : e.c_str());
return false;
}
#else
std::unique_ptr<heif_decoding_options, void (*)(heif_decoding_options*)>
options(heif_decoding_options_alloc(), heif_decoding_options_free);
options->ignore_transformations = !m_reorient;
// print("Got decoding options version {}\n", options->version);
struct heif_image* img_tmp = nullptr;
struct heif_error herr = heif_decode_image(m_ihandle.get_raw_image_handle(),
&img_tmp, heif_colorspace_RGB,
chroma, options.get());
if (img_tmp)
m_himage = heif::Image(img_tmp);
if (herr.code != heif_error_Ok || !img_tmp) {
errorfmt("Could not decode image ({})", herr.message);
m_ctx.reset();
return false;
}
#endif

int bits = m_himage.get_bits_per_pixel(heif_channel_interleaved);
m_spec = ImageSpec(m_ihandle.get_width(), m_ihandle.get_height(), bits / 8,
Expand Down Expand Up @@ -270,21 +298,37 @@ HeifInput::seek_subimage(int subimage, int miplevel)
decode_xmp(metacontents, m_spec);
} else {
#ifdef DEBUG
std::cout << "Don't know how to decode meta " << m
<< " type=" << m_ihandle.get_metadata_type(m)
<< " contenttype='"
<< m_ihandle.get_metadata_content_type(m) << "'\n";
std::cout << "---\n"
<< string_view((const char*)&metacontents[0],
metacontents.size())
<< "\n---\n";
print(
"Don't know how to decode meta {} type='{}' contenttype='{}'\n",
m, m_ihandle.get_metadata_type(m),
m_ihandle.get_metadata_content_type(m));
print("---\n{}\n---\n",
string_view((const char*)metacontents.data(),
metacontents.size()));
#endif
}
}

// Erase the orientation metadata because libheif appears to be doing
// the rotation-to-canonical-direction for us.
m_spec.erase_attribute("Orientation");
int orientation = m_spec.get_int_attribute("Orientation", 1);
if (orientation != 1) {
if (m_reorient) {
// If libheif auto-reoriented, record the original orientation in
// "hei:Orientation" and erase the "Orientation" attribute since we're
// presenting the image to the caller in the usual orientation.
m_spec.attribute("heif:Orientation", orientation);
m_spec.erase_attribute("Orientation");
} else {
// libheif supplies oriented width & height, so if we are NOT
// auto-reorienting and it's one of the orientations that swaps
// width and height, we need to do that swap ourselves.
if (orientation == 5 || orientation == 6 || orientation == 8) {
std::swap(m_spec.width, m_spec.height);
std::swap(m_spec.full_width, m_spec.full_height);
}
}
}

m_subimage = subimage;
return true;
Expand Down
111 changes: 111 additions & 0 deletions testsuite/heif/ref/out-libheif1.4.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ ref/IMG_7702_small.heic : 512 x 300, 3 channel, uint8 heif
FNumber: 1.8
Make: "Apple"
Model: "iPhone 7"
Orientation: 1 (normal)
ResolutionUnit: 2 (inches)
Software: "12.1.2"
XResolution: 72
Expand Down Expand Up @@ -38,3 +39,113 @@ ref/IMG_7702_small.heic : 512 x 300, 3 channel, uint8 heif
Exif:SubsecTimeOriginal: "006"
Exif:WhiteBalance: 0 (auto)
oiio:ColorSpace: "sRGB"
Reading ref/Chimera-AV1-8bit-162.avif
ref/Chimera-AV1-8bit-162.avif : 480 x 270, 3 channel, uint8 heif
SHA-1: F8FDAF1BD56A21E3AF99CF8EE7FA45434D2826C7
channel list: R, G, B
oiio:ColorSpace: "sRGB"
Reading ../oiio-images/heif/greyhounds-looking-for-a-table.heic
../oiio-images/heif/greyhounds-looking-for-a-table.heic : 4032 x 3024, 3 channel, uint8 heif
SHA-1: CDC56B959BAF5567376CDBD6A74AA992C7564C81
channel list: R, G, B
DateTime: "2023:09:28 09:44:03"
ExposureTime: 0.0135135
FNumber: 2.4
Make: "Apple"
Model: "iPhone 12 Pro"
Orientation: 8 (rotated 90 deg CCW)
ResolutionUnit: 2 (inches)
Software: "16.7"
XResolution: 72
YResolution: 72
Exif:ApertureValue: 2.52607 (f/2.4)
Exif:BrightnessValue: 2.7506
Exif:ColorSpace: 65535
Exif:DateTimeDigitized: "2023:09:28 09:44:03"
Exif:DateTimeOriginal: "2023:09:28 09:44:03"
Exif:DigitalZoomRatio: 1.3057
Exif:ExifVersion: "0232"
Exif:ExposureBiasValue: 0
Exif:ExposureMode: 0 (auto)
Exif:ExposureProgram: 2 (normal program)
Exif:Flash: 16 (no flash, flash suppression)
Exif:FocalLength: 1.54 (1.54 mm)
Exif:FocalLengthIn35mmFilm: 17
Exif:LensMake: "Apple"
Exif:LensModel: "iPhone 12 Pro back triple camera 1.54mm f/2.4"
Exif:LensSpecification: 1.54, 6, 1.6, 2.4
Exif:MeteringMode: 5 (pattern)
Exif:OffsetTime: "+02:00"
Exif:OffsetTimeDigitized: "+02:00"
Exif:OffsetTimeOriginal: "+02:00"
Exif:PhotographicSensitivity: 320
Exif:PixelXDimension: 4032
Exif:PixelYDimension: 3024
Exif:SensingMethod: 2 (1-chip color area)
Exif:ShutterSpeedValue: 6.20983 (1/74 s)
Exif:SubsecTimeDigitized: "886"
Exif:SubsecTimeOriginal: "886"
Exif:WhiteBalance: 0 (auto)
GPS:Altitude: 3.24105 (3.24105 m)
GPS:AltitudeRef: 0 (above sea level)
GPS:DateStamp: "2023:09:28"
GPS:DestBearing: 90.2729
GPS:DestBearingRef: "T" (true north)
GPS:HPositioningError: 5.1893
GPS:ImgDirection: 90.2729
GPS:ImgDirectionRef: "T" (true north)
GPS:Latitude: 41, 50, 58.43
GPS:LatitudeRef: "N"
GPS:Longitude: 3, 7, 31.98
GPS:LongitudeRef: "E"
GPS:Speed: 0.171966
GPS:SpeedRef: "K" (km/hour)
oiio:ColorSpace: "sRGB"
Reading ../oiio-images/heif/sewing-threads.heic
../oiio-images/heif/sewing-threads.heic : 4000 x 3000, 3 channel, uint8 heif
SHA-1: 6A061BFE2F0BAC4CC94F5D9D5A6E674634149813
channel list: R, G, B
DateTime: "2023:12:12 18:39:16"
ExposureTime: 0.04
FNumber: 1.8
Make: "samsung"
Model: "SM-A326B"
Orientation: 1 (normal)
ResolutionUnit: 2 (inches)
Software: "A326BXXS8CWK2"
XResolution: 72
YResolution: 72
Exif:ApertureValue: 1.69 (f/1.8)
Exif:BrightnessValue: 1.19
Exif:ColorSpace: 1
Exif:DateTimeDigitized: "2023:12:12 18:39:16"
Exif:DateTimeOriginal: "2023:12:12 18:39:16"
Exif:DigitalZoomRatio: 1
Exif:ExifVersion: "0220"
Exif:ExposureBiasValue: 0
Exif:ExposureMode: 0 (auto)
Exif:ExposureProgram: 2 (normal program)
Exif:Flash: 0 (no flash)
Exif:FocalLength: 4.6 (4.6 mm)
Exif:FocalLengthIn35mmFilm: 25
Exif:MaxApertureValue: 1.69 (f/1.8)
Exif:MeteringMode: 2 (center-weighted average)
Exif:OffsetTime: "+01:00"
Exif:OffsetTimeOriginal: "+01:00"
Exif:PhotographicSensitivity: 500
Exif:PixelXDimension: 4000
Exif:PixelYDimension: 3000
Exif:SceneCaptureType: 0 (standard)
Exif:ShutterSpeedValue: 0.04 (1/1 s)
Exif:SubsecTime: "576"
Exif:SubsecTimeDigitized: "576"
Exif:SubsecTimeOriginal: "576"
Exif:WhiteBalance: 0 (auto)
Exif:YCbCrPositioning: 1
GPS:Altitude: 292 (292 m)
GPS:AltitudeRef: 0 (above sea level)
GPS:Latitude: 41, 43, 33.821
GPS:LatitudeRef: "N"
GPS:Longitude: 1, 49, 34.0187
GPS:LongitudeRef: "E"
oiio:ColorSpace: "sRGB"
Loading

0 comments on commit a4e1e48

Please sign in to comment.