Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: histogram equalization #644

Merged
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
7380a63
Merge pull request #1 from SixLabors/master
brianpopow Jun 12, 2018
d41ece1
added first attempt of histogram equalization
brianpopow Jun 30, 2018
687db06
added HistogramEqualizationTest
brianpopow Jun 30, 2018
5b1b8b1
changed the cdf to be the cumulative histogram
brianpopow Jul 1, 2018
7890186
Merge remote-tracking branch 'upstream/master' into feature/histogram…
brianpopow Jul 1, 2018
8117d82
added support for 16 bit greyscale
brianpopow Jul 1, 2018
1d5a8f7
SixLabors.ImageSharp.Processing.Contrast -> SixLabors.ImageSharp.Proc…
brianpopow Jul 2, 2018
f4d9786
using memoryAllocator
brianpopow Jul 2, 2018
7ecfa0c
removed unnecessary allocation of the lut, using cdf instead
brianpopow Jul 2, 2018
e6ba776
moved test to Normalization folder
brianpopow Jul 2, 2018
a5b180a
fixed rounding issue in calculating the luminance
brianpopow Jul 2, 2018
dd698dd
using GetPixelSpan instead of GetPixelRowSpan
brianpopow Jul 3, 2018
f78d9b7
allocating cdf and histogram buffer with a using statement
brianpopow Jul 3, 2018
ac69aa7
using Vector4 to calculate the luminance and set the pixel value
brianpopow Jul 3, 2018
3e08e30
Merge branch 'master' into feature/histogramEqualization
brianpopow Jul 3, 2018
5708144
luminance levels is now a parameter of the constructor, defaults to 6…
brianpopow Jul 4, 2018
b102652
Merge branch 'master' into feature/histogramEqualization
brianpopow Jul 4, 2018
c36b0df
Merge branch 'master' into feature/histogramEqualization
JimBobSquarePants Jul 9, 2018
4fcefc5
moved extension to the processing namespace and the processor accordi…
brianpopow Jul 15, 2018
8ad5b34
Merge branch 'master' into feature/histogramEqualization
brianpopow Jul 15, 2018
0da5e68
Cleanup and remove double cast.
JimBobSquarePants Jul 16, 2018
2108bf1
Merge remote-tracking branch 'upstream/master' into feature/histogram…
JimBobSquarePants Jul 16, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
// Copyright (c) Six Labors and contributors.
// Licensed under the Apache License, Version 2.0.

using SixLabors.ImageSharp.PixelFormats;

namespace SixLabors.ImageSharp.Processing.Normalization
{
/// <summary>
/// Adds extension that allows applying an HistogramEqualization to the image.
/// </summary>
public static class HistogramEqualizationExtension
{
/// <summary>
/// Equalizes the histogram of an image to increases the global contrast.
/// </summary>
/// <typeparam name="TPixel">The pixel format.</typeparam>
/// <param name="source">The image this method extends.</param>
/// <returns>A histogram equalized grayscale image.</returns>
public static IImageProcessingContext<TPixel> HistogramEqualization<TPixel>(this IImageProcessingContext<TPixel> source)
where TPixel : struct, IPixel<TPixel>
=> source.ApplyProcessor(new HistogramEqualizationProcessor<TPixel>());
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
// Copyright (c) Six Labors and contributors.
// Licensed under the Apache License, Version 2.0.

using System;
using SixLabors.ImageSharp.Advanced;
using SixLabors.ImageSharp.PixelFormats;
using SixLabors.ImageSharp.Processing.Processors;
using SixLabors.Memory;
using SixLabors.Primitives;

namespace SixLabors.ImageSharp.Processing.Normalization
{
/// <summary>
/// Applies a global histogram equalization to the image.
/// </summary>
/// <typeparam name="TPixel">The pixel format.</typeparam>
internal class HistogramEqualizationProcessor<TPixel> : ImageProcessor<TPixel>
where TPixel : struct, IPixel<TPixel>
{
/// <inheritdoc/>
protected override void OnFrameApply(ImageFrame<TPixel> source, Rectangle sourceRectangle, Configuration configuration)
{
var rgb48 = default(Rgb48);
var rgb24 = default(Rgb24);
MemoryAllocator memoryAllocator = configuration.MemoryAllocator;
int numberOfPixels = source.Width * source.Height;
bool is16bitPerChannel = typeof(TPixel) == typeof(Rgb48) || typeof(TPixel) == typeof(Rgba64);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we generally don't have ImageProcesses having any logic dependent on the PixelType do we really need this? do we even need to know that its a 16bit image? normally we would convert the pixel to a Vector4 and process the buffer with no need for branching.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i am using now vector4, but still i need to determine how many gray levels there are with the pixel type, because i need that to create the histogram.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the code the number of levels looks like its something that the user should be in control of. We should allow it to be passed in to the constructor of the processor but have a default value of 65536 this will give a good high quality default plus allow users to adjust as they desire to tweaking the output.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is important for the algorithm to know how many possible grey levels there are. If the input image only has 256 possible grey levels, creating a histogram with 65536 would not change the quality of the outcome.
The algorithm just maps each input grey level to a new output grey level with the goal to use all possible grey values equally. For example the picture i have attached to this PR has only grey values of 120 to 200. After the equalization it uses the the complete range from 0 to 256, but the total number of different grey levels will not change after the equalization.

here is the histogram of the input image:
before_eq

and after the equalization:
after_eq

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it sounds like my suggestion will allow that and will work fine it will just over allocate on smaller pixel sizes but will allow users with higher quality pixel definitions to fully make use of the processor.

Ultimately we can't have processors that have logic dependent on the pixel type. All processors in the core of imagesharp have to have to work consistently no matter which pixel-type is used this includes custom user defined pixel types that might not even map to any sort of bit depth.... they could be using some high precision floating point value for each channel that doesn't map to any of those grayscale depths you are expecting thus we need to allow the user to influence the mapping.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, i understand now your reasoning why you do not want pixel type related code in the processor.

The luminance levels is now a parameter of the constructor as you suggested.


// build the histogram of the grayscale levels
int luminanceLevels = is16bitPerChannel ? 65536 : 256;
Span<int> histogram = memoryAllocator.Allocate<int>(luminanceLevels, clear: true).GetSpan();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be a called with a using statment

using(IBuffer<int> buffer = memoryAllocator.AllocateClean<int>(luminanceLevels))
{
// wrap remaining code in here
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i have changed that

for (int y = 0; y < source.Height; y++)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be done with a single loop of

var pixels = source.GetPixelSpan();
for (int i = 0; i < pixels.Length; i++){
    int luminance = this.GetLuminance(in pixels[i], is16bitPerChannel, ref rgb24, ref rgb48);
    histogram[luminance]++;
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah nice, i did not know that

{
Span<TPixel> row = source.GetPixelRowSpan(y);
for (int x = 0; x < source.Width; x++)
{
TPixel sourcePixel = row[x];
int luminance = this.GetLuminance(sourcePixel, is16bitPerChannel, ref rgb24, ref rgb48);
histogram[luminance]++;
}
}

// calculate the cumulative distribution function (which will be the cumulative histogram)
Span<int> cdf = memoryAllocator.Allocate<int>(luminanceLevels, clear: true).GetSpan();
int histSum = 0;
for (int i = 0; i < histogram.Length; i++)
{
histSum += histogram[i];
cdf[i] = histSum;
}

// get the first none zero value of the cumulative histogram
int cdfMin = 0;
for (int i = 0; i < histogram.Length; i++)
{
if (histogram[i] != 0)
{
cdfMin = cdf[i];
break;
}
}

// creating the lookup table: subtracting cdf min, so we do not need to do that inside the for loop
for (int i = 0; i < histogram.Length; i++)
{
cdf[i] = cdf[i] - cdfMin;
}

// apply the cdf to each pixel of the image
double numberOfPixelsMinusCdfMin = (double)(numberOfPixels - cdfMin);
int luminanceLevelsMinusOne = luminanceLevels - 1;
for (int y = 0; y < source.Height; y++)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as above, as this doesn't actually depend on the relative position of the pixel then this can be done is a single loop over all the pixels.

{
Span<TPixel> row = source.GetPixelRowSpan(y);
for (int x = 0; x < source.Width; x++)
{
TPixel sourcePixel = row[x];

int luminance = this.GetLuminance(sourcePixel, is16bitPerChannel, ref rgb24, ref rgb48);
double luminanceEqualized = (cdf[luminance] / numberOfPixelsMinusCdfMin) * luminanceLevelsMinusOne;
luminanceEqualized = Math.Round(luminanceEqualized);

if (is16bitPerChannel)
{
row[x].PackFromRgb48(new Rgb48((ushort)luminanceEqualized, (ushort)luminanceEqualized, (ushort)luminanceEqualized));
}
else
{
row[x].PackFromRgba32(new Rgba32((byte)luminanceEqualized, (byte)luminanceEqualized, (byte)luminanceEqualized));
}
}
}
}

/// <summary>
/// Convert the pixel values to grayscale using ITU-R Recommendation BT.709.
/// </summary>
/// <param name="sourcePixel">The pixel to get the luminance from</param>
/// <param name="is16bitPerChannel">Flag indicates, if its 16 bits per channel, otherwise its 8</param>
/// <param name="rgb24">Will store the pixel values in case of 8 bit per channel</param>
/// <param name="rgb48">Will store the pixel values in case of 16 bit per channel</param>
private int GetLuminance(TPixel sourcePixel, bool is16bitPerChannel, ref Rgb24 rgb24, ref Rgb48 rgb48)
{
// Convert to grayscale using ITU-R Recommendation BT.709
int luminance;
if (is16bitPerChannel)
{
sourcePixel.ToRgb48(ref rgb48);
luminance = Convert.ToInt32((.2126F * rgb48.R) + (.7152F * rgb48.G) + (.0722F * rgb48.B));
}
else
{
sourcePixel.ToRgb24(ref rgb24);
luminance = Convert.ToInt32((.2126F * rgb24.R) + (.7152F * rgb24.G) + (.0722F * rgb24.B));
}

return luminance;
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
// Copyright (c) Six Labors and contributors.
// Licensed under the Apache License, Version 2.0.

using SixLabors.ImageSharp.PixelFormats;
using SixLabors.ImageSharp.Processing;
using SixLabors.ImageSharp.Processing.Normalization;
using Xunit;

namespace SixLabors.ImageSharp.Tests.Processing.Normalization
{
public class HistogramEqualizationTests
{
[Fact]
public void HistogramEqualizationTest()
{
// arrange
byte[] pixels = new byte[]
{
52, 55, 61, 59, 70, 61, 76, 61,
62, 59, 55, 104, 94, 85, 59, 71,
63, 65, 66, 113, 144, 104, 63, 72,
64, 70, 70, 126, 154, 109, 71, 69,
67, 73, 68, 106, 122, 88, 68, 68,
68, 79, 60, 79, 77, 66, 58, 75,
69, 85, 64, 58, 55, 61, 65, 83,
70, 87, 69, 68, 65, 73, 78, 90
};
var image = new Image<Rgba32>(8, 8);
for (int y = 0; y < 8; y++)
{
for (int x = 0; x < 8; x++)
{
byte luminance = pixels[y * 8 + x];
image[x, y] = new Rgba32(luminance, luminance, luminance);
}
}

byte[] expected = new byte[]
{
0, 12, 53, 32, 146, 53, 174, 53,
57, 32, 12, 227, 219, 202, 32, 154,
65, 85, 93, 239, 251, 227, 65, 158,
73, 146, 146, 247, 255, 235, 154, 130,
97, 166, 117, 231, 243, 210, 117, 117,
117, 190, 36, 190, 178, 93, 20, 170,
130, 202, 73, 20, 12, 53, 85, 194,
146, 206, 130, 117, 85, 166, 182, 215
};

// act
image.Mutate(x => x.HistogramEqualization());

// assert
for (int y = 0; y < 8; y++)
{
for (int x = 0; x < 8; x++)
{
Rgba32 actual = image[x, y];
Assert.Equal(expected[y * 8 + x], actual.R);
Assert.Equal(expected[y * 8 + x], actual.G);
Assert.Equal(expected[y * 8 + x], actual.B);
}
}
}
}
}