-
-
Notifications
You must be signed in to change notification settings - Fork 855
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature: histogram equalization #644
Changes from 11 commits
7380a63
d41ece1
687db06
5b1b8b1
7890186
8117d82
1d5a8f7
f4d9786
7ecfa0c
e6ba776
a5b180a
dd698dd
f78d9b7
ac69aa7
3e08e30
5708144
b102652
c36b0df
4fcefc5
8ad5b34
0da5e68
2108bf1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
// Copyright (c) Six Labors and contributors. | ||
// Licensed under the Apache License, Version 2.0. | ||
|
||
using SixLabors.ImageSharp.PixelFormats; | ||
|
||
namespace SixLabors.ImageSharp.Processing.Normalization | ||
{ | ||
/// <summary> | ||
/// Adds extension that allows applying an HistogramEqualization to the image. | ||
/// </summary> | ||
public static class HistogramEqualizationExtension | ||
{ | ||
/// <summary> | ||
/// Equalizes the histogram of an image to increases the global contrast. | ||
/// </summary> | ||
/// <typeparam name="TPixel">The pixel format.</typeparam> | ||
/// <param name="source">The image this method extends.</param> | ||
/// <returns>A histogram equalized grayscale image.</returns> | ||
public static IImageProcessingContext<TPixel> HistogramEqualization<TPixel>(this IImageProcessingContext<TPixel> source) | ||
where TPixel : struct, IPixel<TPixel> | ||
=> source.ApplyProcessor(new HistogramEqualizationProcessor<TPixel>()); | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,120 @@ | ||
// Copyright (c) Six Labors and contributors. | ||
// Licensed under the Apache License, Version 2.0. | ||
|
||
using System; | ||
using SixLabors.ImageSharp.Advanced; | ||
using SixLabors.ImageSharp.PixelFormats; | ||
using SixLabors.ImageSharp.Processing.Processors; | ||
using SixLabors.Memory; | ||
using SixLabors.Primitives; | ||
|
||
namespace SixLabors.ImageSharp.Processing.Normalization | ||
{ | ||
/// <summary> | ||
/// Applies a global histogram equalization to the image. | ||
/// </summary> | ||
/// <typeparam name="TPixel">The pixel format.</typeparam> | ||
internal class HistogramEqualizationProcessor<TPixel> : ImageProcessor<TPixel> | ||
where TPixel : struct, IPixel<TPixel> | ||
{ | ||
/// <inheritdoc/> | ||
protected override void OnFrameApply(ImageFrame<TPixel> source, Rectangle sourceRectangle, Configuration configuration) | ||
{ | ||
var rgb48 = default(Rgb48); | ||
var rgb24 = default(Rgb24); | ||
MemoryAllocator memoryAllocator = configuration.MemoryAllocator; | ||
int numberOfPixels = source.Width * source.Height; | ||
bool is16bitPerChannel = typeof(TPixel) == typeof(Rgb48) || typeof(TPixel) == typeof(Rgba64); | ||
|
||
// build the histogram of the grayscale levels | ||
int luminanceLevels = is16bitPerChannel ? 65536 : 256; | ||
Span<int> histogram = memoryAllocator.Allocate<int>(luminanceLevels, clear: true).GetSpan(); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should be a called with a using statment using(IBuffer<int> buffer = memoryAllocator.AllocateClean<int>(luminanceLevels))
{
// wrap remaining code in here
} There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. i have changed that |
||
for (int y = 0; y < source.Height; y++) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this can be done with a single loop of var pixels = source.GetPixelSpan();
for (int i = 0; i < pixels.Length; i++){
int luminance = this.GetLuminance(in pixels[i], is16bitPerChannel, ref rgb24, ref rgb48);
histogram[luminance]++;
} There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ah nice, i did not know that |
||
{ | ||
Span<TPixel> row = source.GetPixelRowSpan(y); | ||
for (int x = 0; x < source.Width; x++) | ||
{ | ||
TPixel sourcePixel = row[x]; | ||
int luminance = this.GetLuminance(sourcePixel, is16bitPerChannel, ref rgb24, ref rgb48); | ||
histogram[luminance]++; | ||
} | ||
} | ||
|
||
// calculate the cumulative distribution function (which will be the cumulative histogram) | ||
Span<int> cdf = memoryAllocator.Allocate<int>(luminanceLevels, clear: true).GetSpan(); | ||
int histSum = 0; | ||
for (int i = 0; i < histogram.Length; i++) | ||
{ | ||
histSum += histogram[i]; | ||
cdf[i] = histSum; | ||
} | ||
|
||
// get the first none zero value of the cumulative histogram | ||
int cdfMin = 0; | ||
for (int i = 0; i < histogram.Length; i++) | ||
{ | ||
if (histogram[i] != 0) | ||
{ | ||
cdfMin = cdf[i]; | ||
break; | ||
} | ||
} | ||
|
||
// creating the lookup table: subtracting cdf min, so we do not need to do that inside the for loop | ||
for (int i = 0; i < histogram.Length; i++) | ||
{ | ||
cdf[i] = cdf[i] - cdfMin; | ||
} | ||
|
||
// apply the cdf to each pixel of the image | ||
double numberOfPixelsMinusCdfMin = (double)(numberOfPixels - cdfMin); | ||
int luminanceLevelsMinusOne = luminanceLevels - 1; | ||
for (int y = 0; y < source.Height; y++) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. as above, as this doesn't actually depend on the relative position of the pixel then this can be done is a single loop over all the pixels. |
||
{ | ||
Span<TPixel> row = source.GetPixelRowSpan(y); | ||
for (int x = 0; x < source.Width; x++) | ||
{ | ||
TPixel sourcePixel = row[x]; | ||
|
||
int luminance = this.GetLuminance(sourcePixel, is16bitPerChannel, ref rgb24, ref rgb48); | ||
double luminanceEqualized = (cdf[luminance] / numberOfPixelsMinusCdfMin) * luminanceLevelsMinusOne; | ||
luminanceEqualized = Math.Round(luminanceEqualized); | ||
|
||
if (is16bitPerChannel) | ||
{ | ||
row[x].PackFromRgb48(new Rgb48((ushort)luminanceEqualized, (ushort)luminanceEqualized, (ushort)luminanceEqualized)); | ||
} | ||
else | ||
{ | ||
row[x].PackFromRgba32(new Rgba32((byte)luminanceEqualized, (byte)luminanceEqualized, (byte)luminanceEqualized)); | ||
} | ||
} | ||
} | ||
} | ||
|
||
/// <summary> | ||
/// Convert the pixel values to grayscale using ITU-R Recommendation BT.709. | ||
/// </summary> | ||
/// <param name="sourcePixel">The pixel to get the luminance from</param> | ||
/// <param name="is16bitPerChannel">Flag indicates, if its 16 bits per channel, otherwise its 8</param> | ||
/// <param name="rgb24">Will store the pixel values in case of 8 bit per channel</param> | ||
/// <param name="rgb48">Will store the pixel values in case of 16 bit per channel</param> | ||
private int GetLuminance(TPixel sourcePixel, bool is16bitPerChannel, ref Rgb24 rgb24, ref Rgb48 rgb48) | ||
{ | ||
// Convert to grayscale using ITU-R Recommendation BT.709 | ||
int luminance; | ||
if (is16bitPerChannel) | ||
{ | ||
sourcePixel.ToRgb48(ref rgb48); | ||
luminance = Convert.ToInt32((.2126F * rgb48.R) + (.7152F * rgb48.G) + (.0722F * rgb48.B)); | ||
} | ||
else | ||
{ | ||
sourcePixel.ToRgb24(ref rgb24); | ||
luminance = Convert.ToInt32((.2126F * rgb24.R) + (.7152F * rgb24.G) + (.0722F * rgb24.B)); | ||
} | ||
|
||
return luminance; | ||
} | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
// Copyright (c) Six Labors and contributors. | ||
// Licensed under the Apache License, Version 2.0. | ||
|
||
using SixLabors.ImageSharp.PixelFormats; | ||
using SixLabors.ImageSharp.Processing; | ||
using SixLabors.ImageSharp.Processing.Normalization; | ||
using Xunit; | ||
|
||
namespace SixLabors.ImageSharp.Tests.Processing.Normalization | ||
{ | ||
public class HistogramEqualizationTests | ||
{ | ||
[Fact] | ||
public void HistogramEqualizationTest() | ||
{ | ||
// arrange | ||
byte[] pixels = new byte[] | ||
{ | ||
52, 55, 61, 59, 70, 61, 76, 61, | ||
62, 59, 55, 104, 94, 85, 59, 71, | ||
63, 65, 66, 113, 144, 104, 63, 72, | ||
64, 70, 70, 126, 154, 109, 71, 69, | ||
67, 73, 68, 106, 122, 88, 68, 68, | ||
68, 79, 60, 79, 77, 66, 58, 75, | ||
69, 85, 64, 58, 55, 61, 65, 83, | ||
70, 87, 69, 68, 65, 73, 78, 90 | ||
}; | ||
var image = new Image<Rgba32>(8, 8); | ||
for (int y = 0; y < 8; y++) | ||
{ | ||
for (int x = 0; x < 8; x++) | ||
{ | ||
byte luminance = pixels[y * 8 + x]; | ||
image[x, y] = new Rgba32(luminance, luminance, luminance); | ||
} | ||
} | ||
|
||
byte[] expected = new byte[] | ||
{ | ||
0, 12, 53, 32, 146, 53, 174, 53, | ||
57, 32, 12, 227, 219, 202, 32, 154, | ||
65, 85, 93, 239, 251, 227, 65, 158, | ||
73, 146, 146, 247, 255, 235, 154, 130, | ||
97, 166, 117, 231, 243, 210, 117, 117, | ||
117, 190, 36, 190, 178, 93, 20, 170, | ||
130, 202, 73, 20, 12, 53, 85, 194, | ||
146, 206, 130, 117, 85, 166, 182, 215 | ||
}; | ||
|
||
// act | ||
image.Mutate(x => x.HistogramEqualization()); | ||
|
||
// assert | ||
for (int y = 0; y < 8; y++) | ||
{ | ||
for (int x = 0; x < 8; x++) | ||
{ | ||
Rgba32 actual = image[x, y]; | ||
Assert.Equal(expected[y * 8 + x], actual.R); | ||
Assert.Equal(expected[y * 8 + x], actual.G); | ||
Assert.Equal(expected[y * 8 + x], actual.B); | ||
} | ||
} | ||
} | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we generally don't have ImageProcesses having any logic dependent on the
PixelType
do we really need this? do we even need to know that its a 16bit image? normally we would convert the pixel to a Vector4 and process the buffer with no need for branching.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i am using now vector4, but still i need to determine how many gray levels there are with the pixel type, because i need that to create the histogram.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking at the code the number of levels looks like its something that the user should be in control of. We should allow it to be passed in to the constructor of the processor but have a default value of 65536 this will give a good high quality default plus allow users to adjust as they desire to tweaking the output.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it is important for the algorithm to know how many possible grey levels there are. If the input image only has 256 possible grey levels, creating a histogram with 65536 would not change the quality of the outcome.
The algorithm just maps each input grey level to a new output grey level with the goal to use all possible grey values equally. For example the picture i have attached to this PR has only grey values of 120 to 200. After the equalization it uses the the complete range from 0 to 256, but the total number of different grey levels will not change after the equalization.
here is the histogram of the input image:
and after the equalization:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So it sounds like my suggestion will allow that and will work fine it will just over allocate on smaller pixel sizes but will allow users with higher quality pixel definitions to fully make use of the processor.
Ultimately we can't have processors that have logic dependent on the pixel type. All processors in the core of imagesharp have to have to work consistently no matter which pixel-type is used this includes custom user defined pixel types that might not even map to any sort of bit depth.... they could be using some high precision floating point value for each channel that doesn't map to any of those grayscale depths you are expecting thus we need to allow the user to influence the mapping.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, i understand now your reasoning why you do not want pixel type related code in the processor.
The luminance levels is now a parameter of the constructor as you suggested.