Feature: histogram equalization #644

brianpopow · 2018-07-02T08:38:59Z

Prerequisites

I have written a descriptive pull-request title
I have verified that there are no overlapping pull-requests open
I have verified that I am following matches the existing coding patterns and practice as demonstrated in the repository. These follow strict Stylecop rules 👮.
I have provided test coverage for my change (where applicable)

Description

This PR adds a HistogramEqualization Processor for enhancing the global contrast of an image.

Example image:

After Equalization:

Sync with imagesharp master

…Equalization

JimBobSquarePants

Nice idea, need a little refinement though. 👍

I wonder if we can incorporate some of the work from this early fork of ImageSharp to build more normalization/equalization methods?

https://github.com/JaCraig/Structure.Sketching/tree/master/Structure.Sketching/Filters/Normalization

JimBobSquarePants · 2018-07-02T12:53:25Z

src/ImageSharp/Processing/Contrast/HistogramEqualizationProcessor.cs

+
+            // build the histogram of the grayscale levels
+            int luminanceLevels = is16bitPerChannel ? 65536 : 256;
+            int[] histogram = new int[luminanceLevels];


Big allocation here. Use the MemoryAllocator.

im using MemoryAllocator now, please check if im using it correctly.

JimBobSquarePants · 2018-07-02T12:54:56Z

src/ImageSharp/Processing/Contrast/HistogramEqualizationProcessor.cs

+            }
+
+            // calculate the cumulative distribution function (which will be the cumulative histogram)
+            int[] cdf = new int[luminanceLevels];


JimBobSquarePants · 2018-07-02T12:55:11Z

src/ImageSharp/Processing/Contrast/HistogramEqualizationProcessor.cs

+                }
+            }
+
+            int[] lut = new int[luminanceLevels];


removed this allocation, its not necessary, using the cdf array instead

JimBobSquarePants · 2018-07-02T12:56:27Z

src/ImageSharp/Processing/Contrast/HistogramEqualizationProcessor.cs

+                {
+                    TPixel sourcePixel = row[x];
+
+                    int luminance = this.GetLuminance(sourcePixel, is16bitPerChannel, ref rgb24, ref rgb48);


We appear to be getting the luminance twice?

yes and i see no good way around this.

First i need to access all pixels to calculate the histogram.
After that i need the luminance of each pixel again to remap it to the new value.

JimBobSquarePants · 2018-07-02T13:03:11Z

src/ImageSharp/Processing/Contrast/HistogramEqualizationExtension.cs

+
+using SixLabors.ImageSharp.PixelFormats;
+
+namespace SixLabors.ImageSharp.Processing.Contrast


SixLabors.ImageSharp.Processing.Normalization?

i have changed that

…essing.Normalization

tocsoft

a few things to note

tocsoft · 2018-07-02T17:14:18Z

src/ImageSharp/Processing/Normalization/HistogramEqualizationProcessor.cs

+            var rgb24 = default(Rgb24);
+            MemoryAllocator memoryAllocator = configuration.MemoryAllocator;
+            int numberOfPixels = source.Width * source.Height;
+            bool is16bitPerChannel = typeof(TPixel) == typeof(Rgb48) || typeof(TPixel) == typeof(Rgba64);


we generally don't have ImageProcesses having any logic dependent on the PixelType do we really need this? do we even need to know that its a 16bit image? normally we would convert the pixel to a Vector4 and process the buffer with no need for branching.

i am using now vector4, but still i need to determine how many gray levels there are with the pixel type, because i need that to create the histogram.

Looking at the code the number of levels looks like its something that the user should be in control of. We should allow it to be passed in to the constructor of the processor but have a default value of 65536 this will give a good high quality default plus allow users to adjust as they desire to tweaking the output.

it is important for the algorithm to know how many possible grey levels there are. If the input image only has 256 possible grey levels, creating a histogram with 65536 would not change the quality of the outcome.
The algorithm just maps each input grey level to a new output grey level with the goal to use all possible grey values equally. For example the picture i have attached to this PR has only grey values of 120 to 200. After the equalization it uses the the complete range from 0 to 256, but the total number of different grey levels will not change after the equalization.

here is the histogram of the input image:

and after the equalization:

So it sounds like my suggestion will allow that and will work fine it will just over allocate on smaller pixel sizes but will allow users with higher quality pixel definitions to fully make use of the processor.

Ultimately we can't have processors that have logic dependent on the pixel type. All processors in the core of imagesharp have to have to work consistently no matter which pixel-type is used this includes custom user defined pixel types that might not even map to any sort of bit depth.... they could be using some high precision floating point value for each channel that doesn't map to any of those grayscale depths you are expecting thus we need to allow the user to influence the mapping.

ok, i understand now your reasoning why you do not want pixel type related code in the processor.

The luminance levels is now a parameter of the constructor as you suggested.

tocsoft · 2018-07-02T17:16:50Z

src/ImageSharp/Processing/Normalization/HistogramEqualizationProcessor.cs

+            // build the histogram of the grayscale levels
+            int luminanceLevels = is16bitPerChannel ? 65536 : 256;
+            Span<int> histogram = memoryAllocator.Allocate<int>(luminanceLevels, clear: true).GetSpan();
+            for (int y = 0; y < source.Height; y++)


this can be done with a single loop of

var pixels = source.GetPixelSpan(); for (int i = 0; i < pixels.Length; i++){ int luminance = this.GetLuminance(in pixels[i], is16bitPerChannel, ref rgb24, ref rgb48); histogram[luminance]++; }

ah nice, i did not know that

tocsoft · 2018-07-02T17:18:57Z

src/ImageSharp/Processing/Normalization/HistogramEqualizationProcessor.cs

+
+            // build the histogram of the grayscale levels
+            int luminanceLevels = is16bitPerChannel ? 65536 : 256;
+            Span<int> histogram = memoryAllocator.Allocate<int>(luminanceLevels, clear: true).GetSpan();


should be a called with a using statment

using(IBuffer<int> buffer = memoryAllocator.AllocateClean<int>(luminanceLevels)) { // wrap remaining code in here }

i have changed that

tocsoft · 2018-07-02T17:24:38Z

src/ImageSharp/Processing/Normalization/HistogramEqualizationProcessor.cs

+            // apply the cdf to each pixel of the image
+            double numberOfPixelsMinusCdfMin = (double)(numberOfPixels - cdfMin);
+            int luminanceLevelsMinusOne = luminanceLevels - 1;
+            for (int y = 0; y < source.Height; y++)


as above, as this doesn't actually depend on the relative position of the pixel then this can be done is a single loop over all the pixels.

codecov · 2018-07-02T17:57:56Z

Codecov Report

Merging #644 into master will increase coverage by 0.01%.
The diff coverage is 98.7%.

@@            Coverage Diff             @@
##           master     #644      +/-   ##
==========================================
+ Coverage   89.39%   89.41%   +0.01%     
==========================================
  Files         896      899       +3     
  Lines       38337    38414      +77     
  Branches     2668     2678      +10     
==========================================
+ Hits        34272    34348      +76     
- Misses       3264     3265       +1     
  Partials      801      801

Impacted Files	Coverage Δ
...rs/Normalization/HistogramEqualizationProcessor.cs	`100% <100%> (ø)`
...essing/Normalization/HistogramEqualizationTests.cs	`100% <100%> (ø)`
...Sharp/Processing/HistogramEqualizationExtension.cs	`50% <50%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d53547a...2108bf1. Read the comment docs.

…5536

…ngly into Processors namespace

…Equalization

brianpopow · 2018-07-19T19:00:21Z

@JimBobSquarePants would think an adaptive histogram equalization could be a useful feature for imagesharp? I think it can produce much better results then the global histogram equalization in some cases.

I have hacked together a first rough version in in ImageSharp. Here is an example result where is works pretty good:

(note: click on the image to see it in full size, the preview does not look correct)

Let me know, if you think that this is something you would like to have for imagesharp, then i would look further into it.

JimBobSquarePants · 2018-07-20T03:04:48Z

@brianpopow Definitely! I'd love to get a common set of histogram based operations put together.

brianpopow and others added 6 commits June 12, 2018 20:05

Merge pull request #1 from SixLabors/master

7380a63

Sync with imagesharp master

added first attempt of histogram equalization

d41ece1

added HistogramEqualizationTest

687db06

changed the cdf to be the cumulative histogram

5b1b8b1

Merge remote-tracking branch 'upstream/master' into feature/histogram…

7890186

…Equalization

added support for 16 bit greyscale

8117d82

JimBobSquarePants requested changes Jul 2, 2018

View reviewed changes

brianpopow changed the title ~~Feature: histogram equalization~~ [WIP] Feature: histogram equalization Jul 2, 2018

brianpopow added 5 commits July 2, 2018 15:34

SixLabors.ImageSharp.Processing.Contrast -> SixLabors.ImageSharp.Proc…

1d5a8f7

…essing.Normalization

using memoryAllocator

f4d9786

removed unnecessary allocation of the lut, using cdf instead

7ecfa0c

moved test to Normalization folder

e6ba776

fixed rounding issue in calculating the luminance

a5b180a

tocsoft reviewed Jul 2, 2018

View reviewed changes

brianpopow and others added 4 commits July 3, 2018 11:02

using GetPixelSpan instead of GetPixelRowSpan

dd698dd

allocating cdf and histogram buffer with a using statement

f78d9b7

using Vector4 to calculate the luminance and set the pixel value

ac69aa7

Merge branch 'master' into feature/histogramEqualization

3e08e30

brianpopow changed the title ~~[WIP] Feature: histogram equalization~~ Feature: histogram equalization Jul 3, 2018

brianpopow and others added 7 commits July 4, 2018 20:37

luminance levels is now a parameter of the constructor, defaults to 6…

5708144

…5536

Merge branch 'master' into feature/histogramEqualization

b102652

Merge branch 'master' into feature/histogramEqualization

c36b0df

moved extension to the processing namespace and the processor accordi…

4fcefc5

…ngly into Processors namespace

Merge branch 'master' into feature/histogramEqualization

8ad5b34

Cleanup and remove double cast.

0da5e68

Merge remote-tracking branch 'upstream/master' into feature/histogram…

2108bf1

…Equalization

JimBobSquarePants approved these changes Jul 16, 2018

View reviewed changes

JimBobSquarePants merged commit c1bc017 into SixLabors:master Jul 17, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: histogram equalization #644

Feature: histogram equalization #644

brianpopow commented Jul 2, 2018

JimBobSquarePants left a comment

JimBobSquarePants Jul 2, 2018

brianpopow Jul 2, 2018

JimBobSquarePants Jul 2, 2018

JimBobSquarePants Jul 2, 2018

brianpopow Jul 2, 2018

JimBobSquarePants Jul 2, 2018

brianpopow Jul 2, 2018

JimBobSquarePants Jul 2, 2018

brianpopow Jul 2, 2018

tocsoft left a comment

tocsoft Jul 2, 2018

brianpopow Jul 3, 2018

tocsoft Jul 3, 2018

brianpopow Jul 4, 2018

tocsoft Jul 4, 2018

brianpopow Jul 4, 2018

tocsoft Jul 2, 2018

brianpopow Jul 3, 2018

tocsoft Jul 2, 2018

brianpopow Jul 3, 2018

tocsoft Jul 2, 2018

codecov bot commented Jul 2, 2018 •

edited

Loading

brianpopow commented Jul 19, 2018

JimBobSquarePants commented Jul 20, 2018 •

edited

Loading


		using SixLabors.ImageSharp.PixelFormats;

		namespace SixLabors.ImageSharp.Processing.Contrast

Feature: histogram equalization #644

Feature: histogram equalization #644

Conversation

brianpopow commented Jul 2, 2018

Prerequisites

Description

JimBobSquarePants left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tocsoft left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Jul 2, 2018 • edited Loading

Codecov Report

brianpopow commented Jul 19, 2018

JimBobSquarePants commented Jul 20, 2018 • edited Loading

codecov bot commented Jul 2, 2018 •

edited

Loading

JimBobSquarePants commented Jul 20, 2018 •

edited

Loading