Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changed histogram binning logic #3918

Merged
merged 9 commits into from
Jun 12, 2020
Merged

Changed histogram binning logic #3918

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8688032
Changed how the bins are calculated and where the data goes.
kaitlyndlee Jun 10, 2020
506a638
Changed hist's output to align with the changes to Histogram
kaitlyndlee Jun 11, 2020
fb1a3db
Added changes to cnethist
kaitlyndlee Jun 11, 2020
dfacc2a
Added history entries
kaitlyndlee Jun 11, 2020
8aca563
Resolved conflicts
kaitlyndlee Jun 11, 2020
1fdc5a3
Merge branch 'hist' of https://github.com/kaitlyndlee/ISIS3 into kait…
AustinSanders Jun 12, 2020
b26fafc
Changed histogram binning logic
AustinSanders Jun 12, 2020
a85327d
Added ui.interactive check
AustinSanders Jun 12, 2020
5e8cd79
Removed extra history end tag
AustinSanders Jun 12, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions isis/src/base/apps/hist/hist.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,12 @@
Updated logic such that min/max values are no longer calculated from .cub
if values are provided by the user. Fixes #3881.
</change>
<change name="Kaitlyn Lee" date="2020-06-11">
Added "Pixels Below Min" and "Pixels Above Max" to the CSV output and changed how the data is
outputted. Originally, the CSV output and the GUI histogram would use the DN value in the
middle of a bin to represent that bin. That is, the CSV output used to have a "DN" value that
was the bin's middle pixel DN. This was not intuitive to users. Removed the "DN" value and added "MinInclusive" and "MaxExclusive" to the CSV so that bins are represented by their min/max values. These changes were also reflected in the histrogram creation. The x-axis is now based off of the min value of a bin instead of the middle value. These changes were made alongside changes made to Histogram and cnethist.
</change>
</history>

<oldName>
Expand Down
94 changes: 46 additions & 48 deletions isis/src/base/apps/hist/main.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ void IsisMain() {
Cube *icube = p.SetInputCube("FROM");

UserInterface &ui = Application::GetUserInterface();
if(!ui.WasEntered("TO") && !ui.IsInteractive()) {
if (!ui.WasEntered("TO") && !ui.IsInteractive()) {
QString msg = "The [TO] parameter must be entered";
throw IException(IException::User, msg, _FILEINFO_);
}
Expand Down Expand Up @@ -66,53 +66,60 @@ void IsisMain() {
p.Progress()->CheckStatus();
LineManager line(*icube);

for(int i = 1; i <= icube->lineCount(); i++) {
for (int i = 1; i <= icube->lineCount(); i++) {
line.SetLine(i);
icube->read(line);
hist->AddData(line.DoubleBuffer(), line.size());
p.Progress()->CheckStatus();
}

if(!ui.IsInteractive() || ui.WasEntered("TO") ) {
if (!ui.IsInteractive() || ui.WasEntered("TO") ) {
// Write the results
QString outfile = ui.GetFileName("TO");
ofstream fout;
fout.open(outfile.toLatin1().data());

fout << "Cube: " << ui.GetFileName("FROM") << endl;
fout << "Band: " << icube->bandCount() << endl;
fout << "Average: " << hist->Average() << endl;
fout << "Std Deviation: " << hist->StandardDeviation() << endl;
fout << "Variance: " << hist->Variance() << endl;
fout << "Median: " << hist->Median() << endl;
fout << "Mode: " << hist->Mode() << endl;
fout << "Skew: " << hist->Skew() << endl;
fout << "Minimum: " << hist->Minimum() << endl;
fout << "Maximum: " << hist->Maximum() << endl;
fout << "Cube: " << ui.GetFileName("FROM") << endl;
fout << "Band: " << icube->bandCount() << endl;
fout << "Average: " << hist->Average() << endl;
fout << "Std Deviation: " << hist->StandardDeviation() << endl;
fout << "Variance: " << hist->Variance() << endl;
fout << "Median: " << hist->Median() << endl;
fout << "Mode: " << hist->Mode() << endl;
fout << "Skew: " << hist->Skew() << endl;
fout << "Minimum: " << hist->Minimum() << endl;
fout << "Maximum: " << hist->Maximum() << endl;
fout << endl;
fout << "Total Pixels: " << hist->TotalPixels() << endl;
fout << "Valid Pixels: " << hist->ValidPixels() << endl;
fout << "Null Pixels: " << hist->NullPixels() << endl;
fout << "Lis Pixels: " << hist->LisPixels() << endl;
fout << "Lrs Pixels: " << hist->LrsPixels() << endl;
fout << "His Pixels: " << hist->HisPixels() << endl;
fout << "Hrs Pixels: " << hist->HrsPixels() << endl;
fout << "Total Pixels: " << hist->TotalPixels() << endl;
fout << "Valid Pixels: " << hist->ValidPixels() << endl;
fout << "Pixels Below Min: " << hist->UnderRangePixels() << endl;
fout << "Pixels Above Max: " << hist->OverRangePixels() << endl;
fout << "Null Pixels: " << hist->NullPixels() << endl;
fout << "Lis Pixels: " << hist->LisPixels() << endl;
fout << "Lrs Pixels: " << hist->LrsPixels() << endl;
fout << "His Pixels: " << hist->HisPixels() << endl;
fout << "Hrs Pixels: " << hist->HrsPixels() << endl;

// Write histogram in tabular format
fout << endl;
fout << endl;
fout << "DN,Pixels,CumulativePixels,Percent,CumulativePercent" << endl;
fout << "MinInclusive,MaxExclusive,Pixels,CumulativePixels,Percent,CumulativePercent" << endl;

Isis::BigInt total = 0;
double cumpct = 0.0;
double low;
double high;

for(int i = 0; i < hist->Bins(); i++) {
if(hist->BinCount(i) > 0) {
for (int i = 0; i < hist->Bins(); i++) {
if (hist->BinCount(i) > 0) {
total += hist->BinCount(i);
double pct = (double)hist->BinCount(i) / hist->ValidPixels() * 100.;
cumpct += pct;

fout << hist->BinMiddle(i) << ",";
hist->BinRange(i, low, high);

fout << low << ",";
fout << high << ",";
fout << hist->BinCount(i) << ",";
fout << total << ",";
fout << pct << ",";
Expand All @@ -122,10 +129,10 @@ void IsisMain() {
fout.close();
}
// If we are in gui mode, create a histogram plot
if(ui.IsInteractive()) {
if (ui.IsInteractive()) {
// Set the title for the dialog
QString title;
if(ui.WasEntered("TITLE") ) {
if (ui.WasEntered("TITLE") ) {
title = ui.GetString("TITLE");
}
else {
Expand All @@ -138,19 +145,19 @@ void IsisMain() {
ui.TheGui());

// Set the xaxis title if they entered one
if(ui.WasEntered("XAXIS") ) {
if (ui.WasEntered("XAXIS") ) {
QString xaxis(ui.GetString("XAXIS"));
plot->setAxisLabel(QwtPlot::xBottom, xaxis.toLatin1().data());
}

// Set the yLeft axis title if they entered one
if(ui.WasEntered("FREQAXIS") ) {
if (ui.WasEntered("FREQAXIS") ) {
QString yaxis(ui.GetString("FREQAXIS"));
plot->setAxisLabel(QwtPlot::yRight, yaxis.toLatin1().data());
}

// Set the yRight axis title if they entered one
if(ui.WasEntered("PERCENTAXIS") ) {
if (ui.WasEntered("PERCENTAXIS") ) {
QString y2axis(ui.GetString("PERCENTAXIS") );
plot->setAxisLabel(QwtPlot::yLeft, y2axis.toLatin1().data());
}
Expand All @@ -159,13 +166,16 @@ void IsisMain() {
QVector<QPointF> binCountData;
QVector<QPointF> cumPctData;
double cumpct = 0.0;
for(int i = 0; i < hist->Bins(); i++) {
if(hist->BinCount(i) > 0) {
binCountData.append(QPointF(hist->BinMiddle(i), hist->BinCount(i) ) );

double pct = (double)hist->BinCount(i) / hist->ValidPixels() * 100.;
double low;
double high;
for (int i = 0; i < hist->Bins(); i++) {
if (hist->BinCount(i) > 0) {
hist->BinRange(i, low, high);
binCountData.append(QPointF(low, hist->BinCount(i) ) );

double pct = (double)hist->BinCount(i) / hist->ValidPixels() * 100.0;
cumpct += pct;
cumPctData.append(QPointF(hist->BinMiddle(i), cumpct) );
cumPctData.append(QPointF(low, cumpct) );
}
}

Expand All @@ -184,21 +194,13 @@ void IsisMain() {
cdfCurve->setYAxis(QwtPlot::yLeft);
cdfCurve->setPen(*pen);

//These are all variables needed in the following for loop.
//----------------------------------------------
QVector<QwtIntervalSample> intervals(binCountData.size() );
// double maxYValue = DBL_MIN;
// double minYValue = DBL_MAX;
// // ---------------------------------------------
//
for(int y = 0; y < binCountData.size(); y++) {
for (int y = 0; y < binCountData.size(); y++) {

intervals[y].interval = QwtInterval(binCountData[y].x(),
binCountData[y].x() + hist->BinSize());

intervals[y].value = binCountData[y].y();
// if(values[y] > maxYValue) maxYValue = values[y];
// if(values[y] < minYValue) minYValue = values[y];
}

QPen percentagePen(Qt::red);
Expand All @@ -211,10 +213,6 @@ void IsisMain() {

plot->add(histCurve);
plot->add(cdfCurve);
// plot->fillTable();

// plot->setScale(QwtPlot::yLeft, 0, maxYValue);
// plot->setScale(QwtPlot::xBottom, hist.Minimum(), hist.Maximum());

QLabel *label = new QLabel(" Average = " + QString::number(hist->Average()) + '\n' +
"\n Minimum = " + QString::number(hist->Minimum()) + '\n' +
Expand Down
34 changes: 10 additions & 24 deletions isis/src/base/objs/Histogram/Histogram.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,6 @@ namespace Isis {
Histogram::Histogram(double minimum, double maximum, int nbins) {

SetValidRange(minimum, maximum);
//SetBinRange(minimum, maximum);
SetBins(nbins);
}

Expand Down Expand Up @@ -167,21 +166,9 @@ namespace Isis {

rangesFromNet(net,statFunc);


//stretch the domain so that it is an even multiple of binWidth
//for some reason Histogram makes the end points of the bin range be at the center of
//bins. Thus the +/-0.5 forces it to point the bin range at the ends of the bins.
//SetBinRange(binWidth*( floor(this->ValidMinimum()/binWidth )+0.5),
// binWidth*(ceil( this->ValidMaximum()/binWidth )-0.5) );


//Keep an eye on this to see if it breaks anything. Also, I need to create
//a dataset to give this constructor for the unit test.

//tjw: SetValidRange is moved into SetBinRange
//SetValidRange(binWidth*floor(this->ValidMinimum()/binWidth),
// binWidth*ceil(this->ValidMaximum()/binWidth));

//from the domain of the data and the requested bin width calculate the number of bins
double domain = this->ValidMaximum() - this->ValidMinimum();
int nBins = int ( ceil(domain/binWidth) );
Expand Down Expand Up @@ -266,7 +253,6 @@ namespace Isis {
//set up the histogram ranges

SetValidRange(min, max);
//SetBinRange(min, max);
}


Expand Down Expand Up @@ -396,7 +382,7 @@ namespace Isis {
Histogram::~Histogram() {
}

//2015-08-24, Tyler Wilson: Added Statistics::SetValidRange call to SetBinRange
//2015-08-24, Tyler Wilson: Added Statistics::SetValidRange call to SetValidRange
//So the two functions do not have to be called together when setting
//up a histogram

Expand Down Expand Up @@ -462,8 +448,8 @@ namespace Isis {
index = 0;
}
else {
index = (int) floor((double)(nbins - 1) / (BinRangeEnd() - BinRangeStart()) *
(data[i] - BinRangeStart() ) + 0.5);
index = (int) floor( ((double) nbins / (BinRangeEnd() - BinRangeStart())) *
(data[i] - BinRangeStart()) );
}
if (index < 0) index = 0;
if (index >= nbins) index = nbins - 1;
Expand All @@ -490,8 +476,8 @@ namespace Isis {
index = 0;
}
else {
index = (int) floor((double)(nbins - 1) / (BinRangeEnd() - BinRangeStart() ) *
(data - BinRangeStart() ) + 0.5);
index = (int) floor( ((double) nbins / (BinRangeEnd() - BinRangeStart())) *
(data - BinRangeStart()) );
}
if (index < 0) index = 0;
if (index >= nbins) index = nbins - 1;
Expand Down Expand Up @@ -522,8 +508,8 @@ namespace Isis {
index = 0;
}
else {
index = (int) floor((double)(nbins - 1) / (BinRangeEnd() - BinRangeStart()) *
(data[i] - BinRangeStart()) + 0.5);
index = (int) floor( ((double) nbins / (BinRangeEnd() - BinRangeStart())) *
(data[i] - BinRangeStart()) );
}
if (index < 0) index = 0;
if (index >= nbins) index = nbins - 1;
Expand Down Expand Up @@ -591,7 +577,7 @@ namespace Isis {
}
}

return BinMiddle( (int)p_bins.size() - 1);
return BinMiddle( (int) p_bins.size() - 1);
}


Expand Down Expand Up @@ -653,8 +639,8 @@ namespace Isis {
throw IException(IException::Programmer, message, _FILEINFO_);
}

double binSize = (BinRangeEnd() - BinRangeStart()) / (double)(p_bins.size() - 1);
low = BinRangeStart() - binSize / 2.0 + binSize * (double) index;
double binSize = (BinRangeEnd() - BinRangeStart()) / (double) p_bins.size();
low = BinRangeStart() + binSize * (double) index;
high = low + binSize;
}

Expand Down
5 changes: 5 additions & 0 deletions isis/src/base/objs/Histogram/Histogram.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,11 @@ namespace Isis {
* #1673.
* @history 2018-07-27 Jesse Mapel - Added support for initializing a histogram from
* signed and unsigned word cubes. References #971.
* @history 2020-06-11 Kaitlyn Lee - Changed how to detemine which bin a pixel/measure falls
* into in AddData(). Changed how the bin range is calculated in
* BinRange(). The math for these functions were incorrect and not
* intuitive. These changes were made alongside changes made
* to cnethist and hist.
*/

class Histogram : public Statistics {
Expand Down
5 changes: 5 additions & 0 deletions isis/src/control/apps/cnethist/cnethist.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@
<change name="Orrin Thomas" date="2012-04-19">
Original version
</change>
<change name="Kaitlyn Lee" date="2020-06-11">
Removed "ResidualMagnitude" from the CSV output and added "ResidualMagnitudeMin" and "ResidualMagnitudeMax". This way, the bins are represented by the min/max values of the
bins instead of the DN of the pixel in the middle of the bin. These changes were also
reflected in the histrogram creation. The x-axis is now based off of the min value of a bin instead of the middle value. These changes were made alongside changes made to Histogram and hist.
</change>
</history>

<groups>
Expand Down
Loading