Allowing for small float rounding errors when calculating column widths

[hbrandl]

I found another bug in the table width calculation algorithm.

In some cases arithmetic errors lead to errors.

Testcase to illustrate the problem

    it "table widths should be correctly calculated even for column widths with lots of decimals", :focus do
      data=[["a", "b ", "c ", "d", "e", "f", "g", "h", "i", "j", "k", "l"],
            [{:content=>"Foobar", :colspan=>12}]
          ]
      #we need values with lots of decimals so that arithmetic errors will occur
      #the values are not arbitrary but where found converting mm to pdf pt
      column_widths=[137, 40, 40, 54.69291338582678, 54.69291338582678, 54.69291338582678, 54.69291338582678, 54.69291338582678, 54.69291338582678, 54.69291338582678, 54.69291338582678, 54.69291338582678]
      pdf = Prawn::Document.new({:page_size => 'A4', :page_layout => :landscape})
      table = Prawn::Table.new data, pdf, :column_widths => column_widths
      table.column_widths.should == column_widths
    end

Output of current master

1) Prawn::Table You can explicitly set the column widths and use a colspan > 1 table widths should be correctly calculated even for column widths with lots of decimals
     Failure/Error: table.column_widths.should == column_widths
       expected: [137, 40, 40, 54.69291338582678, 54.69291338582678, 54.69291338582678, 54.69291338582678, 54.69291338582678, 54.69291338582678, 54.69291338582678, 54.69291338582678, 54.69291338582678]
            got: [59.10301837270342, 59.10301837270342, 59.10301837270342, 59.10301837270342, 59.10301837270342, 59.10301837270342, 59.10301837270342, 59.10301837270342, 59.10301837270342, 59.10301837270342, 59.10301837270342, 59.10301837270342] (using ==)
     # ./spec/table_spec.rb:59:in `block (3 levels) in <top (required)>'

This pull request addresses the issue by allowing for an small arithmetic error (e-09) in the two affected algorithms.

[hbrandl]

The test case only addresses the change in column_width_calculator.rb.
I'm pretty sure that the change in cells.rb addresses a bug too. But I don't know how to produce the floating point arithmetic error needed to prove it.

[practicingruby]

@bradediger I am curious of your thoughts on the problem and the fix here. @hbrandl and I chatted over IRC about it a bit and both agree it's not ideal.

Should we thinking about using BigDecimal here? Or something else entirely? Or just stick with the fuzzy float comparison?

[fidothe]

I think you want to avoid BigDecimal like the plague here. PostScript Points -> any sane measurement system involves lots of silly floating point, BigDecimal assumes something rather saner. I was wondering whether some precision limit was worth enforcing (AFAICT InDesign et al don't output anything like the precision Prawn does, they round out at 3 or 4 places (IIRC)). That might, in turn, make this kind of problem largely go away. [I realise that this is not an entirely helpful comment in this context]

[bradediger]

@sandal Yes, I think using a FP epsilon and changing equality tests to closeness tests is the best (least insane) way to go here. BigDecimal is going to be a huge rabbit hole that offers little payoff, IMO.

[practicingruby]

@bradediger OK, Should we decide on a precision and create a constant for it? I feel like 10^-9 is more than enough, but we could use something close to machine precision via Float::EPSILON. I just always forget the right way to do those calculations 😢

[bradediger]

@sandal Errors accumulate during consecutive numerical operations, so we can't necessarily just use the machine epsilon. But I definitely think we could standardize on an epsilon around 1e-9 (give or take a few orders of magnitude) for all FP comparisons.

[practicingruby]

Okay, so it sounds like we just need to add a test for the uncovered case here, and clean up the checks a little bit. I'm unblocked to work on this, but won't get around to it for at least another week. Anyone who wants to take a stab at this sooner than that is welcome to do so.

[practicingruby]

I can't really think of how to construct the test case the possible rounding error in the natural column widths code, so I think we'll need to build one in response to a regression rather than proactively.
That said, if someone has an idea of how to pin this behavior down, please submit a pull request with an updated spec.

I made some minor tweaks, and I'm going to go ahead and squash+merge this now.