Floating point arthmetic #77
Conversation
| @@ -1603,6 +1618,286 @@ | |||
| "level": 2, | |||
| "metadata": {}, | |||
| "source": [ | |||
| "Floating point arithmetic" | |||
There was a problem hiding this comment.
We usually title case section headings: "Floating Point Arithmetic".
|
Good section, and absolutely necessary: but the reaction I always get when I teach the problems with floating point is, "So what am I supposed to do?" We can tell learners to always compare things within tolerances, but then they'll ask, "How do I figure out a good tolerance?" (And if we say, "You have to work through the numerical analysis to figure that out," there's an awkward silence, because they don't know how to, and won't do it anyway, because nobody they know is doing it.) And when it comes to other operations --- rearranging calculations to preserve significant digits, for example --- all the rules of thumb are very specialized. Long story short: I don't know how to wrap this discussion up with a recommendation, but I really feel it needs one... |
|
@gvwilson @ahmadia I like all of these suggested improvements from Greg, but I'm a little unsure about what happens next. Do I need to re-submit a new pull request with the changes incorporated into it? @gvwilson I think the recommendation has to be something along the lines of what you're saying: Floating point issues are often unique to the problem you're working on. However, if you're aware of the common tools (e.g. np.allclose) and techniques (don't use ==) used to deal with them, you can almost always come up with a simple solution (e.g. like the approx_one function in my example). |
|
On 2013-10-20 5:31 PM, DamienIrving wrote:
|
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "This result brings us to the first rule of floating point arithmetic: never use `==` (or `!=`) on floating point numbers. Instead, it is better to check whether two values are within some tolerance, and if they are, treat them as equal. NumPy provides some very convenient tools for doing this, which typically allow you to define the tolerance in terms of the absolute and/or relative error. The [absolute error](glossary.html#aboslute-error) (`atol`) in some approximation is simply the absolute value of the difference between the actual value and the approximation. The [relative error](glossary.html#relative-error) (`rtol`), on the other hand, is the ratio of the absolute error to the value we're approximating. For example, if we're off by 1 in approximating 8+1 and 56+1, the absolute error is the same in both cases, but the relative error in the first case is 1/9 = 11%, while in the second case it is only 1/57 = 1.7%. When we're thinking about floating point numbers, relative error is almost always more useful than absolute error. After all, it makes little sense to say that we're off by a hundreth when the value in question is a billionth. \n", |
There was a problem hiding this comment.
The last sentence is a little confusing. If I understand correctly what you're trying to say, then maybe you could change it to "After all, if we're off by a hundredth, we will be a lot more concerned if the value in question is a billionth than if it is one thousand."
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "The relative error associated with this expression is very small, however it's significant because arccosine is only defined over the range [-1.0, 1.0]. If we pass this derived input value to our simple trigonometic function, it now returns `nan`. This is NumPy's way of representing an undefined or unrepresentable value (and stands for \"not a number\"). " |
There was a problem hiding this comment.
It would be more correct to say "this is the standard way computers represent " instead of "this is NumPy's way of representing". NumPy is simply propagating up a NaN as defined by the IEEE floating point standard.
|
@DamienIrving - thanks for all your work on this one. It's alllllmost ready :) @mattphotonman and @gvwilson - thanks for your reviews. |
ghost
assigned
|
Thanks for the detailed review @wltrimbl. @DamienIrving - are you ready for another review? :) |
|
Whoops, didn't mean to close it! @ahmadia I guess I'm ready for another review, although at this point I'm thinking it's probably ready to be merged. |
|
I'm +1 for merge. |
I've created a new pull request for the floating point arithmetic stuff, since I figured I should do it the correct way (i.e. create a 'floating_point_arithmetic' branch, as opposed to working in gh-pages).