Numerical approximation error

danielfn
21 ●1 ●4

I'm apparently having a very specific rounding/floating point error when using the numerical approximation function .n(). I reproduced it in the example below. The "1" would be a function whose numerical value amounts to 1.

Input:

print(int(-1/6*1.n() + 49/6))
print(int(-1/6 + 49/6))

Output:

7
8

The error does not happen if we specify a precision for approximation, such as: Input:

print(int(-1/6*1.n(1)+ 49/6))
print(int(-1/6 + 49/6))

Output:

8
8

Can this be corrected?

answered 4 years ago

John Palmieri

8906 ●20 ●74 ●193 http://www.math.washin...

Once you use a numerical approximation, you probably shouldn't do further arithmetic and then another approximation (such as applying int) and expect reliable results. The number you're getting is very close to 8, just a little smaller, so when int truncates it, the result is 7.

sage: 8-(49/6-(1/6*1.n()))
8.88178419700125e-16

Perhaps it would be better to use round instead of int: int is a built in Python function, while round is better suited to work with Sage number types. In general, though, it's probably best not to use .n() until the end of the calculation.

link

Comments

One more comment: when you do -1/6 * 1.n(), the use of 1.n() forces all remaining computations to be done using approximate computer arithmetic, as opposed to -1/6 * 1 which Sage recognizes as being a fraction with exact value -1/6. That's why the rounding issues occur.

John Palmieri ( 4 years ago )

add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Numerical approximation error

1 Answer

Comments

Your Answer

Question Tools

Stats

Related questions

Numerical approximation error savecancel

1 Answer

Comments

Your Answer

Question Tools

Stats

Related questions

Numerical approximation error