Numerical approximation error

asked 2021-03-02 15:44:22 +0100

danielfn
21 ●1 ●4

I'm apparently having a very specific rounding/floating point error when using the numerical approximation function .n(). I reproduced it in the example below. The "1" would be a function whose numerical value amounts to 1.

Input:

print(int(-1/6*1.n() + 49/6))
print(int(-1/6 + 49/6))

Output:

7
8

The error does not happen if we specify a precision for approximation, such as: Input:

print(int(-1/6*1.n(1)+ 49/6))
print(int(-1/6 + 49/6))

Output:

8
8

Can this be corrected?

answered 2021-03-03 06:02:54 +0100

John Palmieri

8876 ●19 ●73 ●192 http://www.math.washin...

Once you use a numerical approximation, you probably shouldn't do further arithmetic and then another approximation (such as applying int) and expect reliable results. The number you're getting is very close to 8, just a little smaller, so when int truncates it, the result is 7.

sage: 8-(49/6-(1/6*1.n()))
8.88178419700125e-16

Perhaps it would be better to use round instead of int: int is a built in Python function, while round is better suited to work with Sage number types. In general, though, it's probably best not to use .n() until the end of the calculation.

edit flag offensive delete link

Comments

One more comment: when you do -1/6 * 1.n(), the use of 1.n() forces all remaining computations to be done using approximate computer arithmetic, as opposed to -1/6 * 1 which Sage recognizes as being a fraction with exact value -1/6. That's why the rounding issues occur.

John Palmieri ( 2021-03-03 22:52:49 +0100 )edit

add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Numerical approximation error

1 Answer

Comments

Your Answer

Question Tools

Stats

Related questions

Numerical approximation error edit

1 Answer

Comments

Your Answer

Question Tools

Stats

Related questions

Numerical approximation error