Weird rounding error in n and N

asked 2021-09-15 22:52:41 +0200

jakupl gravatar image

This is weird. While dabbling in sage, It seems I hit a rounding error.

sage: number=180*arccos(14/205*sqrt(41)*sqrt(5))/pi
sage: n(number)
12.0947570770121
sage: n(number,digits=4)
12.10

Is there any good reason that I'm not getting 12.09 as a result? This happens in N as well.

Here's my sage info.

SageMath version 9.2, Release Date: 2020-10-24 
Using Python 3.9.5. Type "help()" for help.
edit retag flag offensive close merge delete

Comments

1

Something fishy is going on here. If we take the numerical value directly. it's rounded correctly:

sage: a = 12.0947570770121                                                                                                                                                                                         
sage: n(a,digits=4)                                                                                                                                                                                                
12.09
Max Alekseyev gravatar imageMax Alekseyev ( 2021-09-16 04:34:39 +0200 )edit

As a temporary workaround you can use n(n(number),digits=4)

Max Alekseyev gravatar imageMax Alekseyev ( 2021-09-16 05:03:58 +0200 )edit

Same thing happens in CoCalc, running SageMath version 9.3

jakupl gravatar imagejakupl ( 2021-09-16 08:51:14 +0200 )edit
2

It may be possible that digits=4 is applied to every term in the expression before it's evaluated, resulting in the loss of precision, like in:

sage: 180*arccos(14/205*n(sqrt(41),digits=4)*n(sqrt(5),digits=4))/n(pi,digits=4)                                                                                                                                   
12.10
Max Alekseyev gravatar imageMax Alekseyev ( 2021-09-16 15:51:58 +0200 )edit
1

That's a solid hypothesis! I did find a better method:

sage: round(number,2)
12.09

This is probably how one should do it. I will however leave this post unanswered, as the original question is still not definitely answered.

jakupl gravatar imagejakupl ( 2021-09-16 16:22:45 +0200 )edit
see more comments