Ask Your Question

Revision history [back]

click to hide/show revision 1
initial version

rounding error .n() and N()

Why does 1.414 - sqrt(2).n(digits=4) not evaluate to zero?

sage: sqrt(2).n(digits=4) 1.414 sage: 1.414 - sqrt(2).n(digits=4) -0.0002136

rounding error .n() and N()

Why does 1.414 - sqrt(2).n(digits=4) not evaluate to zero?

sage: sqrt(2).n(digits=4) 1.414 sqrt(2).n(digits=4)

1.414

sage: 1.414 - sqrt(2).n(digits=4) sqrt(2).n(digits=4)

-0.0002136