stack overflow in computing entropy

asked 2012-09-22 20:42:12 +0100

kayhan
13 ●1 ●1 ●5

Hi,

I am still new to sage, I am trying to compute use sage to do some symbolic computation. Here is my test to compute entropy of the Gaussian distribution:

x, sigma, mu,a = var('x','sigma','mu','a')
q = 1/((2*pi)^(1/2)*sigma)*exp(-(x - mu)^2/(2*sigma^2))
assume(sigma>0)
assume(mu>0)
show(q)
import sympy
integral(q,(x,-oo,oo)) 
integral(-q*log(q),(x,-oo,oo))

First of all, it complains if I don't assume mu is positive. It is right that sigma should be positive but why mu ?! Second, even after assuming mu>0, it produces stack overflow:

WARNING: Output truncated!  
full_output.txt


;;;
;;; Binding stack overflow.
;;; Jumping to the outermost toplevel prompt
;;;


;;;
;;; Binding stack overflow.
;;; Jumping to the outermost toplevel prompt
;;;

Does anyone know why? I am sure I am missing something.

edit retag flag offensive close merge delete

Comments

The semicolons indicate this is a Maxima overflow. (Maxima handles our symbolic integration, and raises the error messages asking for the sign of various parameters.)

kcrisman ( 2012-09-22 23:05:08 +0100 )edit

By the way, importing sympy would not do anything here, unless you used sympy.integral or whatever their syntax is. Which could be an interesting option for you (or so I thought until I saw your other question).

kcrisman ( 2012-09-22 23:05:32 +0100 )edit

add a comment

3 Answers

Sort by » oldest newest most voted

answered 2012-09-23 02:32:57 +0100

achrzesz
2803 ●5 ●25 ●61

updated 2012-09-23 05:40:49 +0100

Maxima can compute the entropy

sage:maxima_console()

(%i1) display2d:false$                                       

(%i2) q:1/((2*%pi)^(1/2)*sigma)*exp(-(x - mu)^2/(2*sigma^2))$

(%i3) assume(sigma>0)$

(%i4) assume(mu>0)$

(%i5) ii:integrate(-q*log(q),x,minf,inf)$

(%i6) changevar(ii,(x-mu)/(sqrt(2)*sigma)-y,y,x)$

(%i7) ev(%,integrate)$

(%i8) ratsimp(%);

(%o8) (2*log(sigma)+log(%pi)+log(2)+1)/2
(%i9) logcontract(%);

(%o9) (log(2*%pi*sigma^2)+1)/2

and the last expression is equal to:

log(sqrt(2*pi)*sigma)+1/2

edit flag offensive delete link

Comments

Nice, except that one has to change the variable explicitly. So Maxima can't really do it without extra intervention :(

kcrisman ( 2012-09-24 15:01:04 +0100 )edit

add a comment

answered 2012-09-23 22:19:05 +0100

kayhan
13 ●1 ●1 ●5

Thank you for your reply.

The answer is correct but just a few points for clarification of my mind:

maxima is able to compute the integral correctly but it totally different software and sage can interface with it. However the syntax is totally different.
The mystery that why sage cannot solve the integral is not resolved, right?
It is not clear why we need to define (mu > 0) even in maxima.

Again thank you for your reply.

edit flag offensive delete link

Comments

You are correct that the overflow is still not resolved. Sage doesn't just interface with Maxima, though, but USES it to do integrals. (You can also use Sympy and some other options, with the `algorithm` keyword.) Check out http://www.sagemath.org/doc/reference/sage/symbolic/integration/integral.html#sage.symbolic.integration.integral.integral

kcrisman ( 2012-09-24 15:02:29 +0100 )edit

add a comment

answered 2012-09-22 23:13:33 +0100

kcrisman
12242 ●42 ●136 ●255

updated 2012-09-24 15:00:33 +0100

This isn't an answer, but apparently the problem isn't Maxima per se. (Recall that Maxima is the normal integration engine within Sage.) After the requisite assumptions, I have (in Sage's native Maxima):

sage: maxima_console()
<snip>
(%i1) q:1/((2*%pi)^(1/2)*sigma)*exp(-(x - mu)^2/(2*sigma^2));
                                             2
                                     (x - mu)
                                   - ---------
                                            2
                                     2 sigma
                                 %e
(%o1)                       -----------------------
                            sqrt(2) sqrt(%pi) sigma
(%i2) assume(sigma>0);
(%o2)                             [sigma > 0]
(%i3) assume(mu>0);
(%o3)                              [mu > 0]
(%i4) integrate(q,x,minf,inf);
(%o4)                                  1

(%i5) integrate(-q*log(q),x,minf,inf);
                                                        2
                                 2              (x - mu)
                         (x - mu)             - ---------
                inf    - ---------                     2
               /                2               2 sigma
               [         2 sigma            %e
               I     %e            log(-----------------------) dx
               ]                       sqrt(2) sqrt(%pi) sigma
               /
                minf
(%o5)        - ---------------------------------------------------
                             sqrt(2) sqrt(%pi) sigma
(%i6)

which shouldn't cause an overflow. See the other answer for more details on how to simplify this.

(In the notebook, one can choose "Maxima" from the combo box of systems, or put %maxima at the beginning of a cell, and then just put the input commands above in that cell to evaluate this.)

edit flag offensive delete link

Comments

where did you define "inf" and "minf" ? did you import a library to get the answer?

kayhan ( 2012-09-22 23:24:01 +0100 )edit

Sorry, I'll edit to make this clear.

kcrisman ( 2012-09-22 23:25:02 +0100 )edit

sorry for my ignorance, but it is still unclear for me. What is maxima_console() ? I am using web-based interface, are you using something else? Do you think it is a bug? You mentioned default mode of integration in sage is maxima, how can I change it?

kayhan ( 2012-09-22 23:38:30 +0100 )edit

Hmm, that's true, you probably can't do the interactive console in Maxima. However, you could just type in the commands - I'll edit again.

kcrisman ( 2012-09-24 14:59:01 +0100 )edit

add a comment

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

stack overflow in computing entropy

Comments

3 Answers

Comments

Comments

Comments

Your Answer

Question Tools

Stats

Related questions

stack overflow in computing entropy edit

Comments

3 Answers

Comments

Comments

Comments

Your Answer

Question Tools

Stats

Related questions

stack overflow in computing entropy