Ask Your Question

stack overflow in computing entropy

asked 2012-09-22 20:42:12 +0100

kayhan gravatar image


I am still new to sage, I am trying to compute use sage to do some symbolic computation. Here is my test to compute entropy of the Gaussian distribution:

x, sigma, mu,a = var('x','sigma','mu','a')
q = 1/((2*pi)^(1/2)*sigma)*exp(-(x - mu)^2/(2*sigma^2))
import sympy

First of all, it complains if I don't assume mu is positive. It is right that sigma should be positive but why mu ?! Second, even after assuming mu>0, it produces stack overflow:

WARNING: Output truncated!  

;;; Binding stack overflow.
;;; Jumping to the outermost toplevel prompt

;;; Binding stack overflow.
;;; Jumping to the outermost toplevel prompt

Does anyone know why? I am sure I am missing something.

edit retag flag offensive close merge delete


The semicolons indicate this is a Maxima overflow. (Maxima handles our symbolic integration, and raises the error messages asking for the sign of various parameters.)

kcrisman gravatar imagekcrisman ( 2012-09-22 23:05:08 +0100 )edit

By the way, importing sympy would not do anything here, unless you used sympy.integral or whatever their syntax is. Which could be an interesting option for you (or so I thought until I saw your other question).

kcrisman gravatar imagekcrisman ( 2012-09-22 23:05:32 +0100 )edit

3 Answers

Sort by ยป oldest newest most voted

answered 2012-09-23 02:32:57 +0100

achrzesz gravatar image

updated 2012-09-23 05:40:49 +0100

Maxima can compute the entropy


(%i1) display2d:false$                                       

(%i2) q:1/((2*%pi)^(1/2)*sigma)*exp(-(x - mu)^2/(2*sigma^2))$

(%i3) assume(sigma>0)$

(%i4) assume(mu>0)$

(%i5) ii:integrate(-q*log(q),x,minf,inf)$

(%i6) changevar(ii,(x-mu)/(sqrt(2)*sigma)-y,y,x)$

(%i7) ev(%,integrate)$

(%i8) ratsimp(%);

(%o8) (2*log(sigma)+log(%pi)+log(2)+1)/2
(%i9) logcontract(%);

(%o9) (log(2*%pi*sigma^2)+1)/2

and the last expression is equal to:


edit flag offensive delete link more


Nice, except that one has to change the variable explicitly. So Maxima can't really do it without extra intervention :(

kcrisman gravatar imagekcrisman ( 2012-09-24 15:01:04 +0100 )edit

answered 2012-09-23 22:19:05 +0100

kayhan gravatar image

Thank you for your reply.

The answer is correct but just a few points for clarification of my mind:

  • maxima is able to compute the integral correctly but it totally different software and sage can interface with it. However the syntax is totally different.

  • The mystery that why sage cannot solve the integral is not resolved, right?

  • It is not clear why we need to define (mu > 0) even in maxima.

Again thank you for your reply.

edit flag offensive delete link more


You are correct that the overflow is still not resolved. Sage doesn't just interface with Maxima, though, but USES it to do integrals. (You can also use Sympy and some other options, with the `algorithm` keyword.) Check out

kcrisman gravatar imagekcrisman ( 2012-09-24 15:02:29 +0100 )edit

answered 2012-09-22 23:13:33 +0100

kcrisman gravatar image

updated 2012-09-24 15:00:33 +0100

This isn't an answer, but apparently the problem isn't Maxima per se. (Recall that Maxima is the normal integration engine within Sage.) After the requisite assumptions, I have (in Sage's native Maxima):

sage: maxima_console()
(%i1) q:1/((2*%pi)^(1/2)*sigma)*exp(-(x - mu)^2/(2*sigma^2));
                                     (x - mu)
                                   - ---------
                                     2 sigma
(%o1)                       -----------------------
                            sqrt(2) sqrt(%pi) sigma
(%i2) assume(sigma>0);
(%o2)                             [sigma > 0]
(%i3) assume(mu>0);
(%o3)                              [mu > 0]
(%i4) integrate(q,x,minf,inf);
(%o4)                                  1

(%i5) integrate(-q*log(q),x,minf,inf);
                                 2              (x - mu)
                         (x - mu)             - ---------
                inf    - ---------                     2
               /                2               2 sigma
               [         2 sigma            %e
               I     %e            log(-----------------------) dx
               ]                       sqrt(2) sqrt(%pi) sigma
(%o5)        - ---------------------------------------------------
                             sqrt(2) sqrt(%pi) sigma

which shouldn't cause an overflow. See the other answer for more details on how to simplify this.

(In the notebook, one can choose "Maxima" from the combo box of systems, or put %maxima at the beginning of a cell, and then just put the input commands above in that cell to evaluate this.)

edit flag offensive delete link more


where did you define "inf" and "minf" ? did you import a library to get the answer?

kayhan gravatar imagekayhan ( 2012-09-22 23:24:01 +0100 )edit

Sorry, I'll edit to make this clear.

kcrisman gravatar imagekcrisman ( 2012-09-22 23:25:02 +0100 )edit

sorry for my ignorance, but it is still unclear for me. What is maxima_console() ? I am using web-based interface, are you using something else? Do you think it is a bug? You mentioned default mode of integration in sage is maxima, how can I change it?

kayhan gravatar imagekayhan ( 2012-09-22 23:38:30 +0100 )edit

Hmm, that's true, you probably can't do the interactive console in Maxima. However, you could just type in the commands - I'll edit again.

kcrisman gravatar imagekcrisman ( 2012-09-24 14:59:01 +0100 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools


Asked: 2012-09-22 20:42:12 +0100

Seen: 865 times

Last updated: Sep 24 '12