ASKSAGE: Sage Q&A Forum - Individual question feedhttp://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Sun, 30 Jan 2011 05:59:16 -0600Plotting and Fittinghttp://ask.sagemath.org/question/7909/plotting-and-fitting/Hi, I have to learn to plot and fit functions (with error bars in x and y direction).
My question is: **1**) Can I do it with sage? Would you prefer gnuplot?
How can I read and plot data like this: http://rn0.ru/show/1430/ easily in sage?
**2**) And if yes how?
I already tried to plot some randomly created data: http://rn0.ru/show/1429/ And that works great.
**3**) How can I fit a function (nonlinear) with considering the errorbars?
Fri, 28 Jan 2011 05:07:37 -0600http://ask.sagemath.org/question/7909/plotting-and-fitting/Answer by DSM for <p>Hi, I have to learn to plot and fit functions (with error bars in x and y direction). </p>
<p>My question is: <strong>1</strong>) Can I do it with sage? Would you prefer gnuplot?</p>
<p>How can I read and plot data like this: <a href="http://rn0.ru/show/1430/">http://rn0.ru/show/1430/</a> easily in sage?
<strong>2</strong>) And if yes how? </p>
<p>I already tried to plot some randomly created data: <a href="http://rn0.ru/show/1429/">http://rn0.ru/show/1429/</a> And that works great.</p>
<p><strong>3</strong>) How can I fit a function (nonlinear) with considering the errorbars?</p>
http://ask.sagemath.org/question/7909/plotting-and-fitting/?answer=12042#post-id-12042I'm not sure what you mean by "with considering the errorbars". Do you mean merely that you want the covariance of the fit (which leastsq returns), or that your data has errors in y (the usual model) and you want that to affect the fitting process, or that your data has errors in both x and y and you want that to affect the fit?
Anyway, here are two relatively straightforward ways to fit and get covariances.
sage: import numpy as np
sage: import scipy.optimize
sage: import scipy.odr
sage: import random
sage: random.seed(3)
sage:
sage: #
sage: # generate the data
sage: #
sage:
sage: x = np.linspace(0, 10, 100)
sage: y = np.sin(0.7*x)+2.4*x # underlying function
sage: y += np.random.randn(len(x))/10 # add some noise
sage: x_err = [0.1]*len(x) # estimated errors
sage: y_err = [0.3]*len(x) # estimated errors
sage:
sage: # first, we can use curve_fit
sage:
sage: def fit1(x, a, b):
....: return np.sin(x*a)+b*x
....:
sage: # note: can't handle errors in x
sage: popt, pcov = scipy.optimize.curve_fit(fit1, x, y, sigma=y_err)
sage: popt
array([ 0.70124584, 2.39768117])
sage: pcov
array([[ 6.97634001e-06, -2.58153187e-06],
[ -2.58153187e-06, 4.22745005e-06]])
sage:
sage: # or use the more powerful odr approach, which unfortunately has
sage: # some different conventions
sage:
sage: def fit2(B, x):
....: return np.sin(x*B[0])+B[1]*x
....:
sage: mymodel = scipy.odr.Model(fit2)
sage: mydata = scipy.odr.RealData(x, y, sx=x_err, sy=y_err)
sage: myodr = scipy.odr.ODR(mydata, mymodel, beta0=[1., 1.], maxit=1000)
sage: myoutput = myodr.run()
sage: myoutput.pprint()
Beta: [ 0.70139136 2.39769177]
Beta Std Error: [ 0.00258362 0.00198363]
Beta Covariance: [[ 9.08706871e-05 -2.35028948e-05]
[ -2.35028948e-05 5.35658971e-05]]
Residual Variance: 0.0734568446584
Inverse Condition #: 0.679929358499
Reason(s) for Halting:
Numerical error detected
Sat, 29 Jan 2011 16:53:18 -0600http://ask.sagemath.org/question/7909/plotting-and-fitting/?answer=12042#post-id-12042Comment by niles for <p>I'm not sure what you mean by "with considering the errorbars". Do you mean merely that you want the covariance of the fit (which leastsq returns), or that your data has errors in y (the usual model) and you want that to affect the fitting process, or that your data has errors in both x and y and you want that to affect the fit?</p>
<p>Anyway, here are two relatively straightforward ways to fit and get covariances.</p>
<pre><code>sage: import numpy as np
sage: import scipy.optimize
sage: import scipy.odr
sage: import random
sage: random.seed(3)
sage:
sage: #
sage: # generate the data
sage: #
sage:
sage: x = np.linspace(0, 10, 100)
sage: y = np.sin(0.7*x)+2.4*x # underlying function
sage: y += np.random.randn(len(x))/10 # add some noise
sage: x_err = [0.1]*len(x) # estimated errors
sage: y_err = [0.3]*len(x) # estimated errors
sage:
sage: # first, we can use curve_fit
sage:
sage: def fit1(x, a, b):
....: return np.sin(x*a)+b*x
....:
sage: # note: can't handle errors in x
sage: popt, pcov = scipy.optimize.curve_fit(fit1, x, y, sigma=y_err)
sage: popt
array([ 0.70124584, 2.39768117])
sage: pcov
array([[ 6.97634001e-06, -2.58153187e-06],
[ -2.58153187e-06, 4.22745005e-06]])
sage:
sage: # or use the more powerful odr approach, which unfortunately has
sage: # some different conventions
sage:
sage: def fit2(B, x):
....: return np.sin(x*B[0])+B[1]*x
....:
sage: mymodel = scipy.odr.Model(fit2)
sage: mydata = scipy.odr.RealData(x, y, sx=x_err, sy=y_err)
sage: myodr = scipy.odr.ODR(mydata, mymodel, beta0=[1., 1.], maxit=1000)
sage: myoutput = myodr.run()
sage: myoutput.pprint()
Beta: [ 0.70139136 2.39769177]
Beta Std Error: [ 0.00258362 0.00198363]
Beta Covariance: [[ 9.08706871e-05 -2.35028948e-05]
[ -2.35028948e-05 5.35658971e-05]]
Residual Variance: 0.0734568446584
Inverse Condition #: 0.679929358499
Reason(s) for Halting:
Numerical error detected
</code></pre>
http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22213#post-id-22213Thanks for providing this example :) I think criticizing someone's English is in poor taste though, since sage has such a broad international user base -- probably you just wanted the question to be more clear :)Sun, 30 Jan 2011 03:25:39 -0600http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22213#post-id-22213Comment by DSM for <p>I'm not sure what you mean by "with considering the errorbars". Do you mean merely that you want the covariance of the fit (which leastsq returns), or that your data has errors in y (the usual model) and you want that to affect the fitting process, or that your data has errors in both x and y and you want that to affect the fit?</p>
<p>Anyway, here are two relatively straightforward ways to fit and get covariances.</p>
<pre><code>sage: import numpy as np
sage: import scipy.optimize
sage: import scipy.odr
sage: import random
sage: random.seed(3)
sage:
sage: #
sage: # generate the data
sage: #
sage:
sage: x = np.linspace(0, 10, 100)
sage: y = np.sin(0.7*x)+2.4*x # underlying function
sage: y += np.random.randn(len(x))/10 # add some noise
sage: x_err = [0.1]*len(x) # estimated errors
sage: y_err = [0.3]*len(x) # estimated errors
sage:
sage: # first, we can use curve_fit
sage:
sage: def fit1(x, a, b):
....: return np.sin(x*a)+b*x
....:
sage: # note: can't handle errors in x
sage: popt, pcov = scipy.optimize.curve_fit(fit1, x, y, sigma=y_err)
sage: popt
array([ 0.70124584, 2.39768117])
sage: pcov
array([[ 6.97634001e-06, -2.58153187e-06],
[ -2.58153187e-06, 4.22745005e-06]])
sage:
sage: # or use the more powerful odr approach, which unfortunately has
sage: # some different conventions
sage:
sage: def fit2(B, x):
....: return np.sin(x*B[0])+B[1]*x
....:
sage: mymodel = scipy.odr.Model(fit2)
sage: mydata = scipy.odr.RealData(x, y, sx=x_err, sy=y_err)
sage: myodr = scipy.odr.ODR(mydata, mymodel, beta0=[1., 1.], maxit=1000)
sage: myoutput = myodr.run()
sage: myoutput.pprint()
Beta: [ 0.70139136 2.39769177]
Beta Std Error: [ 0.00258362 0.00198363]
Beta Covariance: [[ 9.08706871e-05 -2.35028948e-05]
[ -2.35028948e-05 5.35658971e-05]]
Residual Variance: 0.0734568446584
Inverse Condition #: 0.679929358499
Reason(s) for Halting:
Numerical error detected
</code></pre>
http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22212#post-id-22212Hmm, I didn't think of it that way at all: noting that something doesn't make sense in English doesn't seem like a criticism to me, but an observation. Still, I'll defer to your judgment.Sun, 30 Jan 2011 03:34:05 -0600http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22212#post-id-22212Comment by niles for <p>I'm not sure what you mean by "with considering the errorbars". Do you mean merely that you want the covariance of the fit (which leastsq returns), or that your data has errors in y (the usual model) and you want that to affect the fitting process, or that your data has errors in both x and y and you want that to affect the fit?</p>
<p>Anyway, here are two relatively straightforward ways to fit and get covariances.</p>
<pre><code>sage: import numpy as np
sage: import scipy.optimize
sage: import scipy.odr
sage: import random
sage: random.seed(3)
sage:
sage: #
sage: # generate the data
sage: #
sage:
sage: x = np.linspace(0, 10, 100)
sage: y = np.sin(0.7*x)+2.4*x # underlying function
sage: y += np.random.randn(len(x))/10 # add some noise
sage: x_err = [0.1]*len(x) # estimated errors
sage: y_err = [0.3]*len(x) # estimated errors
sage:
sage: # first, we can use curve_fit
sage:
sage: def fit1(x, a, b):
....: return np.sin(x*a)+b*x
....:
sage: # note: can't handle errors in x
sage: popt, pcov = scipy.optimize.curve_fit(fit1, x, y, sigma=y_err)
sage: popt
array([ 0.70124584, 2.39768117])
sage: pcov
array([[ 6.97634001e-06, -2.58153187e-06],
[ -2.58153187e-06, 4.22745005e-06]])
sage:
sage: # or use the more powerful odr approach, which unfortunately has
sage: # some different conventions
sage:
sage: def fit2(B, x):
....: return np.sin(x*B[0])+B[1]*x
....:
sage: mymodel = scipy.odr.Model(fit2)
sage: mydata = scipy.odr.RealData(x, y, sx=x_err, sy=y_err)
sage: myodr = scipy.odr.ODR(mydata, mymodel, beta0=[1., 1.], maxit=1000)
sage: myoutput = myodr.run()
sage: myoutput.pprint()
Beta: [ 0.70139136 2.39769177]
Beta Std Error: [ 0.00258362 0.00198363]
Beta Covariance: [[ 9.08706871e-05 -2.35028948e-05]
[ -2.35028948e-05 5.35658971e-05]]
Residual Variance: 0.0734568446584
Inverse Condition #: 0.679929358499
Reason(s) for Halting:
Numerical error detected
</code></pre>
http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22211#post-id-22211thanks -- this is probably just one of those things that came across a little harsher than it was meant.Sun, 30 Jan 2011 05:59:16 -0600http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22211#post-id-22211Answer by kcrisman for <p>Hi, I have to learn to plot and fit functions (with error bars in x and y direction). </p>
<p>My question is: <strong>1</strong>) Can I do it with sage? Would you prefer gnuplot?</p>
<p>How can I read and plot data like this: <a href="http://rn0.ru/show/1430/">http://rn0.ru/show/1430/</a> easily in sage?
<strong>2</strong>) And if yes how? </p>
<p>I already tried to plot some randomly created data: <a href="http://rn0.ru/show/1429/">http://rn0.ru/show/1429/</a> And that works great.</p>
<p><strong>3</strong>) How can I fit a function (nonlinear) with considering the errorbars?</p>
http://ask.sagemath.org/question/7909/plotting-and-fitting/?answer=12035#post-id-12035You could also use the R interface in Sage to produce things, using their standard fitting and error bar tools.
Edit: R is probably the industry standard for statistical/modeling/fitting stuff. Of the statistical variety. [http://www.r-project.org](http://www.r-project.org) is the site. You can do
sage: r?
for basic info, and see the whole documentation at [this page](http://hg.sagemath.org/sage-main/file/f24ce048fa66/sage/interfaces/r.py) for more info (apparently it's not in the reference manual yet. I also have a sample at [sagenb](http://www.sagenb.org/), but that site is so overloaded that it might be hard to access...
Fri, 28 Jan 2011 15:51:10 -0600http://ask.sagemath.org/question/7909/plotting-and-fitting/?answer=12035#post-id-12035Comment by kcrisman for <p>You could also use the R interface in Sage to produce things, using their standard fitting and error bar tools. </p>
<p>Edit: R is probably the industry standard for statistical/modeling/fitting stuff. Of the statistical variety. <a href="http://www.r-project.org">http://www.r-project.org</a> is the site. You can do </p>
<pre><code>sage: r?
</code></pre>
<p>for basic info, and see the whole documentation at <a href="http://hg.sagemath.org/sage-main/file/f24ce048fa66/sage/interfaces/r.py">this page</a> for more info (apparently it's not in the reference manual yet. I also have a sample at <a href="http://www.sagenb.org/">sagenb</a>, but that site is so overloaded that it might be hard to access...</p>
http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22220#post-id-22220This should work in the notebook, incidentally, if that's where you 'live'.Fri, 28 Jan 2011 15:51:26 -0600http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22220#post-id-22220Comment by manifold for <p>You could also use the R interface in Sage to produce things, using their standard fitting and error bar tools. </p>
<p>Edit: R is probably the industry standard for statistical/modeling/fitting stuff. Of the statistical variety. <a href="http://www.r-project.org">http://www.r-project.org</a> is the site. You can do </p>
<pre><code>sage: r?
</code></pre>
<p>for basic info, and see the whole documentation at <a href="http://hg.sagemath.org/sage-main/file/f24ce048fa66/sage/interfaces/r.py">this page</a> for more info (apparently it's not in the reference manual yet. I also have a sample at <a href="http://www.sagenb.org/">sagenb</a>, but that site is so overloaded that it might be hard to access...</p>
http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22218#post-id-22218hmmm ok... I don't know R... how is it?Sat, 29 Jan 2011 07:10:49 -0600http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22218#post-id-22218Answer by Shashank for <p>Hi, I have to learn to plot and fit functions (with error bars in x and y direction). </p>
<p>My question is: <strong>1</strong>) Can I do it with sage? Would you prefer gnuplot?</p>
<p>How can I read and plot data like this: <a href="http://rn0.ru/show/1430/">http://rn0.ru/show/1430/</a> easily in sage?
<strong>2</strong>) And if yes how? </p>
<p>I already tried to plot some randomly created data: <a href="http://rn0.ru/show/1429/">http://rn0.ru/show/1429/</a> And that works great.</p>
<p><strong>3</strong>) How can I fit a function (nonlinear) with considering the errorbars?</p>
http://ask.sagemath.org/question/7909/plotting-and-fitting/?answer=12034#post-id-12034Have a look at find_fit on the page
http://www.sagemath.org/doc/reference/sage/numerical/optimize.htmlFri, 28 Jan 2011 14:56:19 -0600http://ask.sagemath.org/question/7909/plotting-and-fitting/?answer=12034#post-id-12034Comment by manifold for <p>Have a look at find_fit on the page
<a href="http://www.sagemath.org/doc/reference/sage/numerical/optimize.html">http://www.sagemath.org/doc/reference...</a></p>
http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22217#post-id-22217hmmm I think that is just for very easy fits... And they use the function from scipy: scipy.optimize.leastsq ; that I use currentyl... But I don' t get xhi^2 and the correlation matrix and all that.... and that would be importantSat, 29 Jan 2011 07:47:52 -0600http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22217#post-id-22217Comment by manifold for <p>Have a look at find_fit on the page
<a href="http://www.sagemath.org/doc/reference/sage/numerical/optimize.html">http://www.sagemath.org/doc/reference...</a></p>
http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22219#post-id-22219thanks! hmm.... I found a way to fit without error bars (for now..) with scipy.Sat, 29 Jan 2011 07:10:27 -0600http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22219#post-id-22219Answer by niles for <p>Hi, I have to learn to plot and fit functions (with error bars in x and y direction). </p>
<p>My question is: <strong>1</strong>) Can I do it with sage? Would you prefer gnuplot?</p>
<p>How can I read and plot data like this: <a href="http://rn0.ru/show/1430/">http://rn0.ru/show/1430/</a> easily in sage?
<strong>2</strong>) And if yes how? </p>
<p>I already tried to plot some randomly created data: <a href="http://rn0.ru/show/1429/">http://rn0.ru/show/1429/</a> And that works great.</p>
<p><strong>3</strong>) How can I fit a function (nonlinear) with considering the errorbars?</p>
http://ask.sagemath.org/question/7909/plotting-and-fitting/?answer=12032#post-id-12032Here is an old thread from sage-devel about [error bars on plots](http://www.mail-archive.com/sage-devel@googlegroups.com/msg39806.html), with some suggestions. One of them suggests using matplotlib, and perhaps this would also work for curve fitting.
Sorry for the short answer, but I don't know much about this :(Fri, 28 Jan 2011 07:02:36 -0600http://ask.sagemath.org/question/7909/plotting-and-fitting/?answer=12032#post-id-12032Comment by manifold for <p>Here is an old thread from sage-devel about <a href="http://www.mail-archive.com/sage-devel@googlegroups.com/msg39806.html">error bars on plots</a>, with some suggestions. One of them suggests using matplotlib, and perhaps this would also work for curve fitting.</p>
<p>Sorry for the short answer, but I don't know much about this :(</p>
http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22224#post-id-22224error bar plotting is as I show in this example: http://rn0.ru/show/1429/ not the problem. But fitting is...I Havn't found a clue how to fit a functionto data in sage...Fri, 28 Jan 2011 07:27:40 -0600http://ask.sagemath.org/question/7909/plotting-and-fitting/?comment=22224#post-id-22224