ASKSAGE: Sage Q&A Forum - Individual question feedhttp://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Thu, 08 Dec 2011 03:50:06 -0600a fast function taking either variable or collection as argumenthttp://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/I want to create a function which takes either variable or collection as argument.
If the input is any kind of collection, function should be taken of every element in it and return a numpy array of results.
Currently, I do in this way:
def simplefunc(x):
<some code here> # it is assumed that x is a single variable
return <some single result>
def arrayfunc(x):
try: #check if we have a collection
x_iterator = iter(x)
except TypeError: #this is a single expression
return simplefunc(x)
else: # iterate through iterable x
ret=[]
for xx in x:
ret.append(simplefunc(xx))
return numpy.array(ret)
It works as is, however, I do not think this is the fastest way possible,especially the method to figure out if the input is a collection. I also do not like that strings are considered as collections and split to chars (I can stand it though). Is there a more elegant way?
I tried also to call arrayfunc() recursively on each element of collection (instead of simplefunc()), to handle collections of collections in the same way, however, it runs into infinite loop on strings. I have to check if the input is a string explicitly.
Wed, 07 Dec 2011 05:58:12 -0600http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/Comment by Jason Grout for <p>I want to create a function which takes either variable or collection as argument.
If the input is any kind of collection, function should be taken of every element in it and return a numpy array of results.
Currently, I do in this way:</p>
<pre><code>def simplefunc(x):
<some code here> # it is assumed that x is a single variable
return <some single result>
def arrayfunc(x):
try: #check if we have a collection
x_iterator = iter(x)
except TypeError: #this is a single expression
return simplefunc(x)
else: # iterate through iterable x
ret=[]
for xx in x:
ret.append(simplefunc(xx))
return numpy.array(ret)
</code></pre>
<p>It works as is, however, I do not think this is the fastest way possible,especially the method to figure out if the input is a collection. I also do not like that strings are considered as collections and split to chars (I can stand it though). Is there a more elegant way?</p>
<p>I tried also to call arrayfunc() recursively on each element of collection (instead of simplefunc()), to handle collections of collections in the same way, however, it runs into infinite loop on strings. I have to check if the input is a string explicitly.</p>
http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?comment=20735#post-id-20735Your function above doesn't have non-sequence parameters, so I'm not sure how you plan to deal with those. You could always make a partial function (look up partial in the functools package). List comprehensions deal nicely with this too.Thu, 08 Dec 2011 02:33:01 -0600http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?comment=20735#post-id-20735Comment by ADuC812 for <p>I want to create a function which takes either variable or collection as argument.
If the input is any kind of collection, function should be taken of every element in it and return a numpy array of results.
Currently, I do in this way:</p>
<pre><code>def simplefunc(x):
<some code here> # it is assumed that x is a single variable
return <some single result>
def arrayfunc(x):
try: #check if we have a collection
x_iterator = iter(x)
except TypeError: #this is a single expression
return simplefunc(x)
else: # iterate through iterable x
ret=[]
for xx in x:
ret.append(simplefunc(xx))
return numpy.array(ret)
</code></pre>
<p>It works as is, however, I do not think this is the fastest way possible,especially the method to figure out if the input is a collection. I also do not like that strings are considered as collections and split to chars (I can stand it though). Is there a more elegant way?</p>
<p>I tried also to call arrayfunc() recursively on each element of collection (instead of simplefunc()), to handle collections of collections in the same way, however, it runs into infinite loop on strings. I have to check if the input is a string explicitly.</p>
http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?comment=20738#post-id-20738Thanks a lot... this I may have overlooked while googling.. However, this does not work if I have some other non-sequence parameters in my function
Maybe better to post this as an answer?Wed, 07 Dec 2011 17:59:49 -0600http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?comment=20738#post-id-20738Comment by John Palmieri for <p>I want to create a function which takes either variable or collection as argument.
If the input is any kind of collection, function should be taken of every element in it and return a numpy array of results.
Currently, I do in this way:</p>
<pre><code>def simplefunc(x):
<some code here> # it is assumed that x is a single variable
return <some single result>
def arrayfunc(x):
try: #check if we have a collection
x_iterator = iter(x)
except TypeError: #this is a single expression
return simplefunc(x)
else: # iterate through iterable x
ret=[]
for xx in x:
ret.append(simplefunc(xx))
return numpy.array(ret)
</code></pre>
<p>It works as is, however, I do not think this is the fastest way possible,especially the method to figure out if the input is a collection. I also do not like that strings are considered as collections and split to chars (I can stand it though). Is there a more elegant way?</p>
<p>I tried also to call arrayfunc() recursively on each element of collection (instead of simplefunc()), to handle collections of collections in the same way, however, it runs into infinite loop on strings. I have to check if the input is a string explicitly.</p>
http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?comment=20739#post-id-20739Perhaps using 'map(function, sequence)' would help? 'try: L = map(simplefunc, x) ...'. See http://docs.python.org/tutorial/datastructures.html#functional-programming-toolsWed, 07 Dec 2011 06:15:00 -0600http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?comment=20739#post-id-20739Answer by Jason Grout for <p>I want to create a function which takes either variable or collection as argument.
If the input is any kind of collection, function should be taken of every element in it and return a numpy array of results.
Currently, I do in this way:</p>
<pre><code>def simplefunc(x):
<some code here> # it is assumed that x is a single variable
return <some single result>
def arrayfunc(x):
try: #check if we have a collection
x_iterator = iter(x)
except TypeError: #this is a single expression
return simplefunc(x)
else: # iterate through iterable x
ret=[]
for xx in x:
ret.append(simplefunc(xx))
return numpy.array(ret)
</code></pre>
<p>It works as is, however, I do not think this is the fastest way possible,especially the method to figure out if the input is a collection. I also do not like that strings are considered as collections and split to chars (I can stand it though). Is there a more elegant way?</p>
<p>I tried also to call arrayfunc() recursively on each element of collection (instead of simplefunc()), to handle collections of collections in the same way, however, it runs into infinite loop on strings. I have to check if the input is a string explicitly.</p>
http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?answer=12980#post-id-12980As for the else statement, this is probably faster:
else:
return numpy.array([simplefunc(xx) for xx in x])
I probably wouldn't even put that in an else statement either. Just put it as the last line of the function.
**Update**
It looks like map is slightly faster than a list comprehension in the test below:
sage: def f(x): return x*x
....:
sage: timeit('map(f,xrange(1e6))')
5 loops, best of 3: 223 ms per loop
sage: timeit('[f(x) for x in xrange(1e6)]')
5 loops, best of 3: 277 ms per loop
Wed, 07 Dec 2011 06:12:45 -0600http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?answer=12980#post-id-12980Comment by ADuC812 for <p>As for the else statement, this is probably faster:</p>
<pre><code>else:
return numpy.array([simplefunc(xx) for xx in x])
</code></pre>
<p>I probably wouldn't even put that in an else statement either. Just put it as the last line of the function.</p>
<p><strong>Update</strong></p>
<p>It looks like map is slightly faster than a list comprehension in the test below:</p>
<pre><code>sage: def f(x): return x*x
....:
sage: timeit('map(f,xrange(1e6))')
5 loops, best of 3: 223 ms per loop
sage: timeit('[f(x) for x in xrange(1e6)]')
5 loops, best of 3: 277 ms per loop
</code></pre>
http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?comment=20737#post-id-20737is this variant: map(function, sequence) from comment above slower?Wed, 07 Dec 2011 18:09:46 -0600http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?comment=20737#post-id-20737Answer by jdc for <p>I want to create a function which takes either variable or collection as argument.
If the input is any kind of collection, function should be taken of every element in it and return a numpy array of results.
Currently, I do in this way:</p>
<pre><code>def simplefunc(x):
<some code here> # it is assumed that x is a single variable
return <some single result>
def arrayfunc(x):
try: #check if we have a collection
x_iterator = iter(x)
except TypeError: #this is a single expression
return simplefunc(x)
else: # iterate through iterable x
ret=[]
for xx in x:
ret.append(simplefunc(xx))
return numpy.array(ret)
</code></pre>
<p>It works as is, however, I do not think this is the fastest way possible,especially the method to figure out if the input is a collection. I also do not like that strings are considered as collections and split to chars (I can stand it though). Is there a more elegant way?</p>
<p>I tried also to call arrayfunc() recursively on each element of collection (instead of simplefunc()), to handle collections of collections in the same way, however, it runs into infinite loop on strings. I have to check if the input is a string explicitly.</p>
http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?answer=12982#post-id-12982I agree with the other responders on using 'map'. As to the other issue, detecting if x is a collection: are you attempting to construct a very general tool that can be used in just about any context, or just something that will work for the solution of a particular problem? If it's the latter, then (depending on your particular context) you might only encounter certain types of collections. For example, if the only kinds of collections you expect to see are lists, tuples, or sets, you could define
>itertypes = ["<type 'tuple'>", " <type 'list'>", " <class 'sage.sets.set.Set_object_enumerated_with_category'>"]
then just test if str(type(x)) is in itertypes.
I did a smidge of benchmarking, using a few basic examples, comparing this approach to your "try... except TypeError" approach. (I only tested the step where you determine whether x is a collection or not, omitting the step where you actually apply 'simplefunc'.) When x is a list/tuple/set, my method is only about half as fast as yours, but otherwise it's 3 times as fast. So which is faster might depend on the proportion of the time you expect to encounter collections.
By the way, this may not be a worry for you, but I don't think that Sage would raise any objection if you ask it to iterate over an infinite set like ZZ. It just wouldn't terminate.Thu, 08 Dec 2011 01:58:44 -0600http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?answer=12982#post-id-12982Comment by Jason Grout for <p>I agree with the other responders on using 'map'. As to the other issue, detecting if x is a collection: are you attempting to construct a very general tool that can be used in just about any context, or just something that will work for the solution of a particular problem? If it's the latter, then (depending on your particular context) you might only encounter certain types of collections. For example, if the only kinds of collections you expect to see are lists, tuples, or sets, you could define</p>
<blockquote>
<p>itertypes = ["<type 'tuple'>", " <type 'list'>", " <class 'sage.sets.set.Set_object_enumerated_with_category'>"]</p>
</blockquote>
<p>then just test if str(type(x)) is in itertypes.</p>
<p>I did a smidge of benchmarking, using a few basic examples, comparing this approach to your "try... except TypeError" approach. (I only tested the step where you determine whether x is a collection or not, omitting the step where you actually apply 'simplefunc'.) When x is a list/tuple/set, my method is only about half as fast as yours, but otherwise it's 3 times as fast. So which is faster might depend on the proportion of the time you expect to encounter collections.</p>
<p>By the way, this may not be a worry for you, but I don't think that Sage would raise any objection if you ask it to iterate over an infinite set like ZZ. It just wouldn't terminate.</p>
http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?comment=20736#post-id-20736Isn't easier and clearer to just test for the types directly using isinstance? Like: isinstance(x, (list, tuple))Thu, 08 Dec 2011 02:31:36 -0600http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?comment=20736#post-id-20736Comment by jdc for <p>I agree with the other responders on using 'map'. As to the other issue, detecting if x is a collection: are you attempting to construct a very general tool that can be used in just about any context, or just something that will work for the solution of a particular problem? If it's the latter, then (depending on your particular context) you might only encounter certain types of collections. For example, if the only kinds of collections you expect to see are lists, tuples, or sets, you could define</p>
<blockquote>
<p>itertypes = ["<type 'tuple'>", " <type 'list'>", " <class 'sage.sets.set.Set_object_enumerated_with_category'>"]</p>
</blockquote>
<p>then just test if str(type(x)) is in itertypes.</p>
<p>I did a smidge of benchmarking, using a few basic examples, comparing this approach to your "try... except TypeError" approach. (I only tested the step where you determine whether x is a collection or not, omitting the step where you actually apply 'simplefunc'.) When x is a list/tuple/set, my method is only about half as fast as yours, but otherwise it's 3 times as fast. So which is faster might depend on the proportion of the time you expect to encounter collections.</p>
<p>By the way, this may not be a worry for you, but I don't think that Sage would raise any objection if you ask it to iterate over an infinite set like ZZ. It just wouldn't terminate.</p>
http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?comment=20734#post-id-20734It sure is. I just didn't know about isinstance.Thu, 08 Dec 2011 03:50:06 -0600http://ask.sagemath.org/question/8536/a-fast-function-taking-either-variable-or-collection-as-argument/?comment=20734#post-id-20734