How to slice an unknown length iterator

savecancel

I'm trying to take chunks of an iterator, for which I don't know the length of. The underlying goal is that I have a generator function which makes piles and piles of graphs (via a generator/iterator) that I want to try and test for a property in parallel (because serially testing each element is fast, but the list is really long).

Here's the code I'm working with right now:

#Slice Size:
slsize=500

grpgen=graphs.nauty_geng("10 -c 33:39 -d6")
start=0
stop=slsize

grplst=list()
loopcnt=0
loopt=True
while loopt:

   grplst.append(itertools.islice(grpgen,start,stop))
   #print loopcnt

   try:     
       g=grplst[loopcnt].next()
       print g
   except StopIteration:
       loopt=False
       print "got except"

   start+=slsize+1
   stop+=slsize
   loopcnt+=1

print loopcnt
#print grplst

This snippet will run ok. But as soon as I uncomment the "print grplst" at the end, it raises a strange 'resource unavailable' error. The error is raised on the 'try command' (g=grplst[loopcnt].next()) .... not when I actually print the grplst. And, it happens while _preparsing_ my file. (I'm loading this from a separate file).

When I leave it commented out though, it's clearly going through the 'try' command several times successfully before correctly catching the exception. It just seems that when I want to USE the list I've been creating of iterators, then it doesn't like it!