How to slice an unknown length iterator
I'm trying to take chunks of an iterator, for which I don't know the length of. The underlying goal is that I have a generator function which makes piles and piles of graphs (via a generator/iterator) that I want to try and test for a property in parallel (because serially testing each element is fast, but the list is really long).
Here's the code I'm working with right now:
#Slice Size: slsize=500 grpgen=graphs.nauty_geng("10 -c 33:39 -d6") start=0 stop=slsize grplst=list() loopcnt=0 loopt=True while loopt: grplst.append(itertools.islice(grpgen,start,stop)) #print loopcnt try: g=grplst[loopcnt].next() print g except StopIteration: loopt=False print "got except" start+=slsize+1 stop+=slsize loopcnt+=1 print loopcnt #print grplst
This snippet will run ok. But as soon as I uncomment the "print grplst" at the end, it raises a strange 'resource unavailable' error. The error is raised on the 'try command' (g=grplst[loopcnt].next()) .... not when I actually print the grplst. And, it happens while _preparsing_ my file. (I'm loading this from a separate file).
When I leave it commented out though, it's clearly going through the 'try' command several times successfully before correctly catching the exception. It just seems that when I want to USE the list I've been creating of iterators, then it doesn't like it!