I have a very large file and I read it as follows:
with open('/Users/jianrongli/Downloads/SmallRank4ModulesGr416.txt', 'r') as fp:
L = [sage_eval(line) for line in fp.readlines() if line.strip()]
The file is very large (more than 3 G). Each element in L is a 2d array like [[1,2,3,5],[2,3,4,9]]. Now I want to select those in L whose max number is less than 12.
r1=[]
for j in b2:
t1=[]
for i in j:
#print(t1,list(i))
t1=t1+list(i)
if max(t1)<=12:
r1.append(j)
len(r1)
Since the file is very large, it takes a few hours and has not finished. Is there some way to make it faster? Thank you very much.
