RAM problem in script execution
I wrote the next code. In 1-2 hours of execution time the RAM of my laptop (8gb) is filled and the sistem crash:
from scipy.stats import uniform import numpy as np cant_de_cadenas=[400,600,700,800] os.makedirs('directory') np.savetxt(here 'cant_de_cadenas' is saved as .csv in folder 'directory') A1=mp.array() ... A6=mp.array() import time for N in cant_de_cadenas: start=time.clock() B1=mp.array() ... B6=mp.array() for u in srange (100): array_1 = uniform.rvs(loc=-2,scale=4, size=N) array_2 = uniform.rvs(loc=-2,scale=4, size=N) array_3 = uniform.rvs(loc=-1,scale=7, size=N) array_4 = 1/np.tan(array_3) array_5 = uniform.rvs(loc=-1,scale=1, size=N) array_6 = function(array_5) array_7 = function(array_6, array_5 and array_4) array_8 = function(array_1 and array_7) array_9 = function(array_2 and array_7) M=np.zeros([N+4,N+4]) for j in srange(N+4): if j>0: two arrays (C and D) with len=j-1 are created for k in srange (j): if C[k]<=0 and D[k]<=0: M[j,k]=1 if j+1<N+4: two arrays (C and D) with len=((N+4)-j-1) are created for k in srange ((N+4)-j-1): if C[k]<=0 and D[k]<=0: M[j,k+j+1]=1 An algorithm with matrix M is executed and values 'b1' and 'b2' are generated M_hor=M.copy() Some values in M_hor are changed and an algorithm with matrix M_hor is executed and values 'b3' and 'b4' are generated M_ver=M.copy() Some values in M_hor are changed and an algorithm with matrix M_hor is executed and values 'b5' and 'b6' are generated B1=np.append(B1,b1) ... B6=np.append(B6,b6) A1=np.append(A1,sum(B1)/100) ... A6=np.append(A6,sum(B6)/100)
Like you can see, len(A1)=...=len(A6)=len( cant_de_cadenas) (because in An are the average values of 100 repetitions -included in Bn-).
While many arrays are created (array_1,array_2 etc , with lenght N), in each one of the 100 cycles 'for u in srange (100)' these arrays are overwritten (with the same name). The same applies to B1,...,B6 arrays: in each one of the len(cant_de_cadenas) cyles 'for N in srange (cant_de_cadenas)' these arrays are overwritten (with the same name).
I tryed with gc and gc.collect() and nothing work! In addition, i cant use memory_profiler module in sage.
What am I doing wrong? Why the memory becomes full while running (starts with 10% of RAM used and in 1-2hour is totally full used)?
Please help me, I'm totally stuck!
Thanks a lot!