Problem in memory usage when loading file (Memory leak?)
I need to get coefficients of some polynomials save in my disk. However, the memory usage is increasing in each iteration although garbage collector is used. Here is my program. The memory looks increasing in the command pol = load(f"{foldername}pp{2}.sobj")
def get_co(filename, Mon):
pol = load(filename)
co = [pol.coefficient(m) for m in Mon]
del pol
gc.collect()
return co
def load_monomials(n):
foldername = f"mult/p{n}/"
diname = f"mon/di{n}"
os.makedirs(diname, exist_ok=True) # Ensure directory exists
k = 0
filename = f"{foldername}pp{k}.sobj"
# Take set of monomials with different coefficients
Mon = set()
# Call the polynomials of index 2 (we want random, but not 0 actually)
pol = load(f"{foldername}pp{2}.sobj")
print("Taking the monomials with different coefficients")
Co = set()
for mon in pol.monomials():
coe = pol.coefficient(mon)
if coe not in Co:
Co.add(coe)
Mon.add(mon)
print("The length of monomials is", len(Mon))
while os.path.exists(filename):
print(f"Loading {filename}...")
coname = f"{diname}/co{k}.sobj"
co = get_co(filename, Mon)
save(co, coname)
# Explicitly dereference
del co
gc.collect()
print(f"Memory usage: {get_memory_usage()} MB, GC stats: {gc.get_stats()}")
k += 1
filename = f"{foldername}pp{k}.sobj"
load_monomials(160)
Can you create a minimal example illustrating the problem? (Perhaps, loading the same polynomial in the loop.) Then the issue can be reported at https://github.com/sagemath/sage/issues
For instance:
Even loading the file only, the memory usage keep increasing.
Please report the issue at https://github.com/sagemath/sage/issues
As a workaround, you may try to add
@fork
decorator to yourget_co
function: