I am trying to run a very long (weeks long) process that constructs a large collection of graphs.
The collection is too large to keep in memory and so as each graph is found it is stored on disk (rather than in an array).
However when I set it running, I noticed that the amount of RAM used by the process was continuously increasing until the rest of the machine ground to a halt. My computer has 64Gb RAM and it takes a bit more than a day for the process to grow to that size.
I have tried to isolate the part of the program that was causing the memory usage to grow, and I think that I have isolated it to the canonical labelling algorithm "bliss".
Here is a simple program that just creates random graphs and canonically labels them (but does nothing more):
for it in range(1000000):
g = graphs.RandomGNP(20,0.5)
gc = g.canonical_label(algorithm="bliss")
if it % 1000 == 0:
print(it)
When I run this program the memory consumed grows and grows.
If I change the program to use the built-in Sage method,
for it in range(1000000):
g = graphs.RandomGNP(20,0.5)
gc = g.canonical_label(algorithm="sage")
if it % 1000 == 0:
print(it)
then the program runs to completion with no increase in memory used - the memory is correctly being recovered as it goes out of scope each time through the loop.
What should I do in this situation?
