ASKSAGE: Sage Q&A Forum - RSS feedhttps://ask.sagemath.org/questions/Q&A Forum for SageenCopyright Sage, 2010. Some rights reserved under creative commons license.Mon, 04 Mar 2019 17:08:10 +0100Save/load huge dictionaryhttps://ask.sagemath.org/question/45651/saveload-huge-dictionary/I have a huge dictionary about 50 GB. After generating this dictionary, I do not have any space left on my memory. I still run Sagemath standard `save(mydict, 'myfile')`. However, the save runs almost forever.
What should I do? Storing it in multiple files is also fine to me. I really need to load it again to use in the future.
Maybe another approach is helpful. Besides the above dictionary, I have another huge redundant dictionary `mydict2`, which I tried using `del mydict2` to get some extra memory for the above Sagemath `save`; however, the memory usage still stays the same as before calling `del mydict2`. I guess its keys are still stored in memory. I do not need keys from `mydict2`, but its value is used in `mydict`.imnvshMon, 04 Mar 2019 17:08:10 +0100https://ask.sagemath.org/question/45651/Sagemath heap size limithttps://ask.sagemath.org/question/45347/sagemath-heap-size-limit/Hi all, I am not new to Python, but new to Sagemath.
My code:
v = MatrixSpace(GF(2), 2, 6)
size = binomial(4096,3)
g3 = []
for c in Combinations(range(4096), 3):
m = block_matrix(3, 1, [v[c[0]], v[c[1]], v[c[2]]])
if m.rank() >= 4:
g3.append(c)
The variable "g3" raises up to about 20 GB in total. Sagemath catches "Memory Error" at some point; therefore, I divide "g3" into 1920 different parts to save. Now I need to process further with "g3", i.e. I need to assign "g3" parts again to some variables to use. The 1st solution that I think of is to create 1920 different variables to store the whole "g3" in my code; however, this way is a bit inconvenient.
Is there any better solution?
For example, increasing the limit of "list" size (python list type []), which might help me to save up to 11.444.858.880 lists in the list "g3" (about 11 billion is the size from binomial(4096,3)). I have a computer of 128 GB RAM, and it is very nice if I can utilize the computer's strength.
There is an old topic on this: trac.sagemath.org/ticket/6772. However, I do not really get the idea there.
I wish to be supported :-) Thank you a lot !!!imnvshWed, 06 Feb 2019 23:43:15 +0100https://ask.sagemath.org/question/45347/