# Sagemath heap size limit [closed]

Hi all, I am not new to Python, but new to Sagemath.

My code:

v = MatrixSpace(GF(2), 2, 6)
size = binomial(4096,3)
g3 = []
for c in Combinations(range(4096), 3):
m = block_matrix(3, 1, [v[c], v[c], v[c]])
if m.rank() >= 4:
g3.append(c)


The variable "g3" raises up to about 20 GB in total. Sagemath catches "Memory Error" at some point; therefore, I divide "g3" into 1920 different parts to save. Now I need to process further with "g3", i.e. I need to assign "g3" parts again to some variables to use. The 1st solution that I think of is to create 1920 different variables to store the whole "g3" in my code; however, this way is a bit inconvenient.

Is there any better solution?

For example, increasing the limit of "list" size (python list type []), which might help me to save up to 11.444.858.880 lists in the list "g3" (about 11 billion is the size from binomial(4096,3)). I have a computer of 128 GB RAM, and it is very nice if I can utilize the computer's strength.

There is an old topic on this: trac.sagemath.org/ticket/6772. However, I do not really get the idea there.

I wish to be supported :-) Thank you a lot !!!

edit retag reopen merge delete

### Closed for the following reason question is not relevant or outdated by imnvsh close date 2019-02-12 00:14:10.824559

Why do you need to physically save such an object with an easy generator? We do not have the (maybe $10\times 100$ matrix) v, but even if the range condition makes a rare selection among the many involved c's, what do you further want to do with this list?

Trying to run your code in a fresh Sage session, one gets:

NameError: name 'v' is not defined


I edited my post adding v:

v = MatrixSpace(GF(2), 2, 6)


@dan_fulea, I would like to share with you further process. Example:

g3 = [(1, 2, 3), (1, 2, 4), (1, 2, 5), (2, 3, 5), (1, 3, 5)]


1) g3 is a list of all c involved in good combinations, i.e. condition m.rank() >= 4 is satisfied (its result is still a huge list, which we can see by running @slelievre suggestion sum(1 for g in g3), which almost non-stop) 2) I process each list in g3 to learn relationship of any pairs, e.g:

relatives =
{(1, 2): [3, 4, 5],
(1, 3): [2, 5],
(1, 4): ,
(1, 5): [2, 3],
(2, 3): [1, 5],
(2, 4): ,
(2, 5): [1, 3],
(3, 5): [2, 1]}


3) Consider (1,2,3), we know (1,2) can go with 4, or 5 besides 3. If all (1 ...(more)

Sort by » oldest newest most voted

To expand on @dan_fulea's comment, in general, when we have huge amounts of objects, we don't want to store them in a list!

Using iterators is generally preferred: we can run through an iterator, dealing with its element one at a time without storing all of them at once.

In the case at hand, having defined

sage: C = Combinations(range(4096), 3)


we see that, as you correctly recalled,

sage: C.cardinality()
11444858880


and we could define

sage: g3 = (c for c in C if block_matrix(3, 1, [v[c], v[c], v[c]]).rank() >= 4)


and count these combinations with

sage: sum(1 for g in g3)


and then if we had another thing to study about these combinations we could re-define

sage: g3 = (c for c in C if block_matrix(3, 1, [v[c], v[c], v[c]]).rank() >= 4)


and run through them once more.

more