Ask Your Question
0

How to speed up this code

asked 2022-12-08 06:13:33 +0100

sgmth gravatar image

updated 2022-12-08 09:02:10 +0100

Given a square matrix M, to find (say) five sets of, (say) 4 matrices A1,A2,A3,A4 such that

(i) each Ai is a unique 0,1 or 0,-1 matrix

(ii) each Ai is not - all zeroes or all ones or all minus ones matrix(that is, matrices with all entries same are not allowed)

(iii) M= A1+A2+A3+A4

(iv) Each matrix product AiAj (i,j=1,2,3,4)is a linear combination of A1,A2,A3,A4 with integer coefficients.

This is the same problem which was discussed in :

https://ask.sagemath.org/question/642...

But there we wanted “all possible combinations” satisfying the required conditions. There we basically discussed the process for first getting all combinations satisfying (iii) and then checked condition(iv). But for 3rd order and above no output even after one week.

So thought that rather than first collecting all combinations satisfying(iii) it would be better to check (iv) alongside individually and after say, five outputs , the program ends, as even this much would also help me for my work . I have made the following code:

#Given Matrix 
 M = matrix(2,2, [1, 2, 2, -1])
 show("M=",M)
 n = int(input("Enter the order of the matrix M : "))

 T = Tuples((0,1),(n^2))


 #Removing all-zero and all-one tuples as not needed for our case"
 R=T.list()[1:-1]

 #Forming n by n matrices from  list in R. Contains all {0,1 }matrices.
 P = []
 for v in R:    
       P.append(matrix(ZZ,n,n,v))

 #Forming all n by n , {0,-1} matrices.    
 N=[]
 for a in P:
      b=-a
 N.append(b)   

 test_sum = int(input("In how many matrices you want to break M :"))

 num_nonneg = int(input("How many matrices with non-negative entries you want :"))
 num_nonpos = (test_sum)-(num_nonneg)           

 nonneg_comb = Combinations(P, num_nonneg)
 nonpos_comb = Combinations(N, num_nonpos )

#Defining a function which converts matrix to vector

def mat_to_vector(m):
        return vector( sum(map(list,m.rows()), []), immutable=True )

Counter=0
X = ZZ^(n^2)

for a in nonneg_comb:
      for b in nonpos_comb:
      flag=0
      V = X.span(mat_to_vector((a+b)[i]) for i in range(len((a+b))))
      for i in range(test_sum):
            for j in range(test_sum):   

                if( (sum(a+b)!=M) or  (mat_to_vector((a+b)[i]*(a+b)[j]) not  in V)):
                flag=1

    if(flag==0):

                Counter+=1
                show("(",Counter,").",  a+b)  

    if Counter==5:
         break;

But this is also time taking for higher orders. So, wanted to know how it can be speeded up or how the code given by Max Alekseyev in the above link can accordingly be edited.

edit retag flag offensive close merge delete

1 Answer

Sort by » oldest newest most voted
2

answered 2022-12-21 17:28:11 +0100

cmark gravatar image

updated 2023-01-05 19:30:41 +0100

I couldn't run your original code as-is, I had to make some modifications to it. After that I've made some more (mostly vectorizations) to speed it up, I think it's siginificant (it also goes easier on the machine due to the aforementioned changes). There you go, give it a try:

# Given Matrix
M = matrix(2, 2, [1, 2, 2, -1])
show("M=", M)
n = int(input("Enter the order of the matrix M: "))

T = Tuples((0, 1), (n**2))

# Removing all-zero and all-one tuples as not needed for our case
R = T.list()[1:-1]

# Forming n by n matrices from list in R. Contains all {0, 1} matrices.
P = [matrix(ZZ, n, n, v) for v in R]

# Forming all n by n, {0, -1} matrices.
N = [-a for a in P]

test_sum = int(input("In how many matrices you want to break M: "))

num_nonneg = int(input("How many matrices with non-negative entries you want: "))
num_nonpos = test_sum - num_nonneg

nonneg_comb = Combinations(P, num_nonneg)
nonpos_comb = Combinations(N, num_nonpos)

# Defining a function which converts matrix to vector
def mat_to_vector(m):
    return vector(sum(map(list, m.rows()), []), immutable=True)

# Convert all matrices in P and N to vectors
P_vectors = [mat_to_vector(m) for m in P]
N_vectors = [mat_to_vector(m) for m in N]

# Create the span of all possible vectors
X = ZZ**(n**2)
V = X.span(P_vectors + N_vectors)

# Initialize counter
Counter = 0

# Iterate over all combinations of positive and negative matrices
for a in nonneg_comb:
    for b in nonpos_comb:
        # Check if the sum of the matrices is equal to M and all possible pairwise products are in the span
        if (sum(a+b) == M) and all(mat_to_vector(m1 * m2) in V for m1, m2 in itertools.product(a+b, repeat=2)):
            Counter += 1
            show("(", Counter, ").", a+b)
        if Counter == 5:
            break

This should allow the code to run significantly faster on a machine with multiple CPU cores. Please see the new code:

import itertools

# Given Matrix
#M = matrix(2, 2, [1, 2, 2, -1])
M = matrix(2, 2, [1, 1, 1, -1])
show("M=", M)
n = int(input("Enter the order of the matrix M: "))

T = Tuples((0, 1), (n**2))

# Removing all-zero and all-one tuples as not needed for our case
R = T.list()[1:-1]

# Forming n by n matrices from list in R. Contains all {0, 1} matrices.
P = [matrix(ZZ, n, n, v) for v in R]

# Forming all n by n, {0, -1} matrices.
N = [-a for a in P]

test_sum = int(input("In how many matrices you want to break M: "))

num_nonneg = int(input("How many matrices with non-negative entries you want: "))
num_nonpos = test_sum - num_nonneg

nonneg_comb = Combinations(P, num_nonneg)
nonpos_comb = Combinations(N, num_nonpos)

# Defining a function which converts matrix to vector
def mat_to_vector(m):
    return vector(sum(map(list, m.rows()), []), immutable=True)

# Convert all matrices in P and N to vectors
P_vectors = [mat_to_vector(m) for m in P]
N_vectors = [mat_to_vector(m) for m in N]

# Create the span of all possible vectors
X = ZZ**(n**2)
V = X.span(P_vectors + N_vectors)

# Initialize counter
Counter = 0

# Iterate over all combinations of positive and negative matrices
for a in nonneg_comb:
    for b in nonpos_comb:
        # Check if the sum of the matrices is equal to M and all possible pairwise products are in the span
        if (sum(a+b) == M) and all(mat_to_vector(m1 * m2) in V for m1, m2 in itertools.product(a+b, repeat=2)):
            Counter += 1
            show("(", Counter, ").", a+b)
        if Counter == 5:
            break

My final iteration, using the 'parallel' module:

import itertools
from sage.parallel.decorate import parallel

# Given Matrix
#M = matrix(2, 2, [1, 2, 2, -1])
M = matrix(2, 2, [1, 1, 1, -1])
show("M=", M)
n = int(input("Enter the order of the matrix M: "))

T = Tuples((0, 1), (n**2))

# Removing all-zero and all-one tuples as not needed for our case
R = T.list()[1:-1]

# Forming n by n matrices from list in R. Contains all {0, 1} matrices.
P = [matrix(ZZ, n, n, v) for v in R]

# Forming all n by n, {0, -1} matrices.
N = [-a for a in P]

test_sum = int(input("In how many matrices you want to break M: "))

num_nonneg = int(input("How many matrices with non-negative entries you want: "))
num_nonpos = test_sum - num_nonneg

nonneg_comb = Combinations(P, num_nonneg)
nonpos_comb = Combinations(N, num_nonpos)

# Define a function to convert a matrix to a vector
def mat_to_vector(m):
    return vector(sum(map(list, m.rows()), []), immutable=True)

# Convert all matrices in P and N to vectors
P_vectors = [mat_to_vector(m) for m in P]
N_vectors = [mat_to_vector(m) for m in N]

# Create the span of all possible vectors
X = ZZ**(n**2)
V = X.span(P_vectors + N_vectors)

# Initialize counter
Counter = 0

# Iterate over all combinations of positive and negative matrices
@parallel
def process_combinations(a, b):
    global Counter
    # Check if the sum of the matrices is equal to M and all possible pairwise products are in the span
    if (sum(a+b) == M) and all(mat_to_vector(m1 * m2) in V for m1, m2 in itertools.product(a+b, repeat=2)):
        Counter += 1
        show("(", Counter, ").", a+b)

for a in nonneg_comb:
    for b in nonpos_comb:
        process_combinations(a, b)
        if Counter == 5:
            break

Please note that you may need to adjust the number of worker processes used by the parallel decorator depending on the number of CPU cores available on your machine, like so:

@parallel(ncpus=4)
def process_combinations(

You can give this a go, too.

edit flag offensive delete link more

Comments

Thanks a lot for your effort and time. Yes , this is certainly faster. Tried for 2nd order matrix. Now have run for 4th order. It is still running for the last few hours. Lets' see how long does it take. Can we break the code into , (say) two parts, so that we may first run the first part and then taking its output as input for the second part ? Because I have access to a supercomputer , but there is time limit for running a code. Or can the code be written in parallel to make optimum use of supercomputer ?

sgmth gravatar imagesgmth ( 2022-12-31 16:25:34 +0100 )edit
1

I've edited my answer with a modfied code, have a go at it.

cmark gravatar imagecmark ( 2022-12-31 18:32:49 +0100 )edit

Thanks a lot again for looking into my problem and modifying the code. Let me mention that I have only elementary knowledge of sagemath and absolutely no knowledge of parallel processing. But I think there is typo error in the first line. If I am not mistaken it should be 'multiprocessing' instead of 'iprocessing'. Also 'import itertools' is to be added. When I ran the code after making these changes, I got the error that "check_combination() takes 1 positional argument but 2 were given". I am not able to figure it out. Kindly look into it. Also, right now there is some problem with the supercomputer,I will be able to use it most probably by this weekend.

sgmth gravatar imagesgmth ( 2023-01-03 12:48:34 +0100 )edit

Also, the specifications of the supercomputer I will be using and the commands for requesting the resources in mentioned in the link

https://ask.sagemath.org/question/638...

Kindly guide me how to choose the resources for this code which you have made.

sgmth gravatar imagesgmth ( 2023-01-03 12:57:38 +0100 )edit
1

There was indeed a typo in the beginning of the code. The itertools module was indeed missing, sorry about them (I'm working on this code off the top of my head). Please see the new iteration, I've changed the necessary lines.

cmark gravatar imagecmark ( 2023-01-03 18:04:58 +0100 )edit

Thanks a lot again. I really appreciate the time and effort you are devoting to my problem, I ran the latest code and getting this error message "name 'a' is not defined", pointing towards the line 'show("(", Counter, ").", a+b)'. I tried to make some changes and work it out, but failed to do so.

sgmth gravatar imagesgmth ( 2023-01-04 07:21:00 +0100 )edit
1

Sorry about that, the 'a' and 'b' variables were defined out of scope, now I've edited the code again (also could ran this time, it did run just fine for me), please give it a try (now you see two codes in my answer, use the one under "Please see new code").

cmark gravatar imagecmark ( 2023-01-04 22:13:07 +0100 )edit

Sorry, but I am getting the same error now also.

sgmth gravatar imagesgmth ( 2023-01-05 06:44:47 +0100 )edit
1

Are you quite sure you ran the new code? I ran it with test numbers and received no errors whatsoever.

However, I've adjusted the code a bit around how itertools works, but however I run it now, don't experience any errors. There are two codes no in my answer, please use the latter (the second) one. If you still encounter errors, please give the the input you used so I can try the code with that myself, as at the moment I don't see where the problem could be.

cmark gravatar imagecmark ( 2023-01-05 09:58:11 +0100 )edit

I tried the following input:

M = matrix(2, 2, [1, 1, 1, -1])

In how many matrices you want to break M: 4

How many matrices with non-negative entries you want: 2

sgmth gravatar imagesgmth ( 2023-01-05 10:19:44 +0100 )edit

I just ran my latest code (with the original data in your very first script, you can see I never changed it), please see that I experience no issues:

user@host ~ % sage ask.sage
'M=' [ 1  2]
[ 2 -1]
Enter the order of the matrix M: 1
In how many matrices you want to break M: 4
How many matrices with non-negative entries you want: 2
user@host ~ %

So I couldn't reproduce the errors. If you would want me to try it with another set of matrices, please send me the variable "M" exactly as you used it (I never used any other values so my code works for "M = matrix(2, 2, [1, 2, 2, -1])"). Please give me something easy to compute, I don't want to run this for hours to test. :)

cmark gravatar imagecmark ( 2023-01-05 12:14:25 +0100 )edit

My input and output for M = matrix(2, 2, [1, 1, 1, -1]) are as follows:

          𝙼=(111−1)
          Enter the order of the matrix M: 2
          In how many matrices you want to break M: 4
          How many matrices with non-negative entries you want: 2
          ---------------------------------------------------------------------------
         NameError                                 Traceback (most recent call last)
         <ipython-input-2-019b8e8093bc> in <module>
         56         if result:
         57             Counter += Integer(1)
    ---> 58             show("(", Counter, ").", a+b)
         59         if Counter == Integer(5):
         60             break

        NameError: name 'a' is not defined
sgmth gravatar imagesgmth ( 2023-01-05 13:49:06 +0100 )edit
1

No, what I meant is the actual code (that's what I wrote :)), what you changed in the script. The original, what I'm using is this:

M = matrix(2,2, [1, 2, 2, -1])

This works, as you can see in my output. So plesase give me the actual line for M, what you are using in the new setup.

cmark gravatar imagecmark ( 2023-01-05 13:59:31 +0100 )edit

As I have mentioned in my previous two comments, I have taken M as follows:`

                                  M = matrix(2, 2, [1, 1, 1, -1])`
sgmth gravatar imagesgmth ( 2023-01-05 14:09:03 +0100 )edit
1

Thanks, I didn't notice. However, I can't seem to run the input you gave me (an example run before the problem):

user@host ~ % sage ask.sage
'M=' [ 1  1]
[ 1 -1]
Enter the order of the matrix M: 1
In how many matrices you want to break M: 2
How many matrices with non-negative entries you want: 3
user@host ~ % 

user@host ~ % sage ask.sage
'M=' [ 1  1]
[ 1 -1]
Enter the order of the matrix M: 2
In how many matrices you want to break M: 4
How many matrices with non-negative entries you want: 2
'M=' [ 1  1]
[ 1 -1]
Enter the order of the matrix M: 'M=' [ 1  1]
[ 1 -1]
Enter the order of the matrix M: 'M=' [ 1  1]
[ 1 -1]
Enter the order of the matrix M: 'M=' [ 1  1]
[ 1 -1]
Enter the order of the matrix M:

It seems to ...(more)

cmark gravatar imagecmark ( 2023-01-05 14:26:03 +0100 )edit

What could be possible reasons behind vast diifference between your output and mine? What to do now?

sgmth gravatar imagesgmth ( 2023-01-05 14:43:32 +0100 )edit
1

I don't know. At some point I've introduced an ifninite loop but couldn't find to so I took a step back and went back to one of my previous solutions, this works alright:

user@host ~ % sage ask.sage
'M=' [ 1  1]
[ 1 -1]
Enter the order of the matrix M: 2
In how many matrices you want to break M: 4
How many matrices with non-negative entries you want: 2
'(' 1 ').' [
[1 0]  [1 1]  [-1  0]  [ 0  0]
[0 0], [1 0], [ 0  0], [ 0 -1]
]
'(' 2 ').' [
[0 1]  [1 1]  [ 0 -1]  [ 0  0]
[0 0], [1 0], [ 0  0], [ 0 -1]
]
'(' 3 ').' [
[1 1]  [1 0]  [-1  0]  [ 0  0]
[0 0], [1 0], [ 0  0], [ 0 -1]
]
'(' 4 ').' [
[1 1]  [0 1]  [ 0 -1]  [ 0  0]
[0 0], [1 0], [ 0  0], [ 0 -1]
]
'(' 5 ').' [
[1 1]  [1 1]  [-1  0]  [ 0 -1 ...
(more)
cmark gravatar imagecmark ( 2023-01-05 16:17:37 +0100 )edit

The previously mentioned code is now the second code in my answer. I think this is the end of the road for me; also, I wanted to leave a working code in the answer, now both does.

cmark gravatar imagecmark ( 2023-01-05 16:18:41 +0100 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2022-12-08 06:13:33 +0100

Seen: 703 times

Last updated: Jan 05 '23