Gordon's profile - activity

I am trying to run a very long (weeks long) process that constructs a large collection of graphs.

The collection is too large to keep in memory and so as each graph is found it is stored on disk (rather than in an array).

However when I set it running, I noticed that the amount of RAM used by the process was continuously increasing until the rest of the machine ground to a halt. My computer has 64Gb RAM and it takes a bit more than a day for the process to grow to that size.

I have tried to isolate the part of the program that was causing the memory usage to grow, and I think that I have isolated it to the canonical labelling algorithm "bliss".

Here is a simple program that just creates random graphs and canonically labels them (but does nothing more):

for it in range(1000000):
    g = graphs.RandomGNP(20,0.5)
    gc = g.canonical_label(algorithm="bliss")
    if it % 1000 == 0:
            print(it)

When I run this program the memory consumed grows and grows.

If I change the program to use the built-in Sage method,

for it in range(1000000):
    g = graphs.RandomGNP(20,0.5)
    gc = g.canonical_label(algorithm="sage")
    if it % 1000 == 0:
            print(it)

then the program runs to completion with no increase in memory used - the memory is correctly being recovered as it goes out of scope each time through the loop.

What should I do in this situation?

Excellent - 10.1 does not display the memory leak for bliss, so I'll restart the program and see if it can catch up with

I use the most recent MacOs X binary version as provided at https://github.com/3-manifolds/Sage_macOS/releases which, by

Problems with memory usage with bliss algorithm I am trying to run a very long (weeks long) process that constructs a la

This is quite old, but since it is not answered:

One way to do this is do use the (built-in) version of GAP to test the conjugacy.

This has the function RepresentativeAction(g,g1,g2) which takes three arguments, g, g1, g2 and either returns fail if g1 and g2 are not conjugate in g, or returns a conjugating element if they are.

You need to turn everything into a GAP-object in order to do this.

 gap.RepresentativeAction(gap("SymmetricGroup(3)"), gap(G1), gap(G2))

This returns () which is the identity, indicating that the groups are not only conjugate, but identical.

EDIT: if you define

S = SymmetricGroup(3)
gen1 = Permutation('(1,2)(3)')
gen2 = Permutation('(1,3)(2)')
H1=S.subgroup([gen1])
H2=S.subgroup([gen2])

You'll see that you get

sage: gap.RepresentativeAction(gap(S),gap(H1),gap(H2))
(2,3)

so the subgroups are conjugate, but not identical. Useful to know how to get this info, though!

A perfect answer I think, covering exactly what I needed to know. Awesome.

After nearly killing myself (in frustration) installing Sage 8.4 from source on OSX 10.14.1 (Mojave), I discover that a graph plotting command that I frequently use has vanished, breaking a number of my own programs.

Previously (on SageMath up to 8.2) I would do

from sage.graphs.graph_plot import _circle_embedding

in order to access this.

Now I just get

ImportError: cannot import name _circle_embedding

I can relatively easily replace this particular function myself, but if it were something more complicated, I would be at a loss to know how to track down the cause for its disappearance and figure out how to get it back.

So I guess the questions are: has it only vanished for me due to a botched or incomplete installation, or has it somehow been removed from Sage? In either case, how can I recover it?

I am trying to install Sage 8.3 on OSX (latest version, 10.13.6)

I usually download the disk image and drag to Applications, so I attempted this with the file

sage-8.3-OSX_10.13.6-x86_64.dmg

and checked that the MD5 checksum matched the advertised MD5 sum on the download page.

On opening the disk image, a dialog saying "Verifying" opens up, and starts to verify, but then halts with an error message that I have not seen before:

"The following disk images couldn't be opened, (name of file), image data corrupted"

I downloaded again, and once again the MD5 checksum (called from a terminal window) was correct.

Then I ran Apple's "Disk Utility" and attempt to verify the image, which failed and produced a bunch of diagnostic data that I do not understand. (I have attached this diagnostic output in case it may be helpful)

Any assistance gratefully received.

Diagnostic output from "Disk Utility" starts here:

Checksumming Protective Master Boot Record (MBR : 0)…
Protective Master Boot Record (MBR :: verified   CRC32 $B149A7EF
Checksumming GPT Header (Primary GPT Header : 1)…
GPT Header (Primary GPT Header : 1): verified   CRC32 $D70C9635
Checksumming GPT Partition Data (Primary GPT Table : 2)…
GPT Partition Data (Primary GPT Tabl: verified   CRC32 $D7F0CD0F
Checksumming  (Apple_Free : 3)…
                (Apple_Free : 3): verified   CRC32 $00000000
Checksumming EFI System Partition (C12A7328-F81F-11D2-BA4B-00A0C93EC93B : 4)…
EFI System Partition (C12A7328-F81F-: verified   CRC32 $B54B659C
Checksumming disk image (Apple_APFS : 5)…
     disk image (Apple_APFS : 5): checksum failed with error 1000.
Operation failed with status 1000: image data corrupted
Operation failed…

@david Awesome, thanks.

If I create some connected graphs, and ask SageMath for their spectral radius, the command never returns (well, I have only let it run for a few minutes) if the target is a tree.

Here is a simple example:

 g = graphs.CompleteBipartiteGraph(1,3)
 print g.spectrum()
 print g.spectral_radius()

The only advertised requirement for the target is that it be strongly connected, but as it is a connected graph, this should automatically be satisfied.

Yes, I can work around by computing the spectrum and taking the biggest, but I'd still like to know why this does not work.

I am using SageMath more and more and finding that my current workflow is not scaling up well, but am not sure how to improve it.

Basically I use the notebook interface on multiple computers (home, work etc), and each worksheet rapidly expands as I interactively test, write, develop, experiment etc.

So I frequently end up searching for functions that I remember writing, but I am not sure which computer they are on, or even what worksheet they are defined in.

I have tried to discipline myself to write functions only in a small number of Dropbox files which I can then load with a single command from any particular machine. But this is rather slow and if I "save and reload" every time I change something then I have lost the main advantage of an interpreted language over the "save and compile" workflow of writing in C.

I have searched the forum and found technical questions on aspects of this problem, such as how to work across multiple machines with a single set of worksheets, but I could not really make the answers (symlinking files to Dropbox, changing environment variables and so on) work.

So my question is: what is an effective "best practice workflow" of using SageMath across multiple machines that combines the benefits of the interpreted environment with the development of a re-usable body of code.

If it matters, I'm working with various flavours of Unix (either Linux or MacOS) but I am really more interested in the big picture of how to organise my own high-level work practices than precise technical details.

(Feel free to describe your own set-up and workflow if it works for you, even if you are unsure whether it qualifies as "best practice")

Thanks for your answer, but for some reason I am still having problems . if I use the triple <<< symbol, then I get nothing at all - no errors, but no output. I made the MyProgram1 file exactly as in your example, and when I run it from the shell, it produces what I expect, but when I pipe it in to Sage then nothing happens. (This is on Mac OSX command line, but I can't see why this would matter.)

For the second method I get a bit further, in that I can get some output, but not exactly what I want. My Sage code starts with a couple of function definitions and then lots and lots of calls using those newly defined functions. If I include the function definitions, then I get errors, but if I just have regular Sage code, then it works.

I would like to run Sage as the middle of a Unix pipe, so that some program generates Sage input, pipes it to Sage, and another program reads the output.

$ MyProgram1 | sage | MyProgram2 > final_output

In principle, MyProgram1 will run for days, weeks or months producing hundreds of millions of inputs for Sage to process, while MyProgram2 will look out for the rare successful inputs.

I know that if I have a small amount of input, then I can create a file, say "input.sage" and then run "sage input.sage", but this creates an auxiliary file "input.sage.py" before it does anything else, and so is not suitable for hundreds of millions of inputs/outputs.

I want to work in Sage with a large array of small graphs.

To create this array, I load a textfile that has 100000 lines of the following form

gs = []
gs.append(Graph([(0,1,0),(0,1,1),(0,1,2),(0,1,3),(2,3,4),(2,3,5),(2,3,6),(4,5,7),(4,5,8),(4,5,9),(0,1,10), (0,1,11),(0,1,12),(0,1,13),(0,1,14)]))
gs.append(Graph([(0,1,0),(0,1,1),(0,1,2),(0,1,3),(2,3,4),(2,3,5),(2,3,6),(4,5,7),(4,5,8),(4,5,9),(0,1,10),(0,1,11),(0,1,12),(0,1,13),(0,2,14)]))
gs.append(Graph([(0,1,0),(0,1,1),(0,1,2),(0,1,3),(2,3,4),(2,3,5),(2,3,6),(4,5,7),(4,5,8),(4,5,9),(0,1,10),(0,1,11),(0,1,12),(0,1,13),(2,3,14)]))

The textfile occupies a total of 14Mb of diskspace only, but when imported into Sage, the memory usage of the process jumps by nearly 800Mb, and indeed at intermediate stages of the importing, it jumps to over 5Gb before dropping back down again. (Presumably this is due to some sort of memory allocation strategy that keeps doubling the memory required?)

Clearly I am doing something wrong, because an array of 100000 graphs should be no trouble at all to a modern computer, but I don't know what I am doing wrong and how to fix it.