Ask Your Question
1

Continuing the calculation at Jupyter Notebook on another day

asked 2022-05-01 21:36:35 +0200

Vibap gravatar image

updated 2022-05-02 12:03:38 +0200

I'm running a function at Jupyter Notebook which might cost more than 5 hours to be completed in my estimate.

When we play games, we don't usually complete every mission just in one day but save it at some point, and continue it on another day.

Likewise, to stop and save the calculation at some point and continue the calculation from that saved point on another day, what would be the necessary steps that I must follow?

If I look at menu bar in the upper side, I see 'Run' button and '■'(black square) button next to it.

Would pressing the '■'(black square) button enough to save the calculation? After saving it, if I load the file on another day and press the 'Run' button, would it continue the calculation from the saved point and not restart the whole process again?

For example, if there is a task that costs 5 hours, saving it at some point after calculating it for 4 hours, and loading it on another day to continue and finish the remaining 1 hour of task,

would this happen by just pressing 'Run' and '■'(black square) as what I mentioned above?

If not, I would like to know whether there is any way to do this.

Thanks.

edit retag flag offensive close merge delete

2 Answers

Sort by » oldest newest most voted
2

answered 2022-05-02 12:41:55 +0200

tmonteil gravatar image

Regarding the way you propose, the answer is no : the jupyter notebook does not know and is not able to save the state of a computation.

I see two options:

  • from inside : you can write checkpoints within your code (e.g. every X iterations) that save, within a file on your disk, all information needed to resume the computation from there. Your function should be written in a way that given such a checkpoint file, comtinue the computation.

  • from outside : in Linux, it is possible to stop a process an resume it later (with STOP and CONT signals). However, Sage launches several processes. An option is to use cgroups. Otherwise, you can run Sage within a virtual machine (e.g. qemu) or a container (e.g. LXC) so that you have control over all Linux processes involved in the computation, those tools provide ways to stop the processes and restart them later (even save the state on the disk if you want to restart them after machine reboot).

edit flag offensive delete link more

Comments

I'm thinking of installing Linux soon. Thank you for introducing the method!!!

Vibap gravatar imageVibap ( 2022-05-02 15:35:31 +0200 )edit
1

answered 2022-05-03 21:36:22 +0200

Sébastien gravatar image

updated 2022-05-03 22:30:12 +0200

In SageMath, there is save_session and load_session which you can use to save and load a session:

  sage: a = 5
  sage: filename = 'session-2022-05-03'
  sage: save_session(filename)
  sage: del a
  sage: load_session(filename)
  sage: print(a)
  5

This should work the same in the Jupyter notebook.

edit flag offensive delete link more

Comments

Beware : some Sage objects cannot be saved. Most notably Python functions...

Emmanuel Charpentier gravatar imageEmmanuel Charpentier ( 2022-05-03 23:32:05 +0200 )edit

To complement @emmanuel-charpentier comment, some Python variables point to internal state of an external library for which there is no saving protocol. Example : you use a huge MILP solver to which you add constraints one by one depending on the result of the previous solve (see e.g. the source code for hamiltonian cycles). When you add a constraint to an existing solved MILP, the solver is supposed to work "hot", meaning that it uses the previous computations to do the current one (it does not restart from scratch). Such internal state will not be saved with save_session.

tmonteil gravatar imagetmonteil ( 2022-05-05 07:25:23 +0200 )edit

I will definitely check until what extent it works. Super thanks!!!

Vibap gravatar imageVibap ( 2022-05-05 18:39:02 +0200 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2022-05-01 21:02:09 +0200

Seen: 247 times

Last updated: May 03 '22