A possible reason could be that your plugin starts (a new session of) SageMath each time you run the script. And SageMath is very long to start (1 to 2 seconds on my laptop). Actually, this happens for instance if you invoke sage myscript.sage in a terminal: It takes ~1s + the time for the script to execute.