Your description is in essence correct : the interaction with a Sage plot is possible through the notebook.
You may use any Python library by importing the functions of interest, which allows you to use, for example, matplotlib tools. However, since most of these tools use only Python objects, you'll have to wrap any use of Sage-specific objects in wrapper functions converting from Sage to Python and vice-versa. For example, Sage Integers must be coerced to Python's ints, a Sage symbolic function must be wrapped in a Python function, and so on...
Conversely, you may import the needed Sage objects in a Python workspace and create your plots in Python, calling Sage as required. Again, conversion wrappers may be needed.
One may also consider building a Sage SPKG allowing the use of external Python libraries ; such a package may mecome a worthy addition to Sage.
For example, have a look at the interface to Fricas, which allows to use Fricas function and methods from Sage on Sage objects (this interface, which implements the necessary wrappers., is still incomplete, and uses a console-mode interface to a fricas process).
Further discussion of such an implementation may be best done in the Sage mailing lists.
HTH,
