There are several different convenience functions that allow you to control YT in perhaps unexpected and unorthodox manners. These will allow you to conduct in-depth debugging of processes that may be running in parallel on multiple processors, as well as providing a mechanism of signalling to YT that you need more information about a running process. Additionally, YT has a built-in mechanism for optional reporting of errors to a central server. All of these allow for more rapid development and debugging of any problems you might encounter.
Additionally, yt is able to leverage existing developments in the IPython community for parallel, interactive analysis. This allows you to initialize multiple YT processes through mpirun and interact with all of them from a single, unified interactive prompt. This enables and facilitates parallel analysis without sacrificing interactivity and flexibility.
At http://paste.enzotools.org/ a pastebin is available for placing scripts. With yt the script yt_lodgeit.py is distributed, which allows for commandline uploading and downloading of pasted snippets. To upload script you would supply it to the command:
$ yt_lodgeit.py some_script.py
The URL will be returned. If you’d like it to be marked ‘private’ and not show up in the list of pasted snippets, supply the argument --private. All snippets are given either numbers or hashes. To download a pasted snippet, you would use the --download option:
$ yt_lodgeit.py --download=216
The snippet will be output to the window, so output redirection can be used to store it in a file.
If you are having troubles with yt, you can have it paste the error report to the pastebin by running your problematic script with the --paste option:
$ python2.6 some_problematic_script.py --paste
The --paste option has to come after the name of the script. When the script dies and prints its error, it will also submit that error to the pastebin and return a URL for the error. When reporting your bug, include this URL and then the problem can be debugged more easily.
For more information on asking for help, see asking-for-help.
During startup, yt inserts handlers for two operating system-level signals. These provide two diagnostic methods for interacting with a running process. Signalling the python process that is running your script with these signals will induce the requested behavior.
- SIGUSR1
- This will cause the python code to print a stack trace, showing exactly where in the function stack it is currently executing.
- SIGUSR1
- This will cause the python code to insert an IPython session wherever it currently is, with all local variables in the local namespace. It should allow you to change the state variables.
If your yt-running process has PID 5829, you can signal it to print a traceback with:
$ kill -SIGUSR1 5829
Note, however, that if the code is currently inside a C function, the signal will not be handled, and the stacktrace will not be printed, until it returns from that function.
If you are running a parallel job that fails, often it can be difficult to do a post-mortem analysis to determine what went wrong. To facilitate this, yt has implemented an XML-RPC interface to the Python debugger (pdb) event loop.
Running with the --rpdb command will cause any uncaught exception during execution to spawn this interface, which will sit and wait for commands, exposing the full Python debugger. Additionally, a frontend to this is provided through the yt command. So if you run the command:
$ mpirun -np 4 python2.6 some_script.py --parallel --rpdb
and it reaches an error or an exception, it will launch the debugger. Additionally, instructions will be printed for connecting to the debugger. Each of the four processes will be accessible via:
$ yt rpdb 0
where 0 here indicates the process 0.
For security reasons, this will only work on local processes; to connect on a cluster, you will have to execute the command yt rpdb on the node on which that process was launched.
IPython is a powerful mechanism not only for interactive usage, but also for task delegation and parallel analysis driving. Using the IPython Parallel Multi Engine interface, you can launch multiple ‘engines’ for computation which can then be driven by yt. However, to do so, you will have to ensure that the IPython dependencies for parallel computation are met – this requires the installation of a few components.
Both Twisted and Foolscap can be installed using easy_install but PyOpenSSL requires manual installation. Of course, yt itself requires mpi4py to be installed as well, which is described in Parallel Computation With YT.
The entire section in the IPython manual on parallel computation is essential reading, but for a quick start, you need to launch the engines:
$ ipcontroller
$ mpirun -np 4 ipengine
This will launch the controller, which interfaces with the new computation engines launched afterward. Note that you can launch an arbitrary number of compute processes. Now, launch IPYthon:
$ ipython
and execute the commands:
ipcontroller mpirun -np 4 ipengine
mec = client.MultiEngineClient()
mec.activate()
You have now obtained an object, mec, which is able to interact with and control all of the launched engines. Any command prefixed with the string %px will now be issued on all processors. Any action that would be executed in parallel in yt will be executed in parallel here. For instance,
%px from yt.mods import *
%px pf = load("data0050")
%px pc = PlotCollection(pf)
%px pc.add_projection("Density", 0)
This will load up the name space, the parameter file, and project through data0050 in parallel utilizing all of our processors. IPython can also execute commands on a limited subset of hosts, and it can also turn on auto-execution, to send all of your commands to all of the compute engines, using the %autopx directive.