This module contains some of the most central FireWorks classes:
with its Launches.
FireTasks) - A FWAction encapsulates the output of a FireTask and tells FireWorks what to do next after a job completes
Bases: fireworks.utilities.fw_serializers.FWSerializable
A FWAction encapsulates the output of a FireTask (it is returned by a FireTask after the FireTask completes). The
FWAction allows a user to store rudimentary output data as well as return commands that alter the workflow.
Parameters: | stored_data – (dict) data to store from the run. Does not |
---|
affect the operation of FireWorks. :param exit: (bool) if set to True, any remaining FireTasks within the same FireWork are skipped. :param update_spec: (dict) specifies how to update the child FW’s spec :param mod_spec: ([dict]) update the child FW’s spec using the DictMod language (more flexible than update_spec) :param additions: ([Workflow]) a list of WFs/FWs to add as children :param detours: ([Workflow]) a list of WFs/FWs to add as children ( they will inherit the current FW’s children) :param defuse_children: (bool) defuse all the original children of this FireWork
If the FWAction gives any dynamic action, we skip the subsequent FireTasks
Returns: | (bool) |
---|
Bases: dict, fireworks.utilities.fw_serializers.FWSerializable
FireTaskBase is used like an abstract class that defines a computing task (FireTask). All FireTasks should inherit from FireTaskBase.
You can set parameters of a FireTask like you’d use a dict.
This method gets called when the FireTask is run. It can take in a FireWork spec, perform some task using that data, and then return an output in the form of a FWAction.
Parameters: | fw_spec – (dict) a FireWork spec |
---|---|
Returns: | (FWAction) |
Bases: type
Bases: fireworks.utilities.fw_serializers.FWSerializable
A FireWork is a workflow step and might be contain several FireTasks
Parameters: |
|
---|
FireTask :param launches: ([Launch]) a list of Launch objects of this FireWork :param archived_launches: ([Launch]) a list of archived Launch objects of this FireWork :param state: (str) the state of the FW (e.g. WAITING, RUNNING, COMPLETED, ARCHIVED) :param fw_id: (int) an identification number for this FireWork
Bases: fireworks.utilities.fw_serializers.FWSerializable, object
A Launch encapsulates data about a specific run of a FireWork on a computing resource
Parameters: |
|
---|
automatically if None) :param ip: (str) the IP address where the launch took place (set automatically if None) :param trackers: ([Tracker]) File Trackers for this Launch :param action: (FWAction) the output of the Launch :param state_history: ([dict]) a history of all states of the Launch and when they occurred :param launch_id: (int) launch_id set by the LaunchPad :param fw_id: (int) id of the FireWork this Launch is running
Returns: | (datetime) the time the Launch last pinged a heartbeat that |
---|
it was still running
Returns: | (int) number of seconds the Launch was stuck as RESERVED in |
---|
a queue
Returns: | (int) the number of seconds that the Launch ran for |
---|
Adds the job_id to the reservation
Parameters: | reservation_id – (str) the id of the reservation (e.g., |
---|
queue reservation)
Returns: | (str) The current state of the Launch. |
---|
Returns: | (datetime) the time the Launch was COMPLETED or FIZZLED |
---|
Returns: | (datetime) the time the Launch was RESERVED in the queue |
---|
Returns: | (datetime) the time the Launch started RUNNING |
---|
Updates the update_at field of the state history of a Launch. Used to ping that a Launch is still alive.
Bases: fireworks.utilities.fw_serializers.FWSerializable, object
A Tracker monitors a file and returns the last N lines for updating the Launch object
Reads the monitored file and returns back the last N lines :param launch_dir: directory where job was launched in case of relative filename :return:
Bases: fireworks.utilities.fw_serializers.FWSerializable
A Workflow connects a group of FireWorks in an execution order
Bases: dict, fireworks.utilities.fw_serializers.FWSerializable
An inner class for storing the DAG links between FireWorks
Parameters: |
|
---|
child_id1, child_id2)] :param metadata: (dict) metadata for this Workflow
Apply a FWAction on a FireWork in the Workflow
Parameters: |
|
---|---|
Returns: | ([int]) list of FireWork ids that were updated or new |
Gets leaf FireWorks of this workflow (those with no children)
Returns: | ([int]) FireWork ids of leaf FWs |
---|
Refreshes the state of a FireWork and any affected children.
Parameters: |
|
---|---|
Returns: | ([int]) list of FireWork ids that were updated |
Archives the launches of a FireWork so that it can be re-run. :param fw_id: (int) :return: ([int]) list of FireWork ids that were updated
Gets root FireWorks of this workflow (those with no parents)
Returns: | ([int]) FireWork ids of root FWs |
---|
A set of global constants for FireWorks (Python code as a config file)
This module contains FireWorker, which encapsulates information about a computing resource
Bases: fireworks.utilities.fw_serializers.FWSerializable
Parameters: |
|
---|
The LaunchPad manages the FireWorks database.
Bases: fireworks.utilities.fw_serializers.FWSerializable
The LaunchPad manages the FireWorks database.
Parameters: |
|
---|
Parameters: | wf – a Workflow object. |
---|
(internal method) Finds a FireWork that’s ready to be run, marks it as running, and returns it to the caller. The caller is responsible for running the FireWork.
Parameters: |
|
---|---|
Returns: | a FireWork, launch_id tuple |
(internal method) used to mark a FireWork’s Launch as completed. :param launch_id: :param action: the FWAction of what to do next
Given a FireWork id, give back a FireWork object
Parameters: | fw_id – FireWork id (int) |
---|---|
Returns: | FireWork object |
Return all the fw ids that match a query, :param query: (dict) representing a Mongo query :param sort: [(str,str)] sort argument in Pymongo format :param limit: (int) limit the results :param count_only: (bool) only return the count rather than explicit ids
Given a Launch id, return details of the Launch
Parameters: | launch_id – launch id |
---|---|
Returns: | Launch object |
Checkout the next FireWork id
Checkout the next Launch id
Given a FireWork id, give back the Workflow containing that FireWork :param fw_id: :return: A Workflow object
Return one fw id for all workflows that match a query, :param query: (dict) representing a Mongo query :param sort: [(str,str)] sort argument in Pymongo format :param limit: (int) limit the results :param count_only: (bool) only return the count rather than explicit ids
Create a new FireWorks database. This will overwrite the existing FireWorks database! To safeguard against accidentally erasing an existing database, a password must be entered.
Parameters: |
|
---|
Checks to see if the database contains any FireWorks that are ready to run :return: (T/F)
Note: usernames/passwords are exported as unencrypted Strings!
A Rocket fetches a FireWork from the database, runs the sequence of FireTasks inside, and then completes the Launch
The Rocket fetches a workflow step from the FireWorks database and executes it.
Parameters: |
|
---|
Run the rocket (check out a job from the database and execute it)
This module contains methods for launching Rockets, both singly and in rapid-fire mode
Run a single rocket in the current directory :param launchpad: (LaunchPad) :param fworker: (FWorker) :param fw_id: (int) if set, a particular FireWork to run :param strm_lvl: (str) level at which to output logs to stdout
Keeps running Rockets in m_dir until we reach an error. Automatically creates subdirectories for each Rocket. Usually stops when we run out of FireWorks from the LaunchPad.
Parameters: |
|
---|