Bases: fireworks.utilities.fw_serializers.FWSerializable
A FWAction encapsulates the output of a FireTask (it is returned by a FireTask after the FireTask completes). The
FWAction allows a user to store rudimentary output data as well as return commands that alter the workflow.
Parameters: | stored_data – (dict) data to store from the run. Does not |
---|
affect the operation of FireWorks. :param exit: (bool) if set to True, any remaining FireTasks within the same Firework are skipped. :param update_spec: (dict) specifies how to update the child FW’s spec :param mod_spec: ([dict]) update the child FW’s spec using the DictMod language (more flexible than update_spec) :param additions: ([Workflow]) a list of WFs/FWs to add as children :param detours: ([Workflow]) a list of WFs/FWs to add as children ( they will inherit the current FW’s children) :param defuse_children: (bool) defuse all the original children of this Firework :param defuse_workflow: (bool) defuse all incomplete steps of this workflow
If the FWAction gives any dynamic action, we skip the subsequent FireTasks
Returns: | (bool) |
---|
Bases: collections.defaultdict, fireworks.utilities.fw_serializers.FWSerializable
FireTaskBase is used like an abstract class that defines a computing task (FireTask). All FireTasks should inherit from FireTaskBase.
You can set parameters of a FireTask like you’d use a dict.
This method gets called when the FireTask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.
Bases: abc.ABCMeta
Bases: fireworks.utilities.fw_serializers.FWSerializable
A Firework is a workflow step and might be contain several FireTasks.
Parameters: |
|
---|
FireTask :param launches: ([Launch]) a list of Launch objects of this Firework :param archived_launches: ([Launch]) a list of archived Launch objects of this Firework :param state: (str) the state of the FW (e.g. WAITING, RUNNING, COMPLETED, ARCHIVED) :param created_on: (datetime) - time of creation :param fw_id: (int) an identification number for this Firework :param parents: (Firework or [Firework]) list of parent FWs this FW depends on :param updated_on: (datetime) - last time the STATE was updated
Returns: | (str) The current state of the Firework |
---|
Bases: fireworks.utilities.fw_serializers.FWSerializable, object
A Launch encapsulates data about a specific run of a Firework on a computing resource
Parameters: |
|
---|
automatically if None) :param ip: (str) the IP address where the launch took place (set automatically if None) :param trackers: ([Tracker]) File Trackers for this Launch :param action: (FWAction) the output of the Launch :param state_history: ([dict]) a history of all states of the Launch and when they occurred :param launch_id: (int) launch_id set by the LaunchPad :param fw_id: (int) id of the Firework this Launch is running
Returns: | (datetime) the time the Launch last pinged a heartbeat that |
---|
it was still running
Returns: | (int) number of seconds the Launch was stuck as RESERVED in |
---|
a queue
Returns: | (int) the number of seconds that the Launch ran for |
---|
Adds the job_id to the reservation
Parameters: | reservation_id – (str) the id of the reservation (e.g., |
---|
queue reservation)
Returns: | (str) The current state of the Launch. |
---|
Returns: | (datetime) the time the Launch was COMPLETED or FIZZLED |
---|
Returns: | (datetime) the time the Launch was RESERVED in the queue |
---|
Returns: | (datetime) the time the Launch started RUNNING |
---|
Bases: fireworks.utilities.fw_serializers.FWSerializable, object
A Tracker monitors a file and returns the last N lines for updating the Launch object
Reads the monitored file and returns back the last N lines :param launch_dir: directory where job was launched in case of relative filename :return:
Bases: fireworks.utilities.fw_serializers.FWSerializable
A Workflow connects a group of FireWorks in an execution order
Bases: dict, fireworks.utilities.fw_serializers.FWSerializable
An inner class for storing the DAG links between FireWorks
Parameters: |
|
---|
child_id1, child_id2)] :param name: (str) naem of workflow :param metadata: (dict) metadata for this Workflow :param created_on: (datetime) :param updated_on: (datetime) :param fw_states: (dict) - leave alone unless you are purposefully creating a Lazy-style WF
Method to add a workflow as a child to a Firework Note: detours must have children that have STATE_RANK that is WAITING or below
Parameters: |
|
---|---|
Returns: | ([int]) list of Firework ids that were updated or new |
Apply a FWAction on a Firework in the Workflow
Parameters: |
|
---|---|
Returns: | ([int]) list of Firework ids that were updated or new |
Create a fresh Workflow from an existing one.
Gets leaf FireWorks of this workflow (those with no children)
Returns: | ([int]) Firework ids of leaf FWs |
---|
Refreshes the state of a Firework and any affected children.
Parameters: |
|
---|---|
Returns: | (set(int)) list of Firework ids that were updated |
Archives the launches of a Firework so that it can be re-run. :param fw_id: (int) :return: ([int]) list of Firework ids that were updated
Reset the states of all Fireworks in this workflow to ‘WAITING’.
Parameters: | reset_ids – (bool) if True, give each Firework a new id. |
---|
Gets root FireWorks of this workflow (those with no parents)
Returns: | ([int]) Firework ids of root FWs |
---|
Bases: fireworks.utilities.fw_serializers.FWSerializable
name: the name of the resource, should be unique category: a String describing the computing resource, does not
need to be unique
Bases: fireworks.utilities.fw_serializers.FWSerializable
The LaunchPad manages the FireWorks database.
Parameters: |
|
---|
Parameters: | wf – a Workflow object. |
---|
Append a new workflow on top of an existing workflow
Parameters: |
|
---|
(internal method) Finds a Firework that’s ready to be run, marks it as running, and returns it to the caller. The caller is responsible for running the Firework.
Parameters: |
|
---|---|
Returns: | a Firework, launch_id tuple |
(internal method) used to mark a Firework’s Launch as completed. :param launch_id: :param action: the FWAction of what to do next
Given a Firework id, give back a Firework object
Parameters: | fw_id – Firework id (int) |
---|---|
Returns: | Firework object |
Return all the fw ids that match a query, :param query: (dict) representing a Mongo query :param sort: [(str,str)] sort argument in Pymongo format :param limit: (int) limit the results :param count_only: (bool) only return the count rather than explicit ids :param launches_mode: (bool) query the launches collection instead of fireworks
Given a Launch id, return details of the Launch
Parameters: | launch_id – launch id |
---|---|
Returns: | Launch object |
Checkout the next Firework id
Checkout the next Launch id
Given a Firework id, give back the Workflow containing that Firework :param fw_id: :return: A Workflow object
Given a FireWork id, give back the Workflow containing that FireWork :param fw_id: :return: A Workflow object
Return one fw id for all workflows that match a query, :param query: (dict) representing a Mongo query :param sort: [(str,str)] sort argument in Pymongo format :param limit: (int) limit the results :param count_only: (bool) only return the count rather than explicit ids
A much faster way to get summary information about a Workflow by querying only for needed information.
fw_id (int): A Firework id. mode (str): Choose between “more”, “less” and “all” in terms of
quantity of information.
Rerun a fw at the task level :param fw_id: (int) fw_id to rerun :param rerun_duplicates: (bool) also rerun duplicate FWs :param launch_id: (int) launch id to rerun, if known. otherwise the last launch_id will be used :param recover_mode: (str) use “prev_dir” to run again in previous dir, “cp” to try to copy data to new dir, or None to start from scratch :return: ([int]) list of rerun fw_ids
Create a new FireWorks database. This will overwrite the existing FireWorks database! To safeguard against accidentally erasing an existing database, a password must be entered.
Parameters: |
|
---|
Checks to see if the database contains any FireWorks that are ready to run :return: (T/F)
Note: usernames/passwords are exported as unencrypted Strings!
Update fireworks with a spec. Sometimes you need to modify a firework in progress.
fw_ids: All fw_ids to modify. spec_document: The spec document. Note that only modifications to
the spec key are allowed. So if you supply { “_tasks.1.parameter”: “hello”}, you are effectively modifying spec._tasks.1.parameter in the actual fireworks collection.
Bases: object
A LazyFirework only has the fw_id, and grabs other data just-in-time. This representation can speed up Workflow loading as only “important” FWs need to be fully loaded. :param fw_id: :param fw_coll: :param launch_coll:
Bases: exceptions.ValueError
Error raised if the context manager WFLock can’t acquire the lock on the WF within the selected time interval (WFLOCK_EXPIRATION_SECS), if the killing of the lock is disabled (WFLOCK_EXPIRATION_KILL)
Bases: object
Lock a Workflow, i.e. for performing update operations Raises a LockedWorkflowError if the lock couldn’t be acquired withing expire_secs and kill==False. Calling functions are responsible for handling the error in order to avoid database inconsistencies.
The Rocket fetches a workflow step from the FireWorks database and executes it.
Parameters: |
|
---|
Run the rocket (check out a job from the database and execute it)
Run a single rocket in the current directory :param launchpad: (LaunchPad) :param fworker: (FWorker) :param fw_id: (int) if set, a particular Firework to run :param strm_lvl: (str) level at which to output logs to stdout
Keeps running Rockets in m_dir until we reach an error. Automatically creates subdirectories for each Rocket. Usually stops when we run out of FireWorks from the LaunchPad.
Parameters: |
|
---|