core Package

firework Module

This module contains some of the most central FireWorks classes:

  • A FireTaskBase defines the contract for tasks that run within a FireWork (FireTasks)
  • A FWAction encapsulates the output of a FireTask and tells FireWorks what to do next after a job completes
  • A FireWork defines a workflow step and contains one or more FireTasks.
  • A Launch describes the run of a FireWork on a computing resource.
class fireworks.core.firework.FWAction(command, stored_data=None, mod_spec=None)
A FWAction encapsulates the output of a FireTask (it is returned by a FireTask after the FireTask completes). The
FWAction allows a user to store rudimentary output data as well as return commands that alter the workflow.
__init__(command, stored_data=None, mod_spec=None)
Parameters:
  • command – (str) an item from the list of FWAction.commands
  • stored_data – (dict) any output data to store. Intended to be brief, not store a ton of data.
  • mod_spec – description of how to modify the Workflow according to a set of rules (see tutorial docs)
commands = ['CONTINUE', 'DEFUSE', 'MODIFY', 'DETOUR', 'CREATE', 'PHOENIX', 'BREAK']
classmethod from_dict(*args, **kwargs)
to_dict(*args, **kwargs)
class fireworks.core.firework.FireTaskBase(parameters=None)

Bases: fireworks.utilities.fw_serializers.FWSerializable

FireTaskBase is used as an abstract class that defines a computing task (FireTask). All FireTasks should inherit from FireTaskBase.

__init__(parameters=None)
Parameters:parameters – (dict) Parameters that control the FireTask’s operation (custom depending on the FireTask

type)

classmethod from_dict(*args, **kwargs)
run_task(fw_spec)

This method gets called when the FireTask is run. It can take in a FireWork spec, perform some task using that data, and then return an output in the form of a FWAction.

Parameters:fw_spec – (dict) a FireWork spec
Returns:(FWAction)
to_dict(*args, **kwargs)
class fireworks.core.firework.FireWork(tasks, spec=None, launches=None, state='WAITING', created_on=None, fw_id=-1)

Bases: fireworks.utilities.fw_serializers.FWSerializable

STATE_RANKS = {'RUNNING': 5, 'WAITING': 1, 'RESERVED': 4, 'FIZZLED': 3, 'CANCELED': 6, 'READY': 2, 'COMPLETED': 7, 'DEFUSED': 0}
__init__(tasks, spec=None, launches=None, state='WAITING', created_on=None, fw_id=-1)
Parameters:
  • tasks – (list) a list of FireTasks to run in sequence
  • spec – (dict) specification of the job to run. Used by the FireTask
  • fw_id – (int) the FW’s database id (negative numbers will be re-assigned dynamically when they are entered in the database through the LaunchPad.
  • launches – (list) a list of Launch objects of this FireWork
  • state – (str) the state of the FW (e.g. WAITING, RUNNING, COMPLETED, CANCELED)
classmethod from_dict(*args, **kwargs)
to_db_dict()
to_dict(*args, **kwargs)
class fireworks.core.firework.Launch(state, launch_dir, fworker=None, host=None, ip=None, action=None, state_history=None, launch_id=None, fw_id=None)

Bases: fireworks.utilities.fw_serializers.FWSerializable, object

__init__(state, launch_dir, fworker=None, host=None, ip=None, action=None, state_history=None, launch_id=None, fw_id=None)
Parameters:
  • state – (str) the state of the Launch (e.g. RUNNING, COMPLETED)
  • launch_dir – (str) the directory where the Launch takes place
  • fworker – (FWorker) The FireWorker running the Launch
  • host – (str) the hostname where the launch took place (set automatically if None)
  • ip – (str) the IP address where the launch took place (set automatically if None)
  • action – (FWAction) the output of the Launch
  • state_history – (list) a history of all states of the Launch and when they occurred
  • launch_id – (int) launch_id set by the LaunchPad
  • fw_id – (int) id of the FireWork this Launch is running
classmethod from_dict(*args, **kwargs)
last_pinged
Returns:(datetime) the time the Launch last pinged a heartbeat that it was still running
reservedtime_secs
Returns:(int) number of seconds the Launch was queued
runtime_secs
Returns:(int) the number of seconds that the Launch ran for
set_reservation_id(reservation_id)

Adds the job_id to the reservation

Parameters:reservation_id – the id of the reservation (e.g., queue reservation)
state
Returns:(str) The current state of the Launch.
time_end
Returns:(datetime) the time the Launch was COMPLETED or FIZZLED
time_reserved
Returns:(datetime) the time the Launch was RESERVED in the queue
time_start
Returns:(datetime) the time the Launch started RUNNING
to_db_dict()
to_dict(*args, **kwargs)
touch_history()

Updates the update_at field of the state history of a Launch. Used to ping that a Launch is still alive.

fw_constants Module

A set of global constants for FireWorks (Python code as a config file)

fireworks.core.fw_constants.DATETIME_HANDLER(obj)

fworker Module

This module contains classes relevant for a FireWorker (worker computing resource)

class fireworks.core.fworker.FWorker(name='anonymous worker', category='general worker', query=None, params=None)

Bases: fireworks.utilities.fw_serializers.FWSerializable

__init__(name='anonymous worker', category='general worker', query=None, params=None)
Parameters:
  • name – the name of the resource, should be unique
  • category – a String describing the computing resource, does not need to be unique
  • query – a dict query that restricts the type of FireWork this resource will run
  • params – further descriptions of this resource
classmethod from_dict(*args, **kwargs)
to_dict(*args, **kwargs)

launchpad Module

The LaunchPad manages the FireWorks database.

class fireworks.core.launchpad.LaunchPad(host='localhost', port=27017, name='fireworks', username=None, password=None, logdir=None, strm_lvl=None)

Bases: fireworks.utilities.fw_serializers.FWSerializable

The LaunchPad manages the FireWorks database.

__init__(host='localhost', port=27017, name='fireworks', username=None, password=None, logdir=None, strm_lvl=None)
Parameters:
  • host
  • port
  • name
  • username
  • password
  • logdir
  • strm_lvl
add_wf(wf)
Parameters:wf – a Workflow object.
detect_fizzled(expiration_secs=14400, fix=False)
detect_unreserved(expiration_secs=1209600, fix=False)
classmethod from_dict(d)
get_fw_by_id(fw_id)

Given a FireWork id, give back a FireWork object

Parameters:fw_id – FireWork id (int)
Returns:FireWork object
get_fw_ids(query=None, sort=False)

Return all the fw ids that match a query, :param query: a dict representing a Mongo query

get_launch_by_id(launch_id)

Given a Launch id, return details of the Launch

Parameters:launch_id – launch id
Returns:Launch object
get_new_fw_id()

Checkout the next FireWork id

get_new_launch_id()

Checkout the next Launch id

get_wf_by_fw_id(fw_id)

Given a FireWork id, give back the Workflow containing that FireWork :param fw_id: :return: A Workflow object

maintain()
mark_fizzled(launch_id)
reset(password, require_password=True)

Create a new FireWorks database. This will overwrite the existing FireWorks database! To safeguard against accidentally erasing an existing database, a password must be entered.

Parameters:
  • password – A String representing today’s date, e.g. ‘2012-12-31’
  • require_password – Whether a password is required to reset the DB. Highly recommended to leave this set to True, otherwise you are inviting disaster!
run_exists()

Checks to see if the database contains any FireWorks that are ready to run :return: (T/F)

to_dict()

Note: usernames/passwords are exported as unencrypted Strings!

unreserve(launch_id)

rocket Module

TODO: add docs

class fireworks.core.rocket.Rocket(launchpad, fworker, fw_id)

The Rocket fetches a workflow step from the FireWorks database and executes it.

__init__(launchpad, fworker, fw_id)
Parameters:
  • launchpad – A LaunchPad object for interacting with the FW database
  • fworker – A FWorker object describing the computing resource
run()

Run the rocket (actually check out a job from the database and execute it)

fireworks.core.rocket.ping_launch(launchpad, launch_id, stop_event)

rocket_launcher Module

fireworks.core.rocket_launcher.launch_rocket(launchpad, fworker=None, logdir=None, strm_lvl=None, fw_id=None)

Run a single rocket in the current directory :param launchpad: a LaunchPad object :param fworker: a FWorker object

fireworks.core.rocket_launcher.rapidfire(launchpad, fworker=None, m_dir=None, logdir=None, strm_lvl=None, nlaunches=0, sleep_time=60, max_loops=-1)

Keeps running Rockets in m_dir until we reach an error. Automatically creates subdirectories for each Rocket. Usually stops when we run out of FireWorks from the LaunchPad.

Parameters:
  • launchpad – a LaunchPad object
  • fworker – a FWorker object
  • m_dir – the directory in which to loop Rocket running
  • nlaunches – 0 means ‘until completion’, -1 means ‘infinity’

workflow Module

class fireworks.core.workflow.Workflow(fireworks, links_dict=None, metadata={})

Bases: fireworks.utilities.fw_serializers.FWSerializable

Bases: dict, fireworks.utilities.fw_serializers.FWSerializable

classmethod from_dict(m_dict)
nodes
to_db_dict()
to_dict()
Workflow.__init__(fireworks, links_dict=None, metadata={})
Parameters:
  • fireworks – a list of FireWork objects
  • links_dict – A dict representing workflow links
  • metadata – metadata for this Workflow
Workflow.apply_action(action, fw_id)
classmethod Workflow.from_FireWork(fw)
classmethod Workflow.from_dict(m_dict)
classmethod Workflow.from_tarfile(tar_filename)
Workflow.refresh(fw_id, updated_ids=None)
Workflow.root_fw_ids
Workflow.to_db_dict()
Workflow.to_dict()
Workflow.to_tarfile(f_name='fwf.tar', f_format='json')

Table Of Contents

This Page