core Package

firetask Module

TODO: add docs

class fireworks.core.firetask.FWAction(command, stored_data=None, mod_spec=None)

TODO: add docs

commands = ['CONTINUE', 'DEFUSE', 'MODIFY', 'DETOUR', 'CREATE', 'ADDIFY', 'PHOENIX', 'BREAK']
classmethod from_dict(m_dict)
to_dict()
class fireworks.core.firetask.FireTaskBase(parameters=None)

TODO: add docs

classmethod from_dict(m_dict)
run_task(fw)
to_dict(*args, **kwargs)

firework Module

A FireWork defines a workflow step.

An FWorkflow connects FireWorks by their fw_ids.

A Launch is a describes a FireWork’s run on a computing resource. The same Launch might apply to multiple FireWorks, e.g. if they are identical.

A FWAction encapsulates the output of that launch.

class fireworks.core.firework.FireWork(tasks, spec=None, fw_id=-1, launches=None, state='WAITING', created_at=None)

Bases: fireworks.utilities.fw_serializers.FWSerializable

classmethod from_dict(m_dict)
to_db_dict()
to_dict()

This is a ‘minimal’ or ‘compact’ dict representation of the FireWork

class fireworks.core.firework.Launch(fworker, host=None, ip=None, launch_dir=None, action=None, start=None, end=None, state=None, launch_id=None)

Bases: fireworks.utilities.fw_serializers.FWSerializable

classmethod from_dict(m_dict)
time_secs
to_db_dict()
to_dict()

fw_constants Module

A set of global constants for FireWorks (Python code as a config file)

fireworks.core.fw_constants.DATETIME_HANDLER(obj)

fworker Module

This module contains classes relevant for a FireWorker (worker computing resource)

class fireworks.core.fworker.FWorker(name='anonymous worker', category='general worker', query=None, params=None)

Bases: fireworks.utilities.fw_serializers.FWSerializable

classmethod from_dict(m_dict)
to_dict()
class fireworks.core.fworker.QueueAdapterBase

Bases: fireworks.utilities.fw_serializers.FWSerializable

The QueueAdapter is responsible for all interactions with a specific queue management system. This includes handling all details of queue script format as well as queue submission and management.

A user should extend this class with implementations that work on specific queue systems.

classmethod from_dict(m_dict)
get_njobs_in_queue(rocket_params, username=None)

returns the number of jobs in the queue, probably using subprocess or shutil to call a command like ‘qstat’. returns None when the number of jobs cannot be determined.

Parameters:
  • rocket_params – a RocketParams() instance
  • username – the username of the jobs to count (default is to autodetect)
get_script_str(rocket_params, launch_dir)

returns a (multi-line) String representing the queue script, e.g. PBS script. All details of the queue script must be specified in the RocketParams().

Parameters:
  • rocket_params – A RocketParams() instance
  • launch_dir – The directory the job will be launched in
submit_to_queue(rocket_params, script_file)

submits the job to the queue, probably using subprocess or shutil :param rocket_params: A RocketParams() instance :param script_file: name of the script file to use (String)

to_dict(*args, **kwargs)
class fireworks.core.fworker.QueueParams(queue_adapter, params, logging_dir='.')

Bases: fireworks.utilities.fw_serializers.FWSerializable

A RocketParams instance contains all the information needed to write a queue file and submit to a queue system. Details of the queue file format and queue submission commands should be included in the QueueAdapterBase object. Specific parameters used by the QueueAdapterBase should be included in the params variable.

classmethod from_dict(m_dict)

Note: The QueueAdapter is loaded based on its fw_name alone See the docs for to_dict() for more details

to_dict()

Note: the QueueAdapter is being serialized using the FW name alone This keeps the serialization compact and easy to edit by humans. The from_dict() will dynamically find the correct QueueAdapter using its fw_name.

launchpad Module

The LaunchPad manages the FireWorks database.

class fireworks.core.launchpad.LaunchPad(host='localhost', port=27017, name='fireworks', username=None, password=None, logdir=None, strm_lvl=None)

Bases: fireworks.utilities.fw_serializers.FWSerializable

The LaunchPad manages the FireWorks database.

add_wf(wf)
Parameters:wf – a Workflow object.
classmethod from_dict(d)
get_fw_by_id(fw_id)

Given a FireWork id, give back a FireWork object

Parameters:fw_id – FireWork id (int)
Returns:FireWork object
get_fw_ids(query=None, sort=False)

Return all the fw ids that match a query, :param query: a dict representing a Mongo query

get_launch_by_id(launch_id)

Given a Launch id, return details of the Launch

Parameters:launch_id – launch id
Returns:Launch object
get_new_fw_id()

Checkout the next FireWork id

get_new_launch_id()

Checkout the next Launch id

get_wf_by_fw_id(fw_id)

Given a FireWork id, give back the Workflow containing that FireWork :param fw_id: :return: A Workflow object

maintain()
reset(password, require_password=True)

Create a new FireWorks database. This will overwrite the existing FireWorks database! To safeguard against accidentally erasing an existing database, a password must be entered.

Parameters:
  • password – A String representing today’s date, e.g. ‘2012-12-31’
  • require_password – Whether a password is required to reset the DB. Highly recommended to leave this set to True, otherwise you are inviting disaster!
run_exists()

Checks to see if the database contains any FireWorks that are ready to run :return: (T/F)

to_dict()

Note: usernames/passwords are exported as unencrypted Strings!

queue_launcher Module

This module is used to submit jobs to a queue on a cluster. It can submit a single job, or if used in “rapid-fire” mode, can submit multiple jobs within a directory structure. The details of job submission and queue communication are handled using RocketParams, which specifies a QueueAdapter as well as desired properties of the submit script.

fireworks.core.queue_launcher.launch_rocket_to_queue(rocket_params, launcher_dir='.', logdir=None, strm_lvl=None)

Submit a single job to the queue.

Parameters:
  • rocket_params – A RocketParams instance
  • launcher_dir – The directory where to submit the job
fireworks.core.queue_launcher.rapidfire(rocket_params, launch_dir='.', njobs_queue=10, njobs_block=500, logdir=None, strm_lvl=None, infinite=False, sleep_time=60, launchpad=None)

Submit many jobs to the queue.

Parameters:
  • rocket_params – A RocketParams instance
  • launch_dir – directory where we want to write the blocks
  • njobs_queue – stops submitting jobs when njobs_queue jobs are in the queue
  • njobs_block – automatically write a new block when njobs_block jobs are in a single block

rocket Module

TODO: add docs

class fireworks.core.rocket.Rocket(launchpad, fworker)

The Rocket fetches a workflow step from the FireWorks database and executes it.

run()

Run the rocket (actually check out a job from the database and execute it)

rocket_launcher Module

fireworks.core.rocket_launcher.launch_rocket(launchpad, fworker=None, logdir=None, strm_lvl=None)

Run a single rocket in the current directory :param launchpad: a LaunchPad object :param fworker: a FWorker object

fireworks.core.rocket_launcher.rapidfire(launchpad, fworker=None, m_dir=None, logdir=None, strm_lvl=None, infinite=False, sleep_time=60)

Keeps running Rockets in m_dir until we reach an error. Automatically creates subdirectories for each Rocket. Usually stops when we run out of FireWorks from the LaunchPad.

Parameters:
  • launchpad – a LaunchPad object
  • fworker – a FWorker object
  • m_dir – the directory in which to loop Rocket running

workflow Module

class fireworks.core.workflow.Workflow(fireworks, links_dict=None, metadata={})

Bases: fireworks.utilities.fw_serializers.FWSerializable

Bases: dict, fireworks.utilities.fw_serializers.FWSerializable

classmethod from_dict(m_dict)
nodes
to_db_dict()
to_dict()
Workflow.apply_action(action, fw_id)
classmethod Workflow.from_FireWork(fw)
classmethod Workflow.from_dict(m_dict)
classmethod Workflow.from_tarfile(tar_filename)
Workflow.refresh(fw_id, updated_ids=None)
Workflow.root_fw_ids
Workflow.to_db_dict()
Workflow.to_dict()
Workflow.to_tarfile(f_name='fwf.tar', f_format='json')