FireWorks Changelog
v0.99
.. note:: v0.98 is skipped, as it has a faulty dependency.
.. caution:: Users of the frontend will need to install Flask, pip install flask; pip install flask-paginate. Django is no longer required for the frontend.
- Ability to add FireWorks to existing workflow (launchpad.add_wf_to_fwids)
- Better unit tests for task-level reruns (G. Petretto)
- Redesigned web site using Flask (M. Brafman)
v0.97
- Fix bug in adding multiple detours
- Task-level reruns (G. Petretto)
- Better Fworker default restrictions (G. Petretto)
- Make _launch_dir if doesn’t exist (G. Petretto)
- Bug fixes (G. Petretto)
v0.96
- Address some installation issues (thanks to kpoman)
- fix minor issues and docs
v0.95
- Add decompressdir task (S.P. Ong)
- Fix bugs in offline launch (G. Petretto)
- Improve failure handling in case of FW system failure (G. Petretto)
- Allow embedding error message on FW rerun (G. Petretto)
- Minor testing improvements
v0.94
- Improve performance of get_wflows (S.P. Ong)
- Fix another bug due to performance improvements (B. Medasani)
- Fix bug in de-serialization of non dict-like FireTasks and other serialization issues
v0.93
- Fix bug in performance improvement cached state + unit tests (B. Medasani)
- minor bug fixes, installation changes
v0.92
Caution
This version has a minor bug affecting defusing of FWs and cached states for performance, fixed in v0.94
- Improve large workflow performance using a LazyFirework (B. Medasani, D. Gunter)
- some code cleanups and minor (rare) bugfix to datetime
- Add email option to PBS adapter (S.P. Ong)
- Support for pymatgen as_dict formulation (X. Qu)
v0.91
- Major: Rename FireWork to Firework. Should be fully backward-compatible for the moment, but users must switch by ~v1.0.
- Unicode compatibility for Py3k (S.P. Ong)
v0.90
- Introduce reporting tools via lpad report (W. Chen)
- Fix bug in locking
- Greatly speed up rlaunch rapidfire by removing artificial sleep
- Use monty CLoader (S.P. Ong)
v0.89
- Fix small FireTaskMeta issue (G. Petretto w/S.P. Ong)
- simplify some imports
- Add reservation display mode (S.P. Ong)
- add updated_on to FW which updates whenever FW changes state
- improve docs
v0.88
- Add many more unit tests (B. Medasani)
- Fix tracking when FireTask crashes (B. Medasani)
- Clean up some logging
- Don’t rerun DEFUSED FWs - they must be reignited
- Allow defuse of COMPLETED FWs
- minor internal fixes
v0.87
- Fix major bug causing FIZZLED FWs to rerun spontaneously
- Make WFLock more nimble
- Forcibly remove WFLock after some time in case of catastrophe (tunable in FW_config)
- improve unit tests
v0.86
Warning
This version has a major bug that causes FIZZLED FWs to rerun, patched in v0.87
- add delete_wfs command (w/S.P. Ong)
- add update_fws command (S.P. Ong)
- add ignore_errors option in some default FireTasks (S.P. Ong)
- fix bug in Windows $HOME var (thanks to A. Berg)
- fig bug in reporting of lost FWs; rerun option should be OK in prev. versions
- change FIZZLED to have lower STATE_RANK than READY/RESERVED/RUNNING/etc
v0.85
- fix bug in running daemon mode locally with qlaunch rapidfire (B. Foster)
- better handling of duplicate path detection (S.P. Ong)
- add support for nodes keyword in SLURM adapter (S.P. Ong)
v0.84
- ability to define links when defining FireWorks rather than all at the Workflow level (based on conversation with H. Rusche)
- better handling of config files and better reporting on config file conflicts
v0.83
- misc multiprocessing improvements (X. Qu)
- better handling of dir creation conflicts (X. Qu)
v0.82
- add ability to define links via {fw1:fw2} objects rather than explicit IDs (based on conversation with H. Rusche)
- un-reserve a FW if queue submission goes badly and clean up queue launcher code
- internal cleanups (don’t rerun ARCHIVED jobs, skip reruns of WAITING jobs)
- stop rapidfire upon error in queue launch
- rerun fw on unreserve
- add methods to work with queue ids (cancel_qid, --qid option in get_fws, and get_qid)
v0.81
Note
A major bugfix to dynamic and branching workflows was added in this release
- fix race condition bug in which two FW belonging to same WF simultaneously try to update the WF, and only one succeeds
v0.80
- rerun duplicated FWs on a rerun command (enabled by default), and return back all fw_ids that were rerun
- change default QUEUE_UPDATE_INTERVAL from 15 secs down to 5 secs
- add background tuneup option, and make it the default
- misc. cleanup (S.P. Ong)
v0.79
- Add support for IBM LoadLeveler Queue (F. Brockherde)
v0.78
- Fix spec copy bug as reported by Github user (F. Brockherde)
- Misc fixes (archiving FWs, tuple support)
v0.77
- Support/fix serialization of tuples as list instead of String (S.P. Ong)
- Introduce fw_env variables (S.P. Ong)
v0.76
- Better test for invalid WFs (S.P. Ong)
- Minor internal code cleanup (S.P. Ong)
- add internal profiling tools (D. Gunter)
v0.75
- Fix bug that randomly affected some dynamic workflows
- Add CompressDir and ArchiveDir tasks (S.P. Ong)
- Initial commit of PyTask (S.P. Ong)
- Initial networkx graphing of workflows via lpad (S.P. Ong)
v0.72
Warning
This version has a bug that can affect some dynamic workflows, patched in v0.75
- Include default base site files in pip install
- Optimizations for when WFs contains 1000s of root node FWs
- zopen tracker files
v0.71
- Include default templates in pip install
- Change default formatting in get_wfs (S.P. Ong)
v0.7
Caution
The default behavior is now that mod_spec and update_spec push updates to next Firework AND the next FireTask
Caution
The FWConfig parameters are no longer called via a FWConfig() class instantiation; you can import these parameters directly now.
- Python 3 support! via ‘six’ library (S.P. Ong)
- BackgroundTasks introduced
- Performance improvements to get_wf command (S.P. Ong)
- Deserialization warnings and added stability (S.P. Ong)
- Reservation mode and silencer works in remote launch (S.P. Ong)
- Restore old FileTransferTask behavior
- Tutorial updates
- Various internal improvements, e.g. to FWConfig (S.P. Ong)
- Bug fixes (A. Jain, S.P. Ong)
v0.66
Warning
This version changes the default serialization for custom FireWorks without _fw_name to <project>::<Class> instead of <Class>. If you have custom FireTasks from v0.62-v0.65 that did not specify _fw_name explicitly, this introduces a backward incompatibility. Contact the support list if this affects you - an easy fix is available.
- Fix major bug in dynamic workflows with multiple additions/detours
- Fixed lpad reset that became broken in recent release
- Change default _fw_name for FireTasks to <project>::<Class>, e.g. fireworks::MyTask
v0.65
- Fix bug in qlaunch singleshot introduced in previous release (S.P. Ong)
- Add qlaunch cleanup (S.P. Ong)
- Setup different default config dirs (S.P. Ong)
v0.64
Warning
This version introduced a major bug in qlaunch singleshot via the command line (fixed in v0.65)
Warning
This version introduced a bug in lpad reset via the command line (fixed in v0.66)
Caution
The add_dir command is incorporated into the add command. e.g. lpad add my_dir/*.yaml. Many command line options that allowed comma-separated lists are now space-separated lists to better employ argparse (see updated docs).
- clean up argument parsing (S.P. Ong)
- remote qlaunch handles multiple configs (S.P. Ong)
v0.63
- fix bug in rtransfer mode of FileTransferTask (S.P. Ong)
- improvements to remote qlaunch (S.P. Ong)
v0.62
Caution
The TransferTask is renamed to FileTransferTask (however, existing FireWorks databases should be backwards-compatibile). The names of the default FireTasks no longer have spaces; however, existing FireWorks databases and code should be backwards-compatible.
- Add FIFO and FILO sort options for equal priority FireWorks
- Remove database locks in multiprocessing mode
- Allow multiple scripts in ScriptTask (S.P. Ong)
- Add additional File I/O FireTasks (S.P. Ong)
- Changes to FireTask base implementation (S.P. Ong)
- Allow config file in $HOME/.fireworks (S.P. Ong)
- Add remote options to qlaunch via fabric library (S.P. Ong)
- _fw_name automatically set to class name if unspecified (S.P. Ong)
- Remove ValueError upon not finding a Firework to run and handle this situation better
v0.61
- Include text files needed for queue adapters in distribution (D. Gunter)
v0.6
Caution
The QueueAdapter code has been refactored in a way that is not fully backward compatible. Chances are, you will have to modify any my_qadapter.yaml files you have so that the _fw_name is set to CommonAdapter and a new _fw_q_type parameter is set to PBS, SGE, or SLURM.
- Major refactor of QueueAdapters so it is easy to change template files without adding new code (S.P. Ong)
- restore lpad.maintain()
- minor doc updates
v0.54
- Add --exclude and --include options to Trackers + minor formatting changes
- use config file in current dir if possible
v0.53
- Display name in trackers
- Fix some bugs relating to multiprocessing & offline mode (Xiaohui Qu)
- Don’t require password when tracking many FWs
- Default 25 lines in trackers
v0.52
- add trackers, or the ability to monitor output files
v0.51
- make set_priority work as intended through command line
- invert the -b option on webgui (new -s option skips opening browser)
v0.5
Caution
The command/function detect_fizzled has changed to detect_lostruns, changed old arguments and added additional ones
Caution
The command/function detect_unreserved has changed - refactored “mark” to “fizzle”
- add option to “rerun” when detecting lost runs
- add option to only detect short-lived lost jobs (useful for job packing type failures)
- refactored argument names and method names for clarity
v0.46
v0.45
- allow user to confirm database reset and multi-FW changes via an input prompt rather than password parameter
v0.44
- make it easier to define new queueadapters, and add documentation
v0.43
- fix bug introduced in v0.4 that caused rlaunch rapidfire to stop working
v0.42
- fix bug introduced in v0.4 that caused update_time to be NULL for launches
v0.41
- add set_priority function to LaunchPad
- minor bug fixes related to multi-launcher and default queue params
v0.4
v0.37
Caution
The default behavior in ScriptTask is now fizzle_bad_rc.
- add lpad add_scripts
- fizzle_bad_rc by default in ScriptTask
- add FWorker() by default in rlaunch
v0.36
Caution
The rerun_fw, defuse_fw, and reignite_fw commands are now pluralized, refresh_wf is simply refresh, and rerun_fizzled has been incorporated into rerun_fws.
- much more powerful control for rerun_fws, defuse, archive, reignite, defuse_fws, reignite_fws, refresh.
v0.35
- restore behavior back to v0.33
v0.34
- deprecated - rename FIZZLED to FAILED
v0.33
- concatenate the update_spec and mod_spec of all FireTasks, instead of exiting as soon as a FireTask updates a spec.
v0.32
- change templating language to Jinja2 (and remove heavyweight dependency to Django)
- add ability to manually refresh workflows
v0.31
- fix bug related to interaction between multi job packer and job checkout optimization
v0.3
- multi job launcher to ‘pack’ jobs (Xiaohui Qu)
v0.25
- make paramiko optional as it can cause install problems
v0.24
- TransferTask added
- fix _use_global_spec
v0.23
- delete useless dirs when setting _launch_dir
- ScriptTask and TemplateWriterTask have _use_global_spec option
v0.22
- allow user to control where a FW gets executed using _launch_dir
v0.21
- add TemplateWriterTask plus documentation
- check for duplicate serialized objects
v0.2
- initial (alpha) release of Web GUI from Morgan Hargrove
v0.196
- bugfix to detect_unreserved script
- fixes to pip installation and instructions
v0.18
- add fizzle_bad_rc option to ScriptTask
- major doc additions and updates
v0.17
- minor update to ping()
- major docs reorganization and updates
- document and better support ‘pip’ installation
v0.16
- refactor AVOID_MANY_STATS into more tunable QSTAT_FREQUENCY
- speed up counting operations
- add more indices
- better log queue submission errors
- auto_load() function for LaunchPad
- queue launcher fills in previous block if not full (modifiable in FWConfig)
- many doc updates
v0.15
- add ability to ARCHIVE FireWorks
- update docs regarding enhancements to querying FireWorks and Workflows
- option to avoid overloading the queue management system with status requests
- more robust PBS adapter implementation
v0.14
- pin down and fix known issue of launches sometimes not being updated
- further refine display options and enhancements for get_fws and get_wfs.
- minor enhancements to queue launcher and PBS adapter
- support user indices for workflows
- minor bugfixes and internal code cleanup
v0.13
- multiple query and output display options and enhancements added for get_fws and get_wfs.
- use FW’s name to set more informative PBS job names
- make sure ping_launch only writes on running jobs (prevent race condition)
- minor bugfixes
v0.12
Caution
The get_fw_id and get_fw LaunchPad commands were merged into get_fws.
- better support for getting states of FireWorks and Workflows
- minor bugfix for dynamic FireWorks
v0.11
- rerunning FireWorks
- misc fixes for categories