This is the last release to support Python 2.x as there are hardly any more systems that allow installing such old versions
If the runner finds the environment variables associated with the
job-scheduler SLURM (most noabley SLURM_JOB_ID) it assumes that it is run as part
of a SLURM-job.
This means that the job id is written to the PyFoamServer.info file and that the server has
two new callbacks
Two new options allow displaying additional info (if present):
pyFoamNetShell
JobID is found by the squeue command a *
is displayed after the ID
in addition the option
pyFoamListCase.py now displays serial and
parallel runs in the same listingThe utiliy now automatically uses the parallel timesteps if there are more of those than serial timesteps
It is possible to switch to the old behaviour with a switch
In addition for collated processor data a remark col is added
pyFoamListCase.py optionally displays the number
of restartsIf the option --restart-number is
specified then a column with the number of restarts of the current
solver is displayed for each case (if there were restarts)
pyFoamClearCase.py allows clearing only
unaligned timesThe option --keep-aligned-to allows
specifying an interval where only timesteps that are aligned to this
interval are kept.
For instance
pyFoamClearCase.py --keep-aligned=0.1
removes timesteps 0.15 and 0.22 but keeps 0.1, 0.2 etc
pyFoamRedoPlot.pyThe utility now has two options --pixel-width and --pixel-height that allows setting the sizes of
the pixtures. The default is now 1024x800 instead of the 640x480
previously
pyFoamListProfilingInfo.py writes CSV-filesWhen the option --csv-file is used then
the raw results are written into a CSV-file
pyFoamBinarySize.py no scans user-binaries as
wellWith the option --user-binaries now the
binaries of the current user are added to the scanning
pyFoamListCase.py prints wall-time and core
usage hoursThe utility now has two options for output
These should help estimating the computational cost of the simulation
pyFoamListSolver.py shows solverThe option --show-solver now shows the
detected solver (if found)
pyFoamCompareDictionary.py handles compressed
filesThe utility now automatically compares files if one of them or both are compressed.
If the case has been written as collated then this class recognizes
the directory processorsXXX and the
timesteps in it (cloning is also properly handled)
_ as underscriptStrings for curve names and titles are interpreted by the png terminal in newer GNUPlot versions as
subscript which is usally not what is expected in our plots.
Per default now the unenhanced version for the png terminal is used which treats _ as plain _
To have _ being treated as underscript
select the png_enhanced implementation
RedoPlot now writes xlsx-files by defaultWhen the writing of Excel-files is requested then the utility now
writes the newer xlsx-format per default
(it used to be xls). The extension can be
switched on the command line
pyFoamListCases.py automatically displays
parallel runsThe output of the utility is now different: it automatically displays parallel timesteps. There is a switch to switch the utility back to the old behaviour
RedoPlot writes larger pictures per defaultThe default size of the written pictures now increased from 640x480 to 1024x800 (but can be changed with the new options)
The ThirdParty-library six has been update to avoid problems with Python 3.12
If (for instance because of a typo) the executable could not be found
then the pyFoamRunner-utility failed with
an error message that did not make sense.
Fix: a warning is issued and the runner tries to start anyway
.analyzed directoriesWhen using a Runner-utility to restart a run there are logfile.restartXX-files created but the original
.analyzed directory is removed (including
the pickledPlots).
This has been fixed: now there are corresponding analyzed.restartNN-directories created. They are
removed if a "fresh" (non-restart) run is done
If a process ends befor the amount of memory is checked then an exception is printed.
The exception is now caught
logscale with pyFoamRedoPlot.pyThe reason was that this information was not stored in the CustomPlotInfo because parsed values were set as
a BoolProxy instead of a bool and this was not added to the serialized
data
This was fixed but will only work for new pickledPlot files
For the gnuplot-plotter the titles were
missing from the hardcopies. For the matplotlib-plotter the titles were missing from
the y-axes.
Both were fixed
gzip does not properly flush on some
versionsIn some Python-versions a "on the fly" created gzip.open does not properly flush after writing
(code after it might find an empty file). This caused a
problem in the TemplateFile-class which
has been fixed by assigning to a variable and explicitly closing the
file
pyFoamRunner.pyBy default this utility writes the pickled plot data. The option
-no-pickled-file switches off the writing
of these files (this is the same option that the pyFoamPlotRunner.py-utility has)
For Runners the first keyboard interrupt (Ctrl-C) does not kill the process but sends an interrupt signal that allows finishing gracefully. The second Ctrl-C kills the process
Jinja2This is especially important for pyFoamPrepareCase.py. Utilities that rely on
templates now have options to select the templating engine (or
automatically chose it per file).
This is due to the Jinja2 support
described in the library section below
The sources now support QT5 (tested) as well as QT4 (not tested)
blockMesh with the edited
dictionaryWhen running blockMesh inside the
utility it uses whatever blockMeshDict is
currently being edited instead of the default system/blockMeshDict
--list-custom-regexpThe option -quit-after-list-custom-regexp does just that:
quitting after listing the custom plotting expressions
--list-custom-regexpOutput is now improved by
If a collector plot is enabled but its publisher plot is not then the publisher is automatically enabled. The old behaviour can be switched on with an option but will most likely crash
customRegexp be selected by labelTo ease switching on and off big numbers of entries in customRegexp it is now possible to assign labels
to them and select which are plotted by label.
To enable this you can add label
entries to an entry in the customRegexp
like this:
minMaxTemp {
theTitle "Extremes of the temperature";
expr "Temperature min=(.+) max=(.+)";
labels (
temperature
extreme
);
}
This means that this entry can be either selected with the label
temperature or the label extreme.
No if pyFoamPlotWatcher.py is called
with the option --label-to-use=temperature
then all entries from the customRegexp are
disabled with the exception of those that have the label temperature (like the one above). The option can
be used more than once to select multiple labels.
This feature allows easily switching on and off groups of plots
dynamicCode-directoryThe dynamicCode-directory (that holds
compiled code from coded functionObjects
and boundary conditions) is now automatically removed when --clear is requested (or with the pyFoamClearCase.py-directory. This can be
switched off with --keep-dynamic-code
For Pyratemp-template files when there is a syntax error in the file now the filename is give as well as the line number and the column
The TemplateFile-class (on which pyFoamPrepareCase.py relies heavily) now allows
alternate templating engines in addition to pyratemp (which is included in PyFoam). The class constructor takes an argument
engine that specifies which engine to use.
It can by set to pyratemp or jinja2 (if that library is installed) but the
default is auto. This means that it tries
to determine the type of engine to use from the first few lines of the
file. If it find the string template_engine:X, engine:X (with X
being the name of one of the engines) or an engine name then this engine
is used. If it finds nothing then it defaults to pyratemp (unless another engine is configured as
the fallback default).
For details on the Jinja2 templating
language see https://jinja.palletsprojects.com/en/3.1.x/templates/
PyFoam adds some filters to Jinja that make working with OpenFOAM-cases
easier:
ParsedParameterFile and allow some of the
options of these to read special files (that are not "dictionaries with
a file header")
SolutionDirectory-object from this path. This
allows accessing different information about a case and files in it.
{% set
%} statement
allowExec. Not available for old Jinja2-versions
(2.x)
The output with the --long-progress-option now show entries in a
different order: the old behaviour was that they are sorted
alphabetically. In the new order they are shown in the order in which
they were specified/read.
ReleaseNotesEmacs now adds those automatically on save. This avoids "busy"
commits because of the automatically created anchors in generated ReleaseNotes.md that change at every change of
the original file
Having the suffix .development in the
version string breaks the building of wheels. To allow the building of
wheels from development versions the version of development versions is
now the version number of the last release with an added sufffix .99
--dump-custom-regexp did not abort the
applicationThe help message promised that this exits the application after printing the custom regular expressions but in fact the application continued running.
This has been fixed by quitting the application
Minor enhancements. This release was made necessary because OpenFOAM 10 broke the plotting utilities with a new format for the simulation time
Adapt parser to parse more files that it doesn't understand
= in it is a reaction) in a #includeFunc with parameters (if there is
already a ))openfoamplusThe development version of the ESI-branch is now identified as openfoamplus instead of openfoam
OpenFOAM 10 adds a s (for second) to
the output of the Time. This broke the
regular expression that was used to recognize times during log analysis.
Which broke the plotting utilities
In newer version of OpenFOAM the arc
specification for edges can be different.
This breaks the parsing with ParsedBlockMeshDict
If there are two semicolons in a row (or dictionaries that consist of only a semicolon) the parser fails. This has been fixed by discarding the semicolon
Because development versions don't return a proper version number the execution time analyzer assumed that this is a very old version and did not parse it properly
pyFoamPVSnapshot.py with certain
ParaView versionsFor some ParaView versions the utility fails due to an import problem. This is fixed by trying to import the failing library before doing the actual import because the second import of the library will work
This fix was found in https://discourse.paraview.org/t/issue-with-pv-5-10-with-the-python-shell-and-help/8859
A minor bugfix release
WM_COMPILE_OPTION doesn't workThe --force-debug option doesn't work
because of some changes to the init-scripts. This has been fixed
pyFoamPrepareCase.py reports about 1 processor
directoryBecause of a change in the way processor directories are detected
there was a warning at the end of pyFoamPrepareCase.py about 1 processor directory
being present even if there is none. This has been fixed
master / slave plots to collector / publisherAs the termonology of master / slave -processes (which has been common in
programming for processes where one does the work and the other consumes
the products of that work) is increasingly seen as offensive it was
decided to change this terminology for plots which got their data from
two or more sources.
Instead of the previously used type slave in customRegexp which was used for plots that sent
their data to another plot to be displayed in that plots window now the
type collector should be used. Instead of
the entry master now an entry publisher is used.
This new nomenclature also describes the relationship of the two plots more accurately. The previous names were inaccurate as the "master" was doing the work of displaying the data while the "slave" was doing nothing
In order to keep the transition painless for users the old master / slave
keywords still work so old customRegexp-files don't have to be changed
pyFoamListCases.py indicates if the simulation
ran to the endNow if the first timestep is the startTime and the last time is the endTime then it is indicated by a =
between them (instead of the usual -) in the list
pyFoamBinarySize.py now reports the size of
ThirdParty as wellThe utility now looks for the corresponding ThirdPArty-directory of an installation and sums
that as well. This can be switched off
pyFoamPlotRunner.py now also supports pre- and
post-hooksThe utility now also supports the pre- and post-hooks that pyFoamRunner.py supports. But they have to be
enabled explicitly
pyFoamListProfilingInfo.py now removes nodes
with little computation timeWith the option --threshold-low it is
now possible to remove nodes with little computation time that would
clutter up the output. Input is a number between 0 and 100 and nodes
whose total computation time (including children) is less percent
(compared to the overall computation time) then the node is removed
pyFoamListProfilingInfo.py now allows generation
of graphsWith the option graphviz-dot the
utility produces output that can be piped into the dot-utility of the GraphViz suite.
The nodes are colored by the amount of computation time they use (without their child nodes). The edges are colored by the fraction of computation time the child node uses of the parent node. The thickness of the edges corresponds to the percentage of the overall computation all child nodes use
The utility has options to change the title of the plot and the color theme
The algorithm to reduce the amount of data plotted has now been improved. The previous algorithm produced "coarse" artefacts in the beginning of the plot because the actual time (and thus the resolution of the x-axis) was not considered. The new algorithm tries to distribute the reduction evenly over time so that "every" pixel of the x-axis has at least one value.
The most recent data points are not coarsened. The fraction of points
that is not coarsened can be set with the option --fraction-unchanged-by-splitting with a default
of 0.2
For large arrays the preparation of the data to plot in gnuplot was very inefficiently implemented in
the original Gnuplot-library.
The code now uses numpy and speeds up
that part of the code by 10-20 times (for large arrays)
These changes have been reverted and replaced by a different
implementation as the numpy-implementation
was slower than expected
Now the number of processors in a case is now detected in such a way:
processor followed by numbers are found then it
is assumed that the number of processors is the number of those
directoriesprocessors followed by a number is found then it
is assumed that this number is the number of processors (this is the
case for runs with collated data writing)This is honored by the running utilities with the --autosense-parallel option as well as the --clear-option
Because this is the first version with a leading 2 the pattern (which assumed a leading 1 did not find it. This has been adapted that
until v3912 all versions will be
adapted
0 directory
in the templateReported by David Blacher at https://www.cfd-online.com/Forums/openfoam-community-contributions/60683-first-discussion-thread-about-pyfoam-33.html#post786352
that the handling of the 0-directory is
broken for parameter variation runs where no 0 exists in the template case. This is fixed
according to his suggestion
snappyHexMeshAs reported on https://sourceforge.net/p/openfoam-extend/ticketspyfoam/227/
by Guillaume Broggi the auto-plotting of markers for snappyHexMesh causes problems with pyFoamRunner.py because the Dummy-plotter does not implement the appropriate
methods. This has been fixed
pyFoamPrepareCase failThis was reported on the MessageBoard
If only one parameter is defined then the routine for formatting the parameters failed
Fixed
Currently if a customRegexp is not
matched at one time-step then a data line with the
values 0 is added. The solution is to set this default to NaN
The default value if a custom plot value was not found at a time-step
used to be 0 and was plotted as such. In
the current release this has been changed to NaN (Not a Number). Gnuplot interprets this
correctly as "missing value". For other plotting interfaces this might
lead to undefined behaviour
error method of PyFoamApplicationClass now reports the location
it was calledUntil now the report only printed the location of the method (which was only of limited usefulness). Now it prints the location where the method was called
This is the first version of PyFoam where the version numbers change from pseudo-semantic versioning to date-based versions
Newer versions of Paraview may use Python 3 as the Python-interpreter
which made pyFoamPVSnapshot.py fail.
The Paraview-utiliies (and the library as well) have been adapted to work with Python 3 and 2
pyFoamPrepareCase.py allows automatically
zipping template resultsThe utility now allows automatically zipping the results of template evaluations if the result file has no extension (because then it can be assumed that these files are OpenFOAM-dictionaries which are transparently unzipped).
This allows automatically ignoring this files with proper patterns in
.gitignore or .hgignore
This can be switched on via the command line or the LocalConfigPyFoam file
customRegexp has a type mark to add marks to the plotsPlots of type mark don't plot but there
is a list targets with names of other
plots. Every time the expr matches a
vertical line is added to those plots.
Purpose of this type is to annotate singular events in the graphs
snappyHexMeshThe pyFoamPlotRunner.py and pyFoamPlotWatcher.py now allow plotting the
numbers of cells in different refinement levels in snappyHexMesh
Different phases are annotated with vertical lines
foamyHexMeshThe pyFoamPlotRunner.py and pyFoamPlotWatcher.py now allow plotting the
numbers of inserted points and the total displacement and distance of
foamyHexMesh
typeInstead of an obscure error message it now prints a list of the available types and a short descriptions
customRegexp-specifications
now give better error messagesNow the complete spec is printed (in addition to the missing attribute)
--quiet-plot for plotting utilities
swallows output of the plotting programgnuplot sometimes outputs error
messages which mess up the ncurses-output.
The --quiet-plot-option swallows this
output. This behaviour is not the default because
sometimes this output is useful
pyFoamPlotWatcher for logfiles
from restartsIf following a file which has .restartXX-files then when the file is changed a
red marker line with the label "Restart" is shown in all the plots
writeFiles in a customRegexp-entry writes scanned output to
fileIf a line
writeFiles yes;
is added to an entry in customRegexp
then the data is written to a file as well. These files are found in the
.analyzed-folder
Plotting data is reduced from time to time if the number of data
points exceeds a certain threshold. Usually this threshold is 2048. Now
for most runner utilities (and the PlotWatcher) there are two additional options:
--split-data-points-threshold allows
setting the number of data points to a different value than 2048 and
--no-split-data-points to switch off
splitting altogether
xvalue in customRegexpIf an entry xvalue is found in a customRegexp-specification then the value with
that name is not plotted but used instead of the time
as the x-coordinate for the
plot. So something like
highspeedLocation {
theTitle "Location of the highest velocity";
expr "Expression highSpeedLoc : min=\((.+) (.+) .+\)";
titles (
x
y
);
xlabel "x";
ylabel "y";
xvalue x;
}
plots the location of the highest speed
See above: Paraview-utilities now work in Paraviews that use Python 3
TemplateFile now can write the result as
zippedThe writeFile-method now has an
optional parameter gzip that forces the
file to be written in compressed form with the extension .gz added. If the file already has the extension
.gz it is assumed that gzip is set. If a file of the same name with an
extension .gz exists then it is assumed
that this is to be overwritten. If a file is written zipped and the same
file without .gz exists then it is removed
to avoid confusion which file will be used
alternateTime in single customRegexpThe entry alternateTime now allows to
reference special expressions that will serve as an alternate "time
source". This was for instance used to implement the progress graph for
snappyHexMesh
quiet-option added to plotting
implementationsThis option tells the plotting program to not output anything to the
terminal. Currently only works for Gnuplot
[]-operator of the PyFoamDataFrame is now more flexibleIf that operator gets a single numeric value or a list of numbers it
gets the rows where the index (usually the time) is nearest to the
number(s) and returns a PyFoamDataFrame
All other keys are passed to the "usual" []-operator of the DataFrame but the result is converted to a PyFoamDataFrame
BasicRunner has a unique IDEach run gets a unique id. This allows to easily find out whether the data is from the same run
pyFoamAddCaseDataToDatabase.py allows updating
dataThe --update switch allows updating the
data for runs. Otherwise if the run already exists in the database the
utility will fail
RunDatabase gets method modifyThis method allows updating a run with a unique id by specifying a (nested) dictionary with the values
auto for the solver does not work with
compressed controlDictIf the controlDict was compressed the
value of the application-entry could not
be read. This has been fixed
FileBasisBackup now works with zipped fileIf a file is already zipped then the FileBasisBackup class could not create a backup
file and failed. This works now
controlDict not recognized as a
valid caseIf the controlDict was zipped for some
reason then PyFoam (and therefor utilities like pyFoamListCase.py) did not recognize it as a
valid case
pyFoamDisplayBlockMesh.py not working with newer
VTK-versionsThe utility did not work with newer versions of VTK. This has been fixed. In the process support for Python 2.x has been broken
TemplateFile writes to zipped file if it
existsThe method writeFile looks for a file o
the same name with .gz added. If this
exists then it is assumed that this should be written (in zipped
form)
pyFoamDisplayBlockMesh.py not working with
Python 2.x anymoreThis utility now requires Python 3.
It is not typically run on computation servers (which mostly still have Python 2) so the extra effort to support this outdated version of Python is not worth it
PyFoamDataFrame is more restrictiveThe constructor now checks if
because most additional algorithms (integrate() etc) rely on these assumptions
[]-operator of PyFoamDataFrame returns a PyFoamDataFrameThe old behaviour of this was to return a DataFrame. Shouldn't break existing code but
makes it easier to chain indexes
RunDatabase fails if the same unique ID is
inserted againIf a dataset with the same unique ID is added with add for a second time the method fails with a
KeyError unless the update_existing-flag is set
This has the potential to beak old code (which probably wouldn't work correctly anyway as the old behaviour was to add the data for the rum for a second time)
Gnuplot-libraryThere has been a quiet-option added
that swallows all terminal output of gnuplot. This only works in the
Unix/Linux-implementation. All others ignore it
src-directoryTo make sure that the tox-tests are not
affected the library is moved into the src-subdirectory
Added a file DeveloperNotes with hints
for people who want to contribute
customRegexpMacro expansion in the customRegexp
might break it for some cases
FIFO as the default anymoreSee the relevant entry in Enhancements to Utilities
A potential problem is that the new implementation leaves files in
the /tmp filesystem
customRegexpThere are two new type s available in
customRegexp: data and dataslave. These have to have one of the
enxtries csvName, excelName or txtName with the name of a "spreadsheet type"
file. The data from this file will be plotted according to the current
simulation time. With dataslave it will be
plotted to another plot.
The entry timeName interprets the name
of the column that is to be interpreted as the time and either validData or validMatchRegexp select the other columns to
plot.
skip_header, stripCharacters and replaceFirstLine are other parameters of the
SpreadsheetData-class that preprocess the
file to conform to an expected format
customRegexpIn the customRegexp it is now possible
to use the usual OpenFOAM-macro-expansions with $ etc. This makes
progress entry in customRegexp now allows format stringsThe progress entry in customRegexp-entries now allows strings in the
format of the str.format method in Python.
So instead off
progress "$0: $1";
one can write
progress "{0}: ${1}";
The advantage of this format is additional flexibility. For instance the length of the strings can be fixed
progress "{0:5}: ${1:10}";
Note: the entries are strings. Not numbers as expected
pyFoamRedoPlot.py allows passing terminal
optionsThe utility now allows passing options to the plotting implementation
with the --terminal-options-option. This
can for instance be used to modify the size of the plot
pyFoamPlotWatcher.py stops scanning the file is
--end was specifiedIf the --end-time option is specified
then the solver stops scanning if that time is reached. The plot windows
are killed. To keep them specify --persistent
Instead of the custom00xxx name the
hardcopies of custom plots now have and additional short name that
describes the content of the plot (it is taken from the id in the customRegexp)
The plotting utilities have two new options --gnuplot-use-fifo and --gnuplot-no-use-fifo that switches whether the
plotting windows will use a FIFO-queue or a regular file for the data of
the plot.
Until now FIFO was used. This had two problems
Now the default behavior is to use regular files. The problem with
this is that the files are not removed in the end from /tmp. This shouldn't be a problem as usually
that filesystem is purged of old files at regular intervals
pyFoamPrepareCase.py calls script after copying
initial conditionsA script postCopy.sh is called after
the initial conditions are copied from 0.org
--stop-after-template and --keep-zero improve control in pyFoamPrepareCaseParameters.pyThe combination of these two parameters allow "just" changing something in the templates without running other lengthy operations
pyFoamPVSnapshot.py allows specification of the
image qualityThe option --quality now allows
specifying the quality of the image with a value of 0 being worst (but producing the smallest
pictures) and 100 best (but producing
huge pictures). The default is 50
pyFoamPVSnapshot.pyThe options --x-resolution and --y-resolution allow specifying the resolution
of the image. If only one of them is set the image is scaled
proportionally. This only works for Paraview versions bigger than
5.4
pyFoamPVSnapshot.pyTwo new options were added that allow setting a separation between different views and making the background transparent. This only works for Paraview versions bigger than 5.4
pyFoamPVLoadState.py automatically uses
decomposed or reconstructed dataDepending which of the two sets has more timesteps that state is set
to this before loading. So if more parallel timesteps are present then
these are used even if the state file uses the reconstructed times. The
behavior can be changed with the --decompoes-mode-option
pyFoamPrepareCase.py to
targetThe option --extecute-in-case-directory
changes the working directory to the target directory. THis allows
specifying parameter files that are in that directory without a full
path
pyFoamPrepareCase.py can create an example
caseA command like
pyFoamPrepareCase.py exampleCase --paramter-file=parameters.base --build-example --clone-case=originalCase
creates an example case exampleCase
from a template case originalCase using
the parameter file parameter.base. It
creates a script Allrun that allows
executing the case without PyFoam (if none
of the scripts uses PyFoam-scripts)
This may not work for all configurations (especially cases that use
postTemplate)
pyFoamPrepareCase prints derived valuesThe same way that the utility printed the used values it now prints the derived values as well
pyFoamPVSnapshot allows specifying different
colors for different viewsThe option --color-for-filers now
allows specifying a different color for the same filter in different
view. This is done by specifying a dictionary
alternateLogscale for custom plotsThis is analog to logscale but for the
values that are specified with alternateAxis
pyFoamBinarySize.py now calculates documentation
size as wellIf there is html documentation then
this is counted as well
pyFoamCompareDictionary.py allows specification
of significant digitsWhen comparing numbers now the number of significant digits can be specified. This only works for single numbers. Not compound types like lists and vectors
progress-data is automatically converted to
floatWhen using format-strings for the progress-entry then the library automatically
attempts to convert the data to float
(otherwise it keeps it as str)
FoamInformationTwo functions foamCaseDicts() and foamPostProcessing() have been added that return
the paths to these directories inside $FOAM_ETC
BoolProxy now works correctly with
!=Added a method __ne__ so that the
results of the != operator are consistent with
==
_slave are
problematicThis made the slave plots that had _slave in the name fail
As reported in https://sourceforge.net/p/openfoam-extend/ticketspyfoam/223/
by Rodrigo Leite Prates parsing certain constructs that involve
dimension sets fail. This is because of a problem with the comparison of
Dimension that assumes that the other side
is a Dimension as well. Fixed
pyFoamPVSnapshot.py not working with Paraview
5.6The API now has to be called through a different module. Otherwise it will fail
customRegexp farthes away was usedWhen looking automatically for a customRegexp the one furtherest up in the
directory tree was used. Now instead all the customRegexp are used with the lower ones
overriding the other ones
ParameterFile-class got confused by commented
linesOne of the oldest classes in PyFoam had the problem that it "found"
parameters that were commented out with //. This has been fixed
pyFoamBinarySize.py did not count files in buildSome distros have a directory build
with the intermediate object files. This has not been counted until
now
ParsedParameterFile not working
in Python 3Because Python 3 tries to encode read files as Unicode strings and certain byte combinations are not valid UTF-8 encodings.
Hopefully fixed by reading the file as binary and then create a latin-1 encoded string
Reported in https://sourceforge.net/p/openfoam-extend/ticketspyfoam/225/ by Johan Hidding
Parts of the FileBasis class were not
working correctly in Python 3 because there strings are no longer 'lists
of bytes'. This has been adapted so that these parts work correctly in
Python 2 and 3 and unit tests have been added
This is only a minor release with the main purpose to recognize OpenFOAM 6 installations with their new numbering scheme
pyFoamPrepareCase.py does not execute
decomposition scripts for single processor casesIf numberOfProcessors is smaller than 2
then the decomposition scripts are ignored. This may cause problems in
the unlikely case that the setup process relied on these scripts being
always executed
pyFoamFunkyDoCalc.py to compare data from
funkyDoCalcThis utility compares data written by the funkyDoCalc-utility from swak4Foam (this data is written with the -writeDict-switch
For details on the usage see the online help of the utility
pyFoamListCases.pyThe --recursive-option now recursively
searches the specified directories for cases. Without the option it
behaves the way it did before
customRegexp in parent directoriesThe plotting utilities now look for customRegexp-files in parent directories of the
case directory. This allows to use one file for a number of cases. The
file in the directory is still used. The behavior can be switched off
with the --no-parent-customRegexp-option
pyFoamPrepareCase.py does not execute
decomposition scripts for single processor casesIf numberOfProcessors is smaller than 2
then the decomposition scripts are ignored
pyFoamPrepareCase.py checks for proper
decompositionAt the end the utility now checks if the number of processor
directories is consistent with the specified --number-of-processors
pyFoamPlotWatcher.py automatically uses newest
logfileIf a logfile auto is specified then the
utility uses the newest file with the extension .logfile in the current directory.
Like any automatism this might produce unexpected results. So use with care
FoamFileGenerator handles OrderedDictThe class now preserves the order if an instance of OrderedDict is found (instead of the usual
behaviour of sorting the keys to always get the same output)
#sinclude handled as an alias to #includeIfPresentOpenFOAM v1812 introduces this as an alias. It is now handled by the parser similarly
With OpenFOAM 6 the naming scheme changed again. Instead of 6.0 (which PyFoam would have recognized) it is now plain 6. PyFoam now recognizes both forms in the directory name
pyFoamPrepareCase.py did not remove processor-directoriesNow installations with names like OpenFOAM-6 are recognized
curses-output to UtilitiesOn terminals that support it there is now an additional option --curses that enhances the output of the
utility. It adds a header and a footer-line and colors the output if it
finds special strings like the current time.
Some utilities enhance the output event further: the Runner and
Watches utilities color lines that match regular expressions that are
sought for: the line is colored differently and data items (which are
usually plotted) are colored even more differently. There is also an
additional footer-line with the string that is usally printed by the
--progress-option
Output can be paused by pressing space
and resumed by pressing space again. This
behavior may be overridden by some utilities
Caution: on some terminal implementations resizing the terminal causes a segmentation fault of Python which may stop your simulation
pyFoamPrepareCase.py creates .foam-fileThe utility now automatically creates a file that allows Paraview to open the case
4.0The hardcoded Foam-version that is used if the WM_PROJECT_VERSION environment variable is not
set is set to 4.0 from the rather ancient
version 1.5
none no longer parsed as an equivalent for falseThis breaks the parsing of cases where none is used as a word.
pyFoamJoinTimelines.py to join Timelines from
restartsThis utility joins timeline files from different restarts. The lines from times that will be in the next file are discarded
pyFoamRestartRunner.py to automatically restart
runsFor cases that fail strangely (mostly due to problems in the linear solver) but restart successfully this utility helps running them.
The utility tries to restart the solver until either the endTime is reached or no new time-step is
written to disk (in this case it makes no sense to run again)
There have been variations of the regular pyFoamPVSnapshot.py added. The only difference
they have is that they set a option that enforces the used OpenGL-implementation (especially Mesa). Use
this run the script on a machine that don't have hardware support for
3D-graphics
For the surface film solvers there now properties like the mass, covered surface, thickness and velocity are automatically plotted
pyFoamClearCase.py automatically executes an
existing AllcleanIf present the script (which is usually found in tutorial cases) is executed before other cleaning takes places
pyFoamPrepareCase.py executes tutorial scripts
if availableIf there are scripts present from the original tutorials that do
only case preparation (like Allrun.pre) but no solver running and no special
scripts are present then the original scripts are executed
pyFoamPrepareCase.pyIf a special script clearCase.sh is
found this is used for additional clearing. If instead a script Allclean is found then this is used
pyFoamPlotWatcher.py now can handle multiple
filesIf the utility gets more than one file it tries to concatenate the data from theses files. It assumes:
Create mesh
for time = appears in the file it is assumed that this file is
the last one in the sequenceIf only one file is specified and matching logdfiles from restarts are found then these are automatically added (there is an option to prohibit this)
pyFoamPrepareCase.py now allows separate
decomposition scriptsAfter three of the phases
.org filesthe utility executes a script (decomposeMesh.sh, decomposeFields.sh and decomposeCase.sh) if found. This allows to adapt
for different situations (for instance: the mesh already being generated
in parallel)
If PyFoam thinks that a run is restarted (old time-files exist and
there already exists a logfile) it creates logfiles with .restart appended. This allows joining the
original log and the restart log
pyFoamPVSnapshot.py improves rewriting of
state-filesThe utility now allows changing properties before loading it into paraview. There is also a number of options that allow setting up this process.
First the available sources can be listed with --analyze-state-file. From these sources the
properties (with values) for one of these sources can be listed like
this
pyFoamPVSnapshot.py . --list-prop=probeCSV
then one propery can be changed like this
pyFoamPVSnapshot.py . --set-property=probeCSV:FileName=probes.csv
--set-property can be used more than
once
pyFoamPackCase.py adds parallel dataWith the option --parallel now adds
parallel data
--replacement-option in pyFoamPVSnapshot.py supports Foam-formatThe option can now alternatively use Foam-format instead of Python-format
pyFoamPVSnapshot.py improved error messages with
problems in replacementInstead of a stack trace there is now an output of the template string and the available values
customRegexp now searched in parent
directoriesIf in the directory of the log-file no customRegexp and the log-file is not in the
current directory then PyFoam looks for it in all directories up to the
current directories
Paraview.StateFile extendedThis module has been extended to allow more flexible manipulations of the state-file
BasicRunner now checks for regular EndThe BasicRunner now checks whether the
string End was seen and
no time-information after that. This means that the
simulation reached its "regular" end and is also reported in the PyFoamState.TheState-file
pyFoamPrepareCaser.py ran out of memory for
large script outputsBecause all the output from scripts was stored in memory before being written to a log it was possible that the utility ran out of memory when there was much output. The output is now written directly to disk
WM_PROJECT_VERSION is
unsetScanning for the Courant number defaulted to the versy old version. This has been fixed
pyFoamPVSnapshot.py--rescale-color-to-source did not work
for sources that have no cell values (like streamlines). Fixed.
socketserverSome installations of Python 2.7 already have the socketserver-module which does not have the
required BaseServer-module. Fixed
v1706As the + is not present in the WM_PROJECT_VERSION this distro was detected as
the Foundation fork
pyFoamNet-utilities now work without a
Meta-ServerAs described in other changenotes below the Server process now
announces its presence with ZeroConf. This means that a central
Meta-Server is no longer necessary. But the zeroconf library has to be present on all
involved machines (server and client). It is installed with
pip install zeroconf
PyFoam.Infrastructure.AuthenticationThis module implements a simple public key authentication. For authentication a username and a challenge are sent. If the username is in the set of authenticated keys (or is the own username) then this key is used to check the challenge.
pyFoamClearCase.py now has -dry-run optionThis option doesn't clear anything but prints the things that will be erased
--keep-time for pyFoamClearCase.pyThis option (which can be specified more than once) allows specifying single time-steps that should be kept
pyFoamNetList.py no longer needs a meta-server
to workDue to the addition of ZeroConf this
utility no longer needs a Meta-Server to find running calculations in
the same subnet
If the psutil-library is installed then
the memory used by parallel runs is calculated as well
PyFoam.Sitemodule specified in preRunHook_ and postRunHook_ is now searched in PyFoam.Site. as well. This module is the lib-directory in the directory specified by the
environment variable PYFOAM_SITE_DIR
(which allows adding user-scripts and modules)
OpenFOAM+ v1706This fork has changed its numbering scheme (dropped the + in the version string). This broke a regular
expression and a function to detect the number
Runner-utilities now register as ZeroConf-servicesUnless switched off every pyFoam(Plot)Runner.py-controlled run already had
a server process that allowed controlling the process. Now the process
also advertises itself via ZeroConf (aka
Avahi, Bonjour, mDNS. See
https://www.wikiwand.com/en/Zero-configuration_networking)
This allows the Net-utilities to
operate without a meta-server.
Due to the limitation of the protocol this only works reliable in the same broadcast-subnet
--keep-interval in pyFoamClearCase.py not working for
parallel-casesDue to a copy/past error this option did not work for parallel cases. This is now fixed
The new PLY-version and six now at least requires this Python-version.
If your system has Python 2.5 or older stick with PyFoam 0.6.6
pyFoamPVSnapshot.py
differDue to changes in the time-step numbering the names of the generated
files differ. If your work-flow depends on the old names add the options
--consecutive-index-for-timesteps and
--duplicate-times
pyFoamListProfilingInfo.py to print
profiling dataThe utility reads the profiling info written by
and prints it in a human-readable form
pyFoamBlockMeshConverter.py to convert a
2D-mesh to 3DThis utility (and the required Libraries) was written by Hasan Shetabivash (original at https://github.com/hasansh/blockMeshConverter.git)
This utility takes a file that is written in a similar syntax as the
regular blockMeshDict but with the third
dimension missing. It then converts this file to a regular blockMershDict by either extruding in the z-direction or by rotating around
the x or the y-axis
customRegexp now can scan for textsUntil now data items (groups) in the regular expression had to be
valid numbers. Otherwise a warning was issued. This behavior is still
the default but if a list-variable stringValues is added then the items (whose
numbers are specified in the list) are not being plotted. They are
written to disk with --write-files and
they can be used in progress
PyFoamHistory escapedAdditions to the history file where arguments contain whitespaces and/or quotes are quoted and quotes inside are escaped. This allows these command lines to be copy/pasted to the command line
--values-string of pyFoamPrepareCase.py now accepts
OpenFOAM-formatThese strings are now parsed as OpenFOAM-strings if there is no
starting { and no ending }. With these the old behavior (parsing as a
Python-dictionary) is used
pyFoamRunner.py and pyFoamPlotRunner.py allow automatic selection of
solverIf for those utilities the word auto is
written in place of a proper solver (like interFoam) then the utility looks into controlDict for the application-entry and uses that
customRegexpIf an entry dataTransformations is
specified in the dictionary then the results of these transformations is
are for writing/plotting instead of the real data.
dataTransformations is a list with
valid single-line python-expressions. The groups from the regular
expression expr are inserted textually
into the expression before it is evaluated. $1 is the first group, $2 the second and so on (this convention is the
same as for the progress-string.
The titles-entry corresponds to these
results. If the "raw" (untransformed) data should be plotted as well
you'll have to add identity-transformations of the form "$1"
idNr for dynamic
in customRegexpidNr can now alternatively be a list of
integers (instead of a single integer). In that case the corresponding
matches are concatenated (with a _ between
them) to generate the data id.
If only a number is specified it has the old behavior.
As usual the indexes of the matches stat with 1 (not 0)
pyFoamListCases.py detects dead runsIf a run has not had any output in the last hour it is listed as dead. This threshold can be customized
pyFoamPVSnapshot.pyThe utility now numbers the time-steps by the order in the solution directory. Before that the used time-steps were numbered consecutively (from 0 to N − 1 if N time-steps were specified).
Also: by default each time-step is only processed once
The old behaviour can be reproduced with --consecutive-index-for-timesteps and --duplicate-times
Whether or not plots are plotted automatically (for instance the
execution time) can be set in the [Plotting]-section of the configuration. So it
can be set per-case in the LocalConfigPyFoam-file. If a plot is on by
default it can be switched off with the corresponding --no-option. If off by default the --with-option switches it on
derivedParameters.py-script called from pyFoamPrepareCase.py allows error reportingIn that script that calculates new parameters and also can do
parameter checking there now is a function error available that makes the script and the
complete execution fail
Both distros changed their scheme for the version numbers and the regular expressions have been adapted to detect them
SpreadsheetData now handles string dataIf one of the columns is string data then the ()-operator returns string values (when
interpolating the next value)
TimelineData tolerates string valuesThe class can now read strings without spaces (OpenFOAM words) and pass them to SpreadsheetData
() operator of SpreadsheetData works without nameIf no name parameter is given then the
method returns a dictionary with all the values
setCurrentTimeline in PyFoam.Paraview.Data to get data at timeThis function reads timeline data at a specified position. Return the interpolated values from the current simulation time as a table
Use in Programmable Filter. Set output
type to vtkTable. To get the values from a
timeline postProcessing/swakExpression_pseudoTime/0/pseudoTime
use
from PyFoam.Paraview.Data import setCurrentTimeline
setCurrentTimeline(self.GetOutput(),"swakExpression_pseudoTime","pseudoTime")
To print the values pipe into a PythonAnnotation-filter. Set ArrayAssociation to
Row Data and use an Expression like
"Time = %.1f (%.1f)" % (average,t_value)
(this assumes that there is a column average)
Hint: this reads string values as well. But in that case the value
has to be converted with val.GetValue(0)
in the expression
The method PyFoam.FoamInformation.getUserTempDir makes sure
that a user specific temporary directory exists and returns the path to
that directory
Gnuplot-plots now get better titlesInstead of Gnuplo the window titles are
now set to pyFoam and the actual title of
the plots. This should make it easier to find plot windows in the window
manager
ParsedParameterFile now supports #includeFuncIn case of a #includeFunc-entry the
file is either searched relative to the current file (this is where the
semantics differ from how OpenFOAM because that searches in the system-directory. But as this entry is usually
called from system/controlDict the result
is the same) and if not it looks for it in $FOAM_ETC
findFileInDirThis function in PyFoam.Basics.Utilities looks recursively for a
file in a directory
humandReadableDuration added to PyFoam.Basics.UtilitiesThis function takes a duration (in seconds) and prints it in a human-readable form
pyFoamVersion.py now reports the versions of the
ThirdParty-packagesNow these versions are reported as well for quick reference
The class PyFoamApplication assumes
that the module sys has an element argv. This is not the case inside Paraview
pyFoamPrepareCaseParameters.sh not
working on Mac OS XBecause newer versions of Mac OS X remove LD_LIBRARY_PATH from the environment passed to
scripts (for security reasons) additional libraries (for instance swak4Foam) were not correctly loaded. This has
been fixed by generating a special script that exports LD_LIBRARY_PATH before executing the rest
SolutionDirectoryThe class assumed that processorX
directories were added in the numeric order which is not necessarily the
case. This caused problems with pyFoamCaseReport.py
The utility function to delete directories failed if the directory didn't exist. Fixed
RegionCasesWhen creating a RegionCase only the
"known" files were symlinked. This causes some programs to fail. Now
everything from system in the master-case
is symlinked if there is not already a file of that name there
solver in pyFoamListCase.pyIf another utility was run in the mean-time the wrong solver is listed by the utility. Fixed
tqdm to version 4.8.4No reason. Just because there was an update
PLY to version 3.9This breaks compatibility with Python 2.5 or older
six
to 1.10.0This also breaks compatibiliy with Python 2.5 or older
IPython-notebooks 3.0With IPython 3.0 the names of the Widget classes lost the Widget in the name (for instance DropdownWidget becomes Dropdown). PyFoam
has been changed to conform with this but as a consequence won't work
with old version of the IPython
notebooks
pyFoamPrepareCase.py executes setFields if appropriateIf no setup-script is specified and if there is a setFieldsDict present then setFields is automatically executed
The configuration system now has a section Autoplots. The entries there are all
dictionaries with two entries:
customRegexp: expr, titles etc
If the solver matches solvers then a
custom plot based on plotinfo is
automatically added.
If the solver does not fit the plot is not added
(this helps avoid processing of unused expr. This is important because processing the
expr is one of the things PyFoam uses the most computation time for)
Some common plots (phase fractions for instance or most of the output
of chtMultiRegionFoam) are hardcoded into
PyFoam
Also there is a new section [SolverBase] in the configuration. It is checked
whether the current solver name begins with any of the
keys there. If it does then the list of solvers is assumed to be the
solvers this solver was based on and Autoplots for those solvers are added as well.
For instance
[SolverBase]
myReactingCht: ["chtMultiRegionFoam","reactingFoam"]
assumes that a solver myReactingChtFoam
is based on these two solvers and automatically adds their Autoplots
In addition if a setting like
[Plotting]
autoplots: cloudNumberMass
is set (for instance through LocalConfigPyFoam in the case) then the autoplot
cloudnumbermass is used regardless of the
solver name
alternateAxis-entries now can be regular
expressionsThis allows specifying plots generated with type dynamic on the alternate axis
These utilities now allow with the option --gnuplot-terminal to choose the terminal.
Otherwise the terminal specified in the configuration (usually x11) is used
Names in the legend are now sorted. This improves readability for large numbers of lines in the plot
pyFoamExecute.py allows calling with
debuggerThe option --run-with-debugger runs the
command in the debugger. The arguments are appropriately handled
pyFoamPrepareCase.py fails if execution of a
script failsIf the execution of a script fails (the status code returned by the
script is not 0) then execution of the
utility fails (before the utility just continued). This behavior can be
overridden with the option --continue-on-script-failure.
It is up to the script to ensure that the failure of a utility called
in the script is forwarded to pyFoamPrepareCase.py. For instance with
set -e
in a bash-script
--hardcopy in plotting library now allows
modification of gnuplot-terminalsThis option allows setting the options for the terminal selected with
--format-of-hardcopy. This overrides any
default set in configuration section [Plotting] under the name hardcopyOptions_<term> with <term> being the name of the terminal (for
instance for png the option is hardcopyOptions_png.
pyFoamPrepareCase.py writes state information
about what it is currently doingIt writes to the usual state file. This way pyFoamListCases.py will list this information.
If the scripts call pyFoamRunner.py then
this information will be overwritten
pyFoamBinarySize.py can handle new location of
binaries in OpenFOAM 3.0Since that foam version all binaries (and object files are located in
the directory platforms. The utility now
finds them there
Runner-utilites now can signal on blink(1)-devicesWith the option --use-blink1 these
utilities now flash on a plugged in blink(1) USB-device for every time-step
pyFoamExecute.py can flash a blink(1)To indicate that the utility is still running it is able to play a
pattern on a blink(1)-device. This is
switched on with -use-blink
pyFoamDecompose.py allows using a template
fileWith the option --template-dict it is
possible to initialize the output with an existing file. With this file
it is possible to add 'complicated' settings.
pyFoamTimelinePlot.py now handles new format of
probe filesProbe files now have one comment line per probe to specify the position. This format is now correctly detected and plotted. Old probe files are also handled
ReST-report of pyFoamPrepareCase.py now reports derived
parametersThe .rst-file written by the utility
now adds a section on derived parameters if such parameters were
specified in a script
pyFoamPrepareCase can now ignore
directoriesIt is now possible to specify directories that should be ignored when
looking for templates. Some sensible defaults like postProcessing, processor* and VTK
are already set
pyFoamConvertToCSV.py allows adding formulas to
XLSX-filesThe option --add-formula-to-sheet
allows adding formulas to the Excel-sheet. Something like
--add-formula="massflow:::'inlet'-'outlet'"
adds a column massflow that subtracts
the columns inlet and outlet
pyFoamListCases.py now displays mercurial
infoFor those who use mercurial to keep track of their cases the utility
now has the option -hg-info that displays
the mercurial hash-ID, the local id and the branch name
Using the library lqdm progress bars
have been added to utilities that have a long run-time and where the
progress bars are not disturbing the regular output. These utilities
are
pyFoamListCases.pypyFoamBinarySize.pyBars can be switched off with --no-progress-bar
pyFoamCleasCase.py and all utilities
that have a --clear option now also have a
--verbose-clear option that reports
what is being cleared
pyFoamConvertToCSV.py now allows manipulating
the inputThe utility now has two new options:
OpenFOAM-devA development installation is now also detected and it is assumed
that this uses the new calling convention. Also: PyFoam reports this as
version 9.9.9 (as this is larger than any
version in the foreseeable future
OpenFOAM+ as a forkAn additional fork type openfoamplus
has been added (in addition to openfoam
and extend). Installations of the form
OpenFOAM-vX.X+ (with X.X being the version number) are added to this
fork. Also OpenFOAM-plus is added as the
development version of this fork
blockMeshDictIn newer OpenFOAM-versions blockMeshDict may be located in system. PyFoam detects it either there or in the
old constant/polyMesh-location
ConfigurationLists and dictionaries now can also be specified. Have to be correctly formatted if they are longer than one line (indented by at least one space - convention for configuration files)
Configuration has method getArch for architecture dependent settingsIf an option opt is requested with this
option then it is checked whether an architecture-dependent setting
exists. Architecture arch is the output of
the uname-command. The
architecture-dependent name is opt_arch.
execute-method from PyFoam.Basics.Utilities returns status-codeThis function now has an option that makes it return the status of the execution as well as the output of the execution.
BasicRunner now supports more ways of stopping
runsIn the past this class (and the utilities based on it) looked for a
file stop and stopped the run (with
writing) if it was found. Now two additional files are looked for
Blink1 class to support blink(1) devicesThis class assumes that a blink(1)
USB-device is present and the API-server (from the Blink1Control-program for this is running. It
wraps these calls so that utilities can use them conveniently
ParsedParameterFiles now supports includeEtc#includeEtc is now supported
Uniform fields of the form 1002{42.5}
(Field with 1002 values 42.5) are now
correctly parsed
toNumpy-method added to Unparsed and FieldThese two classes have a method toNumpy
added that transformed the data into a structured NumPy-array. There are no applications for this
in PyFoam yet but an application will be
the parsing of lagrangian data
PyFoam.RunDictionary.LagrangianPatchData to read
data from patch function objectThis module reads data written by the cloud function-object that
writes particle data as particles hit the patches and transforms it into
numpy-array. Which can also be returned as
pandas DataFrames
It adds some properties to the data
globalId constructed from origId and origProcIdPyFoam.RunDictionary.LagrangianCloudData to read
cloud dataThis gets
DataFrameA globalId that is consistent with the
one in LagrangianPatchData is set
code added to =RestructuredTextHelperThis method formats a string assuming that it is a program code.
Default value is python
ParsedParameterFile now parses new dimension
format correctlyNewer OpenFOAM-versions allow dimensions in symbolic format (for
example [ m s^-1 ]). These are now
correctly parsed
ParsedParameterFiel now parses uniform fields
correctlyFields of the form 23 { 4.2 } (meaning
"23 times 4.2") are now correctly parsed
epydoc to sphinxAs expydoc is discontinued the
API-documentation is now generated with sphinx. Just run
make docu
to do so.
Advantage is that now with
make docset
a document set for offline searching with Dash (for Mac OS X) or clones (on other OSes)
can be generated
Untitests only used to run correctly if the OpenFOAM-version was 1.7. Are changed to run with OF 3.0. No effort has been made to support intermediate versions as the changes are mainly about changed tutorials
ExecutionTime breaks plotting
utilitiesIf the ExecutionTime is not as expected
pyFoamPlotWatcher.py and pyFoamPlotRunner.py finish with an error. This
is now more robust
phases not working with dynamic plotsFor dynamic plots the addition of the phase name did not work. Fixed
If phase was set the output of the
function objects got it added to the names even though the function
objects do not belong to the phase. This is fixed
When working with regions one region too many was added in pyFoamChangeBoundaryType.py and pyFoamChangeBoundaryName.py. Fixed
pyFoamClearCase.py fails on write-protected
caseIf a case is write protected then the utility failed. Now it only issues a warning and continues cleaning
pyFoamPrepareCase.py
confused by zipped filesWhen copying one file to another and one of them is zipped then copying doesn't replace the destination correctly but adds the zipped/unzipped variant
pyFoamPVSnapshots.pyIf snapshots were taken from state files with multiple layouts then some of the views had the wrong time (either that from the state-file or from the timestep before). Fixed
The data from the first timestep was not plotted under certain circumstances. This has been fixed
DYLD_LIBRARY_PATH not passed on Mac OS X
10.11Starting with this OS-version as a security feature the system does
not pass LD_LIBRARY_PATH and DYLD_LIBRARY_PATH to a shell. PyFoam detects this and creates these variables
and makes sure they are passed to the processes
pandas broke the writing of
excel files with pyFoamConvertToCSV.pyThe reason is that the old way of making axis data unique did not work anymore. This has been fixed
E in exponential notation for floats
breaks parserThis problem has been reported at https://sourceforge.net/p/openfoam-extend/ticketspyfoam/220/
(the number 1E-2 is not correctly parsed
to 0.01) and has been fixed
Runner-utilities clear processor directories if
first time in parallel data differsIn cases where the parallel data has a different start directory than
0 the pyFoamRunner.py and similar utilities cleared
that data and made a restart impossible. This has been fixed
pvpython not working when installed
through distutilsAs the distutils (and all mechanisms
built on these like pip) replace the used
python in scripts the necessary pvpython
was removed. This has been fixed by generating a temporary script file
that is actually executed with =pvpython)
tqdm for progress barsAdd the library tqdm (https://github.com/tqdm/tqdm)
for adding progress bars to utilities.
Library is under MIT License
PyFoam is now found at https://pypi.python.org/pypi/PyFoam
Recommended way of installing is using https://pip.pypa.io/en/latest/ :
pip install PyFoam
This will also make sure that the required numpy-package is installed
ArchiveDir in SolutionDirectory discouragedAs this was never really used it is discouraged (the option is still there).
If you don't understand what this means it probably doesn't concern you
All pickled files are now written and read in binary mode (as this was the only way that works consistently in Python 3). This may cause problems with old cases (but no effort has been made to check whether this problem actually exists)
PlotRunner and PlotWatcher now don't strip spacesThese two utilities now don't strip leading spaces from the read lines. This preserves formatting in the output but might break scripts that rely on these spaces.
The old behaviour may be reset by overriding stripSpaces in section SolverOutput with a value True
pyFoamConvertToCSV.pyThe enhanced naming of the columns might break scripts that rely on the old naming
pyFoamChangeBoundaryName.py and pyFoamChangeBoundaryType.py automatically modify
processorXIn previous versions these boundary files were not modified. Now they
are. Scripts that rely on unchanged boundary-files in the processorX-directories might fail. Old behavior
can be set with the --no-processor-option
TemplateFile passed to
fileLines with $$ are passed to the file
and make it syntactically incorrect. Fixed
This caused problems in Python 3 were binary strings were not correctly written (actually: attempts to write them to a pickle-file made the application fail)
In a number of classes/applications semantic differences between Python 2 and 3 makes these fail on Python 3. Changes are
map with list comprehension
where possibleCTestRun to work
with Python3print with print_ParsedParameterFile fails if "complete"
dictionary is #include edIf an included dictionary has a header parsing failed. This is fixed by retrying the parsing with the header
ParsedParameterFile fails if there is more info
after #includeIf there is for instance a ; after an
#include statement the regular
OpenFOAM-parser ignores it. The PyFoam-parser failed. This has been
fixed and the parser behaves like regular OpenFOAM
pyFoamDisplayBlockMesh.py not working with VTK
6Due to changes in the API the program did not work. This is now fixed and the program works with VTK 6 as well as VTK 5
pyFoamCreateModuleFile.py failed with
environment variables containing =In that case an overeager split created
lists.
Fix provided by Martin Beaudoin
GeneralVCSInterfaceChange in the syntax of imports in Python 3 broke this one. Fixed. But doesn't matter as Mercurial doesn't support Python3
ParsedBlockMeshDict brokenWrong usage of indexes. Fixed
TemplateFile not correctly working in Python
3Reason was a different calling convention to the exec-function of Python. Fixed
pyFoamPrepareCase in
--quiet was setThis was due to actions being on the same level as the debug-output. Fixed
For certain solvers the plot utilities issued a warning during a phase when no information about the current time is available. This is now fixed
When iterating over dictionaries with redirects the values in the redirected dictionaries were not used. This is now fixed
Due to a difference in the behavior of the eval-function in Python 3 certain expressions
(especially with list comprehension) failed. Fixed
RunDatabaseThese characters were stripped out by SQLite. They are thus normalized to special character combinations (which will be denormalized after reading)
The algorithm to find (Open)FOAM-installations in alternate locations was broken. Now working again
Due to a a change in the socket API if the socket for the network thread (usually port 18000) was already notified the program failed. Fixed. Tested on 2.7 as well
pyFoamPrepareCase recognizes multi-region
casesIf there are multiple regions and no prepareMesh.sh then it will try to execute blockMesh for the regions
pyFoamPrepareCase adds specialized
templatesWith the option --extension-addition
additional templates that override the regular templates can be
specified.
Application may be for instance special templates for potentialFoam
pyFoamPrepareCase keeps data generated by
meshing scriptIf the meshing stage generates a 0-directory then data in that directory will be
kept. This can be switched off if this is not the desired behaviour
pyFoamPrepareCase adds possibility for a file
with default valuesIf a file default.parameters is found
in this directory then this file is automatically read. This file is
read like a regular parameter-file but it can also have added
information about the values:
if an entry is a dictionary and has the entries description and values then it is assumed that this is a
group of values. These groups are only used for grouping in
reports. The values are set in the global
scope.
Groups can be nested
if an entry is a dictionary and has the entries description and default then it is assumed that the dictionary
holds meta-information about the value. The entry default is the value that is actually
used.
an additional enty options in that
dictionary specifies a list of valid values for that parameter. The
utility fails if the set value is not in that list. This is useful for
setting boundary condition types or similar entries where only a limited
number of words is valid
It is recommended that all used values are specified in this file as this will be used for reporting
pyFoamPrepareCase writes report about the
variablesThe utility now writes a Restructured Text file with a
report about the variables. Information from default.parameters like groupings and description are used in this report. Also the
default value and the actual value are reported.
The file can be converted with a utility like rst2pdf
For all utilities that use Gnuplot as the backend for plotting there
is now a configuration option gnuplotCommands in the Plotting-section with the usual set commands of Gnuplot. This is preset to
display a grid and the y-axis.
The settings can be reset with the gnuplotCommands-list in the customRegexp-entries
pyFoamPVSnapshot.py now supports Paraview 4.2
and laterThe API for screenshots changed with Paraview 4.2 and later. This API
now supports layouts if multiple views were specified. The default
behaviour is now to make only one screenshot per layout. The old
behaviour (one screenshot per view) can now be switched on with the
-no-layouts-option.
This allows screenshots exactly the way they look on screen
pyFoamPVSnapshot.py allows switching between
decomposed and reconstructed dataThe utility now rewrites the state file so that either decomposed or reconstructed data is read. The default is that the data set for which more timesteps exist is selected
pyFoamPVSnapshot.py allows changing the field
for sourcesIt is now possible to specify a dictionary with source names and the fields that should be set for them. This allows quickly coloring the same layout with different fields.
This works for
pyFoamPVSnapshot.py allows rescaling the
color-legendThere are now two ways to rescale the color-transfer functions
highest and lowest percent of the cells to be ignored The first method will override the second
pyFoamPVsnapshot reads parameters written by
pyFoamPrepareCase.pyAn option allows specifying that these parameters should be read. They are then available for substitution in the Text source
pyFoamListCases.py allows filteringThe utility now has options to select the cases that should be considered by name of the case. Either substrings or globs can be used. Ignore patterns can be specified as well
pyFoamRunParametervariation.py now allows
dictionariesThe format of the parameter-file has been extended so that instead of values variations can be dictionaries. The contents of the dictionary are then used instead of a single value. An example would look like this
defaults {
a 1;
b 2;
c 3;
}
values {
solver (
interFoam
);
inVel ( 1 2 );
outP ( 0 1 );
sets (
{ name foo; a 3; }
{ name bar; b 4; c 5; }
);
}
where sets has two variations. Values unset in sets will be used from defaults
pyFoamConvertToCSV.py now has all functionality
of pyFoamJoinCSV.pyNow all functionality of the Join-utility is present in the Convert utility (including interpolating
times)
pyFoamJoinCSV.py will be removed in
future versions of PyFoam
dynamic in customRegexp now allows composition from
multiple match-groupsIf idNr is a list then now the actual
id is composed from all the match groups specified in that list. If
idNr is an integer then it behaves like it
used to.
Application for the new behavior would be for instance to have the flow of different species on different patches in one plot
dynamicslave in customRegexpThis combines the properties of the types dynamic and slave:
dynamically generated data sets that are added to another plot
--profile-line-profilerUses the library line_profiler for
profiling. Only of interest for developers. Experimental
The section Template in the
configuration now allows modifying the behavior of the templating engine
(how templates are processed and syntax details)
LocalConfigPyFoam now can be read
before argument parsingThis allows overruling options before they are set be command line options. This has to be enabled by the application (it doesn't make sense for all).
An example is the pyFoamPrepareCase.py-utility where the local
configuration file can overrule the default behavior of the template
engine
pyFoamConvertToCSV.py automatically selects the
output format with --automatic-formatNow if that option is selected and the extension of the output is
xls the option -write-excel-file doesn't have to be set
anymore
pyFoamConvertToCSV.py allows adding original
data as separate sheetsThe input data files now can be added to an excel-file as separate
sheets with the --add-sheets-option
pyFoamConvertToCSV.py has improved naming of
columnsNow if there is more than one source file then the columns from the first source also get a unique prefix.
It is also possible to clean the column names before writing them to
file. The time name can be changed with -write-time-name. Substrings can be replaced
with --column-name-replace and simple
functions can be applied with --column-name-transformation
pyFoamConvertToCSV.py now supports
sets-filesThe utility now can read these files and determine the field names
from the filename. --automatic assumes
that files with the extension .xy are of
this format
pyFoamPrepareCase.py can calculate derived
values with a scriptIf a script derivedParameters.py is
present then it is executed and values set in that script can be used in
the templates as well
pyFoamPrepareCase.py adds a variable numberOfProcessorsIf unspecified by the user this variable is automatically 1. It can be used by the templates to distinguish between different cases
The PrepareCaseJob-class in ClusterJob automatically sets the values
according to the number of processors in the cluster job
pyFoamChangeBoundaryName.py and pyFoamChangeBoundaryType.py now support
decomposed casesBoth utilities now also modify the boundary files in the processorX-direcories. This behaviour can be
switched off from the command line
pyFoamPrepareCase.py has possibility for
templates after the final stageTemplates with the extension .finalTemplate are executed after the caseSetup.sh-script.
pyFoamRunParameterVariation allows adding
postfix to cloned casesThe option --cloned-case-postfix adds a
postfix to the cloned directory names. This can be used if the results
from multiple variations with the same parameter file should be kept
(for instance when comparing OpenFOAM-versions)
pyFoamConvertToCSV now allows setting of default
input file formatThe option --default-read-format now
allows setting a different format than csv
as the default format for input files
pyFoamListCases.py adds the hostname to the
printed informationThe utility tries to determine from the pickled data the host the simulation was run on and prints it (can be switched off with an option)
pyFoamPrepareCase.py allows cloningThe utility now has an option --clone-case to clone a new case before starting
instead of working in the specified directory (in this case the case
will be cloned to this directory). The name of the created directory
can be constructed from the specified parameters with
the --automatic-casename-option
SolutionDirectory detects multiple regionsValid regions are sub-directories of constant that have a polyMesh-directory
BoolProxy now compares like builtin boolComparison used to fail for types where it was not explicitly
implemented like None
PyFoamApplication-class now supports pvpython for debuggingNow --interactive-after-exception also
works in the utilities that use pvpython
TemplateFile now allows more flexible
assignmentsIn the mode where executions are allowed now more flexible assignments to variables are possible. To be specific:
$$ a["t"] = 2
$$ a,b = 2,3
ThirdParty-library six upgraded to 1.9.0This library has been upgraded to the latest released version
RestructuredTextHelperThere are additional methods emphasis,
strong and literal that wrap their arguments in the
appropriate markup
The methods bulletList, enumerateList and definitionList take lists or dictionaries and
mark them as lists
SpreadsheetData can now read files produced by
the sets-functionObjectIf the option isSampleFile is set then
it is assumed that the field names are in the filename and there is no
header with field names in the file
By Oliver Borm. The building of Debian packages for Python libraries has changes. Necessary adaptions were done by Oliver Borm
pytest for unittestingThe nose-library had problems and all
the unit-tests run out-of-the-box with pytest
The only requirement is a python-installation. The main testing and development has been done with 2.7 and 2.6. Python 2.5 should work but not for all utilities. Currently 2.6, 2.7 and 3.4 are used during development.
Certain functionalities need special Python-libraries. The main one
is numpy for plotting. Other libraries
that might be of interest are matplotlib,
pandas and IPython. For pyFoamDisplayBlockmesh.py the libraries PyQt4 and vtk are
needed. Other libraries are tested for and reported by pyFoamVersion.py but not required (only install
if you're sure that you need it)
pyFoamJoinCSV.py and pyFoamConvertToCSV.py unifiedThese two utilities are almost indistinguishable and will be unified into one
customRegexpIf in customRegexp an expr is found with \n then this expression is matched over multiple
consecutive lines. Types like dynamic work
as usual.
This makes it possible to match for instance the output of the forces-function objects
pyFoamPrepare.pyThe utility which was introduced in the last version is becomong more
usable and will be central to all things that set up the case (for
instance a special ClusterJob)
These utilities are now more flexible and allow writing and reading of Excel-files too
PYFOAM_SITE_DIR and PYFOAM_DIRBoth variables are not necessary but PYFOAM_SITE_DIR allows consistent insertion of
site-specific libraries and utilities.
PYFOAM_DIR is set by some
Foam-distributions and tested for by pyFoamVersion.py. It is supposed to point to the
currents PyFoam-installation.
PYFOAM_SITE_DIR points to a directory
with site-specific scripts and configurations. The sub-directories
supported (and thested py pyFoamVersion.py) are
PATH outside of PyFoam (for instance in the
.bashrc)
PyFoam.Site: For instance if a
file Foo.py is present in lib it can be imported as PyFoam.Site.Foo. This directory does not have to
be (in fact: it shouldn't) added to PYTHONPATH
Purpose of PYFOAM_SITE_DIR is to allow
administrators to provide site-wide scripts and settings for all users
on a site
--silent removed from pyFoamPrepareCase.pyOption has been renamed to --no-complain
RunDatabase with column-names that
contain upper-case letters changeSQLite does not support case-sensitive column-names (s_max and S_max
are the same). To change this the upper case letters in the column names
are replaced by an underscore and the letter (S_max becomes _s__max)
This means that old databases might not be read correctly
pyFoamConvertToCSV.pyThe algorithm to make variable names unique has changed (basically it uses the part of the filenames that differ) and scripts relying on these names might fail
PyFoam.IPython-module renamed to PyFoam.IPythonHelpersThe name of the module crashed in certain instances (especially
unit-testing) with the regular IPython-library. To avoid these crashes it has
been renamed to IPythonHelpers. This
raises two potential problems:
import the module have to
be adapted to the new namepyFoamIPythonNotebook.py have two imports
pointing to this module. These notebooks have to be adapted to be usable
againpyFoamPrepareCase.py did not keep
permissionsThis was a problem for script-templates which were not executable any more. Fixed
pyFoamComparator.py failed due to circular
dependencyThis has been fixed by adding an import in BasicRunner.py
pyFoamDumpRunDatabaseToCSV.py fails if
Pandas-data is requestedThis is now fixed
sort for list broke code on Python 3Some calls for sort still used the
cmp-parameter which does not exist for
Python3 anymore. These calls have been replaced with key and reverse
Because the output of subprocess is now
binary instead of a regular string. Fixed
addData in PyFoamDataFrame extrapolates for invalid
valuesThis was due to incorrect use of the interpolate-method
--keep-last did not work for pyFoamClearCase.py and parallel casesThis was because there was a problem in the library code and the utility did not consider the parallel time-steps. Fixed
pyFoamDumpRunDatabaseToCSV.py does not add basic
run informationBasic run information was not added to the file. Now it is with the
prefix runInfo//
FileBasisBackup did not workThe logic for checking whether a file was "backupable" was wrong.
This affected the proper restore of files with utilities for instance
for --write-all
DataStructuresAccording to the bug http://sourceforge.net/p/openfoam-extend/ticketspyfoam/219/
it was not possible to import DataStructures because of a circular dependency
with FoamFileGenerator. Fixed by moving an
import to the back of the file
pyFoamRunParameterVariation.pyThis utility takes a template case and a file specifying the
parameter variation and creates cases with the pyFoamPrepareCase.py-engine, runs a solver on
these cases and collects the data into a database. The database can then
be extracted with pyFoamDumpRunDatabaseToCSV.py
pyFoamBinarySize.pyCalculates the size of the binaries in an OpenFOAM-installation separated by compile-option
pyFoamBlockMeshRewrite.pyAssists the user in rewriting the blockMeshDict by doing simple, but error-prone
transformations. Assumes "sensible" formatting: one block/vertex etc per
line.
Sub-commands are:
number
blockMeshDict, copy over the
vertices-section of that mesh and rewrite
blocks and patches so that they conform to these vertices. The original vertices have to be a sub-set of the vertices in the other mesh
pyFoamChangeBoundaryType.py allows setting
additional valuesThe option --additional-values allows
specifying a dictionary with additional values for the boundary (stuff
that is needed by mappedWall etc)
pyFoamPrepareCase.py now has OF-version and fork
as defined variablesThis allows to write case-templates that can distinguish between different OF-versions
pyFoamPrepareCase.py now allows "overloading"
another directoryBefore doing anything else the contents of different directories are copied into the current case. This allows for instance to use tutorial cases as the basis for a case
pyFoamIPythonNotebook.py adds improvements to
the notebookAdditional code added to the generated notebook:
distributions (generated by some swak-function objects) addedpyFoamListCases.py more tolerant to faulty controlDictIf the controlDict is acceptable to
OpenFOAM but syntactically incorrect for PyFoam (for instance because of
a missing semicolon) the utility does not fail anymore (but no data is
collected for that case).
pyFoamDumpConfiguration.py prints sections and
keys alphabeticallyThis should make it easier to find items
pyFoamJoinCSV.py and pyFoamConvertToCSV.py read and write
Excel-filesBoth utilities now allow writing Excel-files
In addition to regular text files the first sheet from xls-files can be read
pyFoamJoinCSV.py and
pyFoamConvertToCSV.pyNow it is possible to filter for regular expressions
The functionality of the two utilities now is very similar and it is possible that one of them will be discontinued
pyFoamJoinCSV.py and pyFoamConvertToCSV.py can be recalculatedThe two utilities now can add columns or recalculate columns based on the existing column values
Numeric removed from pyFoamVersion.pyTesting for the library Numeric library
removed as it is no longer supported as a fallback for numpy. Test also removed from setup.py
ClusterJob that support PrepareCaseThe class PrepareCaseJob supports cases
that are set up with pyFoamPrepareCase.py.
Additional parameters to the constructor are
ClusterJob that support RunParameterVariationThe class VariationCaseJob supports
cases that are set up with pyFoamRunParameterVariation.py. Additional
parameters to the constructor are
execute in PyFoam/Utilities fails if script is not
executableThe function checks if the file exists and is not executable. The program fails in that case
foamVersion uses a separate wrapper class for
tupleThis ensures that it is printed in a form that is valid in OF-dictionaries
UtilitiesThis has until now only been used in ListCases but moved to a separate
method/function diskUsage in the Utilities-module
--helpAdded the possibility to have an epilog and usage examples with the
epilog and examples-keyword arguments for applications.
These and descriptions now have the possibility for line-breaks: if two line-breaks are encountered in the text a new paragraph is created
which-routine in Utitlities uses native Python-routineFor Python-version where shutil has a
which-function this is used instead of
calling an external program
FileBasis now allows file handles instead of the
filenameThis currently only works for reading, Backups, zipping etc won't work but it makes algorithms more flexible
BlockMesh doesn't force writing to file
anymoreInstead content is stored in memory. Old behaviour is the default to preserve compatibility with old scripts
BlockMesh-classLineReader allows keeping spaces on leftPrevious behaviour was stripping all spaces from the lines. Now the left hand spaces can be ket. Old behaviour is still default for compatibility
TemplateFile now allows writing of
assignment-results in fileThis allows faster debugging of template-files. This can be enabled with a switch in the utilities using templates
SolverJob now allows passing of parameters to
the solverAnd additional parameter solverArgs
will now be passed to the solver (if the solver accepts arguments)
SpreadsheetData now allows reading from an Excel
fileDuring construction if an Excel-file is specified and the xlrd-library and pandas are installed then the first sheet in the
file is read
SpreadsheetData allows recalculating
columnsColumns can be recalculated using expressions. This includes other
data items. Currently present column names are available as variables.
There is also a variable data that can be
subscripted for items that are not valid variable names. A variable
this points to the item to be
recalculated
This manifests with pyFoamRunParameterVariation.py. The custom
timelines are still kept in memory. Not a problem. Just annoying
The only requirement is a python-installation. The main testing and development has been done with 2.7 and 2.6. Python 2.5 should work but not for all utilities. Unit tests run on Python 3.4 but it is currently not used in a production environment (reports of success or failure most welcome)
Certain functionalities need special Python-libraries. The main one
is numpy for plotting. Other libraries
that might be of interest are matplotlib,
pandas and IPython. For pyFoamDisplayBlockmesh.py the libraries PyQt4 and vtk are
needed. Other libraries are tested for and reported by pyFoamVersion.py but not required (only install
if you're sure that you need it)
Now pyFoam supports different versions
of OpenFOAM for switching. Out of the box openfoam and extend are supported. If only the version number
is specified (for instance by --foamVersion=1.7.x) and such a version exists
only for one fork it is correctly expanded with the correct fork( in the
example with openfoam-1.7.x). If more than
one fork has the same version then the fork name has to be specified as
well
Note: additional forks can be easily specified with the
configurations. In section OpenFOAM the
parameter forks has to be extended. For
each new fork a dirpatterns and installation-parameter has to be specified
pyFoamSTLUtility.pyThe selection of what is to be done is now selected by subcommands instead of options. This will break scripts using this
0.org is present pyFoamCloneCase.py and pyFoamPackCase.py ignore 0The reason is that the utilities assume that this directory is
produced from 0.org
The reason was that it used the same throttling that made sense for the PlotRunner. Fixed
pyFoamPVSnapshot.py fails for newer
paraview-versionsReason is that the class vtkPythonStdStreamCaptureHelper does not support
isatty
Reported in https://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=208 and fixed
pyFoamTimelinePlot.py failed Numpy/Pandas output
of vector fieldsVector fields only were added to the data fields if they were the first in the list. Fixed
alternateAxis ignored for slaveThis is now fixed. The alternate values have to be specified in the master (specifying in the slave gives an error)
pyFoamCaseReport.py more stable for binary boundary-filesUsually these files are ascii (even if
the header says binary). In some cases the
parsing failed for these. Fixed by enforcing reading as ascii. Can be switched off
SpreadsheetData returns data which breaks
certain Pandas-operationsThe reason was that if there were duplicate times in the table the index was non-unique which certain Pandas-operations don't appreciate. Solved by dropping duplicate times. Can be switched off
pyFoamCloneCase.py added duplicates to the
archiveIf things are specified twice they were added twice. Now it is checked whether the item already exists in the tar-file before adding them
nonuniform of length 3 not correctly
printedThe reason was that this was interpreted as a vector and the numeric prefix was removed. Reported at http://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=218
Fixed by introducing an extra parameter to FoamFileGenerator
pyFoamPrepareCase.py for case preparationThis utility aims to reduce the need for boilerplate scripts to set up cases. The steps it executes are
0.org is present remove
the 0 folder too.template do template expansion using the
pyratemp-engineblockMeshDict is present by running blockMesh.
If none of these is present assume that there is a valid mesh
present)foo.org that is found to to
foo (recursively if directory).postTemplatepyFoamIPythonNotebook.py for generating and
manipulating IPython-notebooksThis utility creates and manipulates IPython-notebooks for analyzing OpenFOAM cases. It has a number of subcommands:
this is the main command. All other commands assume that the notebooks they work with were created with this.
The utility looks at the case specified and creates a notebook that has the capabilities to quickly build a report about the case:
basically is the capability of the pyFoamCaseReport.py-utility
pyFoamTimelinePlot.py or pyFoamSamplePlot.py and generates selectors that
allow the user to select which data to import. The selectors import the
data as Pandas-DataFrames and create the
commands necessary to do this. It is recommended to erase the selector
afterwards
generated by PyFoam
file. This feature is experimental and has performance issues for medium to large datasets The created notebook can be executed but needs to be edited to be useful
create) or output from the notebook.
Recommended way of working with this utility is
PyFoam.IPythonThe purpose of this submodule is to support pyFoamIPythonNotebook.py. It has the classes
PyFoam.WrappersWraps popular Python-libraries to add functions that make it easier to work with OpenFOAM-data.
Currently only one Wrapper is implemented:
Pandas-wrappers
This provides PyFoamDataFrame as a
wrapper for DataFrame. The functionality
added is
Series, DataFrames. It is assumed that the index is the
same property (time or for samples the distance) but with a different
resolution. The indexes are joined and missing data is interpolated
Series. validLength is the extent on which data is
defined (weightedAverage is basically
integrate divided by validLength). For integrate the trapezoid-rule is used
describe-command
pyFoamSampleplot has option to use index instead
of time in filenamesThe option -index-instead-of-filename
switches this on. This makes it easier to generate movies from the
files
pyFoamListCases.py Allows addition of custom
dataThe option --custom-data now allows the
specification of custom data items. These are read from the pickledData-files and displayed in the table
like regular data items
Now all utilities allow switching the compiler version (for instance
from Gcc47 to Gcc48). The relevant options are --force-system-compiler, --force-openfoam-compiler and --force-compiler
pyFoamVersion.py reports the installed versions
betterNow the location of the installations is reported as well
pyFoamPVSnapshot.pyThis is a workaround where the writer produces a segmentation fault
pyFoamPVSnapshot.pyIn addition to writing out bitmaps allows writing out 3D-data (for importing into other applications). Sources can be selected by name
pyFoamSTLUtilityThe utility can now also:
pyFoamDecomposer.py switches off function
objectsThis now automatically happens for OF-versions that support it (2.0 and greater). They can be switched on again
pyFoamCloneCase.py clones more stuffFiles that are assumed to be used by pyFoamPrepareCase.py are automatically added to
the clone. This includes all files (and directories) with the extensions
.sh, .template and .org. Also IPython notebooks (extension .ipynb are added)
BasicRunner now can print the command line that
is actually usedThis should help with diagnosing problems with MPI etc.
Can be switched on in some utilities with --echo-command-prefix
ClusterJob now can live without a
machinefileUsing the machine-file now can be switched off for job-schedulers with a tight integration
If a item in the case itself is a symlink then it used to be a copy
of the file the symlink is pointing to. Now it is created as a symlink
to the target the original symlink. If the --follow-symlink-option is used the old
behaviour is used (copying). In this case the option noForceSymlink in the Cloning-section of the configuration can be used
to change this behaviour for selected files
AnalyzedCommon clears the analyzed-directoryThe directory is cleared if it exits from a previous run.
TimelineDirectory is more tolerantUsed to fail if incompatible data types were used. Now ignores them
The subclass SubclassFoamOptionParser
now allows the parsing of subclasses. The base class for utilities PyFoamApplication now supports this as an
option. As an example this is implemented in pyFoamSTLUtilities.py
STLUtility accepts file-handlesThe class checks whether arguments are filehandles and in this case doesn't try to open a file for reading or writing but uses the handle
addClone in SolutionDirectory accepts glob patternsIf no file matching the name is found it is assumed that this is a
glob-pattern and all matching files are added. This affects all
utilities that use that method (especially pyFoamCloneCase.py)
execute in Utilities allows specification of working
directory and echoing of outputThis method now allows the specification of a working directory. Before executing the command the method changes to the working directory. Afterwards it changes back to the regular working directory.
There is also an option echo that
immediately prints the output to the screen
rmtree and copytree more tolerantrmtree now also works if the "tree" is
a file.
copytree now has a parameter force that allows removing the destination
directory if it exists
Strings that are usually interpreted as boolean in OF-dictionaries
(for instance on, yes, …) are now stored as a special type that
allows treating them like 'real' booleans.
For instance an expression test no; in
a dictionary now allows things like if d['test']: in the script
Until now the options to be used had to be specified as a list of
string modeled on the way the command line looked like. This is still
possible. In addition it is now possible to specify these things as
keyword parameters on the command line. Rudimentary type checking is
done. The names of the parameters are generated from the command line
options: the - are removed and the words
are converted to CamelCase. So for instance --list-custom-Regexp =
becomes =listCustomRegexp. Also for switches like these a
boolean value has to be passed. So the correct usage in a call would be
listCustomRegexp=True.
SolutionDirector now can classify directories in
the postProcessing-directoryA number of properties has been added that list data generated by function objects:
propes) that can be
visualized by pyFoamTimelinePlot.py
set (assuming it is in raw-format) that can be processed by pyFoamSamplePlot.py
surface (assumes VTK-format) that can be used by pyFoamSurfacePlot.py
sample with distribution
data
These properties only list the subdirectories of the case with that data
Additional properties are
These lists are sorted in descending temporal order (newest first)
pyFoamSamplePlot.py now more flexible for
distributionsTries to determine the names of the values from the first line in the files
DictProxy now has a dict-like update-methodThis also allows enforcing string values
FoamFileGenerator automatically quotes
stringsIf strings are unquoted but contain characters that make it illegal as a word then the string is quoted before output
FileBasis now can be used with the
with-statementThis mainly concerns ParsedParameterFile
pandas-libraryStarting with this version the pandas-library is used for data-analysis. When
possible and appropriate classes return pandas-objects. Currently these are:
CSVCollection. With a call-operator
this class returns the collected data as a DataFrame of the collected dataSpreadsheetData now has methods to
return Series and DataFrame objectsIt is not necessary to install pandas
if these classes are not used (and even then most of their functionality
works)
The class RunDatabase (and therefor
also the utility pyFoamDumpRunDatabaseToCSV.py) now write as a
separator for data from sub-tables a //
instead of the space. This especially means that scripts that rely on a
data-item foo in analyzed might break because this is now called
analyzed//foo instead of analyzed foo. On the other hand this makes the
names more consistent and easier to parse as // is the saperator for other levels of
dictionaries
Instead of col0 this is now coord. This could cause problems with scripts
that use that column name in the resulting SpreadsheetData-object
pyFoamPackCase.py does not handle symbolic links
correctlySymbolic links were copied as is and did not work correctly afterwards. This is fixed. If the symbolic link is an absolute path or points outside the case directory it is replaced with the file it points to. Otherwise it is preserved as a symbolic link
pyFoamPotentialRunner.py not working with
OpenFOAM 2.0 or newerThese versions require an entry potentialFlow in the fvSolution-file instead of the old SIMPLE
pyFoamListCase.py fails with controlDict that use preprocessingFixed by first trying to read that with preprocessing. Without if that fails
Now using a set instead of a list makes sure that no file is cloned twice
pyFoamPotentialRunner.py now allows removing of
functions and libsThe utility now allows removing these entries in case that they don't
work with potentialFoam
Some of the options of pyFoamClearCase.py for clearing cases (for
instance specifying additional files) have been ported to the Runner-utilities. Also is the postProcessing-directory removed by default
SolutionDirectory and TimeDirectory are more tolerantIf there are field files and their zipped counterpart than instead of an error a warning can be given
ClusterJob now handles template filesA new method templateFile gets the name
of a file which is constructed from a template of the same name plus the
extension .template
ClusterJobThe method additionalParameters can
return a dictionary with additional parameters
In the written data in the sub-dictionary analyzed there is now a subdictionary Custom with the values of the custom expressions
with the prefix CustomXX_ removed. This
means that values that were available as
data['Custom02_velCheck']['min']
are now available as
data['Custom']['velCheck']['min']
The advantage is that the number part which was dependent on the order the expressions were specified is now no longer necessary (this should make scripts more stable)
The old notation is still available but deprecated
SolverJob now allows compression of outputThe parameter solverLogCompress
compresses the log-file while writing it to disc.
Attention: This may lead to corrupted log-files if the
run crashes
PyFoamApplication-class now allows quick access
to dataThe dictionary returned by getData()
now allows access to all the elements as attributes.
The hook-module MailToAddress sends a
mail at the end of a run. Prerequisite is an SMTP-Server that doesn't
need authentication
pyFoamCompressCases.pyThis utility goes through cases and compresses single files. The cases can be searched recursively to.
Purpose of this utility is to shrink cases where writeCompression was not turned on during the
run
A new module PyFoam.Paraview.Data reads
additional data usually written by OpenFOAM. These are converted to
vtkArray using the following functions and
can be used in Programmable filters:
RedoPlot
pyFoamRedoPlot.py can plot in XKCD-modeWhen used with the option --implementation=xkcd and version of matplotlib that supports it is installed then
plots are done in the style of the webcomics http://xkcd.com
pyFoamListCases.py now displays disk usage in
human readable formIf the disk usage of the cases is calculated then it is displayed in human readable form (as KB, MB, GB or TB) for sizes larger than one Kilobyte
pyFoamClearCase.py more flexible in selection of
data to be removedOptions to be more flexible in removing data are added:
--keep-interval=0.1 will keep times like 1, 1.1 etc
but remove 1.05
processor-directories. Also are things like
keep-last now honored for processor
directories
pyFoamFromTemplate.py automatically chooses
template and default valuesIf an output file foo is specified and
no template then the utility looks for a file foo.template as a template.
If a file foo.defaults is present then
this file is read and used as default parameter values. Other
specifications override these defaults
pyFoamDumpRunDatabaseToCSV.py can disable
standard-fieldsAdditional option --disable-run-data
pyFoamDumpRunDatabaseToCSV.py prints pandas-objectWith the -pandas-print-option a DataFrame is generated and printed
ipdbIf the ipdb-package (basically pdb with IPython-additions) is installed then it is used.
This gives additions like tab-completion
The option --interactive-after-execution drops the user to
an interactive shell where the namespace can be inspected. If present
IPython will be used, otherwise the
regular shell is used
pandas-data and/or numpyThis is mainly to be used on the interactive shell to do further analysis or write this data out. The utilities are:
dump with the whole data as a
DataFrame
series with all the data as
Series and dataFrame with the same data as a DataFrame
pyFoamTimelinePlot.py
The utilities pyDumpRunDatabaseToCSV.py, pyFoamTimelinePlot.py, pyFoamSamplePlot.py and pyFoamRedoPlot.py now have options to write
Excel-files
GnuPlot in customRegexpIf an item in customRegexp has an item
gnuplotCommands then it is assumed that
this is a list of strings which are executed before the first plotting.
For instance
gnuplotCommands (
"set format y '%.2e'"
);
changes the number format on the y-axis
pyFoamSamplePlot.pyInstead of determining the names of the fields and lines form the filenames it is now also possible to specify them through options.
The option --is-distribution is a
shorthand that sets these options for distribution files
pyFoamSamplePlot.py now allows specification of
x-rangeThe range of the x-axis of the plots can either be set by
automatically scaling to the domains of all the data sets with --scale-domain or by specifying them with --domain-minimum or --domain-maximum.
These domains are set for all plots
controlDict after writeWhen activating an on-demand write the constrolDict was not restored because the
output-line about the file being read was not recognized (due to a
change in the output in recent OF-versions). Now a number of different
formats is recognized
slave not working if no master definedThat plot-type needs a master. Fixed to
fail if none is defined
-list-only did not correctly parse lists with a
numeric prefixThis did affect all utilities that use that option and also calls
with listOnly to the library class
pyFoamBuildHelper.py now allow more than one
actionIf multiple actions like --update and
--build are specified they are executed in
a sensible order (update before build etc)
If the environment variable that determines the OpenFOAM-version is unset a warning is issued by the utilities
pyFoamUpgradeDictionariesTo20.py allows single
filesIf single file is specified then the action to transform it has can be specified
pyFoamUpgradeDictionariesTo20.py transforms
reaction-schemesNow knows how to transform "old" reaction files (where the reactions-entry was a list) to the new format
(where it is a dictionary). Only a limited number of reaction types is
supported.
pyFoamUpgradeDictionariesTo20.py transforms
thermophysical dataNow the old form of thermophysical data (lists) is transformed into the new dictionary-form
pyFoamCloneCase now allows creating directory
that symlinks to the originalNow with the option --symlink-mode
instead of copying the directories from the original new directories art
created and populated with symlinks to the files in the original. The
depth until which no symlinks to directories are created can be
specified. This allows the clone to share the configuration files with
the original
pyFoamClearCase.py now removes postProcessing and allows removal of additional
filesThe directory postProcessing is now
automatically removed (can be switched off with --keep-postprocessing). Also with the --additional-option patterns with additional
files to remove can be specified.
pyFoamVersion.pypython-executableThe files Allrun, Allclean and 0.org
are automatically added during cloning as these are often used by the
standard-utilities
pyFoamDisplayBlockMesh.py uses the same options
for template format as pyFoamFromTemplate.pyThis makes sure that templates are handled consistently and also
allows different delimiters in the blockMeshDict.template
ParsedParameterFileUtilities-class now function to find files
matching a patternAdded a function find that approxiamtes
the find-command
Some more patterns have been added that will be ignored in a VSC-controlled case. All of them concerning files that PyFoam creates during operation
pyFoamSymlinkToFile.pyThis utility replaces a symlink with a copy of the file/directories
it points to. To be used after a pyFoamCloneCase.py in --symlink-mode
Sources are adapted so that PyFoam
works with Python3 too. This breaks support for Python 2.4 and earlier
(possibly also Python 2.5)
Some of the Libraries in PyFoam.ThirdParty had to be adapted to work with
Python3:
six-library (see below) to work with
Python2 and Python3 (inspired by [https://github.com/oblalex/gnuplot.py-py3k/commits/master]
which is a pure port to Python3 without backwards compatibility)
WindowsPort to allow running PyFoam on Windows was done by Bruno Santos of blueCAPE (bruno.santos@bluecape.com.pt)
Patch was originally posted at http://sourceforge.net/apps/mantisbt/openfoam-extend/view.php?id=166
Note: many of PyFoam's features are not yet fully functional on Windows.
pypySources are executed in pypy but it
seems there are problems with numpy and
also with code like for l in open(f).readlines()
ply
to 3.4This brings virtually no changes. README with copyright information has been
added
CTestRun after
initializationThis should help to avoid side-effects
MetaServer rightDue to a previous workaround timeouts when collecting information about new machines was not treated correctly
execute-method to ClusterJobThis allows the execution of a shell-script in the directory of the case
These modules are found in PyFoam.Infrastructure.RunHooks. Two concrete
implementations:
PrintMessageHookSendToWebservicepushover.net added)
Hooks are automatically instantiated from the configuration data (examples are hardcoded))
The Runner-classes now have a parameter parameters. This data (usually it would be a
dictionary) is added verbatim to the run info.
Most runner applications now have the possibility to add this info.
Purpose of this facility is to identify different runs in the database better.
ClusterJob extendedParameter values are now handed to the actual job. Also a dictionary with parameters can be handed to the constructor and will be used in the relevant callbacks
PickledPlotsDuring the run whenever the PickledPlots-file is written a file pickledUnfinishedData gets written. This has the
current solver data and is similar to pickledData.
Also a file pickledStartData gets
written that has the data that is available at the start of the run.
BasicRunner collects error and warning
textsThe runner collects
And stores them in the application data
TemplateFile now uses pyratempThe class TempalteFile now uses an
enhanced templating engine. The old implementation is in the class TemplateFileOldFormat
If used as classes (not as utilities) these classes print the class name instead of the calling utilities name
Error and warning messages don't decorate the output if it goes to files or other non-terminal streams
error-method of application classes now raises
an exceptionAn exception is now raised by self.error(). This makes it easier to handle
such errors if the application class is used. The exception is passed up
until there is a "real" application
ParsedParameterFile now knows how to handle
binary filesWhen the format of a file is binary
lists with a length prefix are being read as binary blobs.
For reading the blobs a simple heuristics is used: a multiple of the
length in bytes is read. If the next character is a ) and the characters after that are a certain
combination of characters (newlines and ;)
then it is assumed that the blob has ended. This may fail on certain
occasions:
It would be hard to work around these restrictions without reprogramming the full functionality of OpenFOAM
LabledReSTTable for more flexible table
generationNew class in the RestructuredTextHelper
allows more flexible generation of tables. Items are added with column and row and
if these don't exist in the first row/column the table is extended
appropriately
xlabelThis is implemented for Gnuplot and
Matplotlib. Default for the label on the
x-Axis is now "Time [s]"
pyFoamFromTemplate.py with new templating
engineThe utility can now use the pyratemp-templating engine which allows templates with loops, conditions and other fancy stuff
pyFoamSamplePlot.py allows using the reference
data as basis for comparisonInstead of using the x-values from the original data the y-values of
the reference data can be used for comparing (with the --use-reference-option)
Same for pyFoamTimelimePlot.py
pyFoamSamplePlot.pyIf scales not equal to 1 and offsets
not equal to 0 are specified they are
used in the gnuplot-output
pyFoamPrintData2DStatistics.py prints relative
average errorWith the --relative-average-error-option
pyFoamVersion.pypyFoamRunner.py allows hooksHooks can be added at the start and the end of a run
pyFoamRedoPlots.py supports range for plotsAdded -end and -start-option to select a range that should be
plotted.
Currently not working with the Matplotlib-implementation (only gnuplot)
pyFoamDisplayBlockMesh.py no supports
templatesIf a file with values is specified then the utility assumes you're editing a template file and will evaluate it before displaying it
pyFoamCaseReport.py is tolerant towards binary
filesNew switch that makes the parser treat files that are declared binary in the header as if they were ascii
pyFoamSamplePlot.py and pyFoamTimelinePlot.py raise error if no plots
are generatedThis makes it easier to catch faulty specifications (or empty timeline-files)
pyFoamSurfacePlot.py can wait for a keyAn option --wait has been added that
makes the utility wait before displaying the next picture
pyFoamEchoDictionary.py is more flexible with
binary filesSwitch allows forcing it to read a binary File as an ASCII
Previously the option --interactive-debug only started the debugger if
the error was no syntax error. This is still the
default behavior, but can be overruled
USR1 and will give
a tracebackThe option --catch-USR1-signal now
installs a signal-handler that prints a traceback and finishes the run.
If the interactive debugger is enabled then it goes to the
debugger-shell.
Option --keyboard-interrupt-trace
triggers the same behaviour for keyboard interrupts with <Ctrl>-C
For the purpose of developing a switch --i-am-a-developer has been added.
With the option xlabel in the customRegexp-file the label on the x-axis of the
plot can be changed. Setting ylabel and
y2label (for the secondary axis) was
already possible
pyFoamTimelinePlot.py and
pyFoamSamplePlot.py support time
rangesNow the options --min-time and --max-time are supported by --metrics and --compare
pyFoamDisplayBlockMesh.py allows graphical
selection of blocks and patchesNew addition by Marc Immer allows the graphical selection of blocks
and patches and adds them to the blockMeshDict
pyFoamCloneCase.py and pyFoamPackCase.py accept additional
parametersThe file LocalConfigPyFoam is read by
these utilities and if there is a parameter addItem in the section Cloning defined then these files are
cloned/packed automatically (no user specification required)
pyFoamListCases.py now calculates estimated
end-timesAdditional option to print the estimated end times. These can be
wrong if the case did not start from the startTime in the controlDict.
Also now allows printing the end and the start-time according to the
controlDict
Plots of type phase in customRegexp don't actually plot anything. The
set a phase-name that is used for subsequent values (for instance to
distinguish the different residuals)
pyFoamChangeBoundaryType.py allows selection of
region and timeOptions --region and --time-directory added that allow selecting
different boundary-files
The class RunDatabase stores the data
from runs. Utility pyFoamAddCaseDataToDatabase.py is one way to
populate the database. pyFoamDumpRunDatabaseToCSV.py allows dumping
that data to a file for further processing (in a spreadsheet for
instance)
Database can also be populated using a special post-run hook
Pattern had to start with 1 (now every digit is possible))
No harm done. But fixed
--clear-option for pyFoamDecompose.py not workingReason was that rmtree does not allow
wildcards. Fixed
pyFoamDisplayBlockmesh.py not working with
variable substitutionThe DictRedirect would not convert to
float. Fixed. Although it might happen again for other data types
--function-object-data of pyFoamClearCase.py not working with
directoriesThe option was only implemented for the list-form of the functions entry in controlDict
Now fixed to also work with the dictionary-form
nonuniform of length 0 not correctly
printedSeems like the length was interpreted as the name of the list. Fixed
pyFoamRunner.py
brokenOnly worked if no region was specified (= not at all). Fixed
pyFoamRedoPlot.py did not correctly honor --end and --startPlots were over the whole data range. This is now fix (also the issue
that --end alone did not work)
WriteParameterFile does not preserve the order
of additionsContents was "only" set as a dictionary which does not preserve the
order in which entries are added. Replaced with a DictProxy
TimelinePlot in positions-modeProblem that was introduced by changes in the fields-mode
ClusterJob uses only metis for decompositionFor OpenFOAM-versions 1.6 and higher the automatic decomposition used
is now scotch
pyFoamSamplePlot.py and pyFoamTimelinePlot.py produced no pictures for
regionsAs regions have their own subdirectories the / from the directory name was inserted into the
filename and if the subdirectory did not exist gnuplot did not create the picture
pyFoamTimelinePlot.py not working if
value is a vectorThe base class didn't correctly handle the ( and ). Fixed
The problem was that during splitting the timeline data an exception was raised. This exception was caught by another part of PyFoam. This left a lock on the data structure locked and the next access to the structure was held indefinitely. Fixed
As there is a tuple of residuals the scanner did not analyze the
output of the output of the block-coupled solver from 1.6-ext correctly. This is now treated as a
special case and each residual is plotted separately (distinguished by a
[x] with x
being the number of the component)
#include not correctly working with macros in
the included fileMacros $var were not correctly
expanded. Fixed
When being expanded to string form macros were not correctly expanded
pyFoamPackCase.py in the working directory
produces 'invisible' tarIf the utility was used in the form
pyFoamPackCase.py .
then an 'invisible' tar ..tgz was
produced. Fixed
Reported in [http://www.cfd-online.com/Forums/openfoam-solving/112278-pyfoam-struggles-adopted-solver-post403990.html] the string is assumed to be part of the iteration number. Fixed
At least for PV 3.14 and 3.98 the way the version number is determined has changed and the PV-utilities failed. This has been fixed but is untested with old versions
pyFoamPVSnapshot.pyFor the first rendered view Paraview automatically resets the camera. This has now been switched off (so the snapshot is rendered correctly)
The changes for older versions can be found on the Wiki-page