stormtracks API

Contents

Running

stormtracks.download – Download Necessary Data

stormtracks.download.download_full_c20(year, variables=None)

Downloads each ensemble member’s values for prmsl, u and v

stormtracks.download.download_full_c20_range(start_year, end_year, variables=None)

Downloads each ensemble member’s values for prmsl, u and v in a given range

stormtracks.download.download_grib_c20(year=2005, month=10, ensemble_member=56)

Downloads the raw data for one ensemble member.

Contains all fields for the given ensemble member (over 100 of them).

stormtracks.download.download_ibtracs()

Downloads all IBTrACS data

Downloads compressed tarball from FTP site to settings.DATA_DIR. Decompresses it to settings.DATA_DIR/ibtracs

stormtracks.download.download_mean_c20(year)

Downloads mean values for prmsl, u and v

stormtracks.download.download_mean_c20_range(start_year, end_year)

Downloads mean values for prmsl, u and v in a given range

stormtracks.run_stormtracks – Run the Project

stormtracks.run_stormtracks.main(num_ensemble_members=56)

Entry point into local running of code

Runs through complete analysis, saves results using a results_manager.

Parameters:num_ensemble_members – how many ensemble members to analyse
Returns:results_manager that has complete set of results.

Main Modules

stormtracks.analysis – Analysis of results

class stormtracks.analysis.StormtracksAnalysis(year, is_setup_logging=False)

Provides a variety of ways of analysing tracking performance

Makes extensive use of its results_manager to load/save results. Used by the pyro code to farm out jobs across the cluster. To a large extent replaces the manual analysis steps. :param year: year on which to run analysis :param is_setup_logging: whether to setup logging (useful in ipython)

get_good_matches(ensemble_member, config)

Either loads or generates (and saves) good_matches

get_matching_configs(**kwargs)

Allows for easy filtering of current config options

get_vort_tracks_by_date(ensemble_member, config)

Either loads or generates (and saves) vort_tracks_by_date

good_matches_key(config)

Returns the good_matches key for the config options

list_stats(ensemble_member=0, sort_on='avgdist', active_configs={})

Runs through all statistics for the requested ensemble member and compiles stats

sorts on the requested column and only looks at the active_configs. This makes it easy to e.g. compare all scale 1 or all 850 configuration options.

next_best_track()

Moves each plotter’s best track on by one

plot()

Uses each plotter to plot the current scene

prev_best_track()

Moves each plotter’s best track back by one

print_stats(ensemble_member=0, sort_on='avgdist')

Prints the stats

run_analysis(ensemble_member, force_regen=False)

For each set of config options, run a tracking analysis and store the results

run_ensemble_matches_analysis(num_ensemble_members=56, force_regen=False)

Looks at a particular config and runs a matching algortihm against each track

run_individual_analysis(ensemble_member, config)

Runs a given analysis based on config dict

run_position_analysis(sort_on='avgdist', active_configs={}, force_regen=False, num_ensemble_members=56)

Runs a positional analysis on the given sort_on col

If sort_on is e.g. avg_dist, summed av dist for each of the active configs are calc’d and they are ranked in terms of which is lowest

run_wld_analysis(active_configs={}, num_ensemble_members=56)

Runs a win/lose/draw analysis on all ensemble members

If a track from a particular analysis has a lower average dist it is said to have ‘won’ i.e. track for Wilma in pressure_level:850/scale:2/tracker:nearest neighbour has a lower av dist than pl:995/../.. .

set_year(year)

Sets the year, loading best_tracks and setting up results_manager appropriately

setup_analysis()

Sets up the current configuration options

setup_display(ensemble_member, active_configs={})

Sets up plotters for displaying of results

vort_tracks_by_date_key(config)

Returns the vort_tracks_by_date key for the config options

stormtracks.analysis.analyse_ibtracs_data(plot=True)

Adds up a freq. distribution and a yearly total of hurricane-timesteps

stormtracks.analysis.run_ensemble_analysis(stormtracks_analysis, year, num_ensemble_members)

Performs a full enesmble analysis on the given year

Searches through and tries to match all tracks across ensemble members without using any best tracks info.

stormtracks.analysis.run_tracking_stats_analysis(stormtracks_analysis, year, num_ensemble_members=56)

Runs a complete tracking analysis, comparing the performance of each configuration option

Compares performance in a variety of ways, e.g. within pressure level or just scale 1.

stormtracks.c20data – C20 Reanalysis Data

class stormtracks.c20data.C20Data(start_year, fields='all', smoothing=False, upscaling=False, verbose=True, pressure_level=850, scale_factor=2)

Class used for accessing data from C20 Reanalysis project.

This acts as a wrapper around netCDF4.Datasets and makes it easy to view data. Typically it exposes the psl, vort, and vort4 fields for one ensemble member. (vort4 is calculated using a 4th order vorticity calculation.) It will load these fields, along with corresponding maxima (vorticity) and minima (pressure) each time a new date is set for an object.

Parameters:
  • start_year – Year from which to take data
  • smoothing – Apply smoothing to data fields
  • upscaling – Upscale data using cubic splines
  • verbose – Prints lots of output
  • pressure_level – Which pressure level (850/995 hPa) to use
  • scale_factor – how much to scale data by
close_datasets()

Closes all open datasets

cvorticity(u, v)

Calculates the (2nd order) vorticity by calling into a c function

cvorticity4(u, v)

Calculates the (4th order) vorticity by calling into a c function

Algorithm was taken from Walsh’s code

first_date(ensemble_member=0)

Sets date to the first date of the year (i.e. Jan the 1st)

fourth_order_vorticity(u, v)

Calculates the (4th order) vorticity using python

Algorithm was taken from Walsh’s code

load_datasets(year)

Loads datasets for a given year

Just sets up the NetCDF4 objects, doesn’t actually load any data apart from lons/lats and dates.

next_date(ensemble_member=0)

Moves date on by one timestep (6hr)

prev_date(ensemble_member=0)

Moves date back by one timestep (6hr)

set_date(date, ensemble_member=0)

Sets date and loads all data for that date

Will have no effect if there is no difference in date or ensemble_member.

Parameters:
  • date – date to load
  • ensemble_member – ensemble member to load
Returns:

date if successful, otherwise None

set_year(year)

Sets a year and loads the relevant dataset

vorticity(u, v)

Calculates the (2nd order) vorticity using python

class stormtracks.c20data.GlobalEnsembleMember(c20data, ensemble_member=0)

Wrapper around a C20Data object

holds state of which ensemble member is currently being analysed

set_date(date)

Sets the date, does nothing if it is the same as the stored date

set_year(year)

Change the year of the stored C20Data object

stormtracks.ibtracsdata – IBTrACS Data

class stormtracks.ibtracsdata.IbStormtrack(year, name)

Holds info about an IBTrACS best track

class stormtracks.ibtracsdata.IbtracsData(data_dir=None, verbose=True)

Class used for accessing IBTrACS data

Wraps the underlying NetCDF4 files and extracts the information required from them.

Parameters:
  • data_dir – directory where IBTrACS NetCDF4 files are held
  • verbose – whether to pring lots of output
load_ibtracks_year(year, basin='NA')

Loads a given year’s worth of data

Parameters:
  • year – year to load
  • basin – which basins to load (‘all’ for all of them)
Returns:

list of loaded best tracks

load_wilma_katrina()

Loads only best tracks corresponding to Wilma and Katrina (2005)

stormtracks.tracking – Tracking of Features

class stormtracks.tracking.VortMax(date, pos, vort)

Holds key info (date, position, vorticity value) about a vorticity maximum.

To serialize this class (or any that contain objects of this class) you must make sure next_vortmax/prev_vortmax are None.

Parameters:
  • date – date of vortmax
  • pos – position (2 element tuple)
  • vort – vorticity value
add_next(vortmax)

Used to make doubly linked list of vortmaxes

class stormtracks.tracking.VortMaxTrack(start_vortmax, ensemble_member)

Stores a collection of VortMax objects in a list and adds them to a dict that is accessible through a date for easy access.

class stormtracks.tracking.VortmaxFinder(gem)

Finds all vortmaxes for a given ensemble member

Parameters:gem – GlobalEnsembleMember to use
find_vort_maxima(start_date, end_date, use_upscaled=False)

Runs over the date range looking for all vorticity maxima

class stormtracks.tracking.VortmaxKalmanFilterTracker(ensemble_member, Q_mult=0.001, R_mult=0.1)

Uses a Kalman Filter to try to track vorticity maxima

The main idea is to use the innovation parameter from the Kalman Filter estimation process as a measure of how likely it is that a vorticity maxima in a subsequent timestep belongs to the same track as a vorticity maxima in the current timestep

track_vort_maxima(vortmax_time_series)

Uses a generated list of vortmaxes to track them from one timestep to another

Parameters:vortmax_time_series – dict of vortmaxes
Returns:self.vort_tracks_by_date (OrderedDict)
class stormtracks.tracking.VortmaxNearestNeighbourTracker(ensemble_member)

Simple nearest neighbour tracker

Assumes that the two nearest points from one timestep to another belong to the same track.

track_vort_maxima(vortmax_time_series)

Uses a generated list of vortmaxes to track them from one timestep to another

Parameters:vortmax_time_series – dict of vortmaxes
Returns:self.vort_tracks_by_date (OrderedDict)

stormtracks.tracking – Matching of Tracks

class stormtracks.matching.BestTrackMatch(best_track, store_all_tracks=False)

Represents one match between many vorticity tracks

add_match(match)

Checks to see whether the macth should be added, then adds it

Compares the overlap and cumalitive distance between the match to the current average track, and if they are below a limit adds the match, which involves recalculating the average track for this match, and updating some fields.

Parameters:match – match to be added
Returns:True if it was added, otherwise False
av_dist()

Returns the average distance between all vorticity tracks

class stormtracks.matching.CycloneMatch(best_track, cyclone)

Represents one match between a best track and a vorticity track

av_dist()

Returns the average distance between the best and vorticity tracks

class stormtracks.matching.EnsembleMatch(vort_track, store_all_tracks=False)

Represents one match between many vorticity tracks

add_track(vort_track)

Checks to see whether the track should be added, then adds it

Compares the overlap and cumalitive distance between the track to the current average track, and if they are below a limit adds the track, which involves recalculating the average track for this match, and updating some fields.

Parameters:vort_track – vorticitiy track to be added
Returns:True if it was added, otherwise False
av_dist()

Returns the average distance between all vorticity tracks

class stormtracks.matching.Match(best_track, vort_track)

Represents one match between a best track and a vorticity track

av_dist()

Returns the average distance between the best and vorticity tracks

stormtracks.matching.combined_match(best_tracks, all_matches)

Uses all best tracks and matches to combine all matches for each best track

Parameters:
  • best_tracks – list of best tracks
  • all_matches – all matches to search through
Returns:

dict of combined_matches (key: best track, value: list of matches)

stormtracks.matching.good_matches(matches, dist_cutoff=None, overlap_cutoff=6)

Returns all matches that meet the requirements

Requirements are that match.av_dist() < dist_cutoff and match.overlap >= overlap_cutoff

stormtracks.matching.match_ensemble_vort_tracks_by_date(vort_tracks_by_date_list)

Takes the normal output from a tracker and flattens them to a list

stormtracks.matching.match_vort_tracks_by_date_to_best_tracks(vort_tracks_by_date, best_tracks)

Takes all vorticity tracks and best tracks and matches them up

Parameters:
  • vort_tracks_by_date – dict with dates as keys and lists of vort tracks as values
  • best_tracks – list of best tracks
Returns:

array of all matches

stormtracks.results – Store and Access Results

exception stormtracks.results.ResultNotFound

Simple exception thrown if result cannot be found in results manager or on disk

class stormtracks.results.StormtracksNumpyResultsManager(name)

Super simple key/value store for numpy arrays.

class stormtracks.results.StormtracksResultsManager(name, cache_loaded=False)

Manager class that is responsible for loading and saving all python results

Simple key/value store. Load/saves to settings.OUTPUT_DIR. Saves each result to e.g.:

settings.OUTPUT_DIR/<self.name>/<year>/<ensemble_member>-<key>.pkl

(using that as a directory/filename structure).

add_result(year, ensemble_member, result_key, result)

Adds a given result based on year, ensemble_member and a user chosen result_key

compress_year(year, delete=False)

Compresses a given year’s dir and then deletes that year

decompress_year(year)

Decompresses a given year’s tarball

delete(year, ensemble_member, result_key)

Deletes a specific result from disk

delete_year(year)

Deletes a year (use with caution!)

get_result(year, ensemble_member, result_key)

Returns a saved/already loaded result if it exists, or result from disk

throws a ResultNotFound error if it can’t find the requested result on disk.

list_ensemble_members(year)

List all results saved for a particular year

list_results(year, ensemble_member)

List all results saved for a particular year/ensemble_member

list_years()

List all saved years

save()

Saves all unsaved results that have been added so far

Utilities

stormtracks.utils.utils.compress_dir(data_dir)

Compresses a given data_dir, taking care to get file names correct

stormtracks.utils.utils.decompress_file(compressed_file)

Decompresses a given tarball, taking care to get file names correct

stormtracks.utils.utils.dist(p1, p2)

Returns the cartesian distance between two points

stormtracks.utils.utils.find_extrema(array)

Takes an array and finds its local extrema.

Returns an array with 0s for not an extrema, 1s for maxs and -1 for mins and a list of the indices of all maximums and minimums

N.B. this function is much faster than the above.

stormtracks.utils.utils.find_extrema_slow(array, print_warning=True)

Takes an array and finds its local extrema.

Returns an array with 0s for not an extrema, 1s for maxs and -1 for mins and a list of the indices of all maximums and minimums

stormtracks.utils.utils.geo_dist(p1, p2)

Returns the geodesic distance between two points

p1, p2 should be of the form (lon, lat) in degrees

stormtracks.utils.utils.pairwise(iterable)

s -> (s0,s1), (s1,s2), (s2, s3), ...

stormtracks.utils.utils.raster_voronoi(extrema, maximums, minimums)

Takes a 2D array and points of max/mins, and returns a 2D array with the voronoi sections filled with different values

stormtracks.utils.utils.upscale_field(lons, lats, field, x_scale=2, y_scale=2, is_degrees=True)

Takes a field defined on a sphere using lons/lats and returns an upscaled version, using cubic spline interpolation.

class stormtracks.utils.kalman.KalmanFilter(F, H)

Implementation of static Kalman Filter

Assumes a static model and observation operator.

Parameters:
  • F – model to use to update x (np.matrix)
  • H – observation operator (np.matrix)
estimate(x_init, P_init, z, Q, R)

Perform predict then update

Parameters:
  • Q – covariance matrix for uncertainty in model
  • R – covariance matrix for uncertainty in observation
Returns:

best estimate for x, P

predict(x_init, P_init, Q)

Predict where x, P will be based on model

update(x, P, z, R)

Update x, P based on new observation

class stormtracks.utils.kalman.Linear2DKalman

Linear 2D version of KalmanFilter

class stormtracks.utils.kalman.RTSSmoother(F, H)

Implementation of Rauch-Tung-Striebel smoother

process_data(zs, x, P, Q, R)
Parameters:
  • zs – measurements
  • x – initial state
  • P – initial covariance matrix
  • Q – process uncertainty covariance matrix
  • R – measurement uncertainty covariance matrix
Returns:

self.xs: all smoothed x values, self.Ps: all smoothed P values