ismn Package

interface Module

Created on Aug 5, 2013

@author: Christoph Paulik Christoph.Paulik@geo.tuwien.ac.at

exception pytesmo.io.ismn.interface.ISMNError[source]

Bases: exceptions.Exception

class pytesmo.io.ismn.interface.ISMN_Interface(path_to_data, network=None)[source]

Bases: object

class provides interface to ISMN data downloaded from the ISMN website

upon initialization it collects metadata from all files in path_to_data and saves metadata information in numpy file in folder path_to_data/python_metadata/ First initialization can take a minute or so if all ISMN data is present in path_to_data

Parameters :

path_to_data : string

filepath to unzipped ISMN data containing the Network folders

network : string, optional

provide name of network to only load the given network

Raises :

ISMNError :

if given network was not found in path_to_data

Attributes

metadata numpy.array metadata array for all stations contained in the path given during initialization
kdTree pytesmo.grid.nearest_neighbor.findGeoNN kdTree for finding nearest insitu station for given lon lat

Methods

find_nearest_station(lon,lat) find nearest station for given coordinates
find_nearest_station(lon, lat, return_distance=False)[source]

finds the nearest station available in downloaded data

Parameters :

lon : float

Longitude of point

lat : float

Latitude of point

return_distance : boolean, optional

if True also distance is returned

Returns :

station : ISMN_station

ISMN_station object

distance : float, optional

distance to station in meters, measured in cartesian coordinates and not on a great circle. Should be OK for small distances

get_station(stationname, network=None)

get ISMN_station object by station name

Parameters :

stationname : string

name of station

network : string, optional

network name, has to be used if stations belonging to different networks have the same name

Returns :

ISMN_station : ISMN_station object

Raises :

ISMNError :

if stationname was not found

list_networks()

returns numpy.array of networks available through the interface

Returns :

networks : numpy.array

unique network names available

list_stations(network=None)

returns numpy.array of station names available through the interface

Parameters :

network : string, optional

if network name is given only stations belonging to the network are returned

Returns :

networks : numpy.array

unique network names available

plot_station_locations()

plots available stations on a world map in robinson projection only available if basemap is installed

Raises :

ISMNError :

if basemap is not installed

stations_that_measure(variable)

Goes through all stations and returns those that measure the specified variable

Parameters :

variable : string

variable name one of

  • ‘soil moisture’,
  • ‘soil temperature’,
  • ‘soil suction’,
  • ‘precipitation’,
  • ‘air temperature’,
  • ‘field capacity’,
  • ‘permanent wilting point’,
  • ‘plant available water’,
  • ‘potential plant available water’,
  • ‘saturation’,
  • ‘silt fraction’,
  • ‘snow depth’,
  • ‘sand fraction’,
  • ‘clay fraction’,
  • ‘organic carbon’,
  • ‘snow water equivalent’,
  • ‘surface temperature’,
  • ‘surface temperature quality flag original’
Returns :

ISMN_station : ISMN_station object

class pytesmo.io.ismn.interface.ISMN_station(metadata)[source]

Bases: object

Knows everything about the station, like which variables are measured there in which depths and in which files the data is stored. This is not completely true for the CEOP format since depth_from and depth_to are not easily knowable without parsing the whole file. For CEOP format depth_from and depth_to will only contain the phrase ‘multiple’ instead of the actual depth

Parameters :

metadata : numpy.array

part of the structured array from metadata_collector.collect_from_folder() which contains only fields for one station

Attributes

network string network the time series belongs to
station string station name the time series belongs to
latitude float latitude of station
longitude float longitude of station
elevation float elevation of station
variables numpy.array

variables measured at this station one of

  • ‘soil moisture’,
  • ‘soil temperature’,
  • ‘soil suction’,
  • ‘precipitation’,
  • ‘air temperature’,
  • ‘field capacity’,
  • ‘permanent wilting point’,
  • ‘plant available water’,
  • ‘potential plant available water’,
  • ‘saturation’,
  • ‘silt fraction’,
  • ‘snow depth’,
  • ‘sand fraction’,
  • ‘clay fraction’,
  • ‘organic carbon’,
  • ‘snow water equivalent’,
  • ‘surface temperature’,
  • ‘surface temperature quality flag original’
depth_from numpy.array shallower depth of layer the variable with same index was measured at
depth_to numpy.array deeper depth of layer the variable with same index was measured at
sensors numpy.array sensor names of variables
filenames numpy.array filenames in which the data is stored

Methods

get_variables() returns the variables measured at this station
get_depths(variable) get the depths in which a variable was measured at this station
get_sensors(variable,depth_from,depth_to) get the sensors for the given variable, depth combination
read_variable(variable,depth_from=None,depth_to=None,sensor=None) read the data for the given parameter combination
data_for_variable(variable, min_depth=None, max_depth=None)

function to go through all the depth_from, depth_to, sensor combinations for the given variable and yields ISMNTimeSeries if a match is found. if min_depth and/or max_depth where given it only returns a ISMNTimeSeries if depth_from >= min_depth and/or depth_to <= max_depth

Parameters :

variable: string :

variable to read one of

  • ‘soil moisture’,
  • ‘soil temperature’,
  • ‘soil suction’,
  • ‘precipitation’,
  • ‘air temperature’,
  • ‘field capacity’,
  • ‘permanent wilting point’,
  • ‘plant available water’,
  • ‘potential plant available water’,
  • ‘saturation’,
  • ‘silt fraction’,
  • ‘snow depth’,
  • ‘sand fraction’,
  • ‘clay fraction’,
  • ‘organic carbon’,
  • ‘snow water equivalent’,
  • ‘surface temperature’,
  • ‘surface temperature quality flag original’

min_depth : float, optional

depth_from of variable has to be >= min_depth in order to be included.

max_depth : float, optional

depth_to of variable has to be <= max_depth in order to be included.

Returns :

time_series : iterator(pytesmo.io.ismn.readers.ISMNTimeSeries)

ISMNTimeSeries object containing data and metadata

get_depths(variable)[source]

get depths at which the given variable was measured at this station

Parameters :

variable : string

variable string best one of those returned by get_variables() or one of

  • ‘soil moisture’,
  • ‘soil temperature’,
  • ‘soil suction’,
  • ‘precipitation’,
  • ‘air temperature’,
  • ‘field capacity’,
  • ‘permanent wilting point’,
  • ‘plant available water’,
  • ‘potential plant available water’,
  • ‘saturation’,
  • ‘silt fraction’,
  • ‘snow depth’,
  • ‘sand fraction’,
  • ‘clay fraction’,
  • ‘organic carbon’,
  • ‘snow water equivalent’,
  • ‘surface temperature’,
  • ‘surface temperature quality flag original’
Returns :

depth_from : numpy.array

depth_to : numpy.array

get_sensors(variable, depth_from, depth_to)[source]

get the sensors at which the variable was measured at the given depth

Parameters :

variable : string

variable abbreviation one of

  • ‘soil moisture’,
  • ‘soil temperature’,
  • ‘soil suction’,
  • ‘precipitation’,
  • ‘air temperature’,
  • ‘field capacity’,
  • ‘permanent wilting point’,
  • ‘plant available water’,
  • ‘potential plant available water’,
  • ‘saturation’,
  • ‘silt fraction’,
  • ‘snow depth’,
  • ‘sand fraction’,
  • ‘clay fraction’,
  • ‘organic carbon’,
  • ‘snow water equivalent’,
  • ‘surface temperature’,
  • ‘surface temperature quality flag original’

depth_from : float

shallower depth of layer the variable was measured at

depth_to : float

deeper depth of layer the variable was measured at

Returns :

sensors : numpy.array

array of sensors found for the given combination of variable and depths

Raises :

ISMNError :

if no sensor was found for the given combination of variable and depths

get_variables()[source]

get a list of variables measured at this station

Returns :

variables : numpy.array

array of variables measured at this station

read_variable(variable, depth_from=None, depth_to=None, sensor=None)[source]

actually reads the given variable from the file. Parameters are required until any ambiguity is resolved. If there is only one depth for the given variable then only variable is required. If there are multiple depths at least depth_from is required. If there are multiple depth_to possibilities for one variable-depth_from combination also depth_to has to be specified. If 2 sensors are measuring the same variable in the same depth then also the sensor has to be specified.

Parameters :

variable: string :

variable to read one of

  • ‘soil moisture’,
  • ‘soil temperature’,
  • ‘soil suction’,
  • ‘precipitation’,
  • ‘air temperature’,
  • ‘field capacity’,
  • ‘permanent wilting point’,
  • ‘plant available water’,
  • ‘potential plant available water’,
  • ‘saturation’,
  • ‘silt fraction’,
  • ‘snow depth’,
  • ‘sand fraction’,
  • ‘clay fraction’,
  • ‘organic carbon’,
  • ‘snow water equivalent’,
  • ‘surface temperature’,
  • ‘surface temperature quality flag original’

depth_from : float, optional

shallower depth of layer the variable was measured at

depth_to : float, optional

deeper depth of layer the variable was measured at

sensor : string, optional

name of the sensor

Returns :

data : readers.ISMNTimeSeries

ISMNTimeSeries object containing the relevant metadata for the time series as well as a .data pointing to a pandas.DataFrame

Raises :

ISMNError: :

if not all ambiguity was resolved by the given input parameters or if no data was found for the given input parameters

metadata_collector Module

Created on Aug 1, 2013

@author: Christoph Paulik christoph.paulik@geo.tuwien.ac.at

pytesmo.io.ismn.metadata_collector.collect_from_folder(rootdir)[source]

function walks the rootdir directory and looks for network folders and ISMN datafiles. It collects metadata for every file found and returns a numpy.ndarray of metadata

Parameters :

rootdir : string

root directory on filesystem where the ISMN data was unzipped to

Returns :

metadata : numpy.ndarray

structured numpy array which contains the metadata for one file per row

readers Module

Created on Jul 31, 2013

@author: Christoph Paulik christoph.paulik@geo.tuwien.ac.at

exception pytesmo.io.ismn.readers.ISMNTSError[source]

Bases: exceptions.Exception

class pytesmo.io.ismn.readers.ISMNTimeSeries(data)[source]

Bases: object

class that contains a time series of ISMN data read from one text file

Attributes

network string network the time series belongs to
station string station name the time series belongs to
latitude float latitude of station
longitude float longitude of station
elevation float elevation of station
variable list variable measured
depth_from list shallower depth of layer the variable was measured at
depth_to list deeper depth of layer the variable was measured at
sensor string sensor name
data pandas.DataFrame data of the time series

Methods

plot(*args, **kwargs)[source]

wrapper for pandas.DataFrame.plot which adds title to plot and drops NaN values for plotting Returns ——- ax : axes

matplotlib axes of the plot
Raises :

ISMNTSError :

if data attribute is not a pandas.DataFrame

exception pytesmo.io.ismn.readers.ReaderException[source]

Bases: exceptions.Exception

pytesmo.io.ismn.readers.get_format(filename)[source]

get’s the file format from the length of the header and filename information

Parameters :

filename : string

Returns :

methodname : string

name of method used to read the detected format

Raises :

ReaderException :

if filename or header parts do not fit one of the formats

pytesmo.io.ismn.readers.get_info_from_file(filename)[source]

reads first line of file and splits filename this can be used to construct necessary metadata information for all ISMN formats

Parameters :

filename : string

filename including path

Returns :

header_elements : list

first line of file split into list

filename_elements : list

filename without path split by _

pytesmo.io.ismn.readers.get_metadata(filename)[source]

reads ISMN metadata from any format

Parameters :filename: string :
Returns :metadata: dict :
pytesmo.io.ismn.readers.get_metadata_ceop(filename)[source]

get metadata from ISMN textfiles in the format called CEOP Reference Data Format

Parameters :

filename : string

path and name of file

Returns :

metadata : dict

dictionary of metadata information

pytesmo.io.ismn.readers.get_metadata_ceop_sep(filename)[source]

get metadata from ISMN textfiles in the format called Variables stored in separate files (CEOP formatted)

Parameters :

filename : string

path and name of file

Returns :

metadata : dict

dictionary of metadata information

pytesmo.io.ismn.readers.get_metadata_header_values(filename)[source]

get metadata from ISMN textfiles in the format called Variables stored in separate files (CEOP formatted)

Parameters :

filename : string

path and name of file

Returns :

metadata : dict

dictionary of metadata information

pytesmo.io.ismn.readers.read_data(filename)[source]

reads ISMN data in any format

Parameters :filename: string :
Returns :timeseries: IMSNTimeSeries :
pytesmo.io.ismn.readers.read_format_ceop(filename)[source]

Reads ISMN textfiles in the format called CEOP Reference Data Format

Parameters :

filename : string

path and name of file

Returns :

time_series : ISMNTimeSeries

ISMNTimeSeries object initialized with metadata and data from file

pytesmo.io.ismn.readers.read_format_ceop_sep(filename)[source]

Reads ISMN textfiles in the format called Variables stored in separate files (CEOP formatted)

Parameters :

filename : string

path and name of file

Returns :

time_series : ISMNTimeSeries

ISMNTimeSeries object initialized with metadata and data from file

pytesmo.io.ismn.readers.read_format_header_values(filename)[source]

Reads ISMN textfiles in the format called Variables stored in separate files (Header + values)

Parameters :

filename : string

path and name of file

Returns :

time_series : ISMNTimeSeries

ISMNTimeSeries object initialized with metadata and data from file

Table Of Contents

Previous topic

io Package

Next topic

test Package

This Page