Introduction to pytesmo a python Toolbox for the Evaluation of Soil Moisture Observations

pytesmo is a package which aims it is to provide a standard library that can be used for the comparison and validation of geospatial time series datasets with a focus on soil moisture.

It contains an expanding collection of readers for different soil moisture datasets (see Supported Datasets) as well as routines for comparing them. Special classes in the module pytesmo.grid.grids provide easy nearest neighbor searching between datasets as well as the calculation of lookup tables of nearest neighbours. They also provide possibilities to easily read all grid points of a dataset in the correct order.

It contains the code used for the calculation of metrics by the Satellite Soil Moisture Validation Tool For ASCAT. See pytesmo.metrics.

Features

  • easily read data from the Supported Datasets
  • anomaly calculation based on climatology or using a moving window see pytesmo.time_series.anomaly
  • easy temporal matching of time series see pytesmo.temporal_matching
  • multiple methods for scaling between different observation domains (CDF matching, linear regreesion, min-max matching) see pytesmo.scaling
  • calculate standard metrics like correlation coefficients, RMSD, bias, as well as more complex ones like triple collocation or MSE as a decomposition of the RMSD see pytesmo.metrics

Supported Datasets

Soil moisture is observed using different methods and instruments, in this version several satellite datasets as well as in situ data are supported.

ERS

ASCAT

Data from the International Soil Moisture Network (ISMN)

ISMN data can be downloaded for free after registration from http://ismn.geo.tuwien.ac.at/

In case of the ISMN, 3 different formats are provided:

  • Variables stored in separate files (CEOP formatted)

    this format is supported 100% and should work with all examples

  • Variables stored in separate files (Header+values)

    this format is supported 100% and should work with all examples

  • CEOP Reference Data Format

    this format can be read with the readers in pytesmo.io.ismn.readers but is not supported for more complex queries since it is missing sensor information , only provides soil moisture and soil temperature and contains several depths in one file.

If you downloaded ISMN data in one of the supported formats in the past it can be that station names are not recognized correctly because they contained the ‘_’ character which is supposed to be the seperator. If you experience problems because of this please download new data from the ISMN since this issue should be fixed.

Installation

Prerequisites

In order to enjoy all pytesmo features python version 2.7.5 with the following packages has to be installed

optional

Windows - new python users

For users with little python experience, using Windows, the easiest way to install everything but matplotlib-basemap and netCDF4 is to install winpython from https://code.google.com/p/winpython/ and then download basemap from http://sourceforge.net/projects/matplotlib/files/matplotlib-toolkits/ and netCDF4 from https://code.google.com/p/netcdf4-python/ and add it to your winpython installation using the winpython Control Panel.

Just make sure that you download both for the same architecture (32/64 bit) and the same python version (2.7.x)

After that you can also use the winpython control panel to add the relevant pytesmo Windows binaries

After that you can open spyder or the Ipython notebook from the winpython installation directory and start testing pytesmo.

If you want a system installation of python download the following files and install them in order.

Linux

If you already have a working python installation with the necessary packages just change directory to the unzipped pytesmo-0.1.1 folder and use the following command in the command line:

python setup.py install

or if you’d rather use pip then use the command:

pip install pytesmo

Contribute

If you would like to help this project by improving the documentation, providing examples of how you use it or by extending the functionality of pytesmo we would be very happy.

Please browse the source code which is available at http://github.com/TUW-GEO/pytesmo

Feel free to contact Christoph Paulik in case of any questions or requests.