Help! I have no variable metadata

This example shows how GSPy can help when you have a large data file and need to do the tedious task of filling out the variable metadata.

By doing a first-pass through GSPy with a data json file that is missing the variable_metadata dictionary, the code will break, but will generate a template file containing placeholder metadata dictionaries for all variables from the data file (in this case the column headers of the CSV data file). The user can then fill in this template and then add it to the data json file.

../../_images/variable_metadata_template_snippet.png
from os.path import join
from gspy import Survey

Generate the Variable Metadata Template for My Dataset

# Path to example files
data_path = '..//..//supplemental'

# Define the Survey metadata file
metadata = join(data_path, "region//MAP//data//Resolve_survey_md.json")

# Initialize the Survey
survey = Survey(metadata)
# Define input data file (CSV format) and the
# associated metadata file (without the variable_metadata dictionary)
d_data = join(data_path, 'region//MAP//data//Resolve.csv')
d_supp = join(data_path, 'region//MAP//data//Resolve_data_md_without_variables.json')

# Attempt to add the raw AEM data from CSV-format
# This will trigger an error message when no variable metadata is found, however the error will
# output a template json file with this dataset's variable names, to then be filled in by the user
survey.add_tabular(type='csv', data_filename=d_data, metadata_file=d_supp)
Traceback (most recent call last):
  File "/Users/sjames/REPOS/gspy/gspy_secondary/documentation_source/source/examples/Creating_GS_Files/plot_help_I_have_no_variable_metadata.py", line 43, in <module>
    survey.add_tabular(type='csv', data_filename=d_data, metadata_file=d_supp)
  File "/Users/sjames/REPOS/gspy/gspy_secondary/gspy/src/classes/survey/Survey.py", line 133, in add_tabular
    out = tabular_csv.Tabular_csv.read(data_filename, metadata_file=metadata_file, spatial_ref=self.spatial_ref, **kwargs)
  File "/Users/sjames/REPOS/gspy/gspy_secondary/gspy/src/classes/data/Tabular.py", line 141, in read
    cls._create_variable_metadata_template(filename, file.df.columns)
  File "/Users/sjames/REPOS/gspy/gspy_secondary/gspy/src/classes/data/tabular_csv.py", line 185, in _create_variable_metadata_template
    raise Exception(s)
Exception:
Variable metadata values are not defined in the metadata file.
Creating a template with filename variable_metadata_template_Resolve.json
Please fill out and add the dictionary to the metadata file.

Total running time of the script: ( 0 minutes 2.178 seconds)

Gallery generated by Sphinx-Gallery