experiments

Experiment Definitions

Below is the schema for an experiment, illustrating all of the different options and their expected data types.

{
  "experiment_definition": {
    "target": "<string>",
    "signal_dimensions": ["<string>"],
    "link_function": "<string>",
    "time_horizons": ["<integer>"],
    "time_index": "<string>",
    "include_features": ["<string>"],
    "drop_features": ["<string>"],
    "time_validation_splits": ["<string>"],
    "train_end": "<string>",
    "train_end": "<string>",
    "forecast_end": "<string>",
    "forecast_end": "<string>",
    "validation_end": "<string>",
    "validation_end": "<string>",
    "encode_features": ["<string>"],
    "scenarios": [
        {
          "feature": "<string>",
          "values": ["<string, integer, float>"],
          "start": "<string>",
          "end": "<string>"
        }
    ],
    "scenario_freq": "<string>",
    "data_path": "<string>",
    "confidence_intervals": ["<integer>", "<integer>"],
    "joins": [
      {
        "data_path": "<string>",
        "join_on": ["<string>", "<string>"],
        "as": "<string>"
      }
    ]
    "bootstrap_sample": "<integer>",
  }
}

Experiment Persistence

Experiment artifacts are persisted either locally or to S3 depending on the use of the –local flag when running the experiment command and will produce a local output structure as shown below:

experiment path
  |- models
  |    |
  |    \- h_{forecast horizon}
  |           |-fit_model.joblib
  |           |-bootstrap
  |                |
  |                |- bootstrap_model_{random seed}
  |
  |- forecast
  |    |
  |    |- common_meta.parquet
  |    |- forecast_partition_0_meta.parquet
  |    |- forecast_partition_0.parquet
  |    \  ...
  |
  |- validation
       |
       |- metrics.json
       \- {validation split}
              |
              |- validation_partition_0_meta.parquet
              |- validation_partition_0.parquet
              \  ...