Run_Pipe, example of running a rail pipeline¶

author: Eric Charles last run successfully: March 16, 2022

This notbook shows how to:

  1. Load that pipeline from a saved yaml file
  2. Run the loaded pipeline
In [1]:
import ceci
In [2]:
p = ceci.Pipeline.read('pipe_example.yml')
No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)
Inserting handle into data store.  model: ../../src/rail/examples/goldenspike/data/pretrained_flow.pkl, flow_engine_test
In [3]:
p.run()
Executing flow_engine_test
Command is:
OMP_NUM_THREADS=1   python3 -m ceci rail.creation.engines.flowEngine.FlowCreator   --model=../../src/rail/examples/goldenspike/data/pretrained_flow.pkl   --name=flow_engine_test   --config=pipe_example_config.yml   --output=./output_flow_engine_test.pq 
Output writing to ./flow_engine_test.out

Job flow_engine_test has completed successfully!

Executing lsst_error_model_test
Command is:
OMP_NUM_THREADS=1   python3 -m ceci rail.creation.degradation.lsst_error_model.LSSTErrorModel   --input=./output_flow_engine_test.pq   --name=lsst_error_model_test   --config=pipe_example_config.yml   --output=./output_lsst_error_model_test.pq 
Output writing to ./lsst_error_model_test.out

Job lsst_error_model_test has completed successfully!

Executing col_remapper_test
Command is:
OMP_NUM_THREADS=1   python3 -m ceci rail.core.utilStages.ColumnMapper   --input=./output_lsst_error_model_test.pq   --name=col_remapper_test   --config=pipe_example_config.yml   --output=./output_col_remapper_test.pq 
Output writing to ./col_remapper_test.out

Job col_remapper_test has completed successfully!

Executing table_conv_test
Command is:
OMP_NUM_THREADS=1   python3 -m ceci rail.core.utilStages.TableConverter   --input=./output_col_remapper_test.pq   --name=table_conv_test   --config=pipe_example_config.yml   --output=./output_table_conv_test.hdf5 
Output writing to ./table_conv_test.out

Job table_conv_test has completed successfully!
Out[3]:
0

Yep, that's it.¶

In [ ]: