raw.flight_tickets:

preprocessed.flight_tickets:

processed.flight_tickets:

# SCENARIO 1 - we get a new excel file

# raw data  --> SEPARATE FOLDER??
1. Place file in storage system of project
2. Create catalog raw entry to load excel file into code
3. Add validation on data that's read

# preprocessed data
1. Change column names into intuitive formats (snake_case, no spaces, no capital letters, logical name)
2. Change data types to required ones, i.e. fix schema
3. Add validation
4. Save as parquet file (schema is included)

# processed data
1. transform, join, etc your data to required format
2. Add validation
3. Save as parquet file (schema is included)
