{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "useeio-title",
   "metadata": {},
   "source": [
    "# USEEIO parser walkthrough\n",
    "\n",
    "This notebook is the practical guide for parsing `USEEIO` workbooks in MARIO."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "useeio-coverage",
   "metadata": {},
   "source": [
    "## What this notebook covers\n",
    "\n",
    "- where the currently supported `USEEIO` workbook exports come from;\n",
    "- what the main published model aliases mean;\n",
    "- what MARIO parses today and what it intentionally does not parse;\n",
    "- how to interpret `format=` correctly;\n",
    "- how to use `path=` and `format=`;\n",
    "- why the resulting database is treated as a `SUT`;\n",
    "- one important caveat on release year versus internal IO year."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "useeio-sources",
   "metadata": {},
   "source": [
    "## Relevant source pages\n",
    "\n",
    "- Data.gov catalog page: [USEEIO v2.5 models](https://catalog.data.gov/dataset/useeio-v2-5-models)\n",
    "- EPA model page: [USEEIO models](https://www.epa.gov/land-research/us-environmentally-extended-input-output-useeio-models)\n",
    "- EPA technical page: [USEEIO technical content](https://www.epa.gov/land-research/us-environmentally-extended-input-output-useeio-technical-content)\n",
    "- Framework and workbook format reference: [USEPA/useeior](https://github.com/USEPA/useeior)\n",
    "\n",
    "MARIO does not download USEEIO workbooks automatically. The expected workflow is to download one workbook manually and then point the parser to that local file."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "useeio-entrypoint",
   "metadata": {},
   "source": [
    "## Main entry point\n",
    "\n",
    "For normal user workflows, the public entry point is:\n",
    "\n",
    "- `mario.parse_useeio(...)`\n",
    "\n",
    "At the moment, MARIO supports only the verified workbook export family behind that entry point."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "useeio-scope",
   "metadata": {},
   "source": [
    "## What MARIO currently parses\n",
    "\n",
    "The current backend is intentionally narrow:\n",
    "\n",
    "- it parses the local Excel workbook export;\n",
    "- it currently supports only the verified `v2.5` workbook structure;\n",
    "- it does **not** parse the full `useeior` build framework directly.\n",
    "\n",
    "This is why the parser exposes an explicit `format=` argument even though only one format is supported today."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "useeio-aliases",
   "metadata": {},
   "source": [
    "## What the main model aliases mean\n",
    "\n",
    "The official `USEPA/USEEIO` model registry distinguishes model **aliases** such as `yellowthroat`, `kingbird`, or `catbird`. These aliases describe the **content** of the model, not the workbook layout.\n",
    "\n",
    "For the currently relevant national `v2.5` families:\n",
    "\n",
    "- `yellowthroat`: BEA Summary 2017, GLORIA-backed, with GHG and material-footprint extensions;\n",
    "- `waxwing`: BEA Detail 2017, GLORIA-backed, with GHG and material-footprint extensions;\n",
    "- `kingbird`: BEA Summary 2017, EXIOBASE-backed, with GHG extensions;\n",
    "- `kinglet`: BEA Detail 2017, EXIOBASE-backed, with GHG extensions;\n",
    "- `oriole`: BEA Summary 2017, CEDA-backed, with GHG extensions;\n",
    "- `catbird`: BEA Detail 2017, CEDA-backed, with GHG extensions.\n",
    "\n",
    "The current MARIO parser targets these national workbook exports, not the `StateEEIO` families that are listed in the same upstream registry.\n",
    "\n",
    "One caution from the upstream registry: `USEEIO v2.5-waxwing-22` is marked as deprecated because it was published with an incorrect extension and replaced upstream by `v2.5.1-waxwing-22`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "useeio-format-meaning",
   "metadata": {},
   "source": [
    "## What `format` means\n",
    "\n",
    "`format=` is a **parser-side** selector. It tells MARIO what workbook organization it should expect.\n",
    "\n",
    "So:\n",
    "\n",
    "- `yellowthroat` and `kingbird` are different models;\n",
    "- but they can still share the same parser `format` if their workbook tabs and matrix semantics are organized in the same way.\n",
    "\n",
    "Today the parser supports:\n",
    "\n",
    "- `format=\"auto\"`: inspect the workbook and resolve the known layout automatically;\n",
    "- `format=\"v2.5_workbook\"`: force the currently verified workbook structure.\n",
    "\n",
    "For this verified format, MARIO expects the workbook to expose the standard `useeior` export components such as `V`, `U`, `B`, `q`, `commodities_meta`, `final_demand_meta`, and `value_added_meta`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b90eeae4",
   "metadata": {},
   "source": [
    "## Key arguments and workbook layout\n",
    "\n",
    "Key public arguments:\n",
    "\n",
    "- `path`: one local `USEEIO*.xlsx` workbook or one directory containing one workbook;\n",
    "- `format`: use `\"auto\"` or `\"v2.5_workbook\"`;\n",
    "- `table`: currently only `\"SUT\"` is supported.\n",
    "\n",
    "Typical local layout:\n",
    "\n",
    "```text\n",
    "USEEIO/\n",
    "├── USEEIOv2.5-yellowthroat-22.xlsx\n",
    "├── USEEIOv2.5-kingbird-22.xlsx\n",
    "└── USEEIOv2.5-catbird-22.xlsx\n",
    "```\n",
    "\n",
    "Inside the workbook, MARIO expects the `useeior` export sheets such as `V`, `U`, `B`, `q`, `commodities_meta`, `final_demand_meta`, and `value_added_meta`. The workbook release label can differ from the internal IO year stored in the parsed metadata."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "useeio-import",
   "metadata": {},
   "outputs": [],
   "source": [
    "import mario"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "useeio-explicit-workbook",
   "metadata": {},
   "source": [
    "## Parse one explicit workbook\n",
    "\n",
    "Use this when you already know the exact workbook you want to ingest."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "useeio-parse-auto",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Parser: USEEIO workbook parsed with 71 activities, 73 commodities, 3 value-added rows and 30 satellite rows.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Metadata: initialized.\n"
     ]
    }
   ],
   "source": [
    "db = mario.parse_useeio(\n",
    "    path=\"/path/to/USEEIOv2.5-yellowthroat-22.xlsx\",\n",
    "    format=\"auto\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "useeio-pin-format",
   "metadata": {},
   "source": [
    "## Pin the workbook format explicitly\n",
    "\n",
    "If you want to be strict and avoid `auto` detection, pin the verified format directly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "useeio-parse-format",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Parser: USEEIO workbook parsed with 402 activities, 402 commodities, 3 value-added rows and 30 satellite rows.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Metadata: initialized.\n"
     ]
    }
   ],
   "source": [
    "db = mario.parse_useeio(\n",
    "    path=\"/path/to/USEEIOv2.5-waxwing-22.xlsx\",\n",
    "    format=\"v2.5_workbook\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "useeio-directory",
   "metadata": {},
   "source": [
    "## Parse from a directory\n",
    "\n",
    "You can also point the parser to a directory, but only when that directory contains a single workbook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "useeio-parse-directory",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Parser: USEEIO workbook parsed with 71 activities, 73 commodities, 3 value-added rows and 30 satellite rows.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Metadata: initialized.\n"
     ]
    }
   ],
   "source": [
    "db = mario.parse_useeio(\n",
    "    path=\"/path/to/USEEIO\",\n",
    "    model_alias=\"yellowthroat\",\n",
    "    release_year=2022,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "useeio-sut-logic",
   "metadata": {},
   "source": [
    "## Why MARIO parses USEEIO as a SUT\n",
    "\n",
    "For the verified workbook family, the structure is closer to a split-native `SUT` than to a symmetric `IOT`:\n",
    "\n",
    "- `V` is the make matrix;\n",
    "- `U` contains the use block plus final demand columns and value-added rows;\n",
    "- `B` is a direct environmental coefficient matrix;\n",
    "- `q` provides the commodity-output vector used to reconstruct direct environmental flows.\n",
    "\n",
    "So the parser returns the native `S`, `U`, `Yc`, `Va`, `Ec`, ... blocks rather than forcing a symmetric `IOT` interpretation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "useeio-ec-logic",
   "metadata": {},
   "source": [
    "## Commodity-side environmental extension\n",
    "\n",
    "One important detail of the verified `v2.5` workbook layout is that `B` is aligned with the **commodity** axis. Because of that, MARIO reconstructs the direct extension on the commodity side:\n",
    "\n",
    "- `Ec = B * q`\n",
    "\n",
    "This is deliberate. For this workbook family, `Ec` is the correct direct extension block, while `Ea` stays zero-filled."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "useeio-year-caveat",
   "metadata": {},
   "source": [
    "## Release year versus internal IO year\n",
    "\n",
    "Some USEEIO workbooks expose a release label that differs from the internal IO year used by the model. MARIO stores the **internal IO year** in the parsed database metadata.\n",
    "\n",
    "So if a workbook is labelled like `...-22.xlsx` but the internal economic base year is `2017`, MARIO will record `2017` as `db.meta.year`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "useeio-meta-check",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('USEEIO v2.5 yellowthroat 2022', 2022, 'Model-year USD')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db.meta.name, db.meta.year, db.meta.price"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "useeio-limitations",
   "metadata": {},
   "source": [
    "## Current limitations\n",
    "\n",
    "- only the verified `v2.5_workbook` format is supported;\n",
    "- there is no MARIO downloader yet;\n",
    "- the parser targets the workbook export, not the full `useeior` framework pipeline.\n",
    "\n",
    "That is also why keeping `format=` explicit is useful: it leaves room for future workbook families without pretending they are already supported."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mario",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
