{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "title-hybrid",
   "metadata": {},
   "source": [
    "# Hybrid EXIOBASE walkthrough\n",
    "\n",
    "This notebook is the practical walkthrough for parsing the hybrid EXIOBASE bundle in MARIO.\n",
    "\n",
    "It focuses on:\n",
    "\n",
    "- hybrid HSUT;\n",
    "- hybrid HIOT;\n",
    "- extension selection.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hybrid-what-covers",
   "metadata": {},
   "source": [
    "## What this notebook covers\n",
    "\n",
    "- which parser entry points to use for hybrid EXIOBASE;\n",
    "- what local bundle MARIO expects;\n",
    "- how to parse the hybrid SUT;\n",
    "- how to parse the hybrid IOT;\n",
    "- how to limit the parsed extensions when needed.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hybrid-links",
   "metadata": {},
   "source": [
    "## Relevant Zenodo releases\n",
    "\n",
    "- [v3.3.18](https://doi.org/10.5281/zenodo.7244919)\n",
    "- newer consequential release not yet covered by the current MARIO hybrid parser: \n",
    "\n",
    "Important limitation: the current hybrid parser targets the **3.3.18** bundle. It does **not** yet include the later consequential developments released separately on [Zenodo](https://zenodo.org/records/15421526).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hybrid-entry-points",
   "metadata": {},
   "source": [
    "## Main entry points\n",
    "\n",
    "For normal user workflows, the public entry points are:\n",
    "\n",
    "- `mario.parse_exiobase(...)`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dff2731e",
   "metadata": {},
   "source": [
    "## Key arguments and local layout\n",
    "\n",
    "The hybrid workflow uses the same `mario.parse_exiobase(...)` entry point with these main selectors:\n",
    "\n",
    "- `table`: choose `\"IOT\"` or `\"SUT\"`;\n",
    "- `unit`: use `\"Hybrid\"`;\n",
    "- `path`: extracted `3.3.18` release root;\n",
    "- `extensions`: `None`, `[]`, `\"all\"`, or one explicit list of extension groups.\n",
    "\n",
    "Typical extracted layout:\n",
    "\n",
    "```text\n",
    "EXIOBASE/3.3.18/\n",
    "├── MR_HSUT_*.txt\n",
    "├── MR_HIOT_*.txt\n",
    "├── Land_*.txt\n",
    "├── Emiss_*.txt\n",
    "├── waste_sup_*.txt\n",
    "└── waste_use_*.txt\n",
    "```\n",
    "\n",
    "Supported extension groups are `resource`, `Land`, `Emiss`, `Emis_unreg_w`, `waste_sup`, `waste_use`, `pack_sup_waste`, `pack_use_waste`, `mach_sup_waste`, `mach_use_waste`, `stock_addition`, and `crop_res`. `Unreg_w` is available for `HSUT` only."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "hybrid-imports",
   "metadata": {},
   "outputs": [],
   "source": [
    "import mario"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hybrid-download",
   "metadata": {},
   "source": [
    "## Optional download step\n",
    "\n",
    "If you do not already have the bundle locally, MARIO exposes a helper for the hybrid release it supports.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "hybrid-download-code",
   "metadata": {},
   "outputs": [],
   "source": [
    "info = mario.download_hybrid_exiobase(\n",
    "    path=\"/path/to/3.3.18\",\n",
    "    )\n",
    "info"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hybrid-sut",
   "metadata": {},
   "source": [
    "## Parse the hybrid SUT\n",
    "\n",
    "This is the standard starting point for the hybrid supply-use table.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "hybrid-sut-code",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Parser: reading EXIOBASE hybrid HSUT from /Users/lorenzorinaldi/Library/CloudStorage/OneDrive-SharedLibraries-PolitecnicodiMilano/DENG-SESAM - Documenti/c-Research/a-Datasets/_Input Output Databases/EXIOBASE/3.3.18.\n",
      "INFO Parser: EXIOBASE hybrid HSUT parsed with 164 activities, 200 commodities, 7 factor rows and 350 extension rows.\n",
      "INFO Metadata: initialized.\n"
     ]
    }
   ],
   "source": [
    "db_sut = mario.parse_exiobase(\n",
    "    path=\"/path/to/3.3.18\",\n",
    "    table=\"SUT\",\n",
    "    unit=\"Hybrid\",\n",
    "    extensions=\"all\", \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hybrid-iot",
   "metadata": {},
   "source": [
    "## Parse the hybrid IOT\n",
    "\n",
    "Use this when you need the hybrid input-output representation instead of the HSUT.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "hybrid-iot-code",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Parser: reading EXIOBASE hybrid HIOT from /Users/lorenzorinaldi/Library/CloudStorage/OneDrive-SharedLibraries-PolitecnicodiMilano/DENG-SESAM - Documenti/c-Research/a-Datasets/_Input Output Databases/EXIOBASE/3.3.18.\n",
      "INFO Parser: EXIOBASE hybrid HIOT parsed with 164 sectors, 7 factor rows and 119 extension rows.\n",
      "INFO Metadata: initialized.\n"
     ]
    }
   ],
   "source": [
    "db_iot = mario.parse_exiobase(\n",
    "    path=\"/path/to/3.3.18\",\n",
    "    table=\"IOT\",\n",
    "    unit=\"Hybrid\",\n",
    "    extensions=[\"resource\", \"Emiss\"],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hybrid-inspect",
   "metadata": {},
   "source": [
    "## Inspect the parsed database\n",
    "\n",
    "Once parsed, the result is a standard MARIO database. From here the normal exploration methods apply.\n",
    "\n",
    "A simple way to inspect a parsed MARIO database is to evaluate the object name in a notebook cell (for example, `db_iot` or `db_sut`).  \n",
    "This prints a compact summary of its core properties."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "5be95832",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "name = None\n",
       "table = SUT\n",
       "tech_assumption = industry-based\n",
       "scenarios = ['baseline']\n",
       "Activity = 164\n",
       "Commodity = 200\n",
       "Factor of production = 7\n",
       "Satellite account = 350\n",
       "Consumption category = 6\n",
       "Region = 48"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db_sut"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71c31a8f",
   "metadata": {},
   "source": [
    "When dealing with hybrid tables, it's fundamental to have clear view of the unit of measures. Units are stored in a dictionary, as shown below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "hybrid-inspect-code",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Paddy rice</th>\n",
       "      <td>tonnes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Wheat</th>\n",
       "      <td>tonnes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cereal grains nec</th>\n",
       "      <td>tonnes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Vegetables; fruit; nuts</th>\n",
       "      <td>tonnes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Oil seeds</th>\n",
       "      <td>tonnes</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           unit\n",
       "Paddy rice               tonnes\n",
       "Wheat                    tonnes\n",
       "Cereal grains nec        tonnes\n",
       "Vegetables; fruit; nuts  tonnes\n",
       "Oil seeds                tonnes"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db_sut.units[\"Commodity\"].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b06a850",
   "metadata": {},
   "source": [
    "The same exploration, can be done for IOTs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "50581bf5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>unit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Employment: Low-skilled male</th>\n",
       "      <td>1000 persons</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Employment: Low-skilled female</th>\n",
       "      <td>1000 persons</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Employment: Medium-skilled male</th>\n",
       "      <td>1000 persons</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Employment: Medium-skilled female</th>\n",
       "      <td>1000 persons</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Employment: High-skilled male</th>\n",
       "      <td>1000 persons</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Ni (soil - Emiss)</th>\n",
       "      <td>tonnes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>C (air - Emiss)</th>\n",
       "      <td>tonnes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>other emissions (undef - Emiss)</th>\n",
       "      <td>tonnes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Carbon dioxide, biogenic (air - Emiss)</th>\n",
       "      <td>tonnes</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Other emissions nec (air - Emiss)</th>\n",
       "      <td>tonnes</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>119 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                unit\n",
       "Employment: Low-skilled male            1000 persons\n",
       "Employment: Low-skilled female          1000 persons\n",
       "Employment: Medium-skilled male         1000 persons\n",
       "Employment: Medium-skilled female       1000 persons\n",
       "Employment: High-skilled male           1000 persons\n",
       "...                                              ...\n",
       "Ni (soil - Emiss)                             tonnes\n",
       "C (air - Emiss)                               tonnes\n",
       "other emissions (undef - Emiss)               tonnes\n",
       "Carbon dioxide, biogenic (air - Emiss)        tonnes\n",
       "Other emissions nec (air - Emiss)             tonnes\n",
       "\n",
       "[119 rows x 1 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db_iot.units[\"Satellite account\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hybrid-notes",
   "metadata": {},
   "source": [
    "## Notes\n",
    "\n",
    "- hybrid parsing is more dataset-specific than the monetary case, so the direct hybrid entry points are often acceptable in documentation examples;\n",
    "- keep extension selection explicit if you want lighter parse results;\n",
    "- use the standard MARIO inspection workflow after parsing, exactly as for other databases;\n",
    "- if you need the newer consequential developments, the current hybrid parser is not there yet.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mario",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
