{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cf891cb1",
   "metadata": {},
   "source": [
    "# Eora parser walkthrough\n",
    "\n",
    "This notebook is the practical guide for parsing the Eora datasets supported by MARIO.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71633ddc",
   "metadata": {},
   "source": [
    "<div class=\"admonition warning\">\n",
    "<div class=\"admonition-title\">Warning</div>\n",
    "At the moment, MARIO supports only <strong>EORA1</strong>.\n",
    "<br><br>\n",
    "Support for <strong>EORA2</strong> is planned as soon as possible.\n",
    "<br><br>\n",
    "EORA website: <a href=\"https://www.eora.org\">https://www.eora.org</a>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1b7a6cbb",
   "metadata": {},
   "source": [
    "## What this notebook covers\n",
    "\n",
    "- which Eora workflows MARIO supports and which it does not;\n",
    "- the difference between `Eora26` and single-region local files;\n",
    "- which files are required for the `Eora26` workflow;\n",
    "- how `multi_region=`, `indeces=`, `country=`, and `price=` are used;\n",
    "- how `name_convention=` and `aggregate_trade=` affect the single-region workflow;\n",
    "- which parser caveats matter in practice.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "841aabce",
   "metadata": {},
   "source": [
    "## Important scope note\n",
    "\n",
    "MARIO does **not** parse the full Eora MRIO release.\n",
    "\n",
    "The supported workflows are only:\n",
    "\n",
    "- `Eora26` as the multi-region workflow;\n",
    "- local single-region Eora tables.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92e8a2ab",
   "metadata": {},
   "source": [
    "## Relevant source page\n",
    "\n",
    "- Official Eora website: [Eora MRIO portal](https://www.worldmrio.com/)\n",
    "\n",
    "Automatic download is not part of the current MARIO workflow. The expected path is to work from local files that you already downloaded.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68e0d540",
   "metadata": {},
   "source": [
    "## Main entry point\n",
    "\n",
    "For normal user workflows, the public entry point is:\n",
    "\n",
    "- `mario.parse_eora(...)`\n",
    "\n",
    "The same function supports:\n",
    "\n",
    "- `Eora26` with `multi_region=True`;\n",
    "- local single-region Eora files with `multi_region=False`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f21705b",
   "metadata": {},
   "source": [
    "## Key arguments\n",
    "\n",
    "The key public arguments are:\n",
    "\n",
    "- `path`: one Eora file or one local dataset directory;\n",
    "- `multi_region`: use `True` for `Eora26` and `False` for local single-region files;\n",
    "- `table`: mainly relevant for single-region parsing;\n",
    "- `indeces`: optional path to the `Eora26` label files;\n",
    "- `name_convention`: `full_name` or `abbreviation` in the single-region workflow;\n",
    "- `aggregate_trade`: whether to collapse detailed import/export rows in the single-region workflow;\n",
    "- `country`: country selector when a directory contains multiple single-region files;\n",
    "- `price`: optional selector when a single-region directory contains multiple price variants.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fca78507",
   "metadata": {},
   "source": [
    "## `Eora26` versus single-region Eora\n",
    "\n",
    "Use `multi_region=True` only for `Eora26`. This workflow expects the standard `Eora26_<year>_<price>_*.txt` files plus the `labels_*.txt` files.\n",
    "\n",
    "Use `multi_region=False` for the local single-region tables. In that case MARIO resolves one file such as `IO_ITA_2015_BasicPrice.txt` and can usually infer whether it behaves like an `IOT` or a `SUT`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c90a1387",
   "metadata": {},
   "source": [
    "## Label files for `Eora26`\n",
    "\n",
    "For `Eora26`, MARIO needs both:\n",
    "\n",
    "- the numeric files such as `T`, `FD`, `VA`, `Q`, and `QY`;\n",
    "- the label files such as `labels_T.txt`, `labels_FD.txt`, `labels_VA.txt`, and `labels_Q.txt`.\n",
    "\n",
    "If the label files already live next to the dataset files, `indeces=` can be omitted. Otherwise, point `indeces=` to the directory that contains them.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "04f451ae",
   "metadata": {},
   "source": [
    "## Local layout examples\n",
    "\n",
    "There is no dedicated Eora 2 parser yet; current MARIO support is still limited to the Eora 1 style workflows covered here.\n",
    "\n",
    "`Eora26` expects a directory like:\n",
    "\n",
    "```text\n",
    "EORA/\n",
    "├── Eora26_2017_bp/\n",
    "│   ├── Eora26_2017_bp_T.txt\n",
    "│   ├── Eora26_2017_bp_FD.txt\n",
    "│   ├── Eora26_2017_bp_VA.txt\n",
    "│   ├── Eora26_2017_bp_Q.txt\n",
    "│   └── Eora26_2017_bp_QY.txt\n",
    "└── indices/\n",
    "    ├── labels_T.txt\n",
    "    ├── labels_FD.txt\n",
    "    ├── labels_VA.txt\n",
    "    └── labels_Q.txt\n",
    "```\n",
    "\n",
    "For single-region workflows, `path` can point to one file or to one directory of files named like `IO_ITA_2015_BasicPrice.txt`. Use `country=` and `price=` when the directory contains multiple candidates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "213937a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "import mario"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d719e61f",
   "metadata": {},
   "source": [
    "## Parse one `Eora26` directory\n",
    "\n",
    "This is the supported multi-region EORA26 workflow.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7229617f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Parser: reading Eora26 from /Users/lorenzorinaldi/Library/CloudStorage/OneDrive-SharedLibraries-PolitecnicodiMilano/DENG-SESAM - Documenti/c-Research/a-Datasets/_Input Output Databases/EORA/EORA1/Eora26/Eora26_2015_bp.\n",
      "INFO Parser: Eora26 parsed with 189 regions, 26 sectors, 7 factor rows and 2728 extension rows.\n",
      "INFO Metadata: initialized.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "name = Eora26_2015_bp\n",
       "table = IOT\n",
       "scenarios = ['baseline']\n",
       "Factor of production = 7\n",
       "Satellite account = 2728\n",
       "Consumption category = 7\n",
       "Region = 189\n",
       "Sector = 26"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db = mario.parse_eora(\n",
    "    path=\"/path/to/Eora26_2017_bp\",\n",
    "    multi_region=True,\n",
    ")\n",
    "\n",
    "db"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3137774",
   "metadata": {},
   "source": [
    "## Parse one local single-region file\n",
    "\n",
    "This is the non-multi-region Eora workflow. When `path` points to a directory, `country=` selects the file to parse.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "86585b47",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Parser: reading EORA single-region file IO_ITA_2017_BasicPrice.txt.\n",
      "INFO Parser: EORA single-region parsed as SUT with 7 factor rows and 2631 extension rows.\n",
      "INFO Metadata: initialized.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "name = IO_ITA_2017_BasicPrice\n",
       "table = SUT\n",
       "tech_assumption = industry-based\n",
       "scenarios = ['baseline']\n",
       "Activity = 61\n",
       "Commodity = 61\n",
       "Factor of production = 7\n",
       "Satellite account = 2631\n",
       "Consumption category = 7\n",
       "Region = 1"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db = mario.parse_eora(\n",
    "    path=\"/path/to/IO_ITA_2017_BasicPrice.txt\",\n",
    "    multi_region=False,\n",
    ")\n",
    "\n",
    "db"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0aeaf0a1",
   "metadata": {},
   "source": [
    "## Select one price variant in the single-region workflow\n",
    "\n",
    "Use `price=` when the directory contains more than one variant for the same country and year.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2b8df613",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Parser: reading EORA single-region file IO_ITA_2017_BasicPrice.txt.\n",
      "INFO Parser: EORA single-region parsed as SUT with 7 factor rows and 2631 extension rows.\n",
      "INFO Metadata: initialized.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "name = IO_ITA_2017_BasicPrice\n",
       "table = SUT\n",
       "tech_assumption = industry-based\n",
       "scenarios = ['baseline']\n",
       "Activity = 61\n",
       "Commodity = 61\n",
       "Factor of production = 7\n",
       "Satellite account = 2631\n",
       "Consumption category = 7\n",
       "Region = 1"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db = mario.parse_eora(\n",
    "    path=\"/path/to/IO_All_2017\",\n",
    "    multi_region=False,\n",
    "    country=\"ITA\",\n",
    "    price=\"BasicPrice\",\n",
    ")\n",
    "\n",
    "db"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b27c57e3",
   "metadata": {},
   "source": [
    "## Aggregate trade rows in the single-region workflow\n",
    "\n",
    "Use `aggregate_trade=True` when you want imports and exports collapsed into totals instead of preserving bilateral trade labels.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99ce8276",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Parser: reading EORA single-region file IO_ITA_2017_BasicPrice.txt.\n",
      "INFO Parser: EORA single-region parsed as SUT with 7 factor rows and 2631 extension rows.\n",
      "INFO Metadata: initialized.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "name = IO_ITA_2017_BasicPrice\n",
       "table = SUT\n",
       "tech_assumption = industry-based\n",
       "scenarios = ['baseline']\n",
       "Activity = 61\n",
       "Commodity = 61\n",
       "Factor of production = 7\n",
       "Satellite account = 2631\n",
       "Consumption category = 7\n",
       "Region = 1"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db = mario.parse_eora(\n",
    "    path=\"/path/to/IO_All_2017\",\n",
    "    multi_region=False,\n",
    "    country=\"ITA\",\n",
    "    price=\"BasicPrice\",\n",
    "    aggregate_trade=True,\n",
    ")\n",
    "\n",
    "db"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "594e41f6",
   "metadata": {},
   "source": [
    "## Notes and caveats\n",
    "\n",
    "- there is no parser here for the full Eora MRIO release;\n",
    "- multi-region means `Eora26` only;\n",
    "- the `Eora26` parser applies a few consistency repairs during parsing; those notes are stored in metadata;\n",
    "- single-region parsing can infer `IOT` versus `SUT` automatically from the local file structure.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "0ac5e45a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2026-05-31 08:46:49]    Table added into metadata with value equal to SUT.\n",
      "[2026-05-31 08:46:49]    Price added into metadata with value equal to BasicPrice.\n",
      "[2026-05-31 08:46:49]    Source added into metadata with value equal to Eora website @ https://www.worldmrio.com/.\n",
      "[2026-05-31 08:46:49]    Year added into metadata with value equal to 2017.\n",
      "[2026-05-31 08:46:49]    Tech_Assumption added into metadata with value equal to industry-based.\n"
     ]
    }
   ],
   "source": [
    "db.meta_history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "7566714a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Activity',\n",
       " 'Commodity',\n",
       " 'Factor of production',\n",
       " 'Satellite account',\n",
       " 'Consumption category',\n",
       " 'Region']"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db.sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "d18f24a6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Activity':                                                                unit\n",
       "Agriculture, hunting and related service activi...  current 000 US$\n",
       "Forestry, logging and related service activities    current 000 US$\n",
       "Fishing, operating of fish hatcheries and fish ...  current 000 US$\n",
       "Mining of coal and lignite; extraction of peat      current 000 US$\n",
       "Extraction of crude petroleum and natural gas; ...  current 000 US$\n",
       "...                                                             ...\n",
       "Recreational, cultural and sporting activities      current 000 US$\n",
       "Other service activities                            current 000 US$\n",
       "Private households with employed persons            current 000 US$\n",
       "FISIM                                               current 000 US$\n",
       "Re-export                                           current 000 US$\n",
       "\n",
       "[61 rows x 1 columns], 'Commodity':                                                                unit\n",
       "Products of agriculture, hunting and related se...  current 000 US$\n",
       "Products of forestry, logging and related services  current 000 US$\n",
       "Fish and other fishing products; services incid...  current 000 US$\n",
       "Coal and lignite; peat                              current 000 US$\n",
       "Crude petroleum and natural gas; services incid...  current 000 US$\n",
       "...                                                             ...\n",
       "Recreational, cultural and sporting services        current 000 US$\n",
       "Other services                                      current 000 US$\n",
       "Private households with employed persons            current 000 US$\n",
       "FISIM                                               current 000 US$\n",
       "Re-export                                           current 000 US$\n",
       "\n",
       "[61 rows x 1 columns], 'Factor of production':                                              unit\n",
       "Compensation of employees D.1     current 000 US$\n",
       "Taxes on production D.29          current 000 US$\n",
       "Subsidies on production D.39      current 000 US$\n",
       "Net operating surplus B.2n        current 000 US$\n",
       "Net mixed income B.3n             current 000 US$\n",
       "Consumption of fixed capital K.1  current 000 US$\n",
       "Imports                           current 000 US$, 'Satellite account':                                                                                      unit\n",
       "Item                                                                                     \n",
       "Natural Gas (I-ENERGY)                                                                 TJ\n",
       "Coal (I-ENERGY)                                                                        TJ\n",
       "Petroleum (I-ENERGY)                                                                   TJ\n",
       "Nuclear Electricity (I-ENERGY)                                                         TJ\n",
       "Hydroelectric Electricity (I-ENERGY)                                                   TJ\n",
       "...                                                                                   ...\n",
       "IEA - DEPRECATED - GDP using purchasing power p...  IEA GHG emissions (Gg) and energy use\n",
       "IEA - DEPRECATED - GDP using exchange rates (I-...  IEA GHG emissions (Gg) and energy use\n",
       "IEA - DEPRECATED - Population (I-IEArev2-TOTAL)     IEA GHG emissions (Gg) and energy use\n",
       "IEA - DEPRECATED - Total primary energy supply ...  IEA GHG emissions (Gg) and energy use\n",
       "IEA - DEPRECATED - Total primary energy supply ...  IEA GHG emissions (Gg) and energy use\n",
       "\n",
       "[2631 rows x 1 columns]}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db.units"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mario",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
