{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ef47ff01",
   "metadata": {},
   "source": [
    "# CEPALSTAT parser walkthrough\n",
    "\n",
    "This notebook is the practical guide for parsing the CEPALSTAT `COU` and `MIP` bundles supported by MARIO.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bcdced6",
   "metadata": {},
   "source": [
    "<div class=\"admonition warning\">\n",
    "<div class=\"admonition-title\">Warning</div>\n",
    "`CEPALSTAT` support in MARIO is currently best treated as an evolving beta interface.\n",
    "<br><br>\n",
    "The parser already covers several important country-specific workbook families, but the source repository still contains many layout edge cases and country exceptions that may not behave perfectly yet.\n",
    "<br><br>\n",
    "Issue reports are therefore especially valuable here: if a country bundle fails or behaves unexpectedly, please open an issue and include the specific country, year, table type, and workbook family when possible.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c275c4d4",
   "metadata": {},
   "source": [
    "## What this notebook covers\n",
    "\n",
    "- where the official CEPALSTAT repository lives;\n",
    "- the difference between `SUT` and `IOT` workflows;\n",
    "- direct-file versus directory parsing;\n",
    "- how `year=`, `country=`, and `iot_mode=` are used;\n",
    "- which CEPALSTAT workbook families are currently supported;\n",
    "- which parser warnings matter in practice.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8c4190a",
   "metadata": {},
   "source": [
    "## Relevant source page\n",
    "\n",
    "- Official CEPALSTAT repository: [Repository of supply and use tables and input-output tables in Latin America and the Caribbean](https://statistics.cepal.org/repository/cou-mip/index.html?lang=en)\n",
    "\n",
    "MARIO does not provide an automatic downloader for this source. The expected workflow is to parse local files that you already downloaded from the repository.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73608c15",
   "metadata": {},
   "source": [
    "## Expected path structure\n",
    "\n",
    "`path` can point either to one downloaded CEPALSTAT bundle archive or to a directory collecting multiple bundle archives.\n",
    "\n",
    "Typical direct-file inputs look like:\n",
    "\n",
    "```text\n",
    "/path/to/COL_COU_2023.zip\n",
    "/path/to/DOM_MIP_2012.zip\n",
    "```\n",
    "\n",
    "When you want MARIO to select from a local directory, a practical layout is:\n",
    "\n",
    "```text\n",
    "cepalstat_directory/\n",
    "|-- ARG_COU_2020.zip\n",
    "|-- BRA_COU_2020.zip\n",
    "|-- COL_COU_2023.zip\n",
    "|-- DOM_MIP_2012.zip\n",
    "`-- CHI_MIP_2020.zip\n",
    "```\n",
    "\n",
    "MARIO does not require every country bundle to follow one identical internal workbook engineering, but it does expect CEPALSTAT-style bundle names and local files that you already downloaded from the repository."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8851f325",
   "metadata": {},
   "source": [
    "## Main entry point\n",
    "\n",
    "For normal user workflows, the public entry point is:\n",
    "\n",
    "- `mario.parse_cepalstat(...)`\n",
    "\n",
    "The same function supports both:\n",
    "\n",
    "- `SUT` bundles (`table=\"SUT\"`);\n",
    "- `IOT` bundles (`table=\"IOT\"`).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3644fd7",
   "metadata": {},
   "source": [
    "## Supported layout families\n",
    "\n",
    "CEPALSTAT is not technically uniform, so MARIO resolves a set of supported families behind the same public API.\n",
    "\n",
    "Current `SUT` support includes:\n",
    "\n",
    "- integrated offer/use workbooks such as Colombia;\n",
    "- two-sheet workbooks such as Argentina;\n",
    "- split offer/demand workbooks such as Brazil;\n",
    "- multi-cuadro workbooks such as Chile.\n",
    "\n",
    "Current `IOT` support includes:\n",
    "\n",
    "- direct matrix workbooks such as Dominican Republic and Guatemala;\n",
    "- cuadro workbooks such as Colombia;\n",
    "- symmetric workbooks such as Argentina;\n",
    "- demand-at-basic-prices workbooks such as Brazil;\n",
    "- matrix workbooks such as Chile.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "87db432d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import mario"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48071773",
   "metadata": {},
   "source": [
    "## Parse one SUT bundle directly\n",
    "\n",
    "Use `year=` when the bundle contains more than one annual workbook or when the workbook itself exposes more than one reference year.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a658ce1f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING Parser: CEPALSTAT Argentina SUT workbook does not expose disaggregated value-added rows. Using the aggregate 'Valor Agregado Bruto pb' row.\n",
      "INFO Parser: CEPALSTAT SUT parsed with 107 activities, 222 commodities and 7 final-demand categories.\n",
      "INFO Metadata: initialized.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "name = CEPALSTAT SUT ARG 2019\n",
       "table = SUT\n",
       "tech_assumption = industry-based\n",
       "scenarios = ['baseline']\n",
       "Activity = 107\n",
       "Commodity = 222\n",
       "Factor of production = 8\n",
       "Satellite account = 1\n",
       "Consumption category = 7\n",
       "Region = 1"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db = mario.parse_cepalstat(\n",
    "    path=\"/path/to/COL_COU_2023.zip\",\n",
    "    table=\"SUT\",\n",
    "    year=2019,\n",
    ")\n",
    "\n",
    "db"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb803521",
   "metadata": {},
   "source": [
    "## Parse one IOT bundle directly\n",
    "\n",
    "Use `iot_mode=` when the bundle exposes both `PxP` and `AxA`, or set `iot_mode=\"auto\"` when the workbook family should decide automatically.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "ea86afaf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Parser: CEPALSTAT IOT parsed with 24 sectors, 7 final-demand categories and 7 factor rows.\n",
      "INFO Metadata: initialized.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "name = CEPALSTAT IOT DOM 2012 PXP\n",
       "table = IOT\n",
       "scenarios = ['baseline']\n",
       "Factor of production = 7\n",
       "Satellite account = 1\n",
       "Consumption category = 7\n",
       "Region = 1\n",
       "Sector = 24"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db = mario.parse_cepalstat(\n",
    "    path=\"/path/to/DOM_MIP_2012.zip\",\n",
    "    table=\"IOT\",\n",
    "    iot_mode=\"pxp\",\n",
    "    calc_all=False,\n",
    ")\n",
    "\n",
    "db"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3503c6b1",
   "metadata": {},
   "source": [
    "## Parse from one directory containing multiple bundles\n",
    "\n",
    "When `path` points to a directory, `country=` and `year=` are the main selectors. This is useful when the local CEPALSTAT folder contains many countries and vintages.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "88628724",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Parser: CEPALSTAT IOT parsed with 124 sectors, 7 final-demand categories and 1 factor rows.\n",
      "INFO Metadata: initialized.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "name = CEPALSTAT IOT ARG 1997 AUTO\n",
       "table = IOT\n",
       "scenarios = ['baseline']\n",
       "Factor of production = 1\n",
       "Satellite account = 1\n",
       "Consumption category = 7\n",
       "Region = 1\n",
       "Sector = 124"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db = mario.parse_cepalstat(\n",
    "    path=\"/path/to/cepalstat_directory\",\n",
    "    table=\"IOT\",\n",
    "    country=\"ARG\",\n",
    "    year=1997,\n",
    "    iot_mode=\"auto\",\n",
    "    calc_all=False,\n",
    ")\n",
    "\n",
    "db"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c37e9e56",
   "metadata": {},
   "source": [
    "## Warnings you should expect\n",
    "\n",
    "Some CEPALSTAT families require controlled fallbacks. Typical examples are:\n",
    "\n",
    "- the offer and use product sets differ slightly, so MARIO keeps the common intersection and warns;\n",
    "- some `SUT` families expose only aggregate value added rather than the full breakdown;\n",
    "- some `IOT` families do not expose explicit factor rows, so MARIO reconstructs aggregate value added residually and warns.\n",
    "\n",
    "These warnings are informative: they mark real differences in the source layout, not parser noise.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fec4c2a",
   "metadata": {},
   "source": [
    "## Practical recommendation\n",
    "\n",
    "For CEPALSTAT, start by validating one country family at a time. The repository is coherent at the metadata level, but the actual workbook engineering differs a lot across countries and vintages. If you are building reproducible ingestion scripts, keep the source file names and years explicit rather than relying on broad directory parsing when avoidable.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mario",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
