{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "wiod-01",
   "metadata": {},
   "source": [
    "# WIOD parser walkthrough\n",
    "\n",
    "This notebook is the practical guide for parsing WIOD 2016 workbooks in MARIO.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-02",
   "metadata": {},
   "source": [
    "## What this notebook covers\n",
    "\n",
    "- where to download each WIOD workbook family;\n",
    "- when to pass a direct file path and when to pass a directory;\n",
    "- how `year=` and `country=` disambiguate candidate files;\n",
    "- how `MRIO`, national `IOT`, and national `SUT` workflows differ;\n",
    "- how `add_extensions=` imports `Socio_Economic_Accounts.xlsx`;\n",
    "- why the international `SUT` parser exposes `row_mode=` and why that part should still be treated with caution.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-03",
   "metadata": {},
   "source": [
    "## Relevant source pages\n",
    "\n",
    "- Official WIOD 2016 release page: [GGDC WIOD 2016 release](https://www.rug.nl/ggdc/valuechain/wiod/wiod-2016-release?lang=en)\n",
    "- MRIO IOT, current prices: [Dataverse 199104](https://dataverse.nl/api/access/datafile/199104)\n",
    "- MRIO IOT, previous-year prices: [Dataverse 199102](https://dataverse.nl/api/access/datafile/199102)\n",
    "- MRIO international SUT: [Dataverse 199100](https://dataverse.nl/api/access/datafile/199100)\n",
    "- National IOT bundle: [Dataverse 199099](https://dataverse.nl/api/access/datafile/199099)\n",
    "- National SUT bundle: [Dataverse 199096](https://dataverse.nl/api/access/datafile/199096)\n",
    "- Socio-economic accounts: [Dataverse 199095](https://dataverse.nl/api/access/datafile/199095)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-04",
   "metadata": {},
   "source": [
    "## Main entry point\n",
    "\n",
    "For normal user workflows, the public entry point is:\n",
    "\n",
    "- `mario.parse_wiod(...)`\n",
    "\n",
    "The same function supports:\n",
    "\n",
    "- multiregional `IOT` workbooks;\n",
    "- multiregional `IOT` `_PYP` workbooks;\n",
    "- multiregional international `SUT` workbooks;\n",
    "- national `IOT` workbooks;\n",
    "- national `SUT` workbooks;\n",
    "- optional socio-economic extensions through `add_extensions=`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-05",
   "metadata": {},
   "source": [
    "## Key arguments\n",
    "\n",
    "The key public arguments are:\n",
    "\n",
    "- `path`: one workbook or one directory containing multiple WIOD workbooks;\n",
    "- `table`: choose `\"IOT\"` or `\"SUT\"`;\n",
    "- `year`: used to disambiguate directories and mandatory for national workbooks with multi-year content;\n",
    "- `country`: useful when one directory contains multiple national workbooks;\n",
    "- `add_extensions`: optional path to `Socio_Economic_Accounts.xlsx`;\n",
    "- `row_mode`: only relevant for the international `SUT` workbook.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-06",
   "metadata": {},
   "source": [
    "## Direct file path vs directory path\n",
    "\n",
    "Use a **direct workbook path** when you already know the exact file to parse. This is the simplest case and often means you do not need `year=` or `country=`.\n",
    "\n",
    "Use a **directory path** when you keep several WIOD files together. In that case MARIO scans the directory and then uses `table=`, `year=`, and optionally `country=` to pick the correct file.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bde99edc",
   "metadata": {},
   "source": [
    "## Example local layouts and caveats\n",
    "\n",
    "Typical extracted layout:\n",
    "\n",
    "```text\n",
    "WIOD/\n",
    "├── WIOTS_in_EXCEL/\n",
    "│   └── WIOT2014_Nov16_ROW.xlsb\n",
    "├── WIOTS_PYP_in_EXCEL/\n",
    "│   └── WIOT2014_PYP_Nov16_ROW.xlsb\n",
    "├── NIOTS/\n",
    "│   └── ITA_NIOT_nov16.xlsx\n",
    "├── SUT_national/\n",
    "│   └── ITA_SUT_nov16.xlsx\n",
    "├── SUT_international/\n",
    "│   └── intsut14_nov16.xlsb\n",
    "└── Socio_Economic_Accounts.xlsx\n",
    "```\n",
    "\n",
    "National `IOT` and `SUT` workbooks contain multiple years in one file, so `year=` is mandatory for those inputs. For the international `SUT`, `row_mode=\"external_account\"` is the recommended mode because `ROW` is not published as a complete endogenous economy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "wiod-07",
   "metadata": {},
   "outputs": [],
   "source": [
    "import mario"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-08",
   "metadata": {},
   "source": [
    "## Optional download step\n",
    "\n",
    "MARIO provides dedicated WIOD download helpers instead of one generic downloader for every asset.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "wiod-09",
   "metadata": {},
   "outputs": [],
   "source": [
    "mario.download_wiod2016(\n",
    "    path=\"/path/to/wiod\",\n",
    "    table=\"IOT\",\n",
    ")\n",
    "\n",
    "mario.download_wiod2016_iot_pyp(\"/path/to/wiod_pyp\")\n",
    "mario.download_wiod2016_national_iot(\"/path/to/wiod_national_iot\")\n",
    "mario.download_wiod2016_national_sut(\"/path/to/wiod_national_sut\")\n",
    "mario.download_wiod2016_socioeconomic_accounts(\"/path/to/wiod_extensions\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-10",
   "metadata": {},
   "source": [
    "## Parse one explicit MRIO IOT workbook\n",
    "\n",
    "Use this when you want to point MARIO to one specific WIOD multiregional IOT file.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "wiod-11",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO Parser: reading WIOD workbook WIOT2006_Nov16_ROW.xlsb sheet 2006.\n",
      "INFO Parser: WIOD IOT payload ready with shapes Z=(2464, 2464), Y=(2464, 220), V=(6, 2464).\n",
      "INFO Metadata: initialized.\n"
     ]
    }
   ],
   "source": [
    "db = mario.parse_wiod(\n",
    "    path=\"/path/to/WIOT2014_Nov16_ROW.xlsb\",\n",
    "    table=\"IOT\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "58a83412",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "name = WIOT2006_Nov16_ROW\n",
       "table = IOT\n",
       "scenarios = ['baseline']\n",
       "Factor of production = 6\n",
       "Satellite account = 1\n",
       "Consumption category = 5\n",
       "Region = 44\n",
       "Sector = 56"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-12",
   "metadata": {},
   "source": [
    "## Parse one explicit MRIO IOT workbook in previous-year prices\n",
    "\n",
    "The `_PYP` variant is selected from the filename directly. No extra parser flag is needed.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "wiod-13",
   "metadata": {},
   "outputs": [],
   "source": [
    "db = mario.parse_wiod(\n",
    "    path=\"/path/to/WIOT2014_PYP_Nov16_ROW.xlsb\",\n",
    "    table=\"IOT\",\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-14",
   "metadata": {},
   "source": [
    "## Parse one international SUT workbook\n",
    "\n",
    "This is the source where the treatment of `ROW` matters.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "wiod-15",
   "metadata": {},
   "outputs": [],
   "source": [
    "db = mario.parse_wiod(\n",
    "    path=\"/path/to/intsut14_nov16.xlsb\",\n",
    "    table=\"SUT\",\n",
    "    row_mode=\"external_account\",\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-16",
   "metadata": {},
   "source": [
    "## Parse one national IOT workbook\n",
    "\n",
    "National WIOD `IOT` workbooks contain multiple years in one workbook, so `year=` is required.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "wiod-17",
   "metadata": {},
   "outputs": [],
   "source": [
    "db = mario.parse_wiod(\n",
    "    path=\"/path/to/ITA_NIOT_nov16.xlsx\",\n",
    "    table=\"IOT\",\n",
    "    year=2014,\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-18",
   "metadata": {},
   "source": [
    "## Parse one national SUT workbook\n",
    "\n",
    "National WIOD `SUT` workbooks also require `year=`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "wiod-19",
   "metadata": {},
   "outputs": [],
   "source": [
    "db = mario.parse_wiod(\n",
    "    path=\"/path/to/ITA_SUT_nov16.xlsx\",\n",
    "    table=\"SUT\",\n",
    "    year=2014,\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-20",
   "metadata": {},
   "source": [
    "## Parse from a directory containing multiple WIOD files\n",
    "\n",
    "This is the important case when you keep several WIOD files together. Use `year=` to select the release and `country=` when the directory contains multiple national workbooks.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "wiod-21",
   "metadata": {},
   "outputs": [],
   "source": [
    "db = mario.parse_wiod(\n",
    "    path=\"/path/to/wiod_directory\",\n",
    "    table=\"SUT\",\n",
    "    country=\"ITA\",\n",
    "    year=2014,\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-22",
   "metadata": {},
   "source": [
    "## Add socio-economic extensions\n",
    "\n",
    "When `add_extensions=` is provided, MARIO imports `Socio_Economic_Accounts.xlsx` as satellite extensions.\n",
    "\n",
    "- for `IOT`, they populate `E`;\n",
    "- for `SUT`, they populate `Ea` and keep `Ec` zero-filled.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "wiod-23",
   "metadata": {},
   "outputs": [],
   "source": [
    "db = mario.parse_wiod(\n",
    "    path=\"/path/to/WIOT2014_Nov16_ROW.xlsb\",\n",
    "    table=\"IOT\",\n",
    "    add_extensions=\"/path/to/Socio_Economic_Accounts.xlsx\",\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wiod-26",
   "metadata": {},
   "source": [
    "## Warnings to expect\n",
    "\n",
    "Typical parser warnings or important notes are:\n",
    "\n",
    "- unreadable `.xlsb` files stored as cloud placeholders instead of real local files;\n",
    "- ambiguous directory selections when `year=` or `country=` is missing;\n",
    "- notes in `db.meta_history` when the international `SUT` parser reclassifies `ROW` under `row_mode=\"external_account\"`.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mario",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
