{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SolasAI Disparity Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quick Start Summary\n",
    "\n",
    "This example serves as an introductory text aiming to teach a user how to measure disparities with the SolasAI library. While the example in this notebook is based on the calculation of a single metric, the Adverse Impact Ratio (AIR), there are numerous other metrics available in SolasAI, as well as a generic metric interface for developing custom disparity metrics. Usage of the other metrics is discussed in other examples.\n",
    "\n",
    "This notebook provides:\n",
    "\n",
    "1. A short background on terms used throughout the SolasAI library.\n",
    "\n",
    "2. An explanation of how to import and call functionality in the SolasAI disparity testing library. \n",
    "\n",
    "3. An example of how to calculate an Adverse Impact Ratio using U.S. Home Mortgage Disclosure Act (HMDA) data.\n",
    "\n",
    "4. An overview of how to make customized or formatted tables and charts using the SolasAI disparity testing library."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Background\n",
    "\n",
    "Before jumping into the code, we provide a short explanation of several terms used in the code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### \"Protected\" and \"Reference\" Groups\n",
    "\n",
    "In SolasAI's current list of curated metrics, there is an assumption that one will test whether some group(s) achieve outcomes that are _at least as good_ as another group's outcomes. The first group(s), identified in SolasAI as `protected_groups`, typically consists of people who have been or are continuing to experience some form of discrimination or disadvantage.  The outcomes of these groups are then compared to those of the `reference_groups`, which typically have not been subject to the same type of discrimination or disadvantages.\n",
    "\n",
    "Outside SolasAI, there are numerous ways that these classifications are made and described. For example, often the word \"protected\" may be replaced with \"minority\", \"disadvantaged\", \"test\", or some other description, while \"reference\" may be replaced with \"majority\", \"advantaged\", \"control\", or some other description. Our use of the \"protected\" and \"reference\" categorization comes from our experience working in litigation and regulatory compliance in the United States. It is not meant to imply any particular value judgement regarding the classifications used.\n",
    "\n",
    "A third attribute, `group_categories` is also used throughout the SolasAI library to delineate between different protected-reference group combinations. For example, `group_categories` might be race, sex, age, medical diagnoses, etc. By way of example, classifications most commonly used by SolasAI's U.S. customers are:\n",
    "\n",
    "<center/>\n",
    "\n",
    "| `group_categories` | `protected_group` | `reference_group` |\n",
    "|--------------------|-------------------|-------------------|\n",
    "| Race               | Black             | White             |\n",
    "| Race               | Hispanic          | White             |\n",
    "| Race               | Asian             | White             |\n",
    "| Sex / Gender       | Female            | Male              |\n",
    "| Age                | Age >= X          | Age < X           |\n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "### Outcomes, Labels, and Segments\n",
    "\n",
    "SolasAI can be used to measure disparities created as the result of automated systems, semi-automated systems, or entirely subjective processes.  Regardless of the use case, when the user calculates disparity, they will need to specify the `outcome` attribute.  In some cases, this will be a binary (i.e., \"Yes\" or \"No\") outcome, such as whether a person was offered a job, loan, or sent a marketing offer.  In other cases, it may be a continuous value, such as a model's probability of loan default, the amount of time it takes for a person to be promoted, or an employee's pay rate.  Of the metrics SolasAI provides, some are appropriate for the binary case (e.g., `adverse_impact_ratio`), while others are appropriate for analyses of continuous values (e.g., `standardized_mean_difference`).\n",
    "\n",
    "When measuring disparities that arise from the use of a model, and when the true outcome is known, a user can specify the `label` attribute.  Certain metrics, such as the `residual_standardized_mean_difference`, require the label to be present because the disparity measurement incorporates the label.\n",
    "\n",
    "Some metrics, including the `segmented_adverse_impact_ratio` perform analyses on subsets of the data and then aggregate the results.  SolasAI refers to these subsets using the `segment` attribute.  Examples of segments might be different store locations, job openings, or job types.  Care should be used when deciding whether to incorporate segmentation into an analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Statistical Significance and Practical Significance\n",
    "\n",
    "SolasAI uses thresholds of statistical significance and \"practical\" significance to determine whether any potential disparities found are sufficiently large to warrant further review.  The SolasAI does not provide guidance as to which standards are appropriate.  What constitute appropriate and reasonable standards may be driven by regulatory and legal requirements, business decisions, or other factors.  We suggest consulting with one's compliance department, legal advisors, or consultants such as [BLDS, LLC](https://www.bldsllc.com/) (the consultancy from which SolasAI was founded) for advice.\n",
    "\n",
    "That said, after one decides what thresholds to use, SolasAI allows users to test whether the disparity metrics exceed those thresholds.  In the case of statistical significance, each metric uses a measurement of statistical significance that is appropriate or commonly used for that metric.  As an example, in the case of the AIR, statistical significance is calculated using either a Fisher's Exact test or a Chi-Squared test (depending on the size of the data).  For the SMD, a t-test is used.  In certain cases, the user can specify either the test itself or certain attributes of the test.  In United States legal and regulatory standards, a two-sided p-value less than 5% (or, equivilently, a one-sided p-value less than 2.5%) is generally considered statistically significant.  SolasAI does not provide guidance on whether this standard is reasonable or appropriate for any particular use case.\n",
    "\n",
    "Importantly, for SolasAI to identify that a result is \"practically significant\", it must _both_ be found to be statistically significant and exceed the practical significance thresholds set.  Most metrics in SolasAI provide the option to specify two thresholds.  The first is a `percent_difference_threshold`.  If a user sets a value greater than zero, then the raw difference in outcomes between the protected and reference groups must exceed a particular value before a result is considered practically significant.  For example, if `percent_difference_threshold = 0.02` for the AIR, and we find that Black applicants receive offers 1% of the time, but White applicants receive them 2.5% of the time, then this disparity will not be significant because the difference in outcomes is only 1.5%, which is less thna the 2% threshold.  \n",
    "\n",
    "The second practical significance metric relates to the value of the metric itself.  For example, when calling the `adverse_impact_ratio`, one sets the `air_threshold` to a particular value.  In the example below, we use `air_threshold=0.80`.  This means that only results that are statistically significantly different from parity _and_ that have AIR values less than 0.80 will be considered practically significant."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Importing the Library and Data for Use in the Analysis\n",
    "\n",
    "Below, we import the `pandas` and `plotly` libraries used to prepare and graph the data.  SolasAI relies on the plotly library for graphing.  The final line of code, `pio.renderers.default = 'svg'` is only necessary because this workbook is hosted in GitHub, which cannot render plotly graphics in their native format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import plotly.io as pio\n",
    "pio.renderers.default = \"svg\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SolasAI's disparity library is imported just like any standard Python package.  Here, we import the library itself, which will be accessed as `sd`.  Within sd, we will also access several other types of functions, including:\n",
    "\n",
    "1. The interface functionality, `sd.ui`, which allows users to do things like create nicely formatted tables.\n",
    "2. The SolasAI constants file, `sd.const`, which allows users to customize numerous settings including column names and plot headings.\n",
    "3. Access to a set of utility functions, `sd.util`, which provide additional useful functionality.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import solas_disparity as sd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Preparation\n",
    "\n",
    "The following code imports a sample of the 2018 Home Mortgage Disclosure Act (HMDA) data.  The HMDA dataset includes information about nearly every home mortgage application in the United States.  This dataset includes information about the mortgage itself, such as the loan term and APR; information about credit characteristics of the borrowers themselves, including the borrowers income and debt-to-income (DTI) ratio; and information about the home being purchased, such as its location and the value of the property.  Importantly, it also includes information about each borrower's race, gender, and ethnicity.  The data we are using is based only for applications where the borrower was approved for the loan."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Low-Priced</th>\n",
       "      <th>Interest Rate</th>\n",
       "      <th>Rate Spread</th>\n",
       "      <th>Loan Amount</th>\n",
       "      <th>Loan-to-Value Ratio</th>\n",
       "      <th>No Intro Rate Period</th>\n",
       "      <th>Intro Rate Period</th>\n",
       "      <th>Property Value</th>\n",
       "      <th>Income</th>\n",
       "      <th>Debt-to-Income Ratio</th>\n",
       "      <th>...</th>\n",
       "      <th>Hawaiian Or Pacific Islander</th>\n",
       "      <th>Hispanic</th>\n",
       "      <th>Non-Hispanic</th>\n",
       "      <th>Male</th>\n",
       "      <th>Female</th>\n",
       "      <th>Age &gt;= 62</th>\n",
       "      <th>Age &lt; 62</th>\n",
       "      <th>Race</th>\n",
       "      <th>Ethnicity</th>\n",
       "      <th>Sex</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>13451</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.04875</td>\n",
       "      <td>0.00596</td>\n",
       "      <td>155000.0</td>\n",
       "      <td>0.9700</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>165000.0</td>\n",
       "      <td>35000.0</td>\n",
       "      <td>0.33</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>White</td>\n",
       "      <td>Hispanic</td>\n",
       "      <td>Male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18248</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.05750</td>\n",
       "      <td>0.01268</td>\n",
       "      <td>305000.0</td>\n",
       "      <td>1.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>295000.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>0.55</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>Unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19610</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.05500</td>\n",
       "      <td>0.01214</td>\n",
       "      <td>485000.0</td>\n",
       "      <td>0.9500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>515000.0</td>\n",
       "      <td>100000.0</td>\n",
       "      <td>0.43</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>White</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td>Unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3339</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.03875</td>\n",
       "      <td>-0.00087</td>\n",
       "      <td>675000.0</td>\n",
       "      <td>1.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>675000.0</td>\n",
       "      <td>190000.0</td>\n",
       "      <td>0.33</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>Black</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td>Unknown</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19675</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.04375</td>\n",
       "      <td>0.00076</td>\n",
       "      <td>275000.0</td>\n",
       "      <td>0.3507</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>775000.0</td>\n",
       "      <td>209000.0</td>\n",
       "      <td>0.25</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>White</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td>Unknown</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 28 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       Low-Priced  Interest Rate  Rate Spread  Loan Amount  \\\n",
       "id                                                           \n",
       "13451         1.0        0.04875      0.00596     155000.0   \n",
       "18248         1.0        0.05750      0.01268     305000.0   \n",
       "19610         1.0        0.05500      0.01214     485000.0   \n",
       "3339          1.0        0.03875     -0.00087     675000.0   \n",
       "19675         1.0        0.04375      0.00076     275000.0   \n",
       "\n",
       "       Loan-to-Value Ratio  No Intro Rate Period  Intro Rate Period  \\\n",
       "id                                                                    \n",
       "13451               0.9700                     1                  0   \n",
       "18248               1.0000                     1                  0   \n",
       "19610               0.9500                     1                  0   \n",
       "3339                1.0000                     1                  0   \n",
       "19675               0.3507                     1                  0   \n",
       "\n",
       "       Property Value    Income  Debt-to-Income Ratio  ...  \\\n",
       "id                                                     ...   \n",
       "13451        165000.0   35000.0                  0.33  ...   \n",
       "18248        295000.0   60000.0                  0.55  ...   \n",
       "19610        515000.0  100000.0                  0.43  ...   \n",
       "3339         675000.0  190000.0                  0.33  ...   \n",
       "19675        775000.0  209000.0                  0.25  ...   \n",
       "\n",
       "       Hawaiian Or Pacific Islander  Hispanic Non-Hispanic Male  Female  \\\n",
       "id                                                                        \n",
       "13451                           0.0       1.0          0.0  1.0     0.0   \n",
       "18248                           NaN       NaN          NaN  NaN     NaN   \n",
       "19610                           0.0       0.0          1.0  NaN     NaN   \n",
       "3339                            0.0       0.0          1.0  NaN     NaN   \n",
       "19675                           0.0       0.0          1.0  NaN     NaN   \n",
       "\n",
       "       Age >= 62  Age < 62     Race     Ethnicity      Sex  \n",
       "id                                                          \n",
       "13451        NaN       NaN    White      Hispanic     Male  \n",
       "18248        NaN       NaN  Unknown       Unknown  Unknown  \n",
       "19610        1.0       0.0    White  Non-Hispanic  Unknown  \n",
       "3339         0.0       1.0    Black  Non-Hispanic  Unknown  \n",
       "19675        0.0       1.0    White  Non-Hispanic  Unknown  \n",
       "\n",
       "[5 rows x 28 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(\"hmda.csv.gz\", index_col=\"id\")\n",
    "df.sample(random_state=161803, n=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We next specify the groups that we will use to test disparities.  In SolasAI, each protected and reference group must be its own variable in the input data.  These variable names are then included in the `protected_group`, `reference_group`, and `group_categories` lists that are used throughout the SolasAI library.\n",
    "\n",
    "For example, if we are going to test for evidence of disparities between Black and White applicants, we must have one variable that identifies whether the person represented by the observation is Black and one variable that identifies whether the person is White.  Importantly, missing values for these characteristics are generally allowed in SolasAI.  \n",
    "\n",
    "While more groups are available for analysis in the HMDA data, we limit the analysis in order to make the output more tractable.  The categorization used in this example is as follows:\n",
    "\n",
    "<center/>\n",
    "\n",
    "| `group_categories` | `protected_group` | `reference_group` |\n",
    "|--------------------|-------------------|-------------------|\n",
    "| Race               | Black             | White             |\n",
    "| Race               | Native American   | White             |\n",
    "| Race               | Asian             | White             |\n",
    "| Sex                | Female            | Male              |\n",
    "| Ethnicity          | Hispanic          | Non-Hispanic      |\n",
    "\n",
    "</center>\n",
    "\n",
    "The three lists must all be the same length, with each element of the list corresponding to the same comparison (e.g., the first element of the lists below have `Black`, `White` and `Race`, meaning that Black applicants are being compared to White applicants, which is a comparison by race. The fifth elements of each list are `Female`, `Male`, and `Sex`, which means that women are being compared to men, and the type of comparison is by Sex).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "protected_groups = [\"Black\", \"Asian\", \"Native American\", \"Hispanic\",     \"Female\"]\n",
    "reference_groups = [\"White\", \"White\", \"White\",           \"Non-Hispanic\", \"Male\"]\n",
    "group_categories = [\"Race\",  \"Race\",  \"Race\",            \"Ethnicity\",    \"Sex\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While not demonstrated in this notebook, a key benefit to the SolasAI library is its ability to calculate disparities on groups where the characteristics are estimated, rather than known.  \n",
    "\n",
    "In this case, each person's probability estimates are put into the fields identifying group membership.  For example, if the estimation procedure finds that a person has a 75% chance of being Black and a 25% chance of being White, then that person's Black and White variables would have values of 0.75 and 0.25, respectively.\n",
    "\n",
    "A common example of this occurs in race and ethnicity estimation, where a person's home address and last name are used to calculate the probability that the person is Black or White (This is known as the Bayesian Improved Surname Geocoding (\"BISG\") method.  See [here](https://github.com/cfpb/proxy-methodology) for more detail).  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculating the Adverse Impact Ratio on Prior Lending Decisions\n",
    "\n",
    "Determining whether there is evidence of discrimination requires, to the extent possible, testing a model or process before it is put into use as well as testing it when it is being used in production (i.e., having an effective monitoring process).  In this example, we focus on how an organization would monitor a process that is already in production.  Here, we use the HMDA data to test whether there is evidence that members of the protected groups were less likely to receive low-priced loans than members of the reference groups.\n",
    "\n",
    "This type of analysis can be performed on subjective decisions, such as loan officer decisions to underwrite a loan, or a manager deciding whom to promote.  It can also be performed on the outcomes of automated decisioning processes, such as the use of a model to screen applicants, give job offers, or some other similar process.  It can also be performed on a model's training or validation datasets prior to the model being used in production.\n",
    "\n",
    "Below, we use the `sd.adverse_impact_ratio` function to calculate the AIR.  More detail about the AIR can be found in the API documentation for <a href=\"../_api/solas_disparity.adverse_impact_ratio.html\">solas_disparity.adverse_impact_ratio</a>."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "air = sd.adverse_impact_ratio(\n",
    "    group_data=df,  # dataset containing the protected and reference group information\n",
    "    protected_groups=protected_groups,\n",
    "    reference_groups=reference_groups,\n",
    "    group_categories=group_categories,\n",
    "    outcome=df[\"Low-Priced\"],\n",
    "    sample_weight=None,\n",
    "    air_threshold=0.80,\n",
    "    percent_difference_threshold=0,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Overview of SolasAI Disparity Objects with the AIR as an Example\n",
    "\n",
    "Before jumping into the results of the analysis, below we discuss common elements of the SolasAI disparity objects."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Disparity Object Output\n",
    "\n",
    "In a notebook, a user can display a formatted summary of results by displaying the results object using standard IPython notebook methods such as referencing the object in the last line of a cell or by calling the `display` function. This summary includes three elements: \n",
    "1. The disparity card, which summarizes information about the inputs and results of the test run.\n",
    "2. A summary table, which prints more detailed results (and is discussed below).\n",
    "3. A Plotly graph of the AIR metric. \n",
    "\n",
    "Two examples are shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "## Disparity Calculation: Adverse Impact Ratio"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┌───────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────┐\n",
       "│ Protected Groups                  │ Black, Asian, Native American, Hispanic, Female                             │\n",
       "│ Reference Groups                  │ White, White, White, Non-Hispanic, Male                                     │\n",
       "│ Group Categories                  │ Race, Race, Race, Ethnicity, Sex                                            │\n",
       "│ AIR Threshold                     │ 0.8                                                                         │\n",
       "│ % Diff Threshold                  │ 0.0                                                                         │\n",
       "│ Affected Groups                   │                                                                             │\n",
       "│ Affected Reference                │                                                                             │\n",
       "│ Affected Categories               │                                                                             │\n",
       "└───────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┌───────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────┐\n",
       "│ Protected Groups                  │ Black, Asian, Native American, Hispanic, Female                             │\n",
       "│ Reference Groups                  │ White, White, White, Non-Hispanic, Male                                     │\n",
       "│ Group Categories                  │ Race, Race, Race, Ethnicity, Sex                                            │\n",
       "│ AIR Threshold                     │ 0.8                                                                         │\n",
       "│ % Diff Threshold                  │ 0.0                                                                         │\n",
       "│ Affected Groups                   │                                                                             │\n",
       "│ Affected Reference                │                                                                             │\n",
       "│ Affected Categories               │                                                                             │\n",
       "└───────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "\\* Percent Missing: Ethnicity: 13.96%, Race: 13.88%, Sex: 46.41%"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## Adverse Impact Ratio Summary Table"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>Group Category</th>\n",
       "      <th>Group</th>\n",
       "      <th>Reference Group</th>\n",
       "      <th>Observations</th>\n",
       "      <th>Percent Missing</th>\n",
       "      <th>Total</th>\n",
       "      <th>Favorable</th>\n",
       "      <th>Percent Favorable</th>\n",
       "      <th>Percent Difference Favorable</th>\n",
       "      <th>AIR</th>\n",
       "      <th>P-Values</th>\n",
       "      <th>Practically Significant</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Black</td>\n",
       "      <td>White</td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.88%</td>\n",
       "      <td>1,337.0</td>\n",
       "      <td>1,065.0</td>\n",
       "      <td>79.66%</td>\n",
       "      <td>11.22%</td>\n",
       "      <td>0.877</td>\n",
       "      <td>0.000</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Asian</td>\n",
       "      <td>White</td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.88%</td>\n",
       "      <td>1,286.0</td>\n",
       "      <td>1,224.0</td>\n",
       "      <td>95.18%</td>\n",
       "      <td>-4.30%</td>\n",
       "      <td>1.047</td>\n",
       "      <td>0.000</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Native American</td>\n",
       "      <td>White</td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.88%</td>\n",
       "      <td>94.0</td>\n",
       "      <td>81.0</td>\n",
       "      <td>86.17%</td>\n",
       "      <td>4.71%</td>\n",
       "      <td>0.948</td>\n",
       "      <td>0.147</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>White</td>\n",
       "      <td></td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.88%</td>\n",
       "      <td>14,461.0</td>\n",
       "      <td>13,142.0</td>\n",
       "      <td>90.88%</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Hispanic</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td>17,207</td>\n",
       "      <td>13.96%</td>\n",
       "      <td>2,032.0</td>\n",
       "      <td>1,593.0</td>\n",
       "      <td>78.40%</td>\n",
       "      <td>13.49%</td>\n",
       "      <td>0.853</td>\n",
       "      <td>0.000</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td></td>\n",
       "      <td>17,207</td>\n",
       "      <td>13.96%</td>\n",
       "      <td>15,175.0</td>\n",
       "      <td>13,943.0</td>\n",
       "      <td>91.88%</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Sex</td>\n",
       "      <td>Female</td>\n",
       "      <td>Male</td>\n",
       "      <td>10,719</td>\n",
       "      <td>46.41%</td>\n",
       "      <td>4,222.0</td>\n",
       "      <td>3,719.0</td>\n",
       "      <td>88.09%</td>\n",
       "      <td>1.28%</td>\n",
       "      <td>0.986</td>\n",
       "      <td>0.043</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Sex</td>\n",
       "      <td>Male</td>\n",
       "      <td></td>\n",
       "      <td>10,719</td>\n",
       "      <td>46.41%</td>\n",
       "      <td>6,497.0</td>\n",
       "      <td>5,806.0</td>\n",
       "      <td>89.36%</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"500\" style=\"\" viewBox=\"0 0 700 500\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"500\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-64b277\"><g class=\"clips\"><clipPath id=\"clip64b277xyplot\" class=\"plotclip\"><rect width=\"476\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip64b277x\"><rect x=\"80\" y=\"0\" width=\"476\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip64b277y\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip64b277xy\"><rect x=\"80\" y=\"100\" width=\"476\" height=\"320\"/></clipPath></g><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"><rect class=\"bg\" x=\"80\" y=\"100\" width=\"476\" height=\"320\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/></g><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"minor-gridlayer\"><g class=\"x\"/><g class=\"y\"/></g><g class=\"gridlayer\"><g class=\"x\"/><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,361.95)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,303.89)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,245.84)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,187.79000000000002)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,129.73)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"><path class=\"yzl zl crisp\" transform=\"translate(0,420)\" d=\"M80,0h476\" style=\"stroke: rgb(0, 0, 0); stroke-opacity: 1; stroke-width: 1px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80,100)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M9.52,320V65.58H85.68V320Z\" clip-path=\"url(#clip64b277xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(47.4359375,79.58)\">0.877</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M104.72,320V16H180.88V320Z\" clip-path=\"url(#clip64b277xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 0.85;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(142.6359375,30)\">1.047</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M199.92,320V44.77H276.08V320Z\" clip-path=\"url(#clip64b277xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 0.7;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(237.8359375,58.77)\">0.948</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M295.12,320V72.34H371.28V320Z\" clip-path=\"url(#clip64b277xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(242, 142, 45); fill-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(333.0359375,86.34)\">0.853</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M390.32,320V33.89H466.48V320Z\" clip-path=\"url(#clip64b277xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(225, 87, 89); fill-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(428.2359375,47.89)\">0.986</text></g></g></g></g><g class=\"scatterlayer mlayer\"><g class=\"trace scatter traceba18b1\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\" clip-path=\"url(#clip64b277xyplot)\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M47.6,87.79L428.4,87.79\" clip-path=\"url(#clip64b277xyplot)\" style=\"vector-effect: none; fill: none; stroke: rgb(0, 0, 0); stroke-opacity: 1; stroke-dasharray: 3px, 3px; stroke-width: 0.25px; opacity: 1;\"/></g><g class=\"points\" clip-path=\"url(#clip64b277xyplot)\"/><g class=\"text\" clip-path=\"url(#clip64b277xyplot)\"/></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><path class=\"xdivider crisp\" transform=\"translate(556,0)\" d=\"M0,421v31.5\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xdivider crisp\" transform=\"translate(460.8,0)\" d=\"M0,421v31.5\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xdivider crisp\" transform=\"translate(365.6,0)\" d=\"M0,421v31.5\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xdivider crisp\" transform=\"translate(80,0)\" d=\"M0,421v31.5\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(127.6,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Black</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(222.8,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Asian</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(318,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Native american</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(413.2,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Hispanic</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(508.4,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Female</text></g><g class=\"xtick2\"><text text-anchor=\"middle\" x=\"0\" y=\"449\" transform=\"translate(222.8,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Race</text></g><g class=\"xtick2\"><text text-anchor=\"middle\" x=\"0\" y=\"449\" transform=\"translate(413.2,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Ethnicity</text></g><g class=\"xtick2\"><text text-anchor=\"middle\" x=\"0\" y=\"449\" transform=\"translate(508.4,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Sex</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,420)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0.000  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,361.95)\">0.200  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,303.89)\">0.400  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,245.84)\">0.600  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,187.79000000000002)\">0.800  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,129.73)\">1.000  </text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"smithlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-64b277\"><g class=\"clips\"/><clipPath id=\"legend64b277\"><rect width=\"122\" height=\"105\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"><path data-index=\"0\" fill-rule=\"evenodd\" d=\"M80,187.78714496009775L556,187.78714496009775\" style=\"opacity: 1; stroke: rgb(0, 0, 0); stroke-opacity: 1; fill: rgb(0, 0, 0); fill-opacity: 0; stroke-dasharray: 3px, 3px; stroke-width: 1.25px;\"/></g></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(565.52,100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" width=\"122\" height=\"105\" x=\"0\" y=\"0\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\"/><g class=\"scrollbox\" transform=\"\" clip-path=\"url(#legend64b277)\"><text class=\"legendtitletext\" text-anchor=\"start\" x=\"2\" y=\"18.2\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Legend</text><g class=\"groups\"><g class=\"traces\" transform=\"translate(0,32.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Race</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"116.703125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,51.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Ethnicity</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(242, 142, 45); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"116.703125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,70.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Sex</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(225, 87, 89); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"116.703125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,89.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">AIR Threshold</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(0, 0, 0); stroke-opacity: 1; stroke-dasharray: 3px, 3px; stroke-width: 0.25px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"116.703125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" x=\"0\" y=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"35\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Adverse Impact Ratio (AIR)</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"318\" y=\"476.3\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Group</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,17.996875000000003,260)\" x=\"17.996875000000003\" y=\"260\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">AIR</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "air"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "## Disparity Calculation: Adverse Impact Ratio"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┌───────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────┐\n",
       "│ Protected Groups                  │ Black, Asian, Native American, Hispanic, Female                             │\n",
       "│ Reference Groups                  │ White, White, White, Non-Hispanic, Male                                     │\n",
       "│ Group Categories                  │ Race, Race, Race, Ethnicity, Sex                                            │\n",
       "│ AIR Threshold                     │ 0.8                                                                         │\n",
       "│ % Diff Threshold                  │ 0.0                                                                         │\n",
       "│ Affected Groups                   │                                                                             │\n",
       "│ Affected Reference                │                                                                             │\n",
       "│ Affected Categories               │                                                                             │\n",
       "└───────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────┘\n",
       "</pre>\n"
      ],
      "text/plain": [
       "┌───────────────────────────────────┬─────────────────────────────────────────────────────────────────────────────┐\n",
       "│ Protected Groups                  │ Black, Asian, Native American, Hispanic, Female                             │\n",
       "│ Reference Groups                  │ White, White, White, Non-Hispanic, Male                                     │\n",
       "│ Group Categories                  │ Race, Race, Race, Ethnicity, Sex                                            │\n",
       "│ AIR Threshold                     │ 0.8                                                                         │\n",
       "│ % Diff Threshold                  │ 0.0                                                                         │\n",
       "│ Affected Groups                   │                                                                             │\n",
       "│ Affected Reference                │                                                                             │\n",
       "│ Affected Categories               │                                                                             │\n",
       "└───────────────────────────────────┴─────────────────────────────────────────────────────────────────────────────┘\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "\\* Percent Missing: Ethnicity: 13.96%, Race: 13.88%, Sex: 46.41%"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "## Adverse Impact Ratio Summary Table"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>Group Category</th>\n",
       "      <th>Group</th>\n",
       "      <th>Reference Group</th>\n",
       "      <th>Observations</th>\n",
       "      <th>Percent Missing</th>\n",
       "      <th>Total</th>\n",
       "      <th>Favorable</th>\n",
       "      <th>Percent Favorable</th>\n",
       "      <th>Percent Difference Favorable</th>\n",
       "      <th>AIR</th>\n",
       "      <th>P-Values</th>\n",
       "      <th>Practically Significant</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Black</td>\n",
       "      <td>White</td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.88%</td>\n",
       "      <td>1,337.0</td>\n",
       "      <td>1,065.0</td>\n",
       "      <td>79.66%</td>\n",
       "      <td>11.22%</td>\n",
       "      <td>0.877</td>\n",
       "      <td>0.000</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Asian</td>\n",
       "      <td>White</td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.88%</td>\n",
       "      <td>1,286.0</td>\n",
       "      <td>1,224.0</td>\n",
       "      <td>95.18%</td>\n",
       "      <td>-4.30%</td>\n",
       "      <td>1.047</td>\n",
       "      <td>0.000</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Native American</td>\n",
       "      <td>White</td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.88%</td>\n",
       "      <td>94.0</td>\n",
       "      <td>81.0</td>\n",
       "      <td>86.17%</td>\n",
       "      <td>4.71%</td>\n",
       "      <td>0.948</td>\n",
       "      <td>0.147</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>White</td>\n",
       "      <td></td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.88%</td>\n",
       "      <td>14,461.0</td>\n",
       "      <td>13,142.0</td>\n",
       "      <td>90.88%</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Hispanic</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td>17,207</td>\n",
       "      <td>13.96%</td>\n",
       "      <td>2,032.0</td>\n",
       "      <td>1,593.0</td>\n",
       "      <td>78.40%</td>\n",
       "      <td>13.49%</td>\n",
       "      <td>0.853</td>\n",
       "      <td>0.000</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td></td>\n",
       "      <td>17,207</td>\n",
       "      <td>13.96%</td>\n",
       "      <td>15,175.0</td>\n",
       "      <td>13,943.0</td>\n",
       "      <td>91.88%</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Sex</td>\n",
       "      <td>Female</td>\n",
       "      <td>Male</td>\n",
       "      <td>10,719</td>\n",
       "      <td>46.41%</td>\n",
       "      <td>4,222.0</td>\n",
       "      <td>3,719.0</td>\n",
       "      <td>88.09%</td>\n",
       "      <td>1.28%</td>\n",
       "      <td>0.986</td>\n",
       "      <td>0.043</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Sex</td>\n",
       "      <td>Male</td>\n",
       "      <td></td>\n",
       "      <td>10,719</td>\n",
       "      <td>46.41%</td>\n",
       "      <td>6,497.0</td>\n",
       "      <td>5,806.0</td>\n",
       "      <td>89.36%</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"500\" style=\"\" viewBox=\"0 0 700 500\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"500\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-148166\"><g class=\"clips\"><clipPath id=\"clip148166xyplot\" class=\"plotclip\"><rect width=\"476\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip148166x\"><rect x=\"80\" y=\"0\" width=\"476\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip148166y\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip148166xy\"><rect x=\"80\" y=\"100\" width=\"476\" height=\"320\"/></clipPath></g><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"><rect class=\"bg\" x=\"80\" y=\"100\" width=\"476\" height=\"320\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/></g><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"minor-gridlayer\"><g class=\"x\"/><g class=\"y\"/></g><g class=\"gridlayer\"><g class=\"x\"/><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,361.95)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,303.89)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,245.84)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,187.79000000000002)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,129.73)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"><path class=\"yzl zl crisp\" transform=\"translate(0,420)\" d=\"M80,0h476\" style=\"stroke: rgb(0, 0, 0); stroke-opacity: 1; stroke-width: 1px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80,100)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M9.52,320V65.58H85.68V320Z\" clip-path=\"url(#clip148166xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(47.4359375,79.58)\">0.877</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M104.72,320V16H180.88V320Z\" clip-path=\"url(#clip148166xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 0.85;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(142.6359375,30)\">1.047</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M199.92,320V44.77H276.08V320Z\" clip-path=\"url(#clip148166xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 0.7;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(237.8359375,58.77)\">0.948</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M295.12,320V72.34H371.28V320Z\" clip-path=\"url(#clip148166xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(242, 142, 45); fill-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(333.0359375,86.34)\">0.853</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M390.32,320V33.89H466.48V320Z\" clip-path=\"url(#clip148166xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(225, 87, 89); fill-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(428.2359375,47.89)\">0.986</text></g></g></g></g><g class=\"scatterlayer mlayer\"><g class=\"trace scatter trace0bcf96\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\" clip-path=\"url(#clip148166xyplot)\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M47.6,87.79L428.4,87.79\" clip-path=\"url(#clip148166xyplot)\" style=\"vector-effect: none; fill: none; stroke: rgb(0, 0, 0); stroke-opacity: 1; stroke-dasharray: 3px, 3px; stroke-width: 0.25px; opacity: 1;\"/></g><g class=\"points\" clip-path=\"url(#clip148166xyplot)\"/><g class=\"text\" clip-path=\"url(#clip148166xyplot)\"/></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><path class=\"xdivider crisp\" transform=\"translate(556,0)\" d=\"M0,421v31.5\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xdivider crisp\" transform=\"translate(460.8,0)\" d=\"M0,421v31.5\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xdivider crisp\" transform=\"translate(365.6,0)\" d=\"M0,421v31.5\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xdivider crisp\" transform=\"translate(80,0)\" d=\"M0,421v31.5\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(127.6,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Black</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(222.8,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Asian</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(318,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Native american</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(413.2,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Hispanic</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(508.4,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Female</text></g><g class=\"xtick2\"><text text-anchor=\"middle\" x=\"0\" y=\"449\" transform=\"translate(222.8,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Race</text></g><g class=\"xtick2\"><text text-anchor=\"middle\" x=\"0\" y=\"449\" transform=\"translate(413.2,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Ethnicity</text></g><g class=\"xtick2\"><text text-anchor=\"middle\" x=\"0\" y=\"449\" transform=\"translate(508.4,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Sex</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,420)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0.000  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,361.95)\">0.200  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,303.89)\">0.400  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,245.84)\">0.600  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,187.79000000000002)\">0.800  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,129.73)\">1.000  </text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"smithlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-148166\"><g class=\"clips\"/><clipPath id=\"legend148166\"><rect width=\"122\" height=\"105\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"><path data-index=\"0\" fill-rule=\"evenodd\" d=\"M80,187.78714496009775L556,187.78714496009775\" style=\"opacity: 1; stroke: rgb(0, 0, 0); stroke-opacity: 1; fill: rgb(0, 0, 0); fill-opacity: 0; stroke-dasharray: 3px, 3px; stroke-width: 1.25px;\"/></g></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(565.52,100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" width=\"122\" height=\"105\" x=\"0\" y=\"0\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\"/><g class=\"scrollbox\" transform=\"\" clip-path=\"url(#legend148166)\"><text class=\"legendtitletext\" text-anchor=\"start\" x=\"2\" y=\"18.2\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Legend</text><g class=\"groups\"><g class=\"traces\" transform=\"translate(0,32.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Race</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"116.703125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,51.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Ethnicity</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(242, 142, 45); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"116.703125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,70.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Sex</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(225, 87, 89); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"116.703125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,89.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">AIR Threshold</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(0, 0, 0); stroke-opacity: 1; stroke-dasharray: 3px, 3px; stroke-width: 0.25px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"116.703125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" x=\"0\" y=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"35\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Adverse Impact Ratio (AIR)</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"318\" y=\"476.3\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Group</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,17.996875000000003,260)\" x=\"17.996875000000003\" y=\"260\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">AIR</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython.display import display\n",
    "display(air)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Disparity Summary Table Output\n",
    "\n",
    "Nearly all the important information about the results of the analysis is contained in the `summary_table`.  The discussion below describes how to access, format, and graph the information in the `summary_table`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The table can be accessed as a pandas DataFrame by referencing the `summary_table` attribute of the results object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Group Category</th>\n",
       "      <th>Group</th>\n",
       "      <th>Reference Group</th>\n",
       "      <th>Observations</th>\n",
       "      <th>Percent Missing</th>\n",
       "      <th>Total</th>\n",
       "      <th>Favorable</th>\n",
       "      <th>Percent Favorable</th>\n",
       "      <th>Percent Difference Favorable</th>\n",
       "      <th>AIR</th>\n",
       "      <th>P-Values</th>\n",
       "      <th>Practically Significant</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Race</td>\n",
       "      <td>Black</td>\n",
       "      <td>White</td>\n",
       "      <td>17224</td>\n",
       "      <td>0.13880</td>\n",
       "      <td>1337.0</td>\n",
       "      <td>1065.0</td>\n",
       "      <td>0.796559</td>\n",
       "      <td>0.112230</td>\n",
       "      <td>0.876506</td>\n",
       "      <td>1.242621e-38</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Race</td>\n",
       "      <td>Asian</td>\n",
       "      <td>White</td>\n",
       "      <td>17224</td>\n",
       "      <td>0.13880</td>\n",
       "      <td>1286.0</td>\n",
       "      <td>1224.0</td>\n",
       "      <td>0.951788</td>\n",
       "      <td>-0.042999</td>\n",
       "      <td>1.047315</td>\n",
       "      <td>2.389532e-08</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Race</td>\n",
       "      <td>Native American</td>\n",
       "      <td>White</td>\n",
       "      <td>17224</td>\n",
       "      <td>0.13880</td>\n",
       "      <td>94.0</td>\n",
       "      <td>81.0</td>\n",
       "      <td>0.861702</td>\n",
       "      <td>0.047087</td>\n",
       "      <td>0.948187</td>\n",
       "      <td>1.467790e-01</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Race</td>\n",
       "      <td>White</td>\n",
       "      <td></td>\n",
       "      <td>17224</td>\n",
       "      <td>0.13880</td>\n",
       "      <td>14461.0</td>\n",
       "      <td>13142.0</td>\n",
       "      <td>0.908789</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Hispanic</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td>17207</td>\n",
       "      <td>0.13965</td>\n",
       "      <td>2032.0</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>0.783957</td>\n",
       "      <td>0.134857</td>\n",
       "      <td>0.853227</td>\n",
       "      <td>1.726213e-82</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td></td>\n",
       "      <td>17207</td>\n",
       "      <td>0.13965</td>\n",
       "      <td>15175.0</td>\n",
       "      <td>13943.0</td>\n",
       "      <td>0.918814</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Sex</td>\n",
       "      <td>Female</td>\n",
       "      <td>Male</td>\n",
       "      <td>10719</td>\n",
       "      <td>0.46405</td>\n",
       "      <td>4222.0</td>\n",
       "      <td>3719.0</td>\n",
       "      <td>0.880862</td>\n",
       "      <td>0.012781</td>\n",
       "      <td>0.985698</td>\n",
       "      <td>4.300645e-02</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Sex</td>\n",
       "      <td>Male</td>\n",
       "      <td></td>\n",
       "      <td>10719</td>\n",
       "      <td>0.46405</td>\n",
       "      <td>6497.0</td>\n",
       "      <td>5806.0</td>\n",
       "      <td>0.893643</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Group Category            Group Reference Group  Observations  \\\n",
       "0           Race            Black           White         17224   \n",
       "1           Race            Asian           White         17224   \n",
       "2           Race  Native American           White         17224   \n",
       "3           Race            White                         17224   \n",
       "4      Ethnicity         Hispanic    Non-Hispanic         17207   \n",
       "5      Ethnicity     Non-Hispanic                         17207   \n",
       "6            Sex           Female            Male         10719   \n",
       "7            Sex             Male                         10719   \n",
       "\n",
       "   Percent Missing    Total  Favorable  Percent Favorable  \\\n",
       "0          0.13880   1337.0     1065.0           0.796559   \n",
       "1          0.13880   1286.0     1224.0           0.951788   \n",
       "2          0.13880     94.0       81.0           0.861702   \n",
       "3          0.13880  14461.0    13142.0           0.908789   \n",
       "4          0.13965   2032.0     1593.0           0.783957   \n",
       "5          0.13965  15175.0    13943.0           0.918814   \n",
       "6          0.46405   4222.0     3719.0           0.880862   \n",
       "7          0.46405   6497.0     5806.0           0.893643   \n",
       "\n",
       "   Percent Difference Favorable       AIR      P-Values  \\\n",
       "0                      0.112230  0.876506  1.242621e-38   \n",
       "1                     -0.042999  1.047315  2.389532e-08   \n",
       "2                      0.047087  0.948187  1.467790e-01   \n",
       "3                           NaN       NaN           NaN   \n",
       "4                      0.134857  0.853227  1.726213e-82   \n",
       "5                           NaN       NaN           NaN   \n",
       "6                      0.012781  0.985698  4.300645e-02   \n",
       "7                           NaN       NaN           NaN   \n",
       "\n",
       "  Practically Significant  \n",
       "0                      No  \n",
       "1                      No  \n",
       "2                      No  \n",
       "3                          \n",
       "4                      No  \n",
       "5                          \n",
       "6                      No  \n",
       "7                          "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "air.summary_table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It can also be viewed as a styled Pandas dataframe by using the SolasAI `sd.ui.show` command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>Group Category</th>\n",
       "      <th>Group</th>\n",
       "      <th>Reference Group</th>\n",
       "      <th>Observations</th>\n",
       "      <th>Percent Missing</th>\n",
       "      <th>Total</th>\n",
       "      <th>Favorable</th>\n",
       "      <th>Percent Favorable</th>\n",
       "      <th>Percent Difference Favorable</th>\n",
       "      <th>AIR</th>\n",
       "      <th>P-Values</th>\n",
       "      <th>Practically Significant</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Black</td>\n",
       "      <td>White</td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.88%</td>\n",
       "      <td>1,337.0</td>\n",
       "      <td>1,065.0</td>\n",
       "      <td>79.66%</td>\n",
       "      <td>11.22%</td>\n",
       "      <td>0.877</td>\n",
       "      <td>0.000</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Asian</td>\n",
       "      <td>White</td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.88%</td>\n",
       "      <td>1,286.0</td>\n",
       "      <td>1,224.0</td>\n",
       "      <td>95.18%</td>\n",
       "      <td>-4.30%</td>\n",
       "      <td>1.047</td>\n",
       "      <td>0.000</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Native American</td>\n",
       "      <td>White</td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.88%</td>\n",
       "      <td>94.0</td>\n",
       "      <td>81.0</td>\n",
       "      <td>86.17%</td>\n",
       "      <td>4.71%</td>\n",
       "      <td>0.948</td>\n",
       "      <td>0.147</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>White</td>\n",
       "      <td></td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.88%</td>\n",
       "      <td>14,461.0</td>\n",
       "      <td>13,142.0</td>\n",
       "      <td>90.88%</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Hispanic</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td>17,207</td>\n",
       "      <td>13.96%</td>\n",
       "      <td>2,032.0</td>\n",
       "      <td>1,593.0</td>\n",
       "      <td>78.40%</td>\n",
       "      <td>13.49%</td>\n",
       "      <td>0.853</td>\n",
       "      <td>0.000</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td></td>\n",
       "      <td>17,207</td>\n",
       "      <td>13.96%</td>\n",
       "      <td>15,175.0</td>\n",
       "      <td>13,943.0</td>\n",
       "      <td>91.88%</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Sex</td>\n",
       "      <td>Female</td>\n",
       "      <td>Male</td>\n",
       "      <td>10,719</td>\n",
       "      <td>46.41%</td>\n",
       "      <td>4,222.0</td>\n",
       "      <td>3,719.0</td>\n",
       "      <td>88.09%</td>\n",
       "      <td>1.28%</td>\n",
       "      <td>0.986</td>\n",
       "      <td>0.043</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Sex</td>\n",
       "      <td>Male</td>\n",
       "      <td></td>\n",
       "      <td>10,719</td>\n",
       "      <td>46.41%</td>\n",
       "      <td>6,497.0</td>\n",
       "      <td>5,806.0</td>\n",
       "      <td>89.36%</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sd.ui.show(air.summary_table)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Individual columns can also be obtained from the disparity object by requesting the particular column from the object. For example, the `percent_favorable` attribute of the results object accesses the AIR column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Group Category</th>\n",
       "      <th>Group</th>\n",
       "      <th>Reference Group</th>\n",
       "      <th>Percent Favorable</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Race</td>\n",
       "      <td>Black</td>\n",
       "      <td>White</td>\n",
       "      <td>0.796559</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Race</td>\n",
       "      <td>Asian</td>\n",
       "      <td>White</td>\n",
       "      <td>0.951788</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Race</td>\n",
       "      <td>Native American</td>\n",
       "      <td>White</td>\n",
       "      <td>0.861702</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Race</td>\n",
       "      <td>White</td>\n",
       "      <td></td>\n",
       "      <td>0.908789</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Hispanic</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td>0.783957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td></td>\n",
       "      <td>0.918814</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Sex</td>\n",
       "      <td>Female</td>\n",
       "      <td>Male</td>\n",
       "      <td>0.880862</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Sex</td>\n",
       "      <td>Male</td>\n",
       "      <td></td>\n",
       "      <td>0.893643</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Group Category            Group Reference Group  Percent Favorable\n",
       "0           Race            Black           White           0.796559\n",
       "1           Race            Asian           White           0.951788\n",
       "2           Race  Native American           White           0.861702\n",
       "3           Race            White                           0.908789\n",
       "4      Ethnicity         Hispanic    Non-Hispanic           0.783957\n",
       "5      Ethnicity     Non-Hispanic                           0.918814\n",
       "6            Sex           Female            Male           0.880862\n",
       "7            Sex             Male                           0.893643"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "air.percent_favorable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `sd.ui.show` command also supports formatting a subset of any summary table:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>Group Category</th>\n",
       "      <th>Group</th>\n",
       "      <th>Reference Group</th>\n",
       "      <th>Percent Favorable</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Black</td>\n",
       "      <td>White</td>\n",
       "      <td>79.66%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Asian</td>\n",
       "      <td>White</td>\n",
       "      <td>95.18%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Native American</td>\n",
       "      <td>White</td>\n",
       "      <td>86.17%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>White</td>\n",
       "      <td></td>\n",
       "      <td>90.88%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Hispanic</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td>78.40%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td></td>\n",
       "      <td>91.88%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Sex</td>\n",
       "      <td>Female</td>\n",
       "      <td>Male</td>\n",
       "      <td>88.09%</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Sex</td>\n",
       "      <td>Male</td>\n",
       "      <td></td>\n",
       "      <td>89.36%</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sd.ui.show(air.percent_favorable)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The user can generate plots for specific columns of the summary table by using the `plot` method of the results object.  Below, we show examples of plotting the `percent_favorable` and `air` values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"500\" style=\"\" viewBox=\"0 0 700 500\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"500\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-472297\"><g class=\"clips\"><clipPath id=\"clip472297xyplot\" class=\"plotclip\"><rect width=\"401\" height=\"243\"/></clipPath><clipPath class=\"axesclip\" id=\"clip472297x\"><rect x=\"92\" y=\"0\" width=\"401\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip472297y\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"243\"/></clipPath><clipPath class=\"axesclip\" id=\"clip472297xy\"><rect x=\"92\" y=\"100\" width=\"401\" height=\"243\"/></clipPath></g><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"><rect class=\"bg\" x=\"92\" y=\"100\" width=\"401\" height=\"243\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/></g><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"minor-gridlayer\"><g class=\"x\"/><g class=\"y\"/></g><g class=\"gridlayer\"><g class=\"x\"/><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,294.49)\" d=\"M92,0h401\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,245.98)\" d=\"M92,0h401\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,197.47)\" d=\"M92,0h401\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,148.97)\" d=\"M92,0h401\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"><path class=\"yzl zl crisp\" transform=\"translate(0,343)\" d=\"M92,0h401\" style=\"stroke: rgb(0, 0, 0); stroke-opacity: 1; stroke-width: 1px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(92,100)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M5.01,243V49.8H45.11V243Z\" clip-path=\"url(#clip472297xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(24.8959375,77.3234375)rotate(90 0.1640625 -4)\">79.66%</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M55.14,243V12.15H95.24V243Z\" clip-path=\"url(#clip472297xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 0.85;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(75.0259375,39.6734375)rotate(90 0.1640625 -4)\">95.18%</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M105.26,243V34H145.36V243Z\" clip-path=\"url(#clip472297xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 0.7;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(125.1459375,61.5234375)rotate(90 0.1640625 -4)\">86.17%</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M155.39,243V22.58H195.49V243Z\" clip-path=\"url(#clip472297xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 1.5px; fill: rgb(78, 121, 167); fill-opacity: 0.55; stroke: rgb(8, 48, 107); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(175.2759375,50.1034375)rotate(90 0.1640625 -4)\">90.88%</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M205.51,243V52.86H245.61V243Z\" clip-path=\"url(#clip472297xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(242, 142, 45); fill-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(225.3959375,80.3834375)rotate(90 0.1640625 -4)\">78.40%</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M255.64,243V20.15H295.74V243Z\" clip-path=\"url(#clip472297xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 1.5px; fill: rgb(242, 142, 45); fill-opacity: 0.85; stroke: rgb(8, 48, 107); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(275.5259375,47.6734375)rotate(90 0.1640625 -4)\">91.88%</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M305.76,243V29.35H345.86V243Z\" clip-path=\"url(#clip472297xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(225, 87, 89); fill-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(325.6459375,56.8734375)rotate(90 0.1640625 -4)\">88.09%</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M355.89,243V26.25H395.99V243Z\" clip-path=\"url(#clip472297xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 1.5px; fill: rgb(225, 87, 89); fill-opacity: 0.85; stroke: rgb(8, 48, 107); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(375.7759375,53.7734375)rotate(90 0.1640625 -4)\">89.36%</text></g></g></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><path class=\"xdivider crisp\" transform=\"translate(493,0)\" d=\"M0,344v119.8125\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xdivider crisp\" transform=\"translate(392.75,0)\" d=\"M0,344v119.8125\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xdivider crisp\" transform=\"translate(292.5,0)\" d=\"M0,344v119.8125\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xdivider crisp\" transform=\"translate(92,0)\" d=\"M0,344v119.8125\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"356\" transform=\"translate(117.06,0) rotate(90,0,350)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Black</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"356\" transform=\"translate(167.19,0) rotate(90,0,350)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Asian</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"356\" transform=\"translate(217.31,0) rotate(90,0,350)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Native american</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"356\" transform=\"translate(267.44,0) rotate(90,0,350)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">White</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"356\" transform=\"translate(317.56,0) rotate(90,0,350)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Hispanic</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"356\" transform=\"translate(367.69,0) rotate(90,0,350)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Non-hispanic</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"356\" transform=\"translate(417.81,0) rotate(90,0,350)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Female</text></g><g class=\"xtick\"><text text-anchor=\"start\" x=\"0\" y=\"356\" transform=\"translate(467.94,0) rotate(90,0,350)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Male</text></g><g class=\"xtick2\"><text text-anchor=\"middle\" x=\"0\" y=\"460.303125\" transform=\"translate(192.25,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Race</text></g><g class=\"xtick2\"><text text-anchor=\"middle\" x=\"0\" y=\"460.303125\" transform=\"translate(342.63,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Ethnicity</text></g><g class=\"xtick2\"><text text-anchor=\"middle\" x=\"0\" y=\"460.303125\" transform=\"translate(442.88,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Sex</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"91\" y=\"4.199999999999999\" transform=\"translate(0,343)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0.00%  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"91\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,294.49)\">20.00%  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"91\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,245.98)\">40.00%  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"91\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,197.47)\">60.00%  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"91\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,148.97)\">80.00%  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"91\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,100.46)\">100.00%  </text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"smithlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-472297\"><g class=\"clips\"/><clipPath id=\"legend472297\"><rect width=\"187\" height=\"143\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(501.02,100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\" width=\"187\" height=\"143\" x=\"0\" y=\"0\"/><g class=\"scrollbox\" transform=\"\" clip-path=\"url(#legend472297)\"><text class=\"legendtitletext\" text-anchor=\"start\" x=\"2\" y=\"18.2\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Legend</text><g class=\"groups\"><g class=\"traces\" transform=\"translate(0,32.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Race</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"181.0625\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,51.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Race Reference Group</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 1.5px; fill: rgb(78, 121, 167); fill-opacity: 0.55; stroke: rgb(8, 48, 107); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"181.0625\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,70.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Ethnicity</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(242, 142, 45); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"181.0625\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,89.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Ethnicity Reference Group</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 1.5px; fill: rgb(242, 142, 45); fill-opacity: 0.85; stroke: rgb(8, 48, 107); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"181.0625\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,108.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Sex</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(225, 87, 89); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"181.0625\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,127.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Sex Reference Group</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 1.5px; fill: rgb(225, 87, 89); fill-opacity: 0.85; stroke: rgb(8, 48, 107); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"181.0625\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\" x=\"0\" y=\"0\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"35\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Percent Favorable</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"292.5\" y=\"487.6125\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Group</text></g><g class=\"g-ytitle\" transform=\"translate(0.3466796875,0)\"><text class=\"ytitle\" transform=\"rotate(-90,12.653125000000003,221.5)\" x=\"12.653125000000003\" y=\"221.5\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Percent Favorable</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "air.plot(column=\"percent_favorable\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"500\" style=\"\" viewBox=\"0 0 700 500\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"500\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-8e1af7\"><g class=\"clips\"><clipPath id=\"clip8e1af7xyplot\" class=\"plotclip\"><rect width=\"476\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip8e1af7x\"><rect x=\"80\" y=\"0\" width=\"476\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip8e1af7y\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip8e1af7xy\"><rect x=\"80\" y=\"100\" width=\"476\" height=\"320\"/></clipPath></g><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"><rect class=\"bg\" x=\"80\" y=\"100\" width=\"476\" height=\"320\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/></g><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"minor-gridlayer\"><g class=\"x\"/><g class=\"y\"/></g><g class=\"gridlayer\"><g class=\"x\"/><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,361.95)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,303.89)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,245.84)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,187.79000000000002)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,129.73)\" d=\"M80,0h476\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"><path class=\"yzl zl crisp\" transform=\"translate(0,420)\" d=\"M80,0h476\" style=\"stroke: rgb(0, 0, 0); stroke-opacity: 1; stroke-width: 1px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80,100)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M9.52,320V65.58H85.68V320Z\" clip-path=\"url(#clip8e1af7xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(47.4359375,79.58)\">0.877</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M104.72,320V16H180.88V320Z\" clip-path=\"url(#clip8e1af7xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 0.85;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(142.6359375,30)\">1.047</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M199.92,320V44.77H276.08V320Z\" clip-path=\"url(#clip8e1af7xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 0.7;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(237.8359375,58.77)\">0.948</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M295.12,320V72.34H371.28V320Z\" clip-path=\"url(#clip8e1af7xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(242, 142, 45); fill-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(333.0359375,86.34)\">0.853</text></g></g></g><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M390.32,320V33.89H466.48V320Z\" clip-path=\"url(#clip8e1af7xyplot)\" style=\"vector-effect: none; opacity: 1; stroke-width: 0px; fill: rgb(225, 87, 89); fill-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(0, 0, 0); fill-opacity: 1; white-space: pre;\" transform=\"translate(428.2359375,47.89)\">0.986</text></g></g></g></g><g class=\"scatterlayer mlayer\"><g class=\"trace scatter trace208cc1\" style=\"stroke-miterlimit: 2; opacity: 1;\"><g class=\"fills\" clip-path=\"url(#clip8e1af7xyplot)\"/><g class=\"errorbars\"/><g class=\"lines\"><path class=\"js-line\" d=\"M47.6,87.79L428.4,87.79\" clip-path=\"url(#clip8e1af7xyplot)\" style=\"vector-effect: none; fill: none; stroke: rgb(0, 0, 0); stroke-opacity: 1; stroke-dasharray: 3px, 3px; stroke-width: 0.25px; opacity: 1;\"/></g><g class=\"points\" clip-path=\"url(#clip8e1af7xyplot)\"/><g class=\"text\" clip-path=\"url(#clip8e1af7xyplot)\"/></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><path class=\"xdivider crisp\" transform=\"translate(556,0)\" d=\"M0,421v31.5\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xdivider crisp\" transform=\"translate(460.8,0)\" d=\"M0,421v31.5\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xdivider crisp\" transform=\"translate(365.6,0)\" d=\"M0,421v31.5\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xdivider crisp\" transform=\"translate(80,0)\" d=\"M0,421v31.5\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; stroke-width: 1px;\"/><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(127.6,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Black</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(222.8,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Asian</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(318,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Native american</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(413.2,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Hispanic</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(508.4,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Female</text></g><g class=\"xtick2\"><text text-anchor=\"middle\" x=\"0\" y=\"449\" transform=\"translate(222.8,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Race</text></g><g class=\"xtick2\"><text text-anchor=\"middle\" x=\"0\" y=\"449\" transform=\"translate(413.2,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Ethnicity</text></g><g class=\"xtick2\"><text text-anchor=\"middle\" x=\"0\" y=\"449\" transform=\"translate(508.4,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">Sex</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,420)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0.000  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,361.95)\">0.200  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,303.89)\">0.400  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,245.84)\">0.600  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,187.79000000000002)\">0.800  </text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,129.73)\">1.000  </text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"smithlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-8e1af7\"><g class=\"clips\"/><clipPath id=\"legend8e1af7\"><rect width=\"122\" height=\"105\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"><path data-index=\"0\" fill-rule=\"evenodd\" d=\"M80,187.78714496009775L556,187.78714496009775\" style=\"opacity: 1; stroke: rgb(0, 0, 0); stroke-opacity: 1; fill: rgb(0, 0, 0); fill-opacity: 0; stroke-dasharray: 3px, 3px; stroke-width: 1.25px;\"/></g></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(565.52,100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" width=\"122\" height=\"105\" x=\"0\" y=\"0\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\"/><g class=\"scrollbox\" transform=\"\" clip-path=\"url(#legend8e1af7)\"><text class=\"legendtitletext\" text-anchor=\"start\" x=\"2\" y=\"18.2\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Legend</text><g class=\"groups\"><g class=\"traces\" transform=\"translate(0,32.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Race</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(78, 121, 167); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"116.703125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,51.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Ethnicity</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(242, 142, 45); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"116.703125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,70.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Sex</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0px; fill: rgb(225, 87, 89); fill-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"116.703125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,89.7)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">AIR Threshold</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"><path class=\"js-line\" d=\"M5,0h30\" style=\"fill: none; stroke: rgb(0, 0, 0); stroke-opacity: 1; stroke-dasharray: 3px, 3px; stroke-width: 0.25px;\"/></g><g class=\"legendsymbols\"><g class=\"legendpoints\"/></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"116.703125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" x=\"0\" y=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"35\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Adverse Impact Ratio (AIR)</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"318\" y=\"476.3\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Group</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,17.996875000000003,260)\" x=\"17.996875000000003\" y=\"260\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">AIR</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "air.plot(column=\"air\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Results can also be exported. Examples of these commands are shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "air.to_csv(file_path=\".output/air_summary_table.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "air.to_excel(file_path=\".output/air_summary_table.xlsx\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "air.to_image(file_path=\".output/air_plot.svg\", column=\"percent_favorable\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "air.to_image(file_path=\".output/air_plot.png\", column=\"AIR\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Utility Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`utils.pgrg_ordered` returns a unique and ordered list of protected AND reference groups.  This can be helpful when working with group data or with SolasAI results outside of SolasAI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Black',\n",
       " 'Asian',\n",
       " 'Native American',\n",
       " 'White',\n",
       " 'Hispanic',\n",
       " 'Non-Hispanic',\n",
       " 'Female',\n",
       " 'Male']"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "protected_and_reference_groups = sd.utils.pgrg_ordered(protected_groups=protected_groups, reference_groups=reference_groups)\n",
    "protected_and_reference_groups"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Customizing Output\n",
    "\n",
    "A user can change column names used in all downstream results using the `solas_disparity.const` interface.  This can be very helpful when customizing output for a particular use case.  As an example, a lender might want to make the \"Percent Favorable\" column display as \"Loans Underwritten\" or \"Accepted Applications\", while an employer might want to make the \"Percent Favorable\" column be \"Job Offers\" or \"Promotions.\"  \n",
    "\n",
    "Below is an example of how column names can be customized for the HMDA data use case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "sd.const.FAVORABLE = \"Total Loan Offers\"\n",
    "sd.const.PERCENT_FAVORABLE = \"Loan Offer Percent\"\n",
    "sd.const.OBSERVATIONS = \"Obs. with Data\"\n",
    "sd.const.PERCENT_MISSING = \"Pct Obs. Missing Data\"\n",
    "sd.const.PERCENT_DIFFERENCE_FAVORABLE = \"Offer Percent Difference\"\n",
    "sd.const.AIR_VALUES = \"Adverse Impact Ratio\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Formatting in the table can also be changed, as occurs in the code below.  Here, `sd.ui.AUTO_FORMATTERS` is a dictionary that contains the formats for each of the attributes in the summary table.  When changing the formats, one specifies the key of the dictionary as the attribute from the const file (e.g., \"FAVORABLE\" when changing `sd.const.FAVORABLE`), and the value as desired python formatter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "sd.ui.AUTO_FORMATTERS['TOTAL'] = \"0,.0f\"\n",
    "sd.ui.AUTO_FORMATTERS['FAVORABLE'] = \"0,.0f\"\n",
    "sd.ui.AUTO_FORMATTERS['P_VALUES'] = \"0.1%\"\n",
    "sd.ui.AUTO_FORMATTERS['PERCENT_MISSING'] = \"0.1%\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "const_modified_air = sd.adverse_impact_ratio(\n",
    "    group_data=df,\n",
    "    protected_groups=protected_groups,\n",
    "    reference_groups=reference_groups,\n",
    "    group_categories=group_categories,\n",
    "    outcome=df[\"Low-Priced\"],\n",
    "    sample_weight=None,\n",
    "    air_threshold=0.80,\n",
    "    percent_difference_threshold=0.0,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The summary table with new variable names and formatted values is printed below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>Group Category</th>\n",
       "      <th>Group</th>\n",
       "      <th>Reference Group</th>\n",
       "      <th>Obs. with Data</th>\n",
       "      <th>Pct Obs. Missing Data</th>\n",
       "      <th>Total</th>\n",
       "      <th>Total Loan Offers</th>\n",
       "      <th>Loan Offer Percent</th>\n",
       "      <th>Offer Percent Difference</th>\n",
       "      <th>Adverse Impact Ratio</th>\n",
       "      <th>P-Values</th>\n",
       "      <th>Practically Significant</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Black</td>\n",
       "      <td>White</td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.9%</td>\n",
       "      <td>1,337</td>\n",
       "      <td>1,065</td>\n",
       "      <td>79.66%</td>\n",
       "      <td>11.22%</td>\n",
       "      <td>0.877</td>\n",
       "      <td>0.0%</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Asian</td>\n",
       "      <td>White</td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.9%</td>\n",
       "      <td>1,286</td>\n",
       "      <td>1,224</td>\n",
       "      <td>95.18%</td>\n",
       "      <td>-4.30%</td>\n",
       "      <td>1.047</td>\n",
       "      <td>0.0%</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>Native American</td>\n",
       "      <td>White</td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.9%</td>\n",
       "      <td>94</td>\n",
       "      <td>81</td>\n",
       "      <td>86.17%</td>\n",
       "      <td>4.71%</td>\n",
       "      <td>0.948</td>\n",
       "      <td>14.7%</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Race</td>\n",
       "      <td>White</td>\n",
       "      <td></td>\n",
       "      <td>17,224</td>\n",
       "      <td>13.9%</td>\n",
       "      <td>14,461</td>\n",
       "      <td>13,142</td>\n",
       "      <td>90.88%</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Hispanic</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td>17,207</td>\n",
       "      <td>14.0%</td>\n",
       "      <td>2,032</td>\n",
       "      <td>1,593</td>\n",
       "      <td>78.40%</td>\n",
       "      <td>13.49%</td>\n",
       "      <td>0.853</td>\n",
       "      <td>0.0%</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Ethnicity</td>\n",
       "      <td>Non-Hispanic</td>\n",
       "      <td></td>\n",
       "      <td>17,207</td>\n",
       "      <td>14.0%</td>\n",
       "      <td>15,175</td>\n",
       "      <td>13,943</td>\n",
       "      <td>91.88%</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Sex</td>\n",
       "      <td>Female</td>\n",
       "      <td>Male</td>\n",
       "      <td>10,719</td>\n",
       "      <td>46.4%</td>\n",
       "      <td>4,222</td>\n",
       "      <td>3,719</td>\n",
       "      <td>88.09%</td>\n",
       "      <td>1.28%</td>\n",
       "      <td>0.986</td>\n",
       "      <td>4.3%</td>\n",
       "      <td>No</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Sex</td>\n",
       "      <td>Male</td>\n",
       "      <td></td>\n",
       "      <td>10,719</td>\n",
       "      <td>46.4%</td>\n",
       "      <td>6,497</td>\n",
       "      <td>5,806</td>\n",
       "      <td>89.36%</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sd.ui.show(const_modified_air.summary_table)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.8.13 ('.venv': poetry)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "42910659b5347e2331926e04cec18e1ab8da9dc7db1d3439bb5f49b584c5f534"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}