
Formulas and Mathematical Detail
================================

Notation
--------

Interest is in recovering the parameter vector from the model

.. math::

   \begin{aligned}
   y_{i} & =\beta^{\prime}x_{i}+\epsilon_{i}\end{aligned}

where :math:`\beta` is :math:`k` by 1, :math:`x_{i}` is a :math:`k` by 1
vector of observable variables and :math:`\epsilon_{i}` is a scalar
error. :math:`x_{i}` can be separated in two types of variables. The
:math:`k_{1}` by 1 set of variables :math:`x_{1i}` are exogenous
regressors in the sense that :math:`E\left[x_{1i}\epsilon_{i}\right]=0`.
The :math:`k_{2}` by 1 variables :math:`x_{2i}` are endogenous. A set of
:math:`p` instruments is available that satisfy the requirements for
validity where :math:`p\geq k_{2}`. The extended model can be written as

.. math::

   \begin{aligned}
   y_{i} & =\underset{\textrm{exogenous}}{\underbrace{\beta_{1}^{\prime}x_{1i}}}+\underset{\textrm{endogenous}}{\underbrace{\beta_{2}^{\prime}x_{2i}}}+\epsilon_{i}\\
   x_{2i} & =\underset{\textrm{exogenous}}{\underbrace{\gamma_{1}^{\prime}z_{1i}}}+\underset{\textrm{instruments}}{\underbrace{\gamma_{2}^{\prime}z_{2i}}}+u_{i}\end{aligned}

The model can be expressed compactly

.. math::

   \begin{aligned}
   Y & =X_{1}\beta_{1}+X_{2}\beta_{2}+\epsilon=X\beta+\epsilon\\
   X_{2} & =Z_{1}\gamma_{1}+Z_{2}\gamma_{2}+u=Z\gamma+u\end{aligned}

The vector of instruments :math:`z_{i}` is :math:`p` by 1. There are
:math:`n` observations for all variables. :math:`k_{c}=1` if the model
contains a constant (either explicit or implicit, i.e., including all
categories of a dummy variable). The constant, if included, is in
:math:`x_{1i}`. :math:`X` is the :math:`n` by :math:`k` matrix of
regressors where row :math:`i` of :math:`X` is :math:`x_{i}^{\prime}`.
:math:`X` can be partitioned into :math:`\left[X_{1}\;X_{2}\right]`.
:math:`Z` is the :math:`n` by :math:`p` matrix of instruments. The
vector :math:`y` is :math:`n` by 1. Projection matrices for :math:`X` is
defined :math:`P_{X}=X\left(X^{\prime}X\right)^{-1}X^{\prime}`. The
projection matrix for :math:`Z` is similarly defined only using
:math:`Z`. The annihilator matrix for :math:`X` is
:math:`M_{X}=I-P_{X}`.

Observation weights, :math:`w_{i}`, are scaled to have a mean of 1. When
a model uses weights, estimation is performed on the weighted variables
:math:`\sqrt{w_{i}}y_{i}`, :math:`\sqrt{w_{i}}x_{i}` and
:math:`\sqrt{w_{i}}z_{i}`, which take the place of :math:`y_{i}`,
:math:`x_{i}` and :math:`z_{i}` in the formulas below. This includes the
covariance estimators, the specification tests and the first-stage
statistics. The parameters, covariance and test statistics of a weighted
model are therefore those of the unweighted model that is estimated
using the weighted variables.

Parameter Estimation
--------------------

Two-stage Least Squares (2SLS)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The 2SLS estimator is

.. math::

   \begin{aligned}
   \hat{\beta}_{2SLS} & =\left(X^{\prime}P_{Z}X\right)^{-1}\left(X^{\prime}P_{Z}y\right)\end{aligned}

Limited Information Maximum Likelihood and k-class Estimators
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The LIML or other k-class estimator is

.. math::

   \begin{aligned}
   \hat{\beta}_{\kappa} & =\left(X^{\prime}\left(I-\kappa M_{Z}\right)X\right)^{-1}\left(X^{\prime}\left(I-\kappa M_{Z}\right)y\right)\end{aligned}

where :math:`\kappa` is the parameter of the class. When
:math:`\kappa=1` the 2SLS estimator is recovered. When :math:`\kappa=0`,
the OLS estimator is recovered. The LIML estimator is recovered when
:math:`\kappa` set to

.. math:: \hat{\kappa}=\min\mathrm{eig\left[\left(W^{\prime}M_{Z}W\right)^{-\frac{1}{2}}\left(W^{\prime}M_{X_{1}}W\right)\left(W^{\prime}M_{Z}W\right)^{-\frac{1}{2}}\right]}

where :math:`W=\left[y\:X_{2}\right]` and :math:`\mathrm{eig}` returns
the eigenvalues.

Generalized Method of Moments (GMM)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The GMM estimator is defined as

.. math::

   \begin{aligned}
   \hat{\beta}_{GMM} & =\left(X^{\prime}ZWZ^{\prime}X\right)^{-1}\left(X^{\prime}ZWZ^{\prime}y\right)\end{aligned}

where :math:`W` is a positive definite weighting matrix.

Continuously Updating Generalized Method of Moments (GMM-CUE)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The continuously updating GMM estimator solves the minimization problem

.. math:: \min_{\beta}n\bar{g}\left(\beta\right)^{\prime}W\left(\beta\right)^{-1}\bar{g}\left(\beta\right)

where
:math:`\bar{g}\left(\beta\right)=n^{-1}\sum_{i=1}^{n}z_{i}\hat{\epsilon}_{i}`
and :math:`\hat{\epsilon}_{i}=y_{i}-x_{i}\beta`. Unlike standard GMM,
the weight matrix, :math:`W` directly depends on the model parameters
:math:`\beta` and so it is not possible to use a closed form estimator.

Jackknife Instrumental Variables (JIVE)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The 2SLS fitted value of a regressor, :math:`\hat{x}_{i}`, the
:math:`i`\ th row of :math:`\hat{X}=P_{Z}X`, is computed from a first
stage that uses observation :math:`i`. This links :math:`\hat{x}_{i}` to
the error :math:`\epsilon_{i}`, which is the source of the bias of 2SLS
towards OLS when there are many instruments. The jackknife IV estimator
(JIVE1) of Angrist, Imbens and Krueger (1999) replaces the fitted values
with leave-one-out predictions. Row :math:`i` of :math:`\tilde{X}` is
the prediction of :math:`x_{i}` from the first-stage regression of
:math:`X` on :math:`Z` that is estimated without observation :math:`i`,

.. math:: \tilde{x}_{i}^{\prime}=z_{i}^{\prime}\left(Z_{(-i)}^{\prime}Z_{(-i)}\right)^{-1}Z_{(-i)}^{\prime}X_{(-i)}

where the subscript :math:`(-i)` indicates that row :math:`i` has been
removed. The first stage does not have to be estimated :math:`n` times,
since

.. math:: \tilde{x}_{i}=\frac{\hat{x}_{i}-h_{i}x_{i}}{1-h_{i}},\qquad h_{i}=z_{i}^{\prime}\left(Z^{\prime}Z\right)^{-1}z_{i}

where :math:`h_{i}` is the leverage of observation :math:`i`, the
:math:`i`\ th diagonal element of :math:`P_{Z}`. The exogenous
regressors are in :math:`Z`, so that they are their own predictions,
:math:`\tilde{x}_{1i}=x_{1i}`, and only the endogenous regressors are
replaced. The JIVE estimator is the IV estimator that uses
:math:`\tilde{X}` as the instruments for :math:`X`,

.. math:: \hat{\beta}_{JIVE}=\left(\tilde{X}^{\prime}X\right)^{-1}\tilde{X}^{\prime}y

which replaces :math:`\hat{X}` with :math:`\tilde{X}` in
:math:`\hat{\beta}_{2SLS}=\left(\hat{X}^{\prime}X\right)^{-1}\hat{X}^{\prime}y`.
The estimator cannot be computed if :math:`h_{i}=1` for an observation.
This happens if the observation is the only one with a non-zero value of
one of the instruments, for example a dummy variable that is 1 for a
single observation.

Basic Statistics
----------------

Let :math:`\hat{\epsilon}=y-X\hat{\beta}`. The residual sum of squares
(RSS) is :math:`\hat{\epsilon}^{\prime}\hat{\epsilon}`, the model sum of
squares (MSS) is :math:`\hat{\beta}^{\prime}X^{\prime}X\hat{\beta}` and
the total sum of squares (TSS) is
:math:`\left(y-k_{c}\bar{y}\right)^{\prime}\left(y-k_{c}\bar{y}\right)^{\prime}`\ where
:math:`\bar{y}` is the sample average of :math:`y`. The model
:math:`R^{2}`\ is defined

.. math::

   \begin{aligned}
   R^{2} & =1-\frac{\hat{\epsilon}^{\prime}\hat{\epsilon}}{\left(y-k_{c}\bar{y}\right)^{\prime}\left(y-k_{c}\bar{y}\right)^{\prime}}=1-\frac{RSS}{TSS}\end{aligned}

and the adjusted :math:`R^{2}` is defined

.. math::

   \begin{aligned}
   \bar{R}^{2} & =1-\left(1-R^{2}\right)\frac{N-k_{c}}{N-k}.\end{aligned}

The :math:`R^{2}` is NaN, with no error or warning, if the total sum of
squares is zero, up to rounding error, since it is not defined then, e.g.,
if the dependent variable does not vary in a model with a constant. The
adjusted :math:`R^{2}` is NaN as well, and it is NaN if :math:`N=k`.

The residual variance is
:math:`s^{2}=n^{-1}\hat{\epsilon}^{\prime}\hat{\epsilon}` unless the
debiased flag is used, in which case a small sample adjusted version is
estimated
:math:`s^{2}=\left(n-k\right)^{-1}\hat{\epsilon}^{\prime}\hat{\epsilon}`.
The model degree of freedom is :math:`k` and the residual degree of
freedom is :math:`n-k`.

The model F-statistic is defined

.. math::

   \begin{aligned}
   F & =\hat{\beta}_{-}^{\prime}\hat{V}_{-}^{-1}\dot{\hat{\beta}_{-}}\end{aligned}

where the notation :math:`\hat{\beta}_{-}` indicates that the constant
is excluded from :math:`\hat{\beta}` and :math:`\hat{V}_{-}` indicates
that the row and column corresponding to a constant are excluded. [1]_
The test statistic is distributed as :math:`\chi_{k-k_{c}}^{2}.` If the
debiased flag is set, then the test statistic :math:`F` is transformed
as :math:`F/\left(k-k_{c}\right)` and a :math:`F_{k-k_{c},n-k}`
distribution is used.

Parameter Covariance Estimation
-------------------------------

Two-stage LS, LIML and k-class estimators
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Four covariance estimators are available. The first is the standard
homoskedastic covariance, defined as

.. math::

   \begin{aligned}
   \hat{\Sigma}=n^{-1}s^{2}\left(\frac{X^{\prime}\left(I-\kappa M_{z}\right)X}{n}\right)^{-1} & =n^{-1}s^{2}\hat{A}.\end{aligned}

Note that this estimator can be expressed as

.. math::

   \begin{aligned}
   \hat{\Sigma}=n^{-1}\hat{A}^{-1}\left\{ s^{2}\hat{A}\right\} \hat{A}^{-1} & =n^{-1}\hat{A}^{-1}\hat{B}\hat{A}^{-1}.\end{aligned}

All estimators take this form and only differ in how the asymptotic
covariance of the scores, :math:`B`, is estimated. For the homoskedastic
covariance estimator, :math:`\hat{B}=s^{2}\hat{A}.` The score covariance
in the heteroskedasticity robust covariance estimator is

.. math::

   \begin{aligned}
   \hat{B} & =n^{-1}\sum_{i=1}^{n}\hat{\epsilon}_{i}^{2}\hat{x}_{i}\hat{x}_{i}^{\prime}=n^{-1}\sum_{i=1}^{n}\hat{\xi}_{i}\hat{\xi}_{i}^{\prime}.\end{aligned}

where :math:`\hat{x_{i}}` is row :math:`i` of :math:`\hat{X}=P_{Z}X` and
:math:`\hat{\xi}_{i}=\hat{\epsilon}_{i}\hat{x}_{i}`.

The kernel covariance estimator is robust to both heteroskedasticity and
autocorrelation and is defined as

.. math::

   \begin{aligned}
   \hat{B} & =\hat{\Gamma}_{0}+\sum_{i=1}^{n-1}K\left(\frac{i}{h}\right)\left(\hat{\Gamma}_{i}+\hat{\Gamma}_{i}^{\prime}\right)\\
   \hat{\Gamma_{j}} & =n^{-1}\sum_{i=j+1}^{n}\hat{\xi}_{i-j}\hat{\xi}_{i}^{\prime}\end{aligned}

where :math:`K\left(\frac{i}{h}\right)` is a kernel weighting function
where :math:`h` is the kernel bandwidth.

The one-way clustered covariance estimator is defined as

.. math::

   \begin{aligned}
   n^{-1}\sum_{j=1}^{g}\left(\sum_{i\in\mathcal{G}_{j}}\hat{\xi}_{i}\right)\left(\sum_{i\in\mathcal{G}_{j}}\hat{\xi}_{i}\right)^{\prime}\end{aligned}

where :math:`\sum_{i\in\mathcal{G}_{j}}\hat{\xi}_{i}` is the sum of the
scores for all members in group :math:`\mathcal{G}_{j}` and :math:`g` is
the number of groups.

If the debiased flag is used to perform a small-sample adjustment, all
estimators except the clustered covariance are rescaled by
:math:`\left(n-k\right)/n`. The clustered covariance is rescaled by
:math:`\left(\left(n-k\right)\left(n-1\right)/n^{2}\right)\left(\left(g-1\right)/g\right)`. [2]_

Standard Errors
~~~~~~~~~~~~~~~

Standard errors are defined as

.. math:: s.e.\left(\hat{\beta}_{j}\right)=\sqrt{e_{j}^{\prime}\hat{\Sigma}e_{j}}

where :math:`e_{j}` is a vector of 0s except in location :math:`j` which
is 1.

T-statistics
~~~~~~~~~~~~

T-statistics test the null :math:`H_{0}:\beta_{j}=0` against a 2-sided
alternative and are defined as

.. math:: z=\frac{\hat{\beta}_{j}}{s.e.\left(\hat{\beta}_{j}\right)}.

P-values
~~~~~~~~

P-values are computes using 2-sided tests,

.. math:: Pr\left(\left|z\right|>Z\right)=2-2\Phi\left(\left|z\right|\right)

If the covariance estimator was debiased, a Student’s t distribution
with :math:`n-k` degrees of freedom is used,

.. math::

   \begin{aligned}
   Pr\left(\left|z\right|>Z\right) & =2-2t_{n-k}\left(\left|z\right|\right)\end{aligned}

where :math:`t_{n-k}\left(\cdot\right)` is the CDF of a Student’s T
distribution.

Confidence Intervals
~~~~~~~~~~~~~~~~~~~~

Confidence intervals are constructed as

.. math:: CI_{i,1-\alpha}=\hat{\beta}_{i}\pm q_{\alpha/2}\times\hat{\sigma}_{\beta_{i}}

where :math:`q_{\alpha/2}` is the :math:`\alpha/2` quantile of a
standard Normal distribution or a Student’s t. The Student’s t is used
when a debiased covariance estimator is used.

Linear Hypothesis Tests
~~~~~~~~~~~~~~~~~~~~~~~

Linear hypothesis tests examine the validity of nulls of the form
:math:`H_{0}:R\beta-r=0` and are implemented using a Wald test statistic

.. math:: W=\left(R\hat{\beta}-r\right)^{\prime}\left[R^{\prime}\hat{\Sigma}R\right]^{-1}\left(R\hat{\beta}-r\right)\sim\chi_{q}^{2}

where :math:`q` is the :math:`rank\left(R\right)` which is usually the
number of rows in :math:`R` . If the debiased flag is used, then
:math:`W/q` is reported and critical and p-values are taken from a
:math:`F_{q,n-k}` distribution.

GMM Covariance estimators
~~~~~~~~~~~~~~~~~~~~~~~~~

GMM covariance depends on the weighting matrix used in estimation and
the assumed covariance of the scores. In most applications these are the
same and so the inefficient form,

.. math:: \hat{\Sigma}=n^{-1}\left(\left(\frac{X'Z}{n}\right)W\left(\frac{Z'X}{n}\right)\right)^{-1}\left(\left(\frac{X'Z}{n}\right)WSW\left(\frac{Z'X}{n}\right)\right)\left(\left(\frac{X'Z}{n}\right)W\left(\frac{Z'X}{n}\right)\right)^{-1}

will collapse to the simpler form

.. math:: \hat{\Sigma}=n^{-1}\left(\left(\frac{X'Z}{n}\right)W\left(\frac{Z'X}{n}\right)\right)^{-1}

when :math:`W=S^{-1}`. When an unadjusted (homoskedastic) covariance is
used,

.. math:: \hat{S}=\tilde{s}^{2}n^{-1}\sum_{j=1}^{n}z_{j}^{\prime}z_{j}

where
:math:`\tilde{s}^{2}=n^{-1}\sum_{i=1}^{n}\left(\epsilon_{i}-\bar{\epsilon}\right)^{2}`
subtracts the mean which may be non-zero if the model is overidentified.
Like previous covariance estimators, if the debiased flag is used,
:math:`n^{-1}` is replaced by :math:`\left(n-k\right)^{-1}`. When a
robust (heteroskedastic) covariance is used, the estimator of :math:`S`
is modified to

.. math:: \hat{S}=n^{-1}\sum_{i=1}^{n}\hat{\epsilon}_{i}^{2}z_{i}^{\prime}z_{i}.

If the debiased flag is used, :math:`n^{-1}` is replaced by
:math:`\left(n-k\right)^{-1}`.

Kernel covariance estimators of :math:`S` take the form

.. math::

   \begin{aligned}
   \hat{S} & =\hat{\Gamma}_{0}+\sum_{i=1}^{n-1}k\left(i/h\right)\left(\hat{\Gamma}_{i}+\hat{\Gamma}_{i}^{\prime}\right)\\
   \hat{\Gamma_{j}} & =n^{-1}\sum_{i=j+1}^{n}\hat{\epsilon}_{i-j}\hat{\epsilon}_{i}z_{i-j}^{\prime}z_{i}\end{aligned}

and :math:`k\left(\cdot\right)` is a kernel weighting function with
bandwidth :math:`h`. If the debiased flag is used, :math:`n^{-1}` is
replaced by :math:`\left(n-k\right)^{-1}`.

The one-way clustered covariance estimator is defined as

.. math:: \hat{S}=n^{-1}\sum_{j=1}^{g}\left(\sum_{i\in\mathcal{G}_{j}}\hat{\epsilon}_{i}z_{i}\right)^{\prime}\left(\sum_{i\in\mathcal{G}_{j}}\hat{\epsilon}_{i}z_{i}\right)

where :math:`\sum_{i\in\mathcal{G}_{j}}\hat{\epsilon}_{i}z_{i}` is the
sum of the moment conditional for all members in group
:math:`\mathcal{G}_{j}` and :math:`g` is the number of groups. If the
debiased flag is used, the :math:`n^{-1}` term is replaced by

.. math:: \left(n-k\right)^{-1}\frac{n-1}{n}\frac{g}{g-1}.

GMM Weight Estimators
~~~~~~~~~~~~~~~~~~~~~

The GMM optimal weight estimators are identical to the the estimators of
:math:`S` with two notable exceptions. First, they are never debiased
and so always use :math:`n^{-1}`. Second, if the center flag is true,
the demeaned moment conditions defined as
:math:`\tilde{g}_{i}=z_{i}\hat{\epsilon}_{i}-\overline{z\epsilon}` are
used in-place of :math:`g_{i}` in the robust, kernel and clustered
estimators. The unadjusted estimator is always centered, and so this
option has no effect.

JIVE Covariance
~~~~~~~~~~~~~~~

JIVE is a just identified IV estimator that solves the moment conditions
:math:`n^{-1}\sum_{i=1}^{n}\tilde{x}_{i}\hat{\epsilon}_{i}=0`, where
:math:`\hat{\epsilon}_{i}=y_{i}-x_{i}^{\prime}\hat{\beta}_{JIVE}` is
computed using :math:`X` and not :math:`\tilde{X}`. Its covariance is

.. math:: \hat{\Sigma}=n^{-1}\left(\frac{\tilde{X}^{\prime}X}{n}\right)^{-1}\hat{S}\left(\frac{X^{\prime}\tilde{X}}{n}\right)^{-1}

where :math:`\hat{S}` is an estimator of the covariance of the moment
conditions :math:`\tilde{x}_{i}\hat{\epsilon}_{i}`. The estimators are
those of the GMM covariance, with :math:`\tilde{x}_{i}` in place of
:math:`z_{i}`. The unadjusted estimator is
:math:`\hat{S}=s^{2}n^{-1}\tilde{X}^{\prime}\tilde{X}` with
:math:`s^{2}=n^{-1}\hat{\epsilon}^{\prime}\hat{\epsilon}`, which is not
centered. The robust estimator is
:math:`\hat{S}=n^{-1}\sum_{i=1}^{n}\hat{\epsilon}_{i}^{2}\tilde{x}_{i}\tilde{x}_{i}^{\prime}`.
The kernel and one-way clustered estimators are defined as for the GMM
covariance. The bandwidth of the kernel estimator is selected using the
data if it is not provided. Two-way clustering is not available. If the
debiased flag is used, :math:`\hat{S}` is rescaled as in the GMM
covariance estimators.

**Inference with many instruments**

These covariance estimators treat :math:`\tilde{X}` as if it were a
fixed set of instruments. Chao, Swanson, Hausman, Newey and Woutersen
(2012) derive the distribution of JIVE when the number of instruments,
:math:`K`, grows with the sample size. JIVE is consistent and
asymptotically normal if :math:`\sqrt{K}/r_{n}\to0`, where :math:`r_{n}`
is the concentration parameter, which measures the strength of the
instruments. The asymptotic variance is the sum of the term that is
estimated above and a second term of order :math:`K/r_{n}`. The second
term depends on the variances of the errors of the first stage and of
the model and on their covariance. It can be ignored if
:math:`K/r_{n}\to0`, but not if :math:`K` is large relative to the
strength of the instruments. The covariance estimators above omit it,
and so they can understate the uncertainty in the latter case. In a
simulation with :math:`n=200`, :math:`K=100` instruments that each have
a first-stage coefficient of 0.05, and a correlation of 0.95 between the
errors of the first stage and of the model, a 95% confidence interval
that used the robust covariance contained the true value in about 85% of
2,000 replications. The variance estimator that includes the second term
is not implemented.

Davidson and MacKinnon (2006) compare two variants of JIVE with 2SLS and
LIML in Monte Carlo experiments. They find that JIVE is always more
dispersed than 2SLS, often very much so, that it is almost always
inferior to LIML, and that it performs particularly badly when the
instruments are weak.

Post-estimation
---------------

A model that uses weights is tested using the weighted variables, as
described in the notation. Sargan’s, Basmann’s, Anderson-Rubin,
Basmann’s F, Durbin’s and the Wu-Hausman tests assume that the errors
are homoskedastic, and so they do not use the covariance estimator that
was used to fit the model. Wooldridge’s score tests of exogeneity and of
the overidentifying restrictions are robust to heteroskedasticity, and
they also do not depend on the covariance estimator. Wooldridge’s
regression test of exogeneity uses the covariance estimator that was
used to fit the model, and so it has the robustness of that estimator.
The J-statistic and the C-statistic of the GMM estimators use the
estimator of the covariance of the moment conditions that is selected by
the weight type, and not the covariance estimator that is selected in
the call to fit.

2SLS and LIML Post-estimation Results
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**Sargan**

Sargan’s test of over-identifying restrictions examines whether the
variance of the IV residuals is similar to that of the OLS residuals.
The test statistic is computed

.. math:: s=n(1-\hat{\epsilon}^{\prime}M_{Z}\hat{\epsilon}/\hat{\epsilon}^{\prime}\hat{\epsilon})\sim\chi_{v}^{2}

where :math:`\hat{\epsilon}` are the IV residuals and :math:`M_{Z}` is
the annihilator matrix using all exogenous variables.\ :math:`\nu` is
the number of overidentifying restrictions, which is the number of
instruments minus the number of endogenous variables, :math:`p-k_{2}`.

**Basmann**

Basmann’s test is a small-sample corrected version of Sargan’s test of
over-identifying restrictions. It has the same distribution. Let
:math:`s` be Sargan’s test statistic, then Basmann’s test statistic is

.. math:: s(n-p)/(n-s)\sim\chi_{v}^{2}

**Wooldridge score**

Wooldridge’s score test of exogeneity examines the magnitude of the
correlation between the OLS residuals and a an appropriately constructed
set of residuals of the instruments. Define :math:`e=M_{X}Y` and
:math:`\nu=M_{X}M_{Z}X_{2}`. Then the test statistic is computed from
the regression

.. math:: 1=\gamma_{1}\hat{\epsilon}_{1}\hat{v}_{1,i}+\ldots+\gamma_{p}\hat{\epsilon}_{1}\hat{v}_{p,i}+\eta_{i}

as :math:`nR^{2}\sim\chi_{k_{2}}^{2}`.

**Wooldridge regression**

Wooldridge’s regression test of exogeneity is closely related to the
score test and is generally more powerful. It also inherits robustness
to heteroskedasticity and/or autocorrelation the comes from the choice
of covariance estimator used in the model. Define :math:`R=M_{Z}X_{2}`.
The test is implemented in a regression of

.. math:: Y=X\beta+R\gamma+\eta

as

.. math:: \hat{\gamma}^{\prime}\hat{\Sigma}_{\gamma}^{-1}\gamma^{\prime}\sim\chi_{k_{2}}^{2}

where :math:`\hat{\Sigma}_{\gamma}` is the block of the covariance
matrix corresponding to the :math:`\gamma` parameters.
:math:`\hat{\Sigma}` is estimated using the same covariance estimator as
the model fit.

**Wooldridge’s Test of Overidentifying restrictions**

Wooldridge’s test is a score test examining whether the component of the
instrument that is uncorrelated with both the included exogenous and the
fitted exogenous is uncorrelated with the IV residuals. Define
:math:`\tilde{Z}=M_{\left[X_{1}\:\hat{X}_{2}\right]}Z_{2,1:q}` where
:math:`\hat{X_{2}}` are the fitted values from the first stage
regression of the endogenous on all exogenous variables and
:math:`Z_{2,1:q}` contains any :math:`q` columns of :math:`Z_{2}`,
:math:`q=p-k_{2}` . The test statistic is computed using a regression of
1s on the test functions :math:`\hat{\epsilon}_{i}\tilde{z}_{i,j}` for
:math:`j=1,\ldots,q` which should have expected value 0.

.. math:: 1=\gamma_{1}\hat{\epsilon}_{i}\tilde{z}_{i,1}+\ldots+\gamma_{q}\hat{\epsilon}_{i}\tilde{z}_{i,q}

The test statistic is :math:`nR^{2}\sim\chi_{q}^{2}` from the
regression.

**Anderson-Rubin**

The Anderson-Rubin test of overidentification examines the magnitude of
the LIML :math:`\hat{\kappa}`\ which should be close to unity when the
model is not overidentified.

.. math:: n\ln(\hat{\kappa})\sim\chi_{q}^{2}

where :math:`q=p-k_{2}`.

**Basmann’s F**

Basmann’s F test of overidentification also examines the magnitude of
the LIML :math:`\hat{\kappa}`. The test statistic is

.. math:: \hat{\kappa}(n-p)/q\sim F_{q,n-n_{instr}}

where :math:`q=p-k_{2}`.

**Durbin and Wu-Hausman**

Durbin’s and the Wu-Hausman tests of exogeneity test of exogeneity is
depends on the variance of the residuals when come endogenous variables
are treated as exogenous against the case where they are treated as
endogenous. Durbin’s test statistic is

.. math::

   \begin{aligned}
   \delta= & \hat{\epsilon}'_{e}P_{[z,w]}\hat{\epsilon}_{e}-\hat{\epsilon}'_{c}P_{z}\hat{\epsilon}_{c}\\
   D= & \delta/(\hat{\epsilon}'_{e}\hat{\epsilon}_{e})/n\sim\chi_{q}^{2}\end{aligned}

and the Wu-Hausman test statistic is

.. math::

   \begin{aligned}
   WH= & \frac{\delta/q}{(\hat{\epsilon}'_{e}\hat{\epsilon}_{e}-\delta)/v}\sim F_{q,\nu}\end{aligned}

where :math:`\hat{\epsilon}_{e}` treats the selected set of endogenous
variables as exogenous (efficient estimate) and
:math:`\hat{\epsilon}_{c}` is a consistent estimator if these variables
are endogenous.\ :math:`P_{\left[Z\,W\right]}` is the projection matrix
containing all exogenous variables including the instrument as well as
the variables being tested for endogeneity
:math:`\left(W\right)`.\ :math:`q` is the number of variables being
tested for exogeneity and :math:`\nu=n-k1-k2-q`.

GMM Post-estimation Results
~~~~~~~~~~~~~~~~~~~~~~~~~~~

**J-stat**

The J-statistic tests whether the model is over-identified, and is
defined

.. math:: n\bar{g}'W^{-1}\bar{g}\sim\chi_{q}^{2}

where :math:`\bar{g}=n^{-1}\sum\hat{\epsilon}_{i}z_{i}` and :math:`W` is
a consistent estimator of the variance of :math:`\sqrt{n}\bar{g}`. The
degree of freedom is :math:`q=p-k_{2}`.

**Centered J-stat**

The estimators of the covariance of the moment conditions that are used
to compute the weight matrix subtract the mean of the moment conditions
only if the center flag is true, and the unadjusted estimator always
does. Let :math:`g_{i}=z_{i}\hat{\epsilon}_{i}`, and let
:math:`W=n^{-1}\sum_{i=1}^{n}g_{i}g_{i}^{\prime}` and
:math:`W_{c}=n^{-1}\sum_{i=1}^{n}\left(g_{i}-\bar{g}\right)\left(g_{i}-\bar{g}\right)^{\prime}`
be the robust estimators that do not center and that center the moment
conditions, so that :math:`W=W_{c}+\bar{g}\bar{g}^{\prime}`. If the
model is correctly specified, the moment conditions have mean zero at
the true parameters and the two estimators are asymptotically
equivalent. If it is not, :math:`\bar{g}` does not converge to zero, and
:math:`W` also contains :math:`\bar{g}\bar{g}^{\prime}`. When both are
evaluated using the same parameters, the Sherman-Morrison formula shows
that the J-statistic and the centered J-statistic,
:math:`J_{c}=n\bar{g}^{\prime}W_{c}^{-1}\bar{g}`, are related by

.. math:: J=n\bar{g}^{\prime}W^{-1}\bar{g}=\frac{J_{c}}{1+J_{c}/n},\qquad J_{c}=\frac{J}{1-J/n}

The J-statistic is therefore smaller than :math:`n` however large the
violation of the moment conditions, while the centered J-statistic is
not bounded. The relationship is exact when the weight matrix is the
robust estimator that does not center, evaluated at the final
parameters, as in iterated GMM that has converged. In two-step GMM the
weight matrix is computed from the first-step parameters, and the
relationship is approximate. The centered J-statistic has the same
asymptotic distribution as the J-statistic when the moment conditions
hold, and it is available as the attribute centered_j_stat of the
results (Hall, 2000; Hansen and Lee, 2021). It uses the estimator of the
covariance that is selected by the weight type of the model, applied to
the centered moment conditions and evaluated at the final parameter
estimates. The cluster sums of centered moment conditions add to zero,
and so a clustered estimate that uses :math:`g` clusters has rank at
most :math:`g-1`. The statistic is not defined if the estimate is
singular, which requires more clusters than moment conditions.

**C-stat**

The C-statistic tests the exogeneity of :math:`m` of the endogenous
variables by treating them as exogenous, so that they are included in
the moment conditions,
:math:`z_{i}^{e}=\left(z_{i}^{\prime},x_{t,i}^{\prime}\right)^{\prime}`,
where :math:`x_{t,i}` are the tested variables. A likelihood-ratio-like
test statistic is computed as

.. math:: C=J_{e}-J_{c}\sim\chi_{m}^{2}

where :math:`J_{e}` is the J-statistic of the model that treats the
tested variables as exogenous and :math:`J_{c}` is the J-statistic of
the model that leaves them as endogenous. Both J-statistics use the same
estimate of the covariance of the moment conditions. Let :math:`S_{e}`
be the estimate of the covariance of :math:`z_{i}^{e}\hat{\epsilon}_{i}`
computed from the first-step residuals of the model that treats the
variables as exogenous. It is computed using the estimator that is
selected by the weight type of the model, for example robust, kernel or
clustered. The weight matrix of that model is :math:`S_{e}^{-1}`, and
:math:`J_{e}` is its J-statistic. The moment conditions of the original
model are a subset of the moment conditions of this model. Let
:math:`S_{e,c}` be the sub-matrix of :math:`S_{e}` that contains the
rows and columns of the original moment conditions. The original model
is estimated using the weight matrix :math:`S_{e,c}^{-1}`, and
:math:`J_{c}` is its J-statistic. Note that :math:`S_{e,c}^{-1}` is the
inverse of the sub-matrix of :math:`S_{e}`, which is not the sub-matrix
of :math:`S_{e}^{-1}`.

Using a common estimate of the covariance ensures that the test
statistic is not negative, since
:math:`g_{e}^{\prime}S_{e}^{-1}g_{e}\geq g_{c}^{\prime}S_{e,c}^{-1}g_{c}`
for any parameters, where :math:`g_{e}` and :math:`g_{c}` are the
average moment conditions of the two models. If the original model is
just identified then :math:`J_{c}=0` and the C-statistic is
:math:`J_{e}`. The statistic is not defined if :math:`S_{e}` is
singular, which requires more clusters than the moment conditions of the
model that treats the variables as exogenous.

First-stage Estimation Analysis
-------------------------------

**Partial R2 and Partial F-statistic**

The :math:`R^{2}` is reported after orthogonalizing the instruments from
included exogenous variables, so that the model estimated is

.. math:: x_{2i}=\gamma_{0}+\tilde{z}_{2i}\gamma+\eta_{i}

where :math:`\tilde{Z}_{2}=M_{X_{1}}\tilde{Z}`. The partial
:math:`F`-statistic tests :math:`H_{0}:\gamma=0` in this regression. It
is the Wald statistic
:math:`W=\hat{\gamma}^{\prime}\hat{V}_{\gamma}^{-1}\hat{\gamma}`, where
:math:`\hat{V}_{\gamma}` is the covariance of :math:`\hat{\gamma}`
estimated using the covariance estimator of the model. If the covariance
is unadjusted, the statistic is reported as :math:`W/p_{2}` with an
:math:`F_{p_{2},n-p}` distribution. This is the classical
:math:`F`-statistic only if the debiased flag is used. Otherwise, the
covariance uses :math:`n^{-1}` in place of :math:`\left(n-p\right)^{-1}`
and the statistic is :math:`n/\left(n-p\right)` times larger. In all
other cases, :math:`W` is reported with an asymptotic
:math:`\chi_{p_{2}}^{2}` distribution. It is not divided by
:math:`p_{2}`, which is how other software reports the
:math:`F`-statistic of a robust test.

**Shea’s R2**

Shea’s :math:`R^{2}` is defined as the ratio of the parameter variances
under OLS and 2SLS estimation standardized by the unexplained variance
under each,

.. math:: \frac{\frac{\hat{\sigma}_{OLS,\beta_{i}}^{2}}{1-R_{OLS}^{2}}}{\frac{\hat{\sigma}_{IV,\beta_{i}}^{2}}{1-R_{IV}^{2}}}=\frac{\hat{\sigma}_{OLS,\beta_{i}}^{2}}{\hat{\sigma}_{IV,\beta_{i}}^{2}}\times\frac{1-R_{IV}^{2}}{1-R_{OLS}^{2}}

If the estimator under 2SLS was as good as under OLS, both ratios would
be 1 and Shea’s :math:`R^{2}=1`. On the other hand, the worse the
:math:`IV` fit in terms of either :math:`R^{2}` or the parameter
variances, the lower the value reported by Shea’s :math:`R^{2}`.

Kernel Weights and Bandwidth Selection
--------------------------------------

**Kernel weights**

In all formulas, :math:`m` is the kernel bandwidth parameter.

-  Bartlett

   .. math:: w_{i}=1-\frac{i}{m+1},\,i<m

-  Parzen

   .. math::

      \begin{aligned}
      z_{i} & =\frac{i}{m+1},\,i<m\\
      w_{i} & =1-6z_{i}^{2}+6z_{i}^{3},z\leq0.5\\
      w_{i} & =2(1-z_{i})^{3},z>0.5\end{aligned}

-  Quadratic-Spectral

   .. math::

      \begin{aligned}
      z_{i} & =\frac{6\pi i}{5m}\\
      w_{0} & =1\\
      w_{i} & =3(\sin(z_{i})/z_{i}-\cos(z_{i}))/z_{i}^{2},\:i\geq1\end{aligned}

**Optimal BW selection**

If the bandwidth of a kernel estimator is not provided, it is selected
using the data with the plug-in method of Newey and West (1994). The
method uses a scalar series, :math:`g_{i}`, that is a sum of the scores
or moment conditions. In the 2SLS and LIML covariances, :math:`g_{i}` is
the sum of :math:`\hat{x}_{ij}\hat{\epsilon}_{i}` over the regressors
other than the constant, where :math:`\hat{x}_{ij}` are the first-stage
fitted values. In the GMM weight matrix and covariance and in the JIVE
covariance, the moment conditions are divided by their standard
deviations before they are summed. The moment conditions are
:math:`z_{ij}\hat{\epsilon}_{i}`, and
:math:`\tilde{x}_{ij}\hat{\epsilon}_{i}` in JIVE. Let

.. math:: \hat{\sigma}_{j}=T^{-1}\sum_{i=j+1}^{T}g_{i}g_{i-j},\qquad\hat{s}^{(0)}=\hat{\sigma}_{0}+2\sum_{j=1}^{m^{*}}\hat{\sigma}_{j},\qquad\hat{s}^{(q)}=2\sum_{j=1}^{m^{*}}j^{q}\hat{\sigma}_{j}

where :math:`T` is the number of observations. The number of
autocovariances, :math:`m^{*}`, and the constants :math:`q` and
:math:`c_{q}` depend on the kernel. For the Bartlett kernel,
:math:`q=1`, :math:`c_{q}=1.1447` and
:math:`m^{*}=\lceil4\left(T/100\right)^{2/9}\rceil`. For the quadratic
spectral kernel, :math:`q=2`, :math:`c_{q}=1.3221` and
:math:`m^{*}=\lceil4\left(T/100\right)^{2/25}\rceil`. For the Parzen
kernel, :math:`q=2`, :math:`c_{q}=2.6614` and
:math:`m^{*}=\lceil4\left(T/100\right)^{4/25}\rceil`. The bandwidth is

.. math:: \hat{m}=\left\lceil c_{q}\left[\left(\frac{\hat{s}^{(q)}}{\hat{s}^{(0)}}\right)^{2}\right]^{\frac{1}{2q+1}}T^{\frac{1}{2q+1}}\right\rceil

or :math:`T-1`, if that is smaller.

The method needs :math:`m^{*}<T-1`. Otherwise :math:`\hat{s}^{(0)}` is
the sum of all of the autocovariances,
:math:`T^{-1}\left(\sum_{i}g_{i}\right)^{2}`, which is zero, up to
rounding error, for a series with a mean of zero, such as the scores of a
least squares regression, and the method divides by it. The condition
holds with at least 4 observations for the Bartlett kernel
(:math:`m^{*}=2`), 5 for the Parzen kernel (:math:`m^{*}=3`) and 6 for the
quadratic spectral kernel (:math:`m^{*}=4`). An error is raised for fewer
observations, and if the bandwidth is not finite, which happens if
:math:`\hat{s}^{(0)}` is zero or the series has values that are not
finite.

Constant Detection
------------------

Whether a model contains a constant or equivalent is tested using three
tests. These are executed in order and so once a constant is detected,
the others are not executed. The simplest method to ensure that a
constant is correctly detected is to include a columns of 1s.

#. A column with only 1.0s

#. A column with a maximum minus minimum equal to 0 and that is not all
   0s.

#. Whether the column of 1s, :math:`1_{N}`, is in the column space of
   :math:`X`. If it is, then the model contains a constant equivalent
   (e.g., dummies for all categories). :math:`X` can have any rank. The
   column of 1s is regressed on the columns of :math:`X`, which are
   scaled to have a largest absolute value of 1, using a singular value
   decomposition. It is in the column space if the norm of the residual,
   relative to the norm of :math:`1_{N}`, is at most the square root of
   the machine precision (about :math:`1.5\times10^{-8}`). The test is
   not used if the rank of :math:`X` is :math:`N`, since the column
   space is then everything.

.. [1]
   If the model contains an implicit constant, e.g., all categories of a
   dummy, one of the categories is excluded when computing the test
   statistic. The choice of category to drop has no effect and is
   equivalent to reparameterizing the model with a constant and
   excluding one category of dummy.

.. [2]
   This somewhat non-obvious choice is driven by Stata compatibility.
