causalis.scenarios.unconfoundedness.refutation.unconfoundedness.sensitivity

Sensitivity analysis and benchmarking for unconfoundedness.

This module provides tools to assess the robustness of causal estimates to potential unobserved confounding. It implements the sensitivity framework based on partial $R^2$ as described in Cinelli & Hazlett (2020), adapted for non-linear and semi-parametric models via influence functions.

The framework assumes the existence of an unobserved confounder $U$ and quantifies the bias as a function of:

• $R^2_{Y \sim U | D, X}$: How much of the outcome variance $U$ explains.

• $R^2_{D \sim U | X}$: How much of the treatment variance $U$ explains.

The bias-aware estimate is then:

.. math::

\hat{\tau}_{adj} = \hat{\tau} \pm \text{bias}(R^2_{Y \sim U}, R^2_{D \sim U})

Module Contents

Functions

compute_irm_sensitivity_elements	Compute DoubleML-style sensitivity elements for binary-treatment IRM.
compute_bias_aware_ci	Compute bias-aware confidence intervals.
format_bias_aware_summary	Render a single, unified bias-aware summary string.
get_sensitivity_summary	Render a single, unified bias-aware summary string.
sensitivity_benchmark	Benchmark confounders one by one using genuine short-model IRM refits.
sensitivity_benchmark_group	Benchmark a group of confounders using one genuine short-model IRM refit.
sensitivity_analysis	Compute bias-aware components and cache them.
interpret_sensitivity_analysis	Run sensitivity analysis and return a structured interpretation.
run_sensitivity_protocol	Run a binary, benchmark-calibrated sensitivity decision protocol.

Data

__all__

API

causalis.scenarios.unconfoundedness.refutation.unconfoundedness.sensitivity.__all__

[‘sensitivity_analysis’, ‘sensitivity_benchmark’, ‘sensitivity_benchmark_group’, ‘run_sensitivity_pr…

causalis.scenarios.unconfoundedness.refutation.unconfoundedness.sensitivity.compute_irm_sensitivity_elements(*, model: Any, y: numpy.ndarray, d: numpy.ndarray, g0: numpy.ndarray, g1: numpy.ndarray, m_hat: numpy.ndarray, w: Optional[numpy.ndarray] = None, w_bar: Optional[numpy.ndarray] = None, psi: Optional[numpy.ndarray] = None, inv_m: Optional[numpy.ndarray] = None, inv_1m: Optional[numpy.ndarray] = None, score: Any = 'ATE') → dict[str, Any]

Compute DoubleML-style sensitivity elements for binary-treatment IRM.

causalis.scenarios.unconfoundedness.refutation.unconfoundedness.sensitivity.compute_bias_aware_ci(effect_estimation: Dict[str, Any] | Any, *, r2_y: float, r2_d: float, rho: float = 1.0, H0: float = 0.0, alpha: float = 0.05, use_signed_rr: bool = False) → Dict[str, Any]

Compute bias-aware confidence intervals.

Returns a dict with:

• theta, se, alpha, z

• sampling_ci

• theta_bounds_cofounding = [theta_lower, theta_upper] = theta ± bound_width

• bias_aware_ci = [theta - (bound_width + zse), theta + (bound_width + zse)]

• max_bias_base, max_bias, bound_width and components (sigma2, nu2)

Parameters

effect_estimation : Dict[str, Any] or Any The effect estimation object. r2_y : float Sensitivity parameter for the outcome (R^2 form, R_Y^2). r2_d : float Sensitivity parameter for the treatment (R^2 form, R_D^2). rho : float, default 1.0 Correlation parameter. H0 : float, default 0.0 Null hypothesis for robustness values. alpha : float, default 0.05 Significance level. use_signed_rr : bool, default False Whether to use signed rr in the quadratic combination of sensitivity components. If True and m_alpha/rr are available, the bias bound is computed via the per-unit quadratic form and RV/RVa are not reported.

Returns

dict Dictionary with bias-aware results.

causalis.scenarios.unconfoundedness.refutation.unconfoundedness.sensitivity.format_bias_aware_summary(res: Dict[str, Any], label: str | None = None) → str

Render a single, unified bias-aware summary string.

Parameters

res : Dict[str, Any] The result dictionary from compute_bias_aware_ci. label : str, optional, default None The label for the estimand.

Returns

str Formatted summary string.

causalis.scenarios.unconfoundedness.refutation.unconfoundedness.sensitivity.get_sensitivity_summary(effect_estimation: Dict[str, Any] | Any, *, label: Optional[str] = None) → Optional[str]

Render a single, unified bias-aware summary string.

If bias-aware components are missing, shows a sampling-only variant with max_bias=0 and then formats via format_bias_aware_summary for consistency.

Parameters

effect_estimation : Dict[str, Any] or Any The effect estimation object. label : str, optional, default None The label for the estimand.

Returns

Optional[str] Formatted summary string or None if extraction fails.

Notes

The summary reports the point estimate, standard error, and confidence intervals under both the null (no unobserved confounding) and the assumed level of confounding ($R^2_Y, R^2_D$).

It also includes the Robustness Value (RV), which is the minimum strength of confounding ($R^2_Y = R^2_D$) required to change the conclusion (e.g., make the estimate non-significant or zero).

Examples

from causalis.scenarios.unconfoundedness.refutation.unconfoundedness import sensitivity_analysis, get_sensitivity_summary

Assuming ‘estimate’ is a fitted CausalEstimate from IRM

res = sensitivity_analysis(estimate, r2_y=0.05, r2_d=0.05) # doctest: +SKIP summary = get_sensitivity_summary(estimate) # doctest: +SKIP print(summary) # doctest: +SKIP

causalis.scenarios.unconfoundedness.refutation.unconfoundedness.sensitivity.sensitivity_benchmark(effect_estimation: Dict[str, Any] | Any, data: causalis.dgp.causaldata.CausalData, benchmarking_set: List[str] | Literal[all], fit_args: Optional[Dict[str, Any]] = None) → pandas.DataFrame

Benchmark confounders one by one using genuine short-model IRM refits.

Each requested confounder is excluded separately. The returned DataFrame therefore contains one row and one short-model estimate per confounder. Pass "all" to benchmark all confounders in data.confounders order.

See Also

sensitivity_benchmark_group Exclude several confounders together in one short-model refit.

causalis.scenarios.unconfoundedness.refutation.unconfoundedness.sensitivity.sensitivity_benchmark_group(effect_estimation: Dict[str, Any] | Any, data: causalis.dgp.causaldata.CausalData, benchmarking_group: List[str], fit_args: Optional[Dict[str, Any]] = None) → pandas.DataFrame

Benchmark a group of confounders using one genuine short-model IRM refit.

All features in benchmarking_group are excluded together. Group-level confounding strengths and rho are calibrated jointly from long/short IRM sensitivity elements and the actual effect-estimate shift.

Parameters

effect_estimation : dict or Any Estimate/model container exposing a fitted IRM-like model. data : CausalData The exact causal dataset and row order used to fit the long model. benchmarking_group : list[str] Non-empty group of confounders to exclude together. Duplicate names are ignored while preserving their first occurrence. fit_args : dict, optional Additional keyword arguments passed to IRM.estimate(...) for the short model. Short-model diagnostics are disabled by default.

Returns

pandas.DataFrame One-row DataFrame retaining benchmark_group, r2_y, r2_d, rho, theta_long, theta_short, and delta and appending element-based gain statistics and calibration audit fields. The group is stored as a tuple in benchmark_group.

Notes

At least one long-model confounder must remain after removing the group. Runtime is normally dominated by the single short-model refit.

causalis.scenarios.unconfoundedness.refutation.unconfoundedness.sensitivity.sensitivity_analysis(effect_estimation: Dict[str, Any] | Any, *, r2_y: float, r2_d: float, rho: float = 1.0, H0: float = 0.0, alpha: float = 0.05, use_signed_rr: bool = False) → Dict[str, Any]

Compute bias-aware components and cache them.

This function turns a fitted estimate into a simple hidden-confounding stress test. In the default mode, the bound width is

.. math::

|\rho| \cdot \sqrt{\sigma^2 \nu^2}
\cdot
\sqrt{\frac{r2_y}{1-r2_y}\frac{r2_d}{1-r2_d}},

so the reported confounding interval is

.. math::

[\theta - \text{bound\_width}, \theta + \text{bound\_width}].

Here :math:r2_y controls how much residual outcome variation an omitted confounder could explain, :math:r2_d does the same for treatment assignment, and :math:\rho sets the sign and strength alignment between the two channels.

Parameters

effect_estimation : Dict[str, Any] or Any The effect estimation object. r2_y : float Sensitivity parameter for the outcome (R^2 form, R_Y^2; converted to odds form internally). r2_d : float Sensitivity parameter for the treatment (R^2 form, R_D^2). rho : float, default 1.0 Correlation parameter. H0 : float, default 0.0 Null hypothesis for robustness values. alpha : float, default 0.05 Significance level. use_signed_rr : bool, default False Whether to use signed rr in the quadratic combination of sensitivity components. If True and m_alpha/rr are available, the bias bound is computed via the per-unit quadratic form and RV/RVa are not reported.

Returns

dict Dictionary with bias-aware results: - theta, se, alpha, z - sampling_ci - theta_bounds_cofounding = (theta - bound_width, theta + bound_width) - bias_aware_ci = faithful CI for the bounds - max_bias and components (sigma2, nu2) - params (r2_y, r2_d, rho, use_signed_rr)

Examples

from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor from causalis.dgp import obs_linear_26_dataset from causalis.scenarios.unconfoundedness.model import IRM data = obs_linear_26_dataset( … n=1000, … seed=3141, … include_oracle=False, … return_causal_data=True, … ) irm = IRM( … data=data, … ml_g=RandomForestRegressor( … n_estimators=200, … max_depth=6, … min_samples_leaf=5, … random_state=3141, … ), … ml_m=RandomForestClassifier( … n_estimators=200, … max_depth=6, … min_samples_leaf=5, … random_state=3141, … ), … n_folds=3, … random_state=3141, … ) estimate = irm.fit().estimate(score=”ATE”) out = sensitivity_analysis(estimate, r2_y=0.02, r2_d=0.02, rho=1.0) out[“theta_bounds_cofounding”] # doctest: +SKIP out[“bias_aware_ci”] # doctest: +SKIP

causalis.scenarios.unconfoundedness.refutation.unconfoundedness.sensitivity.interpret_sensitivity_analysis(effect_estimation: Dict[str, Any] | Any, *, r2_y: float, r2_d: float, rho: float = 1.0, H0: float = 0.0, alpha: float = 0.05, use_signed_rr: bool = False) → Dict[str, Any]

Run sensitivity analysis and return a structured interpretation.

Parameters

effect_estimation : Dict[str, Any] or Any The effect estimation object. r2_y : float Sensitivity parameter for outcome residual confounding strength. r2_d : float Sensitivity parameter for treatment residual confounding strength. rho : float, default 1.0 Correlation parameter for unobserved confounding. H0 : float, default 0.0 Null hypothesis used for significance checks. alpha : float, default 0.05 Significance level. use_signed_rr : bool, default False Whether to use signed rr in the quadratic sensitivity combination.

Returns

Dict[str, Any] Dictionary with: - raw: the output of sensitivity_analysis(...) - interpretation: machine-readable interpretation fields - summary: compact human-readable interpretation

Notes

This function wraps sensitivity_analysis and provides a textual interpretation of the results, including whether the estimate remains significant under the assumed confounding.

Examples

from causalis.scenarios.unconfoundedness.refutation.unconfoundedness import interpret_sensitivity_analysis

Assuming ‘estimate’ is a fitted CausalEstimate

interpretation = interpret_sensitivity_analysis(estimate, r2_y=0.01, r2_d=0.01) # doctest: +SKIP print(interpretation[“summary”]) # doctest: +SKIP

causalis.scenarios.unconfoundedness.refutation.unconfoundedness.sensitivity.run_sensitivity_protocol(effect_estimation: Dict[str, Any] | Any, data: causalis.dgp.causaldata.CausalData, *, benchmark_groups: Mapping[str, List[str]], decision_threshold: float, direction: Literal[positive, negative], alpha: float = 0.05, stress_multiplier: float = 2.0, preconditions_passed: bool = True, fit_args: Optional[Dict[str, Any]] = None) → Dict[str, Any]

Run a binary, benchmark-calibrated sensitivity decision protocol.

The primary decision uses one-times benchmark strength and the benchmark’s estimated rho. Every primary benchmark must pass. A benchmark passes when its bias-aware confidence interval lies strictly beyond the pre-specified practical threshold in direction. The two-times and rho=1 adversarial scenarios are reported as stress tests and do not change the primary PASS/FAIL decision.

Parameters

effect_estimation : dict or Any Fitted IRM-like estimate accepted by :func:sensitivity_analysis. data : CausalData Exact data and row order used to fit the long model. benchmark_groups : mapping[str, list[str]] Non-empty mapping from meaningful benchmark labels to groups of observed pre-treatment confounders. Each group is omitted in its own genuine short-model refit. decision_threshold : float Pre-specified practically meaningful effect boundary. For a positive claim, the lower bias-aware CI endpoint must exceed this value. For a negative claim, the upper endpoint must be below it. direction : {“positive”, “negative”} Direction of the practically meaningful causal claim. alpha : float, default 0.05 Two-sided significance level used for bias-aware confidence intervals. stress_multiplier : float, default 2.0 Strength multiplier for the secondary stress scenario. Each partial R-squared is multiplied on the odds/Cohen-f-squared scale. preconditions_passed : bool, default True External design gate covering the causal adjustment set, overlap, nuisance-model quality, and estimator stability. False forces the overall result to FAIL while still returning sensitivity diagnostics. fit_args : dict, optional Additional keyword arguments for each short-model estimate.

Returns

dict A report with status ("PASS" or "FAIL"), passed, benchmark and scenario DataFrames, raw scenario details, warnings, and a standard human-readable conclusion. primary determines status; stress and adversarial describe the robustness margin.

Notes

The robustness values rv and rva are reported but are not used as universal pass/fail cutoffs. Benchmark selection and decision_threshold must be justified before inspecting the effect estimate. rho for both singleton and multi-feature benchmarks is calibrated from long/short sensitivity elements. A value of +/-1 in primary or stress can mean either that rho_raw saturated at the correlation boundary or that DoubleML’s non-positive-gain fallback was used. Only the adversarial scenario forces rho=1 independently of the benchmark effect shift.

DoubleML-compatible boundary calibration clips non-positive raw gains to zero and keeps the scenario numerically evaluable. Raw values and boundary flags remain available in benchmarks for audit. Missing/non-finite sensitivity elements and an upper-bound cf_d=1 still fail closed, because Causalis sensitivity bounds require finite inputs below one.

The current IRM estimator supports n_rep=1 only. Repeated cross-fitting stability must therefore be assessed outside this function until IRM adds repeated-split support.

Examples

report = run_sensitivity_protocol( # doctest: +SKIP … estimate, … data, … benchmark_groups={“engagement”: [“sessions”, “activity_days”]}, … decision_threshold=0.0, … direction=”positive”, … ) report[“status”] # doctest: +SKIP ‘PASS’ report[“primary”][[“benchmark”, “ci_lower”, “ci_upper”, “passed”]] # doctest: +SKIP