<img src=https://see.fontimg.com/api/renderfont4/KpAp/eyJyIjoiZnMiLCJoIjoxMzAsInciOjEwMDAsImZzIjoxMzAsImZnYyI6IiNGRDhDMDMiLCJiZ2MiOiIjMDYwNTA1IiwidCI6MX0/Q29uVEVTVA/kg-second-chances-sketch.png width=50% height=50%>


<!--
[![DOI](https://zenodo.org/badge/440851447.svg)](https://zenodo.org/badge/latestdoi/440851447) 
<a href="https://ascl.net/2203.014"><img src="https://img.shields.io/badge/ascl-2203.014-blue.svg?colorB=262255" alt="ascl:2203.014" /></a>

<img src="https://github.com/FiorenSt/AutoSourceID-Light/blob/main/Plots/OpticalImagePatch.png " width=50% height=50%><img src="https://github.com/FiorenSt/AutoSourceID-Light/blob/main/Plots/LoGOnOptical.png " width=50% height=50%> 

-->


# Description

ConTEST is a statistical test for assessing the consistency between observations and astrophysical models.
It uses a combination of non-parametric methods and distance measures to obtain a test
statistic that evaluates the closeness of the astrophysical model to the observations; hypothesis testing is then performed using bootstrap.
 

<img src=https://github.com/FiorenSt/ConTEST/blob/main/img/logo_contest_bkg.png width=15% height=15%>


## Table of Contents 
- [Step-by-step setup](#step-by-step-setup)
- [Tutorial](#tutorial)


# Step-by-step setup

_Follow the instructions below to install and start using ConTEST in Python._

1. Install ConTEST:
   ```sh
   pip intall ConTEST
   ```
   or git clone the repository:
    ```sh
   git clone https://github.com/FiorenSt/ConTEST.git
   ```
   

 <br/>

2. Install the statistical software [R](https://www.r-project.org/). R is needed to run some internal functions of ConTEST.
<br/>

3. To ensure that Python can access R's libraries, run the three lines below in Python (of course, modify to match your folders):

   ```sh
    import os
    os.environ['R_HOME'] = '~/Program Files/R/R-4.0.2'  #-> Your installed R folder
    os.environ['R_USER'] = '~/Miniconda3/envs/ConsistencyTest/lib/site-packages/'  #-> Your python environment
    os.environ['R_LIBS_USER'] = "~/Program Files/R/R-4.0.2/library/"  #-> Your R packages library
   ```
<br/>


4. Install Python dependencies (rpy2 needs R already installed):
   ```sh
   pip intall matplotlib
   pip intall numpy
   pip intall pandas
   pip intall scipy
   pip intall seaborn
   pip intall rpy2
   ```
 <br/>


5. If this is the first time you use ConTEST, you need to install the R package used in Smoothed ConTEST. In Python, simply run:

   ```sh
    def install_R_functions(packnames=('np')):
        # import R's utility package
        utils = rpackages.importr('utils')
        # select a mirror for R packages
        utils.chooseCRANmirror(ind=1)  # select the first mirror in the list
        # R package install
        utils.install_packages(packnames)
    
    install_R_functions()
   ```

3. Use ConTEST in Python! Follow the tutorial below for more information about the individual functions.

<br/>


<img src="https://github.com/FiorenSt/ConTEST/blob/main/img/MemeConTEST.png " width=80% height=80%>

<br/>




# Dependencies:

The following combination of package versions works on most Linux and Windows computers, however other package versions may also
work. If a problem with the combination of packages occurs, raise an issue, and we will help you solve it.


### Python 3 (or superior)
* Numpy 1.21.6
* Pandas 1.4.2
* Scipy 1.7.1
* Matplotlib 3.3.4
* Seaborn 0.11.2
* Rpy2 3.5.2 (For R and Python interaction)

### R 3.6.0 (or superior)
* Np 0.60



# Tutorial

ConTEST can be applied in different case scenarios depending on the nature of the model being tested. 
<br/>
For more details check out the paper: _Stoppa et al., in preparation_

There are 4 fundamental functions in ConTEST:

- ConTEST for regression: Test the consistency of a model with respect to an observed dataset and their uncertainties
- Smoothed ConTEST for regression: Test the consistency of a model with respect to an observed dataset and 
  their uncertainties
- ConTEST for outliers: Test if an observed sample is likely to come from a density model (or a simulated dataset)
- ConTEST for densities: Test the consistency of a density model (or a simulated dataset) with respect to an observed 
  dataset 


### Intro script 

   ```sh
    # ensure that Python can access R
    import os
    os.environ['R_HOME'] = '~/Program Files/R/R-4.0.2'  #-> Your installed R folder
    os.environ['R_USER'] = '~/Miniconda3/envs/ConsistencyTest/lib/site-packages/'  #-> Your python environment
    os.environ['R_LIBS_USER'] = "~/Program Files/R/R-4.0.2/library/"  #-> Your R packages library
    
    # load contest functions
    from ConTEST.CONTEST import contest_reg, smoothed_contest_reg, contest_outliers, contest_dens
   ```


## Regression models

Create synthetic model, observations, and uncertainties to test the functions:

   ```sh
   # random sample 
   n=100
   x = np.random.rand(n)
   
   # synthetic model 
   beta1 = -0.3
   beta2 = 8
   m = 2
   model = np.exp(beta1*x)*np.sin(beta2*x) + m
   
   # error function (Not known in real scenarios)
   err_model = model * .05
   
   # sample observations from the model with the correct uncertainties
   obs = np.zeros(N)
      for i in range(N):
     obs[i] = model[i] + stats.multivariate_normal.rvs(mean=0, cov=(err_model[i])**2,size=1)
   
   # assign correct uncertainties to the observations
   err_obs = err_model
   ```

### ConTEST for regression
 
   ```sh
   Test1 = contest_reg(y_obs = obs, x_obs = x, y_mod = model, y_obs_err = err_obs, K=1000,plot=True)
   ```
<img src="https://github.com/FiorenSt/ConTEST/blob/main/img/ConTESTforRegression.png " width=80% height=80%>


### Smoothed ConTEST for regression
 
   ```sh
   Test2 = smoothed_contest_reg(y_obs = obs, x_obs = x, y_mod = model, y_obs_err = err_obs, K=1000,plot=True)   
   ```
<img src="https://github.com/FiorenSt/ConTEST/blob/main/img/SmoothedConTESTforRegression.png " width=80% height=80%>


## Density models

Create synthetic model and observations:

   ```sh
   n=100
   
   #1D example
   obs = stats.multivariate_normal.rvs(mean=5, cov= [1.5],size=n)
   model = stats.multivariate_normal.rvs(mean=5, cov= [1.5],size=1000)
    
   #2D example
   obs_2d =  stats.multivariate_normal.rvs(mean=[5,5], cov= [[1.5,.8],[.8,2.5]],size=n)
   model_2d = stats.multivariate_normal.rvs(mean=[5,5], cov= [[1.5,.8],[.8,2.5]],size=1000)

### ConTEST for outliers 

   ```sh
   Test3 = contest_outliers(mod=model, obs=obs, K=10000, plot=True)
   Test3 = contest_outliers(mod=model_2d, obs=obs_2d, K=10000, plot=True)
   ```
<img src="https://github.com/FiorenSt/ConTEST/blob/main/img/ConTESTforOutliers1D.png " width=80% height=80%>
<img src="https://github.com/FiorenSt/ConTEST/blob/main/img/ConTESTforOutliers2D.png " width=80% height=80%>



### ConTEST for densities 

   ```sh
   Test4 = contest_dens(mod=model, obs=obs, K=10000, plot=True)   
   Test4 = contest_dens(mod=model_2d, obs=obs_2d, K=10000, plot=True)

   ```
<img src="https://github.com/FiorenSt/ConTEST/blob/main/img/ConTESTforDensities1D.png " width=80% height=80%>
<img src="https://github.com/FiorenSt/ConTEST/blob/main/img/ConTESTforDensities2D.png " width=80% height=80%>




