Metadata-Version: 2.4
Name: soilspecdl
Version: 0.0.1
Summary: Practical deep learning for soil infrared spectroscopy, built on the fastai framework.
Author-email: Franck Albinet <franckalbinet@gmail.com>
License: Apache-2.0
Project-URL: Repository, https://github.com/franckalbinet/soilspecdl
Project-URL: Documentation, https://franckalbinet.github.io/soilspecdl/
Keywords: nbdev
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastai
Requires-Dist: soilspectfm
Requires-Dist: soilspecdata
Requires-Dist: pandas
Requires-Dist: seaborn
Requires-Dist: kennard-stone
Dynamic: license-file

# SoilSpecDL


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

**Practical deep learning for soil infrared spectroscopy, built on the
fastai framework.**

`soilspecdl` brings the [fastai](https://www.fast.ai/) philosophy to
soil spectroscopy: state-of-the-art deep learning models should be
accessible to domain scientists, not just ML practitioners. If you are a
soil scientist comfortable with R packages like `resemble` or
`prospectr`, this package is designed with you in mind.

Built on top of [fastai](https://www.fast.ai/) — a library renowned for
making deep learning approachable without sacrificing performance —
`soilspecdl` provides sensible defaults, minimal boilerplate, and a
layered API that lets you go deeper when you need to. Training a CNN on
a large MIR spectral library should take a few lines of code, not a PhD
in machine learning.

`soilspecdl` is part of a broader ecosystem for soil spectroscopy in
Python:

- [`soilspecdata`](https://fr.anckalbi.net/soilspecdata/) — fetch and
  access large spectral libraries (KSSL, OSSL)
- [`soilspectfm`](https://fr.anckalbi.net/soilspectfm/) — preprocessing
  and spectral transforms
- **`SoilSpecDL`** — deep learning modeling

## Get Started

``` python
from soilspecdata.datasets.ossl import get_ossl
from soilspectfm.core import SNV, TakeDerivative
from sklearn.pipeline import Pipeline
import numpy as np
```

Load OSSL/USDA KSSL + ICRAF + … dataset large soil spectroscopy library
using the [SoilSpecData](https://fr.anckalbi.net/soilspecdata) python
package:

``` python
ossl = get_ossl()
mir_data = ossl.get_mir()
wns = mir_data.wavenumbers

X, y, ids = ossl.get_aligned_data(
    spectra_data=mir_data,
    target_cols='k.ext_usda.a725_cmolc.kg')

X.shape, y.shape, ids.shape, wns[:5], wns.shape
```

    ((57676, 1701),
     (57676, 1),
     (57676,),
     array([600, 602, 604, 606, 608]),
     (1701,))

A bit of pre-processing using
[SoilSpecTfm](https://fr.anckalbi.net/soilspectfm) python package:

``` python
pipe = Pipeline([
    ('snv', SNV()),
    ('derivative', TakeDerivative()),
])
X_tfm = pipe.fit_transform(X)
```

``` python
# Log-transforming the target (Exchangeable Potassium)
y_tfm = np.log1p(y.flatten())
```

``` python
from soilspecdl.dataloader import get_spectral_dls
```

Create a Fastai-like data loader with nice batch preview:

``` python
dls, test_idx = get_spectral_dls(X_tfm, y_tfm, wns)
dls.show_batch(max_n=12, ncols=4)
```

![](index_files/figure-commonmark/cell-7-output-1.png)

Importing [albinet et
al.](https://www.sciencedirect.com/science/article/pii/S2589721722000186)
paper CNN architecture, fastai Learner, …

``` python
from soilspecdl.models import MirzaiCNN
from fastai.learner import Learner
from fastai.callback.schedule import lr_find
from fastai.losses import MSELossFlat
from fastai.metrics import R2Score
```

``` python
learn = Learner(dls, MirzaiCNN(), loss_func=MSELossFlat(), metrics=R2Score())
```

Finding “best” learning rate:

``` python
learn.lr_find()
```

    Epoch 1/1 : |█-------------------| 5.07% [73/1441 00:00<00:12... 0.2062]

    SuggestedLRs(valley=0.0020892962347716093)

![](index_files/figure-commonmark/cell-10-output-3.png)

``` python
learn.fit_one_cycle(40, lr_max=1e-3)
```

    epoch     train_loss  valid_loss  r2_score  time    
    0         0.110809    0.084269    0.394419  00:07                            
    1         0.079789    0.069087    0.503516  00:15                            
    2         0.074917    0.060016    0.568708  00:07                            
    3         0.064447    0.064101    0.539350  00:07                            
    4         0.054324    0.056243    0.595819  00:07                            
    5         0.061540    0.059759    0.570552  00:06                            
    6         0.057230    0.138288    0.006216  00:07                            
    7         0.057897    0.121253    0.128639  00:08                            
    8         0.052471    0.092067    0.338378  00:07                            
    9         0.045639    0.044204    0.682335  00:15                             
    10        0.048287    0.184855    -0.328426 00:06                             
    11        0.043516    0.108474    0.220473  00:06                             
    12        0.040384    0.038102    0.726186  00:07                             
    13        0.040395    0.063995    0.540110  00:06                             
    14        0.034513    0.057830    0.584413  00:06                             
    15        0.037132    0.044052    0.683426  00:06                             
    16        0.036089    0.051145    0.632458  00:07                             
    17        0.032155    0.074059    0.467791  00:15                             
    18        0.034771    0.040014    0.712450  00:08                             
    19        0.031389    0.068343    0.508866  00:06                             
    20        0.033233    0.066360    0.523116  00:06                             
    21        0.032475    0.035962    0.741569  00:07                             
    22        0.028055    0.037719    0.728938  00:15                             
    23        0.029821    0.068363    0.508722  00:07                             
    24        0.027841    0.030044    0.784094  00:07                             
    25        0.025044    0.029944    0.784813  00:06                             
    26        0.026742    0.033762    0.757377  00:06                             
    27        0.023458    0.043821    0.685087  00:06                             
    28        0.022406    0.031492    0.773691  00:14                             
    29        0.024227    0.027881    0.799640  00:15                             
    30        0.022663    0.028077    0.798226  00:06                             
    31        0.020502    0.028358    0.796213  00:06                             
    32        0.021859    0.026163    0.811988  00:06                             
    33        0.019432    0.027430    0.802882  00:07                             
    34        0.019836    0.025392    0.817526  00:07                             
    35        0.018150    0.025433    0.817233  00:07                             
    36        0.020574    0.025172    0.819103  00:07                             
    37        0.019895    0.025199    0.818912  00:14                             
    38        0.019346    0.025025    0.820163  00:15                             
    39        0.019031    0.025060    0.819907  00:06                             

## Developer Guide

If you are new to using `nbdev` here are some useful pointers to get you
started.

### Install soilspecai in Development mode

``` sh
# make sure soilspecai package is installed in development mode
$ pip install -e .

# make changes under nbs/ directory
# ...

# compile to have changes apply to soilspecai
$ nbdev_prepare
```

## Usage

### Installation

Install latest from the GitHub
[repository](https://github.com/franckalbinet/soilspecdl):

``` sh
$ pip install git+https://github.com/franckalbinet/soilspecdl.git
```

or from [conda](https://anaconda.org/franckalbinet/soilspecdl)

``` sh
$ conda install -c franckalbinet soilspecdl
```

or from [pypi](https://pypi.org/project/soilspecdl/)

``` sh
$ pip install soilspecdl
```
