Metadata-Version: 2.1
Name: sickness-screening
Version: 1.0.1
Summary: Module for sepsis predictions
Home-page: https://github.com/sslavian812/sepsis-predictions.git
Author: @Margo78, @akp1n
Author-email: timtimk30@yandex.ru
License: UNKNOWN
Description: # Disease-predictions module
        
        ## Instruction
        
        Predictions sepsis is a module based on pandas, torch, and scikit-learn that allows users to perform simple operations with the MIMIC dataset.
        With this module, using just a few functions, you can train your model to predict whether some patients have certain diseases or not. 
        By default, the module is designed to train and predict sepsis. 
        The module also allows users to change different names of tables to aggregate data from.
        
        ### Installation
        
        To install the module, use the following command:
        
        ```bash
        pip install sickness-screening
        ```
        or
        ```bash
        pip3 install sickness-screening
        ```
        ### Usage
        
        You can import functions from the module into your Python file to aggregate data from MIMIC, 
        fill empty spots, compress data between patients, and train your model.
        
        ### Examples
        
        #### Aggregate patient diagnoses Data
        ```python
        import sickness_screening as ss
        
        ss.get_diagnoses_data(patient_diagnoses_csv='path_to_patient_diagnoses.csv', 
                         all_diagnoses_csv='path_to_all_diagnoses.csv',
                         output_file_csv='gottenDiagnoses.csv')
        ```
        
        #### Aggregate patient ssir Data
        ```python
        import sickness_screening as ss
        
        ss.get_analasys_data(chartevents_csv='chartevents.csv', subject_id_col='subject_id', itemid_col='itemid',
                     charttime_col='charttime', value_col='value', valuenum_col='valuenum', valueuom_col='valueuom',
                     itemids=None, rest_columns=None, output_csv='ssir.csv'):
        ```
        
        #### Combine Diagnoses and SSIR Data
        ```python
        import sickness_screening as ss
        
        ss.combine_data(gotten_diagnoses_csv='gottenDiagnoses.csv', 
                                      ssir_csv='path_to_ssir.csv',
                                      output_file='diagnoses_and_ssir.csv')
        ```
        
        #### Aggregate patient blood analysis data from chartevents.csv and labevents.csv and combine it with diagnoses and SSIR Data
        ```python
        import sickness_screening as ss
        
        ss.merge_and_get_data(diagnoses_and_ssir_csv='diagnoses_and_ssir.csv', 
                                               blood_csv='path_to_blood.csv',
                                               chartevents_csv='path_to_chartevents.csv',
                                               output_csv='merged_data.csv')
        )
        ```
        
        #### Compress Data by patient
        ```python
        import sickness_screening as ss
        
        ss.compress(df_to_compress='balanced_data.csv', 
                    output_csv='compressed_data.csv')
        
        ```
        
        #### Choose top non-sepsis patients to balance
        ```python
        import sickness_screening as ss
        
        ss.choose(compressed_df_csv='compressed_data.csv', 
                  output_file='final_balanced_data.csv')
        ```
        
        #### Fill missing values with mode
        ```python
        import sickness_screening as ss
        
        ss.fill_values(balanced_csv='final_balanced_data.csv', 
                       strategy='most_frequent', 
                       output_csv='filled_data.csv')
        ```
        
        #### Train model
        ```python
        import sickness_screening as ss
        from sklearn.ensemble import RandomForestClassifier
        from sklearn.preprocessing import MinMaxScaler
        model = ss.train_model(df_to_train_csv='filled_data.csv', 
                               categorical_col=['Large Platelets'], 
                               columns_to_train_on=['Amylase'], 
                               model=RandomForestClassifier(), 
                               single_cat_column='White Blood Cells', 
                               has_disease_col='has_sepsis', 
                               subject_id_col='subject_id', 
                               valueuom_col='valueuom', 
                               scaler=MinMaxScaler(), 
                               random_state=42, 
                               test_size=0.2)
        ```
        
        #### For example, you can insert models like CatBoostClassifier or SVC with different kernels.
        CatBoostClassifier:
        ```python
        class_weights = {0: 1, 1: 15}
        clf = CatBoostClassifier(loss_function='MultiClassOneVsAll', class_weights=class_weights, iterations=50, learning_rate=0.1, depth=5)
        clf.fit(X_train, y_train)
        ```
        SVC using Gaussian kernel with radial basis function (RBF):
        ```python
        class_weights = {0: 1, 1: 13}
        param_dist = {
            'C': reciprocal(0.1, 100),
            'gamma': reciprocal(0.01, 10),
            'kernel': ['rbf']
        }
        
        svm_model = SVC(class_weight=class_weights, random_state=42)
        random_search = RandomizedSearchCV(
            svm_model,
            param_distributions=param_dist,
            n_iter=10,
            cv=5,
            scoring=make_scorer(recall_score, pos_label=1),
            n_jobs=-1
        )
        ```
        
        ## The Second Method (Transformers TabNet and DeepFM)
        ### Collecting features into a dataset
        #### You can choose any features, but we will take 4 as in MEWS (Modified Early Warning Score) to predict sepsis in the first hours of a patient's hospital stay:
        * Systolic blood pressure
        * Heart rate
        * Respiratory rate
        * Temperature
        ```python
          item_ids_set = set(item_ids)
        
          with open(file_path) as f:
              headers = f.readline().replace('\n', '').split(',')
              i = 0
              for line in tqdm(f):
                  values = line.replace('\n', '').split(',')
                  subject_id = values[0]
                  item_id = values[6]
                  valuenum = values[8]
                  if item_id in item_ids_set:
                      if subject_id not in result:
                          result[subject_id] = {}
                      result[subject_id][item_id] = valuenum
                  i += 1
          
          table = pd.DataFrame.from_dict(result, orient='index')
          table['subject_id'] = table.index
        
        item_ids = [str(x) for x in [225309, 220045, 220210, 223762]]
        ```
        
        #### Adding the target
        ```python
        target_subjects = drgcodes.loc[drgcodes['drg_code'].isin([870, 871, 872]), 'subject_id']
        merged_data.loc[merged_data['subject_id'].isin(target_subjects), 'diagnosis'] = 1
        ```
        
        #### Filling in gaps using the NoNa library. This algorithm fills in gaps using various machine learning methods, we use StandardScaler, Ridge and RandomForestClassifier
        ```python
        nona(
            data=X,
            algreg=make_pipeline(StandardScaler(with_mean=False), Ridge(alpha=0.1)),
            algclass=RandomForestClassifier(max_depth=2, random_state=0)
        )
        ```
        
        #### Addressing class imbalance using SMOTE
        ```python
        smote = SMOTE(random_state=random_state)
        X_resampled, y_resampled = smote.fit_resample(X_train, y_train)
        ```
        
        #### Training the TabNet model. TabNet is an extension of pyTorch. First, we use semi-supervised pretraining with TabNetPretrainer, then create and train a classification model using TabNetClassifier
        ```python
        unsupervised_model = TabNetPretrainer(
            optimizer_fn=torch.optim.Adam,
            optimizer_params=dict(lr=pretraining_lr),
            mask_type=mask_type
        )
        
        unsupervised_model.fit(
            X_train=X_train.values,
            eval_set=[X_val.values],
            pretraining_ratio=pretraining_ratio,
        )
        
        clf = TabNetClassifier(
            optimizer_fn=torch.optim.Adam,
            optimizer_params=dict(lr=training_lr),
            scheduler_params=scheduler_params,
            scheduler_fn=torch.optim.lr_scheduler.StepLR,
            mask_type=mask_type
        )
        
        clf.fit(
            X_train=X_train.values, y_train=y_train.values,
            eval_set=[(X_val.values, y_val.values)],
            eval_metric=['auc'],
            max_epochs=max_epochs,
            patience=patience,
            from_unsupervised=unsupervised_model
        )
        ```
        
        #### Training the DeepFM model
        ```python
        deepfm = DeepFM("ranking", data_info, embed_size=16, n_epochs=2,
                        lr=1e-4, lr_decay=False, reg=None, batch_size=1,
                        num_neg=1, use_bn=False, dropout_rate=None,
                        hidden_units="128,64,32", tf_sess_config=None)
        
        deepfm.fit(train_data, verbose=2, shuffle=True, eval_data=eval_data,
                   metrics=["loss", "balanced_accuracy", "roc_auc", "pr_auc",
                            "precision", "recall", "map", "ndcg"])
        ```
        
        #### Viewing the obtained metrics
        ```python
        result = loaded_clf.predict(X_test.values)
        accuracy = (result == y_test.values).mean()
        precision = precision_score(y_test.values, result)
        recall = recall_score(y_test.values, result)
        f1 = f1_score(y_test.values, result)
        ```
        
        #### Visualization of 2 PCA components was performed
        ![Image alt](./Визуализация_2_PCA_компоненты.png)
        The distribution by components is presented below:
        
        |                  |  Load on the first component  | Load on the second component  |
        | ---------------- | :---: | :---: |
        | Heart rate       |           -0.101450           |            0.991611           |
        | Temperature      |            0.001178           |            0.013098           |
        | Systolic BP      |            0.994771           |            0.100169           |
        | Respiratory rate |            0.011673           |            0.080573           |
        | MEWS             |           -0.000660           |            0.003313           |
        
        No patterns were found.
        
        #### A variational encoder was trained to build a separable 2D space
        ![Image alt](./Вариационный_кодировщик.png)
        We can see that they overlap and are inseparable.
        
Keywords: sepsis,predictions,python,disease,screening
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3.11
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.7
Description-Content-Type: text/markdown
