Metadata-Version: 2.1
Name: MedReportEval
Version: 0.0.4
Summary: A brief description of your package
License: MIT
Classifier: Development Status :: 1 - Planning
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.6
Description-Content-Type: text/markdown
Requires-Dist: absl-py==2.1.0
Requires-Dist: accelerate==0.33.0
Requires-Dist: aiohappyeyeballs==2.3.6
Requires-Dist: aiohttp==3.10.3
Requires-Dist: aiosignal==1.3.1
Requires-Dist: annotated-types==0.7.0
Requires-Dist: asttokens
Requires-Dist: async-timeout==4.0.3
Requires-Dist: attrs==24.2.0
Requires-Dist: backports.tarfile==1.2.0
Requires-Dist: bert-score==0.3.13
Requires-Dist: blis==0.7.11
Requires-Dist: build==1.2.2.post1
Requires-Dist: catalogue==2.0.10
Requires-Dist: certifi==2024.7.4
Requires-Dist: cffi==1.17.1
Requires-Dist: charset-normalizer==3.3.2
Requires-Dist: click==8.1.7
Requires-Dist: cloudpathlib==0.20.0
Requires-Dist: comm
Requires-Dist: confection==0.1.5
Requires-Dist: contourpy==1.2.1
Requires-Dist: cryptography==44.0.0
Requires-Dist: cycler==0.12.1
Requires-Dist: cymem==2.0.10
Requires-Dist: Cython==3.0.11
Requires-Dist: datasets==2.21.0
Requires-Dist: debugpy
Requires-Dist: decorator
Requires-Dist: dill==0.3.8
Requires-Dist: docutils==0.21.2
Requires-Dist: entrypoints
Requires-Dist: example_package_shengyuan==0.0.1
Requires-Dist: exceptiongroup
Requires-Dist: executing
Requires-Dist: filelock==3.15.4
Requires-Dist: fonttools==4.53.1
Requires-Dist: frozenlist==1.4.1
Requires-Dist: fsspec==2024.6.1
Requires-Dist: green_score==0.0.5
Requires-Dist: huggingface-hub==0.24.5
Requires-Dist: idna==3.7
Requires-Dist: importlib_metadata==8.5.0
Requires-Dist: iniconfig==2.0.0
Requires-Dist: ipykernel
Requires-Dist: ipython
Requires-Dist: jaraco.classes==3.4.0
Requires-Dist: jaraco.context==6.0.1
Requires-Dist: jaraco.functools==4.1.0
Requires-Dist: jedi
Requires-Dist: jeepney==0.8.0
Requires-Dist: Jinja2==3.1.4
Requires-Dist: joblib==1.4.2
Requires-Dist: jsonschema==4.23.0
Requires-Dist: jsonschema-specifications==2024.10.1
Requires-Dist: jupyter-client
Requires-Dist: jupyter_core
Requires-Dist: keyring==25.5.0
Requires-Dist: kiwisolver==1.4.5
Requires-Dist: langcodes==3.5.0
Requires-Dist: language_data==1.3.0
Requires-Dist: marisa-trie==1.2.1
Requires-Dist: markdown-it-py==3.0.0
Requires-Dist: MarkupSafe==2.1.5
Requires-Dist: matplotlib==3.9.2
Requires-Dist: matplotlib-inline
Requires-Dist: mdurl==0.1.2
Requires-Dist: medspacy==1.3.1
Requires-Dist: medspacy_quickumls==3.2
Requires-Dist: medspacy_unqlite==0.9.6
Requires-Dist: more-itertools==10.5.0
Requires-Dist: mpmath==1.3.0
Requires-Dist: multidict==6.0.5
Requires-Dist: multiprocess==0.70.16
Requires-Dist: murmurhash==1.0.11
Requires-Dist: nest_asyncio
Requires-Dist: networkx==3.3
Requires-Dist: nh3==0.2.19
Requires-Dist: nltk==3.9.1
Requires-Dist: numpy==1.26.4
Requires-Dist: nvidia-cublas-cu12==12.1.3.1
Requires-Dist: nvidia-cuda-cupti-cu12==12.1.105
Requires-Dist: nvidia-cuda-nvrtc-cu12==12.1.105
Requires-Dist: nvidia-cuda-runtime-cu12==12.1.105
Requires-Dist: nvidia-cudnn-cu12==9.1.0.70
Requires-Dist: nvidia-cufft-cu12==11.0.2.54
Requires-Dist: nvidia-curand-cu12==10.3.2.106
Requires-Dist: nvidia-cusolver-cu12==11.4.5.107
Requires-Dist: nvidia-cusparse-cu12==12.1.0.106
Requires-Dist: nvidia-nccl-cu12==2.20.5
Requires-Dist: nvidia-nvjitlink-cu12==12.6.20
Requires-Dist: nvidia-nvtx-cu12==12.1.105
Requires-Dist: opencv-python==4.10.0.84
Requires-Dist: packaging
Requires-Dist: pandas==2.2.2
Requires-Dist: parso
Requires-Dist: patsy==0.5.6
Requires-Dist: pexpect
Requires-Dist: pickleshare
Requires-Dist: pillow==10.4.0
Requires-Dist: pkginfo==1.10.0
Requires-Dist: platformdirs
Requires-Dist: pluggy==1.5.0
Requires-Dist: preshed==3.0.9
Requires-Dist: prompt_toolkit
Requires-Dist: protobuf==5.27.3
Requires-Dist: psutil
Requires-Dist: ptyprocess
Requires-Dist: pure_eval
Requires-Dist: pyarrow==17.0.0
Requires-Dist: pycparser==2.22
Requires-Dist: pydantic==2.10.1
Requires-Dist: pydantic_core==2.27.1
Requires-Dist: PyFastNER==1.0.10
Requires-Dist: Pygments
Requires-Dist: pyparsing==3.1.2
Requires-Dist: pyproject_hooks==1.2.0
Requires-Dist: PyRuSH==1.0.9
Requires-Dist: pysbd==0.3.4
Requires-Dist: pysimstring==1.2.1
Requires-Dist: pytest==8.3.3
Requires-Dist: python-dateutil
Requires-Dist: pytz==2024.1
Requires-Dist: PyYAML==6.0.2
Requires-Dist: pyzmq
Requires-Dist: quicksectx==0.4.0
Requires-Dist: readme_renderer==44.0
Requires-Dist: referencing==0.35.1
Requires-Dist: regex==2024.7.24
Requires-Dist: requests==2.32.3
Requires-Dist: requests-toolbelt==1.0.0
Requires-Dist: rfc3986==2.0.0
Requires-Dist: rich==13.9.4
Requires-Dist: rouge_score==0.1.2
Requires-Dist: rpds-py==0.21.0
Requires-Dist: safetensors==0.4.4
Requires-Dist: scikit-learn==1.5.1
Requires-Dist: scipy==1.14.0
Requires-Dist: SecretStorage==3.3.3
Requires-Dist: sentence-transformers==3.0.1
Requires-Dist: sentencepiece==0.2.0
Requires-Dist: shellingham==1.5.4
Requires-Dist: six
Requires-Dist: smart-open==7.0.5
Requires-Dist: spacy==3.7.5
Requires-Dist: spacy-legacy==3.0.12
Requires-Dist: spacy-loggers==1.0.5
Requires-Dist: srsly==2.4.8
Requires-Dist: stack-data
Requires-Dist: statsmodels==0.14.2
Requires-Dist: sympy==1.13.2
Requires-Dist: thinc==8.2.5
Requires-Dist: threadpoolctl==3.5.0
Requires-Dist: tokenizers==0.19.1
Requires-Dist: tomli==2.1.0
Requires-Dist: torch==2.4.0
Requires-Dist: torchvision==0.19.0
Requires-Dist: tornado
Requires-Dist: tqdm==4.66.5
Requires-Dist: traitlets
Requires-Dist: transformers==4.44.0
Requires-Dist: triton==3.0.0
Requires-Dist: twine==5.1.1
Requires-Dist: typer==0.13.1
Requires-Dist: typing_extensions
Requires-Dist: tzdata==2024.1
Requires-Dist: Unidecode==1.3.8
Requires-Dist: urllib3==2.2.2
Requires-Dist: wasabi==1.1.3
Requires-Dist: wcwidth
Requires-Dist: weasel==0.4.1
Requires-Dist: wrapt==1.17.0
Requires-Dist: xxhash==3.4.1
Requires-Dist: yarl==1.9.4
Requires-Dist: zipp==3.21.0

### Model Preparation

Our codebase integrates CheXbert [3] for report labeling and RadGraph [4] for evaluation. Download the required model checkpoints from the [repository here](https://github.com/rajpurkarlab/CXR-Report-Metric) and place them in the `./models/` directory.

### Environment Preparation

To set up the project environment, use the following command:

```
conda env create --name=<env_name> -f cxr-green.yml
```

If you're already on DSI cluster, use the following command to activate environment:

```
conda activate /net/projects/chai-lab/miniconda3/envs/cxr-green
```

### Test Example

Here is an example how to run evaluation:

```
python test.py \
--gt_csv /net/projects/chacha/cxr_eval/test/gt_imp_sample.csv \
--gen_csv /net/projects/chacha/cxr_eval/test/gen_imp_sample_3_1.csv \
--output_metrics /net/projects/chacha/cxr_eval/results/metrics.csv \
--output_combined /net/projects/chacha/cxr_eval/results/metrics_w_green.csv \
--output_dir /net/projects/chacha/cxr_eval/results
```

### Running Notes

1. Based on our testing, we can successfully run metrics without (GREEN, BertScore, RadGraph F1) on A40 GPU. We'll continue updating requirement for environment and gpu to run (GREEN, BertScore, RadGraph F1).
2. Each input csv should contain columns **['study_id', 'report']**. Each column needs to be str and 'study_id' should not include any other characters except numbers.
3. Ensure each input csv has no empty values.

