Metadata-Version: 2.4
Name: MedicalMultitaskModeling
Version: 1.6.3
Summary: Multitask learning for medical data
Project-URL: Repository, https://github.com/FraunhoferMEVIS/MedicalMultitaskModeling
Project-URL: Issues, https://github.com/FraunhoferMEVIS/MedicalMultitaskModeling/issues
Author-email: Raphael Schäfer <jan.raphael.schaefer@mevis.fraunhofer.de>, Till Nicke <till.nicke@mevis.fraunhofer.de>
License: # FRAUNHOFER »MedicalMultitaskModeling« LICENSE FOR SCIENTIFIC NON-COMMERCIAL RESEARCH PURPOSES
        (hereinafter referred to as »License«) 
        
        Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V., Hansastraße 27c, 80686 Munich, Germany (“Fraunhofer”), is an organization for application-oriented research and pursues exclusively and directly non-profit purposes.
        The Fraunhofer Institute for Digital Medicine MEVIS (MEVIS) develops real-world software solutions for image and data supported early detection, diagnosis, and therapy and has developed the software »MedicalMultitaskModeling« as part of its activities. Fraunhofer has the rights to use and exploit MedicalMultitaskModeling, with the exception of those components that constitute third-party software within the meaning of this License. The software is currently in the development stage; it shall therefore not be used for medical purposes. 
        Fraunhofer is willing to grant institutions limited rights to use MedicalMultitaskModeling for testing and evaluation purposes for scientific non-commercial research purposes only without the right to sublicense or transfer MedicalMultitaskModeling; the use for other purposes is not allowed, but can be requested at »histo.app@mevis.fraunhofer.de«.
        
        ## 1 Definitions
        
        »Field of Use« is defined as scientific non-commercial research including the right to publish test and evaluation results. 
        »Internal Purposes« means the Licensee’s internal purposes of testing and using the Software in the Field of Use.
        »Licensee« means an institution granted a right to use the Software for scientific non-commercial research purposes under this License.
        »Third Party Software« means any computer program provided by third parties as licensors and/or authors/right holders under their own license terms, including but not limited to open source software as defined by the Open Source Initiative (www.opensource.org). Third-party software used or included is listed in Annex A. If Fraunhofer software is listed as Third Party Software in Annex A, Fraunhofer shall also be deemed to be a third party and the software shall be deemed to be Third Party Software for the purposes of this License.
        »Object Code« is the intermediate result of a compiler or translation process of the Source Code of the Software and is not subject of this License.
        »Patent« Patent means the patent applied for in the name of Fraunhofer at the European Patent Office under the file number 23209015.9.
        »Source Code« means the code of the Software in the version of the programming language(s).
        »Software« means the computer program made available by Fraunhofer in the Github repository under »https://github.com/FraunhoferMEVIS/MedicalMultitaskModeling«.
         
        ## 2 License
        
        2.1 Subject to the terms and conditions of this License, the Licensee’s acceptance of the terms of this License and the Licensee’s compliance with the provisions of this License, Fraunhofer hereby grants the Licensee the perpetual, non-exclusive, non-sublicensable, non-transferable and royalty-free right, limited in terms of content to Internal Purposes in the Field of Use only, to use the Software on Licensee's computers, i.e. to store and load it permanently or temporarily, to display it and to run it, even to the extent that duplications are necessary for this purpose. The use of results generated or obtained by using the Software for diagnostic or therapeutic purposes is not permitted under this License.
        
        2.2 The right under clause 2.1 includes the right to modify the Software. Licensee shall not reproduce and distribute copies of the Software (neither in Source Code nor in Object Code form) on any medium, with or without modifications.
        
        2.3 Regarding the Patent and subject to the terms and conditions of this License and compliance with the provisions of this License, the Licensee is hereby granted a royalty-free and non-exclusive patent license limited to the rights required to use the Software in accordance with this License. For avoidance of doubt, said patent license shall cover only those rights deriving from the Patent which are indispensable in order not to infringe the Patent and only to the extent that the use of the Software in accordance with this License would otherwise infringe the Patent. The grant of license shall not include rights deriving from the Patent which may in future become necessary for the lawful use of the Software unless such future release of the Software by Fraunhofer grants such patent license.
        
        2.4 No further rights of use or exploitation are granted to the Licensee. Fraunhofer shall retain all intellectual property rights, including software, patents and know-how unless expressly licensed hereunder. The Licensee agrees to refrain from any further use or commercialization of the Software not permitted by the license granted herein.
        
        2.5 Fraunhofer does not grant any rights to Third Party Software to the Licensee, therefore the Licensee must independently ensure proper licensing under their respective applicable license terms. The Third Party Software may be copyrighted in favour of a third party or third parties, e.g. in favour of the authors indicated in the copyright notices included in the corresponding source files or other materials provided together with the Third Party Software, e.g. an open source software. Annex A contains the Third Party Software used by Fraunhofer together with the Software. Fraunhofer does not warrant or represent that Third Party Software is free of errors or claims or fit for a particular purpose. Before using the Third Party Software, the Licensee shall independently verify whether the use of the Third Party Software is permissible and suitable for its purposes. The Licensee undertakes not to use the Software or parts thereof in a way that would result in the Software or parts thereof being licensable under an open source software license. A right of use which allows the Licensee to make the Software or parts thereof available to third parties in its original or edited form as free software in the sense of open source is expressly not granted under this License.
        
        ## 3 Liability, Warranty
        
        3.1 Except in cases of intentional and grossly negligent conduct, Fraunhofer, its legal representatives, trustees, officers and employees shall not be liable for direct or indirect, material or immaterial loss or damage of any kind arising from the License or the use of the Software; this applies, among other things, but not exclusively, to loss of goodwill, loss of production, computer failures or errors, loss of data or economic loss or damage, even if Fraunhofer has been notified of the possibility of such loss or damage.
        
        3.2 Irrespective of clause 3.1, Fraunhofer shall only be liable within the scope of statutory product liability to the extent that the respective provisions are applicable to the Software.
        
        3.3 The Licensee indemnifies Fraunhofer against any claims of third parties, including related costs such as reasonable lawyers' fees and court costs, asserted against Fraunhofer in connection with the Licensee's exercise of the license or due to a breach of the terms of this License by the Licensee. 
        
        3.4 Regarding the Software Fraunhofer has not complied with any regulatory requirements (including norms or other standards) applicable to medical devices, in-vitro diagnostics or medicinal products.
        
        Fraunhofer has not provided any technical documentation, quality assurance, risk management or risk analysis (e.g. in accordance with EN ISO 13485, 14971 or 62304).
        Fraunhofer makes no representations regarding the therapeutic, diagnostic or other medical purpose, usability, performance or safety of the Software or its suitability for conformity assessment purposes of medical devices or in-vitro diagnostics. Fraunhofer is not responsible for the usability and quality of the Software being unaffected if implemented in other results or used in conjunction with other results.
        
        Licensee understands and accepts that the Software is considered research software and is not a medical device. Furthermore, the Licensee knows and accepts that all research results generated or obtained by using the Software may not be used for diagnostic or therapeutic purposes. Furthermore, the Software is not intended for use in therapeutic or diagnostic applications on humans or for other medical purposes or for any other purpose.
        
        ## 4 Limited warranty
        
        This License is granted free of charge and thus constitutes a gift. Accordingly, any warranty is excluded. The Software is a work in progress. The Software is not complete and may therefore contain errors ("bugs"), as is inherent in this type of development.
        
        ## 5 No Trademarks
        
        This License does not grant permission to use the trade names, trademarks, service marks or product names of Fraunhofer.
        
        ## 6 Applicable Law, Place of Jurisdiction
        
        The laws of the Federal Republic of Germany apply exclusively, to the exclusion of the conflict of law provisions and the United Nations Convention on Contracts for the International Sale of Goods (CISG).
        
        The courts of Munich are hereby agreed as place of jurisdiction for all disputes arising from or connected with the License.
        
        ## Annexes
        Annex A: Third Party Software (third-party-licenses.txt, auto-generated by pip-licenses)
Requires-Python: >=3.10
Requires-Dist: albumentations>=2.0.0
Requires-Dist: deepdiff<7,>=6.6.1
Requires-Dist: einops<0.9,>=0.8.0
Requires-Dist: imageio<3,>=2.31.5
Requires-Dist: json5<0.10,>=0.9.14
Requires-Dist: logfire[celery,fastapi]>=4.13.2
Requires-Dist: minio<8,>=7.1.17
Requires-Dist: monai<2,>=1.3.0
Requires-Dist: nibabel<6,>=5.2.1
Requires-Dist: pandas<3,>=2.1.1
Requires-Dist: redis
Requires-Dist: scikit-learn<2,>=1.3.1
Requires-Dist: segmentation-models-pytorch<0.6,>=0.5.0
Requires-Dist: timm<2,>=1.0.15
Requires-Dist: toml<0.11,>=0.10.2
Requires-Dist: torch<3,>=2.1.2
Requires-Dist: torchinfo<2,>=1.8.0
Requires-Dist: torchvision
Requires-Dist: tqdm<5,>=4.66.1
Requires-Dist: typer>=0.9.0
Requires-Dist: wandb>=0.23.0
Provides-Extra: api
Requires-Dist: aiohttp<4,>=3.9.0; extra == 'api'
Requires-Dist: celery[msgpack,redis]<6,>=5.5.3; extra == 'api'
Requires-Dist: fastapi[standard]; extra == 'api'
Requires-Dist: label-studio-sdk; extra == 'api'
Provides-Extra: detection
Requires-Dist: mmdet<4,>=3.3.0; extra == 'detection'
Provides-Extra: export
Requires-Dist: onnx; extra == 'export'
Requires-Dist: onnxscript; extra == 'export'
Provides-Extra: interactive
Requires-Dist: black[jupyter]==23.12.1; extra == 'interactive'
Requires-Dist: ipykernel; extra == 'interactive'
Requires-Dist: ipywidgets; extra == 'interactive'
Requires-Dist: matplotlib; extra == 'interactive'
Requires-Dist: nbconvert; extra == 'interactive'
Requires-Dist: nbformat; extra == 'interactive'
Requires-Dist: plotly; extra == 'interactive'
Requires-Dist: seaborn; extra == 'interactive'
Requires-Dist: streamlit>=1.51.0; extra == 'interactive'
Provides-Extra: survival
Requires-Dist: lifelines<0.31,>=0.30.0; extra == 'survival'
Provides-Extra: testing
Requires-Dist: pytest; extra == 'testing'
Requires-Dist: pytest-cov; extra == 'testing'
Description-Content-Type: text/markdown

# MedicalMultitaskModeling

MedicalMultitaskModeling (M3) enables training foundational medical imaging models using multi-task learning.

The software is provided on "AS IS" basis, i.e. it comes without any warranty, express or implied including (without limitations) any warranty of merchantability and warranty of fitness for a particular purpose.

Please note that this software is licensed under the LICENSE FOR SCIENTIFIC NON-COMMERCIAL RESEARCH PURPOSES, see [license.md](license.md).

## Installation

```bash
pip install medicalmultitaskmodeling m3-sdk
# Extra dependency groups recommended for development:
pip install "medicalmultitaskmodeling[interactive,testing]" m3-sdk
```

Verify system dependencies and installation:

```python
import cv2; import torch; assert torch.cuda.is_available()
from mmm.interactive import *
from mmm.api.M3Model import M3Model
```

See [pyproject.toml](pyproject.toml) for all available extras.

This project depends on [m3-sdk](https://github.com/FraunhoferMEVIS/m3-sdk).

## Usage and available models

- The `DEFAULT_MODEL` is the model presented in [Overcoming data scarcity in biomedical imaging with a foundational multi-task model](https://doi.org/10.1038/s43588-024-00662-z)
- The weights for `TC_NOPANDA` are connected to [Tissue Concepts: supervised foundation models in computational pathology](https://www.sciencedirect.com/science/article/pii/S0010482524017062?via%3Dihub)
- The `UNICORN_ENCODER` achieved first place in the first foundation model benchmark, UNICORN.
- The `WSC_MTL_TINY` is a tiny model presented in [Whole Slide Concepts: A Supervised Foundation Model For Pathological Images](https://arxiv.org/abs/2507.05742v3)


For **pure inference** based on a pre-trained model (downloaded automatically), import it as follows:
```python
from mmm.api.M3Model import M3Model, M3_MODELS

from mmm.api.M3Model import DEFAULT_MODEL  # default UMedPT weights
from mmm.api.M3Model import TC_NOPANDA  # default Tissue Concepts weights 
from mmm.api.M3Model import UNICORN_ENCODER  # First Place UNICORN challenge
from mmm.api.M3Model import WSC_MTL_TINY  # default Whole Slide Concepts weights
```

```python
# Load the selected model
model = M3Model(M3_MODELS[UNICORN_ENCODER])
# The individual components can be accessed through keys
print(model.keys())

import torch; import torch.nn as nn
with torch.inference_mode():
    feature_pyramid: list[torch.Tensor] = model["encoder"](torch.rand(1, 3, 224, 224).to(model.device))
    hidden_vector = nn.Flatten(1)(model["squeezer"](feature_pyramid)[1])
```

- More examples for deep learning researchers: [M3 examples](https://github.com/FraunhoferMEVIS/MedicalMultitaskModeling/tree/main/examples)
- More information for building software with M3: [m3-sdk examples](https://github.com/FraunhoferMEVIS/m3-sdk/tree/main/examples)



## Citation

If you use this project, please cite our work:
- [Overcoming data scarcity in biomedical imaging with a foundational multi-task model](https://doi.org/10.1038/s43588-024-00662-z)
- [Tissue Concepts: supervised foundation models in computational pathology](https://www.sciencedirect.com/science/article/pii/S0010482524017062?via%3Dihub)

```
@article{SchaeferOvercoming2024,
    title = {Overcoming data scarcity in biomedical imaging with a foundational multi-task model},
    journal = {Nature Computational Science},
    issn = {2662-8457},
    doi = {10.1038/s43588-024-00662-z},
    author = {Schäfer, Raphael and Nicke, Till and Höfener, Henning and Lange, Annkristin and Merhof, Dorit and Feuerhake, Friedrich and Schulz, Volkmar and Lotz, Johannes and Kiessling, Fabian},
    year = {2024},
}

@article{nicke2025tissue,
  title={Tissue concepts: Supervised foundation models in computational pathology},
  author={Nicke, Till and Sch{\"a}fer, Jan Raphael and H{\"o}fener, Henning and Feuerhake, Friedrich and Merhof, Dorit and Kie{\ss}ling, Fabian and Lotz, Johannes},
  journal={Computers in biology and medicine},
  volume={186},
  pages={109621},
  year={2025},
  publisher={Elsevier}
}
```
