{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SumMed Demo (Python Module) \n",
    "\n",
    "This is a notebook to give a short overview of how to use the SumMed library. It's showing key features of SumMed in a Python notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"summed_logo.png\" width=600/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Prepare environment\n",
    "First we start here with some technical preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dotenv import load_dotenv\n",
    "from summed.data import PlatformConfig, AnalysisConfig, FileInfo, DocumentSource, Document, SearchResult, TermExplanation\n",
    "from summed.analysis.configurations import DETECT_SENTENCES, CREATE_SUMMARY, CREATE_ABSTRACTIVE_SUMMARY, DETECT_ENTITIES, DETECT_HEALTH_ENTITIES, GLOSSARY_LOOKUP, TRUSTED_SEARCH, CALCULATE_TRUST_SCORE \n",
    "\n",
    "from summed.summed_api import SumMedAPI\n",
    "\n",
    "\n",
    "load_dotenv(\"../.env\")\n",
    "load_dotenv(\"../.env.testing\")\n",
    "\n",
    "import logging\n",
    "logger = logging.getLogger()\n",
    "logger.setLevel(logging.ERROR)\n",
    "\n",
    "SumMed = SumMedAPI(name=\"Test\")\n",
    "\n",
    "# https://www.breastcancer.org/research-news/risk-reducing-effects-of-arimidex-last-years\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Load a medical web article\n",
    "We always need to start with uploading a file into our \"Space\". This way we will have a single source of truth for the raw data that we can always refer back to. The idea is that users have their own spaces for all their documents, which will simplify compliance and data privacy.\n",
    "Once the have the source file stored into our space, we can extract the text and metadata from it into a __Document__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracted text from file 'prevention_causes_risk_factors.html' with title: 'Prevent Breast Cancer - California Protons'\n"
     ]
    }
   ],
   "source": [
    "# The URL to the document we want to analyse\n",
    "url = \"https://www.californiaprotons.com/breast-cancer/prevention-causes-risk-factors/\"\n",
    "\n",
    "# Load it into our \"space\"\n",
    "source_file = SumMed.upload (url)\n",
    "\n",
    "# and then extract it as a new document \n",
    "document = SumMed.extract (source_file)\n",
    "\n",
    "print (f\"Extracted text from file '{document.source_file.filename}' with title: '{document.title}'\" )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Detect sentences\n",
    "One of the first steps for analysis is to identify correct sentence boundaries. \n",
    "Depending on the source, this is almost never \"perfect\", but should catch the most important sentences. that we can then use for further analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Detected 54 sentences in the document: \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[\"SEE HOW WE'RE PROVIDING SAFE IN-PERSON CARE AND TELEMEDICINE APPOINTMENTS\\nLearn More\\nTelemedicine Options Now Available\\nSearch Patient Portal 858.283.4771\\nConditions Treated\\nBreast\\nProstate\\nPediatric\\nLung\\nBrain & Spine\\nBladder\\nGastrointestinal\\nStomach\\nColon\\nRectal\\nAnal\\nPancreatic & Bile\\xa0Duct\\nEsophageal\\nLiver\\nHead & Neck\\nMouth\\nThroat\\nTongue\\nNasal\\nThyroid\\nLymphoma\\nSarcoma\\nTesticular\\nRecurrent & Secondary\\nProton Therapy\\nPatient Support\\nPatient Support Overview\\nPlan Visit\\nCost & Coverage\\nTelemedicine\\nNutrition Services\\nFAQs\\nWhy Us\\nWhy Us\\nAffiliations\\nOur Physicians\\nNews\\nCovid-19 Update\\nPatient Testimonials\\nVirtual Tour\\nRequest Appointment\\nSEE HOW WE'RE PROVIDING SAFE IN-PERSON CARE AND TELEMEDICINE APPOINTMENTS\\nLearn More\\nTelemedicine Options Now Available\\nSearch\\nConditions Treated\\nBreast\\nProstate\\nPediatric\\nLung\\nBrain & Spine\\nBladder\\nGastrointestinal\\nStomach\\nColon\\nRectal\\nAnal\\nPancreatic & Bile\\xa0Duct\\nEsophageal\\nLiver\\nHead & Neck\\nMouth\\nThroat\\nTongue\\nNasal\\nThyroid\\nLymphoma\\nSarcoma\\nTesticular\\nRecurrent & Secondary\\nProton Therapy\\nPatient Support\\nPatient Support Overview\\nPlan Visit\\nCost & Coverage\\nTelemedicine\\nNutrition Services\\nFAQs\\nWhy Us\\nWhy Us\\nAffiliations\\nOur Physicians\\nNews\\nCovid-19 Update\\nPatient Testimonials\\nVirtual Tour\\nPatient Portal\\n858.283.4771\\nRequest Appointment\\n858.283.4771\\nPrevent BreastCancer\\nHome / Breast Cancer / Prevent Breast Cancer\\nOverview\\nDiagnosis\\nTreatments\\nCauses & Risks\\nStages & Grades\\nOnce in your 20s, it’s important to stay on top of your health and perform regular breast exams.\",\n",
       " 'Breast cancer is the most common female cancer worldwide and about 1 in 8 women in the United States will be diagnosed with it sometime in their lifetimes.',\n",
       " 'This percentage increases to 72% for those who have a BRCA1 mutation or BRCA2 mutation.',\n",
       " 'However, breast cancer can be highly curable, especially when caught early.',\n",
       " '\\nCancer is largely caused by mutations or changes to the genetic material found in your cells.',\n",
       " 'Some mutations you’re born with, while others are acquired later in life due to errors in the gene replication process.',\n",
       " 'In most cases, cancer develops when cells cease their normal functions and start to allow rapid growth, fail to stop uncontrolled cell growth or make mistakes when repairing DNA errors.',\n",
       " 'Keep in mind, not all acquired or inherited genetic mutations become cancer.',\n",
       " '\\nWhile doctors have found some correlations between certain health conditions and cases diagnosed, the majority of cancers occur in people without any known risk factors.',\n",
       " 'Your best approach is to manage what you can control, change what you can and stay aware of changes in your health that could be a cause of breast cancer.',\n",
       " '\\nAge: Cancer can develop at any age, but 2 out of 3 invasive breast cancers are found in women 55 or older.',\n",
       " '\\nRace/ethnicity: Ashkenazi Jewish (Eastern European) heritage can put you at greater risk.',\n",
       " 'Caucasian women are slightly more likely to develop breast cancer than those of African American, Hispanic and Asian descent.',\n",
       " 'However, African American women are more likely to develop more aggressive breast cancer at a younger age.',\n",
       " '\\nInherited genetic mutations: BRCA1, BRCA2 and CHEC2 abnormalities can be traced to 5-10% of all cases.',\n",
       " '\\nFamily history: Your genetic risk of breast cancer doubles if you’ve had one first-degree relative—sister, mother or daughter—diagnosed with breast cancer.',\n",
       " 'If you’ve had two first-degree relatives diagnosed, your risk is 5 times higher than average.',\n",
       " 'Blood relatives diagnosed before the age of 50 or with triple-native breast cancer can also increase your risk.',\n",
       " '\\nChildbirth and menstruation cycles: If you haven’t had a full-term pregnancy or your first child before the age of 30, you might have a higher risk of getting breast cancer.',\n",
       " 'Starting your period before the age of 12 and going through menopause after the age of 55 may also increase your risk.',\n",
       " '\\nLifestyle and environmental causes: Smoking, drinking, lack of exercise, a BMI over 25, chronic inflammation, low levels of vitamin D, excessive exposure to the sun or cancer-causing chemicals and having unsafe sex can contribute to the development of cancerous cells.',\n",
       " '\\nThere is no proven strategy for prevention but forming a healthy lifestyle and taking the proper health measures may help reduce your risk and improve your immunity.',\n",
       " 'Recent studies suggest that diet is at least partly responsible for 30 to 40% of all cancers.',\n",
       " '\\nSmoke contains carcinogenic compounds that can cause cancer.',\n",
       " 'It also makes you more prone to treatment-related side effects.',\n",
       " 'If radiation is required, you may face a greater risk of developing lung cancer and/or pneumonitis.',\n",
       " '\\nExercise helps you control blood sugar and limits insulin levels of a hormone that can affect how breast cells grow and develop cancer-causing genetic mutations.',\n",
       " 'Aim for at least 150 minutes a week of moderate aerobic activity or 75 minutes of vigorous aerobic activity.',\n",
       " '\\nFat cells make estrogen, which can stimulate the development and growth of certain types of breast cancer.',\n",
       " 'Extra fat cells can also trigger inflammation in the body, which has been linked to a higher chance of recurrence.',\n",
       " '\\nMaintain a diet rich in fruit and vegetables\\nPick whole grains, lean proteins and non-fat milk and dairy products\\nRemove the skin and fat from meat, poultry and fish\\nLimit sugar, refined carbohydrates and alcohol\\nEat smaller sized portions\\nMore studies are starting to research the environmental causes of breast cancer and impact of certain chemicals found in plastic, cosmetics, food, water and other consumer products.',\n",
       " 'The use of pesticides, antibiotics, and hormones in food production may also play roles.',\n",
       " '\\nEarly detection is still the best way to prevent cancer from growing or spreading.',\n",
       " 'With 5-year survival rates around 100% for Stage 0 and Stage 1 breast cancer, catching it early is key.',\n",
       " '\\nFatigue\\nLump or thickening area felt under the skin\\nUnintended weight loss or gain\\nSkin changes, such as yellowing, darkening or redness of the skin\\nUnexplained bleeding, bruising or sores that won’t heal\\nChanges in bowel or bladder habits\\nPersistent cough, trouble breathing, indigestion or discomfort after eating\\nHoarseness or difficulty swallowing\\nPersistent, unexplained fevers, night sweats or muscle and/or joint pain\\nA family history of breast cancer and/or a known genetic mutation can cause stress and anxiety for many women.',\n",
       " 'Rather than wait and see, some women choose to take a more proactive and aggressive approach to reduce their chances.',\n",
       " 'The most common being the removal of their breast(s) and/or ovaries and suppression of the estrogen hormone linked to cancer cell growth.',\n",
       " 'While there is no guarantee, it’s good to familiarize yourself with the types of preventive treatments available to you.',\n",
       " '\\nAccording to the National Cancer Institute, women who carry a BRCA1 or BRCA2 gene mutation and opt for a preventive mastectomy may be able to reduce their risk of cancer developing by 95%.',\n",
       " 'This percentage drops slightly to 90% for women with a strong family history of breast cancer.',\n",
       " 'During this surgical operation, one or both breasts will be removed to reduce the risk cancer forming in your breast.',\n",
       " 'The amount of tissue removed depends on the type of mastectomy (e.g., total, skin-sparing and nipple-saving) performed.',\n",
       " 'Breast reconstruction may also be an option during or after surgery.',\n",
       " '\\nLearn More\\nThis type of surgery gives women an option who want to lower their risk without causing major visible changes to their body.',\n",
       " 'The removal of your ovaries lowers the amount of estrogen in the body, which helps prevent breast cancers that require estrogen to grow.',\n",
       " 'The National Cancer Institute has found this surgery reduces the number of new cases among high-risk women by around 50%.',\n",
       " 'It’s important to note this type of surgery only benefits pre-menopausal women and eliminates the ability to have children.',\n",
       " '\\nLearn More\\nThis type of treatment is often recommended for those who haven’t been diagnosed but are thought to be at risk for ER- and PR-positive breast cancers.',\n",
       " 'Certain hormones in the blood can cause these cancers to form in the breast tissue.',\n",
       " 'By taking these hormone-blocking drugs, you can lower the production of estrogen in the blood and stop the stimulation of breast cells.',\n",
       " 'Hormone therapy doesn’t affect tumors that are ER- and PR-negative because they’re not influenced by hormones.',\n",
       " '\\nLearn More\\nExams & Diagnosis for Breast Cancer\\nStages & Grades of Breast Cancer\\nTreatment Options for Breast Cancer\\nBlog\\nProton Therapy Treatment FAQs\\nGlossary of Terms\\nDownload Guide\\nCareers\\nContact Us\\nPlan Your Visit\\n9730 Summers Ridge Rd San Diego, CA 92121\\n858.549.7400\\nM–F 8am-5pm PT\\n© 2022 California Protons Cancer Therapy Center Privacy Policy Terms & Conditions\\nEnglish\\nAfrikaans Albanian Amharic Arabic Armenian Azerbaijani Basque Belarusian Bengali Bosnian Bulgarian Catalan Cebuano Chichewa Chinese (Simplified) Chinese (Traditional) Corsican Croatian Czech Danish Dutch English Esperanto Estonian Filipino Finnish French Frisian Galician Georgian German Greek Gujarati Haitian Creole Hausa Hawaiian Hebrew Hindi Hmong Hungarian Icelandic Igbo Indonesian Irish Italian Japanese Javanese Kannada Kazakh Khmer Korean Kurdish (Kurmanji) Kyrgyz Lao Latin Latvian Lithuanian Luxembourgish Macedonian Malagasy Malay Malayalam Maltese Maori Marathi Mongolian Myanmar (Burmese) Nepali Norwegian Pashto Persian Polish Portuguese Punjabi Romanian Russian Samoan Scottish Gaelic Serbian Sesotho Shona Sindhi Sinhala Slovak Slovenian Somali Spanish Sudanese Swahili Swedish Tajik Tamil Telugu Thai Turkish Ukrainian Urdu Uzbek Vietnamese Welsh Xhosa Yiddish Yoruba Zulu\\nFirst Name*\\nLast Name*\\nEmail Address*\\nPhone Number*\\nHow did you hear about us?*\\nMessage\\nSubmit\\nFirst Name*\\nLast Name*\\nTitle*\\nIf other, please specify\\nAffiliation*\\nStreet Address\\nAddress Line 2\\nCity\\nState\\nZIP Code\\nEmail*\\nSpecialty\\nIf other, please specify\\nHow did you hear about this activity?',\n",
       " '\\nBrochure / Postcard in the Mail\\nEmail Announcement\\nPersonal Letter / Invitation\\nDirector / Supervisor\\nOther\\nIf other, please specify\\nPhone\\nThis field is for validation purposes and should be left unchanged.',\n",
       " '\\nSubmit\\n']"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Detect the sentence boundaries in the document\n",
    "document = SumMed.analyze (document, DETECT_SENTENCES)\n",
    "\n",
    "print (f\"Detected {len(document.sentences)} sentences in the document: \")\n",
    "document.sentences"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate a summary\n",
    "First step is to create an so called ___extractive summary___. This is basically a ranked list of the most important sentences from the document. Think of it like going with a text marker over the document, and marking the key sentences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here are the 5 most important sentences, which we think contain the essence of the document: \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['Breast cancer is the most common female cancer worldwide and about 1 in 8 women in the United States will be diagnosed with it sometime in their lifetimes.',\n",
       " 'However, breast cancer can be highly curable, especially when caught early.',\n",
       " 'Caucasian women are slightly more likely to develop breast cancer than those of African American, Hispanic and Asian descent.',\n",
       " '\\nChildbirth and menstruation cycles: If you haven’t had a full-term pregnancy or your first child before the age of 30, you might have a higher risk of getting breast cancer.',\n",
       " 'The removal of your ovaries lowers the amount of estrogen in the body, which helps prevent breast cancers that require estrogen to grow.']"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a summary of the document\n",
    "document = SumMed.analyze (document, CREATE_SUMMARY)\n",
    "\n",
    "print (f\"Here are the {len(document.summary)} most important sentences, which we think contain the essence of the document: \")\n",
    "document.summary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate an \"abstractive\" summary\n",
    "The next step can be to create an __abstractive summary__. This is an A.I. generated summary of the original text, e.g. the systems writes a complete new text (not in the document) about what it THINKS is the essence of the document. This is not always perfect, and we will need to optimize it more for medical texts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here's how our A.I. would paraphrase the key ideas in the document: \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[\"Hormone replacement therapy: If you took hormone replacement therapy (HRT) for menopause symptoms for more than five years, you might have an increased risk of breast cancer.\\n\\nThere are three key points in this medical text. The first is that childbirth and menstruation cycles can affect a woman's risk of developing breast cancer. The second is that hormone replacement therapy can also affect a woman's risk of developing breast cancer. The third is that Caucasian women are slightly more likely to develop breast cancer than those of African American, Hispanic and Asian descent.\"]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create n \"abstractive\" (paraphrasing) summary of the document\n",
    "document = SumMed.analyze (document, CREATE_ABSTRACTIVE_SUMMARY)\n",
    "\n",
    "print (f\"Here's how our A.I. would paraphrase the key ideas in the document: \")\n",
    "document.abstractive_summary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Find medical entities within the text\n",
    "___Named Entity Recognition___ (NER) is a way to identify named \"things\" and their categories.\n",
    "SumMed currently support a __\"basic\"__ version, which only detects very common entities (not very useful in our context), \n",
    "and __\"health\"__ entity recognition, which is using Microsoft Azure Text Analytics for Health - a \"paid\" service)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here's a list of 150 different medical 'things' that are mentioned in the text, orderd by frequency: \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['Diagnosis: Malignant neoplasm of breast (22)',\n",
       " 'Diagnosis: Malignant Neoplasms (18)',\n",
       " 'Gender: Woman (12)',\n",
       " 'TreatmentName: Operative Surgical Procedures (5)',\n",
       " 'BodyStructure: Breast (4)',\n",
       " 'FamilyRelation: Family (3)',\n",
       " 'GeneOrProtein: BRCA2 gene (3)',\n",
       " 'GeneOrProtein: BRCA1 gene (3)',\n",
       " 'TreatmentName: Proton Therapy (3)',\n",
       " 'GeneOrProtein: PGR gene (2)',\n",
       " 'GeneOrProtein: ESR1 gene (2)',\n",
       " 'MedicationClass: Hormone Therapy (2)',\n",
       " 'BodyStructure: breast cells (2)',\n",
       " 'TreatmentName: Prophylactic treatment (2)',\n",
       " 'Diagnosis: Inflammation (2)',\n",
       " 'FamilyRelation: Offspring (2)',\n",
       " 'FamilyRelation: First Degree Relative (2)',\n",
       " 'Age: 55 (2)',\n",
       " 'Diagnosis: COVID19 (disease) (2)',\n",
       " 'HealthcareProfession: Physicians (2)',\n",
       " 'HealthcareProfession: Nutrition Services (2)',\n",
       " 'Diagnosis: Testicular dysfunction (2)',\n",
       " 'Diagnosis: Sarcoma (2)',\n",
       " 'Diagnosis: Lymphoma (2)',\n",
       " 'BodyStructure: Thyroid Gland (2)',\n",
       " 'BodyStructure: Nose (2)',\n",
       " 'BodyStructure: Tongue (2)',\n",
       " 'BodyStructure: Pharyngeal structure (2)',\n",
       " 'BodyStructure: Oral cavity (2)',\n",
       " 'BodyStructure: Neck (2)',\n",
       " 'BodyStructure: Head (2)',\n",
       " 'BodyStructure: Liver (2)',\n",
       " 'Diagnosis: Esophageal Diseases (2)',\n",
       " 'BodyStructure: Bile duct structure (2)',\n",
       " 'BodyStructure: Pancreas (2)',\n",
       " 'BodyStructure: Anus (2)',\n",
       " 'BodyStructure: rectal (2)',\n",
       " 'BodyStructure: Colon structure (body structure) (2)',\n",
       " 'BodyStructure: Stomach (2)',\n",
       " 'BodyStructure: gastrointestinal (2)',\n",
       " 'BodyStructure: Urinary Bladder (2)',\n",
       " 'BodyStructure: Vertebral column (2)',\n",
       " 'BodyStructure: Brain (2)',\n",
       " 'BodyStructure: Lung (2)',\n",
       " 'BodyStructure: Prostate (2)',\n",
       " 'TreatmentName: Self-care interventions (2)',\n",
       " 'Gender: Male population group (1)',\n",
       " 'Diagnosis: Neoplasms (1)',\n",
       " 'TreatmentName: Hormone Therapy (1)',\n",
       " 'BodyStructure: Blood (1)',\n",
       " 'MedicationClass: hormone-blocking drugs (1)',\n",
       " 'BodyStructure: Breast tissue sample (1)',\n",
       " 'TreatmentName: Therapeutic procedure (1)',\n",
       " 'Diagnosis: Premenopausal - menopausal status (1)',\n",
       " 'TreatmentName: removal of your ovaries (1)',\n",
       " 'SymptomOrSign: changes to their body (1)',\n",
       " 'TreatmentName: lower their risk (1)',\n",
       " 'TreatmentName: Mammaplasty (1)',\n",
       " 'TreatmentName: nipple-saving (1)',\n",
       " 'TreatmentName: skin-sparing (1)',\n",
       " 'TreatmentName: Total (1)',\n",
       " 'TreatmentName: Mastectomy (1)',\n",
       " 'TreatmentName: removal technique (1)',\n",
       " 'Direction: both (1)',\n",
       " 'ExaminationName: Percent (qualifier value) (1)',\n",
       " 'TreatmentName: Prophylactic Mastectomy (1)',\n",
       " 'TreatmentName: Visual Suppression (1)',\n",
       " 'BodyStructure: Ovary (1)',\n",
       " 'TreatmentName: removal of their breast(s) (1)',\n",
       " 'TreatmentName: reduce their chances (1)',\n",
       " 'TreatmentName: Entity Risk - aggressive (1)',\n",
       " 'SymptomOrSign: Anxiety (1)',\n",
       " 'SymptomOrSign: Stress (1)',\n",
       " 'SymptomOrSign: Arthralgia (1)',\n",
       " 'SymptomOrSign: Muscle Tissue (1)',\n",
       " 'SymptomOrSign: Night sweats (1)',\n",
       " 'SymptomOrSign: Fever (1)',\n",
       " 'SymptomOrSign: Deglutition Disorders (1)',\n",
       " 'SymptomOrSign: Hoarseness (1)',\n",
       " 'SymptomOrSign: discomfort after eating (1)',\n",
       " 'SymptomOrSign: Dyspepsia (1)',\n",
       " 'SymptomOrSign: Dyspnea (1)',\n",
       " 'SymptomOrSign: Coughing (1)',\n",
       " 'SymptomOrSign: Abnormal urination (1)',\n",
       " 'SymptomOrSign: Changes in bowel (1)',\n",
       " 'SymptomOrSign: Sore to touch (1)',\n",
       " 'SymptomOrSign: Contusions (1)',\n",
       " 'SymptomOrSign: Hemorrhage (1)',\n",
       " 'SymptomOrSign: Erythema (1)',\n",
       " 'SymptomOrSign: darkening (1)',\n",
       " 'SymptomOrSign: Yellow color (1)',\n",
       " 'SymptomOrSign: Skin symptom change (1)',\n",
       " 'SymptomOrSign: Gain (1)',\n",
       " 'SymptomOrSign: Weight decreased (1)',\n",
       " 'BodyStructure: Skin (1)',\n",
       " 'Direction: under (1)',\n",
       " 'SymptomOrSign: thickening area (1)',\n",
       " 'SymptomOrSign: Mass of body structure (1)',\n",
       " 'SymptomOrSign: Fatigue (1)',\n",
       " 'ExaminationName: Five-year Survival Rate (1)',\n",
       " 'Diagnosis: Role Playing DIsorder (1)',\n",
       " 'MedicationClass: Antibiotics (1)',\n",
       " 'TreatmentName: Alcohols (1)',\n",
       " 'TreatmentName: Refined carbohydrate (1)',\n",
       " 'TreatmentName: Limit sugar (1)',\n",
       " 'TreatmentName: Fish (substance) (1)',\n",
       " 'TreatmentName: Poultry preparation (1)',\n",
       " 'TreatmentName: Remove the skin and fat from meat (1)',\n",
       " 'TreatmentName: diet rich in fruit and vegetables (1)',\n",
       " 'BodyStructure: Structure of body of caudate nucleus (1)',\n",
       " 'BodyStructure: Extra fat cells (1)',\n",
       " 'GeneOrProtein: estrogen (1)',\n",
       " 'BodyStructure: Adipocytes (1)',\n",
       " 'TreatmentName: vigorous aerobic activity (1)',\n",
       " 'TreatmentName: moderate aerobic activity (1)',\n",
       " 'Frequency: a week (1)',\n",
       " 'MedicationName: Insulin (1)',\n",
       " 'SymptomOrSign: Blood Glucose (1)',\n",
       " 'Diagnosis: Pneumonitis (1)',\n",
       " 'Diagnosis: Malignant neoplasm of lung (1)',\n",
       " 'TreatmentName: Therapeutic radiology procedure (1)',\n",
       " 'SymptomOrSign: Adverse effects (1)',\n",
       " 'TreatmentName: Measures (attribute) (1)',\n",
       " 'Diagnosis: Tumor cells, malignant (1)',\n",
       " 'MedicationName: vitamin D (1)',\n",
       " 'ExaminationName: Body mass index (1)',\n",
       " 'Diagnosis: Menopause (1)',\n",
       " 'Age: 12 (1)',\n",
       " 'Age: 30 (1)',\n",
       " 'Diagnosis: Pregnancy (1)',\n",
       " 'SymptomOrSign: menstruation cycles (1)',\n",
       " 'Diagnosis: Childbirth (1)',\n",
       " 'Age: 50 (1)',\n",
       " 'FamilyRelation: Biological Relative (1)',\n",
       " 'FamilyRelation: Daughter (1)',\n",
       " 'FamilyRelation: Mother (person) (1)',\n",
       " 'FamilyRelation: Sister (1)',\n",
       " 'GeneOrProtein: CHEC2 (1)',\n",
       " 'Age: Youth (1)',\n",
       " 'SymptomOrSign: changes in your health (1)',\n",
       " 'Diagnosis: Health conditions (1)',\n",
       " 'HealthcareProfession: doctors (1)',\n",
       " 'GeneOrProtein: DNA (1)',\n",
       " 'SymptomOrSign: Cell (compartment) (1)',\n",
       " 'SymptomOrSign: Replicon (1)',\n",
       " 'BodyStructure: Cells (1)',\n",
       " 'GeneOrProtein: Genetic Materials (1)',\n",
       " 'ExaminationName: regular breast exams (1)',\n",
       " 'Age: 20s (1)',\n",
       " 'TreatmentName: PREVENT (product) (1)']"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Detect the healt-related named entities in the document\n",
    "document = SumMed.analyze (document,DETECT_HEALTH_ENTITIES)\n",
    "\n",
    "\n",
    "print (f\"Here's a list of {len(document.health_entities)} different medical 'things' that are mentioned in the text, orderd by frequency: \")\n",
    "[f\"{e.label}: {e.text} ({e.count})\" for e in document.health_entities]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Find trusted information\n",
    "Our trusted web search will look for highly relevant online articles that are closely rfelated to our document and the key topics in it. Only known, trusted websites (such as from renowned medical institutes) are considered."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Hormone Replacement Therapy - What You Need to Know (https://www.drugs.com/cg/hormone-replacement-therapy.html) ',\n",
       " 'Hormone Replacement Therapy | HRT | Menopause | MedlinePlus (https://medlineplus.gov/hormonereplacementtherapy.html) ',\n",
       " 'Hormone therapy: Is it right for you? - Mayo Clinic (https://www.mayoclinic.org/diseases-conditions/menopause/in-depth/hormone-therapy/ART-20046372) ',\n",
       " 'Hormone Replacement Therapy for Menopause - Healthline (https://www.healthline.com/health/menopause/hormone-replacement-therapy-menopause) ',\n",
       " 'Hormone Replacement Therapy - WebMD (https://www.webmd.com/menopause/features/hormone-replacement-therapy) ',\n",
       " 'Hormone replacement therapy after breast cancer: Yes, No or maybe? (https://pubmed.ncbi.nlm.nih.gov/33508379/) ',\n",
       " 'Try This: 36 Alternatives to Hormone Replacement Therapy (HRT) (https://www.healthline.com/health/menopause/alternatives-to-hrt) ']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Lookup web articles from trusted sources that are highly related to the document \n",
    "document = SumMed.analyze (document, TRUSTED_SEARCH)\n",
    "\n",
    "[f\"{x.name} ({x.url}) \" for x in document.search_results]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build a glossary of medical terms\n",
    "We can add unknown terms to the documents \"glossary\" section, and let SumMed find an definition / explanation from an external medical terms dictionary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here's aglossary for selected medical terms in the document: \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['cancer:   not yet implemented',\n",
       " 'Hemorrhage:   not yet implemented',\n",
       " 'Erythema:   not yet implemented']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We add a few terms to the glossary, and then SumMed will look up and fill in their explanations\n",
    "document.glossary = {}\n",
    "document.glossary[\"cancer\"] =  TermExplanation(term=\"cancer\")\n",
    "document.glossary[\"Hemorrhage\"] = TermExplanation(term=\"Hemorrhage\") \n",
    "document.glossary[\"Erythema\"] = TermExplanation(term=\"Erythema\")\n",
    " \n",
    "document = SumMed.analyze (document, GLOSSARY_LOOKUP)\n",
    "\n",
    "print (f\"Here's aglossary for selected medical terms in the document: \")\n",
    "[f\"{k}:   {v.explanation}\" for k,v in (document.glossary).items()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cleanup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SumMed.space.delete_all_files ()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "4c6215dfdde890b0c92d28e5f5d04ada27e42c51ca17043b19be22459f51c8d1"
  },
  "kernelspec": {
   "display_name": "Python 3.9.1 ('.venv': venv)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
