{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SumMed Demo - Multi-language suppport (Portuguese -> German + Russian)\n",
    "\n",
    "This notebook demonstrate a scenario where the patients preferred language is either **German** or **Russian**, while the document is available in **Portuguese**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"summed_logo.png\" width=600/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. The source material\n",
    "Two portuguese websites completely dedicated to hereditary cancer, all the webpages of these websites are dedicated to HC \n",
    "\n",
    "- http://www.cancronafamilia.org\n",
    "-  http://www.cancrohereditario.pt/pt/\n",
    "\n",
    "A Therapeutic Guidance Guide for people affected by hereditary cancer developed by the Portuguese Institute of Oncology \n",
    "- http://www.ipoporto.pt/dev/wp-content/uploads/2018/12/Normas-de-orienta%C3%A7%C3%A3o-Oncogen%C3%A9tica.pdf \n",
    "\n",
    "Webpage from Evita's website that includes support materials such as leaflets about oncofertility, lymphedema, BRCA mutations, advice on how women can take proper care of themselves while undergoing radiotherapy or after having a mastectomy \n",
    "\n",
    "- https://www.evitacancro.org/cancro-hereditario/apoio-internacional/ \n",
    "\n",
    "\n",
    "\n",
    "Websites of some portuguese medical societies that may be relevant:\n",
    "- Portuguese Society of Senology - https://www.spsenologia.pt/ \n",
    "- Portuguese Society of Human Genetics - http://spgh.net/ \n",
    "- Portuguese Society of Gynecology - https://spginecologia.pt/ \n",
    "- Portuguese Association of Cancer Research - http://www.aspic.pt/ \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dotenv import load_dotenv\n",
    "from summed.data import PlatformConfig, AnalysisConfig, FileInfo, DocumentSource, Document, SearchResult, TermExplanation\n",
    "from summed.analysis.configurations import DETECT_SENTENCES, CREATE_SUMMARY, CREATE_ABSTRACTIVE_SUMMARY, DETECT_ENTITIES, DETECT_HEALTH_ENTITIES, GLOSSARY_LOOKUP, TRUSTED_SEARCH, CALCULATE_TRUST_SCORE, PREPROCESS_TEXT, PROFILE_BASIC, PROFILE_FULL, TRANSLATE_TEXT \n",
    "\n",
    "from summed.summed_api import SumMedAPI\n",
    "\n",
    "\n",
    "load_dotenv(\"../.env\")\n",
    "load_dotenv(\"../.env.testing\")\n",
    "\n",
    "import logging\n",
    "logger = logging.getLogger()\n",
    "logger.setLevel(logging.ERROR)\n",
    "\n",
    "SumMed = SumMedAPI(name=\"Test\")\n",
    "\n",
    "# http://www.ipoporto.pt/dev/wp-content/uploads/2018/12/Normas-de-orienta%C3%A7%C3%A3o-Oncogen%C3%A9tica.pdf\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Load the document\n",
    "Upload a PDF document into our __Space__, and then extract the initial __Document__.\n",
    "We get some initial metadata and the extracted text, ready for further processing.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The filename of the  document we want to analyse\n",
    "from pathlib import Path\n",
    "file_name = Path (\"./unittests/data/normas-de-orientação-oncogenética.pdf\").resolve()\n",
    "\n",
    "# Load it into our \"space\"\n",
    "source_file = SumMed.upload (file_name)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracted text from file 'normas_de_orienta_C3_A7_C3_A3o_oncogen_C3_A9tica.pdf' with title: 'None'\n"
     ]
    }
   ],
   "source": [
    "# and then extract it as a new document \n",
    "document = SumMed.extract (source_file)\n",
    "\n",
    "print (f\"Extracted text from file '{document.source_file.filename}' with title: '{document.title}'\" )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Translate into English\n",
    "For now, the best we can do is translating the text into english first. This actually requires two processing steps on the _Document__:\n",
    "1. PREPROCESS the text: this will do language independent cleanup (e.g. normalizing whitespace or hypenation)\n",
    "2. DETECT_SENTENCES: find sentence boundaries in original language (pt). \n",
    "3. TRANSLATE to english\n",
    "4. DETECT_SENTENCES again (this time specialized english sentencizer)\n",
    "5. CREATE_SUMMARY\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of original text : 31628\n",
      "text starts with: 'IPOPORTO INSTITUTO PORTUGUÊS DE ONCOLOGIA DO PORTO FRANCISCO GENTIL, EPE Guia de Orientação Terapêutica CANCRO HEREDITÁRIO Para cuidar de si! Ficha Técnica: Edição: IPO - Porto Propriedade: IPO - Port'\n",
      "Detected 9 sentences in the document (first 3): ['ções de rastreio e/ou redução do risco.', 'cia de cancro hereditário não está descartada.', 'ta a história familiar (incluindo os indivíduos saudáveis).']\n",
      "Detected 9 sentences in the document (first 3): ['screening and/or risk reduction.', 'hereditary cancer is not ruled out.', 'family history (including healthy individuals).']\n"
     ]
    }
   ],
   "source": [
    "# Detect the sentence boundaries in the document\n",
    "#document.language = \"pt\"\n",
    "document = SumMed.analyze (document, PREPROCESS_TEXT)\n",
    "print (f\"Length of original text : {len(document.text)}\")\n",
    "print (f\"text starts with: '{document.text[:200]}'\")\n",
    "print (f\"Detected {len(document.sentences)} sentences in the document (first 3): {document.sentences[:3]}\")\n",
    "\n",
    "\n",
    "document = SumMed.analyze (document, TRANSLATE_TEXT[\"en\"])\n",
    "print (f\"Detected {len(document.sentences)} sentences in the document (first 3): {document.sentences[:3]}\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Detected 50 sentences in the document (first 3): ['Genetic counselling helps people make informed decisions about whether the tests available are useful, both for themselves and for their families.', 'The two most common hereditary predisposition syndromes for cancer are hereditary breast and ovarian cancer and Lynch syndrome (hereditary colorectal cancer without polyposis, HNPCC).', '2- Genetic tests Genetic tests of hereditary cancer are not \"just another blood test\" nor are they suitable for the general population.']\n"
     ]
    }
   ],
   "source": [
    "document = SumMed.analyze (document, DETECT_SENTENCES)\n",
    "\n",
    "print (f\"Detected {len(document.sentences)} sentences in the document (first 3): {document.sentences[:3]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'IPOPORTO PORTUGUESE INSTITUTE OF ONCOLOGY OF PORTO FRANCISCO GENTIL, EPE Guidance Therapeutic Cancer HEREDITARY To take care of yourself! Fact Sheet: Edition: IPO - Port Property: IPO - Porto Texts: IPO - Porto Photography: Median - Global Communication and IPO - Port Design and Production: Median - Global Communication 2 HEREDITARY CANCER Hereditary Cancer 1- Genetic Counseling Genetic counseling is recommended to individuals with a history of cancer, personal and family, who suggest hereditary syndrome. The aim of genetic counseling is to enable individuals to better understand hereditary cancer, understand their own risks of developing cancer, and know the various screening and/or risk reduction options. Genetic counselling helps people make informed decisions about whether the tests available are useful, both for themselves and for their families. Genetic counseling includes: detailed review of family history, paying special attention to those who have already suffered from cancer; assistance with the collection of relevant medical records in order to provide accurate risk assessment; explanation of the differences between sporadic cancers (they occur by chance in the population, so all are at the same risk) and hereditary cancers (which appear in certain families and may be associated with a mutation inherited from a specific gene); interpretation of the pattern or patterns of cancer in family history – some people may realize that the risk of getting cancer is lower than they thought and others that the risk is higher; discussion of the possibility of performing genetic tests and, if indicated, who is the best index case of the family; review of the procedure of a genetic test, if available, including possible outcomes, what potential impact on family dynamics and other risks, benefits and limitations; help inform family members; recommendations for the screening and/or prevention of cancers for which the consul may have a higher risk. Genetic counseling is indicated in individuals from families with hereditary predisposition to cancer with a mutation already identified, as well as in individuals with a family history suggestive of hereditary predisposition to cancer on the paternal or maternal side (in close relatives on the same side of the family), and in this case the genetic study of the family should be initiated by an affected family member at a young age. Generally, hereditary predisposition to cancer should be suspected when: two or more close relatives have the same type of cancer; there is a pattern of cancer in at least two generations; there are cancers diagnosed at a younger age than usual in the general population; multiple primary neoplasms are diagnosed in the same individual; there is an unusual association of cancers in the family; there is a pattern of cancers associated with a known hereditary syndrome. The two most common hereditary predisposition syndromes for cancer are hereditary breast and ovarian cancer and Lynch syndrome (hereditary colorectal cancer without polyposis, HNPCC). 2- Genetic tests Genetic tests of hereditary cancer are not \"just another blood test\" nor are they suitable for the general population. There are eligibility criteria established to ensure a coherent and equitable approach. The objective, within the scope of current resources, is to maximize the availability of appropriate genetic tests that have an impact on the medical management of individuals and their families. In order to provide informed consent, genetic counselling is required before possible collection of blood samples for genetic testing of hereditary predisposition to cancer. A genetic test can be a long and complicated laboratory process. Once the eligibility criteria have been verified, there are two main approaches to testing a specific hereditary cancer gene: index test or carrier test. Index test first test to the gene(s) in the family; usually recommended to those who have had cancer (urgent storage of DNA should be carried out when there is a concern about the health status of a potential case index of a family); the goal is to confirm whether a particular gene is associated with the pattern of cancer in the family; usually involves testing whole gene(s) in order to try to identify a mutation that interferes with IPOPORTO 3 with the function of the gene itself; the result may take several months; the test can confirm the existence of hereditary cancer, but it cannot be ruled out: • if a pathogenic mutation is identified, hereditary cancer syndrome is confirmed and carrier tests can be performed on other family members. The children or siblings of the index case in most situations have a 50% chance of having inherited the same mutation; • if a mutation is not identified, the existence of hereditary cancer is not ruled out. It is possible that there is a mutation that is not detectable with current technology or there may be another gene associated with the family history of cancer. In some cases, it may be indicated to repeat the test on another affected family member; • 10-15% of the results have undefined clinical significance (variants of unknown significance). Carrier testing is only available when a specific mutation of a gene is already confirmed in the family; tests only for the presence/absence of this specific mutation; the result confirms the carrier status and associated cancer risks, but cannot predict when or if the individual will develop cancer: • positive result: the person has inherited a high risk for developing specific types of cancer; • negative result: the person did not inherit a high risk, presenting the risk of the general population; if possible, the application for genetic counseling should include a copy of a family member\\'s genetic test report. 3- Hereditary Breast and Ovarian Cancer (HBOC) Breast or ovarian cancer can be classified as sporadic (70-75%), multifactorial (20-25%), or hereditary (5-10%). Only persons with a personal or family history suggesting hereditary predisposition should be referred for genetic counseling and possible genetic testing. 4 HEREDITARY CANCER Hereditary cancer of the breast and ovary there are currently two known genes associated with hereditary breast and ovarian cancer: BRCA1 and BRCA2; approximately 1:1000 to 1:500 people in the general population are born with an inherited mutation in the BRCA1 or BRCA2 gene; the frequency of mutation in one of these genes is about 1:40 in people with Ashkenazi Jewish ancestry; hereditary breast and ovarian cancer has an autosomadominant transmission; genetic counseling is made available to all individuals and families who meet the benchmarking criteria. Genetic testing for hereditary breast and ovarian cancer is clinically available to individuals who meet the specific criteria. Reference criteria The reference criteria for genetic counseling for suspected hereditary breast and ovarian cancer are (alternatively): three cases of breast cancer in first-degree (or 2nd degree) relatives of the same branch of the family, one of them diagnosed before the age of 50; two cases of breast cancer at any age, provided that there is a case of ovarian carcinoma in the same branch of the family; a case of breast cancer before the age of 45 and ovarian cancer at any age; bilateral breast cancer before the age of 50 and a family member with breast or ovarian cancer; two relatives with breast cancer before the age of 40 or one of them with ovarian cancer before the age of 50; breast or ovarian cancer before the age of 30; breast cancer before the age of 40 with the histological diagnosis of poorly differentiated, medullary breast carcinoma or with negative hormonal and HER2 receptors; breast cancer in men; individual belonging to the family with identified pathogenic mutation. Following the genetic counselling consultation, the probability of a germ line in the BRCA1 and BRCA2 genes is calculated taking into account family history (including healthy individuals). The genetic test of the BRCA1 and BRCA2 gene index case tests will be offered if the probability of mutation is equal to or greater than 10%. In cases that meet the referencing criteria but with a probability of mutation of less than 10%, research of the most common BRCA1 and BRCA2 gene mutations in Portugal may be offered. Cancer risk Brca1 or BRCA2 mutated individuals have a higher lifetime risk for cancer compared to the general population, as shown in the table below. Risks and ages at diagnosis vary between individuals and families. Cancer Type Risk general population Risk carrier mutation BRCA1 Carrier risk BRCA2 mutation genetic test the BRCA1 and BRCA2 gene testing is available to individuals/families who meet the eligibility criteria; the carrier test is offered to individuals in a family in which a pathogenic mutation in the BRCA1 or BRCA2 gene has already been identified; the index test in a family usually begins with a blood sample of: • a woman with ovarian cancer, or • a woman with breast cancer diagnosed at a younger age, or • a man with breast cancer. possible results of the index test for BRCA1/BRCA2: • positive: a pathogenic mutation is identified in the BRCA1 or BRCA2 gene. Family members are eligible for the carrier test for the specific family mutation; • negative: a mutation in the BRCA1 and BRCA2 genes is not identified. Carrier testing is not possible for family members. It does not confirm, but also does not exclude, the presence of hereditary predisposition in the family; • variant of unknown significance: when there is a change in a gene whose clinical significance is not yet established, and may or may not be related to a higher risk of cancer. Carrier testing is not possible for family members. It does not confirm, but also does not exclude, the presence of hereditary predisposition in the family. Additional studies (segregation or functional) are required. Cancer screening and prophylaxis For people with BRCA1 or BRCA2 mutation (or who have a 50% probability of inheriting the known mutation in the family), the current recommendations for screening are: Women: • annual mammography, starting at 25 years or 5-10 years before the youngest diagnostic age of breast cancer in the family; • breast MAGNETIC RESONANCE, starting at 25 years or 5-10 years before the youngest diagnostic age of breast cancer in the family (scheduled to alternate with mammography every 6 months); • careful clinical examination of the breasts and regional lymph nodes every six months, simultaneously with imaging examinations; • monthly breast self-examination (individual option). Men: • regular physical examination; • special attention to any change in the chest/ breasts; • screening for prostate cancer following the recommendations for the general population, unless family history suggests high risk. Currently not recommended: • mammography in men; • ovarian cancer screening due to lack of evidence that it is effective. Women with pathogenic mutation in the BRCA1 or BRCA2 gene may also consider prophylactic surgery options: IPOPORTO 5 Mama, woman 11% 50-85% 50-85% Breast, man 0.1% <1% 6% Ovary 1-2% 40% 20% Prostate 12% 24-36% 24-36% bilateral prophylactic mastetomia with breast reconstruction: • associated with a reduction of at least 90% of the risk of breast cancer; • it can be a complex personal decision due to the potential long-term impact on body image, sexuality and other aspects related to quality of life; • the removed breast tissue should be analysed by a specialist pathologist to exclude cancer. bilateral prophylactic salpingo-oophorectomy: • should be strongly weighed, given the lack of effective screening for ovarian cancer; • excised ovaries and fallopian tubes should be examined by a specialist pathologist to exclude cancer; • in addition to reducing the risk of ovarian and fallopian tube cancer, bilateral prophylactic salpingo-oophorectomy reduces the risk of breast cancer by 50% if performed before menopause; • is most effective if it is carried out between the age of 35-40, and/or when the reproductive project is finished; • the impact of surgical menopause is important in the decision-making process and on postoperative surveillance. Barring some contraindication, hormone replacement therapy is usually an important component in the management of early menopause symptoms. Chemoprevention options should also be considered. Some studies have shown that oral contraceptives reduce the risk of ovarian cancer, which may be an important possibility for women with brca1 or BRCA2 mutation of childbearing age. Tamoxifen is not currently prescribed to reduce the risk of breast cancer in healthy women with germ mutation, unless clinical trials are in clinical trials. A woman with a BRCA1 or BRCA2 mutation diagnosed with breast cancer has a risk of about 30% in 10 years for contralateral breast cancer. Therefore, screening or prophylaxis measures for remaining breast and/or contralateral breast cancer are recommended, in addition to possible prophylaxis of ovarian cancer. 6 HEREDITARY CANCER 4- Lynch syndrome (Hereditary Colorectal Cancer) Colorectal cancer can be classified as sporadic (70-75%), multifactorial (20-25%), or hereditary (5-10%). Only persons with a personal or family history suggesting hereditary predisposition should be referred for genetic counseling and possible genetic testing. Lynch syndrome the terms Lynch syndrome and hereditary colorectal cancer without polyposis (HNPCC) have both been used for the same disease, but the term Lynch syndrome is currently considered the most correct; Lynch syndrome accounts for 2 to 5% of all cases of colorectal cancer; four genes associated with Lynch syndrome are known: MLH1, MSH2, MSH6 and PMS2. All of them are involved in DNA repair; Lynch syndrome has an autosomadominant transmission; genetic counseling is made available to all individuals and families who meet the benchmarking criteria. Genetic testing for Lynch syndrome is clinically available to individuals who meet the specific criteria. Benchmarking criteria There are two types of referencing criteria: the Amsterdam criteria and the Bethesda criteria. The Amsterdam criteria are currently defined as (cumulatively): at least three relatives with cancers associated with Lynch syndrome (colorectal carcinoma or endometrial carcinoma, small intestine, ureter or bacinet); one patient should be first-degree familiar with the other two; at least two successive generations affected; at least one cancer diagnosed before the age of 50; family adenomatous polyposis should be excluded in cases of colorectal cancer; tumors should be checked by pathological examination. Bethesda criteria include (alternatively): colorectal cancer diagnosed before the age of 50; synchronous or metachronous tumors of the lynch syndrome spectrum (colorectal carcinoma or endometrial carcinoma, stomach, small intestine, ovary, pancreas, biliary tract, ureter or bacinete, brain tumor, adenomas or carcinomas of the sebaceous glands, or keratoacantomas) regardless of age; colorectal carcinoma with phenotype associated with microsatellite instability (lymphocytic infiltration, Crohn\\'s-like lymphocytic reaction, mucinous or syncet ring cell differentiation, medullary growth pattern) diagnosed before 60 years; colorectal cancer patient with at least one first-degree family member with colorectal cancer or other lynch syndrome spectrum neoplasm diagnosed before the age of 50 (or adenoma before the age of 40); colorectal cancer patient with at least two first or second-degree relatives with colorectal cancer or other Lynch syndrome spectrum neoplasm diagnosed at any age. Families who meet the Amsterdam criteria have indication for genetic counseling and germ change research. Patients who do not meet the Amsterdam criteria but meet the Bethesda criteria have indication for a tumor pre-test involving microsatellite instability research or immunohistochemical analysis of MLH1, MSH2, MSH6 and PMS2 proteins, and germ mutations are investigated if this analysis is positive. Individuals belonging to the family with identified pathogenic mutation should also be referenced. Cancer risk Individuals with Lynch syndrome have a higher lifetime risk for cancer compared to the general population, as shown in the table below. Risks and ages at diagnosis vary between individuals and families. Type of cancer Risk general population Risk Lynch Colorectal syndrome 6% up to 80% Endometrium 2% 30-60% Ovary 1-2% 10-12% Stomach, hepatobiliary tract, 12% 24-36% Ureter and bacinete, small intestine, <2% <10% brain People with Lynch syndrome may also develop colorectal adenomas or adenomas of the sebaceous glands. Genetic testing available for individuals/families who meet the eligibility criteria; Carrier testing is available to individuals with mutation already identified in the family; the index test should start preferably by: • individual diagnosed with colorectal cancer at a younger age in the family, or • individual with multiple primary tumors. Normally, the tissue of the colorectal tumor of the index case for microsatellite instability and/or immunohistochemical analysis of the proteins MLH1, MSH2, MSH6 and PMS2 is analyzed. If the tumor tissue result is normal (no high microsatellite instability and/or no changes in protein expression), the likelihood of Lynch syndrome is low and screening based on family history of cancer is recommended. Approximately 90% of lynch syndrome colorectal tumors present microsatellite instability when there is a germ mutation in the MLH1, MSH2, MSH6 or PMS2 genes. If the result of tumor tissue suggests Lynch syndrome: a germ line is searched in a blood sample; possible index test results: • positive: a pathogenic mutation is identified in a gene associated with Lynch syndrome. Family members are eligible for carrier testing for the specific family mutation; • negative: a pathogenic mutation is not identified in genes associated with Lynch syndrome. Carrier testing is not possible for family members. It does not confirm, but also does not exclude, the presence of hereditary predisposition in the family; • variant of unknown significance: when there is a change in a gene whose clinical significance is not yet established, and may or may not be related to a higher risk of cancer. Carrier testing is not possible for family members. It does not confirm, but also does not exclude, the presence of hereditary predisposition in the family. Additional studies (segregation or functional) are required. IPOPORTO 7 Cancer screening and prophylaxis For people with Lynch syndrome and identified mutation of the MLH1, MSH2, MSH6 or PMS2 genes (or who have a 50% probability of inheriting the known mutation in the family), the current recommendations for screening and prophylaxis are: colonoscopy: • onset at 25 years (or 5-10 years before the youngest diagnostic age in the family); • repeat every 1-2 years until the age of 40 and then annually; • it has been shown in the scientific literature that this screening protocol significantly reduces the incidence of colorectal cancer in people with Lynch syndrome; annual gynecological examination, as well as immediate assessment of any unusual bleeding between menstrual periods or after menopause; hysterectomy and prophylactic bilateral salpingo-oophorectomy should be considered after 35-40 years or when the reproductive project is finished; Annual endometrial biopsy with transvaginal ultrasonography can be considered: • onset at 25-35 years if family history includes endometrial cancer; • onset at 35 years of age if family history does not include endometrium cancer; • the effectiveness of this approach has not yet been proven; there is no screening recommendation for other types of cancer associated with Lynch syndrome because its efficacy is controversial. 5- Other Family Adenomatous Polypose Hereditary Cancer Syndromes (FAP) is the most common hereditary polyposis syndrome, accounting for up to 1% of all colorectal cancers; associated with germ line mutations in the APC gene; autosomic dominant transmission (20-25% of new mutations); characteristic of >100 colorectal adenomatous polyps, typically diagnosed before the age of 40; 8 HEREDITARY CANCER progression to colorectal cancer is inevitable in the absence of prophylactic cholectomy; Additional features may include: gastric polyps, duodenal adenomatous polyps, osteomas, dental anomalies (e.g. supernumerary teeth), congenital hyperplasia of the pigmented epithelium of the retina, soft tissue tumors (epidermoid cysts and fibromas), dismoid tumors, and other associated cancers; there is an attenuated variant of FAP characteristic of: • 10-100 colorectal adenomas, with a tendency to save the rethane; • later onset polyposis; • diagnosis of later colorectal cancer; • the penetrance for lifelong colorectal cancer is high but not complete; • more limited expression of extraintestinal manifestations of FAP; • gastric and duodenal adenomas are common; Gardner syndrome: fap variant with association of colorectal adenomatous polyposis with osteomas and soft tissue tumors (epidermoid cysts, fibromas and desoid tumors); Turcot syndrome: fap variant with association of colorectal adenomatous polyposis with brain tumors, usually medulloblastoma. The reference criteria are: family member with pathogenic mutation identified in the APC gene (from 12 years); personal (or close family history) with clinical diagnosis of FAP: • a large number of colorectal adenomatous polyps (histopathological reporting is required); • additional features may include: gastric polyps, duodenal adenomatous polyps, osteomas, dental anomalies, congenital hyperplasia of the pigmented epithelium of the retina, soft tissue tumors, desmoid tumors and other associated cancers; Attenuated FAP is suggested by personal (or close family history) of 10 or more histologically confirmed adenomatous polyps. MUTYH-associated polyposis (MAP) associated with biallelic mutations in the MUTYH gene (MYH alias); autosomic recessive transmission; characteristicized by multiples (usually 15-100, but may be >100) colorectal adenomas, usually diagnosed after 40 years; may be difficult to distinguish from FAP or fap attenuated with de novo mutation; high risk for colorectal cancer, predominantly distal; polyps are usually small, moderately dysplastic, tubular adenomas or tubulo-villous, with some hyperplastic polyps. The reference criteria are: sibling with biallenic mutation identified in the MUTYH gene; personal history (or brother) with more than 15 colorectal polyps (histopathological reporting is required). Hereditary diffuse gastric cancer (HDGC) autososol dominant transmission; associated with germ line mutations in the CDH1 gene; cumulative risk for diffuse gastric cancer (or with sythete ring cells) up to 80 years of 67% in men and 83% in women (mean age of diagnosis 38 years, with interval 14-69 years); women also have a cumulative risk of 39% for lobular breast carcinoma; there may be families with colorectal cancer. The reference criteria are: a family member with pathogenic mutation identified in the CDH1 gene; patient with diffuse gastric cancer diagnosed <35 years; two or more confirmed cases of gastric cancer in the family, at least one of them being a diffuse gastric cancer diagnosed <50 years; three or more confirmed cases of gastric cancer in the family regardless of age, at least one of them being a diffuse gastric cancer. Hereditary paraganglioma/pheochromocytoma (PGL) PGL1: associated with germ mutations in the SDHD gene: • sex-dependent dominant autosochrocytoma transmission, with penetrance only when inherited from the father (imprinting); • mainly associated with head and neck paragangliomas, usually multifocal and non-functional; PGL3: associated with germ line mutations in the SDHB gene: • autosomadominant transmission; • mainly performed by functional paragangliomas in the abdomen, pelvis and thorax (extraadrenal); • increased malignant potential and possible increased risk for extra-paraganglionic neoplasms; PGL4: associated with germ line mutations in the SDHC gene: • autosomadominant transmission; • rare, it seems to predispose primarily to paragangliomas of the head and neck. The referencing criteria are: a family member with pathogenic mutation identified in the SDHD, SDHB or SDHC genes; personal (or close family) history of multiple paragangliomas and/or multiple pheochromocytomas (including bilateral and multifocal), recurrent or diagnosed at a young age (<50 years); other associated tumors may include gastrointestinal stromal tumor and possibly clear cell renal carcinoma and papillary thyroid carcinoma. Multiple endocrine neoplasia type 1 (MEN1) autosomic dominant transmission; associated with germ line mutations in the MEN1 gene; characterized by pituitary tumors, parathyroid neoplasia and pancreatic endocrine neoplasia; Primary hyperparathyroidism is the most common manifestation and usually the first sign; neuroendocrine carcinomas of the pancreas: • gastrinomas, insulinomas; • less often, VIPomas, glucagonomas, somatostatinomas; IPOPORTO 9 other associated manifestations may include: duodenal carcellinoid, cortical lesion of the adrenal, thyroid adenoma, pheochromocytoma, ependymoma, leiomyoma, facial angiofibroma, collagengenome, lipoma and meningioma; Additional skin findings may include: \\'coffee with milk\\' or hypopigmented macules and multiple gingival papules. The reference criteria are: familiar with pathogenic mutation identified in the MEN1 gene; personal history of at least two endocrine tumors of origin in parathyroid, pituitary or pancreas. Multiple endocrine neoplasia type 2 (MEN2A, FMTC, MEN2B) autososol dominant transmission; associated with germ line mutations in the RET gene; the most common clinical subtype is MEN2A: • medullary thyroid carcinoma, pheochromocytoma, and hyperparathyroidism; the least common clinical subtype is MEN2B: • medullary thyroid carcinoma, pheochromocytoma, marphanoid habitus and mucous ganglioneuromas; • clinically more aggressive than MEN2A; may present only medullary thyroid carcinoma, and may be called family thyroid medullary carcinoma (FMTC). The reference criteria are: familiar with pathogenic mutation identified in the RET gene; personal or family history that includes one or more cases of medullary thyroid cancer. Von Hippel-Lindau syndrome (VHL) autosomic dominant transmission; associated with germ line mutations in the VHL gene; the most common manifestations include: cerebral, spinal cord and retinal hemangioblastomas (hemangioblastomas in the retina may be the first manifestation and may cause vision loss); renal cysts and clear cell renal carcinoma (40% of patients); pancreatic and epididymis cysts, pheochromocytoma (may be asymptomatic, but may cause sustained or episodic hypertension); 10 HEREDITARY CANCER less common manifestations include: pancreatic tumors, wide ligament adenomas and endolymphatic sac tumors (may cause hearing loss of varying severity). The reference criteria are: familiar with pathogenic mutation identified in the VHL gene; hemangioblastomas in the brain, spinal cord and/or retina; renal cysts and clear cell renal carcinomas (multiple in the absence of other manifestations); pheochromocytomas (in combination with other manifestations); tumors of the endolymphatic sac. Li-Fraumeni syndrome (LfS) autososol dominant transmission; associated with germ line mutations in the TP53 gene; predisposes to multiple primary neoplasms, ~20% diagnosed before the age of 20; the most typical malignancies are: • sarcomas (bone and soft tissues); • breast cancer; • brain tumours; • adrenocortical carcinomas; • leukaemia; other cancers have been reported (e.g. melanoma and pancreatic, lung and colorectal cancer); risk of cancer in men is 70-90% over life, being 45% up to 45 years; risk of cancer in women is close to 100%, being 84% up to 45 years (mainly breast cancer); there is a great variability in the age of diagnosis and type of cancer. The classic criteria of Li-Fraumeni syndrome are: patient diagnosed with sarcoma before the age of 45, and a first-degree relative diagnosed with any cancer before the age of 45, and a first- or second-degree relative diagnosed with any cancer before the age of 45 or sarcoma at any age. Patients with the following criteria should also be referred to as: patient diagnosed with sarcoma, brain tumour, breast cancer or adrenocortical carcinoma before the age of 36 and first or second degree relatives with can cro (not breast cancer if the patient has this diagnosis) before the age of 46 or multiple primary tumours at any age; patient with multiple primary tumours, two of which are sarcoma, brain tumour, breast cancer and/or adrenocortical carcinoma, with the first diagnosis made before the age of 36; adrenocortical carcinoma at any age. Melanoma/hereditary pancreatic cancer autosomic dominant transmission; associated with germ line mutations in the CDKN2A gene or, rarely, in the CDK4 gene; the most common manifestations include: melanoma fa miliar, multiple melanomas, and may be associated with pancreatic cancer in certain families; in Europe the penetrance for melanoma is 13% at 50 years and 58% at 80 years, with the values being found in Australia 32% and 91%, respectively; the cumulative risk of pancreatic cancer can reach 25% by the age of 80. The reference criteria are: familiar with pathogenic mutation identified in the CDKN2A or CDK4 genes; personal or family history that includes at least one melanoma and at least one more case of mela noma or pancreatic cancer (in relatives of primary or second degree); individuals with melanoma and pancreatic cancer (synchronous or metachronous); at least two primary melanomas (synchronous or metachronous). IPOPORTO 11 IPOPORTO Rua Dr. António Bernardino de Almeida 4200-072 Porto T +351 225 084 000 F +351 225 084 001 www.ipoporto.min-saude.pt diripo@ipoporto.min-saude.pt C MEDIANA S.A. www.mediana.pt'"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "document.text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate summaries\n",
    "Generate **extractive** (= top ranked sentences) and **abstractive** (= paraphrased by text generator model) summaries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here are the 5 most important sentences, which we think contain the essence of the document: \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['The aim of genetic counseling is to enable individuals to better understand hereditary cancer, understand their own risks of developing cancer, and know the various screening and/or risk reduction options.',\n",
       " 'Genetic counseling includes: detailed review of family history, paying special attention to those who have already suffered from cancer; assistance with the collection of relevant medical records in order to provide accurate risk assessment; explanation of the differences between sporadic cancers (they occur by chance in the population, so all are at the same risk) and hereditary cancers (which appear in certain families and may be associated with a mutation inherited from a specific gene); interpretation of the pattern or patterns of cancer in family history – some people may realize that the risk of getting cancer is lower than they thought and others that the risk is higher; discussion of the possibility of performing genetic tests and, if indicated, who is the best index case of the family; review of the procedure of a genetic test, if available, including possible outcomes, what potential impact on family dynamics and other risks, benefits and limitations; help inform family members; recommendations for the screening and/or prevention of cancers for which the consul may have a higher risk.',\n",
       " \"Carrier testing is only available when a specific mutation of a gene is already confirmed in the family; tests only for the presence/absence of this specific mutation; the result confirms the carrier status and associated cancer risks, but cannot predict when or if the individual will develop cancer: • positive result: the person has inherited a high risk for developing specific types of cancer; • negative result: the person did not inherit a high risk, presenting the risk of the general population; if possible, the application for genetic counseling should include a copy of a family member's genetic test report.\",\n",
       " '4 HEREDITARY CANCER Hereditary cancer of the breast and ovary there are currently two known genes associated with hereditary breast and ovarian cancer: BRCA1 and BRCA2; approximately 1:1000 to 1:500 people in the general population are born with an inherited mutation in the BRCA1 or BRCA2 gene; the frequency of mutation in one of these genes is about 1:40 in people with Ashkenazi Jewish ancestry; hereditary breast and ovarian cancer has an autosomadominant transmission; genetic counseling is made available to all individuals and families who meet the benchmarking criteria.',\n",
       " 'IPOPORTO 7 Cancer screening and prophylaxis For people with Lynch syndrome and identified mutation of the MLH1, MSH2, MSH6 or PMS2 genes (or who have a 50% probability of inheriting the known mutation in the family), the current recommendations for screening and prophylaxis are: colonoscopy: • onset at 25 years (or 5-10 years before the youngest diagnostic age in the family); • repeat every 1-2 years until the age of 40 and then annually; • it has been shown in the scientific literature that this screening protocol significantly reduces the incidence of colorectal cancer in people with Lynch syndrome; annual gynecological examination, as well as immediate assessment of any unusual bleeding between menstrual periods or after menopause; hysterectomy and prophylactic bilateral salpingo-oophorectomy should be considered after 35-40 years or when the reproductive project is finished; Annual endometrial biopsy with transvaginal ultrasonography can be considered: • onset at 25-35 years if family history includes endometrial cancer; • onset at 35 years of age if family history does not include endometrium cancer; • the effectiveness of this approach has not yet been proven; there is no screening recommendation for other types of cancer associated with Lynch syndrome because its efficacy is controversial.']"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a summary of the document\n",
    "document = SumMed.analyze (document, CREATE_SUMMARY)\n",
    "\n",
    "print (f\"Here are the {len(document.summary)} most important sentences, which we think contain the essence of the document: \")\n",
    "document.summary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['The aim of genetic counseling is to enable individuals to better understand hereditary cancer, understand their own risks of developing cancer, and know the various screening and/or risk reduction options.',\n",
       " 'Genetic counseling includes: detailed review of family history, paying special attention to those who have already suffered from cancer; assistance with the collection of relevant medical records in order to provide accurate risk assessment; explanation of the differences between sporadic cancers (they occur by chance in the population, so all are at the same risk) and hereditary cancers (which appear in certain families and may be associated with a mutation inherited from a specific gene); interpretation of the pattern or patterns of cancer in family history – some people may realize that the risk of getting cancer is lower than they thought and others that the risk is higher; discussion of the possibility of performing genetic tests and, if indicated, who is the best index case of the family; review of the procedure of a genetic test, if available, including possible outcomes, what potential impact on family dynamics and other risks, benefits and limitations; help inform family members; recommendations for the screening and/or prevention of cancers for which the consul may have a higher risk.',\n",
       " \"Carrier testing is only available when a specific mutation of a gene is already confirmed in the family; tests only for the presence/absence of this specific mutation; the result confirms the carrier status and associated cancer risks, but cannot predict when or if the individual will develop cancer: • positive result: the person has inherited a high risk for developing specific types of cancer; • negative result: the person did not inherit a high risk, presenting the risk of the general population; if possible, the application for genetic counseling should include a copy of a family member's genetic test report.\",\n",
       " '4 HEREDITARY CANCER Hereditary cancer of the breast and ovary there are currently two known genes associated with hereditary breast and ovarian cancer: BRCA1 and BRCA2; approximately 1:1000 to 1:500 people in the general population are born with an inherited mutation in the BRCA1 or BRCA2 gene; the frequency of mutation in one of these genes is about 1:40 in people with Ashkenazi Jewish ancestry; hereditary breast and ovarian cancer has an autosomadominant transmission; genetic counseling is made available to all individuals and families who meet the benchmarking criteria.',\n",
       " 'IPOPORTO 7 Cancer screening and prophylaxis For people with Lynch syndrome and identified mutation of the MLH1, MSH2, MSH6 or PMS2 genes (or who have a 50% probability of inheriting the known mutation in the family), the current recommendations for screening and prophylaxis are: colonoscopy: • onset at 25 years (or 5-10 years before the youngest diagnostic age in the family); • repeat every 1-2 years until the age of 40 and then annually; • it has been shown in the scientific literature that this screening protocol significantly reduces the incidence of colorectal cancer in people with Lynch syndrome; annual gynecological examination, as well as immediate assessment of any unusual bleeding between menstrual periods or after menopause; hysterectomy and prophylactic bilateral salpingo-oophorectomy should be considered after 35-40 years or when the reproductive project is finished; Annual endometrial biopsy with transvaginal ultrasonography can be considered: • onset at 25-35 years if family history includes endometrial cancer; • onset at 35 years of age if family history does not include endometrium cancer; • the effectiveness of this approach has not yet been proven; there is no screening recommendation for other types of cancer associated with Lynch syndrome because its efficacy is controversial.']"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "document.summary\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here's how our A.I. would paraphrase the key ideas in the document: \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['1. The aim of genetic counseling is to help people understand their risks for developing cancer, and know the various screening and/or risk reduction options.\\n2. Genetic counseling includes: detailed review of family history, paying special attention to those who have already suffered from cancer; assistance with the collection of relevant medical records in order to provide accurate risk assessment; explanation of the differences between sporadic cancers (they occur by chance in the population, so all are at the same risk) and hereditary cancers (which appear in certain families and may be associated with a mutation inherited from a specific gene); interpretation of the pattern or patterns of cancer in family history – some people may realize that the risk of getting cancer is lower than they thought and others that the risk is higher; discussion of the possibility of performing genetic tests and, if indicated, who is the best index case of the family; review of the procedure of a genetic test, if available, including possible outcomes, what potential impact on family']"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create n \"abstractive\" (paraphrasing) summary of the document\n",
    "document = SumMed.analyze (document, CREATE_ABSTRACTIVE_SUMMARY)\n",
    "\n",
    "print (f\"Here's how our A.I. would paraphrase the key ideas in the document: \")\n",
    "document.abstractive_summary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Find medical entities within the text\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Here's a list of 337 different medical 'things' that are mentioned in the text, orderd by frequency: \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['FamilyRelation: Family (67)',\n",
       " 'Diagnosis: Malignant Neoplasms (50)',\n",
       " 'Diagnosis: Malignant neoplasm of breast (22)',\n",
       " 'Diagnosis: Colorectal Carcinoma (20)',\n",
       " 'Diagnosis: Lynch Syndrome (19)',\n",
       " 'GeneOrProtein: BRCA1 gene (16)',\n",
       " 'GeneOrProtein: BRCA2 gene (15)',\n",
       " 'GeneOrProtein: Genes (15)',\n",
       " 'ExaminationName: Genetic screening method (14)',\n",
       " 'Gender: Woman (11)',\n",
       " 'FamilyRelation: Family member (11)',\n",
       " 'Diagnosis: AMYLOIDOSIS, HEREDITARY, TRANSTHYRETIN-RELATED (9)',\n",
       " 'Diagnosis: Malignant neoplasm of ovary (9)',\n",
       " 'Diagnosis: Hereditary Breast and Ovarian Cancer Syndrome (8)',\n",
       " 'Diagnosis: melanoma (7)',\n",
       " 'Diagnosis: Pheochromocytoma (7)',\n",
       " 'Gender: Male population group (7)',\n",
       " 'Diagnosis: Paraganglioma (6)',\n",
       " 'ExaminationName: Screening procedure (6)',\n",
       " 'Age: 40 (6)',\n",
       " 'Age: 50 (6)',\n",
       " 'ExaminationName: Carrier testing (6)',\n",
       " 'Diagnosis: Malignant neoplasm of pancreas (5)',\n",
       " 'Diagnosis: Sarcoma (5)',\n",
       " 'Diagnosis: Medullary carcinoma of thyroid (5)',\n",
       " 'Diagnosis: Malignant neoplasm of stomach (5)',\n",
       " 'Diagnosis: Brain Neoplasms (5)',\n",
       " 'GeneOrProtein: MSH6 gene (5)',\n",
       " 'GeneOrProtein: MSH2 gene (5)',\n",
       " 'GeneOrProtein: MLH1 gene (5)',\n",
       " 'Direction: bilateral (5)',\n",
       " 'ExaminationName: Laboratory Procedures (5)',\n",
       " 'ExaminationName: index test (5)',\n",
       " 'Diagnosis: Primary Neoplasm (5)',\n",
       " 'TreatmentName: Prophylactic treatment (5)',\n",
       " 'Diagnosis: Hereditary Malignant Neoplasm (5)',\n",
       " 'Diagnosis: Adrenocortical carcinoma (4)',\n",
       " 'Diagnosis: Von Hippel-Lindau Syndrome (4)',\n",
       " 'BodyStructure: Retina (4)',\n",
       " 'Diagnosis: polyps (4)',\n",
       " 'Diagnosis: Adenoma of large intestine (4)',\n",
       " 'Diagnosis: Endometrial Carcinoma (4)',\n",
       " 'ExaminationName: Amsterdam Criteria (4)',\n",
       " 'GeneOrProtein: PMS2 gene (4)',\n",
       " 'BodyStructure: Ovary (4)',\n",
       " 'ExaminationName: Risk (4)',\n",
       " 'Age: 45 (4)',\n",
       " 'FamilyRelation: Second Degree Relative (4)',\n",
       " 'Diagnosis: Breast (4)',\n",
       " 'FamilyRelation: Relative (related person) (4)',\n",
       " 'Diagnosis: Li-Fraumeni Syndrome (3)',\n",
       " 'SymptomOrSign: Manifestation of (3)',\n",
       " 'Diagnosis: Hemangioblastoma (3)',\n",
       " 'Diagnosis: Multiple Endocrine Neoplasia Type 2a (3)',\n",
       " 'GeneOrProtein: MEN1 gene (3)',\n",
       " 'Diagnosis: Conventional (Clear Cell) Renal Cell Carcinoma (3)',\n",
       " 'GeneOrProtein: SDHC gene (3)',\n",
       " 'Age: 50 years (3)',\n",
       " 'ExaminationName: cumulative risk (3)',\n",
       " 'Diagnosis: Soft Tissue Neoplasms (3)',\n",
       " 'Diagnosis: Osteoma (3)',\n",
       " 'Diagnosis: Neoplasms (3)',\n",
       " 'BodyStructure: Ureter (3)',\n",
       " 'ExaminationName: Bethesda criteria (3)',\n",
       " 'Diagnosis: Menopause (3)',\n",
       " 'BodyStructure: Breast (3)',\n",
       " 'ExaminationName: Mammography (3)',\n",
       " 'ExaminationName: carrier test (3)',\n",
       " 'ExaminationName: Blood specimen (3)',\n",
       " 'Diagnosis: hereditary predisposition to cancer (3)',\n",
       " 'GeneOrProtein: CDK4 gene (2)',\n",
       " 'GeneOrProtein: CDKN2A gene (2)',\n",
       " 'Age: 36 (2)',\n",
       " 'Age: 45 years (2)',\n",
       " 'Diagnosis: Endolymphatic sac tumor (2)',\n",
       " 'Diagnosis: Pancreas (2)',\n",
       " 'Diagnosis: Renal cyst (2)',\n",
       " 'BodyStructure: Spinal Cord (2)',\n",
       " 'GeneOrProtein: RET gene (2)',\n",
       " 'Diagnosis: Multiple Endocrine Neoplasia Type 2b (2)',\n",
       " 'Diagnosis: Familial medullary thyroid carcinoma (2)',\n",
       " 'GeneOrProtein: SDHB gene (2)',\n",
       " 'BodyStructure: Neck (2)',\n",
       " 'BodyStructure: Head (2)',\n",
       " 'GeneOrProtein: SDHD gene (2)',\n",
       " 'Age: 80 years (2)',\n",
       " 'GeneOrProtein: CDH1 gene (2)',\n",
       " 'GeneOrProtein: autososol (2)',\n",
       " 'Diagnosis: Hereditary Diffuse Gastric Cancer (2)',\n",
       " 'GeneOrProtein: MUTYH gene (2)',\n",
       " 'Diagnosis: MUTYH-Associate Polyposis (2)',\n",
       " 'ExaminationName: Histopathological aspects (2)',\n",
       " 'Diagnosis: Adenoma of duodenum (2)',\n",
       " 'Diagnosis: fibroma (2)',\n",
       " 'Diagnosis: Epithelial cyst (2)',\n",
       " 'Diagnosis: congenital hyperplasia of the pigmented epithelium (2)',\n",
       " 'Diagnosis: Abnormality of the dentition (2)',\n",
       " 'Diagnosis: Polyposis, Gastric (2)',\n",
       " 'GeneOrProtein: APC gene (2)',\n",
       " 'Age: 35 years (2)',\n",
       " 'BodyStructure: Brain (2)',\n",
       " 'ExaminationName: immunohistochemical analysis (2)',\n",
       " 'Diagnosis: Adenoma (2)',\n",
       " 'BodyStructure: Pancreas (2)',\n",
       " 'BodyStructure: Intestines, Small (2)',\n",
       " 'BodyStructure: Stomach (2)',\n",
       " 'Diagnosis: Adenomatous Polyposis Coli (2)',\n",
       " 'Direction: contralateral (2)',\n",
       " 'Age: 35 (2)',\n",
       " 'TreatmentName: prophylactic salpingo-oophorectomy (2)',\n",
       " 'ExaminationName: Physical Examination (2)',\n",
       " 'Age: 25 years (2)',\n",
       " 'Frequency: annual (2)',\n",
       " 'Age: younger age (2)',\n",
       " 'GeneOrProtein: probability (2)',\n",
       " 'Diagnosis: Fit and well (2)',\n",
       " 'ExaminationName: Histology Procedure (2)',\n",
       " 'FamilyRelation: Sibling (2)',\n",
       " 'Diagnosis: Hereditary Nonpolyposis Colorectal Cancer (2)',\n",
       " 'Diagnosis: hereditary colorectal cancer without polyposis (2)',\n",
       " 'Diagnosis: Neoplastic Syndromes, Hereditary (2)',\n",
       " 'Age: young age (2)',\n",
       " 'TreatmentName: Genetic Counseling (2)',\n",
       " 'Diagnosis: primary melanomas (1)',\n",
       " 'FamilyRelation: second degree (1)',\n",
       " 'Diagnosis: Primary malignant neoplasm (1)',\n",
       " 'Diagnosis: Noma (1)',\n",
       " 'Diagnosis: MELAS Syndrome (1)',\n",
       " 'Age: 80 (1)',\n",
       " 'Age: 46 (1)',\n",
       " 'FamilyRelation: first- (1)',\n",
       " 'FamilyRelation: First Degree Relative (1)',\n",
       " 'ExaminationName: Cancer Risk (1)',\n",
       " 'Diagnosis: Lung (1)',\n",
       " 'Diagnosis: leukemia (1)',\n",
       " 'BodyStructure: soft tissue (1)',\n",
       " 'BodyStructure: Skeletal bone (1)',\n",
       " 'Age: 20 (1)',\n",
       " 'GeneOrProtein: TP53 gene (1)',\n",
       " 'SymptomOrSign: Hearing Loss (1)',\n",
       " 'Diagnosis: wide ligament adenomas (1)',\n",
       " 'Diagnosis: Pancreatic Neoplasm (1)',\n",
       " 'Diagnosis: Hypertensive disease (1)',\n",
       " 'Diagnosis: Epididymal Cyst (1)',\n",
       " 'SymptomOrSign: Abnormal vision (1)',\n",
       " 'BodyStructure: Cerebral hemisphere structure (body structure) (1)',\n",
       " 'Diagnosis: mucous ganglioneuromas (1)',\n",
       " 'SymptomOrSign: habitus (1)',\n",
       " 'Diagnosis: marphanoid (1)',\n",
       " 'Diagnosis: Hyperparathyroidism (1)',\n",
       " 'Diagnosis: Multiple endocrine neoplasia Type 2 (1)',\n",
       " 'BodyStructure: Pituitary Gland (1)',\n",
       " 'BodyStructure: Parathyroid gland (1)',\n",
       " 'Diagnosis: Endocrine Gland Neoplasms (1)',\n",
       " 'SymptomOrSign: Papule (1)',\n",
       " 'SymptomOrSign: Gingival (intended site) (1)',\n",
       " 'SymptomOrSign: Hypopigmented macule (1)',\n",
       " 'SymptomOrSign: Coffee (1)',\n",
       " 'SymptomOrSign: findings aspects (1)',\n",
       " 'BodyStructure: Skin (1)',\n",
       " 'Diagnosis: Meningioma (1)',\n",
       " 'Diagnosis: Lipoma (1)',\n",
       " 'Diagnosis: collagengenome (1)',\n",
       " 'Diagnosis: Fibrous skin tumor of tuberous sclerosis (1)',\n",
       " 'Diagnosis: Fibroid Tumor (1)',\n",
       " 'Diagnosis: Ependymoma (1)',\n",
       " 'Diagnosis: Thyroid Gland Follicular Adenoma (1)',\n",
       " 'BodyStructure: Adrenal Glands (1)',\n",
       " 'Diagnosis: cortical lesion (1)',\n",
       " 'Diagnosis: duodenal carcellinoid (1)',\n",
       " 'Diagnosis: IPOPORTO (1)',\n",
       " 'Diagnosis: Somatostatinoma (1)',\n",
       " 'Diagnosis: Glucagonoma (1)',\n",
       " 'Diagnosis: Vipoma (1)',\n",
       " 'Diagnosis: insulinoma (1)',\n",
       " 'Diagnosis: Gastrinoma (1)',\n",
       " 'Diagnosis: neuroendocrine carcinoma of pancreas (1)',\n",
       " 'Diagnosis: Hyperparathyroidism, Primary (1)',\n",
       " 'Diagnosis: Pancreatic endocrine neoplasia (1)',\n",
       " 'Diagnosis: Parathyroid Neoplasms (1)',\n",
       " 'Diagnosis: Pituitary Neoplasms (1)',\n",
       " 'GeneOrProtein: Multiple endocrine neoplasia type 1 (1)',\n",
       " 'Diagnosis: Papillary thyroid carcinoma (1)',\n",
       " 'Diagnosis: Gastrointestinal Stromal Tumors (1)',\n",
       " 'Diagnosis: PARAGANGLIOMAS 4 (1)',\n",
       " 'Diagnosis: extra-paraganglionic neoplasms (1)',\n",
       " 'Diagnosis: increased malignant potential (1)',\n",
       " 'BodyStructure: Extra-Adrenal (1)',\n",
       " 'BodyStructure: Chest (1)',\n",
       " 'BodyStructure: Pelvis (1)',\n",
       " 'BodyStructure: Abdomen (1)',\n",
       " 'FamilyRelation: Father (person) (1)',\n",
       " 'Diagnosis: Familial Paraganglioma (1)',\n",
       " 'Diagnosis: Carcinoma, Lobular (1)',\n",
       " 'Age: 69 years (1)',\n",
       " 'Age: 38 years (1)',\n",
       " 'Diagnosis: diffuse gastric cancer (1)',\n",
       " 'Diagnosis: Polyp of large intestine (1)',\n",
       " 'FamilyRelation: Brothers (1)',\n",
       " 'Diagnosis: tubulo (1)',\n",
       " 'Diagnosis: Tubular adenoma (1)',\n",
       " 'Direction: distal (1)',\n",
       " 'GeneOrProtein: MYH alias (1)',\n",
       " 'Diagnosis: Adenomatous Polyps (1)',\n",
       " 'Diagnosis: Fibromatosis, Aggressive (1)',\n",
       " 'Diagnosis: Colorectal Adenomatous Polyp (1)',\n",
       " 'Diagnosis: Medulloblastoma (1)',\n",
       " 'Diagnosis: Colorectal (1)',\n",
       " 'Diagnosis: Turcot syndrome (disorder) (1)',\n",
       " 'Diagnosis: desoid tumors (1)',\n",
       " 'Diagnosis: colorectal adenomatous polyposis (1)',\n",
       " 'Diagnosis: Gardner Syndrome (1)',\n",
       " 'Diagnosis: Gastric (qualifier value) (1)',\n",
       " 'SymptomOrSign: extraintestinal manifestations (1)',\n",
       " 'Diagnosis: Multiple polyps (1)',\n",
       " 'Diagnosis: dismoid tumors (1)',\n",
       " 'Diagnosis: Malignant neoplasm of retina (1)',\n",
       " 'SymptomOrSign: Tooth, Supernumerary (1)',\n",
       " 'Diagnosis: duodenal adenomatous polyps (1)',\n",
       " 'TreatmentName: prophylactic cholectomy (1)',\n",
       " 'Diagnosis: hereditary polyposis syndrome (1)',\n",
       " 'Diagnosis: Adenomatous Polypose Hereditary Cancer Syndromes (1)',\n",
       " 'Age: 35 years of age (1)',\n",
       " 'Age: 25 (1)',\n",
       " 'ExaminationName: Transvaginal ultrasonography (1)',\n",
       " 'ExaminationName: Endometrial biopsy (1)',\n",
       " 'Frequency: Annual (1)',\n",
       " 'Age: 40 years (1)',\n",
       " 'TreatmentName: prophylactic bilateral salpingo-oophorectomy (1)',\n",
       " 'TreatmentName: Hysterectomy (1)',\n",
       " 'SymptomOrSign: menstrual symptoms (1)',\n",
       " 'SymptomOrSign: Hemorrhage (1)',\n",
       " 'ExaminationName: Gynecological examination (1)',\n",
       " 'Frequency: annually (1)',\n",
       " 'Frequency: 2 years (1)',\n",
       " 'Frequency: every 1 (1)',\n",
       " 'Age: 10Y (1)',\n",
       " 'Age: 5 (1)',\n",
       " 'ExaminationName: colonoscopy (1)',\n",
       " 'ExaminationName: carrier testing (1)',\n",
       " 'ExaminationName: Tumor tissue sample (1)',\n",
       " 'Diagnosis: lynch syndrome colorectal tumors (1)',\n",
       " 'SymptomOrSign: changes in protein expression (1)',\n",
       " 'SymptomOrSign: Neoplasms (1)',\n",
       " 'GeneOrProtein: Proteins (1)',\n",
       " 'SymptomOrSign: Microsatellite Instability (1)',\n",
       " 'Diagnosis: Colorectal Neoplasms (1)',\n",
       " 'BodyStructure: Body tissue (1)',\n",
       " 'Diagnosis: Sebaceous adenoma (1)',\n",
       " 'Diagnosis: Lynch (1)',\n",
       " 'BodyStructure: hepatobiliary tract (1)',\n",
       " 'BodyStructure: Endometrium (1)',\n",
       " 'Diagnosis: Lynch Colorectal syndrome (1)',\n",
       " 'ExaminationName: Analysis of substances (1)',\n",
       " 'GeneOrProtein: Mismatch repair endonuclease PMS2 (1)',\n",
       " 'ExaminationName: microsatellite instability research (1)',\n",
       " 'ExaminationName: tumor pre-test (1)',\n",
       " 'Diagnosis: Lynch syndrome spectrum neoplasm (1)',\n",
       " 'Diagnosis: lynch syndrome spectrum neoplasm (1)',\n",
       " 'FamilyRelation: first-degree family member (1)',\n",
       " 'Age: 60 years (1)',\n",
       " 'SymptomOrSign: medullary growth pattern (1)',\n",
       " 'SymptomOrSign: syncet ring cell differentiation (1)',\n",
       " 'SymptomOrSign: Mucinous (1)',\n",
       " \"SymptomOrSign: Crohn's-like lymphocytic reaction (1)\",\n",
       " 'Diagnosis: Lymphocytic infiltration (1)',\n",
       " 'Diagnosis: keratoacantomas (1)',\n",
       " 'Diagnosis: Sebaceous Adenocarcinoma (1)',\n",
       " 'Diagnosis: bacinete (1)',\n",
       " 'Diagnosis: Tract (1)',\n",
       " 'ExaminationName: Pathologic Examination (1)',\n",
       " 'FamilyRelation: successive generations (1)',\n",
       " 'FamilyRelation: first-degree (1)',\n",
       " 'Diagnosis: bacinet (1)',\n",
       " 'Diagnosis: Intestines (1)',\n",
       " 'GeneOrProtein: DNA Repair (1)',\n",
       " 'Diagnosis: Disease (1)',\n",
       " 'Diagnosis: Hereditary Colorectal Cancer (1)',\n",
       " 'TreatmentName: prophylaxis measures (1)',\n",
       " 'MedicationName: tamoxifen (1)',\n",
       " 'Age: childbearing age (1)',\n",
       " 'MedicationClass: Contraceptive Agents (1)',\n",
       " 'BodyStructure: Oral (1)',\n",
       " 'TreatmentName: Chemoprevention (1)',\n",
       " 'SymptomOrSign: Symptoms (1)',\n",
       " 'TreatmentName: Hormone replacement therapy (1)',\n",
       " 'ExaminationName: Medical Surveillance (1)',\n",
       " 'Diagnosis: Postsurgical menopause (1)',\n",
       " 'Diagnosis: Reproduction (1)',\n",
       " 'Diagnosis: Fallopian Tube Carcinoma (1)',\n",
       " 'Diagnosis: Ovarian (1)',\n",
       " 'HealthcareProfession: pathologist (1)',\n",
       " 'BodyStructure: Fallopian Tubes (1)',\n",
       " 'TreatmentName: Excision (1)',\n",
       " 'HealthcareProfession: specialist pathologist (1)',\n",
       " 'TreatmentName: removed breast (1)',\n",
       " 'TreatmentName: Mammaplasty (1)',\n",
       " 'TreatmentName: prophylactic mastetomia (1)',\n",
       " 'BodyStructure: Prostate (1)',\n",
       " 'FamilyRelation: Mama (1)',\n",
       " 'ExaminationName: IPOPORTO (1)',\n",
       " 'TreatmentName: Preventive surgery (1)',\n",
       " 'ExaminationName: Screening for Ovarian Cancer (1)',\n",
       " 'Diagnosis: Malignant neoplasm of prostate (1)',\n",
       " 'SymptomOrSign: change in the chest (1)',\n",
       " 'ExaminationName: Breast Self-Examination (1)',\n",
       " 'Frequency: monthly (1)',\n",
       " 'ExaminationName: imaging examinations (1)',\n",
       " 'Frequency: every six months (1)',\n",
       " 'Frequency: every 6 months (1)',\n",
       " 'ExaminationName: breast MAGNETIC RESONANCE (1)',\n",
       " 'Age: year (1)',\n",
       " 'GeneOrProtein: BRCA2 gene testing (1)',\n",
       " 'GeneOrProtein: genetic test (1)',\n",
       " 'HealthcareProfession: genetic counselling (1)',\n",
       " 'GeneOrProtein: HER2 receptors (1)',\n",
       " 'GeneOrProtein: hormonal (1)',\n",
       " 'Diagnosis: Medullary carcinoma of breast (1)',\n",
       " 'Age: 30 (1)',\n",
       " 'Diagnosis: Ovarian Carcinoma (1)',\n",
       " 'FamilyRelation: first-degree ( (1)',\n",
       " 'FamilyRelation: Offspring (1)',\n",
       " 'ExaminationName: carrier tests (1)',\n",
       " 'ExaminationName: Deoxyribonucleic Acid Measurement (1)',\n",
       " 'ExaminationName: Index test (1)',\n",
       " 'Diagnosis: Predisposition -- attribute (1)',\n",
       " 'ExaminationName: Hematologic Tests (1)',\n",
       " 'Diagnosis: Syndrome (1)',\n",
       " 'FamilyRelation: generations (1)',\n",
       " 'FamilyRelation: close relatives (1)',\n",
       " 'ExaminationName: Genetic studies (1)',\n",
       " 'FamilyRelation: close (1)',\n",
       " 'FamilyRelation: Mother (person) (1)',\n",
       " 'FamilyRelation: Paternal Relative (1)',\n",
       " 'Diagnosis: sporadic cancers (1)',\n",
       " 'TreatmentName: Risk Reduction (1)',\n",
       " 'Diagnosis: hereditary syndrome (1)']"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Detect the healt-related named entities in the document\n",
    "document = SumMed.analyze (document,DETECT_HEALTH_ENTITIES)\n",
    "\n",
    "\n",
    "print (f\"Here's a list of {len(document.health_entities)} different medical 'things' that are mentioned in the text, orderd by frequency: \")\n",
    "[f\"{e.label}: {e.text} ({e.count})\" for e in document.health_entities]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Find trusted information\n",
    "Our trusted web search will look for highly relevant online articles that are closely rfelated to our document and the key topics in it. Only known, trusted websites (such as from renowned medical institutes) are considered."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Genetic counseling - Wikipedia (https://en.wikipedia.org/wiki/Genetic_counseling) ',\n",
       " 'Genetic Counseling | CDC (https://www.cdc.gov/genomics/gtesting/genetic_counseling.htm) ',\n",
       " 'Family communication about genetic risk: the little that is known (https://pubmed.ncbi.nlm.nih.gov/15475667/) ',\n",
       " 'Family history of breast cancer: what do women understand and recall ... (https://pubmed.ncbi.nlm.nih.gov/9733031/) ',\n",
       " 'What are the benefits of genetic testing?: MedlinePlus Genetics (https://medlineplus.gov/genetics/understanding/testing/benefits/) ',\n",
       " 'Retinoblastoma: A Major Review - PubMed (https://pubmed.ncbi.nlm.nih.gov/34226484/) ',\n",
       " 'Eugenics - Wikipedia (https://en.wikipedia.org/wiki/Eugenics) ',\n",
       " 'Psychosocial Support Options for People with Cancer (https://www.cancer.org/treatment/survivorship-during-and-after-treatment/coping/understanding-psychosocial-support-services.html) ']"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Lookup web articles from trusted sources that are highly related to the document \n",
    "document = SumMed.analyze (document, TRUSTED_SEARCH)\n",
    "\n",
    "[f\"{x.name} ({x.url}) \" for x in document.search_results]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cleanup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SumMed.space.delete_all_files ()"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "4c6215dfdde890b0c92d28e5f5d04ada27e42c51ca17043b19be22459f51c8d1"
  },
  "kernelspec": {
   "display_name": "Python 3.9.1 ('.venv': venv)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
