How Big Data and Machine Learning are Uniting Against Cancer

Cancer is not one disease. It is many diseases. Let us understand the cause of cancer by a simple example. If you take a photocopy of a document, due to some issues, other dots or smears appear on it even though they are not present in the original copy. In the same way, in gene replication processes, errors occur inadvertently. Most of the time the genes with errors will not be able to sustain and will ultimately perish.

In some rare cases, the mutated gene with mistakes will survive and get further replicated uncontrollably. Uncontrollable replication of mutated genes is the primary cause of cancer. This mutation can happen in any of the twenty thousand genes in our body. Variation in any one or a combination of genes makes cancer a severe disease to conquer. To eradicate cancer, we need methods to destroy the rogue cells without harming the functional cells of the body; which makes it doubly hard to defeat.

Cancer and its complexity

Cancer is a disease with a long tail distribution. Long tail distribution means there are various reasons for this condition to occur and there is no single solution for eradicating it. There are diseases which affect a large percentage of the population but have a sole cause of occurrence. For example, let us consider Cholera. Eating food or drinking water contaminated by the bacterium Vibrio Cholerae is the cause of cholera. Cholera can occur only because of Vibrio Cholerae, and there is no another reason. Once we find out the only cause of a disease, then it is relatively easy to conquer it.

What if a condition occurs because of multiple reasons? A mutation can occur in any of the twenty thousand genes in our body. Not only that, but we also need to consider their combinations. Cancer may not just happen because of a random mutation in a gene but also because of a combination of gene mutations. The number of causes for cancer becomes exponential, and there is no single mechanism to cure it. For example, a mutation of any of these genes ALK, BRAF, DDR2, EGFR, ERBB2, KRAS, MAP2K1, NRAS, PIK3CA, PTEN, RET, and RIT1 can cause lung cancer. There are many ways for cancer to occur and that’s why it is a disease with long tail distribution.

In our arsenal for waging this war on cancer and conquering it, big data and machine learning are critical tools. How can big data help in fighting this war? What does machine learning have to do with cancer? How are they going to help in fighting a disease with many causes, a condition with a long tail distribution? Firstly, how and where is this big data generated? Let us find answers to these questions.

Gene Sequencing and explosion in data

Gene sequencing is one area which is producing humongous amounts of data. Exactly how much data? According to the Washington Post, the human data generated through gene sequencing (approximately 2.5 lakh sequences) takes up about a fourth of the size of YouTube’s yearly data production. If all this data were combined with all the extra information that comes with sequencing genomes and recorded on 4GB DVDs, it would be a stack about half a mile high.

Explore Our Software Development Free Courses

The methods for gene sequencing have improved over the years, and the cost for the same has plummeted exponentially. In the year 2008, the cost of gene sequencing was 10 million dollars. As of today, it is only a 1000 dollars. In the future, it is expected to reduce further. It is estimated that one billion people will have their genes sequenced by 2025. So, within the next decade, the genomics data generated will be somewhere between 2 – 40 exabytes in a year. An exabyte is ten followed by 17 zeros.

Before coming to how data will help in curing cancer, let us take one concrete example and see how data can help in conquering a disease. Data and its analysis helped in finding out the cause of one infectious disease and fight it, not now but in nineteenth-century itself! Yes, in the nineteenth century! The name of that disease is Cholera.

Clustering in the Nineteenth Century – the Cholera breakthrough

John Snow was an anesthesiologist and cholera broke out in September 1854 near Snow’s house. To know the reason for cholera, Snow decided to note the spatial dimensions of the patients on the city map. He marked the location of the home address of patients on London’s city map. With this exercise, John Snow understood that people suffering from cholera were clustered around some specific water wells. He firmly believed that a contaminated pump was responsible for the epidemic and against the will of the local authorities replaced the pump. This replacement drastically reduced the spread of cholera.

Snow subsequently published a map of the outbreak to support his theory, showing the locations of the 13 public wells in the area, and the 578 cholera deaths mapped by home address. This map ultimately led to the understanding that cholera was an infectious disease and quickly spread through the medium of water. John Snow’s experiment is the earliest example of applying the clustering algorithm to know the cause of illness and help eradicate it. In the nineteenth century, John Snow could apply clustering algorithm on a London city map with a pencil. With cancer as the target disease, this level of analysis is not possible with the same ease as John Snow’s Analysis. We need sophisticated tools and technologies to mine this data. That is where we leverage the capabilities of modern technologies like Machine Learning and Big Data.

Explore our Popular Software Engineering Courses

Big data and Machine learning – tools to fight cancer

Vast amounts of data along with machine learning algorithms will help us in our fight with cancer in many ways. It can help us with diagnosis, treatment, and prognosis. Mainly, it will help customise the therapy according to the patient, which is not possible otherwise. It will also help deal with the long tail of the distribution.

Given the enormous amounts of Electronic Medical Records (EMR), data generated and recorded by various hospitals; it is possible to use ‘labelled’ data in diagnosing cancer. Techniques like Natural Language Programming (NLP) are utilised for making sense of doctor’s prescriptions and Deep Learning Neural Networks are deployed to analyse CT and MRI scans. The different types of machine learning algorithms search the EMR databases and find hidden patterns. These hidden patterns will help in diagnosing cancers.

A college student was able to design an Artificial Neural Network from the comfort of her home and developed a model that can diagnose breast cancer with a high degree of accuracy.

In-Demand Software Development Skills

Diagnosis with Big Data and Machine Learning

Brittanny Wenger was 16 years old when her older cousin was diagnosed with breast cancer. This inspired her to make the process better by improving the diagnostics. Fine Needle Aspiration (FNA) was a less invasive method of biopsy and the quickest method of diagnosis. The doctors were reluctant to use FNA because the results are not reliable. Brittanny thought of using her programming skills to do something about it. She decided to improve the reliability of FNA which would enable the women to choose less invasive and comfortable diagnostic methods.

Brittanny found public domain data from the University of Wisconsin that included Fine Needle Aspiration. She coded an Artificial Neural Network (ANN) which is inspired by the design of human brain architecture. She used cloud technologies to process the data and train the ANN to find the similarities. After many attempts and errors finally, her network was able to detect breast cancer from an FNA test data with 99.1% sensitivity to malignancy. This method is applicable for diagnosing other cancers as well.

The accuracy of diagnosis is dependent upon the amount and quality of the data available. The more the data available, the more the algorithms will be able to query the database, find similarities and come out with valuable models.

Treatment with Big Data and Machine Learning

Big data and Machine learning will be helpful not only for diagnosis but treatment as well. John and Kathy were married for three decades. At the age of 49, Kathy was diagnosed with stage III breast cancer. John, CIO of a Boston hospital helped plan her treatment with the help of big data tools that he designed and brought into existence.

In 2008, five Harvard affiliated hospitals shared their databases and created a powerful search tool known as ‘Shared Health Research Information Network’ (SHRINE). By the time of Kathy’s diagnosis, her doctors could sift through a database of 6.1 million records to find insightful information. Doctors queried ‘SHRINE’ with questions like “50-year-old Asian women, diagnosed with stage III breast cancer and their treatments”. Armed with this information doctors were able to treat her with chemotherapy drugs by targeting the estrogen-sensitive tumour cells by avoiding surgery.

By the time Kathy completed her chemotherapy regimen the radiologists could no longer find any tumour cells. This is one example of how big data tools can help in customising the treatment plan according to the requirement of each.

As cancer is a long tail distribution a ‘one size fits all’ philosophy will not work. For customising treatments depending on the patient’s history, their gene sequence, results of diagnostic tests, a mutation found in their genes or a combination of their genes and environment, big data and machine learning tools are indispensable.

upGrad’s Exclusive Software Development Webinar for you –

SAAS Business – What is So Different?

Drug Discovery with Big Data and Machine Learning

Big data and Machine learning will not only help in diagnosis and treatment but also will revolutionise drug discovery. Researchers can use open data and computational resources to discover new uses for the drugs which are already approved by agencies like FDA for other purposes. For example, scientists at University of California at San Francisco found by number crunching that a drug called ‘pyrvinium pamoate’ which is used to treat pinworms – could shrink hepatocellular carcinoma, a type of liver cancer, in mice. This disease which is associated with the liver is the second highest contributor to cancer deaths in the world.

Not only is big data used for discovering new uses for old drugs but can also be used for detecting new drugs. By crunching data related to different drugs, chemicals, and their properties, symptoms of various diseases, the chemical composition of the drugs used for those conditions and side effects of these medications collected from different media; new drugs can be devised for various types of cancer. This will significantly reduce the time taken to come up with new medicines without wasting millions of dollars in the process.

Using big data and machine learning will no doubt improve the process of diagnosis, treatment and drug discovery in treating cancer, but it is not without challenges. There are many stumbling blocks and problems on the road ahead. If these blocks are not removed, and these challenges are not faced, then our enemy will get the upper hand and will defeat us in the future battle.

Read our Popular Articles related to Software Development

Challenges in using Big Data and Machine Learning to fight Cancer

Digitisation

Except for a few large and technically advanced hospitals, most of them are yet to be digitised. They are still following the old methods of capturing and recording data in massive stacks of files. Due to lack of technical expertise, affordability, economies of scale and various other reasons, digitisation has not taken place. Provision of open source EMR software, teaching how helpful these digital records could be in treating the patients and how profitable it is to the hospitals are some steps in the right direction.

Data locked in enterprise warehouses

As of today, only a few hospitals can digitally capture patient records. This apparatus too is locked away in enterprise warehouses and inaccessible to the world at large.

Hospitals are reluctant to share their databases with other hospitals. Even if they are willing, they are plagued by the different database schemas and architectures. Critical thinking is required on this front about how hospitals can share their databases among themselves for their mutual benefit without being suspicious of each other. A consensus needs to be reached about the schema in which this data should be shared as well, for the benefit of all hospitals. This patient data should be democratised and utilised for the betterment of the future of mankind.

Patient data should not be allowed to be employed for the growth of a single organisation. Utmost care should be taken to anonymise the individual to whom the data belongs. If a person’s lipstick preference is leaked, then there is not much harm. If a person’s medical history is leaked, then it will have a significant impact on his life and prospects.

The government should take positive steps in this direction and should help create a big data infrastructure for storing medical records of patients from all hospitals. It should make it compulsory for all hospitals to share their database within this shared infrastructure. Access to this database should be made free for patient treatment and research.

Improvement in efficiency of Machine Learning Algorithms

Machine learning is not a magic pill for cancer diagnosis and treatments. It is a tool that if used well can help in our journey to conquer cancer. Machine learning is still in a nascent stage and has its disadvantages. For example, the data on which these algorithms are trained needs to be very close to the data on which they are utilised for producing results. If there is a huge difference in them, then the algorithm will not be able to provide meaningful results which can be employed.

There are many machine learning algorithms which exist with their own peculiar assumptions, advantages, and disadvantages. If we can find a way to combine all these different algorithms for achieving the results required by us, i.e. curing cancer, needless to say, we would have found a hugely beneficial outcome. The famous machine learning scientist Pedro Domingos calls it “The Master Algorithm”, who also wrote a popular science book of the same name.
According to Pedro, there are five different schools of thought in machine learning. The symbolist, connectionist, Bayesian, evolutionaries and analogisers. It is difficult to go into all these different types of machine learning systems in this article. I will cover all the five types of machine learning systems in one of my future blogs. For now, we need to understand that all these different methods have advantages and disadvantages of their own. If we can combine them, then we can derive highly impactful insights from our data. This will be immensely useful not only for all kinds of predictions and forecasts but also for our fight against a vengeful enemy – cancer.

To summarise, cancer is a formidable enemy which keeps changing its form frequently. We do possess new weapons in our arsenal now in the form of big data and machine learning, however, to face it competently. But to demolish it entirely we need a more powerful weapon than what we presently possess. The name of that weapon is ‘The Master Algorithm’.

We also need to make some changes in the strategies and methods with which we are fighting this enemy. These changes are creating a big data infrastructure, making it compulsory for hospitals to share anonymised patient records, maintaining the security of the database and allowing free access to the database for patient treatment and research to cure cancer.

Get data science certification from the World’s top Universities. Learn Executive PG Programs, Advanced Certificate Programs, or Masters Programs to fast-track your career.

Wrapping up

If you are interested to know more about Big Data, check out our Advanced Certificate Programme in Big Data from IIIT Bangalore.

Learn Software Engineering degrees online from the World’s top Universities. Earn Executive PG Programs, Advanced Certificate Programs, or Masters Programs to fast-track your career.

Plan your Data Science Career now.

Apply for Professional Certificate Program in Data Science from IIM-Kozhikode