At Google Brain, a computer scientist focuses on data so doctors can focus on their patients instead

The artificial intelligence Maithra Raghu studies at Google Brain doesn’t have a bedside manner. But she’s betting it can still help restore a deeply human, disappearing aspect of modern medicine: personal connection.

In a health care system flooded with paperwork and patient data, Raghu sees a natural place for neural networks, which analyze vast amounts of information to find patterns that escape the human eye and use them to churn out diagnoses or health care predictions. To her, the technology could prove to be a powerful tool for processing data that can spare providers more time to spend with patients one-on-one.

“Machine learning isn’t a magic tool here,” said Raghu, a senior research scientist who was recently named a STAT Wunderkind. “Medicine is fundamentally human. In those times when you’re very scared, the human component is really, really critical.”


For the past six years, first as a Ph.D. student at Cornell and now at Google Brain, Raghu has been uncovering basic principles for building neural networks. The systems are notoriously complex – and equally mysterious. While neural networks grant a level of analysis far beyond the human mind, they offer scientists few clues of how they arrive at their decisions.

By developing mathematical techniques comparing how different types of neural networks learn, Raghu has uncovered surprising insights into their design and given researchers new tools for bringing AI into the clinic. Those efforts could one day help clinicians deliver diagnoses faster to patients, or spend more time explaining a patient’s care and listening closely to any concerns.


“I am really hopeful that neural networks can help with the overload of patient data,” she said. “So doctors will have more time and resources for people.”

Some of Raghu’s success in computer science comes from the winding path she took to the field.

As a child, she grew up in five countries: India, France, South Africa, the United States, and the United Kingdom. She was interested in mathematics from a young age, participating in international Math Olympiad competitions and studying mathematics at the University of Cambridge, before switching fields to computer science.

“[Life] is most enjoyable when you are learning lots of things,” said Raghu. “And that’s been a recurring theme throughout my education and career in science.”

Raghu’s Ph.D. adviser, Jon Kleinberg, sees imprints of her wide-ranging interests — not just mathematics, but also medicine — in her current work with AI.

“She’s often been able to bridge between different things,” said Kleinberg, a professor of computer science at Cornell University. He pointed to her work between academia and industry, as well as at the intersection of computer science and medicine. “She’s been very effective at having impacts in different communities.”

Her interest in combining AI with medicine was shaped, in part, by one of her own encounters with the medical system.

In 2014, when Raghu joined Cornell’s computer science department as a Ph.D. student, she observed neural networks being trained to recognize images such as a cat or firetruck from a database containing millions of pictures. She instantly thought about a recent ski accident. She had undergone an X-ray and MRI, but the results were inconclusive and she found herself in a painstaking waiting game. It took days for a doctor to interpret the results and confirm she had torn her ACL.

The really stressful part of the injury was not just being in pain but also not knowing what was wrong, said Raghu. “We actually got a family friend of ours, who was a doctor, to give us an informal read on the MRI.”

Watching AI sift through images, Raghu considered its potential to help doctors return results to patients like her sooner. Neural networks may have been able to scan her MRI and offer a preliminary diagnosis that would later be confirmed by a radiologist, or could have flagged patients with more serious injuries that needed prompt attention. For Raghu, knowing that her ACL was potentially torn would have helped her to take some basic precautions — such as avoiding walking — to prevent additional damage to her knee.

The situation piqued her interest at just the right time. New insights, greater amounts of computing power, and more data meant that neural networks had “just exploded in the level of performance they were able to achieve,” said Kleinberg.

But because of the complexity involved in building neural networks, it wasn’t yet clear how to translate their raw computing power to a medical setting.

At their most basic, neural networks are a cluster of densely interconnected processing nodes — also known as neurons — that are grouped into layers. As a network learns to perform a task, layers of neurons are trained to send data to each other, mimicking how information is transmitted through the human brain. Large networks may contain millions of neurons spanning several layers, leaving researchers with a multitude of design options, with each decision potentially having dramatic impacts on the network’s performance.

“You’re like, ‘I have something that is working this good, and I would like it to get better. But what do I do and where do I go?’” Raghu said.

One shortcut to designing neural networks that can analyze medical images is to adapt a network that has already been trained to recognize pictures from the internet or a large database. The hope is that, with a little refinement and retraining, the network can use apply its general knowledge of images to help decipher specialized images like X-rays or MRI scans.

Raghu’s research has focused on making that process — termed “transfer learning” — as efficient as possible as well as uncovering basic rules for building networks so that scientists can rely less on guessing and checking.

Her key insight has been the development of mathematical tools to study how differently structured networks analyze images. Instead of focusing on individual neurons, which are hard to compare across networks, Raghu concentrated on how layers or groups of neurons work together to find patterns in the data. This approach has started to answer several fundamental questions. Among them: How do small networks stack up against larger networks when it comes to adapting to new pictures? And how much of a network’s original training is forgotten during transfer learning?

“She tends to find creative approaches which other people haven’t thought about.”

Jon Kleinberg, Cornell University professor and Ph.D. adviser to Raghu

“Her creativity stands out,” said Kleinberg. “She tends to find creative approaches which other people haven’t thought about.”

Raghu’s work has also turned up some counterintuitive findings, including that smaller and simpler neural networks perform nearly as well as larger and more complex networks when it comes to medical analysis.

Her work suggests that instead of retraining entire neural networks for medical applications, it may be more efficient to recycle its lowest layers and redesign the upper layers for custom applications, such as diagnosing certain medical conditions.

“Looking forward,” said Raghu, “I’m really excited about the potential for some of these techniques to help inform the design of systems for new types of data.” In particular, Raghu wants to build neural networks that combine insights from a wide range of patient information including images, doctor’s notes, and lab test results.

“You’re going to have even more choices on how the design process should work, and this is a place where techniques giving you insight are going to help,” she added.

In the future, Raghu also wants to expand her research beyond the design of artificial intelligence to explore how neural networks can be deployed in hospitals in a way that is sensitive to patients’ needs.

Her childhood experience living in several parts of the world has given her special insight into how those needs might vary from one patient to the next, Raghu said. That might mean a difference in how often people want to see a doctor, how much medication they’re willing to take, or even how much interaction they want with a provider, she said.

“Those are things that are really important to be respectful of,” Raghu said. “For medicine and health care, the people part is just so important.”

Source: STAT