Dan Riskin, MD, MBA, FACS, CEO and Founder of Verantos.
We all want to think that our doctor learns from the experience of other doctors. But, as patients ourselves, we know that healthcare is complex and not always efficient. Individual doctors learn from treating their patients, but rarely is that information used to make care delivered by other doctors better.
In 2007, the Institute of Medicine published The Learning Healthcare System. It was thought that, in the future, healthcare could start to learn from routine care. This would allow doctors to systematically provide better care to the next patient.
Since then, other industries have started to learn from routine data. For example, Google Maps learns from cell phone location and movement to predict traffic, Amazon learns from other shoppers to make recommendations and Spotify creates playlists based on what others with similar preferences have liked.
If healthcare is not using routine care to learn, what data are doctors using to make decisions?
How Trials Have Worked Historically
Primarily, healthcare uses an approach defined before the personal computer was invented. Randomized trials are set up over several years using a small number of patients, results are analyzed and over the next two decades results slowly work their way into clinical practice.
While trials remain a bedrock of clinical evidence, they have limitations. They attempt to generalize results from a small population to the broader population, which may not respond the same way to a treatment. At an average cost of more than $40,000 per patient, the trials are small, and wealthy Americans are overrepresented.
While it’s possible to understand whether a treatment is safe and effective, there typically isn’t enough information to compare treatment options and discover which works best. In short, while the information captured in these trials is good, it is not sufficient. There is not enough data to enable tailored therapy or rapid learning within healthcare.
Where We Are Now
It’s reasonable to expect that healthcare as an industry is ready to start learning from real-world care. Since The Institute of Medicine report in 2007, and Congress’s passing of the 21st Century Cures Act in 2016, an entire field of “real-world evidence” has been created. The infrastructure has been put in place over the last decade. The recent confluence of data (electronic health records), technology (artificial intelligence) and compute power (cloud) creates an environment where a learning health system is possible and expected.
When healthcare finally does learn from everyday care, it will be transformed. Learning will support a better understanding of the unique characteristics each of us has, a recognition of how unique characteristics influence effectiveness of available treatment options, and a tailoring of care to the individual.
But, what if the learning health system learns the wrong lessons?
In 1989, the American Fertility Society recommended that all postmenopausal women be offered estrogen replacement. This was based on data from routine care, but it was a limited and biased data set. When studied in a more robust way in 2002, the Women’s Health Initiative showed that hormone replacement therapy had more detrimental effects than beneficial for many postmenopausal women. Based on bad data, millions of women received treatment that provided no benefit and increased risk of cancer and other diseases.
Learning the wrong lesson from bad data is not just a problem from long ago. In 2020, two of the most prestigious medical journals, the Lancet and the New England Journal of Medicine (NEJM), published manuscripts on treatments and risk factors for severe Covid-19. The Lancet results were retracted at the request of three authors who stated they could “no longer vouch for the veracity of the primary data sources.” The NEJM publication was retracted because the authors were “unable to validate the primary data sources underlying our article.”
People made decisions based on these recommendations. Patients were harmed, and confidence in the validity of evidence was shaken.
Why Data Quality Is Key
The lesson is clear: If we’re going to learn from routine care, we must protect patients by ensuring sufficiently high data quality to justify the recommendations.
The Food and Drug Administration, aligned with its mandate to protect patients, is in the process of defining a data-quality standard sufficient for their use of real-world evidence. This standard can enable key decision-makers such as doctors, insurance firms and regulators to determine whether real-world evidence is sufficiently credible to influence the standard of care. The data quality standard includes measurable accuracy, completeness and traceability.
There is hope for a bright future. Similar to earlier efforts in maps, shopping and music, companies like mine, Verantos, are deploying modern technologies to learn from the most robust healthcare data. But, data quality must be measured and paramount.
Without an emphasis on accuracy, completeness and traceability, there is significant risk. Not all firms generating healthcare evidence are using high-quality data or even measuring data quality; in fact, very few are. If we power this healthcare transformation with low-quality data, not only will we repeat the sins of the past, but their impacts will be felt a thousand-fold. Whereas relatively few treatment decisions today are guided by real-world evidence, in a learning healthcare system, nearly every treatment decision will be influenced by previous practice. Relying on evidence based on poor quality data would be catastrophic.
This transition to a learning health system is a critical time for the healthcare industry. The availability of electronic health data, compute power and artificial intelligence will be transformative. We must distinguish high-quality data from low-quality data and ensure that we learn the right lessons. We owe it to ourselves to reach that bright future. The payoff is better, safer, more efficient care for us all.