So it's pretty remarkable to have watched the evolution of the machine learning techniques being deployed in electronic medical records that has allowed us to do research at a scale never before done in history. So previously you might have opened up the charts manually for thousands of patients and now at my fingertips I have access to more than 5 million cancer patients where in just you know the snap of fingers we can pull out information that's trapped within paragraphs of notes about their experiences. So it's a really exciting moment in history to um be a part of this type of research. [Music]

Good. So, hello BL and welcome to the show. Um, I'm glad you know we could find the time for this episode. It will be actually I think the number 49 in the series if I'm not mistaken and it was the first time we have someone with a background in epidemiology like you. Um, so I'm very curious to hear you know about your journey uh and your work at the interface of epidemiology, economics, public health. Uh I think we'll also talk about your activities with flat iron health um around cancer research and cancer care. Um to be honest it's like an area that I'm not too close to um but I believe is also

then undergoing like you know some profound changes in terms of you know new tools and approaches um for us to understand to treat uh to cure certain types of cancer. Um so I'm looking forward to to your input on these aspects. Um I'll also underline that we're actually colleagues. So many listeners will know that my job my main job is with RO diagnostics uh which is part of the RO group where flat iron half also belongs. Um so I'll just highlight as I did in former episodes you know that the views and uh the opinions that I express in the podcast um are you know my own that do not

reflect those of my main employer. Um so that being said and you know to kick us off I would you know invite you as I do on the other episodes as well um to present yourself. Thank you. My name is Ble Adamson. I'm an epidemiologist, health economist, data scientist, and I work at Flat Iron Health where I am the head of outcomes research and evidence generation for international. Thank you for having me. — Good. So, what drew you know what drew you to work you know what was your initial interest at say you know I mentioned epidemiology, economics and public health in the introduction. uh what was your initial interest in these

fields? — Well, when I was a child, my family was affected by the HIV pandemic and this is what first drew me into being interested in microbiology and epidemiology with fascination of how diseases spread around the world and impacted populations. And later an interest in economics grew from seeing how resource allocation and policy decisions really were shaping health outcomes at scale and around the world. So the skill set that I've built over my career uh is bringing together these disciplines, combining data, math modeling, informing policy to drive better health decisions and outcomes. And a lot of that's in identifying things that are the most valuable to do. So is it a policy or is it a medicine or

a vaccine or a diagnostic and what are we willing to pay for these things? — I guess the you know one recent event you were I guess many of what you said crystallized was during you know the co9 pandemic I think where you were also very much involved um in in the US. Um can you tell us a bit about you know what has changed or I guess on the one side there is an aspect where it revealed some problems that we have in you know getting us ourselves prepared for pandemics and also what it changed for you as an epidemiologist in terms of you know consideration for your work or

maybe now resources available for that type of work. I guess there's a lot of you know change that happened because people were depending on the input of experts like you um to help us you know navigate um that that difficult period.

Yes, I was steeply involved and when COVID hit, I'd been serving as a private sector epidemiologist economist, but had built this incredible skill set and coding and modeling and real world data analysis abilities at during my time at Flat Iron Health. And so when the White House called and asked if I would come and serve as an emergency responder, um I understood that I had had advantages in on the job experience and training at Flat Iron that allowed me this unique ability to act very quickly and generating real-time insights for policy. Um so I served as the lead data scientist in the West Wing directly advising and working with President

Trump at the time. Um and it it exposed so many things about our country, about the world. um the lack of global uh

public health data infrastructure and the difficulty that that made for countries to um uh have policy informed by the data. Um so yeah I learned so much about global data infrastructure uh and really I think the biggest reinforced lesson was first listening to what the decision dilemmas are and then designing the right research questions and evidence to rapidly generate intelligence that can help reduce uncertainty in those decisions. Can you double click on that aspect when you say you know we're there there is a lag and maybe there's still a lack um to some degree um in terms of accessing public health information you know at scale. Can you give us some examples of

you know maybe certain types of information that you thought would be available when you kind of like jumped in when you realized okay actually we we need to put you know different pieces together because that's that's not there. Well, it's interesting. I know you mentioned that your your main job is working in diagnostics and we saw a lot exposed about um the performance of diagnostics during co um you know these rapid COVID tests were approved initially under emergency youth use authorization — but there was still a lot of uncertainty about things like are they turning positive if someone's infectious or not infectious or Um and this is I think a

chance where it was the real world data that was able to um to continue uh improving our understanding of the performance of diagnostic tests once they're deployed into the real world. Um and then when omccron hit you know for each new strain we had of COVID um are we still operating under the assumption that the sensitivity and specificity of these diagnostic tests is constant across every new strain or you know are is it changing and what's the responsibility of the manufacturer versus scientists and policy makers in the field trying to learn how to responsibly deploy these tools for public health benefit. And so I really saw this um shared responsibility and

opportunity of rapidly updating our understanding of um the performance of diagnostic tests. — So your mandate with the White House, how long did it last like was it like over the two years almost of the pandemic or was a particular point? believe. Yeah, I was there March, April, May of 2020 and then I was brought in in the fall for um operation warp speed uh when they were having difficulty with their trying to deploy machine learning models in a way that would help accelerate the vaccine clinical trials. And when that had difficulty, they brought me in as an a IML expert to be able to um uh help unblock some of the

um strategies that they were trying. Understood. Um so you know maybe we can shift gears towards you know your let's say more recent work with flat iron. So can you tell us a bit about and you mentioned you know AI ML um that you know your expertise in those in those fields. Um I think there's also an aspect in your current work that's related to real world data. Um I think it would be good for you know if you could you know define to us what that means exactly for those who might hear the term for the first time and you know how this combination of you know intellig artificial intelligence real

world data apply you know in your research that you do and work around cancer um cancer care. — Yes. Thank you. When I first finished my PhD and was looking at all the different, you know, job opportunities that are available, I was really looking for something that I thought was going to be the future of where things were moving. And I could see that old types of data sources like claims data, registries, physician surveys, um, a lot of them were like garbage in, garbage out for these models. Um but because electronic medical records which had more recently been adopted contained all of this information but trapped within the notes, you know, paragraphs of narrative

written by clinicians during their visits or within a lab report that was printed out, faxed to a site, scanned in as a PDF. Um, all of these things were are trapped within electronic medical records and Flat Iron Health was one of the first to discover how to pull out the rich clinically meaningful details from documents from this, you know, trillions of pages of content about these patients. Um really for me coming as an health economist um it gave the opportunity for me to compare the effectiveness of different cancer treatments for specific patients once I was able to learn from the lab

reports, the pathology reports um all of this rich information and that can be curated both manually by trained oncology nurses following standardiz policies and procedures which you know is can easily create regulatory grade you know rich real world data sets but also when I joined I could see that flat iron had one of the most innovative machine learning teams um where you know maybe 8 to 10 years ago things were more focused on natural language processing and beginning to train deep learning models for extracting things like the biomarker status of a patient mentioned um they've really continued to innovate and are now um deploying large language models to be able to perform a lot of

these tasks with the same accuracy as a trained oncology nurse. So it's pretty remarkable to have watched the evolution of the machine learning techniques being deployed in electronic medical records that has allowed us to do research at a scale never before done in history. So previously you might have opened up the charts manually for thousands of patients and now at my fingertips I have access to more than 5 million cancer patients where in just you know the snap of fingers we can pull out information that's trapped within paragraphs of notes about their experiences. So it's a really exciting moment in history to um be a part of this type of research. Can

you explain us you know what is the in a way what's the potential of like real world data because so to me my understanding of it is like real world data is data that is like you know as it says like in the real world so kind of like captured about patients outside of let's say a control environments like you know clinical trials um and I know that's you know there's a company called I think resilience in Europe that is also quite uh successful in terms of you know remote patient monitoring for cancer and I think what they kind like a kind of like worked on or transformed was the way you know patients were able

to you know enter um symptoms or you know responding to you know surveys about how they felt about the symptoms um you know digitally and this was you know look back to to the to the let's say the caregivers and the physicians and the oncologist and so on. Um and so to me this is you know a bit an example of how to leverage that type of information from the real world and how this is being fed back into the health system. Um is there is that is is this

my understanding kind of like correct or is there you know more to it? — Oh no you're absolutely right within this space of digital health there are so many different new types of sources of information. So patient reported outcomes as you're describing with um how patients are feeling and and functioning that really matters. Um and

there's an opportunity to learn both at a person level, so how is their personal experience changing over time and making predictions on which treatments may work best for them in the future and then also a population level. So once a new drug or diagnostic comes onto the market, we've understood how well it works in this perfect bubble of an clinical randomized clinical trial understanding the efficacy under idealized circumstances. But often drugs perform differently in the real world and the effectiveness based on adherence or a broader label indication with patients that might be older and more frail than those that were in the clinical trial. This is the opportunity where you can have 10 to 100

times more patients in the real world who you could learn from their experience than were ever included in that initial authorization clinical trial. So there's we can use that then to inform policies at a government level

or at a clinical guidelines level. So understanding that on average patients like this you know may have something we call a heterogeneous treatment effect. So on average a drug might work one way but within subgroups it may work completely differently. It may be that within this subgroup they gain a ton of benefit and this one it's neutral compared to another one and there might be some tiny slice of you know a genotype where that patient would be worse off if they received that. But if you had only looked at them on average you never would have known. So once we are able to expand to these massive sample sizes and across different

geographies, so different countries have different patient profiles, we're really able to tease out what's the right drug for the right person at the right time. — And I guess to some degree, it's also something that's being leveraged more and more by pharmaceutical companies, right? to develop well I guess first to develop new treatments but also identify throughout the development life cycle like which are the populations that respond the best versus you know are there so kind of like tailoring a bit the the way clinical trials are are done based on that input — absolutely — makes sense um so yeah the you know you mentioned in the in your introduction

that you know we are facing or one of the challenges we face when it comes to real world data is you know accessing that information um from you know an infrastructure perspective. Um I was you know wondering are there you know some or what are like some of the biggest barriers when it comes to you know real world data in oncology specifically um and yeah how do you or what are some ways we can we can address these these problems? M — well, it's interesting. You're sitting in Switzerland. I'm sitting in the United States right now. And we're both living in countries that have very different laws and regulations about how

to handle um personally identifiable information you or patient records. And with GDPR in Europe, AAPI in Japan, um it's necessary to be respectful and compliant of local laws and regulations while still also trying to find a way to benefit people with cancer by learning from experiences. And so Flat Iron has done a lot over the last 5 6 years to be

able to na navigate the regulations for patient privacy in in these countries to be able to allow us in the UK, Germany,

and Japan uh partner with hospitals, cancer clinics and be able to access the electronic medical records of patients in these countries, curate them in similar ways that we have in the US. Um and one of you asked you know what are some of the biggest barriers. Um previously it was very difficult to combine patient level data from different countries because often that data is trapped within that country's borders because of local laws. But now it is possible to curate and deidentify patient data sets within a country and then upload it to a trusted research environment which is a cloud-based secure environment where analyses can be done where now you can combine all the

patients from the different countries where you've got every variable that you need to use in your analysis. And so it's it's incredible that this this technical barrier has now been overcome um with trusted research environments that allow us to build this global

health data infrastructure that when I was working in the White House I only dreamed um be available. I think the you know we recorded I think it was episode 40 um with professor Jean Philip from Okin who you might know probably and we talked about you know federated learning which was the approach they were taking to you know leverage um local information from how systems they were collaborating with and you know he was explaining a bit how you know way more let's say elegantly than I could um but how they were able to leverage those you know that data and that information in a way that was not you let's say or going against you know

any confidentiality and you know um data safety issues. Um yeah, I was wondering you know if you could give us um an example of a maybe like a recent um AI model or machine learning model that you worked on that leveraged you know real world data and how it has helped let's say you know concretely the either you know our understanding of you know how cancer works or you know paving the way for for new treatments. M well just recently presented at the ASCO conference is our work with LLM extraction of CTDNA

and this is a really hot topic right now in the field of oncology because it's a measure that is starting to become more of the standard of care. Uh, but these ctna measurements are often within complex PDF reports and depending on what lab the CTA was CTDNA is measured at, the reports are completely different depending on the lab that ran them. And so harmonizing, you know, hundreds of thousands of pages of different labs, PDF documents trying to measure the same thing is an incredibly difficult task. And that's something that has been a huge breakthrough of our AI machine learning team being able to now um at scale be able to pull out every single

measurement of CTDNA for um millions of patients who have had it done. A second example um another thing that's a trendy drug right now that everyone's talking about are these GLP1 drugs, you know, WGOImpic. uh and there's there's already been studies showing that they are likely going to uh decrease the risk of developing cancer and there's new theories now about people using these drugs alongside cancer treatments and it affecting the tumor biology and slowing tumor growth. So, we were able to use an LLM model to identify more than 50,000

women with breast cancer who have used GLP-1 drugs at the same time as their cancer treatment, allowing us to match those to similar women with breast cancer who didn't use any GLP1s. So now our team is able to measure and look at, you know, did did they respond to to their cancer treatments better with while using a GLP-1 is their survival longer? Uh and so being able to understand the use of these other drugs even when you know the LLMs are able to pull out at the same time all this other information like why were they using the GLP1? Is it for weight loss or sleep apneoa or diabetes?

Um, and so to have all that information so quickly pulled out of the charts for us and curated. Um, I mean the benefit that of where I'm sitting at Flat Iron is I also still have access to these incredible oncology nurses who can validate whether or not the LLM is doing its job accurately. Um and so at the same time we can abstract hundreds of patients uh who are using these GLP-1 drugs to make sure that the LLM model is pulling out the details correctly but then we can apply that to hundreds of thousands of women to be able to learn very quickly and at scale um how this is

affecting the outcomes in cancer. — Yeah, super helpful. you know, I I think I re I listened to like um um an episode

of um I don't remember which podcast it was, but it was with Nav Raviant and it was so he's not like you know specialist about healthcare in general but one of his kind of like predictions about the future was to say well GLP1 I think we start to see some very interesting things that apply to cancer care. So beyond you know diabetes and cardio metabolic conditions and um so I find it quite fascinating that you know you're now looking into this and how it can be you know leveraged or combined to some of the you know current cancer treatments. Um before the next question I wanted to ask you what is CTDNA?

— Oh it's — to the first example that you mentioned. — Yes. um it's uh a lab measurement of how

much of the cancer is floating around someone's body. And so yeah, it's a measurement of h the volume of DNA from

the tumor is still in someone's body. So if if it's a way then to basically monitor patients more quantitatively over time whereas before there maybe were more binary outcomes like responded to treatment or didn't respond to treatment. It's this is a little more of a quantitative tool to be like, okay, we can see how the drug is making the measurement of CTDNA go down or there's no measurable um DNA from this tumor that we can detect in this person's body anymore. — Understood. Um so — I hope I got that correct. Yeah, — we may have people you may have most of your comments related to this like life really doesn't understand cDDNA but from

my understanding that that's that's how it's used. — Understood. Um so you've spoken about you know the trap data in unstructured clinical documents and um electronic health records. um what might be you know some lesser known let's say sources of you know valuable data in oncology that you know researchers might might be overlooking or that we don't think about at first. — Yes. Uh well there's so much that that doctors capture in their notes um where that aren't just what drugs are prescribing but give us clues to why they're prescribing certain drugs for certain patients. So trapped in those notes, you may find the reasons a patient has stopped using a drug or the

reasons why maybe they want to switch. Perhaps it's not because their disease has progressed, but because the side effects are not tolerable or the outofpay, the outofpocket costs for them in America are too high for them to be able to afford continuing this drug. And that those are important pieces of information for us to be able to learn from. Um, another could be about uh that we're investigating now through um details that we're pulling out with AI at Flat Iron in this oncology data uh are related to why physicians are prescribing certain ways or changing their prescribing behaviors after, for example, if let's say you're prescribing someone a drug and one of your patients

has a serious adverse event on it. How does that change how that individual physician keeps prescribing to more patients at their clinic in the future? — Um, so or you know, how does news about something coming out change the way that you you see physicians behave? So, I think there's a a lot of new rich qualitative information that we're beginning to um pull out uh in understanding why physicians give different drugs at different times. So you mentioned you know two nice examples of you know your recent work with at iron health um you know on the CTDNA aspect and also on you know on the understanding that GLP1s have for you

know enhancing potentially the cancer treatments. Um are there you know other areas in you know still in in this field of cancer care and research where um you see you know or where AI is currently underutilized or where there's still you know some kind of like untapped potential. M I see some of the greatest

explosion in innovation and research happening now in Japan. So this is really interesting because you know I I had really seen Japan as a country that was very conservative but neglected to appreciate how forwardthinking they are in adoption of new technologies. And so as Flat Iron now, we have a a a team and there's a Flat Iron KK, you know, Japan um office based in Tokyo where we are partnering with cancer care sites all through Japan and curating this EHR data. I think it's fascinating because you know there are many things in Japan where that explain why cancer um many

types of cancer uh have a different natural history of disease than they do in Europe or in the US. Um so for example gastric and colarctal cancer happens at way higher rates in Japan and

over the last three decades we can we're now seeing that gastric and colorectal cancer are happening to younger and younger people in Japan um like at the age that they get diagnosed and there's a lot to unpack and unravel there about what does that mean for what treatments work best for someone living in Japan with colarct al cancer who gets diagnosed at a really young age, you know, should they be receiving a different treatment than an elderly person who's frail in the US, should they be getting a different dose? Um, is their side effect profile going to be the same as a clinical trial um that maybe didn't even include Japan in it?

And it's there that I see a lot of opportunity also for using large language models. Now um when I my I started when I was deeply embedded on the machine learning team that was training deep learning models one of the limitations is for a deep learning model type like LSTM you're training it on data that's in the English language and so you're limited to applying that model to patients that are documented in the English language. So it held us back from taking deep learning models and having them be useful even in they don't even work as well in the UK. Like British English — and American English oncology are two

pretty different things. So you wouldn't even expect a model to perform as well in in London as it does in New York. But now that we have these LLMs that have been trained in every single language possible and even some that have been fine-tuned for the complexity of the Japanese language, this now is allowing us to write prompts in any language, have them read medical charts written in any language, and return back to you the information you wanted to learn about that patient in whatever language you request. And so this I think is really going to unlock so much for Japan of

oncology research at scale. So now that Flaturn um really has made the investments for building uh EHR data infrastructure throughout cancer clinics and hospitals in Japan. I think we're going to see a huge leap forward over the next five years that are going to benefit not just people with cancer in Japan, but in many other countries as we continue to pioneer the application of AIML in um figuring out which drugs work

best in in different settings. You know what it made me think um what you just said regarding you know our ability now to understand let's say um yeah information from across geographies in a way I believe as well that this is also a way to kind of like um because I understand as well you know medical practices are different from you know one country to another and know from let's say my focus of my work which is mostly on cardio metabolic conditions we have you know the European Europe European Society of Cardiology that establishes those guidelines. We have the American Heart Association in the US. We have different ones also in in

Asia. And so I guess that you know the fact that we are able to leverage real world data from different locations in different languages does a potential to inform let's say clinical guidelines at a broader scale than how it's currently made you know regionally in a way. Do you have you know some some thoughts in that regard? No, you've got it. Exactly. That's exactly the type of use case that is possible now because of LLMs. It's harmonizing things across countries now in a way that was incredibly difficult to do before, especially when you need when it was it's local experts that developed clinical guidelines in their local languages. But you know it's interesting

too because even a country like Germany each physician in Germany each oncologist that of when I'm digging into different charts often describes cancer in a completely different way from each other or has different strong opinions about what they think is actually the best treatment or not. So even within a country you can see differences between physicians but there's the opportunity now with this harmonization across countries and across languages that we didn't have before in the ability to figure out maybe you know these types of cancer doctors in Germany have figured out something really special or maybe these ones in this part of Japan for this type of cancer that they see more

of maybe they've figured out something and there are ways that we can then have that information be better shared between countries whereas a lot of those were trapped in guidance documents written in languages that were not easily accessible. I mean even look at systematic literature reviews that we see you know in in PubMed those often

the ones that I read have always said this was restricted to manuscripts published in English you know and there is a lot of good research that comes out in local languages presented at more regional conferences you know that uh have captured really incredible insights that I think are now becoming more accessible with thanks to LLMs. — Could you share with us you know some of I guess some of your hopes in terms of you know what um artificial intelligence some of the you know current or future um computational methods that we have at hand. You know what it would change in terms of um you know um yeah how we

might approach cancer in the next you know 5 to 10 years. M well I think that we're going to see AI ML really speed the time to discovery of

new drugs. We're going to um reduce the uh the effort that it takes to test and understand the safety and effectiveness of new drugs. And once drugs become available on the market, it's going to allow us to in real time better update the information that we know about the safety and effectiveness of new medicines so that we can be using them in the most responsible ways. Uh I think

though that there's going to be a backwards like if you think chronologically like how long is it going to take for us to do all of those things? think we're in the moment right now where we can start capturing that third category of benefit, understanding how well drugs work in real time, you know, in the real world. We can do that now with AI for the identification of new molecules in development. I think that this is where it will be really interesting to see which pharmaceutical companies make the choice of integrating a IML into that discovery stage identification of new molecules. But it will probably take us you know 5 to 15 years to figure out

the payoff of that. — Yeah. Yeah. The benefit — because we need to see like you know is the probability of success actually higher? Well, if it takes 10 years to develop a drug, we may have to take wait 10 years to figure out if the companies that applied those technology or are having a higher success rate than those who chose to wait and be more cautious. — Yeah. Yeah. So looking at you know either number of cleared let's say treatments in you know in the evolution of that if there's a clear increase or yeah level of success of clinical trials are probably some good measures to to look into for the future. Um so BL I'm

you know kind of like keeping an eye on the time and um you know it's been a very very very interesting you know conversation for for me. I learned a lot about you know cancer research and how you know some of these new technologies like AI and so on are applicable and I guess he gives a lot of hope because it's still an area from what I know where there's also like quite a I mean there are people who have let's say not very high risk profiles who still have you know to deal with certain types of cancer so there's also like an amount of like bad luck in it and so you know the

better treatments we can find and and propose I guess the the better it is for society in general. Um for you know our listeners who might be you know interested in learning more about you know the field in which you work and um and you know deep dive into some of the topics we mentioned today um be it you know through books, publications or websites um what what resources would you like to to share with them? — Well I enjoy reading books about science that are written by journalists not scientists. I think that they're better storytell I think journalists are really good storytellers. Um not related to cancer but one of my favorite books is

called Spillover by David Quan and it reads like an Indiana Jones adventure. Um that's one of my favorite books. Um I have a website lifeadamson.com where I blog about different things that I think are interesting. Um but I I don't have a favorite AIML book yet. Um,

— okay. I'll put the links in the in the show notes. Um, can you share with us an additional maybe anecdote from your work at Flatron Health, you know, that made you realize the the impact that you were having on on people's lives? H it would probably be from when the United States Congress was debating uh health insurance and Medicaid expansion which

is offering more health care coverage to the working poor. And when they referenced our flat iron study that showed how that Medicaid expansion reduced racial disparities in access to timely cancer treatment. Um it was just very inspiring to see how cutting edge research using electronic medical records was on the forefront of a congressional debate and informing new laws and regulations in our country. really cool. Um, if you would recommend, you know, a fellow healthcare innovator um, as a potential guest for the show, um, who would that be and why would you recommend her or or him?

— I would recommend the healthc care innovator Marta Brilic Karns. She's the founder of Pomelo Care, which I think is a really fascinating, innovative startup company in maternity care um that's providing um virtual and technology access so that pregnant women and newborns can have um full more comprehensive care. So, they're doing quite a bit at the intersection of technologies that can benefit this in a new way. So, I'd recommend you talk to her. — Yeah, that sounds good. And I think we I've been wanting to do, you know, an episode on, you know, kind of like either, you know, health tech or whim the indication on, you know, women's health. So, yeah, I'll consider it for

sure. — BL, thank you so much for, you know, coming on the show and, you know, sharing your your story and the inspiring work you do with, you know, Flat Iron Health. I'll continue to, you know, follow closely your activities. Um, as I mentioned, I'll put, you know, all the all the links in the show notes. Um, anything else you want to to to end with? — Just stay tuned to more exciting work uh coming out of Flat Iron Health. — Thank you so much. — Thank you. — Thanks for watching the episode. I hope you enjoyed it. You can subscribe by clicking on the podcast channel at the top right and watch another episode here

at the bottom.