Failure in Science

Targeted gene delivery to the lung

A five minute primer on our new paper, out at Science Immunology!

Using local cytokine delivery to prevent lung pathology

We have an exciting new story out at Science Immunology! It uses AAV-mediated cytokine delivery to change the lung environment. We can boost lung Tregs or deliver anti-inflammatory cytokines, preventing fatal respiratory collapse.

This story started during the COVID pandemic. Our first Cambridge PhD students joined during lockdown, and a talented student, Ntombizodwa Makuyana (now Dr Ntombizodwa Gentry), wanted to work on a potential therapeutic.

James and I had only just moved to Cambridge, and we still had a team in Belgium. They were processing clinical samples from COVID patients, so we already had a good idea that the respiratory failure was driven by excessive inflammation.

At the time we had been working on a system to boost Tregs in the brain. Our AAV system was working great, so we thought "what if we tried to do the same thing in the lung?" It took quite some trial and error, but eventually we found that intranasal delivery of AAV6.2 with the CC10 promoter limited expression of our cargo limited to the lung.

The system works great! AAV6.2.CC10 driving IL2 production gives an expansion of lung Tregs without impacting other sites, even the draining LN of the lung. And it is not just IL2 - in the same way we can drive IL10 or IL1RA in the lung without altering systemic levels. Pick your own cargo!

We never ended up testing it in SARS-CoV2 infection - by that point the vaccine had come along. But COVID isn't the only important lung infection. You might not even have heard of one of the most deadly: Influenza-Associated Pulmonary Aspergillosis (IAPA).

Aspergillus is a fungus common in decaying soil, that can infect the lung. For healthy individuals it is rarely a problem, and is easily cleared. But Aspergillus is a hidden killer. The Joost Wauters, Greetje Vande Velde and Stephanie Humblet-Baron teams teams had found that coinfection of influenza and aspergillus is extremely deadly, with an ICU mortality rate of >50%, and lethal coinfection in mice.

So James, Oliver and Milla headed over to Belgium and teamed up with Laura Seldeslachts and Lauren Michiels to test our AAV-cytokine delivery approach in coinfected mice. Success! In every measure we tested, our treatment reduced severity from fatal respiratory failure down close to a regular flu infection.

The best part is, because we only altered lung immunology, the anti-viral responses from the LN were intact, so we could reduce lung inflammation without giving the infection a free-pass. This is why tissue immunology has such potential - only hit the site needed!

Thanks to the ERC, Wellcome Trust and FWO for funding. 

Read the full story here

AI solves enigmatic immune disease?

Unfortunately, in my write up of our new paper, I committed the cardinal sin of science communication:

I forgot to say we used machine learning!

Clearly I should have led with "AI solves enigmatic immune disease!"

This paper is actually a pretty good example of what AI actually does for understanding immune disorders. Maybe it gives an incremental advance over modern statistical methods. Random Forests (the best performing AI approach) identified the same immunological signatures as multivariable logistic regression (a conventional statistical analysis), so we didn't learn any new biology from AI.

It did improve diagnostic AUC by ~2.6% though. That's not nothing - incremental improvements are at the heart of clinical advances. I'll take a ~2.6% improvement where ever I can get it.

But will even that ~2.6% increase in diagnostic discrimination be reproducible and useful? Here I am not so sure. The main immunological signatures look very robust, but the "AI boost" doesn't come from identifying additional signature factors, it comes from interaction between these signals, and that interaction is the part that is most prone to over-fitting. On a repeat study, the overall diagnostic capacity is likely to drop (regression to the mean), and I suspect that the "AI boost" of ~2.6% may drop to 0% or even be negative. If we increase the complexity of the AI model, we do find that it performs worse than the classical statistical model, which points in that direction.

I also doubt that any clinical diagnostic test would use the "AI boost", even if it was reproducible and robust. Actual clinical diagnostics are usually highly simplified, focused on fewer parameters than the research-grade complexity we used here. So if this advance gets into routine clinical diagnostics (which I hope!), I suspect it would focus on just the very strongest individual immune signals, which were identified by both AI and classical statistics.

So overall, AI was a marginally-useful tool in this study, which helped at the edges. But it wasn't transformative. That isn't just a feature of our study - and it isn't simply a matter of better machine-learning algorithms being built. It is a fundamental limitation of AI - medical data is rarely of the right data structure to give the types of advantages you see in other areas (like weather prediction or molecular structure prediction). AI is another useful tool, and every new tool helps. But it will hardly "transform medicine", and only replaces experiments in niche use cases.

Let's use AI where it helps, without contributing to the excessive hype.

AI solves enigmatic immune disease?

Unfortunately, in my write up of our new paper, I committed the cardinal sin of science communication:

I forgot to say we used machine learning!

Clearly I should have led with "AI solves enigmatic immune disease!"

This paper is actually a pretty good example of what AI actually does for understanding immune disorders. Maybe it gives an incremental advance over modern statistical methods. Random Forests (the best performing AI approach) identified the same immunological signatures as multivariable logistic regression (a conventional statistical analysis), so we didn't learn any new biology from AI.

It did improve diagnostic AUC by ~2.6% though. That's not nothing - incremental improvements are at the heart of clinical advances. I'll take a ~2.6% improvement where ever I can get it.

But will even that ~2.6% increase in diagnostic discrimination be reproducible and useful? Here I am not so sure. The main immunological signatures look very robust, but the "AI boost" doesn't come from identifying additional signature factors, it comes from interaction between these signals, and that interaction is the part that is most prone to over-fitting. On a repeat study, the overall diagnostic capacity is likely to drop (regression to the mean), and I suspect that the "AI boost" of ~2.6% may drop to 0% or even be negative. If we increase the complexity of the AI model, we do find that it performs worse than the classical statistical model, which points in that direction.

I also doubt that any clinical diagnostic test would use the "AI boost", even if it was reproducible and robust. Actual clinical diagnostics are usually highly simplified, focused on fewer parameters than the research-grade complexity we used here. So if this advance gets into routine clinical diagnostics (which I hope!), I suspect it would focus on just the very strongest individual immune signals, which were identified by both AI and classical statistics.

So overall, AI was a marginally-useful tool in this study, which helped at the edges. But it wasn't transformative. That isn't just a feature of our study - and it isn't simply a matter of better machine-learning algorithms being built. It is a fundamental limitation of AI - medical data is rarely of the right data structure to give the types of advantages you see in other areas (like weather prediction or molecular structure prediction). AI is another useful tool, and every new tool helps. But it will hardly "transform medicine", and only replaces experiments in niche use cases.

Let's use AI where it helps, without contributing to the excessive hype.

New insights into a mysterious inflammatory disease

We have a new story out at Nature Communications!

This time our lab tackled the enigmatic "autoinflammation of unknown origin". These #autoinflammatory patients don't fit the criteria of classical syndromes. It wasn't even clear they were a single group, to be honest

We worked with the amazing Dr Carine Wouters and clinicians across Europe to collect samples for a systems immunology analysis. Critically, we we able to access samples at the point of diagnosis, many still untreated, so we could see the primary immunological effects. A decade (yes! planning started in 2016) of sample collection and flow cytometry data generation followed, led by the KU Leuven team under the leadership of Prof Stephanie Humblet-Baron. Our own Dr Rafael Veiga led the data analysis.

Fast-forward through the slow science and the final outcome is that immune status distinguishes patients compared to healthy controls (AUC 0.83), with CD38+ T cells elevated and memory B cells way down. This shared phenotype suggests autoinflammation of unknown is a distinct condition, not a diverse set of patients let down by the diagnostic process. That by itself was a significant clinical observation!

But the patients already knew they weren't healthy. The clinical challenge is to distinguish them from other autoinflammatory conditions. Fortunately, we ran in parallel samples from confounding conditions with similar demographics, and the patients still had a distinct immune profile! It was striking, however, that the signature immunological changes were shared with patients with Still's Disease. We could still distinguish the conditions (AUC 0.79), but changes such as CD38 and BAFF moved in parallel between the conditions.

We wrote the paper up, submitted and waited for the reviewer comments. Quite constructive and improved the clinical aspects of our paper. The hardest ask was for serum proteomics on all samples - fortunately Dominique De Seny's team at Universite de Liege had being doing just this! Sometimes luck is on your side!

Completely independent immune phenotype platform, and the same message - autoinflammation of unknown origin patients clustered together, and shared many signature changes with Still's disease patients

Could autoinflammation of unknown origin and Still's disease share an immunological basis? Could the treatments used in Still's disease work in the other patients? Right now, we don't know, but perhaps we are finally on the right path to finding out!

Many thanks to the research teams at Cambridge, Leuven and Liege, the clinical team, and most of all the patients who were at the heart of the study! May your contribution help find new treatments!

Read the full paper here.

Tissue Tregs in Annual Reviews of Immunology

Our latest review is out, a comprehensive synthesis of tissue Tregs. It has been a decade since Annual Reviews of Immunology last reviewed tissue Tregs, and there have been enormous advances and conceptual leaps forward in the field.

Tissue Tregs have now been found in essentially all tissues, and have broadly conserved properties of enhancing repair and rejuvenation as well as controlling local inflammation. While the impact on tissues differ, molecular mediators are largely shared across tissues. The molecular cues that induce the tissue Treg phenotype are only partially understood, but key external signals from the tissue environment seem to be important in upregulating a core transcription factor set, which remodels the epigenetic and transcriptional landscape.

The cellular kinetics are not fully understood, however the majority of evidence using parabiosis, TCR retrogenics, cell transfers and fate-mappers suggest that the majority of tissue Tregs are pan-tissue, multi-tissue or tissue-cycling in their behaviour during homeostasis.

We also cover the increasingly promising attempts to exploit the properties of tissue Tregs in the clinic, and outline the key open questions for the field. 

Read the full article here.

Wetenschapper in wording

Spreek je Nederlands? Wetenschapper in wording is nu ook in het Nederlands verkrijgbaar! Dankjewel Liesbeth Aerts en Annelies Van Dyck.

Self-Doubt: An Anthology of Experiences in the Biomedical Sciences

Introducing our new book, "Self-Doubt: An Anthology of Experiences in the Biomedical Sciences". 

Have you ever had a crisis of self-doubt? A feeling that you are out of your depth and are not cut out for a career in science? I have. At the time I thought it was just me.

After decades of mentoring PhD students and postdocs, I now believe self-doubt is near-ubiquitous, an occupational hazard in science. The hardest part of dealing with your self-doubt believe you are alone in your thoughts. So my lab members and alumni have shared their own stories of career self-doubt.

If you know anyone in science who is doubting their path, please share this book with them (Amazon, Great British Bookshop) so that they know they are not alone

Lung Tregs at the Midwinter Conference

I am very fortunate to be at the Harald von Boehmer Midwinter Conference, courtesy of amazing conference organisers Ludger Klein and Lisa von Boehmer. Highly recommended as one of the best immunology conferences around - I've really been inspired by the great talks in every session. For those of you who would like a taster of the conference, here is my presentation - all unpublished work, covering FlowCode analysis of tissue Tregs, resolving spectral flow cytometry limitations through AutoSpectral, and using a novel AAV-based system of cytokine delivery to the lung to treat Influenza-Associated Pulmonary Aspergillosis. A sneak peak at the work currently going on in the lab!