I had a family member reach out and ask my thoughts on GRAIL’s Galleri blood test recently.
So after going through a couple of their study papers, I thought I’d post publicly what I have learnt.
This technology looks to have a lot of future promise. However, version 1, which is out currently, still needs a lot of work.
The below is what I shared with my family member:
Short(er) Answer
1) Study shortfalls
They designed their only study, to date, in such a way that it makes the efficacy of the test look better than it is. The headline figure I took from it, is that they have a 51.5% cancer detection rate.
However… if the key idea is “early detection”, we’re at most looking for detection at stage 2 (I think?), but ideally stage 1.
So in their cohort of 2,823 patients known to have cancer, they were able to detect 16.8% of the patients who had stage 1, and if you include stage 2, then the sensitivity went up to 27.5%. Including 3 and 4 gets them to 51.5%. See table 2 in their study study for this data.
2) Potential study manipulation
*Putting on my sceptical hat* – I can’t tell if they designed their study in such a way that they included people with cancer types that they think they might have higher predictive value on. Essentially these would be cancer types where you naturally get more “circulating cell-free DNA (cfDNA)”. For example, it seems prostate cancer doesn’t have much cfDNA, and thus if you avoid them, you could boost your total sensitivity.
3) Risk of false positives
With a 99.5% specificity, it means 1 in 200 people might get a false positive. That’s quite high. Therefore, prior to taking the test, it’s probably worth coming up with a plan of “what would I do if I got a positive result, before I panic unnecessarily”. It’s unclear to me how much re-doing the test would help, as the false positive could be as a result of the way the test works (machine learning classifier issue), rather than a mistake made somewhere along the way.
4) Other tests to be aware of
Another interesting testing modality I’m keeping an eye on are these full body MRI scans. Afaik they require solid tumors to be above a certain size already (maybe 1.5cm). Prenuvo is the popular silicon valley one that’s so far only available in the US.
Longer answer:
This seems like a very exciting technology, which in the long run, could become very useful.
As far as I understand, they built a machine learning model (classifier) to look at epigenetic changes (cytosine methylation) in the cell free DNA in the blood, which indicates if regions of DNA are being transcribed, or silenced, with the intention to test for early cancer signs.
How they did this is explained in this study.
The study that their results are based on is this one. This is the only study we can draw from currently, however there’s an NHS trial due to publish results this year.
In fairness to the study writers, their graphics and tables do help us tease out the actual metrics for how well the test can detect certain cancers and cancer stages.
I’ll include the most useful ones below:
Figures A and B are relatively self explanatory, but they explain more about them in the study paper.
Figure C covers the 12 cancer types that they pre specify as accounting for 2/3 of cancer deaths in USA *and* they estimated that they’d have better success rates with.
It shows how for some of them they had good success at early detection, others not so much.
And Table 2 gives us an idea of overall detection success at the different stages. Go to the paper to scroll down (if you need to).
Roundup
As mentioned before, this technology looks to have a lot of future promise. However, version 1, which is out currently, still needs a lot of work.
I think there’s a good chance that as they iterate on their machine learning classifier model, they can increase their success at detection.
One easy win is to increase the amount of training data they feed it.
So the future versions of their tests could get a lot better! Which is exciting.
In the meantime, hopefully the results from the UK’s NHS trial, due this year, will help inform the effectiveness of the test.
Resources I found helpful
- The main study where they tested their model in humans, and estimated its accuracy – link
- Podcast where Peter Attia interviews someone who works on liquid biopsy techonology – link – it gets more relevant around 1hr 30mins point.
- The study explaining the building of GRAIL’s machine learning classifier – link
- YouTube video on how bisulphate sequencing works – link