Can synthetic brain MRI from latent diffusion models train diagnostic classifiers?

By Soheil Fallah · Data Scientist & AI Consultant · peer-reviewed researcher in generative AI

Published 24 June 2026 · Updated 6 August 2026 · 4 min read

A classifier trained only on synthetic brain MRI reached an AUC of 0.754 on real test scans. The same architecture trained on real data reached 0.810. So the generated scans carried genuine diagnostic signal, though they did not fully close the gap to real data. Usefulness for training is also a separate question from whether patient privacy held, and both need reporting.

Why train on brains that never existed?

Medical imaging datasets are small and hard to move. Hospitals hold scans under strict privacy rules, and building a large labelled cohort is slow. One way around this is to train a generative model on the scans you already have, then use it to produce new labelled scans you can train on or share. The hard word is "useful." A generated scan can look convincing and still teach a classifier almost nothing. My dissertation set out to measure usefulness directly, not realism.

What I did

I built a two-stage generative pipeline on a 295-subject cohort from the Alzheimer's Disease Neuroimaging Initiative (ADNI): a VAEGAN autoencoder to learn a compact latent space, then a conditional latent diffusion model trained inside it with classifier-free guidance. Conditioning on the diagnostic label is what makes the output a labelled training set rather than an unlabelled pile of plausible brains.

I then used that pipeline to generate a synthetic training set for separating Alzheimer's-stage subjects from cognitively normal ones, trained a classifier on the synthetic scans alone, and tested it on a held-out set of real scans. This is the TSTR protocol: train on synthetic, test on real. As a reference I trained the same classifier on real scans and tested it on the same real set, which gives the TRTR baseline (train real, test real). If you want the protocol on its own terms, I wrote it up separately in TSTR vs TRTR.

A 295-subject cohort is small for generative modelling, and that was the point. Most of this literature is demonstrated at a scale hospitals rarely have.

The result

The synthetic-trained classifier reached an AUC of 0.754 on real scans. The real-trained baseline reached 0.810. That 0.056 difference is the cost of routing training through the synthetic pipeline. A model that never saw a real scan while learning still classified real scans well, and not far below a model trained on real data.

One framing to avoid: 0.754/0.810 is about 93%, and it is tempting to write "93% of the diagnostic signal retained." AUC's floor is 0.5, not 0, so the share of above-chance discriminative signal retained is (0.754 − 0.5)/(0.810 − 0.5), or roughly 82%. The honest headline is the gap itself.

What the result does not mean

It does not mean synthetic beat real. It did not, and the gap is worth stating plainly. It also says nothing about privacy. TSTR measures task usefulness only. A generator that quietly memorised and replayed real patients would post a high TSTR score while leaking the very data you were trying to protect, so a separate privacy check belongs next to the utility numbers.

I used a nearest-neighbour memorisation check for that, and it turned out to need fixing before it could be trusted: the standard formulation is confounded by set size, to the tune of 33 percentage points, which means the number it reports depends on how many samples you happen to compare rather than on how much the model memorised. Corrected, the pipeline showed no severe memorisation across the 880 samples checked. That correction matters more than the headline AUC, because a memorisation metric that moves with set size will reassure you exactly when you should be worried.

The figures here are also tied to one dataset, one task, and one resolution. I would not assume they transfer to other modalities without testing.

Where this leaves things

Synthetic brain MRI from a latent diffusion model looks promising enough to be worth testing as augmentation on a small cohort — but that is a claim this experiment did not measure. TSTR trains on synthetic data alone; whether mixing synthetic with real actually beats real-only is a separate arm, and mixed training can underperform. Measure it before assuming it.

It is also not yet a stand-in for real data. If you report TSTR, put the TRTR baseline beside it so the gap stays visible, and report a privacy measure so usefulness is not mistaken for safety. For the modelling side of that, see latent diffusion models for medical imaging.

References

Related

Related