A recent comparative evaluation on brain tumour segmentation captures the field’s situation in one sentence. Its authors find that several foundation models achieve Dice scores of up to 0.894 when supplied with extremely accurate bounding box prompts, which exceeds the segmentation performance of the reference specialised architecture, but that this architecture remains the dominant medical image segmentation network due to the impracticality of providing such accurate prompts.
In other words, the generalist model beats the specialised one provided it is told exactly where the structure is. But if the location is already known to that precision, the problem is largely solved.
This article examines what the available data actually says about this transition and its consequences for corpus construction. It extends the complete guide to medical image segmentation and closes this series.
Two medical image segmentation paradigms
The two approaches differ not only in architecture but in what they presuppose of a project. The specialised paradigm trains a model on the precise task at hand, from a corpus annotated for it. Its most developed form is a self-configuring framework that automatically adapts its architecture and parameters to the characteristics of the data. It demands a corpus and produces a model that does one thing, very well. The foundation paradigm, by contrast, trains a single model on massive heterogeneous corpora, then steers it towards a task through a prompt, click, box or text. It demands no specific corpus and produces a model that does many things, less well. The question that actually interests a project is not which is superior in the abstract, but which reaches the clinical objective at the lowest total cost, corpus included.What medical image segmentation comparisons say
Several evaluations conducted under comparable conditions converge, and their figures deserve reading precisely. A study published in 2026 on bone metastasis segmentation in CT, covering 88 patients and 11,006 expert-annotated image-label pairs, reports Dice scores of 0.6280 for a medical foundation model, 0.4480 for a fine-tuned general foundation model, and 0.6849 for the specialised framework. A second study, on pelvic segmentation in magnetic resonance imaging, observes that foundation models used without adaptation score ten to twenty per cent below the specialised framework on well defined convex structures, and encounter great difficulty on disjoint bilateral structures, where scores fall below 0.6. These results together sketch a coherent profile. On a defined task with an annotated corpus, the specialised approach remains ahead. The gap is not enormous and it is consistent.The prompt paradox
The result cited in the introduction deserves developing because it is routinely misread. Foundation models do indeed surpass specialised models when supplied with a very high quality prompt. But that prompt encodes precisely the information that makes the task difficult: knowing where the structure is and roughly how far it extends. A medical image segmentation evaluation supplying prompts derived from ground truth therefore does not measure what it appears to. It measures the model’s ability to refine a location already given, which is useful but different from autonomous segmentation. The practical consequence is twofold. To compare two approaches honestly, the prompt must come from a realistic source, human or automatic, rather than from the reference. And for assisted use, where an annotator supplies the prompt, this limitation is not one: the paradox dissolves once a human is in the loop, which explains why these models are far more convincing as annotation tools than as autonomous systems.Where foundation models win
An exactly opposite result deserves equal attention, since it identifies the real weakness of the specialised approach. A study on adapting a foundation model to volumetric segmentation observes that, on data from a different source than the training set, the specialised framework encounters complete failure in the zero-shot context on magnetic resonance images, obtaining only 10.9 per cent Dice, while the adapted model retains a performance of 60.4 per cent. Ten per cent against sixty per cent. The gap here is far larger than the one observed under favourable conditions, and it runs the other way. That result is probably the most important in this article for a real project, because the situation it describes is frequent. A model trained on one institution’s data meets, on deployment, images from another scanner, another acquisition protocol or another population. Robustness to domain shift is not an incidental property; it is often what decides whether a deployment succeeds.Specialisation closes the gap
A third result completes the picture and it is decisive for the question at hand. The same pelvic segmentation study observes that after fine-tuning on the project’s data, the lightweight foundation model reaches 0.864, on par with the specialised framework, with no significant difference across most structures. In other words, the initial gap reflects not an intrinsic limit but the absence of specialisation. And specialisation, under either paradigm, runs through data annotated for the task at hand. That is precisely where the analysis touches corpus construction. The debate between paradigms is not about whether annotation is needed but about how much and of what.Multi-organ models already available
One concrete development deserves separating from the theoretical debate, since it produces measurable effects today: the availability of public models covering dozens of anatomical structures. These models change the starting point of a medical image segmentation project on standard structures. Where an initial corpus had to be built, it becomes possible to produce a complete pre-annotation and concentrate human effort on correction and on the structures not covered. Three reservations accompany that, and they follow from the above. These models inherit the quality of their training corpora, and several studies have documented imperfections concentrated on particular structures. Their coverage is very uneven by modality, CT being far better served than ultrasound or pathology. And their performance on pathological cases is markedly below what is observed on normal anatomy, whereas pathological cases motivate most projects. The practical consequence is to use them as a starting point and never as a reference. A project accepting their outputs without verification imports their biases, and that import is all the harder to detect for being uniform.What changes for annotation
Three shifts are emerging for annotation in medical image segmentation, and denying them would be as unserious as exaggerating them.Volume falls on common tasks
This is the most visible shift and the most commented on. For a structure well covered by available models, a compact contrasted organ, the number of studies needed to reach a given performance falls markedly. A project that would have required several thousand studies a few years ago now requires a fraction of that. That shift is real and should be acknowledged. It does not mean the need disappears but that it narrows onto the cases where annotation contributes something the model does not have.The nature of the work shifts
This second shift is less visible and weighs more on organisation. The dominant gesture moves from drawing to correcting and verifying, which changes the competence profile required: less dexterity, more judgement and convention discipline. That shift has a consequence projects underestimate. Correcting requires knowing when not to correct, a decision resting on knowledge of the protocol rather than on perception. An assisted annotator poorly trained on conventions produces a worse corpus than an unassisted annotator well trained on them.Some tasks resist
This is the category most of the residual need depends on, and it deserves delimiting precisely. The results cited identify exactly where generalist models fail: disjoint and bilateral structures, non-convex shapes, conventional boundaries. To which are added tasks where the difficulty is not perceptual but definitional, meaning those where the question is not where the boundary is but where it should be placed. These tasks are not residual. They constitute most of what has clinical value, since a structure a generalist model segments correctly without specific data is also a structure whose segmentation differentiates nobody.The shift towards evaluation
One less commented effect deserves flagging because it may become the most important. As training requires less annotated data, the relative share of evaluation grows. A model fine-tuned on few examples must nonetheless be evaluated rigorously, and that evaluation requires a high quality test corpus, annotated under a demanding regime and representative of deployment conditions. That evolution appreciably changes the profile of demand. Fewer studies, but annotated with greater care, with multiple annotation and arbitration, and deliberately covering difficult conditions rather than average cases. It also changes the nature of the value delivered. Building a representative evaluation corpus means understanding deployment conditions, stratifying by site and by population, and measuring inter-expert variability to situate performance. That work belongs more to method than to production.Modalities that remain poorly covered
One asymmetry structures the landscape and is rarely stated: not every area of medical image segmentation advances at the same pace. CT concentrates most public corpora and available models, for understandable reasons: standardised density values, regular geometry, abundance of studies. It is where automation advances fastest and where the annotation need narrows most sharply. Magnetic resonance imaging follows at a lag owing to the absence of absolute values and the multiplicity of sequences, which multiplies the configurations to cover. Ultrasound and pathology remain clearly behind. The former through its operator dependence, quality variability and frequent artefacts; the latter through the gigapixel scale of slides and staining variability between laboratories. That asymmetry has a direct consequence for buyer and provider alike. Conclusions drawn from CT do not transfer, and a project in ultrasound or pathology today sits in a position comparable to CT some years ago, with the volumes and methods that come with it.What will not change in medical image segmentation
Three elements resist any evolution of the models, and they are worth stating explicitly. The first is definition. No model can decide whether papillary muscles belong to the cavity, whether the septum enters right ventricular mass, or how far to extend an incomplete fissure. These questions are conventional and rest on a documented human decision. The second is clinical qualification. Deciding that an image genuinely shows the pathology sought, rather than an artefact or a variant, rests on medical judgement that no training corpus transfers entirely. The third is measurement. Establishing what is attainable requires knowing the variability between experts on the task, and that ceiling follows from no model. It is measured, and it is measured on human annotations produced for the purpose.The regulatory frame as brake and driver
One dimension outside the technical debate bears on this transition and it pulls in two directions. As a brake, the requirement to trace training data sits poorly with a model whose corpus is massive, heterogeneous and partly undocumented. A medical device must demonstrate the provenance and quality of its data, which is simpler on a corpus built for the project than on a foundation model whose training is outside the manufacturer’s control. As a driver, the same requirement favours fine-tuning on owned data, since it gives the manufacturer control of at least the final stage and a documented corpus to present. The regime emerging therefore combines a generic model that is inherited and a documented specialisation that is owned. That division has a consequence for corpus construction. The fine-tuning corpus and the evaluation corpus become the elements the manufacturer must control and document fully, which tightens the quality requirement on reduced volumes rather than relaxing it.How to prepare
From the standpoint of a team building a medical image segmentation corpus today, four decisions extend the working life of the effort. Annotate in the richest format the data permits, with full geometry and metadata, since conversion to a lighter format stays possible and the reverse does not. Keep individual annotations alongside the reference mask, since methods exploiting variability are developing and that information is irrecoverable once fused. Document the protocol and version it, since a corpus whose rules are not explicit can be neither aggregated nor re-evaluated. Invest more in the evaluation corpus than in the training corpus, since the second is shrinking and the first is not. Those four decisions share the property of costing almost nothing when taken and being impossible to retrofit.Signals worth following
Rather than predictions, a few indicators allow the evolution to be tracked without depending on announcements. Zero-shot performance on difficult structures, disjoint or non-convex, is the best indicator of real progress, since that is where current models clearly fail. The quantity of data needed to close the gap through fine-tuning, which directly determines a project’s annotation volume. Robustness to domain shift measured between sites, since it is the specialised paradigm’s weak point and the foundation paradigm’s principal argument. And the availability of models covering structures and modalities currently poorly served, notably ultrasound and pathology, whose coverage lags well behind CT.A cautious outlook
It would be dishonest to close this series on medical image segmentation without acknowledging what this analysis does not permit. The figures cited describe a fast moving field, and an evaluation published today describes models designed a year ago. The gaps observed may close faster than their recent consistency suggests. Two scenarios remain open and are better named than left implicit. In the first, fine-tuning continues to require task-specific data, and the annotation need shifts without disappearing, as this article describes. In the second, models reach coverage such that fine-tuning becomes marginal on most standard structures, and the need then falls sharply, concentrating on evaluation, poorly covered modalities and definitional tasks. Nothing in the available data settles this, and an annotation provider asserting the first scenario with certainty would have an obvious interest in doing so. The defensible position is more modest: both scenarios leave a need standing, they do not leave it standing at the same level, and the four preparation decisions above have the advantage of being right in either case.What this means for a buyer today
Translating all of this into a medical image segmentation decision is worth doing explicitly, since the abstract debate rarely helps someone scoping a project this quarter. If the target structures are standard, well contrasted and on CT, start from a public multi-organ model and size the annotation effort for correction and for the structures it does not cover. The volume required will surprise a team that last scoped a project three years ago. If the target is a lesion type, a disjoint structure or a conventional boundary, size the project much as before. The evidence indicates no shortcut exists there yet, and assuming otherwise produces a corpus too small to reach the objective. If the modality is ultrasound or pathology, treat published gains with caution and measure on your own data before committing. The coverage gap is real and the figures circulating come predominantly from CT. And in every case, protect the evaluation corpus. It is the element whose requirements rise rather than fall, and it is the one most often sacrificed when a schedule tightens, because its value is not visible until the moment a model has to be trusted.Common errors of judgement
These misreadings recur often enough that naming them is usually enough to avoid them.- Reading a performance obtained with ground-truth-derived prompts as autonomous performance.
- Transferring a result obtained on compact organs to lesions or disjoint structures.
- Concluding from falling training volumes that the annotation need disappears.
- Neglecting robustness to domain shift, which often decides deployment.
- Shrinking the evaluation corpus alongside the training corpus.
- Expecting a model to settle a conventional question.
- Comparing paradigms without stating where the prompts came from.
- Annotating in a poor format on the assumption that future needs will match current ones.
- Fusing multiple annotations and discarding the individual ones.
- Relaxing quality control as assistance improves.
- Transferring conclusions drawn from CT to ultrasound or pathology.
- Using a public multi-organ model as a reference rather than a starting point.
