Medical imaging shifts the question of quality. What limits a corpus there is not the precision of the tracing but the disagreement between experts about what should be traced, and that disagreement is the ceiling method will not cross.
This article sets out what this context demands. It extends the article on Segment Anything and assisted segmentation.
What distinguishes this context
Five properties separate this image segmentation vertical from the others. The boundary carries a consequence, an area traced being able to guide a clinical decision. The definition of the structures belongs to a knowledge the appearance does not carry. Transitions are frequently progressive, a lesion merging into the neighbouring tissue. Variability between experts is measured and documented in the field’s literature. And the regulatory frame imposes a traceability the other image segmentation verticals do not require. One important practical consequence follows. The fourth property separates this domain from all others, the limit of what an image segmentation corpus can reach being known there before anyone starts.The ceiling of expert disagreement
This notion governs the whole vertical. It deserves stating at the outset. Two qualified experts delimiting the same structure obtain different results. Three consequences follow in image segmentation. No corpus can be more consistent than that disagreement permits. A quality requirement above that value is untenable by construction. And a system evaluated against a single reference is judged against one opinion. One important observation follows for an image segmentation project. The third consequence justifies double reading on the evaluation corpus, agreement between two experts constituting a firmer reference than the tracing of whichever of them was available.The structures and their difficulty
Four categories differ in variability in medical image segmentation. Organs with a sharp boundary, where disagreement stays low and the task is reproducible. Organs with a partial boundary, a portion of which is poorly distinguished from the neighbouring tissue. Lesions, whose limit depends on the threshold the expert applies. And functional structures, whose definition coincides with no boundary that is visible in the image. One practical consequence follows. Those four categories call for distinct requirements, a single value applied across a corpus being either too loose on sharp organs or untenable on lesions.Measuring expert disagreement
One operation precedes any contractual requirement and it is conducted upstream. Four stages compose that measurement in medical image segmentation. Selecting a sample representing the real difficulties, sharp structures and edge cases alike. Having the same examinations delimited by two experts working independently. Measuring the gap per structure and per boundary type rather than globally. And confronting the cases of strong disagreement to separate a divergence of rule from a divergence of judgement. One important practical consequence follows. The fourth stage is the most productive, a disagreement arising from a different implicit rule being correctable by a written convention while a disagreement of judgement constitutes the irreducible ceiling.What the acquisition protocol changes
Four parameters modify what is observable in medical image segmentation. The modality, each technique making different tissues visible. The settings, which change the contrast between neighbouring structures. Slice thickness, which determines the fineness of the structures represented. And the use of a contrast agent, which reveals certain structures while masking others. One important observation follows for an image segmentation project. Those four parameters must be recorded as metadata, a corpus mixing different protocols presenting a contour variability nothing explains and that the model will learn as noise.Who should annotate
Three configurations occur in medical image segmentation. Their cost differs strongly. Fully expert annotation, the most expensive and the most limited in volume. Annotation by trained operators with expert review, which distributes the effort by difficulty. And annotation by trained operators alone, practicable on the sharp structures only. One important practical consequence follows. The second configuration is the most used and the most effective, provided the split between what is delegated and what escalates is decided structure by structure rather than globally.What the training must cover
Five competences distinguish an operator trained on this vertical. Recognising the target structure in its normal anatomical variants. Applying the placement rule adopted on progressive boundaries. Distinguishing a normal variant from an anomaly, which calls for escalation. Reading the modality used correctly, each technique having its display conventions. And flagging rather than deciding personally on the cases the convention does not cover. One observation follows for an image segmentation project. The third competence is the most decisive, an operator who conscientiously delimits an abnormal structure without flagging it producing a correct annotation and a lost piece of information.The conventions specific to this vertical
Six decisions are added to those of an ordinary image segmentation project. The placement rule on progressive boundaries, decided with a domain expert. The treatment of structures partly outside the field. What to do with acquisition artefacts, excluded or flagged. The treatment of anatomical variants, included in the class or distinguished. The size threshold below which a structure is not delimited. And the precise definition of each structure, taken from an existing clinical reference rather than written for the project. One practical consequence follows. The last decision is the most important and the least practised, a definition written for the project inevitably diverging from the one clinicians use.What quality control requires
Five verifications apply specifically to this vertical. Measuring disagreement between experts on a sample, a precondition for any requirement. Systematic double reading on the evaluation corpus. Comparing mean volumes per structure, revealing a placement bias. Verifying anatomical consistency, a structure being unable to occupy certain positions. And traceability of each annotation, with operator, reviewer and date. One important observation follows. The last verification is imposed by the regulatory frame and it also serves diagnosis, a systematic defect attaching to an operator or a period rather than to the whole corpus.The volumetric acquisitions
A frequent property of this vertical modifies how the work is organised. An examination produces a stack of slices rather than an isolated image. Four consequences follow in image segmentation. The work volume is counted in slices and not in examinations, a ratio clients underestimate. Consistency between neighbouring slices becomes a requirement in its own right. Propagation from one slice to the next sharply reduces the load on regular structures. And an error repeats across a whole series whenever it comes from an unverified propagation. One important practical consequence follows. The last consequence calls for a check specific to this context, examining the variation in area along the stack revealing in one graph a propagation that has drifted.What assistance brings here
Three observations situate automatic help in medical image segmentation. It works correctly on organs with sharp boundaries, precisely those the task already handled well. It fails on lesions and progressive boundaries, where the limit belongs to an expert threshold. And it produces sharp contours on soft transitions, which masks the real underlying uncertainty. One important practical consequence follows for an image segmentation project. The third finding is the most problematic in this context, a sharp contour suggesting a certainty the data does not carry, which can guide a clinical decision on an arbitrary boundary.What the uncertainty should carry
One practice stays rare and it would change the value of these corpora. A contour can be accompanied by an indication of confidence rather than delivered alone. Three forms exist for carrying it in image segmentation. An attribute per region, flagging that the boundary belongs to a judgement. A band of uncertainty, the zone where the boundary could legitimately sit. And keeping several readings rather than merging them into one single reference. One important observation follows. The third form is the most informative and the most expensive, it permits evaluating a system against the range of expert opinions rather than against one of them, which corresponds better to the reality of the field.What confidentiality imposes
Four constraints distinguish data handling in medical image segmentation. Pseudonymisation before any transfer, patient identifiers not being allowed to accompany the images. Control of the processing location, often restricted by the applicable frame or by contract. Access logging, each consultation having to be attributable to a person. And removal of residual information, some images carrying text burned directly into them. One observation follows for an image segmentation project. The last constraint is the most often forgotten, identifying information burned into the image itself surviving any pseudonymisation that bears on the metadata.What the regulation imposes
Four requirements weigh on image segmentation corpora intended for a medical device. Full traceability of production, from the source file to the validated annotation. Documented qualification of the people involved. Description of the annotation protocol and of its evolutions. And retention of the data for a duration that is defined in advance. One observation follows. Those four requirements largely cover the good practices set out in the preceding chapters, which makes compliance a by-product of properly conducted work rather than a separate burden.What this work costs
Four factors determine the image segmentation load on this vertical. The type of structure, from the sharp contour to the diffuse lesion. The level of expert intervention required, the principal budget line. The number of slices per examination, a volumetric acquisition considerably multiplying the work. And the double reading rate adopted, which adds entirely on top of production. One practical consequence follows. The second factor dominates all the others, which makes the split between delegated work and expert review the principal economic decision of a medical image segmentation project.Organising the expert review
Five principles make this organisation effective in medical image segmentation. Reserving the expert for the structures whose measured disagreement is high, a measurement that precedes the split. Escalating the flagged cases rather than submitting the whole corpus to review. Grouping reviews by structure, which avoids constant context switching. Giving the expert the comparison with the initial annotation rather than the image alone. And recording each correction with its reason, which feeds the convention used for the next batch. One important practical consequence follows. The last principle turns review into a source of improvement, a recurring reason revealing a missing rule the next tracing will apply with no further intervention.The pilot batch on this vertical
Four examinations compose a pilot batch suited to this image segmentation vertical. An examination showing the structure in its commonest form, which supplies the reference throughput. An examination where the boundary is partly indistinct, which tests the placement rule. An examination containing an anatomical variant, which tests the flagging instruction. And an examination from a different acquisition protocol, which measures the effect of the resulting contrast on throughput. One observation follows. The fourth examination is the one most often omitted and the one revealing the largest variability factor, two protocols producing appreciably different throughputs and disagreements on the same structure.What the corpus must declare
Six pieces of information accompany a medical image segmentation delivery. The definition used for each structure, with its clinical source. The placement rule adopted on progressive boundaries. The disagreement between experts measured on the target structures. The acquisition protocols present in the corpus, with their distribution. The review arrangement applied, structure by structure. And full traceability, with operator, reviewer and date recorded for each annotation. One practical consequence follows. The third is the one most often missing and the one making the corpus interpretable, an announced performance having meaning only against the variability the experts themselves present.What a provider must demonstrate
Four elements establish a provider’s credibility on this image segmentation vertical. A traceability arrangement operational before the first examination, not added afterwards. A documented training procedure for operators, with assessment of their qualification. A disagreement measurement conducted on the target structures, which shows the limit is known. And a confidentiality arrangement covering the processing location, the accesses and any burned-in text. One observation follows for a medical image segmentation provider. The third element is the most distinctive, it shows a provider has understood this vertical is judged on reproducibility rather than on the fineness of the tracing, a distinction few articulate.Approaching a medical project
Five questions scope a medical image segmentation project. Has disagreement between experts been measured. Its value bounds any requirement. Which structure definition applies. It must come from a clinical reference. What share of the work is delegated to trained operators. That answer determines the budget. Are the acquisition protocols homogeneous. A negative answer calls for a metadata field. And is traceability organised from the very start. It never reconstitutes afterwards. Those five answers determine feasibility and cost. Asking them before starting avoids a corpus whose variability exceeds what the clinical use tolerates.The question that frames the requirement
One question determines the level of reproducibility to aim for. Will the delivered quantity be read by a clinician or consumed by a calculation. A reading-oriented answer tolerates a variability comparable to that of the experts, the result being interpreted by someone who knows that variability. A calculation-oriented answer, volumetric follow-up or a decision threshold, requires higher reproducibility, a gap between two successive measurements having to come from the patient and not from the tracing. That question is asked in one sentence, it belongs to the clinical use and not to the technique, and it determines whether the corpus must reach the expert ceiling or merely approach it.Three decisions before the first examination
Three decisions commit a medical image segmentation project. Measuring disagreement between experts on the target structures, a value that bounds the contractual requirement. Obtaining the definition of the structures from a clinical reference rather than writing it. And organising traceability before production, since it does not reconstitute afterwards. Those three decisions cost a few days and access to an expert, they precede the first examination, and their absence produces a careful corpus whose reproducibility nobody can establish.What this vertical brings to the others
Three practices born here hold beyond medical imaging. Measuring disagreement between annotators before fixing a requirement, applicable anywhere the task involves a judgement. Keeping several readings rather than merging them, transferable as soon as a single reference masks a real variability. And taking a definition established by the domain rather than writing one, necessary whenever a trade has already named what is being annotated. One observation follows. Those three practices answer one same situation, that where annotation transcribes an expert judgement rather than an observation, a configuration this vertical makes visible and that other domains meet without naming it.What this chapter leaves to the next
One vertical presents the opposite difficulty and deserves examining next. Satellite imaging shares the question of blurred boundaries and differs on everything else. Three differences characterise it. Objects there are seen from above, which removes occlusion and replaces it with other difficulties. Continuous materials dominate, where medical annotates identified structures. And scale varies considerably, a single corpus mixing resolutions with no relation to each other. Those three differences make this vertical a useful counterpoint, it tests the same conventions in a context where expert judgement gives way to the variability of the terrain.What this chapter teaches
One cross-cutting observation deserves closing this examination. This vertical is the only one where the quality limit is known before starting. Three findings compose it. Disagreement between experts constitutes a documented ceiling, which makes any higher requirement untenable by construction. The definition of the structures comes from a clinical knowledge external to the project, which forbids writing it. And assistance produces sharp contours on soft transitions, which masks an uncertainty the context makes important. That finding extends the preceding chapter, the gap between contrast and convention reaching here its most marked form.Why the ceiling is good news for a provider
One observation about how this vertical is negotiated belongs at the close. A documented ceiling protects the provider more than it constrains them. Three reasons produce that in image segmentation. A requirement above the expert ceiling cannot be met by anyone, so accepting one is accepting a failure scheduled in advance. The ceiling comes from the field’s own literature, not from the provider, which makes citing it a statement of fact rather than a plea. And a client who understands the ceiling stops comparing providers on a promised figure and starts comparing them on reproducibility, which favours whoever has measured. One practical consequence follows for a medical image segmentation provider. Raising the ceiling before quoting converts an argument about numbers into a discussion about method, and it is one of the few conversations in this field where the honest position is also the commercially stronger one.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Fixing a requirement without having measured disagreement between experts.
- Writing a structure definition instead of taking a clinical reference.
- Applying a single requirement to sharp organs and to lesions.
- Assigning all the work to experts with no split.
- Neglecting to record the acquisition parameters.
- Mixing different protocols with no metadata field.
- Evaluating a system against a single-reader reference.
- Accepting a sharp assisted contour on a progressive transition.
- Organising traceability after production has begun.
- Training operators to trace without training them to flag.
