One challenge result says most of what matters about bone segmentation. On vertebral contouring, the best performing algorithm achieved a vertebrae identification rate of 95 per cent alongside a Dice coefficient of 90 per cent.
Two figures for two distinct tasks. The second measures drawing quality, the first measures the ability to say which vertebra it is. And the second problem is the one specific to bone: on an organ, knowing you are looking at the liver poses no difficulty; on a spine, knowing you are looking at the ninth thoracic vertebra rather than the eighth is a decision in its own right, and it can be wrong while the contour is perfect.
This article covers that specificity and its practical consequences: identification, instance separation, anatomical variants, pathological cases and what clinical use imposes. It extends the complete guide to medical image segmentation.
Two medical image segmentation tasks, not one
Bone medical image segmentation combines two operations often assumed to be one, and which fail independently. Delineation assigns each voxel to bone or to something else. On CT this task is eased by high contrast, bone appearing markedly denser than surrounding tissue. It is the simpler part of the work, which is why the field posts high overlap scores. Identification assigns each bone its anatomical name. It depends not on local contrast but on context, on counting and sometimes on knowledge of variants. It is independent of the first: a perfectly drawn but wrongly labelled mask is unusable, whereas an approximate but correctly labelled mask remains serviceable for many uses. That distinction structures everything else. A bone segmentation protocol must treat the two separately, with their own conventions, controls and metrics, otherwise the overlap score will mask identification errors that are the most expensive kind.The counting problem
This is the field’s central difficulty, and it is well documented by the teams who built the large corpora.Why counting is necessary
The authors of a multi-structure segmentation model explain it plainly: neighbouring ribs or vertebrae are sometimes labelled incorrectly, since the total number of ribs and vertebrae can vary, so physicians typically resort to counting from top to bottom to identify individual ribs or vertebrae; however, on some images only a subset is visible, making it difficult for physicians as well as for the model to identify the correct numbers. The mechanism is clear. A thoracic vertebra carries no distinctive marking allowing it to be recognised in isolation: it is identified by its position in a series, therefore by counting from a reliable landmark. If that landmark is outside the field, identification becomes an inference.The truncated field
Most studies do not cover the whole spine. An abdominal CT shows part of the thoracic and lumbar spine without the cervical landmark and not always the sacrum, depriving the annotator of a starting point. The consequence for a protocol is that it must anticipate this case explicitly rather than leave it to judgement. Three options exist: identify from an alternative landmark where anatomy permits, rely on clinical information available outside the image, or abandon absolute identification and assign only relative positions. That third option is often the most honest, and it is acceptable where the use does not require naming.Anatomical variants
The second failure factor is biological. Reference corpora have taken it seriously: the second edition of a vertebral challenge comprises 300 subjects with 4,142 annotated vertebrae, enriched with cases exhibiting anatomical variants such as enumeration abnormalities, numbering 77, and transitional vertebrae, numbering 161. These variants are not curiosities. A supernumerary vertebra, an atypical lumbosacral junction or a cervical rib shifts the entire count, and an annotator mechanically applying standard numbering will produce a whole series of wrong labels from the point of divergence. The same source notes that the challenge’s first edition showed algorithms perform well on normal anatomy and fail on variants not frequently present in the training data. That finding applies to human annotators too, and it justifies a case library containing variant examples specifically rather than exemplary spines.The three characteristic errors
A concordance study between anatomical segmentation models identifies the failure modes on the spine precisely: inclusion of parts of adjacent vertebrae, merging of multiple vertebrae under a single label, and incorrect assignment of the vertebra label. Those three cover exactly what annotators do wrong, and they call for different corrections. Inclusion of neighbouring parts belongs to the interface separation convention. Merging belongs to instance detection. Incorrect assignment belongs to counting, and it is the most serious because it propagates: one label error mechanically shifts every subsequent one. The same study observes an instructive disparity by level, mid-thoracic vertebrae showing the weakest agreement between models, with overlaps approaching zero in places. That region is precisely where landmarks are furthest away and counting most fragile, confirming that the difficulty is identification rather than drawing.Instance separation
Bone poses a medical image segmentation separation problem organs do not: the pieces touch. Two articular surfaces separated by thin cartilage often appear joined on imaging, and any density-based method fuses them. The same applies to two vertebrae at the facet joints, or to a rib and the vertebra bearing it. The practical consequence is that the protocol must set an explicit separation convention, and that convention determines the difficulty of the work far more than contrast quality does. Three questions recur: where the limit falls between two touching pieces, how the joint space itself is handled, and what to do when the two pieces are genuinely fused by a pathological bony bridge. That last question needs a written rule. A vertebral fusion, spontaneous or surgical, makes separation anatomically arbitrary, and two annotators will produce different limits with neither being wrong. The convention adopted, separating along the presumed limit or treating the whole as a single entity labelled as such, must follow from the use rather than from individual intuition.Contrast and its traps in bone medical image segmentation
The apparent ease of bone delineation leads to errors worth naming. Density thresholding works well enough that people stop checking it. A single threshold overestimates dense cortical bone and loses trabecular or demineralised bone, producing appreciably different results between a young adult and an osteoporotic patient. The protocol must therefore state whether the threshold is adapted per case, and document the value used. Some regions have cortical bone thinner than the study’s effective resolution: orbital wall, skull base, facial skeleton. Thresholding produces perforations there with no anatomical reality, and the protocol must state what to do rather than leaving each annotator to fill them their own way. Finally, slice thickness strongly conditions the result. A thick-slice reconstruction produces a partial volume effect that, on thin structures, simply erases the cortex. On those studies, demanding precise delineation amounts to requesting a reconstruction.Pathological cases
Three situations fall outside the standard frame and must be addressed explicitly in the protocol. Fracture raises the instance question: do two fragments of one bone constitute one piece or two? The answer depends on use, surgical planning generally requiring fragments to be distinguished where morphological analysis requires the opposite. The protocol must also state whether the fracture line itself is annotated as a distinct entity. Implanted hardware, prosthesis or osteosynthesis, generates artefacts that saturate density values within a sometimes substantial radius and render thresholding inoperative. On revision surgery studies this case is the rule rather than the exception, and its cost must be budgeted separately. The protocol must say whether hardware is segmented, ignored or treated as its own class. Demineralisation, finally, reduces contrast and blurs the bone boundary where it is usually sharp. In an elderly population this case is frequent and it shifts the task from the geometric register towards the perceptual one, which changes the competence profile required.What multi-structure models contribute
One economic feature clearly separates bone from other areas of medical image segmentation: public models exist covering most of the skeleton, which changes a project’s starting point. A bone segmentation project can therefore start with complete free pre-annotation on standard pieces, and concentrate human effort on correction rather than initial drawing. The gain is considerable and has no equivalent in lesion segmentation. Two reservations accompany that, and they follow from everything above. The first is that these models inherit the identification difficulties described earlier: the errors they make bear precisely on labelling, and an annotator accepting without checking the count propagates a systematic error. The second is that their own training corpora contain known imperfections, several studies noting that some models exhibit problematic vertebral segmentations inherited from their source data. The practical consequence is that pre-annotation should be used for drawing and systematically verified for identification. That asymmetry of trust is specific to bone and must appear explicitly in the instructions, otherwise the team will grant the model a uniform confidence it does not merit.What clinical use imposes
As elsewhere in medical image segmentation, the mask’s destination sets the requirement level, and bone has a particularity: its uses are very heterogeneous. Surgical planning and guide manufacture demand surface precision, since the object produced comes into contact with the bone. It is the most demanding use, and it adds geometric validity constraints covered elsewhere. Morphometry and population analysis rest on dimensional measurements and mainly require reproducibility between studies, absolute precision mattering less than consistency of conventions. Bone density analysis does not require fine delineation but reliable positioning of measurement regions, which shifts the requirement towards identification rather than drawing. Preoperative localisation and navigation, finally, require above all that identification be correct, an error of vertebral level having direct consequences no contour quality compensates for. That diversity explains why a bone corpus cannot be designed without its use having been settled, and why one dataset serves two opposing uses poorly.The bone medical image segmentation protocol
Five points must be settled in writing, and their absence explains most disagreements observed in production. The list of pieces to segment and their exact nomenclature, specifying the anatomical reference used, since several conventions coexist across specialties. The identification rule for truncated fields, with what to do when the counting landmark is not visible, and whether relative labelling is permitted. The separation convention at articular interfaces, illustrated with examples rather than stated abstractly. The handling of anatomical variants, with the numbering rule to apply in case of an abnormality, a question a clinician must settle rather than the annotation team. The handling of pathological cases, fractures, fusions, implanted hardware and demineralisation, each calling for a specific rule.Quality control of a bone medical image segmentation
The skeleton’s regular structure permits automatic checks other domains cannot support, and that is an advantage to exploit. Four checks are easy to program. Numbering consistency, meaning no gap or duplicate in the label series. Order consistency, each piece having to sit between those flanking it along the expected axis. Symmetry, where volume difference between left and right pieces is a simple and effective indicator, an approach used by work on automatic detection of segmentation anomalies. And connectivity, a bone fragmented into isolated components generally signalling an error. These checks do not replace human review but they direct sampling effectively, flagging the studies to examine first. On a large corpus they are the main efficiency lever in the quality arrangement. To these is added targeted review of the risk zones identified above: levels far from landmarks, junctions, variant cases. Uniform control over-represents easy regions and says nothing about the problematic ones.From segmentation to a manufactured object
Some bone projects do not stop at the mask: the result feeds a manufacturing chain, cutting guide, planning model or custom implant. That destination adds three requirements segmentation alone does not carry. The surface becomes the object, whereas volumetric overlap mainly measures the structure’s core, which makes the usual metrics uninformative. Geometric validity of the mesh produced after conversion becomes a binary condition, an impeccable mask potentially yielding a surface the manufacturing chain rejects. And physical scale must be preserved without approximation, a slightly resized model being not approximate but wrong. For an annotation project the practical consequence fits in one sentence: where the destination is a manufactured object, control must include superimposing the converted result on the source study, not merely inspecting the mask. The conventions specific to that chain, interface separation, thin cortex handling, smoothing parameters and unit preservation, belong to a scoping exercise distinct from segmentation itself.The annotator profile
The competence required in bone segmentation differs from other medical areas, and that difference is economically favourable. Drawing demands less judgement than on a lesion, contrast doing most of the work. What demands competence is identification, and it rests on precise but perfectly teachable anatomical knowledge: recognising a landmark, applying a counting rule, spotting a variant and knowing it must be escalated to the clinician. Two practical consequences follow. Training must focus mainly on anatomy and nomenclature rather than on the gesture, which is the opposite of lesion segmentation. And qualification before production benefits from an identification test on truncated-field studies and on variant cases, which is markedly more predictive than contour quality on a standard case.Cost and what governs it
The cost profile of a bone project differs from an organ segmentation on one important point. The number of pieces to individualise dominates, a full spine or rib cage representing dozens of instances per study where an abdomen contains a handful. Time does not grow proportionally, delineation being eased by contrast, but the identification verification load does grow with the number of pieces. The proportion of pathological cases comes next, and it is the most frequently underestimated factor. A corpus from a trauma or revision surgery department carries a density of fractures and implanted hardware that multiplies time per study without the image count changing. The useful estimation method therefore times normal and pathological studies separately, then weights by the corpus’s real composition. A costing built on standard studies systematically underestimates clinical corpora.Delivering identification alongside the mask
One delivery point specific to bone medical image segmentation deserves settling early, because it determines whether the corpus can be audited at all. A label volume encodes identification implicitly: the voxel value carries the name. That is compact and it loses everything about how the identification was reached. When a downstream disagreement arises, and on truncated fields it will, nothing in the file says whether the annotator counted from a visible landmark, inferred from context, or applied a relative convention. The remedy costs almost nothing at production time. Alongside the mask, record per study the landmark used for counting, whether the labelling is absolute or relative, and whether a variant was identified and how it was handled. Three fields, filled as the work proceeds, that turn an unauditable corpus into one where a disputed label can be traced to its reasoning. That record has a second use. Aggregated across the corpus, it shows what proportion of studies were labelled by inference rather than by direct counting, which is exactly the information needed to weight confidence in the identification metric and to decide where additional clinical review is worthwhile.The most common mistakes
These failures recur often enough across bone projects that naming them is usually enough to avoid them.- Treating delineation and identification as a single task.
- Reporting an overlap score with no identification indicator.
- Providing no rule for identification in a truncated field.
- Building a case library with no anatomical variant examples.
- Letting the annotation team settle numbering in case of an abnormality.
- Omitting the separation convention at articular interfaces.
- Applying a single density threshold whatever the patient’s bone condition.
- Demanding fine delineation on thick-slice reconstructions.
- Not exploiting the automatic checks the skeleton’s regularity permits.
- Costing a trauma corpus on normal studies.
