A thread-like defect breaks the usual metrics before it breaks the models. A crack two pixels wide and a thousand long has a perimeter-to-area ratio such that a one-pixel offset along its whole length halves its overlap score. The same offset on a circular stain a hundred pixels across is undetectable.
That asymmetry is not theoretical. A recent façade inspection dataset documents it: crack instances form thin, elongated structures with short sides sometimes below five pixels, whereas spalling regions may exceed 4,800 pixels along their longer dimension. Three orders of magnitude separate them, within one corpus and under one metric.
This article covers fine defects in defect detection: why they require segmentation rather than a box, which boundary conventions to write, how to measure without picking the wrong metric, and how to organise the annotation. It extends the complete guide to defect detection.
When segmentation becomes necessary in defect detection
The four annotation primitives used in defect detection are not interchangeable, and the field’s literature states plainly what each supports.
A synthesis on surface defect segmentation puts it this way: image classification provides sample-level decisions with high efficiency but no spatial localisation, object detection offers instance-level localisation with coarse boundaries suited to rapid triage of discrete or sparse defects, while semantic segmentation outputs pixel-accurate defect morphology, supporting dimensional quantification, area statistics and rework planning.
That hierarchy gives the primitive selection criterion directly. If the quality decision rule concerns a dimension, crack length, spalling area, scratch width, only segmentation supports the decision. If it concerns presence or count, a lighter primitive suffices.
The particular case of thread-like defects
On an elongated defect, the bounding box becomes frankly misleading. A thin diagonal crack produces a box whose overwhelming majority is sound background, so the box conveys neither real length, nor path, nor affected area. Two very different cracks can produce identical boxes.
That is why fine defects require segmentation regardless of the budget available. It is not a precision requirement but a representation requirement: the box simply does not encode the information the quality decision needs.
Instance or semantic
The distinction matters as soon as counting enters into the decision rule. Semantic segmentation says which pixels belong to the crack class without separating cracks from each other; instance segmentation separates them. A network of crazing, multiple porosity, several parallel scratches all raise the question of how many objects. If the decision rule mentions a number, instance segmentation is mandatory, and the protocol must then define explicitly what constitutes a distinct instance.
Why the usual metrics mislead
Mean intersection over union is the default metric of defect detection, and that is precisely the problem on fine defects.
The same synthesis notes that evaluation for industrial defect segmentation still largely borrows the mean intersection over union used in natural-image tasks, although industrial objects have neither the size nor the geometry of those in general-purpose corpora.
The distortion mechanism is purely geometric. Overlap compares an intersection against a union, and on a thin object the boundary accounts for an enormous share of total area. An annotator slightly more generous than another on delineation produces a considerable score gap, although both annotations describe the same crack and would lead to the same quality decision.
Two consequences follow directly. First, a low overlap score on a thread-like class does not necessarily mean poor annotation, and a high score on an extended class does not necessarily mean good annotation. Second, comparing classes of different geometry by overlap is meaningless: the measured gap reflects their geometry as much as their difficulty.
The metrics to add
Three complementary metrics correct that distortion. Boundary intersection over union measures agreement within a narrow band around the contour rather than across the whole area, which neutralises the size effect. Measurement error compares directly the quantity that enters the decision rule, length or area, which is the metric a quality engineer can interpret most directly. And the miss rate counts entirely missed defects, information no overlap metric surfaces, since an absent object contributes nothing to the calculation.
A study on wood defects illustrates the value of that triad. On fine cracks under two millimetres wide, it reports an initial miss rate of 24.67 per cent due to weak feature response, then after improvement a boundary intersection over union of 80 per cent and a measurement error of 3.12 per cent, meeting industrial grading requirements. Three distinct metrics, three different pieces of information, and it is their combination that supports a conclusion.
Boundary conventions in fine defect detection
On a fine defect, the delineation convention weighs more on corpus consistency than annotator care does. Four points must be settled in writing.
The fading contour
A crack does not stop sharply anywhere: it fades, branches, becomes invisible before disappearing. The protocol must say where to stop, for instance at the limit of visibility at the reference magnification, or at an explicit contrast threshold. Without that rule, each annotator sets their own, and the corpus carries variability that nothing signals.
The zone of influence
Some defects come with alteration of their surroundings: discolouration around a pit, heat-affected zone around a weld, peripheral spalling around a crack. Should that zone be included? Both answers are defensible, neither follows from common sense, and the right one depends on what the quality rule measures.
Interruption
Is a discontinuous crack one instance or several? The question is decisive as soon as counting or cumulative length enters the rule. The protocol must set an explicit merge distance, for instance two aligned segments separated by less than a given value form one instance, and illustrate it with examples rather than stating it abstractly.
Branching
Branched networks are where conventions matter most and where most protocols stay silent.
A branched crack network raises the opposite question: one instance with branches, or several connected instances. The convention chosen radically changes the count, and therefore the conformity measurement.
The annotator profile
Fine segmentation calls on a particular competence, distinct from the one defect recognition requires.
Recognising that a crack is present belongs to the perceptual register and is acquired quickly. Delineating it reproducibly is a manual skill, comparable to a draughtsman’s: steadiness of hand, consistency of zoom level, sweeping discipline. That skill improves with practice and is measured objectively by an annotator’s agreement with themselves.
Two practical consequences follow. Training must include manual practice with immediate feedback on reference cases, not only transmission of criteria. And team stability matters more here than on classification tasks, since the skill takes weeks to acquire and decays if the person moves to another task.
Ergonomics as the dominant cost factor
Fine defect segmentation is the most time-expensive annotation task in all of defect detection, and a substantial share of that cost is attributable to the tool.
The reason is straightforward. Drawing a long narrow contour at high magnification requires many precise actions, and every friction compounds. Three features make a measurable difference. A drawing tool suited to elongated structures, allowing an axis to be followed and a width applied, rather than point-by-point polygon tracing. Persistence of navigation state between images, avoiding rebuilding the viewpoint. And contour assistance, local thresholding or propagation, whose output the annotator corrects rather than drawing from scratch.
One practice deserves flagging: annotating by centreline rather than by contour. On a thread-like defect of relatively constant width, tracing the central line and specifying a width produces a usable mask in a fraction of the time a full contour takes. The method does not suit defects of varying width, but it transforms project economics where it applies.
Resolution as a prior constraint
No annotation convention in defect detection compensates for insufficient resolution, and that constraint is fixed before collection.
The practical rule is that a defect must span several pixels in its smallest dimension to be delineable reproducibly. Below that, two competent annotators produce different masks because the signal itself is ambiguous, and no training corrects it.
That constraint collides with an industrial reality: raising resolution shrinks the field, therefore multiplying acquisitions and cycle time. The trade-off is made at scoping, starting from the smallest dimension to be detected under the quality rule and working back to the optics. A project that discovers this constraint after collection has to start again.
Assisted pre-annotation on fine defects
General-purpose segmentation models, trained to isolate compact objects in natural images, perform poorly on thread-like structures. That is an important difference from other defect detection tasks, where assistance brings an immediate gain.
Three mechanisms explain the weakness. The progressive internal resolution reduction in standard architectures erases structures one or two pixels wide, a phenomenon the wood study cited above attributes explicitly to weak feature response. Overlap-based supervision penalises missing a thin object only lightly, so the model learns that missing it is cheap. And topology is unconstrained: a model can produce an acceptable overlap score while fragmenting a continuous crack, which destroys the length measurement.
Three forms of assistance nonetheless remain genuinely useful. Axis suggestion, where the model proposes a centreline the annotator adjusts. Guided local thresholding, where the annotator points at the defect and the tool propagates by contrast. And pre-triage, where the model flags images likely to contain a fine defect without delineating it, which saves traversal without risking anchoring on a wrong mask.
Tiling and truncated objects
High-resolution industrial images are frequently cut into tiles for training, and that operation raises a problem specific to elongated defects.
The problem is easy to miss because it is created by the pipeline rather than by the data. A crack crossing several tiles is cut into fragments, each becoming a distinct instance from the model’s point of view. Three consequences follow. Counting is wrong if reconstruction is not planned. Measured length is underestimated on every fragment. And an isolated fragment can become uninterpretable, a piece of crack without context resembling a scratch.
The remedies are well known and belong in the design: sufficient overlap between tiles so every fragment retains context, and an instance merging procedure at reconstruction based on geometric continuity at the edges. That procedure must be tested before production, like any conversion chain.
What the corpus must contain
The composition of a fine defect detection corpus follows two requirements projects underestimate.
The first concerns geometric coverage of the defect population. Thread-like defects vary in orientation, curvature, width and contrast, and a corpus covering only part of that range produces a model that fails on the rest. A horizontal crack and a diagonal one do not look alike to a convolutional network, and coverage must be verified explicitly rather than assumed.
The second concerns hard negatives. On a textured material many elements resemble a fine crack without being one: joint, tool mark, hair, fibre, handling scratch. A corpus containing only real cracks and ordinary background produces a model that flags anything elongated. Deliberately annotating these confusions as documented negative cases is among the highest-return investments in a fine defect detection project.
Quality control of fine defects in defect detection
The control arrangement follows the field’s general logic, with two inflections imposed by geometry.
The first is mandatory stratification by geometry rather than by class alone. Pooling thread-like and extended defects into one measurement produces a score describing neither. Metrics are reported separately per shape family.
The second is specific measurement of directional bias. A right-handed annotator tends to trace more precisely in one direction than the other, and the effect is documented: on fine defects it produces a systematic mask offset invisible in any aggregate metric. Detecting it means comparing, on one defect annotated by several people, the position of the contours rather than only their overlap.
One further practice helps considerably: asking for a few defects to be annotated twice by the same annotator weeks apart. Intra-annotator agreement on a fine defect is revealing, and an annotator disagreeing with themselves signals an insufficiently defined boundary convention.
What precision actually costs
An economic trade-off runs through every fine defect detection project, and it is better stated explicitly.
Annotation time rises steeply with the precision required, and that rise is not linear. Moving from an approximate contour to a careful one roughly doubles the time; aiming for pixel accuracy on branched structures multiplies it further. The marginal cost of the last points of precision is high.
The useful question is therefore not what precision to aim for but what precision the decision rule requires. If product grading depends on crack length to the millimetre, precision is justified. If the threshold is sharp and observed defects fall well below or well above it, approximate delineation suffices and the saving is considerable.
A mixed strategy is very often the right answer. Annotating a representative subset finely, enough for the model to learn the fine geometry, and the remaining volume more coarsely, produces an effective corpus at markedly lower cost. That approach requires documenting precisely which part of the corpus belongs to which level, otherwise the metrics become uninterpretable.
Delivering a usable mask
The output format deserves particular attention on this task, because fine defects tolerate conversion poorly.
Three representations are available and they behave very differently here. A raster mask preserves geometry faithfully but weighs heavily on high-resolution images. A polygonal representation is more compact and degrades sharply on thin structures, a contour two pixels wide becoming a polygon that simplification erases entirely. An axis-and-width representation is the most compact and the most faithful for regular thread-like defects, provided the machine learning team can consume it.
The prudent rule is to agree the output format with the receiving team before annotation and to test the conversion on extreme cases, the thinnest and most branched defect in the corpus, rather than on average ones. A simplification that preserves a fifty-pixel contour can erase a two-pixel crack.
Estimating effort on fine segmentation
Two variables dominate the cost of a fine defect detection corpus, and image count is neither of them.
The first is defect density per image. A material where cracks appear singly costs a fraction of one where each image carries a branched network, since time scales with total contour length rather than with instance count. Timing on a pilot drawn to reflect the real density distribution is the only credible basis for a quotation.
The second is the proportion of ambiguous boundary cases. A corpus of high-contrast defects on a clean background annotates quickly; one where most defects fade into texture spends its time on decisions rather than on drawing. That proportion is measurable during the pilot and is the figure that most often separates an accurate estimate from an overrun.
A third factor is worth watching although it is not a cost driver in itself: the share of images containing no defect at all. On fine defect tasks that share is usually high, and traversing them consumes real time while producing no annotation. Model-assisted pre-triage addresses exactly this, which is why it remains worthwhile even where assistance on delineation does not.
The most common mistakes
These failures recur often enough across fine segmentation projects that naming them is usually enough to avoid them.
- Choosing bounding boxes for thread-like defects, which do not encode the useful information.
- Comparing classes of different geometry by mean overlap alone.
- Omitting the miss rate, which no overlap metric surfaces.
- Setting no stopping rule for fading contours.
- Leaving the merge distance for interrupted segments implicit.
- Not settling the treatment of branched networks, which distorts counting.
- Fixing resolution after collection rather than from the smallest dimension to detect.
- Tiling without overlap or an instance merging procedure.
- Demanding pixel accuracy where the decision rule does not require it.
- Neglecting directional bias measurement, invisible in aggregate metrics.
Talking to the quality team about precision
One conversation is worth having early and is routinely skipped: asking the quality function how precisely they actually need the defect measured.
The instinctive answer is always as precisely as possible, and it costs a great deal. and it is almost never the operative one. Pressed further, quality teams usually reveal that the grading rule has thresholds with wide gaps, that measurement in the plant is itself approximate, and that a difference of a tenth of a millimetre changes no decision. That conversation frequently halves the required precision, and with it a large share of the annotation budget.
The productive framing is to ask what decision the measurement feeds, then work backwards. If a crack over ten millimetres downgrades the part and observed cracks cluster around two or around thirty, precision at the millimetre buys nothing. If the population clusters at nine and eleven, it buys everything. That distribution is knowable from the pilot batch, and it should drive the precision requirement rather than an abstract standard.
What to take away
Fine defects require segmentation not out of a quality requirement but out of representational necessity: no lighter primitive encodes the length, path and area the quality decision needs. That constraint is paid for in annotation time, and most of the control consists of paying only for what the rule requires.
Three decisions structure a successful project. Write the four boundary conventions, fading contour stopping rule, zone of influence, merging of interruptions and treatment of branching, with visual examples rather than statements. Measure with a triad, boundary overlap, measurement error and miss rate, rather than with mean overlap alone which distorts everything on thin structures. And fix resolution from the smallest dimension to be detected, before any collection, since no convention recovers an absent signal.
For approaches, ontology and annotation tasks, the complete guide to defect detection sets the frame. For an extension of the field towards tasks where geometric precision gives way to reading and dimensioning, the article on computer vision in logistics covers the constraints specific to parcels, pallets and automated sorting.
To explore delivery arrangements, supported formats and applicable control mechanisms, see our dedicated page on annotation for industry. And if you are preparing a fine segmentation corpus and want the conventions and precision level scoped before production starts, let us discuss your project.
