The precision of a mask does not depend on the patience of whoever traces it. It depends on the method used, the tool available and above all on whether the contour sought exists in the image or belongs to a decision.
This article sets out the techniques. It extends the article on panoptic segmentation.
The four tracing methods
Four approaches produce an image segmentation mask. Their return differs strongly. Polygon tracing, a sequence of vertices the annotator places along the intended contour. The brush, which paints the region directly at pixel resolution. Selection by similarity, which extends a zone on a colour or texture criterion. And assisted designation, where a point or a rough box produces a complete region. One important practical consequence follows. Those four methods do not replace one another, most image segmentation corpora being produced by a combination where each handles what it handles best.What the polygon permits
Four properties characterise this image segmentation method. The result is easily modified, a vertex moving without the tracing being redone. Vertex density adapts to the local complexity of the contour. Storage stays compact, a few dozen values sufficing to describe a whole region. And precision remains bounded by the distance between vertices, which smooths fine irregularities. One observation follows for an image segmentation project. The last property is a defect on fragmented contours and an advantage on regular ones, which makes this method suited to manufactured objects and unsuited to vegetation.What the brush permits
Four properties distinguish it from the polygon in image segmentation. Precision reaches the pixel, no interpolation smoothing the result. Non-convex regions and holes are handled naturally. Correction is local, a zone being retouched without affecting the rest. And storage becomes heavy, every pixel of the image having to carry a value. One practical consequence follows. The second property is decisive on openwork structures, a mesh or foliage being paintable where a polygon describes them only at the price of an impracticable contour.What selection by similarity permits
Three observations situate this method in image segmentation. It handles homogeneous regions very fast where the contrast with the surroundings is sharp. It fails on progressive transitions, the similarity criterion having no natural limit. And it produces spillovers the annotator must rework, which sometimes cancels the gain entirely. One important observation follows. Those three findings make this method a conditional accelerator, worthwhile on contrasted scenes and expensive on soft ones, which requires testing it on the real data before adopting it.What assisted designation changes
Four effects accompany this image segmentation method. Throughput rises sharply on well-contrasted objects of medium size. The work shifts towards verification, production becoming validation. The boundaries produced follow the contrast rather than the project’s convention. And fine structures are frequently omitted, assistance smoothing everything it does not distinguish. One important practical consequence follows for an image segmentation project. The third effect is the most insidious, a contour perfectly fitted to the contrast being able to contradict the placement rule adopted with no imprecision visible at all.Choosing the method by contour type
Four situations each call for a dominant image segmentation method. A regular sharp contour calls for the polygon, fast and compact. A fragmented or openwork contour calls for the brush, the only practicable method. A large highly contrasted homogeneous region calls for selection by similarity. And a common well-detached object calls for assisted designation, subject to a verification. One observation follows. Those four situations coexist within one image, which makes choosing a single method per project less effective than a combination decided class by class in image segmentation.Tracing at uncertain boundaries
One difficulty escapes every image segmentation method. A progressive transition has no position the tool’s precision would permit reaching. Three placement rules coexist in image segmentation. The boundary at the start of the transition, which produces minimal areas. The boundary in the middle, the most widespread compromise. And the boundary at the end of the transition, which produces maximal areas. One important practical consequence follows. Those three rules produce substantial area differences on blurred objects, which makes them decisive for any measurement use and requires writing them rather than leaving them to judgement.The tolerance of contour
One notion permits expressing a precision requirement in a verifiable way. It designates the permitted gap between the traced contour and the reference contour, expressed in pixels. Three uses follow in image segmentation. It translates an intention of precision into a measurable criterion, which a qualitative instruction does not. It permits restricting the agreement computation to a band around the boundaries. And it bounds the time spent per region, an annotator finally knowing when to stop. One important observation follows. The third use is the most underestimated, an unquantified requirement leading each annotator to decide the level of finish alone, which produces a throughput variability greater than that of competence.What the tool must permit
Six tooling functions condition mask production in image segmentation. Fluid zoom and panning with no change of tool, gestures repeated thousands of times a day. Fine adjustment of brush width, down to a single pixel. Undo at fine granularity, an unfortunate gesture being able to affect a whole region. Temporary masking of already traced classes, indispensable for working beneath an existing layer. Display of the contour overlaid on the image, rather than as an opaque fill. And conversion between polygon and mask, which permits changing method in the middle of a tracing. One important practical consequence follows. The fifth function is the most decisive for precision, an opaque fill masking exactly the zone the annotator must examine to place their boundary.The characteristic tracing errors
Five defects recur in image segmentation corpora. The systematically inner contour, produced by the annotator’s caution. The stair-stepped contour, the signature of too wide a brush on an oblique edge. Slivers, regions of a few pixels left between two neighbouring zones. Unintentional holes, background pixels forgotten inside an object. And propagation spillovers, a region having invaded its neighbour without the annotator noticing. One important observation follows. The first defect is the most frequent and the only systematic one, which makes it detectable by comparing mean areas between annotators rather than by visual examination.Fine structures
This category of objects resists every method. Cables, stems, hair and mesh have a thickness close to the resolution. Four difficulties follow in image segmentation. The object partly occupies the pixels, none belonging entirely to it. Brush tracing produces an exaggerated thickness, the tool’s minimum width exceeding that of the real object. The polygon is impracticable, the contour holding as many vertices as pixels. And automatic assistance omits them, no sharp contrast delimiting them. One practical consequence follows for an image segmentation project. Those four difficulties justify an explicit convention for these classes, the inclusion or exclusion of fine structures having to be decided rather than endured.The order of tracing within an image
An organisation of the gesture affects throughput and quality in image segmentation. Four principles govern that order of tracing. Handling the large homogeneous regions first, which reduces the remaining surface and clarifies the image. Tracing the boundary between two classes once only, the second region inheriting the first’s contour. Reserving fine structures and doubtful zones for the end, when the context is established. And checking complete coverage last, rather than after each region traced. One important practical consequence follows. The second principle avoids the commonest defect of exhaustive corpora, two independent annotations of the same boundary systematically leaving a sliver or an overlap between them.What throughput depends on
Four factors determine the time per region in image segmentation. Contour length, the dominant factor the area does not predict. Contour sharpness, a blurred transition requiring a decision at every point. The method used, whose fit to the contour type weighs more than competence. And tool fluidity, a jerky display making any precise tracing impossible. One observation follows. The third factor produces the largest gap between teams, an annotator trained in method choice working several times faster than one who uses the brush everywhere.The configurations that slow the work
Five situations make throughput fall in an image segmentation project. Partly transparent objects, where the membership of the overlap pixels is undecidable. Cast shadows, which belong to the ground or to the object according to a convention. Reflections, which reproduce an object where it is not. Overexposed or underexposed zones, where the contour disappears into uniformity. And objects partly outside the frame, whose traced limit is the image’s and not the object’s. One practical consequence follows. Those five situations are handled by convention rather than by effort, and their frequency in the real data is measured on the pilot batch, which permits quantifying their effect before production.Training an annotator to trace
Six competences distinguish an annotator trained to trace in image segmentation. Choosing the method by contour rather than always using the same one. Working at a suitable zoom level, neither too wide nor excessively close. Applying the placement rule rather than following the apparent contrast. Checking for holes and slivers before validating a region. Flagging a case the convention does not cover rather than deciding alone. And recognising the configurations where assistance fails, notably fine structures and soft transitions. One observation follows. The second competence is the least obvious, an excessive zoom leading to pixel-by-pixel tracing whose precision exceeds what the convention requires, which consumes considerable time without improving the corpus.What control verifies on the tracing
Five verifications bear specifically on mask quality in image segmentation. The distribution of areas per class, whose shift signals a placement bias. The presence of regions of a few pixels, the signature of slivers. The presence of undeclared holes inside regions. Contour regularity, a stair-stepped profile signalling a badly set tool. And comparison between annotators, restricted to a narrow band around the boundaries. One important practical consequence follows. The last verification is the only one measuring what counts, agreement computed over the whole surface being dominated by region interiors where nobody makes mistakes.Measuring the precision actually reached
Four measurements qualify the actual tracing of an image segmentation corpus. The mean gap between two annotators along the boundaries, expressed in pixels. The proportion of boundaries where that gap exceeds the adopted tolerance. The mean area difference per class between annotators, revealing a placement bias. And the distribution of those gaps by contour type, whether sharp, blurred or openwork. One important observation follows. The last measurement is the most useful for steering, it shows that the precision reached is not a property of the corpus but varies strongly by contour, which permits targeting training and conventions where the gap is real.The polygon or the mask as deliverable
A format decision follows in part from the tracing method. Three considerations guide that choice in image segmentation. A corpus meant to be corrected gains from the polygon, whose vertices move without the tracing being redone. A corpus containing openwork structures requires the mask, no polygon describing foliage. And a corpus mixing both can deliver both, conversion from polygon to mask being immediate while the reverse stays approximate. One important practical consequence follows. The third consideration guides a decision of prudence, keeping the vector representation where it exists permitting masks to be regenerated at any resolution while a mask alone fixes the corpus’s fineness.The pilot batch for tracing
Four images test the methods adopted in image segmentation. An image with sharp regular contours, which supplies the reference throughput for the polygon. An image containing an openwork structure, which verifies that a brush is available and adjustable. An image with soft transitions, which tests the placement rule and the tolerance adopted. And an image containing fine structures, which tests the convention decided for those particular classes. One observation follows. The second image frequently reveals a tooling defect rather than a method defect, a platform with no fine brush making a whole class of the reference impracticable.Approaching mask production
Five questions scope mask production in image segmentation. What contour types does the corpus contain. That answer determines the methods to provide. What placement rule applies to blurred transitions. Its absence produces area variability. Are fine structures part of the scope. That answer calls for a dedicated convention. Are the annotators trained in method choice. That answer conditions throughput. And does control bear specifically on the boundaries. Its absence lets most tracing defects through. Those five answers determine the quality achievable and its cost. Asking them before starting avoids a corpus whose precision varies with the annotator and the class.The question that frames the requirement
One question determines the level of precision to ask for. Does the downstream process measure an area or recognise a shape. A recognition-oriented answer tolerates an approximate contour, a model learning the general shape without requiring pixel fineness. A measurement-oriented answer imposes a quantified tolerance and a strict placement rule, the contour gap translating directly into an area gap. That question is asked in one sentence, it belongs to the use and not to the technique, and it avoids this chapter’s commonest expense, pixel-level tracing produced for a model that will make nothing of it.Three decisions before tracing
Three decisions commit the precision of an image segmentation corpus. Quantifying the contour tolerance, which turns an intention into a verifiable criterion and bounds the time per region. Writing the placement rule for blurred transitions, before each annotator adopts a different one. And associating a tracing method with each contour type, which produces a throughput gain greater than any change of tool. Those three decisions cost one meeting and a written instruction, they precede the first image, and their absence produces a corpus whose precision varies with the annotator, the class and the time of day.What this chapter leaves to the next
A recent development modifies what this chapter has just set out. Assisted designation has advanced considerably and it changes how the work divides. Three questions follow for an image segmentation project. What becomes of the method choice when a designation produces most of the regions. What becomes of the placement rule when the contour is proposed rather than traced. And what becomes of control when the annotator validates instead of producing. Those three questions constitute the subject of the following chapter, which examines what assistance really changes and what it leaves intact.What this chapter teaches
One cross-cutting observation deserves closing this examination. The precision of a mask lies in the method and the convention rather than in care. Three findings compose it. Each contour type calls for a different method, and the choice weighs on throughput more than the annotator’s competence does. The placement rule determines the area delivered on every blurred contour, which makes it a measurement decision and not a matter of style. And the inner placement bias is the only systematic defect, hence the only one detectable statistically. That finding extends the preceding chapters, the boundary remaining where the corpus’s quality is decided.Why this chapter matters commercially
One observation about how providers are compared belongs at the close. Mask quality is the part of a segmentation project a client can judge without expertise. Three reasons produce that in image segmentation. A sliver, a hole or a stair-stepped edge is visible to anyone who zooms in, unlike a convention error that requires knowing the convention. Those defects are cheap to look for and they cluster, so a sample of three images tells a client most of what they will find in three hundred. And they signal method rather than effort, since none of them is what a careless annotator produces occasionally. One practical consequence follows for an image segmentation provider. Running the sliver and hole checks before delivery costs minutes and removes the defects a client will look for first, which matters more than it should because these are the defects that shape an impression of the whole corpus.The instruction sheet that fits on one page
One document carries most of what this chapter has set out. It states five things and it sits beside the annotator rather than in a shared folder. Which method to use for each contour type present in the corpus. Where the boundary goes in a blurred transition, with an example image. The contour tolerance in pixels, which tells the annotator when a region is finished. What happens to fine structures, shadows and reflections, class by class. And which cases to flag rather than decide alone. One practical consequence follows for an image segmentation project. This sheet costs an afternoon to write and it removes the decisions annotators would otherwise take silently and differently, which is the difference between a corpus with a convention and a corpus with several.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Using a single tracing method across a whole corpus.
- Tracing an openwork contour with a polygon rather than a brush.
- Leaving the placement rule to each annotator’s judgement.
- Accepting an assisted contour that follows the contrast rather than the convention.
- Not deciding explicitly what happens to fine structures.
- Neglecting training in method choice.
- Checking agreement over the whole surface rather than at the boundaries.
- Leaving slivers between neighbouring regions.
- Using too wide a brush on oblique edges.
- Confusing imprecision with systematic bias in the diagnosis.
