A cable, a stem or a distant object occupy a few pixels none of which belongs entirely to them. That physical property, and not a lack of care, explains why these categories concentrate disagreement and escape the usual measurements.
This article sets out those difficulties. It extends the article on the commonest segmentation mistakes.
Two distinct problems
Two categories are often conflated in image segmentation. They call for different treatments. The small object, compact but of low area, a distant sign or an insect. And the fine structure, elongated and of a thickness close to the resolution, a cable or a stem. One important practical consequence follows. The first belongs to a problem of scale and the second to a problem of representation, a fine structure staying impossible to delimit cleanly even at a large apparent size.What the resolution imposes
Four effects follow from a size close to that of the pixel in image segmentation. No pixel belongs entirely to the object, each mixing the object and its background. The contour does not exist as a sharp boundary but as a zone of mixture. A gap of a single pixel considerably changes the relative area. And two equally reasonable annotators will produce appreciably different results. One important observation follows. Those four effects mean the variability on these categories belongs to the data rather than to execution, which forbids handling it through a raised precision requirement.Why the metrics ignore them
Three mechanisms erase these categories from image segmentation measurements. An average weighted by area gives them negligible weight. An identical contour gap weighs more on a small region, which produces low values read as a failure. And a uniform control sample encounters them only very rarely. One practical consequence follows for an image segmentation project. Those three mechanisms justify reporting these categories separately, a global average saying nothing about how they were handled and a low value not necessarily meaning a defect.Why assistance fails
Four reasons explain that failure in image segmentation. No sharp contrast delimits an object whose pixels are mixed. Models smooth the details they do not distinguish. An openwork structure simply has its voids filled in. And designation by point or by box presupposes an object extended enough to be aimed at. One important observation follows. Those four reasons mean the assistance gain is nil or negative on these categories, which requires switching them to manual tracing rather than leaving them in an assisted chain.Why the tool cannot keep up
Three tooling limits appear on these categories in image segmentation. The brush’s minimum width frequently exceeds the object’s thickness. The polygon becomes impracticable, the contour holding as many vertices as pixels. And the zoom required loses the visual context that alone permits identifying the object. One practical consequence follows. The first limit produces a systematic defect, a fine structure traced with the brush appearing thicker than it is, which falsifies any area measurement on those classes.The conventions to write
Five decisions handle these categories in image segmentation. The size threshold below which the object is not individually delimited. The fate of fine structures, included in a class, excluded or grouped. The contour placement rule within the zone of mixture. The treatment of openwork structures, voids included or excluded. And what to do when an object is identifiable but not cleanly delimitable. One observation follows. The last decision is the most useful and the rarest, it permits flagging an object’s presence without claiming a delimitation the data does not carry.The treatment strategies
Four approaches occur in image segmentation. They do not exclude one another. The exclusion threshold, which discards objects below a given size. The generic class, which groups small objects without distinguishing them. Marking by point rather than by region, which flags without delimiting. And full tracing despite the difficulty, reserved for the few classes whose use requires it. One important practical consequence follows for an image segmentation project. The third approach is the most underused, it keeps the information of presence, which suffices for many uses, without paying the cost of an arbitrary delimitation.Marking by point
One solution deserves separate treatment since it solves a problem tracing does not. A point placed on the object replaces the region where delimitation has no meaning. Four benefits follow in image segmentation. The information of presence is kept, which suffices for detection and counting. Time per object becomes negligible compared with a trace. Variability between annotators falls sharply, a point being placed without hesitation. And control simplifies markedly, the comparison bearing on counts rather than on contours. One important practical consequence follows. This mode presupposes the format and the downstream process accept it, a condition verified at scoping and one that rules the solution out when it is considered too late.What the threshold must express
Three ways of expressing a threshold coexist in image segmentation. Their robustness differs strongly. The threshold in pixels, which changes meaning as soon as resolution varies. The threshold in world units, which stays valid across a heterogeneous corpus. And the threshold expressed as a proportion of the image, intermediate and rarely relevant. One observation follows. The second mode is the only defensible one on a corpus mixing resolutions or distances, which covers most real image segmentation projects.What the threshold must preserve
A precaution accompanies any exclusion threshold in image segmentation. Three pieces of information must survive the decision to discard. The value of the threshold itself, without which nobody knows what is missing. The distinction between a zone discarded by threshold and a zone left unannotated. And a count of everything that was discarded, which bounds the corpus’s real exhaustiveness. One important observation follows. The third is the rarest and the most useful, a system evaluated on a corpus that silently discards small objects appearing better than it is on the scenes that contain them.The openwork structures
A sub-category deserves isolating since it raises a question of definition. A mesh, foliage or a grille occupy a surface part of which is void. Three conventions coexist for an image segmentation project. The envelope, which includes the voids and describes the object as a solid zone. Faithful tracing, which excludes the voids at the price of considerable work. And the dedicated class, which declares the zone as openwork without deciding it pixel by pixel. One important practical consequence follows. Those three conventions produce radically different areas on the same objects, which makes them decisive for any measurement use and requires writing them rather than leaving them to the tracing.What control must look for
Five image segmentation verifications bear specifically on these categories. Counting occurrences per class, which reveals a class omitted across a whole batch. Comparing counts between annotators on the classes concerned. The distribution of thicknesses on fine structures, a shift signalling too wide a brush. The presence of regions just above the threshold, which verifies the threshold is being applied. And human review targeted on these specific classes rather than on a uniform random sample. One practical consequence follows. The second verification is the most revealing, a count gap between annotators signalling a systematic omission where a contour comparison would show only a tracing disagreement.What the downstream use really needs
Four needs occur in image segmentation, and they do not call for the same treatment. Counting, which is satisfied by marking with a point. Localisation, which is satisfied by a rough bounding box. Length measurement, which requires a trace but tolerates a conventional thickness. And area measurement, the only need that requires a faithful delimitation. One observation follows for an image segmentation project. The third need is frequent on fine structures, cables or stems, and it permits an important simplification, a thickness fixed by convention removing the difficulty without altering the quantity sought.What these categories cost
Four factors determine their load in image segmentation. Time per object, high relative to the area obtained. The number of occurrences, often large on urban or natural corpora. The absence of any assistance gain, which leaves these classes entirely manual. And control time, the targeted review adding entirely on top of the usual sampling. One important observation follows. Those four factors mean these categories consume a budget share unrelated to their share of surface, which justifies deciding their fate at scoping rather than discovering it in production.What ignoring these classes costs
Four consequences follow a decision to handle them by default in image segmentation. Variability between annotators installs itself with nobody having decided it. The budget grows on classes whose share of surface is negligible. The delivered corpus mixes objects traced and objects omitted depending on the annotator. And the trained system inherits a detection boundary that reflects fatigue rather than a rule. One important observation follows for an image segmentation project. The last consequence is the most damaging, a detection threshold learned from irregular omissions producing behaviour nothing in the reference explains.What the training must cover
Five competences distinguish an annotator trained on these categories in image segmentation. Recognising that an object falls under the exclusion threshold rather than tracing it approximately. Switching to manual tracing as soon as a class of this kind appears in an assisted chain. Setting the brush width to the real thickness rather than to the default value. Applying the convention on voids without reinterpreting it case by case. And flagging an object that is identifiable but not delimitable rather than producing an invented contour. One practical consequence follows. The last competence is the hardest to install, a conscientious annotator naturally preferring to deliver an approximate contour rather than a flag, while the second carries correct information and the first carries false information.What the corpus must declare
Five pieces of information accompany an image segmentation delivery holding these categories. The size threshold applied, with its unit of expression. The convention adopted on openwork structures. The conventional thickness applied to fine structures, where one exists. The representation mode used, region or point, class by class. And a precise count of the objects discarded by application of the threshold. One practical consequence follows for an image segmentation project. The fourth is indispensable to exploitation, a corpus mixing regions and points without declaring it producing reading errors in whoever uses it unaware.Three checks suited to these classes
Three verifications suffice to frame these categories in image segmentation. Counting occurrences per class and per batch, which reveals a systematic omission. Comparing counts between annotators on these specific classes. And the distribution of thicknesses on fine structures, which detects a badly set brush width. Those three checks run on a delivered export, they need no external reference, and they bear on counts and distributions rather than on contours, which makes them applicable where geometric comparison has no meaning.What these categories reveal to the client
Four observations make this conversation productive in image segmentation. Variability on these classes does not reduce through a raised requirement, it belongs to the data. A low overlap value on these classes does not necessarily signal a defect. Assistance brings nothing here, which bounds any promise of saving. And several representation modes exist, some of which suffice for the use without paying its cost. One observation follows for an image segmentation provider. The second avoids a recurring dispute, a client comparing values per class spontaneously reading a low figure as insufficient work when it measures a physical limit of the image.The pilot batch on these categories
Four images test the handling of these classes in image segmentation. An image containing many small objects, which measures the real throughput and the number of occurrences. An image containing an isolated fine structure, which verifies the brush setting and the thickness convention. An image containing an openwork structure, which tests the convention adopted on voids. And an image mixing objects either side of the threshold, which verifies the threshold is applicable in practice. One observation follows. The fourth image frequently reveals that a threshold defensible on paper becomes inapplicable on screen, an annotator being unable to estimate a size in world units with no visual reference, which requires providing a measurement aid in the tool.Approaching these categories
Five questions scope these categories in image segmentation. Do these categories feature in the reference. Their presence calls for dedicated conventions. Does the downstream use exploit their shape or only their presence. That answer opens up point marking. Is the threshold expressed in world units. Otherwise it varies with the resolution. Are these classes excluded from the assisted chain. Keeping them in it produces omissions. And does control target them specifically. A uniform sample almost never encounters them. Those five answers determine their treatment and their cost. Asking them before starting avoids a corpus whose most difficult classes were handled by default.The question that settles the treatment
One question determines the representation mode to adopt. Does the downstream use need the shape of these objects or only their presence. A presence-oriented answer opens up marking by point, which divides the cost and removes most of the variability. A shape-oriented answer requires tracing and its cost, but it then calls for an explicit convention on thickness and on voids. That question is asked in one sentence, it belongs to the downstream process and not to annotation, and it produces the largest cost variation in this whole chapter.Three decisions before starting
Three decisions handle these categories in image segmentation. Fixing the size threshold in world units and providing a measurement reference in the tool. Choosing the representation mode class by class, region or point, according to what the use exploits. And writing the convention on voids and on thickness, since those two rules determine the areas delivered. Those three decisions cost one meeting, they precede the first image, and their absence produces a corpus whose most expensive classes were handled by default.What these categories bring to the rest of the corpus
Three practices born here hold beyond these classes. Separate reporting by size band, applicable as soon as a corpus mixes regions of unequal extent. The representation mode chosen by use rather than by default, transferable to any class whose shape is not exploited. And the distinction between variability of the data and variability of execution, necessary anywhere a boundary does not frankly exist. One observation follows for an image segmentation project. Those three practices answer one same situation, that where the usual requirement asks for more than the image contains, a configuration these categories make obvious and that other classes present to a lesser degree.What this chapter leaves to the next
The decisions set out here belong to a wider frame. Three of them concern the design of the image segmentation corpus rather than the annotation gesture. The size threshold, which bounds what the corpus contains. The representation mode, which determines what the corpus permits to be learned. And the coverage of these classes, which decides whether the system will handle them at all. Those three decisions belong to the design of an image segmentation dataset, and they constitute the subject of the following chapter.What this chapter teaches
One cross-cutting observation deserves closing this examination. These categories belong to a limit of the data rather than to a difficulty of execution. Three findings compose it. No pixel belongs entirely to the object, which makes the variability irreducible by a raised requirement. The usual metrics erase them, which requires reporting them separately. And marking by point keeps the information of presence without paying for an arbitrary delimitation. That finding extends the preceding chapters, convention handling what precision cannot reach.The line worth writing into the quote
One line in a proposal prevents the argument these classes usually cause. State the size threshold and the representation mode per class, in the quote itself. Three things follow from that in image segmentation. The client sees what they are buying on these classes before the corpus exists. A later request to lower the threshold becomes a change of scope with a price, not a complaint about quality. And the provider is protected against the commonest dispute on this task, a client discovering after delivery that small objects were handled differently from what they assumed. One practical consequence follows for an image segmentation provider. These classes are where scope is most often assumed rather than agreed, so writing two sentences about them converts the likeliest source of friction into a decision the client made themselves.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Treating small objects and fine structures as one problem.
- Expressing the size threshold in pixels on a multi-resolution corpus.
- Expecting assistance to handle these categories.
- Tracing a fine structure with a brush wider than it is.
- Reporting these classes within a global average.
- Reading a low value as an execution defect.
- Sampling uniformly to control these classes.
- Omitting the convention on the voids of openwork structures.
- Requiring a delimitation where a marking would suffice.
- Discovering the cost of these classes in production.
