The choice between a box, a polygon and a mask presents itself as a trade-off between precision and budget. It rarely is one. What actually decides is what the result will have to make possible, and that question is settled before opening any tool at all.
This article compares those three geometries. It extends the bounding box guide.
What each geometry produces
Three representations dominate image annotation, in bounding box work and beyond. The bounding box, a plain rectangle defined by four numbers. The polygon, a series of points following the object’s outline. And the mask, a decision taken separately for every pixel. One important practical consequence follows. Those three representations do not differ only in fineness, a rectangle that locates, a polygon that delimits and a mask that assigns, which makes their comparison more useful in terms of use than in terms of precision.What cost separates
Four gaps distinguish these geometries at the moment of execution. The drawing time, running from the fastest to the slowest. The training period an operator needs. The difficulty of the control conducted by a reviewer. And the weight of the files finally produced. One important observation follows for a bounding box project. The first gap is the most often cited and the least decisive, a polygon costing several times the price of a plain box without that ratio sufficing to settle anything, which shifts the decision towards what each geometry makes possible.What only the box permits
Four situations make the bounding box preferable. A large volume to handle on a very constrained budget. An object whose overall shape approaches a rectangle. A use of mere location or of counting. And a first pass meant to be refined afterwards. One practical consequence follows. The last situation is often underestimated, a box serving merely to reduce the search space before a precise outline, which makes the two geometries complementary rather than competing.What only the polygon permits
Four situations make it necessary despite its extra cost. An object that is elongated, tilted or else irregular in shape. Two adjoining objects the use requires separating. A measurement of surface or else of dimension. And a scene where the background included would come to distort the learning. One observation follows. The last situation justifies the extra cost better than the others, a box drawn around a thin object including more background than object itself, which teaches the system to recognise the context rather than the target.What only the mask permits
Four needs impose this geometry. An assignment of each one of the pixels to a class. An object whose outline holds genuine holes. A boundary between zones rather than between objects. And an exact rather than merely approximate surface calculation. One important practical consequence follows. The third need clearly separates the mask from the other two, a boundary drawn between two textures delimiting no identifiable object, which places this case beyond the reach of any geometry founded on the object.What the background proportion reveals
Five objects illustrate the gap between the box and the outline. A compact object, whose box almost exactly matches its shape. An elongated but horizontal object, still reasonably well framed. The same object once tilted, whose box doubles in area. An object shaped like an L or else a U, whose whole centre is empty. And a thread-like object, almost entirely absent from its own box. One important observation follows for a bounding box project. That progression is measured before deciding, a calculation of the ratio between the object’s area and the box’s over a few dozen examples indicating at once whether this format is suitable, which replaces a debate of opinion with a figure.What mixed geometry permits
Four arrangements avoid choosing a single geometry. A box on the common classes and a polygon on the critical classes alone. A box everywhere and a polygon on a mere fraction of the corpus. A box on the first pass followed by a targeted later refinement. And a box on compact objects, a polygon on thin ones. One important practical consequence follows. Those four arrangements demand strict documentation, a corpus mixing two geometries without ever signalling it becoming unusable for whoever handles it unknowingly, which turns an intelligent saving into an invisible defect.What conversion between geometries allows
Four passages are possible, only two of them without loss. A mask converts into a polygon with a loss that is usually negligible. A polygon converts into a box without the slightest loss. A box on the other hand never becomes a polygon again. And a polygon never becomes a perfectly exact mask again. One important observation follows for a bounding box project. Those four passages describe a one-way street, the fine geometry always degrading towards the coarse and never the reverse, which makes a finely annotated corpus reusable for every use and a box corpus permanently limited to its own.What throughput separates
Four factors widen the time gap between the geometries. The number of points that must be placed on an irregular outline. The rework made necessary as soon as a point is badly placed. The zoom required to follow an edge with precision. And the fatigue accumulated on genuinely continuous drawing work. One important observation follows for a bounding box project. The last factor explains why the observed cost ratio often exceeds the theoretical time ratio, an operator holding a high pace on rectangles and slowing progressively on outlines, which makes an estimate based on the first hours systematically optimistic.What quality control changes here
Four differences separate the control of the three geometries. A box is checked at a single glance across a whole image. A polygon demands a zoom to judge its drawing properly. A mask requires a superimposed display to be appreciated. And the acceptable disagreement varies very strongly with the geometry. One important practical consequence follows. The last difference is neglected at scoping, a gap of two pixels staying insignificant on a box and becoming notable on an outline, which makes a single tolerance threshold inapplicable as soon as a corpus mixes several geometries.What the first batch must establish
Four results justify a trial before fixing the geometry. The measured ratio between object area and box area. The throughput gap observed between box and polygon on the same images. The real frequency of adjoining objects that must be separated. And the performance obtained on a sample annotated in each geometry. One important observation follows for a bounding box project. The last result settles matters better than any argument, one same sample annotated twice and then evaluated on the same use giving a perfectly quantified performance gap, which shifts the debate from a question of principle towards a measured comparison.What the type of object suggests
Five families of object naturally orient the choice. Vehicles as well as street furniture, compact and well framed. People, whose box includes a variable share of background. Animals in motion, whose posture changes the shape. Thin objects, such as cables, rods or cracks. And zones devoid of an outline of their own, such as vegetation or water. One important practical consequence follows for a bounding box project. That progression is a starting point and not a rule, one same type of object shifting from one geometry towards the other with the scale and with the use, which makes measurement on a real sample preferable to any list by category.Three errors proper to this choice
Three defects accompany the geometry decision in bounding box work. A rectangle adopted on very thin objects out of mere economy. A polygon paid for on objects the box already framed well. And a mixed geometry used without ever being documented. Those three defects are decided in minutes and paid for across the whole corpus, they are corrected only by redoing the production, and their common point is to settle matters without having measured what could have been measured.What the client must specify here
Four pieces of information come from the client and orient the choice. What the result will concretely have to allow computing or deciding. The objects they know from experience are hard to tell apart in their field. The later uses they may envisage for the same corpus. And the real budget constraint, stated before the recommendation. One important observation follows for a bounding box project. The last piece gains from being given early rather than withheld, a provider who knows the budget being able to propose a suitable mixed geometry, whereas one who does not recommends a uniform geometry the client will refuse without explaining why.What the delivery format requires here
Four choices condition the reuse of a mixed corpus. A field stating the geometry used for each object. A single coordinate convention whatever the geometry. A tolerance threshold documented separately for each of them. And a note explaining why the corpus mixes two levels. One important practical consequence follows for a bounding box project. The first choice looks trivial and is almost always missing, a file holding rectangles and outlines with no distinction at all forcing whoever takes it up to guess the intention, which loses the very benefit of mixed geometry.What the annotator must know here
Five kinds of knowledge condition correct work on a mixed corpus. The geometry precisely expected for each of the reference’s classes. The rule applying to an object whose class hesitates between two levels. The number of points wanted on an outline, neither too many nor too few. The conduct to adopt when the planned geometry looks plainly unsuitable. And the moment to flag that unsuitability rather than work around it. One important practical consequence follows for a bounding box project. The last kind carries scoping errors back to those who can correct them, an operator observing daily that a box hardly suits a class holding information the initial scoping did not have, which makes those reports a corrective rather than a complaint.What this choice commits over time
Four effects show themselves long after delivery. A fine corpus still adapts to a use that was not foreseen. A coarse corpus forces starting over for any new use. A mixed geometry stays usable provided it is documented. And an undocumented corpus loses its value as those who produced it leave. One important observation follows for a bounding box project. The last effect is observed two or three years later, a wholly renewed team taking up a corpus without knowing which convention produced it, which makes documentation as determining as the geometry itself for the work’s lifespan.What pre-annotation changes about the choice
Four effects modify the trade-off once a tool proposes shapes. The automatic outline appreciably reduces the cost gap between box and polygon. Checking an outline stays slower than checking a plain box. The hardest objects still have to be drawn entirely by hand. And the tool’s own convention often differs from the project’s. One important observation follows for a bounding box project. The first effect genuinely moves the boundary of what is reasonable, an assisted polygon now costing markedly less than a wholly manual one, which makes an old comparison between the two geometries progressively obsolete and justifies redoing the calculation rather than reusing it.What the geometry changes about the reference
Four points of the reference depend on the geometry adopted. The rule applying to hidden parts, which differs with the geometry. The treatment given to an object made of several separate pieces. The way of representing an object with a hole at its centre. And the number of points judged sufficient on an irregular outline. One important practical consequence follows for a bounding box project. The second point illustrates the gap between geometries, an object whose two pieces are separated by an obstacle calling either for one enclosing box or for two distinct outlines, which produces a different count depending on the geometry and distorts any comparison of volumes between two corpora.What the provider brings here
Four contributions distinguish a serious scoping in bounding box work. A measurement of the background proportion before any recommendation. A proposal of mixed geometry where it is justified. Explicit documentation of the geometry used for each class. And a warning about the irreversible nature of the coarse choice. One important practical consequence follows. The last contribution hurts the provider in the short term, a client so warned sometimes choosing a geometry more expensive than the one they had in mind at the outset, whereas it avoids the full rework of the corpus for which the provider would have carried the responsibility.Approaching the choice of a geometry
Five questions scope it in bounding box work. Is it locating or measuring. The answer settles nearly everything. Are the objects compact. Otherwise the box holds too much background. Must adjoining objects be separated. The box does not permit it. Does the boundary bear on zones. Then the mask is required. And is the volume very large. Cost then becomes decisive. Those five answers determine the useful geometry. Asking them before production avoids paying for useless precision or redoing an insufficient corpus.The question that frames the choice
One question determines the useful geometry in bounding box work. Will the corpus serve this project or several successive projects. A single project permits the coarsest geometry the use tolerates, the corpus disappearing with the model it served. Several projects justify the finest geometry the budget permits, each later use being able to degrade it towards what it needs. That question belongs to the client’s ambition and not to the immediate need, it is asked before the first image, and it separates a one-off saving from a reusable asset.Three decisions before producing
Three decisions commit the choice of geometry in bounding box work. Measuring the background proportion included on a real sample. Checking whether adjoining objects will have to be separated. And deciding whether the corpus will serve one use or several. Those three decisions cost half a day, they precede production, and their absence produces either precision paid for nothing or a corpus to redo.Three checks before settling
Three checks illuminate the choice of a geometry. The average ratio between the object’s area and that of its box. The frequency of adjoining objects in real scenes. And the list of uses envisaged for the corpus. Those three checks are each asked in one question, they require only a sample of a few dozen images, and their absence turns a technical decision into a budgetary preference.What this chapter teaches
One cross-cutting observation deserves closing this examination. The geometry is chosen from the use and not from the budget. Three findings compose it. A rectangle locates, a polygon delimits, a mask assigns. A box around a thin object holds more background than object. And a boundary between two textures delimits no identifiable object. That finding extends the preceding chapter, the question of the final use returning here as a quantifiable trade-off.What this chapter shares with the previous one
Three findings recur from one chapter to the next. The decision is taken from the final use and not from the cost. A figure measured on a sample is worth more than an opinion. And a badly scoped choice is paid in rework rather than in a visible defect. One important observation follows for a bounding box project. Those three findings will return in the following chapters under other forms, which suggests that this geometry’s difficulties belong less to its nature than to the scant attention its simplicity earns it.What this chapter leaves to the next
Choosing the rectangle leaves the question of how to draw it entirely open. Three questions stay open in bounding box work. How tightly a box must enclose the object it surrounds. What a tolerated margin changes about the final use. And how to make that rule applicable in the same way by everyone. Those three questions belong to the drawing rules, which constitute the subject of the following chapter.Why recommending the more expensive geometry is defensible
One point about how this recommendation lands belongs at the close. Advising a finer geometry looks like a provider selling more work. Three reasons follow in bounding box work. The provider does invoice more when the client accepts. The client cannot easily verify the advice before committing. And the cheaper option would have looked acceptable for months. One important practical consequence follows for a bounding box provider. Showing the measured background ratio is what removes the suspicion, since a number computed on the client’s own sample makes the recommendation checkable rather than assertable, and a provider who offers the measurement before the quotation is arguing from evidence instead of authority.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Choosing the geometry from the budget rather than from the use.
- Using a box on thin or very elongated objects.
- Paying for a polygon where a box would have sufficed.
- Ignoring the proportion of background held in the box.
- Confusing object separation with outline precision.
- Assuming a mask converts into a box without loss.
- Mixing several geometries in one corpus without documenting it.
- Neglecting the training cost proper to each geometry.
- Deciding without having measured the background proportion on a sample.
- Changing geometry mid-project without reworking what exists.
