The bounding box is the simplest geometry an annotator can draw. It is also the one that produces the most disagreement, because its apparent simplicity removes the need to ask the questions every other geometry forces a project to settle.
This article opens a course devoted to this geometry and its conventions.
What a box actually represents
Five bounding box definitions coexist across projects. The smallest rectangle holding the visible part of the object alone. The smallest rectangle holding the whole object, hidden parts included. The rectangle holding the identifiable part of the object alone. The rectangle enclosing the object as well as its attached accessories. And the rectangle drawn around what the final use will actually need to locate. One important practical consequence follows. Those five definitions produce boxes that all differ on the same image, which makes a corpus inconsistent as soon as the convention has not been written before the very first annotation.The uses that structure this format
Six applications rest on this geometry. Detecting objects present in a scene. Counting repeated elements on one same image. Following one same object from one frame to the next. Extracting a zone with a view to later processing. Searching for an element within a large volume of images. And preparing a finer segmentation task. One important observation follows for a bounding box project. The last use often justifies this format on its own, a box serving merely to delimit the zone a precise outline will afterwards refine, which makes it a stage rather than a final deliverable.What this format saves
Five advantages explain the dominance of the bounding box. A drawing time markedly lower than that of other geometries. Immediate uptake, even by a non-specialist operator. A compact representation in four numbers. Compatibility with most tools and models. And very rapid visual verification by a reviewer. One practical consequence follows. The first advantage is paid for in precision, a box always including some background around the object, which makes this format unsuitable as soon as the exact surface matters.What this format cannot express
Five limits of the bounding box recur across every project. The real shape of an object that is not rectangular. The clean separation of two objects that overlap. The orientation of an elongated and steeply tilted object. The hidden part of a partly masked object. And the distinction between the object and its immediate surroundings. One observation follows. The second limit weighs most in dense scenes, two boxes overlapping heavily saying nothing about which of the two objects occupies the shared zone.What the class reference adds
Five decisions accompany the definition of the classes. The number of classes adopted as well as their mutual proximity. The rule applying to an object whose class is uncertain. The treatment of an object that is visible but not identifiable. The existence of a catch-all class as well as its exact scope. And the hierarchy possibly established between neighbouring classes. One important observation follows for a bounding box project. The third decision recurs more often than one imagines, an object clearly present but too blurred to be classified placing the annotator before a choice nothing has ever settled, which produces either an omission or an arbitrary classification depending on the operator.What the convention must fix
Six points are settled before any bounding box production. The treatment given to the hidden parts of an object. The fate given to an object cut by the edge of the image. The size threshold below which an object is ignored. The visibility threshold below which an object is not annotated. Whether or not elements attached to the object are included. And the margin tolerated between the drawn box and the object. One important practical consequence follows for a bounding box project. Those six points look minor and determine most of the disagreement rate, a reference that leaves them implicit producing divergences the annotators will never report since they do not know they are diverging.What the measurement must reflect
Five indicators describe the quality of a bounding box corpus. The proportion of objects plainly present and yet not annotated. The proportion of boxes drawn with no corresponding object at all. The average overlap between the boxes of two operators. The correctness of the class assigned. And the consistency of the treatment given to limiting cases. One important practical consequence follows. The third indicator is the only one that reveals a disagreement of convention, two boxes each correct under a different definition overlapping imperfectly without either of the two being wrong, which makes it a scoping signal rather than a quality signal.What this work’s throughput presupposes
Five factors determine the time spent per image. The number of objects genuinely present in the scene. The proportion of objects that are partly masked. The fineness of the class reference. The size threshold finally adopted for small objects. And the frequency of cases the instruction does not settle. One important observation follows for a bounding box project. The last factor costs more than it appears to, each unsettled case coming to interrupt the work and call for an adjudication, which makes a complete instruction more profitable than a fast operator.What the file formats impose
Five differences separate the common bounding box formats. The coordinate system employed, absolute or relative to the image. The description of the box, by corners or by centre and dimensions. The origin of the axes, placed top left or else elsewhere. The way classes are named or numbered. And the space reserved for any additional attributes. One important practical consequence follows for a bounding box project. The second difference causes silent conversion errors, a box described by its centre and interpreted by its corners producing a rectangle that is offset yet entirely plausible, which lets a defect pass every automatic check.What the corpus must cover here
Five axes structure a bounding box dataset. The object sizes encountered, from the smallest detectable to the largest. The occlusion levels, from the clear object to the heavily masked one. The scene densities, from the isolated object through to a crowd. The shooting as well as lighting conditions. And the rare classes, systematically under-represented. One important observation follows for a bounding box project. The third axis is the most often neglected, a corpus made of consistently legible scenes in no way preparing for the case where twenty objects overlap, which produces a detector satisfying in demonstration and disappointing in real use.What the annotator must know here
Five kinds of knowledge condition correct work on this format. The definition finally adopted for what the box must hold. The rule applying to the hidden parts of an object. The precise threshold below which an object is not annotated. The distinction between a barely visible object and a genuinely absent one. And the precise moment when they should flag rather than settle alone. One important practical consequence follows for a bounding box project. The last kind protects the consistency of the whole, an operator who settles an unforeseen case alone creating a parallel convention nobody else knows about, which produces a corpus consistent per operator and inconsistent overall.What the delivery format requires here
Four choices condition the client’s reuse of the corpus. The file format adopted as well as its coordinate convention. The attributes attached to each box, such as occlusion or truncation. The marking of images checked and holding no object to annotate. And the full documentation of the reference handed over with the data. One important observation follows for a bounding box project. The third choice avoids a costly ambiguity, an image devoid of boxes being able to mean it holds no object or else that it was never handled, a distinction no inspection of the file permits restoring afterwards.What pre-annotation permits here
Three uses appreciably lighten the work on this format. A proposal of boxes bearing on the well contrasted objects. An automatic flagging of images with no object detected. And an automatic check of plainly aberrant boxes. Two caveats accompany it in bounding box work. An automatic proposal applies its own convention, rarely the project’s. And an operator who validates without adjusting reproduces that offset across the whole corpus.What the first batch must establish
Four results justify a trial batch before production. The average number of objects genuinely met per image. The agreement rate obtained between two operators on the same images. The proportion of cases the instruction does not yet settle. And the real time spent on an image judged representative. One important observation follows for a bounding box project. The third result is worth more than the others at the start, each case reported by an operator at once becoming a written rule that will no longer arise, which makes the first batch an exercise in drafting the reference as much as a measurement of throughput.What revising a reference imposes
Four situations force reopening a convention mid-project. A case become frequent that the initial instruction did not foresee. A class added well after production started. A final use specified later than the scoping was. And a persistent disagreement between two experienced operators. One important observation follows for a bounding box project. The last situation almost always signals an ambiguity in the reference rather than a fault, two competent operators diverging lastingly only where the rule does admit two readings, which makes their disagreement a diagnostic tool rather than a problem to correct.Three errors proper to this format
Three bounding box defects belong to its very simplicity. A convention left implicit on the sole grounds that it looked obvious. A box perfectly correct under one definition and wrong under another. And a format conversion producing an entirely plausible offset. Those three defects are not visible on an isolated image, they reveal themselves when the corpus is put to use, and their common point is to belong to the scoping rather than to the annotator’s hand.What the client must supply here
Four elements come from the client rather than from the provider. The final use the corpus is destined for. The exact meaning of each of the reference’s classes. The object cases they consider to be limiting. And the precise format expected by their own processing chain. One important practical consequence follows for a bounding box project. The third element is rarely asked for and proves decisive, a client who knows from experience which objects cause trouble in their field sparing the provider from discovering those cases one by one during production.What this format changes about a project’s economics
Four effects distinguish the budget of a bounding box project. The cost per object stays low compared with that of other geometries. Volume handled therefore becomes the dominant factor. Object density then weighs more than the mere number of images. And the initial scoping represents an unusually high share of the total. One important observation follows. The last effect often surprises, drafting the reference and the initial adjudications weighing proportionally more than on an expensive geometry, which makes this format economical in production and demanding in preparation.What this format has that is particular
Four traits separate this geometry from the others. It is learned in minutes and mastered only far more slowly. Its defects belong to scoping alone rather than to the operator’s dexterity. Its apparent simplicity discourages writing a reference. And its low unit cost pushes towards handling very large volumes. One important practical consequence follows for a bounding box project. Those four traits combine unfavourably, an easy format, used on large volumes and with no written reference, producing a massive and barely visible inconsistency, which explains why the most disappointing corpora are often the simplest to produce.What the provider brings here
Four contributions distinguish a seriously conducted bounding box engagement. A written reference settling the limiting cases well before production. A measurement of agreement between operators on a fraction of the corpus. Deliberate coverage of dense scenes as well as masked objects. And a format conversion checked visually rather than merely assumed. One important practical consequence follows. The first contribution attracts little notice and avoids the most widespread defect, a reference of a few pages costing far less than a single day of production while determining the consistency of the whole, which makes it the most profitable investment in this kind of project.Approaching a detection project
Five questions scope a bounding box project. What must the box hold. Five definitions coexist. What about hidden parts. The rule must be written. What size threshold applies. It weighs on the cost. Are cut objects annotated. The edge divides opinion. And does the exact surface matter. Otherwise this format suffices. Those five answers determine the corpus’s consistency. Asking them before production avoids a dataset whose convention nobody will be able to reconstitute.The question that frames the project
One question determines the choice of this geometry. Does the use call for locating or for measuring. Locating, detecting, counting or following an object, is fully satisfied by a rectangle and requires no finer geometry at all. Measuring, computing a surface or separating two adjoining objects, calls for an outline and makes the rectangle insufficient whatever its precision. That question belongs to the final use and not to the budget, it is asked before the first image, and it separates a legitimate saving from a corpus that will have to be redone.Three decisions before producing
Three decisions commit a bounding box project. Writing down what the box holds, five definitions coexisting. Fixing the rule applying to hidden parts and to cut objects. And settling the size threshold below which an object is not annotated. Those three decisions cost one meeting, they precede the first image, and their absence produces a corpus whose convention nobody will be able to reconstitute.Three checks on an existing corpus
Three checks qualify a bounding box set already assembled. The written definition of what the box holds. The rule effectively applied to objects cut by the edge of the image. And the agreement rate measured between two operators on the same images. Those three checks are each asked in one question, they require no technical competence, and their absence indicates a corpus whose consistency rests on habits rather than on a rule.What this chapter teaches
One cross-cutting observation deserves closing this opening. The simplicity of this geometry removes the prompt to ask the right questions. Three findings compose it. Five definitions of the box coexist and produce different results. A box always includes some background around the object. And annotators who diverge do not report it since they do not know. That finding opens the fourteen chapters that follow, each returning to a convention a project believes obvious.What this course will handle
Four families of question occupy the following chapters. The choice of geometry as well as the useful level of precision. The difficult situations, occlusion, small objects, dense scenes. The operational aspects, namely formats, cost, quality and instructions. And the extensions, namely video, the third dimension and automation. One important observation follows for a bounding box project. Those four families all return to a convention a project believes obvious, which explains why a format this simple occupies fifteen chapters although it fits in four numbers.What this chapter leaves to the next
The rectangle is only one geometry among several. Three questions stay open in bounding box work. When a polygon genuinely justifies its extra cost over a box. What a mask brings that neither of the two supplies. And how to decide without over-investing or having to redo the corpus. Those three questions belong to the choice of geometry, which constitutes the subject of the following chapter.Why the cheap format deserves the expensive conversation
One point about how these projects get sold belongs at the close. A low per-object price invites skipping the scoping conversation entirely. Three reasons follow in bounding box work. The client sees a simple task and expects a simple quotation. The provider fears that questions will look like padding the bill. And both discover the missing conventions only once volume is committed. One important practical consequence follows for a bounding box provider. Insisting on the scoping meeting despite the low unit price is what separates a corpus that works from one that merely looks finished, since the cost of that meeting is bounded and known while the cost of rebuilding an inconsistent corpus falls on whoever is holding the contract.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Starting production without defining what the box holds.
- Leaving the treatment of hidden parts implicit.
- Omitting the rule applying to objects cut by the edge.
- Choosing this format when the exact surface matters.
- Neglecting the size threshold and its effect on cost.
- Assuming two annotators will draw the same box.
- Confusing a loose box with a box tolerant of error.
- Ignoring the ambiguity of overlap zones.
- Changing convention mid-project without reworking what exists.
- Measuring quality without measuring agreement between operators.
