An object half hidden raises a question its shape does not answer. Should the box enclose what is seen, or what is guessed, and beyond what degree of masking does one stop annotating it at all.
This article sets out those rules. It extends the article on oriented boxes.
What masks an object
Five causes reduce the visible part of an object. Another object coming to stand in front of it. The edge of the image, which cuts it without hiding it. A deep shadow area or else an overexposure. An element of the scene coming to cross it. And the object itself, one part of which comes to hide behind another. One important practical consequence follows. Those five causes call for different rules in bounding box work, the image edge hiding nothing while forbidding any guess, whereas an object in front of another still leaves deduction possible.What the box can represent
Three bounding box conventions occur when facing a masked object. The box drawn around the visible part alone. The box drawn around the whole object, hidden part included. And the box around the visible part alone, completed by an attribute. One important observation follows for a bounding box project. The third convention reconciles the first two, the box staying verifiable since it describes what can be seen, the attribute indicating that the object extends beyond that frame, which leaves the use to decide what to make of it.What guessing presupposes
Four conditions make an extrapolation acceptable in bounding box work. A visible part large enough to recognise the object. A shape regular enough to permit extending the outline. A masking cause clearly identifiable in the scene. And a written convention explicitly authorising such a deduction. One practical consequence follows. Those four conditions are often missing together, an object of which only a fragment is seen calling for an extrapolation nothing guides, which produces boxes whose extent depends on each person’s imagination.What the image edge changes
Five particularities separate truncation from plain occlusion. The missing part exists absolutely nowhere in the image. No clue really permits estimating its extent. The cut follows a straight line whose position is perfectly known. A truncated object nonetheless stays often identifiable. And the visible proportion is then easily calculated. One observation follows. The second particularity settles the debate, extrapolation beyond the edge belonging to invention rather than to deduction, which makes the box limited to the image preferable in almost every case.What attributes permit preserving
Five pieces of information usefully accompany a masked object’s box. The visible proportion of the object, expressed in bands rather than as an exact value. The masking cause, occlusion by an object or truncation by the edge. The side by which the object comes to continue outside the frame. The degree of certainty bearing on the object’s identification. And an indication of self-occlusion by the object’s own shape. One important practical consequence follows. The first piece gains from staying coarse, an estimate in bands reproducing from one operator to another whereas an exact percentage varies considerably, which makes the approximate attribute more reliable than the precise one.What the visibility threshold fixes
Five effects accompany the choice of this threshold in bounding box work. The number of objects that must be annotated per image. The time spent on the most difficult cases. The proportion of barely recognisable objects within the corpus. The consistency between operators when facing limiting cases. And the system’s capacity to detect partly hidden objects afterwards. One important practical consequence follows for a bounding box project. The last effect opposes the first three, a high threshold producing a clean corpus and a system unable to recognise what it has never seen, which makes this threshold a trade-off rather than a quality setting.What the measurement must separate here
Four indicators are broken down by degree of masking. The omission rate, always far higher on partly hidden objects. The disagreement bearing on the box’s extent, proper to masked objects. The correctness of the class assigned from a reduced visible part. And the consistency of the occlusion attributes observed between operators. One important observation follows for a bounding box project. The first indicator alone justifies this breakdown, a corpus showing a good overall rate being able to miss half the partly hidden objects, which stays invisible as long as the measurement does not separate the two populations.What self-occlusion adds
Four situations make an object hide itself. A part of the object comes to pass behind another. A joint comes to fold the object back on itself. A face turned away from the viewpoint disappears. And a hollow object comes to conceal its own interior. One important practical consequence follows for a bounding box project. Those four situations often escape the instruction, a reference describing masking by another object while saying nothing about the masking the object inflicts on itself, which leaves every operator to decide whether the box encloses the folded part or only what can be seen.What throughput loses to masked objects
Four factors lengthen the time spent on a partly hidden object. Identifying the object from a reduced portion. Estimating its visible proportion for the attribute. Hesitation over the threshold when the case is a limiting one. And recourse to context to settle what the object shows. One important practical consequence follows for a bounding box project. The third factor is strongly reduced by scoping, a threshold accompanied by visual examples sparing the operator any deliberation at all, which turns a hesitation repeated thousands of times into an immediate recognition.Three errors proper to this subject
Three bounding box defects belong to masked objects. One same rule applied to occlusion and to truncation. An extrapolation beyond the edge, which invents an extent. And a visibility threshold chosen for annotation convenience. Those three defects produce plausible boxes taken in isolation, they appear by comparing several annotations of one same scene, and their common point is to bear on what the image does not show.What the annotator must know here
Five kinds of knowledge condition correct work on this subject. The convention finally adopted, visible alone or whole object guessed. The separate rule applying to the edge of the image. The visibility threshold as well as the examples that illustrate it. The attributes that must be filled in according to the masking cause. And the conduct to adopt before an object become quite unrecognisable. One important observation follows for a bounding box project. The last kind avoids a widespread defect, an operator guessing the class of an object reduced to a few visible pixels producing a confident annotation on an identification they never made, which is better flagged than settled.What the delivery format requires here
Four choices condition the use of a corpus holding masked objects. The presence of an occlusion attribute on each of the boxes. The distinction between masking by an object and cutting by the edge. An indication of the visibility threshold effectively applied to the corpus. And explicit mention of the extent convention finally adopted. One important practical consequence follows for a bounding box project. The third choice conditions any later comparison, two corpora produced under different thresholds showing incomparable object densities, which distorts an evaluation as much as an estimate of the volume to annotate.What pre-annotation changes here
Four effects follow from using an automatic proposal. Clearly visible objects then receive a reliable box. Heavily masked objects are frequently omitted. Occlusion attributes are practically never proposed. And the operator reviews what exists rather than what is missing. One important observation follows for a bounding box project. The last effect worsens precisely this chapter’s defect, an operator checking proposals ceasing to look for the partly hidden objects no box flags, which concentrates omissions on the population that already suffered most from them.What the first batch must establish
Four results justify a trial batch before production. The exact proportion of partly masked objects in the real scenes. The agreement rate obtained between operators on the extent of these boxes. The list of cases where the threshold proved hard to apply. And the throughput gap observed between visible and masked objects. One important practical consequence follows for a bounding box project. The first result regularly surprises both parties, a proportion of masked objects far above expectation turning a marginal rule into a principal one, which sometimes justifies revisiting the threshold before even starting.What the client must settle here
Four decisions belong to the client rather than to the provider. The real importance of masked objects in the use they foresee. The precise threshold below which an object no longer interests them. The conduct expected before an object cut by the edge of the image. And the usefulness of occlusion attributes in their processing chain. One important observation follows for a bounding box project. The first decision is taken badly without field experience, a client imagining their scenes clearer than they are systematically underestimating the place of masked objects, which makes a joint examination of a few real images more illuminating than any discussion of principle.What the control must target here
Four arrangements suit masked objects. A double annotation bearing on the most crowded scenes alone. A separate control of the omission rate by degree of visibility. A comparison of the occlusion attributes filled in between operators. And a targeted rereading of objects close to the adopted threshold. One important observation follows for a bounding box project. The last arrangement concentrates the effort where it pays, an object clearly above or below the threshold causing no difficulty at all whereas the border zone gathers nearly every disagreement, which makes a targeted control more effective than a uniform rereading.What the corpus must cover here
Five situations deserve an explicit place in the corpus. Fully visible objects, serving as a reference. Objects masked to varying degrees up to the adopted threshold. Objects cut by each one of the image’s four edges. Objects mutually masking each other within a dense scene. And objects one part of which hides behind themselves. One important observation follows for a bounding box project. The second situation is most often missing, a corpus setting whole objects against absent ones in no way preparing for the intermediate cases, which produces a system effective at both extremes and hesitant across the whole zone between them.What the threshold changes about the project’s economics
Four effects accompany a lowering of the visibility threshold. The number of objects that must be annotated rises appreciably per image. The time per object grows across the population added. Quality control becomes markedly heavier on the border cases. And the corpus gains value for uses in crowded scenes. One important practical consequence follows for a bounding box project. Those four effects make the threshold directly quantifiable, a lowering translating into a measurable extra cost and an estimable gain in use, which permits settling by calculation where discussion usually turns to judgement.What this subject has that is particular
Four traits distinguish this question from the course’s others. It bears on information the image quite simply does not hold. It concerns a whole population rather than isolated cases. It measures badly without a breakdown by degree of visibility. And it decides what the system will manage in real conditions. One important observation follows for a bounding box project. The last trait justifies the attention given to this chapter, real scenes almost always presenting more masked objects than demonstration images, which makes this rule the most frequent difference between a system that impresses and one that works.What the provider brings here
Four contributions distinguish a serious engagement on this subject. A rule separating occlusion from truncation from the scoping onwards. A masking attribute systematically filled in. A measurement of the omission rate broken down by degree of visibility. And visual examples covering the limiting cases of the adopted threshold. One important practical consequence follows. The second contribution costs little and changes the corpus’s value, a filled-in attribute permitting the client to discard or keep masked objects as their needs require, which replaces a decision taken once and for all with a choice they will be able to revisit.Approaching the question of occlusion
Five questions scope occlusion on a bounding box project. Does the box cover the visible or the whole object. A choice is needed. Does the image edge follow the same rule. It should not. What visibility threshold applies. It weighs on the corpus. Does an occlusion attribute exist. It avoids settling. And do masked objects count in the use. That changes everything. Those five answers determine the corpus’s consistency. Asking them before production avoids boxes whose extent varies with the operator.The question that frames the rule
One question determines the treatment of masked objects in bounding box work. Will the system have to encounter partly hidden objects. An always clear scene, an inline inspection or a neutral background, permits a high threshold and a corpus made of fully visible objects. A crowded scene, a street or a warehouse, requires annotating masked objects on pain of delivering a system blind to the most common situation. That question belongs to the conditions of use and not to annotation convenience, it is asked before the first image, and it separates a clean corpus from a useful one.Three decisions before producing
Three decisions commit the handling of masked objects in bounding box work. Choosing between the visible part alone and the whole object, then writing it down. Explicitly separating truncation by the edge from plain occlusion. And fixing the visibility threshold below which an object is not annotated. Those three decisions cost half an hour, they precede the first image, and their absence produces boxes whose extent varies with nothing to explain it.Three checks on an occlusion instruction
Three checks qualify a bounding box rule on this subject. The written distinction between occlusion and truncation by the edge. The visibility threshold, quantified rather than left to judgement. And the existence of an attribute flagging the masking. Those three checks are each asked in one question, they require no technical competence, and their absence indicates a corpus whose masked objects will be handled in three different ways.What this chapter teaches
One cross-cutting observation deserves closing this examination. Annotating what one does not see demands a rule and not a talent. Three findings compose it. The image edge hides nothing but forbids any guess. An object of which only a fragment is seen calls for an extrapolation nothing guides. And a high threshold produces a clean corpus and a helpless system. That finding extends the preceding chapters, the ambiguity bearing here on the object’s extent rather than on its limits.What this chapter shares with the previous ones
Three findings recur from the start of this course. A global measurement masks a failure on one particular population. An attribute preserved beats a decision taken in the client’s place. And pre-annotation worsens the defect proper to each chapter. One important practical consequence follows for a bounding box project. The third finding is confirmed once more here, an automatic aid always reinforcing what the corpus already knew how to do, which makes its use all the riskier when the chapter’s subject bears on the difficult cases.What this chapter leaves to the next
A tiny object raises a neighbouring and yet distinct difficulty. Three questions stay open in bounding box work. Below what size an object ceases to be annotatable. What a scene holding hundreds of objects changes about the method. And how to measure quality where each object is only a few pixels across. Those three questions belong to small objects and dense scenes, which constitute the subject of the following chapter.Why the demonstration images mislead both parties
One point about how these projects get scoped belongs at the close. The images shown at scoping are rarely the images that will be processed. Three reasons follow in bounding box work. A client selects clear examples to explain what they want. Clear examples make the task look simpler than it is. And nobody chooses a crowded image to illustrate a specification. One important practical consequence follows for a bounding box provider. Asking for a random sample rather than a chosen one costs a single email and changes the whole estimate, since the proportion of masked objects in that sample sets the threshold, the throughput and the price, and a provider who quotes from the demonstration images is quoting for a project that does not exist.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Applying the same rule to occlusion and to truncation.
- Extrapolating beyond the edge of the image.
- Leaving the visibility threshold to each person’s judgement.
- Omitting the attribute flagging that an object is masked.
- Excluding masked objects when the use encounters them.
- Guessing the extent of an object of which only a fragment is seen.
- Confusing a masked object with an indeterminate one.
- Measuring quality without separating masked objects from the rest.
- Changing threshold mid-project without reworking what exists.
- Neglecting an object’s self-occlusion by its own shape.
