Adding depth to a rectangle looks like a natural extension. It is in fact a change of nature: the drawing stops describing what the image shows and starts describing what the scene holds, and nothing in a flat image permits verifying what is being asserted.
This article sets out that shift. It extends the article on pre-annotation.
What the cuboid adds
Five pieces of information separate a cuboid from a flat bounding box. The object’s position in space rather than in the image. Its real dimensions in physical units. Its orientation about the vertical axis. Its distance from the sensor observing it. And its height, absent from any overhead view. One important practical consequence follows. Those five pieces describe the scene and not the image, which makes verification impossible on a plain photograph and requires a data source able to measure depth.What depth demands as data
Four sources permit annotating in three dimensions rather than in a flat bounding box. A point cloud from a ranging sensor. A pair of images taken from two viewpoints. A sequence whose movement reveals the relief. And an image accompanied by a depth map. One important observation follows for a bounding box project. The first source dominates serious uses, a point cloud supplying a direct measurement where the others rest on a reconstruction, which shifts the difficulty towards reading a representation the eye is not accustomed to.What the point cloud imposes
Five difficulties accompany this representation of the scene. No single view shows the object in its entirety. The density of points falls with distance. Rear surfaces return no points at all. The object is often guessed from a few dozen points. And changing view becomes a constant operation. One practical consequence follows. The second difficulty explains a large share of the disagreements, a distant object being described by only a handful of points, which makes its dimensions largely inferred rather than measured.What the convention must fix
Six points are settled before any production of cuboids. The frame of reference used and its origin. The unit of the dimensions and the order of the axes. The origin of the rotation angle and its direction. The treatment of partly observed objects. The distance beyond which an object is no longer annotated. And how to handle an object standing on a sloping surface. One important observation follows for a bounding box project. The first point produces the most expensive errors, two frames differing by the orientation of a single axis giving valid files and false positions, which reproduces at the scale of the scene the conversion defect described for flat formats.What the control must become here
Five arrangements suit work in three dimensions. A rereading conducted from several viewpoints. A projection of the cuboid onto the matching camera image. A check of dimensional consistency within each class. A control of the annotated objects’ position on the ground. And a double annotation broken down by distance band. One important practical consequence follows for a bounding box project. The second arrangement makes the control affordable, a correct cuboid projecting exactly onto the object visible in the image whereas a badly placed one produces an immediately perceptible offset, which brings a spatial verification back to a flat examination.What throughput becomes here
Five factors lengthen the time spent per object in three dimensions. The change of view needed to judge each face. The setting of seven parameters instead of four. The inference of dimensions on the unobserved faces. The check of consistency between the views. And the fatigue proper to manipulating a space. One important practical consequence follows for a bounding box project. Those five factors place the cost of a cuboid several times above that of a rectangle, which makes the choice of the third dimension markedly more committing than moving from the rectangle to the polygon.What distance changes about reliability
Four effects accompany an object’s growing remoteness. The number of points describing the object falls rapidly. Its dimensions become inferred rather than measured. The class itself sometimes becomes uncertain. And the gap between two operators rises sharply. One important observation follows for a bounding box project. Those four effects require breaking down every quality measurement by distance band, a global rate mixing near and far objects describing above all the distribution of distances in the corpus rather than the quality of the work delivered.What inferring hidden faces presupposes
Four supports permit completing a partly observed object. Knowledge of the usual dimensions of its class. The assumed symmetry of the object about its axis. Its position on the ground, which fixes one of the faces. And its consistency with the preceding frames of the sequence. One important practical consequence follows for a bounding box project. The first support produces an ambiguous effect, an operator completing an object from the usual dimensions of its class teaching the system a regularity that comes from the instruction rather than from the world, which makes this practice useful for consistency and misleading for measurement.What the sequence adds here
Four requirements appear as soon as the frames follow one another. An object keeps the same identity from one frame to the next. Its dimensions must not vary through the sequence. Its trajectory must stay physically plausible. And its orientation must follow its movement. One important practical consequence follows for a bounding box project. The second requirement supplies an almost free control, the dimensions of one same object having to stay constant while nothing in the drawing enforces it, which turns any variation observed into an error signal with no reference annotation being needed.Three errors proper to the third dimension
Three defects belong to the move from rectangle to cuboid. An undocumented frame producing systematically false positions. Annotation continued beyond the usable distance. And a control conducted from a single view. Those three defects produce perfectly valid files, they reveal themselves when the positions are used to decide, and their common point is to bear on information the image does not permit verifying.What the client must supply here
Four elements come from the client rather than from the provider. A description of the frame used by their processing chain. The parameters linking the depth sensor to the cameras. The maximum distance at which their use requires a detection. And the positional precision they genuinely need. One important observation follows for a bounding box project. The second element conditions the whole control, a projection of the cuboid onto the camera image becoming impossible without those parameters, which deprives the project of the only affordable verification arrangement described in this chapter.What this format changes about the project’s economics
Four effects distinguish the budget of a three-dimensional project. The cost per object exceeds that of a rectangle several times over. Training an operator lengthens by several weeks. Control demands a specific and expensive arrangement. And acquiring the data itself weighs on the total. One important observation follows for a bounding box project. The last effect clearly separates this format from the others, flat annotation making do with images already available whereas the third dimension presupposes a dedicated acquisition, which shifts part of the budget upstream of the annotation work itself.What the annotator must know here
Five kinds of knowledge condition correct work in three dimensions. Reading a point cloud and choosing the useful views. The order of movements avoiding back and forth between parameters. The inference rule applying to unobserved faces. The distance beyond which an object is no longer annotated. And the reflex of checking the projection before validating. One important practical consequence follows for a bounding box project. The first kind takes several weeks and not a few hours, reading a point cloud being acquired only through practice, which makes replacing an operator markedly more expensive than on flat work.What the delivery format requires here
Five elements usefully accompany a corpus of cuboids. The definition of the frame and the orientation of each axis. The unit used for the dimensions and the positions. The angle convention with its origin and its direction. The parameters linking the sensor to the cameras. And the distance threshold applied to the corpus. One important observation follows for a bounding box project. The fourth element is neglected and deprives the client of a means of verification, a corpus delivered without those parameters no longer permitting the cuboids to be projected onto the images, which makes any resumption of the control impossible without returning to the producer.What the working tool changes here
Five functions make this work practicable. A fast switch between several viewpoints. A permanent projection of the cuboid onto the camera image. An independent adjustment of each dimension. An automatic snap to the ground plane. And a propagation of the cuboid between two frames of a sequence. One important observation follows for a bounding box project. The fourth function removes a large share of the work, height above ground being deduced from the detected plane rather than from a manual setting, which removes one degree of freedom without taking anything from the object’s description.What this subject has that is particular
Four traits distinguish this question from the course’s others. The annotation describes a measurement and no longer a designation. No single view permits verifying the result on its own. The data itself must be acquired for this purpose. And the operator’s competence becomes rare and slow to build. One important practical consequence follows for a bounding box project. Those four traits make the cuboid a distinct trade rather than a variant, which explains why a provider competent in flat annotation is not automatically so in three dimensions and justifies checking before entrusting this kind of project.What the corpus must cover here
Five situations deserve an explicit place in the corpus. Near objects, entirely described by the points. Objects at medium distance, only partly observed. Objects close to the distance threshold adopted. Objects partly masked by others. And scenes holding ground that is not horizontal. One important observation follows for a bounding box project. The last situation is almost always missing, a corpus acquired on flat terrain in no way preparing for slopes or camber, which produces a system effective in test conditions and caught out at the first relief encountered.What the first batch must establish
Four results justify a trial batch before production. The point density observed at the different distances. The real time spent on a representative scene. The dimensional gap obtained between two operators. And the validity of the frame verified by projection. One important practical consequence follows for a bounding box project. The last result must precede everything else, a badly understood frame invalidating the entirety of the work produced afterwards, which makes this verification the very first operation of a three-dimensional project.What this chapter shares with the previous ones
Three findings recur from the start of this course. An unwritten frame convention produces data both false and valid. A global measurement masks a failure on a whole population. And the most expensive defect stays the one no examination reveals. One important practical consequence follows for a bounding box project. The first finding recurs here in its heaviest form, a badly interpreted frame displacing every object in a scene without producing a single invalid file, which extends to the scale of volume what flat format conversion already produced on the plane.What the bounding box keeps nonetheless
Four roles survive for the rectangle within a three-dimensional project. It serves as a first pass before the cuboid is placed. It describes the object on the camera images accompanying the scene. It permits a fast control by projection. And it suffices for the uses that ignore depth. One important observation follows for a bounding box project. The second role explains the lasting coexistence of the two formats, one same scene being described in volume for the decision and in rectangles for the verification, which makes the rectangle a companion to the cuboid rather than a format it would replace.What the provider brings here
Four contributions distinguish a serious engagement in three dimensions. A frame documented and verified on a first batch. A distance threshold fixed from the real density of points. A control by projection onto the camera images. And quality measurements broken down by distance band. One important practical consequence follows. The second contribution protects both parties, a threshold fixed after examining the cloud avoiding invoicing objects whose dimensions belong to invention, which is worth more than an extended corpus of which one part is unusable.Approaching a three-dimensional project
Five questions scope such a bounding box project. Is depth measured or reconstructed. That changes everything. Is the frame documented. Otherwise the positions will be false. Up to what distance does one annotate. The points thin out. Are hidden faces inferred. A rule is required. And can the control judge a volume. Flat rereading does not suffice. Those five answers determine the project’s feasibility. Asking them before production avoids a corpus whose correctness nobody will be able to verify.The question that frames the shift
One question determines the interest of the cuboid over the bounding box. Does the use need to know where the object is or only that it is there. Knowing it is there, detecting a presence or counting passages, is fully satisfied by a rectangle and requires no depth measurement. Knowing where it is, computing a distance or anticipating a trajectory, requires a cuboid and the corresponding cost. That question belongs to the decision the system will have to take, it is asked before any acquisition, and it separates two projects whose budgets are nothing alike.Three decisions before producing
Three decisions commit a three-dimensional cuboid project. Documenting the frame, the order of axes and the direction of rotation. Fixing the distance beyond which an object is no longer annotated. And writing the inference rule for the unobserved faces. Those three decisions cost half a day, they precede the first scene, and their absence produces a corpus whose positions will be false with nothing to signal it.Three checks on a three-dimensional corpus
Three controls qualify a set of cuboids already produced. The documentation of the frame and the order of the axes. The correct projection of the cuboids onto the camera images. And the breakdown of quality measurements by distance band. Those three controls are each asked in one question, they require no particular competence, and their absence indicates a corpus whose positions nobody can affirm to be correct.Three questions before committing
Three questions precede accepting a cuboid project. Are the frame and the calibration parameters available. Up to what distance does the use require reliable detection. And does the point density stay sufficient there. Those three questions are asked before the quotation, they call for examining a real scene, and their absence leads to committing to work of which one part will be invention.What this chapter teaches
One cross-cutting observation deserves closing this examination. The drawing stops describing the image and starts describing the scene. Three findings compose it. No single view shows the object in its entirety. A distant object is described by only a handful of points. And two frames differing by a single axis give false positions. That finding extends the preceding chapters, annotation becoming a measurement rather than a designation.What this chapter leaves to the next
The rectangle survives all these shifts without ever disappearing. Three questions stay open in bounding box work. What recent models change about annotation work. Why this format outlives richer geometries. And what will remain to be done once automation has advanced. Those three questions belong to what comes next, which constitutes the subject of the following chapter.Why declining this work is sometimes the right answer
One point about taking on cuboid projects belongs at the close. A provider experienced in flat annotation is tempted to accept these. Three reasons follow in bounding box work. The client assumes the two are variants of one skill. The rate per object looks generous compared with rectangles. And the difficulty only becomes visible once production starts. One important practical consequence follows for a bounding box provider. Saying plainly that this is a different trade costs one project and protects the relationship, since a corpus whose frame was misunderstood or whose distant objects were invented is discovered months later by the client, and no explanation offered at that point repairs what the initial honesty would have avoided.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Treating a cuboid as a rectangle with one dimension added.
- Omitting to document the frame of reference used.
- Annotating beyond the distance where the points thin out.
- Leaving the inference of hidden faces to each person’s judgement.
- Controlling a volume by a plain rereading in overhead view.
- Costing a cuboid at the rate of a rectangle with a surcharge.
- Neglecting the fatigue proper to manipulating a space.
- Confusing measured depth with reconstructed depth.
- Ignoring the effect of distance on the reliability of dimensions.
- Assuming a flat drawing tool suits this work.
