Robotics and logistics differ from the other 3D annotation verticals in one particular requirement. The point is not to avoid objects but to grasp them, and that difference changes what the data must describe.This article sets out what this context demands. It extends the article on point cloud segmentation.
What distinguishes this context
Five properties characterise this 3D annotation vertical and clearly separate it from the others.The environment is controlled, a warehouse or a production cell presenting bounded variability.The objects are known, a catalogue of references replacing an open nomenclature.The working distance is short, which gives a high point density on the objects.The purpose is manipulation, which requires the shape and not the footprint.And the tolerance is fine, an error of a few millimetres making a grasp fail.One important practical consequence follows. The first three properties considerably simplify the work compared with autonomous driving, and the last two raise the requirement on what remains.What manipulation requires
Four pieces of 3D annotation information condition a successful grasp.The object’s exact shape, a cuboid including empty space around a non-convex part.The complete orientation, all three angles mattering where the object does not rest flat.The grasping surfaces, planar faces or graspable zones identified separately.And accessibility, a correctly described object potentially staying unreachable in a pile.One important observation follows. The third piece distinguishes this vertical from all the others, annotation not merely describing an object but designating the zones through which a gripper can reach it.The tasks requested
Five 3D annotation formulations occur in this vertical.Their cost differs strongly.Presence detection, which suffices for a count or a conformity check.Pose estimation, which gives the complete position and orientation of a known reference.Instance segmentation, which separates identical stacked objects.Grasp point annotation, which designates where to take hold.And state classification, a parcel intact or damaged, a pallet conforming or not.One practical consequence follows for a 3D annotation project. The second formulation dominates demand, pose estimation on a known catalogue constituting the most frequent need in manipulation robotics.What the known catalogue changes
This property transforms the economics of this vertical.The objects handled come from a finite and documented set.Four direct consequences follow.Dimensions are known, which removes extrapolation and turns annotation into positioning a model.Control becomes very powerful, any dimension diverging from the catalogue signalling an error.The nomenclature is closed, which rules out the difficulty of unanticipated classes.And a digital model of the object can serve as a template, the annotator aligning a known shape rather than delimiting an unknown one.One observation follows. The last consequence changes the nature of the gesture, annotation becoming a registration rather than a delimitation, which makes it faster and more reproducible than in an open environment.What pairing with a model brings
A method specific to this vertical deserves setting out since it changes the nature of the work.A digital model of each reference permits replacing delimitation with alignment.Four concrete benefits follow in 3D annotation.The shape delivered is exact, including on unobserved faces, since it comes from the model and not from an extrapolation.Consistency between annotations is total on a given reference.Control reduces to verifying an alignment rather than assessing a delimitation.And the gesture transmits quickly, aligning being easier to teach than delimiting.One important observation follows. Those four benefits presuppose that the client supplies their digital models, a simple request that rarely succeeds at the first exchange and that is worth making explicitly at scoping.The difficulty of stacking
This configuration dominates the 3D annotation difficulties in this context.Identical objects stacked in bulk mutually overlap.Four effects compound in 3D annotation.Each object is only partially observed, one face at best being visible.Boundaries between identical objects have no geometric signature.Orientation becomes hard to establish from a partial observation.And counting itself becomes uncertain in the deeper layers.One important practical consequence follows for a 3D annotation project. Those four effects justify a depth convention, only the objects accessible from above being annotated, which corresponds moreover to what a robot can actually grasp.What short distance brings
Four advantages accompany a sensor close to the scene in 3D annotation.Point density is high, which makes shapes describable with precision.The class is rarely ambiguous, geometry sufficing to identify a known reference.Boundaries between objects are better resolved where a gap exists.And lighting conditions are controlled, which makes the data reproducible.One important observation follows. Those four advantages move the difficulty from perception towards precision, this vertical not posing the problem of knowing what one sees but that of describing it finely enough to act on it.The difficult surfaces
Four materials common in logistics degrade 3D annotation data.Transparent plastic film, ubiquitous on pallets and invisible to an optical sensor.Polished metal, which produces reflections and aberrant points.Matt black surfaces, whose return is too weak to be measured.And glossy cardboard, whose behaviour varies with the angle of incidence.One practical consequence follows for a 3D annotation project. Those four materials make up most of a logistics environment, which makes the convention on missing and aberrant points more decisive here than in any other vertical.Annotating grasp points
A distinct task often accompanies the geometric description and it deserves separate treatment.Designating where to grasp an object presupposes criteria shape alone does not supply.Five criteria compose it in practice.Local flatness, a suction cup requiring a continuous surface of a minimum size.The available clearance, a gripper needing space on either side.The zone’s strength, soft packaging not tolerating the same hold as a rigid wall.Balance, a grasp point far from the centre of mass causing a rotation.And effective accessibility, a perfect but covered surface being unusable.One important practical consequence follows for a 3D annotation project. Those five criteria depend on the gripper used, which makes a grasp point corpus specific to one piece of equipment rather than reusable, a distinction to state before building it.The adapted quality control
Five verifications exploit the properties of this 3D annotation context.Conformity to the catalogue, any divergent dimension signalling a pose error or a reference confusion.Non-intersection between objects, two solids not being able to occupy the same volume.Gravitational consistency, an object floating with no support signalling a position error.Declared accessibility, an object marked graspable and physically covered constituting a contradiction.And registration stability, two annotators aligning the same model having to obtain close poses.One important observation follows. The first check is the most powerful of all the 3D verticals, the catalogue supplying an exact dimensional truth neither the road nor the forest supplies in 3D annotation.What repeatability permits
One property of the controlled environment opens a possibility open verticals do not offer.The same scene can be captured several times under identical conditions.Three exploitations follow in 3D annotation.Building difficult examples by deliberate staging, a dense bulk or a badly oriented object being reproducible at will.Measuring the sensor’s variability, two captures of a still scene revealing the equipment’s own noise.And verifying an annotation by a capture from another angle, which reveals the faces wrongly extrapolated.One important practical consequence follows for a 3D annotation project. The first exploitation resolves the rare class difficulty the other verticals suffer, the problematic configurations occurring on demand rather than being awaited.What this work costs
Four factors determine the 3D annotation load in this vertical.The overlap rate between objects, the dominant factor determining the difficulty per object.The number of objects per scene, often high in a bin or on a pallet.The requirement on orientation, a precise grasp demanding all three angles.And the presence of grasp points to be annotated separately.One practical consequence follows. The first factor produces a considerable variation, a scene of separated objects and one of dense bulk not comparing even at equal object counts in 3D annotation.The uses outside grasping
Four applications belong to this vertical without concerning manipulation, and they are less demanding in 3D annotation.Load checking, which verifies that a pallet or a container matches the order.Occupied volume measurement, which optimises filling and is satisfied by an envelope.Obstacle detection for an autonomous truck, which belongs to avoidance logic rather than to grasping.And integrity checking, a deformed parcel or an unstable pallet being detected by comparison with an expected shape.One observation follows for a 3D annotation provider. Those four applications are satisfied by a cuboid or an envelope, which places them in the most economical regime, and they represent a substantial share of logistics operators’ real needs.What pre-annotation contributes here
Three observations situate the contribution of 3D annotation assistance in this context.Automatic registration of a known model works well on isolated and well-observed objects.It fails in dense bulk, partial observation admitting several equally compatible poses.And verification stays fast, a misalignment being immediately visible when the model is overlaid on the cloud.One observation follows for a 3D annotation project. The third finding makes assistance particularly worthwhile here, the verification cost being low and the gain high, a more favourable configuration than in the open-nomenclature verticals.What this vertical does not permit
Four limits bound what a 3D annotation can establish here.An entirely covered object produces no point and stays out of reach.A transparent surface is not measured, which leaves a void in the data where an object exists.A deformable object, a sack or a textile, has no stable geometric model to register.And the orientation of an object with rotational symmetry stays undetermined about its axis.That third limit deserves emphasis. It means a catalogue mixing rigid and soft objects calls for two distinct treatments, the second belonging to segmentation rather than to pose estimation.How the corpus ages
One property distinguishes this vertical over time and it deserves setting out to the client.A catalogue of objects evolves, references entering and leaving regularly.Three practical consequences follow.A withdrawn reference does not devalue the corpus, its examples staying valid for what they describe.A new reference calls for a targeted enrichment rather than a complete rework.And a packaging change on an existing reference invalidates the corresponding examples without touching the others.One important observation follows for a 3D annotation project. Those three consequences make the reference metadata indispensable, with no product identifier on each example it becomes impossible to selectively withdraw the obsolete objects and the corpus degrades globally rather than locally.The conventions to write first
Five rules cover most of the disagreement in this vertical.The annotation depth in bulk, expressed by an accessibility criterion rather than by a number of layers.The fate of the data-free zones caused by transparent or reflective surfaces.The treatment of deformable objects, excluded from registration and routed to segmentation.What to do with objects having rotational symmetry, whose orientation stays undetermined about the axis.And the overlap threshold beyond which an object is declared non-graspable.One practical consequence follows for a 3D annotation project. Those five rules all concern situations the catalogue does not document, which confirms that this vertical’s difficulty lies in the presentation of the objects rather than in their identification.The first batch of a robotics project
Four scenes compose a pilot batch that genuinely tests the conventions.A scene of separated and well-presented objects, which supplies the reference throughput.A dense bulk scene, which measures the vertical’s principal cost factor.A scene containing a problematic material, transparent film or polished metal, which tests the convention on missing points.And a scene with a deformable object if the catalogue contains one, which verifies that the distinct treatment is provided for.One observation follows. The throughput gap between the first two scenes is this batch’s most useful figure, it directly measures the effect of overlap and it supplies the basis for a costing the object count does not permit in 3D annotation.Approaching a robotics project
Five questions scope a 3D annotation project in robotics.Is the object catalogue closed and documented. A positive answer turns annotation into registration.Are the objects presented separated or in bulk. That answer determines the principal cost factor.Which materials compose the objects. That answer predicts the data-free zones.Are grasp points expected. That answer adds a distinct task.And what tolerance does the grasp require. That answer bounds what must be promised.Those five answers determine the load and the feasibility. Asking them before starting avoids a corpus precise on objects no gripper will reach.The question that frames the project
One question determines the cost regime and it concerns how the objects are presented.Do the objects arrive separated or in bulk.An answer indicating an ordered presentation, a conveyor or a tray, places the project in the most economical regime, each object being well observed and easily registered.An answer indicating bulk requires several times the work, partial observation and the absence of a geometric boundary dominating the difficulty.That question concerns the physical upstream rather than the annotation, and it sometimes opens a correction the client had not considered, a change of presentation costing less than a bulk corpus.Three checks before delivering
Three checks suffice to rule out most defects in a robotics batch.Compare each annotated object with the dimensions of its catalogue reference, a gap signalling an erroneous registration or a reference confusion.Detect intersections between objects, two solids not being able to occupy the same volume and an overlap betraying an approximate pose.And verify that objects declared graspable are not covered by others, a frequent contradiction in dense bulk.Those three checks run on a delivered export, they exploit the catalogue rather than an external reference, and they cover what distinguishes a corpus usable by a robot from a visually correct one.Three decisions before the first batch
Three decisions commit the feasibility of a 3D annotation project in robotics.Obtaining the digital models of the references, which transform the work and the control.Fixing the accessibility criterion in bulk, which bounds the number of objects to handle per scene.And declaring the tolerance the gripper requires, which determines the precision to reach and what can be promised.Those three decisions cost one meeting and a few file exchanges, they precede the first capture, and their absence produces a careful corpus whose precision matches no identified requirement.What this vertical brings to the others
Three practices born here hold beyond robotics.Registering a known model rather than delimiting, applicable anywhere the objects come from a finite and documented set.Control by comparison with an external dimensional truth, transferable as soon as a size reference exists.And the deliberate construction of difficult configurations, possible whenever the scene can be set up rather than observed.Those three practices presuppose a controlled environment, a condition few verticals meet, which explains why they stay little diffused despite their effectiveness.What this chapter teaches
One cross-cutting observation deserves closing this examination.This vertical inverts the usual relation between perception difficulty and precision requirement.Three findings compose it.The controlled environment and the closed catalogue remove most of the class ambiguities that dominate the other verticals.The grasping requirement imposes a precision neither driving nor mapping demands.And the catalogue supplies an exact dimensional truth, which makes control more powerful here than anywhere else.That finding matches the one the preceding series established about controlled environments: the constraint moves from identification towards measurement, and the corpus is worth what its precision is rather than what its coverage is.Why this vertical suits a smaller provider
One observation about market position belongs here, since this vertical differs commercially from autonomous driving.Robotics and logistics projects are smaller, more numerous and less concentrated than automotive ones.Three properties produce that.A corpus is bounded by a catalogue rather than by a world, which caps the volume at something a mid-sized team can deliver.The buyer is often an integrator or an end user rather than a research programme, which shortens the decision and the payment terms.And a successful first engagement extends naturally, new references and new sites arriving as the deployment grows.One practical consequence follows for a 3D annotation provider. Those three properties make this vertical a more realistic entry point than autonomous driving, where the volumes and the qualification cycles favour whoever is already there.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Using a cuboid where grasping requires the real shape.
- Neglecting the annotation of grasping surfaces.
- Annotating inaccessible objects at the bottom of a bulk pile.
- Ignoring transparent plastic film in the convention.
- Treating deformable objects as objects to register.
- Requiring an orientation on an object with rotational symmetry.
- Omitting the catalogue conformity check, the vertical’s most powerful.
- Costing by the number of objects without accounting for overlap.
- Accepting a registration proposal in dense bulk without verification.
- Not declaring the tolerance the gripper requires.
