The third dimension changes the nature of annotation. An object no longer has an outline in an image, it occupies a volume in a space, and that shift alters the representations, the tools, the difficulties and the cost.This article sets out the panorama of 3D annotation and serves as the entry point to this cluster.
What 3D data covers
Four families of data coexist and their confusion produces quotations unrelated to the real work.The point cloud, a set of coordinates produced by a range sensor, with no surface and no continuity.The mesh, a surface reconstructed from points, which restores a continuity and introduces an interpretation.The voxel, a regular division of space into cells, used in volumetric imaging and in occupancy mapping.And the multimodal scene, where a cloud coexists with one or several calibrated images.One important practical consequence follows. Those four families call for neither the same tools nor the same conventions, which makes the format question prior to any discussion of scope in 3D annotation.What distinguishes 3D from images
Six properties separate this work from image annotation.The data is sparse, a distant object being represented by only a few points.There is no colour and no texture, unless an image supplies them.Density varies strongly with distance from the sensor, which makes one object more or less describable according to its position.Occlusion is absolute, what lies behind an obstacle not existing in the data.The viewpoint is free, the annotator being able to move around the scene.And measurements are metric, the distances read being real distances.One important observation follows. The last property is the principal advantage of 3D annotation, a dimension expressed in metres permitting plausibility checks no image permits.The 3D annotation tasks
Five representations occur in order of increasing cost.The point of interest, which locates without describing extent.The cuboid, an oriented box giving position, dimensions and orientation.The ground footprint polygon, used in mapping and in planning.Point-wise segmentation, which assigns a class to every point in the cloud.And instance segmentation, which adds the object’s identity to the class and separates two adjacent objects of the same type.One practical consequence follows for a 3D annotation project. Moving from the cuboid to point-wise segmentation multiplies the load by a substantial ratio, which makes that choice the scope decision with the heaviest consequences.Where the data comes from
Four sources feed a 3D annotation project and their data differs.LiDAR, which measures a distance by time of flight and produces a precise and sparse cloud.Photogrammetry, which reconstructs a cloud from images and produces dense, coloured but noisy data.Depth cameras, which give a dense distance map over a limited range.And radar, which produces few points and works in conditions where the others fail, rain and fog notably.One practical consequence follows. Those four sources produce different artefacts, which makes a project’s conventions non-transferable from one sensor to another without verification.The cuboid and what it demands
One representation dominates usage and its economics deserve setting out.The cuboid describes an object by nine values, position, dimensions and orientation.Three advantages characterise it.It is fast to place, a box being adjusted in a few gestures.It directly supplies the quantities a perception system exploits.And it permits automatic checks, an aberrant dimension being detectable with no human examination.Three limits accompany it.It badly describes non-convex or articulated objects.It includes empty space around the object, which hinders manipulation tasks.And it presupposes a definable orientation, which is not always the case.Orientation, a difficulty specific to 3D
One quantity has no equivalent in image annotation.A cuboid must indicate which way the object is facing.Three difficulties follow.Front-back ambiguity, a vehicle seen from the side not revealing its direction where the data is too sparse.The reference convention, an object’s principal axis having to be defined by a rule rather than left to intuition.And the precision expected, an error of a few degrees passing unnoticed visually and strongly affecting a trajectory prediction.One important observation follows for a 3D annotation project. The first difficulty justifies a confidence attribute on orientation, failing which a guessed value becomes indistinguishable from an observed one.What density changes
One property governs feasibility and it is quantifiable.The number of points on an object falls with the square of the distance.Four consequences follow.A distant object becomes unclassifiable, a few points not carrying enough information.Dimension becomes an estimate, an unsampled face forcing extrapolation.Disagreement between annotators grows strongly with distance.And the cost per object rises while its training value falls.One important observation follows. Those four consequences justify fixing a maximum annotation distance, a figure that saves work and removes a major source of inconsistency in 3D annotation.What occlusion becomes in 3D
One difference from images deserves emphasis.What lies behind an obstacle is not hidden, it is absent from the data.Three consequences follow.A partially observed object must be dimensioned by extrapolation rather than by measurement.The extrapolation convention determines whether two annotators produce comparable dimensions.And an object entirely behind an obstacle produces no point at all, which makes its annotation impossible rather than difficult.One practical consequence follows for a 3D annotation project. The second consequence requires writing the rule explicitly, an extrapolation left to judgement producing a dispersion of dimensions that no check can subsequently reduce.What fusion with images brings
A frequent configuration combines two sources and it deserves situating.A camera calibrated with the range sensor supplies the colour and texture the cloud lacks.Three contributions follow.Classification becomes more reliable, an object ambiguous in geometry often being obvious in an image.Distant objects stay identifiable, the image keeping a resolution the cloud has lost.And verification is easier, a checker finding in the image what they cannot distinguish among the points.One practical consequence follows. This configuration adds a requirement, calibration and synchronisation between sensors becoming a condition of the corpus’s validity rather than a technical detail.The conventions to write
Six decisions cover most of the difficulties in 3D annotation and they are written before production.The minimum number of points below which an object is not annotated.The maximum distance covered.The extrapolation rule for unobserved faces.The definition of the principal axis for each class.The treatment of points reflected by glass or polished metal.And the attribute permitting an uncertain decision to be marked.One important observation follows. The first two decisions are the most economical, each removing a large share of the objects whose annotation costs the most and contributes the least.The fields of application
Five sectors regularly require this type of work.Autonomous driving and driver assistance, the field’s principal historical driver.Robotics and logistics, where manipulation requires a volumetric description.Building and infrastructure, where the survey feeds a digital model.Agriculture and forestry, where plant structure is measured in volume.And medicine, where volumetric imaging follows a related logic with its own constraints.One observation follows for a 3D annotation provider. The second and third sectors are growing with more affordable volumes than the first, which makes them accessible without requiring the scale of the major automotive programmes.What 3D annotation costs
Five factors determine the load of a 3D annotation project.The geometry adopted, the dominant factor separating the cuboid from point-wise segmentation.The number of objects per scene, the real basis of sizing.The density of the cloud, a sparse cloud slowing the decision as much as a dense one slows the tracing.The number of attributes, orientation and state increasing the time per object.And the requirement for temporal continuity where the scenes form a sequence.One observation follows. The first factor dominates the others, which makes scoping the geometry more decisive for the budget than any production optimisation.What the tooling must permit
Five capabilities condition the quality of the work in 3D annotation.Navigating freely around the scene, an orientation not being judged from a single viewpoint.Switching between perspective and orthogonal views, the latter permitting a dimension to be adjusted precisely.Displaying the cloud and the projected image simultaneously where a camera is available.Adjusting one face of the cuboid independently of the others, an extrapolation bearing only on the unobserved face.And marking an uncertain decision without interrupting the work.One practical consequence follows. The second capability is the most decisive for throughput, a top view permitting the ground footprint and the orientation to be placed in one gesture where a perspective view demands several successive adjustments.What quality control exploits
Four verifications are specific to this field and they are more powerful than in images.Dimensional plausibility, an object of a given class having a bounded physical size.Consistency with the ground, an object floating or sunk signalling an error.Counting the points contained, a nearly empty box signalling an annotation with no support.And non-intersection, two solid objects not being able to occupy the same volume.One important practical consequence follows. Those four checks rest on metric quantities and are programmed once, which makes automatic control markedly more effective in 3D annotation than in image annotation.The exchange formats
Four format decisions commit the exploitation of a 3D annotation corpus.The representation of the cloud itself, common formats differing on the presence of intensity, colour and per-point timestamp.The frame convention, the origin and axis orientation having to be declared and not assumed.The representation of cuboids, the orientation parameters admitting several incompatible conventions.And the link to the associated images, with the calibration parameters permitting projection.One important observation follows. The second decision produces the most frequent and most expensive error, two corpora using different axis conventions appearing compatible until a model trained on one fails entirely on the other.What does not resolve
Four limits bound what 3D annotation can establish.What has not been sampled does not exist, occlusion removing the information rather than masking it.An object too distant stays unclassifiable whatever effort is made.A reflective or transparent surface produces erroneous or absent points.And two objects in contact can be inseparable in geometry alone.That first limit deserves emphasis. It means a requirement for complete dimensions on an object seen from a single face demands an extrapolation, which must be declared as a convention rather than presented as a measurement.The pilot batch of a 3D project
Four scenes compose a representative pilot batch in 3D annotation.A scene of ordinary density, which supplies the reference throughput.A scene containing distant objects, which tests the distance threshold adopted.A scene with mutual occlusions, which tests the extrapolation convention for unobserved faces.And a scene containing a problematic surface, glazing or polished metal, which verifies what to do with aberrant points.One practical consequence follows. Those four scenes represent a few minutes of work, they produce most of the disagreements that would otherwise surface in production, and their absence explains most of the gaps between an announced throughput and an observed one.Comparing two proposals
Six questions make two 3D annotation offers comparable.What geometry per class, cuboid, ground polygon or point-wise segmentation.What maximum annotation distance is adopted.Is orientation part of the scope and with what tolerance.Must unobserved faces be extrapolated or left open.Does the plausibility check feature as a distinct line.And is the evaluation corpus included.One observation follows. The second question is the one where answers diverge most between providers, a generous maximum distance inflating the object volume substantially without improving a system that will not exploit them.Approaching a 3D project
Five questions scope this type of project.What geometry is expected. That answer determines the cost by a substantial ratio.Which sensor produces the data. That answer determines the artefacts and the conventions.Is a calibrated image available. Its presence changes the feasibility of classification.What maximum distance must be covered. That answer bounds the usable share of each scene.And do the scenes form a sequence. A positive answer adds the continuity requirement.Those five answers determine the load and the feasibility. Asking them before starting avoids committing to a precision the data does not carry.The question that frames a 3D project
One question determines the cost before any other consideration.Does the downstream system need to know where an object is or what shape it has.A position-oriented answer, detection and obstacle tracking, is satisfied by a cuboid and places the project in the most economical regime.A shape-oriented answer, manipulation or volume measurement, requires point-wise segmentation and multiplies the load.That question is asked in one sentence, it belongs to the use and not to the technique, and it produces the largest budget variation in a 3D annotation project.Three decisions before the first batch
Three decisions commit the whole spend of a 3D annotation project.Choosing the geometry per class, which separates an economical project from one several times heavier.Fixing the maximum annotation distance, which bounds the object volume without reducing usefulness.And writing the extrapolation convention for unobserved faces, which makes dimensions comparable between annotators.Those three decisions cost one meeting, they precede the first scene, and their absence produces a corpus whose dimensions are neither reproducible nor verifiable.What this cluster will examine
Fourteen chapters extend this panorama and they fall into four groups.The representations, cuboids and cloud segmentation, which set out what each formulation demands.The verticals, robotics and logistics, forestry and agriculture, building and infrastructure, where conventions differ by use.The questions of method, sensors, fusion, tooling, cost, quality, difficulties, corpus building and pre-annotation. The chapter on the cost of 3D annotation details the budget lines.And a forward reading, which separates what is under way from what remains uncertain.One observation follows. Each of those chapters examines a facet of the shift set out here, metric measurement offering unprecedented checks while the absence of visual cues makes the class decision harder than in image annotation.What this chapter teaches
One cross-cutting observation opens this series.The third dimension supplies a measurement and removes cues.Three findings compose it.The quantities are metric, which permits plausibility checks no image permits.The data is sparse and uneven, which makes feasibility depend on distance rather than on effort.And the visual cues are absent, which moves the difficulty from the precision of the outline towards the class decision.That finding orients the fourteen chapters that follow, each examining one facet of that shift.Why this field rewards a provider differently
One observation belongs at the close of a panorama, since it concerns position rather than method.3D annotation has a higher barrier to entry than image work and a more durable one.Three properties of 3D annotation produce that.The tooling is heavier, a point cloud platform demanding more than an image tool and fewer providers having invested in one.The competence transfers poorly, an annotator trained on images needing genuine retraining to judge an orientation or an extrapolated face.And the conventions are less standardised, which means an established convention set is an asset rather than a formality.One practical consequence follows for a 3D annotation provider. Those three properties make the field slower to enter and slower to be displaced from, which changes the calculation on whether to invest in it: the payback is longer and the position, once held, is harder for a competitor to take.One qualification keeps this honest. A higher barrier protects a competent provider and traps an incompetent one, since the same difficulty that keeps competitors out will produce failed projects for anyone entering without the tooling and the conventions.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Requesting point-wise segmentation where a cuboid would have sufficed.
- Transposing image annotation conventions without verification.
- Not fixing a maximum annotation distance.
- Neglecting the orientation convention and its ambiguity.
- Assuming an object seen from one face can be dimensioned without extrapolation.
- Ignoring calibration between sensors in a multimodal project.
- Comparing corpora produced with different sensors without verification.
- Omitting dimensional plausibility checks, cheap and effective.
- Costing per scene without having counted the objects.
- Treating a reflective surface as ordinary data.
