A three-dimensional corpus is not built by accumulating scenes. What determines its value is the number of distinct configurations it covers, and that quantity depends more on the acquisition geometry than on the volume of data produced.
This article sets out the method. It extends the article on density, occlusion and sparsity.
Why the number of scenes misleads
Four properties distinguish a 3D annotation corpus from an image corpus. Redundancy between nearby scenes is high, two neighbouring acquisitions describing the same space. Storage volume grows with no relation to the diversity contributed. Coverage depends on the acquisition geometry as much as on the number of stations. And a scene can contain hundreds of objects or a handful according to the context. One important practical consequence follows. Those four properties mean a corpus with many scenes can cover fewer configurations than a small well-designed one, which shifts a 3D annotation project’s issue from collection to selection.The dimensions to cover
Six axes define the coverage of a 3D annotation corpus. They multiply against one another. Object configurations, density, arrangement and proximity. Distances, one object having to appear both near and far. Acquisition conditions, equipment, density and number of stations. The materials present, notably those that deceive the sensor. Scene types, indoor, outdoor, open or cluttered. And rare classes, whose absence only appears once the system is in use. One important observation follows. The second axis is specific to this field, an object observed only at short range not preparing a system to recognise it sparse, which requires coverage by distance band rather than by mere presence of the class.Selecting rather than collecting
Four selection strategies apply to a 3D annotation corpus. Random drawing, which reflects the real distribution and over-represents the mundane. Selection by configuration, which retains the scenes containing the difficult arrangements. Selection by uncertainty, which retains what an existing model does not master. And selection by coverage, which completes the plan’s missing combinations. One practical consequence follows. Those four strategies combine rather than replace one another, the first supplying a representative base and the other three enriching it on what counts.Dividing into scenes
This decision structures the corpus. It is rarely examined in 3D annotation. A continuous survey must be divided into annotatable units. Four criteria guide the division of a 3D annotation corpus into scenes. The number of objects per scene, which determines a task’s duration and its parallelisation. Natural boundaries, a change of room or of station supplying cut points with no consequence. Homogeneity of density, a scene mixing near and far zones producing an uneven load. And self-contained comprehension, a scene having to hold enough context to be annotatable alone. One important observation follows for a 3D annotation project. The second criterion removes by construction the problem of objects cut by a task boundary, a difficulty that otherwise produces partial annotations on either side.Separating training from evaluation
One rule governs that separation. Breaching it produces flattering and false figures. Two scenes from the same survey share too much context to sit on either side. Four separation levels exist and their demands increase. By scene, insufficient where several scenes come from the same survey. By survey, the acceptable minimum for most uses. By site, correct as soon as the system must work elsewhere. And by campaign or by sensor, the most demanding, which measures what the system will do on new data. One important practical consequence follows. The first level is still practised and it produces a performance with no meaning, two scenes from the same survey sharing the same objects seen from neighbouring angles.The metadata to record
Seven pieces of information condition the exploitation of a 3D annotation corpus. They cost a few fields at capture. The site and the station, indispensable to a correct separation. The date and time, which permit reconstructing the conditions. The sensor used and its parameters. The sensor’s position for each scene. The calibration in a multi-sensor configuration, with its identifier. The treatment applied, subsampling and filtering notably. And the coverage mask, which distinguishes observed zones from unobserved ones. One observation follows for a 3D annotation project. The fourth is specific to this field and rarely recorded, without it becoming impossible to know which face of an object was measured and which was extrapolated.Sizing
Four factors determine the volume needed in 3D annotation. They do not reduce to a number of scenes. The number of configurations to cover, the product of the six axes adopted. The number of objects per class and per distance band. The internal variability of each class. And the difficulty of the task, a fine distinction requiring more examples. One practical consequence follows. The volume is reasoned in objects per class and per distance band rather than in scenes, a near object and a distant one of the same class constituting two different examples for a system.The rare classes
This difficulty dominates building a 3D annotation corpus. It is handled upstream. A rare object is not met by random acquisition. Three methods permit recovering them. Targeted acquisition, practicable where the locations are known. Deliberate staging, possible in a controlled environment and impossible elsewhere. And querying an existing model, whose uncertain detections locate candidates. One important observation follows. The second method distinguishes controlled verticals from open ones, a warehouse permitting the difficult configurations to be produced at will where a road requires waiting for them.The evaluation corpus in 3D
One distinction deserves stating since this corpus does not follow the same rules. Four differences characterise this corpus in 3D annotation. Exhaustiveness becomes imperative, an omitted object counting as a false detection by the system evaluated. The sparsity threshold does not apply, which forces annotating objects the training corpus discarded. Stratification by distance must be deliberate, a set dominated by near objects measuring a performance with no meaning. And stability over time conditions comparison between two versions. One important practical consequence follows for a 3D annotation project. The first two differences make this corpus disproportionately expensive per scene, which justifies sizing it in scenes and not as a proportion of the training corpus.Progressive construction
An organisation in batches is better than a mass acquisition. A corpus built in one block fixes conventions before they are tested. Four advantages accompany a progressive approach in 3D annotation. Conventions are corrected between two batches rather than after the whole. Throughput and cost are measured early and projected. A first model permits pre-annotating the following batches. And the gaps that model reveals direct the next batch’s content. One practical consequence follows. The fourth advantage is the most important, it turns construction into a loop rather than a straight line and concentrates effort on what is actually missing.Organising an acquisition campaign
Five logistical decisions condition the success of a 3D annotation campaign. Scouting the sites, checking that they cover the target configurations and that access is authorised. The station plan per site, which determines the residual occlusion more than the choice of sensor does. The acquisition protocol, height, density and duration per station, recorded so as to be reproducible. Metadata capture, organised to be filled in on site and not reconstituted afterwards. And checking the clouds the same day, a missed or badly registered station being correctable while on site. One important practical consequence follows. The second point is what distinguishes a well-designed campaign, an additional station costing a few minutes on site and removing hours of extrapolation in annotation.Exploiting an existing collection
One frequent situation deserves treating, a client often holding surveys accumulated before any project. Four verifications establish what those surveys permit in 3D annotation. Configuration coverage, an exercise that almost always reveals a concentration on a few sites. The presence of sensor and station metadata, whose absence strongly limits exploitation. Equipment homogeneity, a collection built over several years mixing sensors of different characteristics. And representativeness, a collection built for demonstrations systematically presenting favourable conditions. One practical consequence follows for a 3D annotation project. The third verification produces the most problematic finding, a mix of sensors making the corpus heterogeneous in density and in artefacts, which requires keeping the sensor as metadata rather than treating the whole as one corpus.The coverage table
A simple document steers this dimension and it fits on one page. It crosses the target configurations with what was actually acquired. Five columns compose that table in 3D annotation. The configuration, expressed in observable categories rather than continuous values. The distance band, an additional dimension specific to this field. The number of scenes acquired for that combination. The number of annotated objects arising from them. And the performance measured, filled in after the first evaluation. One practical consequence follows for a 3D annotation project. The second column doubles the table’s row count and it almost always reveals an imbalance, distant objects being under-represented while they constitute the hardest part.What storage imposes
Four specific constraints weigh on organising a 3D annotation corpus. The volume of the source clouds, several orders of magnitude above an image corpus of equivalent coverage. The choice between full cloud and subsampled cloud, the second lightening the work and forbidding certain re-exploitations. Transfer, whose duration becomes a scheduling constraint rather than a technical detail. And retention of the clouds after delivery, without which no re-exploitation is possible. One observation follows. The second constraint deserves an explicit trade-off, a subsampling suited to a cuboid task making a later segmentation impossible on the same corpus, which argues for keeping the full cloud and delivering the lightened version.What not to do
Four practices degrade a 3D annotation corpus. They nonetheless appear reasonable. Discarding degraded scenes, which will be present in operation. Multiplying stations on one site rather than seeking new ones. Dividing at constant volume with no regard for natural boundaries. And keeping only the annotations while deleting the source clouds. One observation follows. The second practice is the most frequent, an accessible site easily producing many stations whose marginal contribution falls rapidly.The pilot batch as a test of the plan
Four verifications apply to the first batch and they concern the plan rather than the corpus. Object density per scene and per distance band, which replaces the volume estimate with a measurement. The feasibility of the division adopted, a scene too crowded or too sparse being correctable at this stage. The real coverage of one acquisition day, often lower than planned. And the completeness of the metadata, a field forgotten at acquisition not being recoverable. One observation follows for a 3D annotation project. The third verification produces the most useful finding, a campaign plan regularly assuming more sites per day than logistics permit, which is corrected before the calendar is consumed.What the documentation must contain
Six elements accompany a 3D annotation delivery. Their absence limits reuse. The sampling plan actually carried out. The table of counts per class and per distance band. The annotation conventions, thresholds and extrapolation rule included. The separation criterion adopted between training and evaluation. The quality report, with the values per distance band. And the list of configurations not covered, which explicitly bounds the domain of use. One important practical consequence follows. The last element is the rarest and the most useful, declaring what the corpus does not cover turning an invisible limit into a known one.What the corpus will be worth afterwards
Four properties determine whether a 3D corpus behaves as an asset. Retention of the source clouds, an annotation separated from its data losing all value. Declaration of the frame, threshold and extrapolation conventions, without which it is combinable with none. Completeness of the metadata, sensor and station included, which permits selecting a subset by condition. And the coverage mask, which distinguishes observed and empty zones from unobserved ones. One observation follows for a 3D annotation project. The third property takes on particular importance here, a corpus from which the scenes of a given sensor can be extracted staying usable when the equipment changes, where an undifferentiated corpus becomes unusable in its entirety.What the second campaign brings
One observation concerns what a first corpus structurally cannot contain. A campaign run over a short period captures one hardware configuration and one arrangement of the sites. Three dimensions escape it. Equipment change, a replaced sensor producing a different density and different artefacts. The evolution of the sites, layout and occupancy changing over months. And variation in acquisition practice, a different operator not placing their stations the same way. One practical consequence follows for a 3D annotation project. A second spaced campaign adds a dimension no increase in the first campaign’s volume supplies, which makes it a qualitatively different contribution rather than a supplement.Approaching the construction of a 3D corpus
Five questions scope this phase of a 3D annotation project. They are asked before any acquisition. How many distinct configurations does the scope contain. That product determines the coverage needed. Is coverage by distance band planned. Its absence produces an unbalanced corpus. What separation criterion will be adopted. That answer conditions the validity of any measurement. Will the sensor’s position be recorded. Its absence deprives the annotation of decisive information. And are the rare classes accessible. Their absence makes the project infeasible as framed. Those five answers determine the volume and the validity of the result. Asking them before collecting avoids a corpus that is voluminous and poor in diversity.The question that frames the collection
One question replaces the debate about volume and it is asked in one sentence. In how many distinct configurations will the system operate in production. An answer limited to a few stable sites leads to a modest corpus and to effort going into covering distances rather than places. An answer in dozens of configurations leads to a campaign planned across many sites, the number of scenes per site mattering little. That question concerns the deployment and not the collection, it is asked of the end user rather than of the technical buyer, and it produces a plan where a discussion about scene counts produces a figure with no foundation.Three decisions before the first acquisition
Three decisions determine a 3D annotation corpus’s validity. Establishing the list of configurations to cover by crossing scene types with distance bands, which replaces an objective in scenes with an objective in diversity. Fixing the separation criterion between training and evaluation, survey or site rather than scene. And settling the metadata to record, the sensor’s position first among them. Those three decisions cost one meeting, they precede the first station, and their absence produces a corpus whose announced performance cannot be verified.What this chapter teaches
One cross-cutting observation deserves closing this examination. A 3D corpus’s value lies in its diversity of configurations and not in its volume. Three findings compose it. Distance constitutes a coverage axis in its own right, a near object and the same object at range being two distinct examples for a system. The acquisition geometry determines what is observable as much as the sensor does, which makes the number of stations a coverage variable and not a volume one. And the sensor’s position is a metadata with no equivalent in image annotation, without which the distinction between measurement and extrapolation is lost. That finding matches the one the video cluster established: what determines a corpus’s value is what it deliberately covers, and that coverage is designed before acquisition rather than observed after.The one axis clients forget
One omission recurs in 3D corpus plans and it is worth naming separately. Clients plan coverage in places and forget to plan it in distances. Three reasons explain that in 3D annotation. Distance is not a property of the scene, it is a property of each object within it, so it does not appear on a site list. The near objects are the ones people look at when reviewing a cloud, which makes the imbalance invisible on inspection. And the corpus that results performs well in demonstration, since demonstrations use the conditions the corpus covers. One practical consequence follows for a 3D annotation provider. Producing the distance-band count on the client’s first batch is the cheapest way to surface this, since the imbalance is arithmetic rather than arguable and it changes the acquisition plan rather than the annotation method.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Sizing a corpus in scenes rather than in objects per distance band.
- Multiplying stations on an accessible site instead of seeking new ones.
- Separating training from evaluation by scene rather than by survey or site.
- Omitting the sensor’s position from the metadata.
- Dividing at constant volume with no regard for natural boundaries.
- Discarding degraded scenes that will be present in operation.
- Deleting the source clouds after extracting the annotations.
- Covering a class without covering its different observation distances.
- Acquiring in bulk before testing the conventions.
- Delivering without the list of configurations not covered.
