The Cost of Box Annotation

An annotation quotation often fits on one line: a price, a unit, a volume. That simplicity hides five items that decide the real amount, and two quotations of comparable appearance regularly cover scopes with nothing in common. This article details that economics. It extends the article on video and tracking.

What the rate covers

Five items compose the price of bounding box work. The drawing itself, the only item the unit rate makes visible. The initial scoping, reference and first adjudications included. Training the operators in the reference finally adopted. Quality control as well as the rework it triggers. And the final delivery, conversion and documentation included. One important practical consequence follows. Four items out of five disappear purely behind the unit rate, which lets two quotations showing the same price per object cover services that are in reality very different.

What makes the unit price vary

Five parameters weigh on the cost of an object annotated in bounding box work. The number of objects per image, which dominates all the others by far. The size of the objects as well as the zoom it imposes. The proportion of objects masked or else ambiguous. The number of classes as well as their mutual proximity. And the level of control required on the final deliverable. One important observation follows for a bounding box project. The first parameter decides most of it, one single image holding a hundred objects costing far more than ten images holding ten each, which makes a per-image rate misleading as soon as density varies.

What the billing unit changes

Four units occur in bounding box quotations. The price per object annotated, which follows the real load most closely. The price per image, wholly insensitive to density. The price per hour, which shifts all the risk to the client. And the fixed price, which shifts it entirely to the provider’s side. One practical consequence follows. Those four units distribute the density risk differently, a fixed price committing the provider on a volume of work they have not yet seen, which explains why an honest fixed price builds in a margin the per-object price has no need to carry.

What scoping costs and returns

Four expenditures compose the phase preceding any production. Drafting the reference as well as its visual examples. The trial batch as well as the measurements it produces. The adjudications given on the limiting cases reported. And the full setting up of the control arrangement. One observation follows. Those four expenditures look high set against one single day of production and derisory set against the whole corpus, which makes it the most profitable item and the most often sacrificed.

What estimating a volume demands

Five measurements ground a defensible estimate. The average number of objects recorded on a sample taken at random. The spread of that number between the busiest images and the rest. The real time observed on an image judged representative. The proportion of ambiguous cases effectively met. And the rework rate observed after quality control. One important observation follows for a bounding box project. The second measurement matters as much as the first, an average density that is nonetheless acceptable being able to mask a few extreme images consuming a quite disproportionate share of the time, which makes the average alone insufficient for building a schedule.

What control represents

Five activities compose this item on a bounding box project. The double annotation bearing on a fraction of the corpus. The calculation as well as the reading of the indicators adopted. The targeted rereading of all the cases flagged. The rework of all the annotations set aside. And the production of the report finally handed to the client. One important practical consequence follows for a bounding box project. This item is easily reduced and rarely seen, a provider who lightens their control delivering a corpus of rigorously identical appearance at a lower price, which explains a large share of the gaps between competing quotations.

What the revision clause must provide for

Four points deserve writing down before production. The average density serving as the basis for the rate finally agreed. The precise gap beyond which the price is renegotiated. The treatment given to atypical batches spotted along the way. And the conduct to adopt if the reference changes mid-project. One important practical consequence follows. The first point protects both parties rather than one, a density below the agreed reference justifying a downward adjustment as much as the reverse, which makes this clause easier to have accepted than a one-sided protection.

What pre-annotation changes about cost

Four effects modify the project’s economics. Drawing time falls markedly on well contrasted objects. Checking time replaces part of the drawing time. The hardest cases stay entirely the operator’s responsibility. And quality control keeps rigorously the same weight. One important observation follows for a bounding box project. Those four effects reduce the volume without reducing the average difficulty, the easiest share of the work always disappearing first, which raises the unit cost of what remains and makes a gain announced in volume misleading about the budget.

Three errors proper to costing

Three bounding box defects belong to the quotation rather than to the work. A unit price compared without ever looking at what it covers. A density merely assumed rather than counted on a sample. And a scoping absent from the quotation then invoiced along the way. Those three defects are committed before the first image, they are discovered once the budget is already committed, and their common point is to bear on the scope rather than on the amount.

What the client must supply here

Four elements condition the accuracy of a costing. A sample of images taken at random rather than carefully chosen. The total volume envisaged as well as how it spreads over time. The class reference where one already exists on their side. And the quality level they expect, stated rather than assumed. One important practical consequence follows for a bounding box project. The first element decides everything else, a sample carefully chosen by the client showing clean and legible images whereas the real corpus holds far busier scenes, which produces a costing that is accurate for images that will never be handled.

What volume changes about the unit price

Four effects accompany an increase in the volume ordered. The initial scoping is spread across a far greater number of objects. Training the operators is amortised over a longer period. Throughput improves as the reference becomes familiar. And control tightens progressively without losing any effectiveness. One important observation follows for a bounding box project. Those four effects justify a real tapering rather than a commercial gesture, a decreasing rate then reflecting a genuinely observed fall in cost and not a concession granted, which makes that tapering defensible and lasting where a discount is not.

What this subject has that is particular

Four traits distinguish this question from the course’s others. It is settled well before the slightest image is handled. It commits both parties across whole months of production. It is discussed over one single figure that masks four items. And it is corrected only by renegotiating, never by working better. One important observation follows for a bounding box project. The last trait explains the importance of the pricing scope, an initial costing error being recovered by no improvement in production at all, which sets it apart from every other defect described across this course.

What budget monitoring must observe

Four indicators are followed during production. The number of objects handled set against the number of images delivered. The effective throughput compared with that of the trial batch. The rework rate observed after the control pass. And the proportion of cases reported for adjudication. One important practical consequence follows for a bounding box project. The last indicator announces drifts before they show in the figures, a rising flow of reported cases signalling either an incomplete reference or a batch different from the previous ones, which leaves time to act rather than merely to observe.

What the schedule changes about cost

Four effects accompany a tight deadline constraint. The number of operators to mobilise rises very appreciably. Dispersion between operators grows mechanically with their number. Control must intensify to offset that dispersion. And training is compressed at the direct expense of consistency. One important observation follows for a bounding box project. Those four effects accumulate unfavourably, a deadline halved costing in practice markedly more than double since it degrades consistency at the same time as it increases the means, which makes an extension of a few weeks often more economical than a discount obtained through negotiation.

What the contract must provide for here

Four clauses deserve writing down before the start. The reference density as well as the renegotiation threshold. The exact scope of the quality control included in the rate. The treatment of rework caused by a change of reference. And ownership of the corpus produced as well as of its documentation. One important practical consequence follows for a bounding box project. The third clause avoids the most frequent dispute, a reference modified mid-project causing the rework of annotations that were nonetheless compliant with the initial instruction, which raises the question of who pays for work correctly executed and then rendered obsolete.

What a low price most often hides

Four savings explain a large gap between two offers. A scoping reduced to a plain reading of the brief. Training brought down to a few quickly commented examples. A control bearing only on a very reduced fraction of the corpus. And a delivery devoid of documentation as of conversion checking. One important observation follows for a bounding box project. Those four savings produce a deliverable that passes acceptance without difficulty, the defects they let through showing themselves only at the moment the corpus is actually used, which explains why a price gap is badly judged at the moment it is observed.

What this chapter presupposes of the provider

Four practices distinguish a sustainable costing from an optimistic one. Counting first before announcing rather than the reverse. Refusing any fixed price bearing on a corpus never sampled. Detailing all the items even when the client does not ask. And announcing a tapering grounded in effects genuinely observed. One important practical consequence follows. The second practice sometimes costs a deal and saves several, a fixed price accepted blind turning into a clear loss as soon as density exceeds the implicit hypothesis, which makes refusal a commercial decision rather than excessive caution.

What the trial batch changes about costing

Four benefits follow from a paid trial before commitment. Real density comes to replace a density until then assumed. Observed throughput comes to replace a throughput merely estimated. Ambiguous cases are discovered before being invoiced as rework. And both parties discuss figures rather than impressions. One important observation follows for a bounding box project. That batch changes the nature of the commitment that follows, an estimate being able to become a firm price as soon as it rests on a real measurement, which benefits the client as much as the provider and makes a paid trial cheaper than a badly calibrated quotation.

What the provider brings here

Four contributions distinguish a serious bounding box quotation. A count carried out on a real sample before any costing. A full breakdown of the items rather than a unit price alone. A density clause written well before the start. And a control arrangement described with its exact coverage. One important practical consequence follows. The second contribution hurts the provider at first sight and serves them afterwards, a detailed quotation looking dearer than one fitting on a single line whereas it shifts the discussion towards the scope, which is the only tenable position against a cheaper and less complete offer.

Approaching a costing

Five questions scope the costing of a bounding box project. How many objects per image on average. The dominant item. Which billing unit applies. It distributes the risk. What level of control applies. It explains the gaps. Is scoping included in the rate. It is often missing. And can the density vary. A clause is required. Those five answers determine the comparability of two quotations. Asking them before choosing avoids comparing services that bear no resemblance to each other.

The question that frames the comparison

One question determines the comparability of two bounding box quotations. Do the two offers cover the same scope or the same hand movement. The same movement, drawing rectangles, is compared on unit price and is found everywhere at roughly the same rate. The same scope, scoping, control and delivery included, is compared on the arrangement and reveals gaps the price alone did not show. That question belongs to what is bought and not to what is invoiced, it is asked before choosing, and it separates a real saving from an apparent one.

Three decisions before costing

Three decisions commit the costing of a bounding box project. Counting the objects on a sample taken at random rather than carefully chosen. Adopting a billing unit that follows the real load most closely. And showing scoping as well as control as distinct lines. Those three decisions cost only half a day, they precede sending the quotation, and their absence produces a commitment whose content nobody knows.

Three checks on a quotation received

Three controls qualify a bounding box offer. The presence of scoping as a line distinct from that of drawing. The description of the control arrangement as well as of its coverage. And the existence of an explicit clause tied to object density. Those three controls are each asked in one question, they require no technical competence, and their absence indicates a quotation whose scope stays to be discovered during the project.

Three questions before committing

Three questions precede signing a bounding box quotation. On exactly which sample was the announced density counted. What fraction of the corpus does quality control genuinely cover. And what happens if density departs from the reference. Those three questions are asked in a few minutes, they require no technical competence, and their absence commits both parties to a scope neither of them has verified.

What this chapter teaches

One cross-cutting observation deserves closing this examination. An identical unit price does not designate the same service. Three findings compose it. Four items out of five disappear behind the rate shown. One image holding a hundred objects costs far more than ten holding ten. And a lightened control produces a corpus of identical appearance at a lower price. That finding extends the preceding chapters, the invisible bearing here on the scope rather than on the data.

What this chapter shares with the previous ones

Three findings recur from the start of this course. A deliverable of identical appearance can cover two realities. Scoping weighs more than speed of execution. And an isolated figure misleads as long as one ignores what it measures. One important practical consequence follows for a bounding box project. The third finding holds for price as it held for quality, an amount devoid of scope being as uninformative as a rate devoid of breakdown, which suggests that the same demand for detail serves the buyer and the producer alike.

What this chapter leaves to the next

A control arrangement is badly costed as long as one ignores what it measures. Three questions stay open in bounding box work. Which indicator genuinely says something about a corpus. Why an excellent score can accompany an unusable deliverable. And how to conduct an audit that teaches something. Those three questions belong to quality measurement, which constitutes the subject of the following chapter.

Why losing on price is sometimes the right outcome

One point about competing on cost belongs at the close of this chapter. A provider who quotes the full scope will lose deals to one who does not. Three reasons follow in bounding box work. The buyer compares the two totals before reading either breakdown. The cheaper offer looks like the same work at a better price. And nothing in the delivered corpus will later prove otherwise. One important practical consequence follows for a bounding box provider. Quoting the full scope anyway is the only sustainable position, since winning a deal by omitting the control item means either absorbing it unpaid or delivering what the price actually bought, and neither outcome survives a second project with the same client.

Common mistakes

These failures recur often enough that naming them is usually enough to avoid them.
  • Comparing two quotations on unit price alone.
  • Charging per image for a corpus whose density varies.
  • Omitting scoping from the costing and discovering it afterwards.
  • Accepting a fixed price on a volume never sampled.
  • Neglecting the control item when comparing offers.
  • Estimating a volume without having counted on a sample.
  • Ignoring the effect of object size on throughput.
  • Confusing a low price with superior productivity.
  • Omitting a revision clause tied to density.
  • Costing from the client’s demonstration images.

What to take away

The unit rate masks most of what composes the price. Three readings emerge. Four items out of five disappear behind the rate shown, which lets two quotations of comparable appearance cover very different services and makes comparison on price alone misleading. One image holding a hundred objects costs far more than ten images holding ten, which makes a per-image rate unsuitable as soon as density varies from one batch to the next. And a lightened control produces a corpus of identical appearance at a lower price, which explains a large share of the gaps between competing offers and shifts the comparison towards the arrangement rather than the amount. For quality measurement, the article on IoU and audits details the indicators. For drafting the reference, the article on annotation instructions sets it out. To explore delivery arrangements, supported formats and applicable control mechanisms, see our dedicated page on 2D and 3D annotation. And if you are preparing a detection project, let us discuss your need.
Tags

Découvrez nos articles