What Does Video Annotation Cost

The cost of a video project does not follow from its duration. A minute can cost a few euros or several hundred depending on factors that do not appear in a specification, and that dispersion explains most of the gaps between quotation and reality.This article breaks down that cost and sets out what reduces it. It extends the article on temporal segmentation.

Why duration says nothing

Four reasons make this unit misleading in video annotation.Object density varies by a considerable factor between a country road and an urban junction.The annotation rate is a project decision and not a property of the video.The geometry required changes the cost per object by a substantial ratio.And part of the duration sometimes contains nothing to annotate.One important practical consequence follows. Those four reasons mean two video annotation projects of the same duration can differ by an order of magnitude, which makes costing by the minute defensible only after those parameters have been measured.

The right unit of measure

One unit makes projects comparable.It is computed on a sample.The number of object-frames annotated, the product of the number of objects and the rate, replaces duration.Three advantages follow.It makes sequences of very different densities comparable.It is measured directly on a pilot batch rather than estimated.And it converts into working time through an observed rate, which produces a verifiable costing.One observation follows. This unit does not suit temporal segmentation, where the occurrence replaces the object-frame, which requires treating the two task types separately in one quotation.

The seven lines of a budget

Separating them conditions any realistic video annotation estimate.Preparing the sequences, extraction, cutting and anonymisation where applicable.Building the conventions, reference and pilot batch.Annotation itself, the dominant line by volume.Quality control, distinct from annotation and regularly absorbed into it.The evaluation corpus, whose requirement exceeds that of the training corpus.Documentation, written conventions and quality report.And infrastructure, storage and processing environment, a negligible line for still images and a significant one for video.

What the rate changes

This decision dominates every other lever in a video annotation project.It is taken at scoping.Annotating one frame in two rather than all of them halves the volume.Three considerations guide that choice.Object speed, rapid movement requiring a higher rate to stay usable.The downstream use, a system working at a given rate having no need of a finer corpus.And the interpolation strategy, which permits delivering one position per frame while observing only a fraction.One important practical consequence follows. This decision produces the largest cost variation in a video annotation project, and it is frequently taken by default at the native rate when no use requires it.

What geometry changes

This second factor multiplies the cost per object.It is decided independently of the rate.Four levels occur in video annotation, in order of increasing cost.The point or the count, which requires a coarse localisation.The bounding box, the standard for most projects.The polygon, which follows the outline and costs several times a box.And the precise mask, whose cost over a sequence becomes prohibitive without assistance.One important observation follows. Moving from the second to the third level rarely justifies itself in video, interpolation working poorly on outlines and the precision gain being rarely exploited by the downstream use.

What attributes change

This third factor is regularly omitted from a video annotation costing.Every variable attribute must be filled across a track’s whole duration.Three attribute categories are distinguishable.Constant attributes, class or colour, filled once per track and of negligible cost.Variable attributes, occlusion or orientation, whose cost follows the track’s duration.And interpretive attributes, intent or state, which require judgement and slow the work considerably.One practical consequence follows for a video annotation project. A single variable attribute can double a project’s load, which justifies enumerating them explicitly at scoping rather than accepting them as a detail.

What tracking costs

This video-specific requirement deserves separate costing.Maintaining an identity costs more than locating an object.Three surcharges are added.The track-by-track method, which requires traversing the sequence as many times as there are objects.Handling occlusions, each interruption requiring a linking decision.And control on sequences, more expensive than control on isolated frames.One observation follows. That surcharge justifies the question raised in the tracking article, the downstream use frequently requiring a count that identity does not improve, in which case this line disappears entirely.

The pilot batch as the only estimate

This method turns those factors into a defensible figure.A few minutes of real sequences annotated supply what no estimate produces.Four decisive pieces of information follow.The throughput observed per scene type, which replaces any assumption.The number of objects per frame and per class, the real basis of sizing.Disagreement between annotators, which bounds what can be promised.And the feasibility of the conventions, including the edge cases revealed.One important practical consequence follows. The gap between a sparsely populated scene and a dense one reaches a factor nobody anticipates correctly, which makes a costing without a pilot arbitrary rather than imprecise in video annotation.

Relative orders of magnitude

Four ratios help at scoping without quoting a rate and they are verified on a pilot batch.A dense scene costs a multiple of a sparsely populated one, at identical duration and rate.Tracking with identity costs several times frame-by-frame detection, on the same content.A polygon costs several times a bounding box, and that gap widens further with interpolation since it works poorly on outlines.And a variable attribute adds a load proportional to track duration, independent of the number of objects.One observation follows for a video annotation project. Those four ratios are measured on a single pilot batch and they supply the client with a conversion grid they apply themselves to their trade-offs, which shifts the discussion from price to scope.

What quality control costs

This line is the most frequently absorbed into annotation and it deserves one of its own.Four elements compose it in video.Double annotation of a sample, whose rate determines the surcharge.Automatic checks on sequences, position jumps and track durations, whose cost is in setup and not in production.Human verification of the alerts those checks produce.And periodic drift measurement on a constant sample.One observation follows for a video annotation project. The second element costs a few hours once and then runs on every batch, which makes it the best return in this line and justifies planning it from the first batch rather than adding it after a contested delivery.

What storage costs

This video annotation line deserves isolating.Data volumes exceed those of an image project by several orders of magnitude.Four elements compose it in practice.Storing the source sequences, often kept uncompressed to preserve quality.Transfer, whose duration becomes a scheduling constraint on large volumes.The working infrastructure, a video annotation platform demanding more than an image tool.And retention after delivery, whose duration must be settled contractually.One observation follows. The second element is the most underestimated, transferring several terabytes potentially delaying a project start by several days, a constraint to build into the schedule rather than discover.

What genuinely reduces cost

Five levers work in video annotation.They act before production.Reducing the rate, the most powerful lever and the most often neglected.Cutting the sequences to exclude what contains nothing, which removes work rather than accelerating it.Simplifying the geometry where the use does not demand better.Removing variable attributes the downstream decision does not exploit.And abandoning identity where a count suffices.One practical consequence follows. Those five levers act on the scope and degrade nothing, whereas attention naturally goes to accelerating production, which produces a smaller gain and a greater risk.

What assistance changes in the budget

One lever modifies a project’s economics and its effect is real and overestimated.Interpolation and assisted tracking reduce the volume produced manually.Three effects compound.Throughput rises on regular segments, which represent most of the duration.Effort concentrates on crossings and occlusions, which are the decisive moments.And the marginal cost of an additional batch falls once the conventions are established.Three caveats accompany it.Identifying the break points is not counted in the announced gain.Verification stays necessary and it bears on the whole.And validation bias requires targeted checks that consume part of the saving.One practical consequence follows. The gain stays substantial and it sits markedly below the raw ratio, a gap that should feature in a costing rather than produce an untenable promise.

The false economies

Four apparent reductions cost more than they save.Skipping the pilot batch, which saves a day and produces a costing whose gap is paid in production.Absorbing quality control into annotation, which hides a line rather than removing it.Discarding difficult sequences, which accelerates production and produces a system that fails in real conditions.And omitting documentation, which saves hours and makes the corpus unextendable.Those four reductions share a common structure. Each removes a cost visible now and creates an invisible one later, a configuration the preceding series identified as the most deceptive.

The line nobody budgets

One cost appears after the first delivery and it rarely features in a proposal.Three elements compose it in video annotation.Enrichment on the failures observed, which presupposes a recording arrangement decided upfront.Extension to new conditions, a moved camera or an additional site constituting a context the initial corpus does not cover.And convention rework, an evolution of the downstream use invalidating a rule applied across a whole corpus.One commercial consequence follows. A proposal presenting only the build understates the client’s real spend, and separating build from operation prevents a competitor appearing cheaper with an incomplete figure.

The spending calendar

A cash-flow observation completes the video annotation estimate.Preparation and transfer precede any production and tie up time without producing a deliverable.Conventions and the pilot batch occupy the first days and determine everything after.Annotation spreads out and is a regular charge.Control and documentation arrive at the end of production, when the budget is already committed.And operation arrives after the project’s formal closure.One practical consequence follows. The two lines determining whether a corpus will be defensible and reusable fall at the moment a budget has least flexibility, which explains why they are precisely the ones compressed.

Billing models

Four modes coexist and their choice redistributes risk.Billing per object-frame, the most faithful to the real work and the least legible to a client.Billing per minute, legible and strongly exposing the provider to density variability.Billing by time, suited to work whose difficulty is poorly known.And a fixed price, which gives the client full visibility and transfers risk to the provider.One recommendation follows for a video annotation project. A billed pilot batch followed by a fixed price combines the advantages, the pilot removing the density uncertainty that made the fixed price risky.

Comparing two proposals

Six questions make two video annotation offers comparable.What annotation rate is adopted.What geometry per class.Which attributes are included and which are variable.Is persistent identity part of the scope.Does quality control feature as a distinct line.And is the evaluation corpus included.One observation follows. Two proposals showing very different per-minute prices frequently describe different scopes rather than different efficiencies, which makes those six questions more informative than the price itself.

Signals of a fragile costing

Five signs indicate an estimate will not hold and they are visible before commitment.The scope is expressed in hours of video with no mention of density or rate.No pilot batch has been run on the client’s real sequences.The attributes are neither enumerated nor distinguished between constant and variable.Quality control does not appear as a distinct line.And the evaluation corpus does not feature in the scope.One observation follows. Those five signs are verified by reading a proposal, they require no technical competence, and their presence predicts an overrun better than any analysis of the headline price.

Presenting a budget to a client

Four framings allow a budget dominated by human work to be accepted.Show the calculation in object-frames, whose result explains a volume duration was masking.Set out what the pilot batch revealed on their own sequences, which convinces better than a rate card.Separate build from operation, which avoids the distorted comparison with an incomplete offering.And propose a costed reduction in scope, rate or attributes, which demonstrates that the objective is a usable result rather than a billed volume.One observation follows. The first framing is the most effective in video, a client frequently discovering that their sequences contain ten times more objects than they imagined, a finding that makes the budget obvious where a justification by hourly rate would stay arguable.

Three questions that halve the budget

Three questions asked in one meeting frequently reduce the cost by half.At what rate will the system work in production. An answer below the native rate removes half the volume or more.Does the result serve to count or to track. A count-oriented answer removes the identity line entirely.Which attributes does the downstream decision actually exploit. An attribute filled and never read is pure load.Those three questions concern the use and not the technique, they are asked before any price discussion, and a client answers them readily once they understand what each answer costs.

What the corpus is worth afterwards

Three properties determine whether a video corpus behaves as an asset or as a consumable.Documentation of the conventions, notably the rate, the occlusion policy and the boundary tolerance, without which a corpus is not combinable with another.Retention of the source sequences, an annotation separated from its video losing all value.And the stability of the conventions over time, a corpus whose rules drifted during production being hard to reuse.Those three properties cost little at the time of building and cannot be added afterwards, which places them in the same category as the scoping decisions set out above.

Approaching the costing of a video project

Five questions produce a defensible estimate in video annotation.What rate does the downstream use actually require. That answer is the principal lever.What object density do the sequences present. That answer determines the volume.Is identity necessary. A negative answer removes an entire line.Which variable attributes are required. Each multiplies the load.And is a pilot batch accepted. Its refusal makes any fixed price imprudent.Those five answers determine the volume and the structure of the offering. Asking them before costing avoids committing to a scope whose extent surfaces in production.

What this chapter teaches

One cross-cutting observation deserves closing this examination.The cost of a video project is determined by scope decisions rather than by the quantity of material.Three findings compose it.Rate, geometry and attributes multiply against one another and determine the budget more than the duration of the sequences.Those three decisions are taken in one meeting and commit the whole spend.And they belong to the downstream use rather than to annotation technique, which places a provider’s value at scoping as much as in production.That finding matches the one the preceding series established in very different domains: an interlocutor able to bring a scope back to what the use requires spares their client a spend out of all proportion to the price of their engagement.

The calculation to do in the first meeting

One arithmetic exercise settles more than any discussion of rates, and it takes minutes.Take one representative frame, count the objects in it, multiply by the rate under consideration, multiply by the duration.Three things happen when that number appears.The client sees why an hour of footage is not a small job, since the figure is usually far larger than the mental model they arrived with.The rate stops being a technical detail and becomes the visible lever it is, halving it halving the number on the board.And the conversation moves from what the provider charges to what the project contains, which is where it should have started.One practical consequence follows for a video annotation provider. This calculation can be done on the client’s own footage in front of them, it requires no tooling, and it converts a price objection into a scope discussion more reliably than any explanation of methodology.

Common mistakes

These failures recur often enough that naming them is usually enough to avoid them.
  • Costing by the minute without having measured object density.
  • Adopting the native rate without examining what the use requires.
  • Accepting variable attributes without costing their load.
  • Requiring a fine geometry where a box would have sufficed.
  • Including persistent identity where a count would have sufficed.
  • Absorbing quality control into the annotation line.
  • Omitting the evaluation corpus from the costed scope.
  • Neglecting data transfer in the schedule.
  • Comparing two proposals on the per-minute price.
  • Committing to a fixed price with no prior pilot batch.

What to take away

A video’s duration does not determine the cost of annotating it, the object-frame being the relevant unit and being measured on a pilot batch.Three readings emerge. The annotation rate produces the largest cost variation in a project and it is frequently adopted by default at the native value when no downstream use requires it. A single variable attribute can double the load, which justifies enumerating them explicitly at scoping rather than accepting them as a detail. And two proposals showing very different per-minute prices most often describe different scopes, which makes six scoping questions more informative than the headline price.For the tooling that conditions part of these costs, the article on video annotation tools details the options. For the line this chapter isolates, the article on quality control in video annotation sets out the method.To explore delivery arrangements, supported formats and applicable control mechanisms, see our dedicated page on 2D and 3D annotation. And if you want a defensible estimate for a video corpus, let us discuss your project.
Tags

Découvrez nos articles