Behind every corpus are people who looked at data, one item at a time, for weeks. What that work costs them and what it earns them never appears in the deliverable, which does not stop the question from arising.
This article sets out those stakes. It extends the article on labeling errors.
What this work demands of people
Five demands weigh on those who practise data labeling. A sustained attention on necessarily repetitive tasks. A constant application of often numerous rules. A throughput that often conditions the pay. An exposure sometimes repeated to difficult content. And an absence of feedback on the work’s final use. One important practical consequence follows. The last demand weighs more than one supposes, work whose destination nobody sees losing the meaning that makes its repetition bearable.What conditions change about quality
Five links tie the conditions to the result produced. An excessive throughput degrades attention well before any indicator. Pay by volume discourages the reporting of cases. An absence of training quickly produces divergent conventions. A high turnover erases all knowledge of the limiting cases. And a climate of sanction quickly makes the questions disappear. One important observation follows for a data labeling project. The second link explains many mediocre corpora, an operator paid per described element having every interest in settling matters alone rather than reporting a case.What the ordering party must know
Five pieces of information concern the data labeling production chain. The exact place where the work is effectively carried out. The precise status of the people who execute it. The pay practised set against the local cost of living. The existence or absence of subcontracting organised in cascade. And the arrangements provided for handling difficult content. One practical consequence follows. The fourth piece is almost always missing, a client assuming they work with the organisation chosen while part of the work circulates towards a party they do not know.What offshoring raises
Five observations concern work carried out at a distance. The cost gap comes first from real differences in living standards. Pay that is low elsewhere can stay decent locally. Comparing gross salaries says nothing about the real situation. The applicable protections depend entirely on the country’s law. And the geographical distance makes any verification far harder. One important practical consequence follows for a data labeling project. The third observation calls for caution in both directions, pay judged low from outside possibly constituting a qualified and stable job on the spot, which makes judgement at a distance more delicate than it looks.What difficult content imposes
Five arrangements frame this kind of data labeling work. A prior briefing on the exact nature of the content. A consent obtained before any effective assignment. A strict limitation on the daily exposure time. A possibility of withdrawing from the task with no consequence. And support available for the people concerned. One observation follows. The fourth arrangement has value only if it is real, a withdrawal entailing a loss of income or of assignment not constituting a free choice.What invisibility produces
Five effects follow the erasure of people behind the corpus. The work ends up appearing as a purely technical operation. The production conditions then escape all discussion. The price is compared with nothing explaining it. The competence nonetheless acquired is valued nowhere. And the difficulties genuinely met never get reported. One important practical consequence follows for a data labeling project. The third effect ties this question directly to the sector’s economics, an unexplained price gap sometimes being explained by conditions the client would not accept if they knew them.What a quality job presupposes
Five elements distinguish a durable post from a disposable task. A genuine employment contract rather than a per-task assignment. An initial training followed by regular accompaniment. A possible progression towards reviewing or supervisory roles. Pay entirely independent of the volume produced alone. And a stable team permitting competence to accumulate. One important observation follows for a data labeling project. Those five elements serve quality as much as the people, a stable and trained team producing precisely the documented and consistent corpus the preceding chapters described as hard to obtain.What corpus bias engages
Five observations concern the representativeness of what is described. A corpus reflects exactly the populations its data holds. An absence in the data produces the same absence in the system. The categories adopted themselves carry a point of view. The operators themselves apply culturally situated conventions. And the result will nonetheless apply to everyone. One important practical consequence follows for a data labeling project. The third observation goes beyond the technical question, a division into categories expressing a way of seeing rather than an objective reality, which makes the choice of reference a decision whose effects reach well outside the project.What the meaning of the work changes
Five contributions follow from work whose destination is known. The adjudications given on new cases gain in relevance. Attention holds up longer on repetitive tasks. The reports end up bearing on what genuinely matters. Team turnover then falls very markedly. And the competence acquired is passed on instead of disappearing. One important observation follows for a data labeling project. The first contribution is easily verified, an operator knowing a corpus will serve to spot a manufacturing defect settling an ambiguous case in the direction of the use, which no written rule could have prescribed in advance.Three errors proper to this question
Three defects recur in data labeling projects. A low price accepted without ever asking about its origin. Pay by volume expected alongside the reporting of cases. And an assignment to difficult content with no preparation at all. Those three defects appear in no deliverable, they translate into the quality produced before translating anywhere else, and their common point is to bear on what nobody judged it useful to ask.What the client can ask at no cost
Five requirements barely add to the price of a service. The declaration of the precise place where the work will be executed. The explicit declaration of any subcontracting envisaged. An undertaking on the absence of pay by volume alone. Information on the difficult content present in the corpus. And the possibility of visiting the site where production happens. One important practical consequence follows for a data labeling project. Those five requirements cost the provider transparency rather than budget, which makes them accessible to any project and makes their absence a client’s choice far more than an economic constraint.What this chapter shares with the previous ones
Three findings recur from the start of this course. Competence builds slowly where it is built. A reported case is worth more than a case settled alone. And a low price is always explained by something. One important observation follows for a data labeling project. Those three findings converge here on one same conclusion, the conditions permitting a team to last, to question and to learn being exactly those the technical chapters called for without ever naming them that way.What this subject has that is particular
Four traits distinguish this question from the course’s others. It bears on people rather than on data. It is read in no indicator of the deliverable. It engages a responsibility the contract does not transfer. And it nonetheless rejoins quality by several paths. One important practical consequence follows for a data labeling project. The last trait deserves underlining, an ordering party indifferent to these questions exposing themselves to very real technical defects, which makes this chapter useful even to someone who would see in it only a moral subject.What recognising the work presupposes
Five practices make visible a contribution usually erased. Naming the people who produced a corpus within its documentation. Reporting back to the teams the results obtained through their work. Formally recognising the competence acquired on a precise field. Involving experienced operators in drafting the reference. And presenting this work as a real qualification rather than a stopgap. One important observation follows for a data labeling project. The fourth practice directly serves the project, an operator having met hundreds of limiting cases knowing the field better than whoever drafts the rules, which makes their involvement in scoping useful well beyond the gesture of recognition.What the subcontracting chain dilutes
Five effects follow the lengthening of a production chain. Each intermediary takes a share of the price paid along the way. The final pay shrinks at every level crossed. The undertakings given transmit worse and worse. The client loses all visibility on the real conditions. And nobody feels genuinely responsible for the whole. One important practical consequence follows for a data labeling project. The last effect characterises this organisation, each party being able to state honestly that they respect their own undertakings while the overall result escapes everyone, which makes the chain’s length more determining than anyone’s good faith.What this chapter presupposes of the provider
Five practices distinguish a serious employer from a plain intermediary. Employing under contract rather than per task as soon as the need lasts. Training the teams rather than replacing them at the first gap. Ruling out pay that penalises the reporting of cases. Preparing people well before any exposure to difficult content. And refusing a price that would permit none of these practices. One important practical consequence follows. The last practice conditions all the others, a provider accepting an untenable rate necessarily having to pass it on to working conditions, which makes refusing a bad price more determining than any undertaking displayed.What these conditions really cost
Five items compose the gap between two organisations. An employment contract rather than a one-off assignment. An initial training that produces nothing in an immediate way. A supervisor available to handle the daily questions. Pay independent of the volume produced. And accompaniment provided for facing difficult content. One important observation follows for a data labeling project. Those five items explain part of the gap between two quotations, which rejoins the finding of the chapter on cost, a markedly lower price often covering an organisation the client would not have chosen had they known it.What client requirements change
Five developments are progressively modifying the sector’s practices. Large ordering parties now audit their providers themselves. Social undertakings appear more and more in tenders. Transparency on the chain is gradually becoming a condition of access. Difficult content is made the subject of specific clauses. And tracing the work’s origin is generalising rapidly. One important practical consequence follows for a data labeling project. Those five developments come from demand rather than from regulation, an important client requiring verifiable conditions obtaining in months what no text had imposed, which places the principal lever on the side of whoever orders.What automation changes about this question
Five effects accompany the reduction of repetitive work. The most mechanical tasks effectively and progressively disappear. The remaining share demands more judgement and more training. The number of people concerned then falls slowly. The qualification of the posts that persist rises instead. And exposure to difficult content does not fall. One important observation follows for a data labeling project. The last effect deserves attention, automation absorbing the simple tasks and leaving precisely those a system cannot settle, which concentrates the human work on what is most demanding in it without reducing its difficulty.What this chapter does not claim to settle
Four questions stay open despite everything above. The level of pay genuinely fair in a given context. The boundary between a quality requirement and excessive pressure. The exact share of responsibility falling to the final client. And how to compare conditions fairly across several countries. One important practical consequence follows for a data labeling project. Those four questions call for no general answer, which does not stop them being asked in each particular situation, an ordering party who asks them already behaving differently from one who ignores them.What the provider brings here
Four contributions distinguish a responsible production. A transparency on place and status without being asked for it. Pay that does not depend on the volume produced alone. An explicit declaration of any subcontracting envisaged. And a refusal of projects whose conditions cannot be met. One important practical consequence follows. The second contribution shows in the delivered corpus, a team whose pay does not penalise reporting sending up limiting cases and producing precisely the documented catalogue a serious project requires.Approaching these questions
Five questions scope these stakes on a data labeling project. Where is the work carried out. The answer is often missing. What status do the people have. It determines their protection. Does subcontracting exist. It is discovered late. Is the content difficult. That imposes arrangements. And is the low price explained. Sometimes by the conditions. Those five answers determine the responsibility engaged. Asking them before ordering avoids discovering afterwards what one was funding.The question that frames the responsibility
One question determines what an ordering party engages in data labeling. Do they know under what conditions their corpus was produced, or not. Knowing engages a clear responsibility, an informed client choosing knowingly and being able to require changes. Not knowing exonerates nothing, a subcontracting chain nobody asked about staying the result of a decision not to ask. That question belongs to what one chose to know, it is asked before any order, and it separates an owned responsibility from one merely displaced.Three decisions that engage responsibility
Three decisions structure a data labeling project. Asking where the work will effectively be carried out and by whom. Ruling out pay tied to the volume produced alone. And providing an arrangement as soon as difficult content is involved. Those three decisions cost almost nothing, they precede the order, and their absence commits the ordering party to conditions they have never examined.Three checks before ordering
Three controls qualify a data labeling chain. The real place where the work will be executed. The status of the people who take part in it. And the existence or absence of undeclared subcontracting. Those three controls are each asked in one question, they require no technical competence, and their absence leads to funding conditions nobody has ever examined.Three questions to ask a provider
Three questions precede awarding a project. What status do the people who will produce the corpus have. Does the pay depend on the volume described alone. And will part of the work be entrusted to a third party. Those three questions are asked in a few minutes, they require no technical competence, and their absence leads to funding an organisation one knows nothing about.What this chapter teaches
One cross-cutting observation deserves closing this examination. The production conditions are read in the corpus produced. Three findings compose it. Pay by volume discourages the reporting of cases. A climate of sanction makes the useful questions disappear. And work with no visible destination loses the meaning that sustains it. That finding extends the preceding chapters, quality depending on conditions nothing in the deliverable reveals.What this chapter leaves to the next
These conditions change as the work itself changes. Three questions stay open in data labeling. What automation genuinely displaces in this work. Which competences gain value and which lose it. And what this work will look like in a few years. Those three questions belong to labeling trends, which constitute the subject of the following chapter.Why the chain question is rarely asked
One point about how these projects are awarded belongs at the close. Almost no buyer asks who will actually produce their corpus. Three reasons follow in data labeling. The question sounds like distrust at the start of a relationship. The answer is hard to verify from the other side of the world. And nothing in the deliverable will ever reveal what it was. One important practical consequence follows for a data labeling provider. Volunteering the answer before it is asked removes the awkwardness entirely, since a provider who states their chain unprompted has made verification unnecessary, and one who waits to be asked has already made the client wonder why they waited.What the two sides gain here
Four returns follow from conditions that hold up. The client gets a corpus produced by people who stayed. The provider keeps the competence it spent months building. The operators get a job rather than a sequence of tasks. And every party can say where the work came from. One important observation follows for a data labeling project. The second return is the one providers underestimate, a team retained across several projects carrying the accumulated knowledge of limiting cases that no reference document fully captures, which turns decent conditions into a commercial asset rather than a cost.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Paying by volume for work from which reports are expected.
- Ignoring the existence of subcontracting in cascade.
- Assigning difficult content with no prior briefing.
- Treating a reported question as an admission of incompetence.
- Comparing two prices without asking about the gap.
- Neglecting the meaning of the work in the daily organisation.
- Confusing an absence of complaint with an absence of difficulty.
- Considering conditions to have no effect on quality.
- Offering a withdrawal that entails a loss of income.
- Assuming a provider applies the same rules as oneself.
