Weed detection is one of the rare computer vision problems where the object to recognise and the object to preserve belong to the same kingdom, present the same colour and often share the same morphology. That proximity founds the whole difficulty of the field.
This article sets out what the task covers, why it resists, what it requires upstream and what it enables downstream. It opens a series of fifteen articles and constitutes their entry point.
What weed detection covers
The term denotes a family of tasks rather than a single operation. Four formulations coexist and they do not call for the same work. Binary detection distinguishes the crop plant from everything that is not it, the simplest formulation and sufficient for many weeding uses. Species detection identifies each weed by name, a requirement of selective treatment programmes where the product depends on the species targeted. Functional group detection groups weeds by their response to treatment, grasses against broadleaves notably, a frequent compromise between the previous two. And infestation mapping measures a density rather than locating individuals, a formulation suited to decisions at field scale. One practical consequence follows. Those four formulations produce corpora incompatible with one another, and the choice is made at scoping rather than corrected mid-project, a binary corpus not being promotable to a species corpus without complete re-annotation.Why weed detection resists
Five properties distinguish this problem from most object detection tasks. The visual proximity between target and non-target is extreme, a grass weed and a young cereal presenting an almost identical morphology at an early stage. Within-class variability is considerable, one weed species changing appearance radically between emergence and flowering. The background is alive and changing, soil, crop residue and cast shadows producing textures models confuse with vegetation. Object density is high, an image of an infested field potentially containing dozens of plants to locate individually. And occlusion is the rule rather than the exception, leaves overlapping as the canopy closes. One important observation follows. Those five properties compound rather than offset, and they reach their maximum at exactly the moment detection carries the most agronomic value, early in the cycle, which the article on early growth stages develops.Detecting the crop or detecting the weed
This alternative structures any weed detection system design. It is too rarely stated explicitly. The first strategy learns to recognise the crop plant and treats everything else as undesirable. Three advantages recommend it. The target class is homogeneous, a sown crop presenting far less variability than the weed flora. The corpus is cheaper to build, a single class needing annotation. And the system stays valid against a weed never encountered, which will be classified as undesirable by default. The second strategy learns to recognise the weeds themselves. Three reasons justify it. It permits selective treatment by species. It produces agronomic information usable beyond weeding, notably infestation pressure monitoring. And it remains applicable where the crop is not sown in an identifiable row. One practical consequence follows for a weed detection project. The first strategy is almost always the right answer where weeding is the only use, and it is regularly set aside by reflex in favour of the second, which is more expensive and more fragile.Row information, a bonus from context
One peculiarity of row crops deserves setting out because it changes the nature of the problem. A crop sown in lines produces a geometric regularity the weed does not share, which supplies information independent of the plants’ appearance. Three uses follow. Inter-row detection, where all vegetation is undesirable by construction, permitting treatment with no botanical recognition. Reinforcement of the decision on the row, an aligned plant having a higher probability of belonging to the crop. And consistency checking, a crop plant detected out of alignment signalling an error more probably than a stray plant. One limit accompanies those three uses. It disappears in dense broadcast-sown crops, and it weakens as the crop develops and closes the inter-row, which the article on row crops details.The acquisition modalities
Three configurations feed a weed detection system and they impose very different constraints. Acquisition mounted on the implement places the camera close to the ground, which gives the finest resolution and imposes a real-time constraint, the decision having to be taken while the machine advances. Drone acquisition covers the field in a separate pass, which relaxes the latency constraint and reduces ground resolution, often to the point of making early stages undetectable. And satellite imagery does not reach the individual plant, which confines it to mapping infestation zones. One practical consequence follows. Those three modalities do not produce interchangeable corpora, an annotation made on mounted-camera images not serving to train a system on drone images, and that incompatibility is regularly discovered after the corpus is built rather than before.What annotation requires in this field
Six requirements distinguish annotation for weed detection from ordinary object detection. Distinguishing neighbouring species requires botanical competence, a generalist annotator not reliably separating two close broadleaves at the cotyledon stage. The growth stage convention must be written, one plant at two stages potentially belonging to the same class or to two classes depending on the project. Handling dense zones calls for an explicit rule, infestations forming continuous mats where individuals no longer separate visually. Partially visible plants at the image edge require an inclusion convention. Unusable images require a dedicated class rather than a silent omission, motion blur and overexposure being frequent in mounted acquisition. And consistency between annotators on doubtful cases must be measured rather than assumed. One important observation follows. The third requirement produces the most variability in production: with no written rule, one annotator will enclose a weed mat in a single box where another draws forty, and that gap is not an individual error but a convention defect.The question of the weed mat
This case deserves separate treatment because it is frequent, expensive and rarely anticipated. A dense infestation zone presents no separable individuals, the plants overlapping and forming a continuous surface. Four possible conventions exist and each serves a distinct use. Splitting into size-constrained boxes, which preserves consistency with the rest of the corpus at the price of an arbitrary division. A single enclosing box, economical and producing a localisation too coarse for targeted treatment. Switching to a segmentation mask over those zones, precise and creating a mixed corpus requiring particular handling downstream. And documented exclusion of the zone, a legitimate solution where the system does not have to decide in those configurations, provided it is recorded. One practical consequence follows for a weed detection project. Those four conventions are chosen according to what the sprayer or the robot will do with the information, and the question belongs to scoping with the end user rather than to an arbitration internal to the annotation team.The downstream uses
Three families of application exploit this detection and their requirements differ markedly. Targeted spraying applies a herbicide only where a weed is detected, which reduces applied volumes significantly and imposes a latency constraint, the nozzle having to open while the machine advances. Robotic mechanical weeding physically destroys the plant, which precludes error on the crop, an uprooted plant not being replaceable. And infestation mapping feeds a deferred decision, which relaxes the real-time constraint and shifts the requirement towards spatial representativeness. One observation runs across those three families. Error tolerance differs by an order of magnitude between them, a false detection costing a drop of herbicide in the first case and a destroyed plant in the second, which precludes reusing the same decision threshold from one use to another.How to evaluate a weed detection system
Four principles govern that evaluation and they rule out the most commonly used indicators. Measurement must be per class rather than global, an excellent average performance being compatible with complete failure on the most damaging species. Omissions and false detections must be reported separately, their agronomic cost being asymmetric and that asymmetry changing with the use. The evaluation set must be separated geographically and temporally from the training set, two images of the same field taken the same day sharing too much context to sit on either side. And performance must be measured by growth stage, a system excellent on developed plants and inoperative at emergence not being usable where the value sits. One important observation follows. That last principle is the most often neglected, and it restates the finding the preceding series established about global indicators: the average masks precisely the configuration that determines the use.What weed detection does not solve
Four limits deserve stating because this field attracts disproportionate expectations. It does not replace agronomic diagnosis, the presence of a weed often signalling a rotation, tillage or sowing date problem that weeding treats at the surface. It does not remove resistance, a treatment targeted with the same active substance exerting the same selection pressure as a blanket one. It does not work outside its training domain, a system calibrated on one crop and one region degrading markedly elsewhere. And it does not detect what is not visible, weeds under the canopy escaping any acquisition from above. That third limit is the most consequential in practice. It means a system performing well in demonstration can prove unusable on a neighbouring farm, and it explains why building local corpora remains this field’s principal expense.The role of agronomic expertise
This dependency characterises weed detection more than any other agricultural vision task. It carries organisational consequences. Ground truth is not read from the image, it is decided by someone able to recognise the species. Three consequences follow. The case library must be established with an agronomist, and not by an annotation team working alone from general instructions. Annotator training concerns visual recognition of a limited number of species at defined stages, learning conducted in a few days and following from no generalist annotation competence. And arbitration of doubtful cases must escalate, an annotator settling alone on a plant they do not recognise introducing an error statistical control will not detect. One observation completes those three consequences. This finding matches the one the medical imaging series established: where ground truth belongs to expert judgement, corpus quality is bounded by access to that expertise and not by annotation throughput, which the article on weed annotation quality develops.The regulatory context
A dynamic outside the technical field drives demand and it deserves situating. European policies on reducing plant protection products create pressure on farms and an interest in any technology reducing applied volumes. Three effects follow for weed detection. Demand grows independently of technical progress, driven by regulatory constraint rather than by the appeal of the technology. Evaluation criteria shift towards measurable input reduction, which makes the economic case simpler to build. And mechanical weeding regains an interest it had lost, which widens the market beyond spraying. One caveat accompanies those three effects. The regulatory calendar undergoes revision and postponement, which makes this dynamic reliable in direction and uncertain in pace.Approaching a weed detection project
Five questions scope a project and they are asked in one meeting. Which decision will the system trigger. Spraying, hoeing and mapping require neither the same precision nor the same error tolerance. Is the species needed, or only the crop against weed distinction. That answer determines the corpus cost by a considerable ratio. At what stage does detection intervene. An early stage maximises agronomic value and complicates recognition. Which crops and which regions must the system cover. That answer bounds the generalisation expected and sizes the corpus. And is agronomic competence available to establish ground truth. Its absence is a disqualifying obstacle, not a difficulty to work around. Those five answers determine the formulation, the corpus, the schedule and the feasibility. Asking them before building an image base avoids this field’s most expensive error, annotating at length under a formulation the real use did not require.What a first project should aim for
Four choices make a first weed detection project usable rather than merely instructive. One crop and one region, which bounds the variability the corpus must cover and makes performance measurable. The simplest formulation compatible with the use, the binary distinction sufficing for the majority of weeding needs. A case library established with an agronomist from the first batch, rather than a rework after divergence between annotators is observed. And an evaluation set built on fields distinct from the training ones, a condition without which the performance announced predicts nothing. One observation follows. Those four choices reduce ambition and raise the probability of a usable result, a trade-off the pressure to demonstrate a general capability regularly pushes teams to invert.What this cluster will cover
The fourteen articles that follow each treat an aspect this panorama has only opened. Four groups organise them. The foundations, with the distinction between weed and crop classes, the species that concentrate the difficulty and row crops. The uses, with targeted spraying, robotic mechanical weeding and detection at early stages. Production, with building the corpus, unfavourable field conditions, annotation quality, project cost, sensor choice and the errors to avoid. And context, with European regulation on plant protection products and the field’s developments. One remark accompanies that organisation. The production articles are the ones determining a project’s success, and they are the ones available literature treats least, most publications stopping at architecture comparison.What this field teaches
One cross-cutting observation deserves closing this panorama because it matches a finding established in very different domains. The central difficulty of weed detection is not model performance. Three findings compose it. The available architectures handle this type of problem correctly as soon as a suitable corpus exists. A suitable corpus presupposes an explicit botanical definition, written conventions on stages and dense zones, and expertise to settle doubtful cases. And that definition does not transfer from one context to another, a crop, a region and a stage forming a combination whose corpus is not worth much elsewhere. That finding matches exactly the one from the series on satellite image segmentation and aerial LiDAR, formulated in domains with opposite constraints: the data is abundant, the models are available, and what is missing is the documented ground truth permitting the first to be interpreted through the second.The question a prospective buyer should ask first
One conversation recurs when a farm equipment manufacturer or an agtech company approaches this field, and it deserves preparing. The instinct is to ask which model performs best on published benchmarks. That question has a well-documented answer and it predicts almost nothing about a deployed system. Three questions predict considerably more. What does the training corpus cover, in crops, regions and growth stages. A system’s usable domain is the intersection of these three, not their union. How was ground truth established, and by whom. An answer naming an agronomist and a written case library indicates a corpus worth its cost; an answer describing a general instruction does not. And how was the evaluation set separated from training. Fields shared between the two produce announced figures that field deployment will contradict. Those three answers characterise a weed detection system more reliably than any benchmark figure. They also happen to be answerable in one conversation, which makes them the practical starting point rather than an ideal.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Choosing species detection where the binary distinction sufficed for the use.
- Building a corpus before fixing the task formulation.
- Omitting the convention for handling weed mats.
- Not writing the growth stage convention.
- Having annotation done with no botanical reference established with an agronomist.
- Omitting a class for unusable images.
- Reusing a decision threshold between spraying and mechanical weeding.
- Expecting generalisation beyond the crops and regions of the corpus.
- Neglecting row information in a line-sown crop.
- Evaluating a system on images acquired under the same conditions as training.
