The choice of sensor precedes that of the model and it determines what the system will be able to distinguish. A weed detection project that settles this question last discovers late that its corpus did not carry the information needed.This article sets out what each modality brings, what it costs and how to arbitrate. It extends the article on what a project costs.
What each modality measures
Four sensor families serve weed detection.They differ in the information they capture.The colour sensor records three visible bands, which reproduces human perception and supplies shape, texture and apparent colour.The multispectral sensor adds a few chosen bands, notably in the near infrared, which gives access to physiological properties the eye does not perceive.The hyperspectral sensor records hundreds of narrow bands, which produces a detailed spectral signature per pixel.And depth sensors, stereo vision or time of flight, measure a three-dimensional geometry rather than an appearance.One important practical consequence follows. Those four families do not measure more or less refined versions of the same thing, they measure different quantities, which makes the choice qualitative rather than quantitative.What the near infrared brings
This band alone justifies moving to multispectral.It deserves setting out.Vegetation reflects strongly in the near infrared and soil far less, which produces a contrast out of all proportion to that of the visible.Three uses follow.Separating vegetation from soil becomes almost trivial, which cheaply resolves the green on brown problem.Detecting seedlings on wet soil improves markedly, the weakness of visible contrast having little effect in this band.And physiological state becomes observable, a stressed or senescing plant presenting a response distinct from a vigorous one.One important observation follows for a weed detection project. Those three uses concern the separation between vegetation and non-vegetation rather than the distinction between species, which bounds this band’s contribution to the green on brown problem and leaves green on green almost untouched.What hyperspectral promises and what it delivers
This modality attracts disproportionate expectations in weed detection.It is worth situating them.A plant’s detailed spectral signature does contain species information the visible does not carry.Three obstacles limit its practical exploitation.The sensor cost stays high against the saving targeted weeding seeks.Spatial resolution is generally low, which is a problem on seedlings occupying little area.And the signature depends strongly on lighting conditions, which imposes demanding calibration under working conditions.One practical consequence follows. This modality finds its place in research and mapping rather than in mounted weeding, and a project contemplating it should first verify that the problem genuinely resists simpler modalities.What depth brings
This geometric information usefully complements the spectral modalities.A plant’s height and the structure of its foliage are discriminating characters colour does not carry.Four distinct uses follow.Distinguishing crop from weed by height, a sown crop presenting a homogeneous height spontaneous emergence does not reproduce.Separating objects that overlap in projection, two superimposed plants being distinguishable in depth.Removing cast shadows as a source of confusion, a shadow having no relief.And biomass estimation, agronomic information distinct from weeding.One important observation follows. The third use is the most underestimated, shadows being a major source of error in full sun as the article on conditions set out, and depth eliminates them by construction.The resolution and information trade-off
This trade-off governs the choice of a sensor.It is rarely stated explicitly.At constant budget, increasing the number of bands is generally paid for in spatial resolution.Three consequences follow for a weed detection project.A high-resolution colour sensor sees a seedling a multispectral sensor at the same price does not resolve.Spectral information is useless on an object too small to occupy a whole pixel, the signal then mixing plant and soil.And the target stage therefore determines which way to lean, an early stage arguing for resolution and a developed stage permitting spectral enrichment.One practical consequence follows. This trade-off is settled by the smallest object to detect, a question whose answer determines the sensor well before any consideration of model performance.What each modality costs
Four lines vary with the choice of sensor.Two of them are regularly forgotten.The sensor itself, whose price rises steeply with the number of bands.Calibration, negligible in colour and demanding in multispectral as in hyperspectral.Data volume, a hyperspectral cube weighing several orders of magnitude more than a colour image.And annotation, whose cost rises where the annotator can no longer visually judge what they are annotating.One important observation follows. The last line is the most often omitted: a multispectral or hyperspectral image is not directly interpretable by an annotator, which requires supplying a visible composition alongside and complicates the whole annotation chain.Annotation in multimodal
This difficulty accompanies enriched modalities in weed detection.It is poorly documented.An annotator judges on a visible representation and the corpus carries bands they do not see.Four requirements follow.Supplying a consistent visible composition, on which the annotator actually works.Registration between bands, an offset between channels producing an annotation valid on one and wrong on the others.Temporal consistency, bands acquired sequentially by some sensors presenting an offset on a moving object.And documenting the composition used, without which a later re-annotation will not work on the same view.One practical consequence follows for a weed detection project. Those four requirements mean multimodal annotation costs more than colour annotation at equal volume, a surcharge that should feature in a costing and is frequently omitted.Vegetation indices
A family of treatments accompanies multispectral sensors and its status deserves clarifying.An index combines several bands into a single value supposed to reflect a property of the vegetation.Three uses stay solid.Separating vegetation from soil, an operation for which these indices are robust and largely sufficient.Preliminary masking, which reduces the area to analyse and lightens downstream processing.And estimating vegetation cover across a zone, agronomic information useful independently of any individual detection.Three limits bound their reach.They do not distinguish species, two green plants presenting close values.They depend on lighting, which imposes normalisation under variable conditions.And they saturate under dense canopy, the value ceasing to vary beyond a certain cover.One observation follows for a weed detection project. These indices make an excellent pre-treatment and a poor decision system, a distinction some projects conflate by expecting from a threshold on an index what a classifier should produce.Active lighting as an alternative
One hardware response deserves comparing against the spectral modalities since it addresses a neighbouring problem by another means.A shroud with controlled lighting removes lighting variability rather than compensating it.Three advantages follow.Stability of conditions, which makes a simple colour sensor perform better than an enriched one under variable natural light.The possibility of choosing the lighting wavelength, which brings part of the multispectral benefit at lower cost.And night working, which widens the operating window and removes cast shadows.Three constraints accompany it.The shroud imposes proximity to the ground and a fixed geometry.Power consumption adds to the machine’s balance.And the corpus becomes specific, images under controlled lighting not mixing with images in natural light.One important observation follows. This option is frequently more profitable than spectral enrichment, and it is less often considered because it belongs to mechanics rather than to optics.What the sensor does not solve
Four limits deserve stating.The choice of sensor is sometimes presented as the answer to the problem.No modality distinguishes two grasses at the cotyledon stage, the difference existing neither in geometry nor in spectrum at that stage.No modality sees under the canopy, the limitation being geometric rather than spectral.No modality compensates an insufficient corpus, information available and not learned staying unused.And no modality removes the need for conventions, the nomenclature belonging to a decision independent of the sensor.That third limit deserves emphasis. It means enriching the sensor without correspondingly enriching the corpus produces a spend with no effect, a frequent situation where the hardware choice precedes the corpus design.Verifying a sensor before committing
A simple method settles between two modalities and it takes a day.The principle is to acquire the same scene with both sensors under consideration and to measure what each permits.Four verifications compose it.The visibility of the smallest target object, counted in pixels actually occupied.The separability of the target classes, measured on an annotated sample rather than supposed.Behaviour under unfavourable conditions, the same scene being retaken at another hour.And the data volume produced per unit area, which conditions real-time processing.One practical consequence follows for a weed detection project. This comparison costs an equipment hire and a day in the field, and it replaces a debate of opinion with a measurement, which makes it disproportionately profitable before an equipment purchase.Arbitrating by the use
A three-step method settles this choice and it starts downstream.Establish the distinction the system must make, vegetation against soil, crop against weed, or species against species.Identify the simplest modality carrying that information, verifying on a sample rather than trusting the literature.And verify that resolution stays sufficient at the target stage after that choice.One practical consequence follows. This sequence leads to the colour sensor in the majority of weeding projects, to the near infrared where separating vegetation from soil is at issue, and to depth where shadows or overlaps dominate the errors.The multimodal arrangement
One configuration combines several sensors and it deserves situating.Pairing colour with near infrared, or colour with depth, brings more than each separately.Three benefits emerge.Complementarity of information, colour carrying texture and depth carrying structure.Robustness to conditions, one modality compensating the other’s failure according to lighting.And cross-verification, a disagreement between modalities signalling a difficult configuration.Three costs accompany it.Registration between sensors, a delicate operation whose failure distorts everything.Temporal synchronisation, demanding on a moving machine.And chain complexity, each sensor adding a failure point.What the sensor imposes on the corpus
Four consequences follow from the hardware choice and they commit the corpus as much as the system.Non-interchangeability, a corpus acquired with one sensor not serving to train a system using another.Dependence on mounting geometry, height and angle being parameters of the corpus as much as the sensor itself.Hardware-linked obsolescence, a change of sensor generation potentially devaluing a corpus built with the previous one.And the need to retain raw images, a corpus stored after processing not being re-exploitable with a different pre-treatment.One important observation follows. The last consequence is the easiest to respect and the most often neglected, retaining the raw costing storage and preserving the ability to re-exploit a corpus when the chain evolves, which matches the principle every preceding series established on retaining source data.The sensor and the nomenclature
A direct link connects these two decisions and it is rarely examined together.Each class in the nomenclature presupposes that some information permits distinguishing it, and that information comes from the sensor.Three verifications apply class by class.Does the class rest on a difference of colour or texture, in which case the visible suffices.Does it rest on a physiological state, in which case the near infrared becomes useful.Or does it rest on a structure or a height, in which case depth brings what the first two do not carry.One practical consequence follows for a weed detection project. This verification is conducted by going through the nomenclature line by line, it takes an hour, and it frequently reveals that a requested class rests on no information the chosen sensor records.The drone case
Its trade-offs differ from those of mounted weed detection.This configuration deserves separate treatment.Distance to the ground alters the balance between modalities.Three consequences follow.Resolution becomes the absolute limiting factor, which often rules out enriched sensors whose resolution is lower at equal price.The latency constraint disappears, analysis happening after the flight, which permits treatments mounted use precludes.And spectral coverage becomes more accessible, light multispectral sensors being designed for this configuration.One observation follows. The drone suits infestation mapping and suits individual detection at the early stage poorly, which makes it a decision tool at field scale rather than a weeding sensor.What hardware developments will change
Three developments are under way and it is worth distinguishing their real effect.Compact multispectral sensors are becoming commonplace, which lowers the cost of the near infrared without altering what it brings.Depth sensors are gaining outdoor robustness, a difficulty that has so far limited their field use.And on-board computing power is growing, which relaxes the latency constraint the spraying article set out.One observation follows for a weed detection project. None of those three developments creates information about the distinction between species at the early stage, which leaves this field’s central difficulty intact and confirms that hardware progress shifts the cost without shifting the bottleneck.Approaching the choice of a sensor
Five questions scope this decision and they are asked before any acquisition.Which distinction must the system make. That answer bounds the modality needed.What is the smallest object to detect. That answer fixes the resolution and settles the trade-off with the number of bands.Will the system work under controlled lighting. A positive answer simplifies multispectral calibration.What forward speed is targeted. That answer bounds the data volume processable in real time.And has multimodal annotation been costed. Its omission produces a systematic overrun.Those five answers determine the modality, the resolution and the corpus cost. Asking them before buying a sensor avoids the situation where performant hardware produces images the project cannot exploit.The question to ask first
One question orients this choice and it is asked before any hardware comparison.Does the current problem belong to green on brown or to green on green.In the first case, separating vegetation from soil suffices, the near infrared makes it robust, and the sensor debate is almost closed.In the second, the difficulty lies in distinguishing two vegetations, no modality resolves it alone, and effort should go to the corpus rather than to the hardware.That question precedes all the others. It avoids this chapter’s most expensive error, funding an enriched sensor for a problem it does not address.What this chapter teaches
One cross-cutting observation deserves closing this examination because it matches a finding of the preceding series.Enriching the sensor does not move the bottleneck of a weed detection project.Three findings compose it.Enriched modalities improve the separation between vegetation and non-vegetation, a problem the visible already handled correctly in most cases.They bring almost nothing to the distinction between species at the stage where it matters, the difference not yet existing physically.And they raise the annotation cost, which moves resources away from the line that determined quality.That finding matches exactly the one the series on aerial LiDAR established about fusion with imagery: an additional modality is justified where a precise class in the nomenclature requires it, and it is an overcost where it is added on principle.Who decides the sensor, and when
One sequencing problem recurs and it explains a substantial share of wasted hardware budgets.The sensor is typically chosen by the engineering team, early, and often before the annotation provider is involved at all.Three consequences follow for a weed detection project.The corpus inherits a modality nobody checked against the nomenclature, which is the verification set out above and which takes an hour.The annotation surcharge appears late, once the images arrive and prove not to be directly readable.And the choice is hard to revisit, hardware having been bought and integration work already done.One practical response follows. An annotation provider asked to quote on a project should ask which sensor was chosen and why, not out of curiosity but because the answer frequently reveals a mismatch worth naming before the corpus is built rather than after.One observation completes that. Raising it costs one question and it positions the provider as someone who checks the chain rather than someone who processes what arrives, which is a distinction clients notice.Common mistakes
These failures recur often enough that naming them is usually enough to avoid them.- Choosing the sensor before establishing the distinction to make.
- Expecting multispectral to distinguish two species at the early stage.
- Sacrificing spatial resolution for bands unusable on a small object.
- Omitting the annotation surcharge specific to multimodal work.
- Neglecting registration between sensors in a combined arrangement.
- Contemplating hyperspectral without having verified that the visible fails.
- Neglecting depth’s contribution on cast shadows.
- Enriching the sensor without correspondingly enriching the corpus.
- Comparing sensors on catalogue rather than on a real sample.
- Omitting the data volume constraint at the target working speed.
