A vision system that works in the lab and fails in production usually does not have an algorithm problem. It has an integration problem: a line speed that leaves no time to acquire a sharp image, lighting that drifts over weeks, a tolerance rule never translated into a computable criterion, or no link to the control system, which makes the decision unusable.
A study of vision and programmable logic controller integration illustrates that shift well. Across three distinct industrial cases, detection accuracy exceeded 95 per cent, and triple verification at controller level reduced false classifications by 28 per cent compared with camera-only operation. In other words, more than a quarter of the errors were corrected outside the model, through how the system was wired into the line.
This article covers that integration layer: throughput, lighting, tolerances, the link to the control system and how to run the deployment. It extends the complete guide to defect detection by computer vision, which it refers to for approaches and ontology.
What separates a pilot from a line system
A defect detection pilot demonstrates that a defect is detectable under chosen conditions. A production system must hold performance over months, with mispositioned parts, different material batches, ageing lighting and a line that does not stop.
The same integration study is explicit about the limits encountered: sensitivity to lighting variation, limited dataset size, and challenges in scaling to full production environments. Those three cover most observed failures, and none is a matter of model architecture.
The practical consequence is that a defect detection project must be designed from the outset for operation, rather than validated in the lab and then transposed. The structuring decisions, throughput, optics, lighting, operating point, are taken before the first batch of images.
Throughput and its downstream consequences
Line speed is the constraint from which almost every other decision in a defect detection project follows, and it translates first into a time budget.
The time budget per part
Everything downstream is bounded by one number, so it is worth computing first. The reasoning is arithmetic and worth setting out explicitly, because it rules out certain architectures immediately. A line at one hundred parts per minute leaves six hundred milliseconds per part, within which triggering, acquisition, transfer, inference and transmission of the decision must all fit. Integration guidance places inference on local compute between ten and fifty milliseconds per image, comfortable at that rate, and recommends above three hundred parts per minute the use of line-scan cameras with strobed illumination to obtain continuous coverage without mechanical stops.
Inference is therefore rarely the actual bottleneck in practice. More often it is image transfer, data writing and synchronisation with the controller. Establishing that time budget item by item, before any hardware choice, avoids discovering late that a high-performing architecture cannot hold the line rate.
Area or line-scan camera
The choice follows from product geometry and throughput, and it is not reversible without rebuilding the corpus. Two families dominate and they are not interchangeable. An area camera captures a complete image at each trigger, which suits discrete, correctly positioned parts. A line-scan camera reconstructs a continuous image from the movement, which becomes necessary on webs, coils and high rates. The choice has direct annotation consequences: a reconstructed image has no natural boundaries between parts, which requires explicitly defining the inspection unit and therefore the annotation unit.
Triggering and positioning
Repeatability beats software correction. Triggering from a presence sensor or a controller signal guarantees the part is in the same position at every acquisition. That repeatability is the single factor that most simplifies the model’s task, and its degradation is a frequent cause of performance decline after several months: a slide developing play is enough to shift parts and push the scene outside the training distribution.
Local or centralised processing
The last trade-off concerns where the computation happens. Neither answer is universally right, and it follows from response time requirements rather than preference. Local processing, on the camera or on a compute node at the line, gives response times compatible with a stop or an ejector. Centralised processing eases model version management and data collection. Most deployments end up with a mixed architecture: local decision, with images and results sent to a central environment for analysis and retraining.
Lighting, the leading cause of false detections in defect detection
This is where a defect detection project gets its best return. One principle circulates in integration guidance and deserves to be taken seriously: incorrect lighting causes more false positives than model limitations, and should be specified before the camera is even selected.
Design before purchasing
The order of operations matters here as much as the choice itself. Lighting is designed from the defect sought, not from a catalogue. Grazing light brings out relief and scratches, diffuse light removes reflections on glossy surfaces, dark-field lighting isolates fine defects on polished surfaces, backlighting reveals contours and missing matter. A setup that makes the defect obvious to the eye makes the model’s task nearly trivial, and the reverse is equally true.
The budget consequence is significant and often misunderstood. A properly designed optical setup simultaneously reduces annotation cost, since defects are easier to spot, and modelling cost, since the problem becomes simpler. It is the only line item in a defect detection project that reduces both at once.
Stability over time
Design is necessary and not sufficient. Correctly designed lighting nonetheless degrades. Sources lose intensity, optics accumulate dust, ambient light varies with time of day and season, and a maintenance intervention occasionally shifts a mount by a few millimetres. Each of these moves normal appearance and triggers unjustified alerts.
None of these is dramatic on its own; their accumulation is what moves the system out of its validated envelope. The countermeasure is simple and rarely implemented: place a reference target in the field, a grey patch or contrast pattern, whose values are measured at every acquisition. Any drift then becomes automatically detectable before it affects results, and the system can alert maintenance rather than produce scrap.
Optics ageing
Beyond drift monitoring, one physical point deserves anticipating at installation: access to the optics for cleaning. A camera mounted in an awkward position will not be cleaned at the necessary frequency, whatever the written procedure says, and progressive contamination produces exactly the kind of slow drift aggregate indicators do not catch. Providing easy access and a cleaning frequency written into the maintenance schedule costs a trivial amount at design stage and avoids months of degraded performance.
Enclosure
Where conditions permit it, one measure dominates all the others. Isolating the station from ambient light with an opaque enclosure remains the most effective way to stabilise acquisition conditions durably. It is sometimes refused for accessibility or footprint reasons, but its absence must then be compensated by stricter drift monitoring, and that constraint belongs in the design rather than being discovered in operation.
Tolerances and the decision rule
A defect detection system does not merely detect, it decides. Translating a quality specification into a computable rule is the most neglected part of integration.
From specification to rule
The gap between the two languages is wider than most teams expect. A quality specification is often expressed in natural language: no visible scratch in the cosmetic zone, porosity tolerated if isolated and small. A computable rule requires numeric thresholds, a definition of the zones, an aggregation criterion for multiple defects and a procedure for ambiguity.
That translation work is the bridge between the quality department and the technical team, and it retroactively determines the annotation primitive. If the rule concerns the cumulative length of scratches within a zone, binary classification will never suffice, and a mask or polygon becomes necessary. Establishing the rule before launching annotation avoids producing a corpus that cannot support the decision.
The operating point
Once the rule is set, the cursor has to be placed. Two indicators govern operation and pull against each other: the escape rate, meaning non-conforming parts accepted, and the false reject rate, meaning conforming parts discarded. A system is tuned along a curve between these two errors, and choosing the operating point is an economic decision, not a technical one. The cost of an escape and the cost of a false reject differ across sectors by considerable margins, and the setting must reflect that ratio rather than a statistical optimum.
The consequence for corpus construction is direct and frequently missed: it must contain enough conforming parts to estimate a false reject rate credibly. A dataset composed mainly of defective examples, which happens often because they are harder to collect and therefore more prized, answers only half the question.
Adjustment in operation
Requirements move, and a fixed setting will not survive them. The operating point must finally remain adjustable without retraining the model, through a parameter accessible from the operator interface. That flexibility is indispensable at product changeovers, during runs on different material, and in periods of quality pressure. It presupposes that the model outputs a continuous score rather than a binary decision, an architecture choice to make at the outset.
Integration with the control system
The output of the defect detection system must reach the actuator, and that link matters more than it appears.
Two directions of flow matter, and they serve different purposes. The result travels to the controller over a deterministic industrial protocol, which guarantees the decision arrives within the ejector’s time window. Reporting upward to production tracking systems provides traceability, associates the result with the batch and feeds quality indicators.
The most interesting and least expected lesson of the study cited in the introduction concerns cross-verification performed at controller level. Combining the model’s decision with other available signals, part presence, position, independent dimensional measurements, removes a significant share of errors without touching the model. It is an improvement route frequently ignored by teams focused on algorithmic performance, even though it is cheap and immediately available.
Product changeover and variability
A line rarely produces a single reference, and switching between them is one of the moments where a defect detection system proves fragile.
Three configurations occur, and none is obviously right. A single model covering all references, which presupposes sufficiently homogeneous normal appearance and quickly becomes untenable once shades or shapes differ. One model per reference, better performing but multiplying corpora, training runs and versions to maintain. Or a single model receiving the reference as an input, which presupposes the controller transmits that information and that the corpus covers each reference.
The choice is made at scoping and it has direct annotation consequences: in the third case, the product reference becomes a mandatory metadata field on every image, and its absence makes the corpus unusable for that architecture. It is exactly the kind of information nobody thinks to keep at collection time and that is sorely missed six months later.
Running the deployment
A successful defect detection deployment follows a phased progression, each phase validating the next and none parallelising.
The most common failure pattern is attempting full coverage before validating on one representative station. Selecting inspection points comes first, and it should be restrictive: identify the few highest-value checks rather than aiming for full coverage. Installation and optical validation follow, verifying that each defect type sought is actually visible in the chosen setup, which is tested with known defective parts before any mass collection.
Then comes the most important and the most frequently curtailed phase of all: parallel running. The system operates alongside the existing inspection without deciding, and every disagreement between machine and operator is examined. That phase produces three outcomes: an operating point tuned on real data, a corpus of difficult cases particularly valuable for retraining, and operator buy-in, since they see their objections handled.
The switchover to automated control happens only once target performance is reached, measured under real conditions rather than in the lab. Compressing that sequence to hold a schedule is the field’s most expensive shortcut.
Handling the reject
One downstream question is settled surprisingly late in many projects: what physically happens to a part the system rejects. The answer shapes the acceptable error profile more than any model parameter.
Three regimes are worth distinguishing. Where rejection is automatic and irreversible, scrapping or diverting to waste, a false reject is a direct loss and the operating point must be set conservatively. Where rejection routes the part to a manual review station, a false reject costs only inspection time, and the system can afford to be more sensitive, since a human filters the output. Where rejection merely flags the part for downstream attention, the tolerance for false alerts is higher still but attention fatigue becomes the binding constraint.
These three regimes call for different operating points and, more importantly, for different corpora: the second and third require the system to rank its outputs by confidence so review effort can be prioritised, which in turn requires annotation that captures severity rather than mere presence.
Operator acceptance
A non-technical factor frequently decides the fate of a defect detection installation, and it rarely appears in specifications: buy-in from the people working with the system.
It is also the factor least likely to be measured, and the most likely to be dismissed as a soft issue. The failure mechanism is well known. A system generating too many false alerts early in operation teaches operators to disregard it, and that habit persists long after the setting has been corrected. Conversely, a system perceived as reliable gets used, and more importantly its errors get reported, which feeds the improvement loop.
Three simple practices markedly improve buy-in. Set the operating point conservatively at start-up, accepting more escapes rather than saturating with alerts. Make visible what the system saw, through a viewable annotated image, so the operator can judge rather than merely comply. And provide an easy way to contest a decision, whose outcome feeds directly into the retraining corpus. These three cost little and largely determine whether the system is used or worked around.
What operation demands from defect detection annotation
A line system generates a continuous annotation need, which is better organised than absorbed. That need is not a sign of a failing system, it is the condition of its maintenance.
Three streams feed a defect detection corpus once the system is live. Disagreement cases between the system and the operator, which form the most informative corpus since they concentrate on the decision boundary. Defects of a new type, flagged through the class provided for that purpose and triggering an ontology extension. And images collected after a change of material, tooling or settings, needed to verify that normal appearance has not drifted.
Budgeting for them is the practical test of whether a project has understood what it is buying. These campaigns follow the same methodological requirements as the original corpus: versioned protocol, sampled control, documented arbitration. They are smaller, more frequent and richer in difficult cases, which makes them proportionally more expensive per unit. Anticipating them contractually rather than improvising them is the difference between a system that holds and one that decays.
Supervising a defect detection system in operation
A defect detection system should be monitored like a measuring instrument, with indicators tracked over time.
Without this, degradation is discovered through customer complaints rather than through data. Four indicators cover the essentials, and tracking them requires no particular tooling. The detection rate per defect type, which reveals targeted degradation before it affects the aggregate. The false reject rate, whose rise usually signals drift in conditions rather than a model problem. The distribution of scores the model produces, whose shift is the earliest signal of a change in the inspected population. And the reference target values, which separate optical drift from product drift.
These indicators are read as trends over weeks and never as isolated point values. Reacting to every daily variation is as counterproductive as monitoring nothing, and bounds established from the variability observed in steady state distinguish noise from signal.
The most common mistakes
These failures recur often enough across deployments that naming them is usually enough to avoid them.
- Selecting the camera and optics before designing and validating the lighting.
- Establishing the per-part time budget after buying hardware rather than before.
- Launching annotation without translating the quality specification into a computable rule.
- Building a corpus with too few conforming parts to estimate the false reject rate.
- Delivering a model producing a binary decision, not adjustable in operation.
- Curtailing the parallel running phase to hold a schedule.
- Neglecting cross-verification at controller level, which corrects a share of errors cheaply.
- Installing no reference target to distinguish optical drift from product drift.
- Planning a system delivered once, with no subsequent annotation campaigns.
- Saturating operators with alerts at start-up and losing their confidence lastingly.
- Not retaining the product reference as metadata, which rules out any multi-reference architecture.
What to take away
Line integration is where the real performance of a defect detection system is decided, and most of the levers there are physical and organisational rather than algorithmic. Lighting, positioning repeatability, translation of the quality rule and the link to the control system explain most of the gap between a convincing pilot and a durable operation.
Three decisions structure a deployment that holds. Design lighting and the time budget before any hardware purchase, since they condition everything else. Translate the quality specification into a computable rule before launching annotation, so the corpus can support the decision. And plan drift monitoring and subsequent annotation campaigns from the design stage, without which the system degrades silently.
For available approaches, ontology and annotation tasks, the complete guide to defect detection sets the general frame. For the class imbalance problem, central as soon as the initial corpus is being built, the article on annotating rare defects covers the applicable strategies.
To explore delivery arrangements, supported formats and applicable control mechanisms, see our dedicated page on annotation for industry. And if you are preparing an automated inspection deployment and want the annotation strategy scoped upstream, let us discuss your project.
