Cost and ROI of a Defect Detection Project

Searching for the return on investment of a vision inspection project produces a remarkable convergence of figures. The cost of poor quality sits at around twenty per cent of revenue, three-year return exceeds three hundred and fifty per cent, and payback falls between seven and eight months. Those values appear almost identically across dozens of pages, nearly all published by solution vendors.That convergence should invite suspicion rather than confidence. The figures circulate, cite one another, and their primary source is rarely accessible. At best they describe successful deployments, selected because they were worth publicising, which is the opposite of a representative average.This article therefore offers no benchmark ratio. It offers a method for building a costing specific to one context: which cost lines to count, which gains are genuinely measurable, how to structure a pilot that produces the inputs to the calculation, and which traps make most estimates wrong. It extends the complete guide to defect detection.

Why published ratios do not apply

Three reasons make a benchmark ratio unusable for deciding, and they are worth knowing before basing a decision on one.

Selection bias

Published case studies cover completed projects. Projects abandoned at pilot stage, those whose system was unplugged after six months for lack of acceptance, those whose performance degraded after a material change, are never written up. The visible average is therefore computed on the favourable tail of the distribution.

Situational heterogeneity

Return depends on factors varying by an order of magnitude between sites: product unit value, initial defect rate, hourly cost of inspection, the inspection’s position in the chain, and the cost of an escape reaching the customer. The same system installed on two lines produces two unrelated savings.

Variable scope

A costing announcing rapid payback generally counts hardware and software. It rarely counts corpus construction, integration with the control system, change management, or subsequent annotation campaigns. Since those lines represent a substantial share of the total, the gap between the quoted figure and the full cost is structural.

The cost lines of a defect detection project

A credible defect detection costing has seven lines, of which only three are usually anticipated.

Hardware

Cameras, optics, lighting, compute hardware, mechanical structure and cabling. It is the best estimated line because it is the most tangible, and generally not the largest. A frequent error is under-specifying lighting, which represents a fraction of hardware cost and determines a major share of performance.

Corpus construction

Collection, de-identification where needed, preparation, annotation, quality control and documentation. This line is the most often underestimated, and it is the only one that benefits from no economy of scale at low volume: training a team on a product costs roughly the same for two thousand images as for twenty thousand.

Development and training

Architecture selection, training, evaluation, iterations. This is now the least expensive of the three technical lines, available tooling having considerably reduced the work required. The imbalance between a modest modelling line and a heavy data line often surprises teams coming from software.

Integration

Connection to the control system, reporting to production tracking systems, operator interface, version management. This is conventional engineering work, predictable, but absent from costings centred on model performance.

Change management

Operator training, parallel running phase, operating point adjustment, adaptation of quality procedures. This line is intangible and yet decisive: parallel running consumes time on both sides for several weeks, and shortening it is the field’s most expensive shortcut.

Ongoing maintenance

Drift monitoring, optical cleaning, supplementary annotation campaigns, retraining. This line is recurrent by nature, and its absence from an initial costing produces a system that degrades silently after delivery.

Rework

A defect detection project that does not budget for it will have it anyway. Explicitly provisioning for protocol revision after the pilot, new sources arriving or an unanticipated constraint being discovered is more honest than discovering the need midway.

Defect detection gains, and what they are actually worth

Four families of gain are invoked in defect detection, and they differ sharply in credibility.

Scrap reduction

Detecting a defect earlier avoids adding value to a part that will be discarded. That gain is real and calculable: it is the number of parts concerned multiplied by the value added between the current detection point and the proposed one.One caution applies. The gain comes not from detection itself but from moving it earlier in the chain. A system installed at the same point as the existing check does not reduce scrap, it only changes who finds it.

Escape reduction

This is the largest gain and the hardest to establish, because it presupposes knowing the current escape rate. That rate is rarely measured: it is inferred from customer returns, which systematically understate reality since not every shipped defect comes back.The methodological consequence is that establishing the baseline is preliminary work in its own right. Without it, the announced improvement is a hypothesis and the costing rests on an invented figure.

Inspection cost reduction

This gain is the easiest to compute and the most often overstated. An inspection system does not remove human checking, it changes its nature: operators stop examining every part and instead verify those the system flags. The real gain therefore depends on the false call rate, and an imprecise system can generate more verifications than it saves.That dependency deserves stating explicitly in the costing, since it links return on investment directly to the operating point setting covered elsewhere in this cluster.

Indirect gains

Traceability, process data, reduced variability between shifts, the ability to document quality to a demanding customer. These gains are real but hard to monetise, and including them in a return calculation weakens its credibility. Better to mention them qualitatively and base the calculation on the first three.

Establishing the baseline before promising improvement

This is the step most defect detection projects skip, and its absence makes any costing arbitrary.Four quantities must be measured before any deployment. The real defect rate of production, which is not the detected rate. The escape rate of the current arrangement, estimable through double checking on a sample. The current false reject rate, equally rarely measured. And the inspection time actually spent, distinct from the theoretical figure.That measurement costs a few days of audit and has two virtues. It gives the calculation a basis, which is its obvious function. And it frequently reveals that the existing arrangement performs less well than assumed, which moves the discussion onto ground where an automatic system is structurally superior.

The pilot as a costing instrument in defect detection

A well designed pilot is not there to demonstrate that the technology works, which few people still doubt. It is there to produce the inputs to the economic calculation.Four measurements come out of it, and none can be estimated any other way. Real annotation time per class, which gives the cost of the full corpus. The disagreement rate between annotators, which gives the necessary control intensity and therefore the quality cost. Achievable performance on real data, which gives the plausible gain. And the false call rate at the chosen operating point, which gives the residual verification workload.Those four figures turn a theoretical costing into a defensible estimate. It is also why a pilot is designed as a measuring instrument rather than a demonstration: the cases it contains must be representative, including difficult ones, rather than chosen to succeed.

The time structure of the return

The cash flow profile of a defect detection project is distinctive and deserves presenting explicitly.Most of the cost concentrates before any operation: hardware, corpus, development, integration, change management. Gains begin only after switchover, and progressively, since the operating point is adjusted over the first weeks. Between the two, parallel running produces cost without gain, which is normal and must be announced.Two practical consequences. A costing presented as an average annual gain hides that structure and sets up disappointment in the first quarter. And breaking the project into successive scopes, one station then an extension, considerably improves the cash flow profile while reducing risk, since the first station partly funds the next ones.

What makes the return vary tenfold

Five variables explain most of the dispersion between defect detection projects, and identifying them allows a case to be assessed quickly.Product unit value determines what an escape costs. A project on low-value parts is justified only by volume. The initial defect rate determines the size of the opportunity: on already well-controlled production, the possible improvement is small in absolute terms. Position in the chain determines the value added saved by early detection. The current inspection cost, depending on operator numbers and shift pattern, determines the labour gain. And technical difficulty, number of classes, subtlety of defects, product variability, determines the corpus cost.Those five variables can be gathered in a single meeting. Crossing them before committing to a project quickly rules out cases where the return will not materialise, which serves the client as much as the provider.

Buy or build

One structuring decision precedes the costing of a defect detection project and alters every line: adopting a market solution or building a bespoke system.Turnkey solutions reduce development and integration cost, bring proven tooling and support. They suit standard tasks on non-specific products. Their limit is that they generally assume the client supplies the annotated corpus, or pays for annotation separately: the data line does not disappear, it simply moves to another budget.Bespoke development is justified where the task falls outside the standard, where integration is complex, or where the corpus is itself an asset the company wants to own. That last point is often decisive and rarely discussed: a corpus annotated on a proprietary product has lasting value, independent of whichever model exploits it today.In both cases the annotation line remains and is costed the same way. It is the only line no purchasing option removes, which is why it deserves treating first in the costing rather than as an adjustment variable.

The small volume case

One observation deserves stating frankly, because it prevents projects doomed economically.The cost of building a corpus does not fall proportionally with production volume. A small series with demanding quality requirements can therefore present a disproportionate entry cost, whatever the technical interest of the project.Three routes exist in that case and they deserve presenting together rather than in sequence. Grouping several similar references into one corpus, which presupposes an ontology designed at defect level rather than product level. Unsupervised anomaly detection, which learns on conforming parts and sharply reduces the annotation requirement. And postponing the project until volume or value justify it, an option an honest provider should be able to recommend.

The commonest defect detection costing errors

Certain costing errors recur with enough consistency to be anticipated.The first counts the whole inspection time as saved, when part of it persists as verification of flags. The second applies an improvement rate taken from a publication to a situation whose baseline was never measured. The third omits the ongoing maintenance line, producing a flattering first-year return followed by degradation.The fourth is subtler: it counts as gain the detection of defects the current inspection already catches. Only the difference counts, and it is often markedly lower than the system’s raw performance.The fifth and last reasons from a single pilot line and extrapolates linearly to the whole site. Lines differ in product, throughput and equipment, and a system validated on one requires adaptation on the others, whose cost is not nil even if lower than the initial one.

What the client must supply for a reliable costing

A credible defect detection quote depends on information only the manufacturer holds, and its absence explains most wrong estimates.Five elements are needed at an absolute minimum. The list of defects actually encountered, with approximate frequency, rather than the theoretical list from the quality standard. The quality decision rule, expressed with its thresholds, which determines the annotation primitive. A representative image sample, including difficult cases. Baseline measurements of the current arrangement, or acceptance that they will be established. And the intended position of the system in the chain, with or without a downstream check.A provider costing without those five produces an estimate that will not survive the project. Asking for them is not excessive caution, it is the condition of a commitment that can be held.

How to present the costing

The form of the presentation influences decision quality as much as the substance of the costing.Three principles markedly improve the usefulness of a defect detection costing. Present a range rather than a point, with the assumptions that move each bound, which reflects real uncertainty rather than masking it. Separate initial from recurring cost, since they often come from different budgets on the client side. And express gains in physical units before converting to value, a number of parts saved per shift speaking louder than a percentage.A sensitivity analysis very usefully completes the exercise. Showing how the return evolves if the improvement achieved is half the assumed figure gives the client the measure of the risk, and that transparency is generally better received than a single optimistic number.

Return beyond the calculation

Part of the value resists being put into an equation, and saying so beats attempting to cost it.Three elements fall into that category. The ability to document quality to a customer, which sometimes conditions market access without any calculation reflecting it. Building a defect database, which feeds process analysis well beyond part sorting and reveals correlations with machine settings. And reduced variability between shifts and stations, which improves the customer relationship without appearing in any accounting line.These deserve mentioning in a presentation without being folded into the calculation. Including them would weaken the credibility of the whole; omitting them would deprive the decision of arguments that genuinely weigh in an investment committee. The honest formulation presents a calculation based on measurable gains, then states that these three add to it without being quantified.

Who owns the business case

One organisational point determines whether a defect detection costing survives contact with an investment committee: who is accountable for the numbers.A business case built entirely by the supplier is treated, correctly, as a sales document. A business case built entirely by the client’s operations team often lacks the technical inputs to be realistic. The workable arrangement splits it: the provider supplies the measurable technical inputs from the pilot, annotation time, achievable performance, false call rate, and the client supplies the economic inputs, unit values, current inspection cost, escape cost, and owns the resulting figure.That split has a practical benefit beyond credibility. A client who has assembled their own economic inputs understands where the sensitivity lies, and is far better placed to judge whether an underperformance mid-project is a problem or a rounding difference. A client handed a completed calculation has no such grip, and tends to treat any deviation as a failure.

The most common mistakes

These failures recur often enough across inspection projects that naming them is usually enough to avoid them.
  • Adopting a published return ratio without checking its source or scope.
  • Omitting corpus construction, a line often larger than hardware.
  • Not provisioning ongoing maintenance, which is recurrent and indispensable.
  • Promising improvement without having measured the current baseline.
  • Counting the whole inspection time as saved.
  • Counting as gain the detection of defects already caught today.
  • Extrapolating linearly from one pilot line to the whole site.
  • Presenting an average annual gain that hides an unfavourable first-year cash flow.
  • Costing without an image sample or a quality decision rule.
  • Committing to a small-volume project without examining alternatives to exhaustive annotation.
  • Treating the annotation line as an adjustment variable when no purchasing option removes it.

When the answer is no

A costing method is only useful if it can produce a negative answer, and a provider willing to state one is worth more than one who never does.Four situations recur where the numbers do not support a defect detection project. Low unit value combined with low volume, where no plausible improvement covers the entry cost. An already well-controlled defect rate, where the remaining opportunity is too small in absolute terms. A product changing faster than a corpus can be built, which makes the data investment perpetually obsolete. And an absent or unmeasurable baseline, where no improvement can be demonstrated and therefore none can be paid for.Saying so early costs a sale and buys something more valuable. A client told frankly that their case does not stand up will return when volume, value or maturity has changed, and will bring the recommendation with them. A client sold a project that fails will attribute the failure to the technology and to the provider, in that order, and will not return at all.

What to take away

The return on investment of a defect detection project follows from no published ratio. It is computed from five site-specific variables, from a measured rather than assumed baseline, and over a cost scope extending well beyond hardware.Three decisions structure a defensible costing. Measure the current arrangement’s performance before promising improvement, which costs a few days and underpins everything else. Use the pilot as a measuring instrument rather than a demonstration, since it produces the four figures the calculation needs. And present a range with a sensitivity analysis, rather than a single point whose apparent precision is false.For approaches, ontology and annotation tasks, the complete guide to defect detection sets the frame. For the question of protecting production images, which governs whether outsourcing is feasible at all and represents a cost line in its own right, the article on securing production images covers the applicable requirements.To explore delivery arrangements, supported formats and applicable control mechanisms, see our dedicated page on annotation for industry. And if you are considering an inspection project and want a costing built from your own data rather than a generic ratio, let us discuss your project.
Tags

Découvrez nos articles