Making it possible: automated cosmetic inspection
How artificial intelligence mimics human expertise
Successful brands are aware that maintaining their image requires maintaining their quality standards for every product batch. For complex and delicate products such as lenses, this requires very precise inspection, since even smallest deviations from the optimal set up may cause irregularities at the lens surface. And even when the optics might remain correct, cosmetic defects highly affect the lens quality and thus matter for the overall brand quality. To date, trained human experts have the sole responsibility to check each lens for surface irregularities. This purely manual process is strictly regulated by each lab’s in-house quality standards. It is both labor intensive and dependent on the inspectors’ perception. With the development of a smart system – that can automate the process with the help of artificial intelligence (AI) – this has changed. The system takes over a vital role in supporting the quality control: it inspects lenses and mimics the companies’ individual decision-making patterns to reflect a labs’ unique quality standard.
Every lab strives for high-quality lenses and high throughput to build and maintain a strong brand with economic success. However, no production process is flawless, and even minor deviations from the optimal conditions may have a negative impact on the optical character of a lens – or may cause cosmetic irregularities. Therefore, every lab has quality control as part of their lab routine.
Until now, this cosmetic inspection has been a purely manual process: Cosmetic inspection is usually done using dark field inspection devices, arc lamps among other tools. Operating under these conditions is extremely exhausting for the human eye and demands consistently high focus. Still, each quality check is expected to be done in a matter of seconds for the lab to remain productive.
Aligning and standardizing cosmetic inspection is very challenging: An inspector’s verdict may deviate from another’s as no single eye is like the other, and despite looking at the same lens, the verdicts may not always be the same. Maintaining a high-quality standard on this basis is very demanding.
In addition, a time-consuming manual report is needed to collect every job’s result in the system and determine good quality lenses versus ones that are sorted out. These reports simply contain the inspection result such as go, no go or rework as well as the defect type.
A more detailed documentation containing images of the lenses for example is not deployed. With this limited and manual data collection, a statistical breakage analysis – which would allow to draw a conclusion on the optimization of the lens production – is not yet possible.
As a result, it has been a desire to automate this important quality check with the intention to facilitate decision making at a new level, based on data, to ultimately standardize the process.
While to date automating cosmetic inspection was considered impossible, this has changed with the development of a fully automated system that can analyze lenses and evaluate their surface quality with the help of AI.
The smart system sees what only trained experts have been able to see so far: It screens the surface for any irregularities, characterizes and evaluates them consistently and without human bias. And even more, it really supports lab work by taking the final decision on how to proceed with the respective lenses.
“Our system works with a supervised neural network,” explains Gunter Schneider, President at Schneider. “This is inspired by the human brain and can be categorized as deep learning.” The system is trained by Schneider’s own AI and lens quality experts. The required data base for training is built from thousands of reference lenses.
Trusting AI-based pattern recognition: The computer vision approach
Whenever an intelligent system is employed at a vital position – and in this case ensures a lab’s quality standards – it is reasonable to desire an understanding of how this machine can learn to consider multiple criteria and finally take over a major decision.
Schneider’s cosmetic inspection system comprises three main steps: the image processing, the defect detection and the decision making. At the beginning of a new inspection, raw images are being taken automatically inside the system, before the data analysis can start: Within seconds, the smart neuronal network calculates and evaluates the image data based on its knowledge, which is compiled in the existing data base.
This first facts-based analysis considers universal standards, unaffected by individual assessments. It builds the consistent basis for any lab. “During this lab independent process, the system will recognize any occurring irregularities and define what kind of defect it is, in which area of the lens (zone) it is located and how distinct it is,” explains Stephan Huttenhuis, Vice President Technology.
Technically, the new cosmetic inspection solution detects scratches, pits, center dots, haze, chatter, cutting marks, spiral, fringes among many other defect types. Even dust particles, which can simply be cleaned, can be distinguished from severe defects. It can easily measure polarized lenses as well as bifocal lenses.
What sounds logical and easy to understand gets more complex, if we consider that almost no defect looks exactly the same. But if every scratch is just a little different, how can one be sure that the intelligent system doesn’t miss any unknown irregularity?
An in-depth understanding of the high-end image analysis might help to erase this worry: “The AI-based pattern recognition is a method that analyses each pixel of an image. It literally decomposes the image into fragments and assesses for every pixel, in relation to the other pixels, if it shows a defect or not,” says Huttenhuis. Therefore, the system does not need to recognize a complete defect, but it detects it piece by piece.
This is extremely worthwhile confirms Schneider: “Deep learning methods, such as neural networks, are replacing classical algorithm-based methods as a superior form of image analysis, since it allows more accuracy compared to the rather simple structure of algorithms.”
The smart process: How a system mimics the individual company quality standards
However, this basic analysis will not yet provide an immediate answer how to proceed with a respective job. It lacks the lab’s individual perspective on the matter. “Just like a human mind, the system needs contextual knowledge: Get to know the specific lab that is implemented in, their overall expectations and even the brand’s image and use of the lenses,” says Schneider.
This is because every brand has their own quality requirements and production standards. So, whether a defect is acceptable or not depends on many aspects, for example the defined criteria for the final product and how the production process will continue.
Is the defect in an area of the lens that is going to be edged off anyway? Will a minor scratch still be visible at all after hard coating?
The cosmetic inspection system goes beyond the pure recognition of the defects. Labs can implement truly individual standards to perfectly cater to their own quality requirements. “This simply requires further human input on lab-dependent criteria.
During the implementation of the system, labs can teach the system by feeding it with their individual decisions. With this input the system learns – supervised by a Schneider expert – to really mimic their decision-making, all while eliminating variability” explains Huttenhuis.
With this added information the cosmetic inspection system can better understand and further refine its perception of whether the job is a go, no go, or requires ‘rework’. The system considers what kind of defect, in which combination, in what intensity and in which zones passes or not. “It really proves to be trustworthy to ensure a labs’ unique quality standard,” Schneider says.
Economic benefit of AI-based automated cosmetic inspection
Finally, this system allows automated quality check 24/7 with standardized evaluation even across different brands or production labs.
This is not only beneficial to assure and comply with quality standards of outgoing goods, but there is even more to it: The cosmetic inspection system can be installed in-between different production steps, after the surfacing steps are completed or for a final quality check after coating.
This implementation in the production cycle will significantly reduce costs: Low quality lenses can be identified as breakage and sorted out at an earlier stage and subsequent finishing processes are avoided.
Being integrated in Schneider’s Modulo system, the smart solution will even serve as an overall assessment to help to detect and solve production problems and fix errors that have caused the occurring irregularities early on, such as worn tools etc., when connected with and reporting to the smart Management Execution System.
Outlook: Future perspectives on the future lab
This best-case scenario proves once again how AI is being introduced to the ophthalmic sector and comes along with a huge success factor. “It really marks another huge step forward towards a smart lab,” says Schneider. And they are already working on the following step to extend the functionality of the inspection system.
“Next, it will be possible to automatically measure the diopter of a lens and thus it’s optical effect.” What they could achieve in cosmetic inspection so far is just one example of what Schneider strongly believes in from a holistic point of view: “AI can further revolutionize the ophthalmic sector – and we will follow our vision for a smart and fully automated future lab.”