A Brief Look at Autumn Trade Fairs. Part III: Product Sorting, Defect Detection and Completeness Verification
The aim of our next product demo was to simulate possible use of PEKAT VISION software in a manufacturing environment, where different products move on a conveyer belt, are sorted and inspected for defects and missing parts. To show the universality of our software, we used products from the electronics, pharmaceutical, and cosmetics/ personal care markets. It is probably not a realistic use for any one manufacturer, but highlights the versatility of application of PEKAT VISION.
In the next paragraphs, we will briefly describe what an onlooker could see, in the following ones, how it was achieved using the PEKAT VISION software.
Objects: PCB, pill blister, and a small personal care kit containing a medical ampule, a dental floss, and a night cream
PEKAT VISION modules: Deep-learning Detector and Classifier
Hardware: Wahtari nCam Smart Camera with Jetson Xavier NX AI Accelerator including LED lighting and motorized lens in IP67 housing
Overview of the Inspection Process
We used three sets of items: a PCB board, a pill blister, and an open personal care carton box containing a night cream, a dental floss, and an ampule. All three sets contained a flawless sample as well as samples that were either damaged or incomplete.
The samples were placed randomly, or based on interest, on the conveyor belt and moved along. When they reached a certain point, a trigger placed on the side of the belt sent an impulse to the camera mounted above the belt to take an image. The image was subsequently sent to the PEKAT VISION software for evaluation. From PEKAT VISION instructions were sent to a PLC and from there to a signaling beacon. Either a green light, for OK samples, or a red light, for NOK samples, was turned on. Simultaneously, an image of the sample was displayed on a monitor, showing either an OK sample in a green frame, or NOK sample in a red frame highlighting any defect, or missing parts.
In a real-life scenario, an impulse from the PLC would be sent to a robotic arm to remove the damaged/ incomplete product from the conveyor belt, or the belt would be stopped and the item removed manually. The conveyor belt would then have to be switched back on. The goal of the demonstration was to identify the damaged and incomplete samples using PEKAT VISION Detector and Classifier modules.
Defect Detection and Classification – A look Behind the scenes
PEKAT VISION offers four main modules: Anomaly Detector, Surface Detector, Detector and Classifier, and OCR/ OCV (Optical Character Recognition/ Optical Character Verification). Often times more than one module can be used for a particular task. The modules can be combined and interwoven with coding. It is our feasibility team working closely with our customers who usually decide on the best course of action. For many applications PEKAT VISION software is ready to be deployed as it is. For our scenario, however, it required some simple coding for optimal results.
First, sample images from each set of products had to be uploaded and annotated. As opposed to the Anomaly detector, the Detector and Classifier modules we opted for, require images without defects as well as with defects. The samples, the individual components of the samples, and their state had to be annotated. Subsequently, PEKAT VISION trained itself to recognize the samples, its components and their state (OK/ NOK). Following are the steps that had to be set for the inspection to run smoothly.
As the scheme above indicates, the image was first cropped to the area of interest (Preprocessing, STEP 1). Next, it was split into two branches: one that remained unmodified, Branch 1, and Branch 2 that was processed in order to establish, in as little time as possible, what product was moving on the conveyor belt – STEP 2.
STEP 2 comprised of three measures – pre-processing, classifying, and coding. Using the Preprocessing feature of PEKAT VISION, the image was resized to a lower resolution in order to increase the processing speed. Using the Classifier module, the image was assigned to one of three classes representing the three sample categories – a PCB, a pill blister, or a carton box containing personal care items. In the last step we used a Python script to send the information about the sample from Branch 2 to Branch 1 containing the original image.
STEP 3: Using a simple script, the original image from STEP 1 was sent to its appropriate branch “PCB“, “Pill Blister“, or “Carton Box“ based on the identified class in STEP 2. In all three branches, the image was examined using the Detector and Classifier modules. The individual components/ parts were detected and classified based on their state. Accordingly, the whole sample was classified into the OK or NOK class.
Looking at the example of the pill blister, each pocket of the blister was marked as either full with the correct and intact pill, full with a bad or damaged pill, or as empty. If the sample contained any pocket that was marked as anything other than full with a correct and intact pill, the entire sample was classified as NOK.
In a similar way PEKAT VISION proceeded in the case of the PCB boards and personal care kits. We used an additional script at the end of the PCB branch to highlight any missing part on the monitor. An extra script was added also to the personal care kit branch in order to further increase the processing speed.
The whole process, from the trigger sending the first impulse to the light being turned on, took well under a second. PEKAT VISION is designed for industrial use and is well suited for manufacturing of large batches at high speed. If you are concerned whether PEKAT VISION would be suitable for your specific need, please contact us or request a demo.
We would like to thank the Wahtari team for providing us with their new nCam Smart Camera that helped us greatly to achieve these results.
