Deep Learning based Computer Vision System for Automated Tyre Defect Detection

Business photo created by freepik.com

As the global manufacturing industry faces the pressures of bringing to market multiple variants of the highest quality products in the shortest possible time, a shift towards AI-driven automation across all functions has become inevitable.

In quality inspection, AI-driven computer vision systems are already enabling the streamlining of the production process so that the products are compliant with the quality criteria set by the company. This in turn brings in advantages of greater efficiency, lower operational costs while enabling 24/7 production and quicker decision making.  

Global Tyre manufacturers have been one of the early adopters of AI-powered technology in various areas including quality assurance. One major application of AI here is defect detection in tyres using deep learning-based computer vision systems. Due to the nature of raw materials used in the tyre manufacturing process, tyre components may be contaminated by metallic or nonmetallic impurities (e.g., steel threads, screws, and plastic fragments), bubble, and overlap. When a vehicle with a defective tyre is run at high speeds, these defects lead to reduced lifetime or even blowout of the tyre.

Background 

The Indian tyre industry manufactures 150+ distinct types of tyres with diameters varying from 8” to 19” and width from 400mm to 1000mm for applications ranging from bikes to cranes. To add to this, they are required to detect 80+ classes of tyre defects and are constantly struggling with a dearth of trained human resources. 

Their challenges with human resources are manifold – shortage of experienced manpower, time taken in nurturing new resources to the expertise level required, person dependency in identifying the faults and repeatability as teams change. To overcome these challenges, tyre manufacturers need an automated solution for tyre quality inspection and defect detection.  

In one of our customer engagements, Ignitarium engineers discovered that the major problems to be solved, to accurately detect defects in tyres were:

• Mimicking the manual test and integrating to the existing manufacturing and quality control process flow
• Having a mechanical fixture which will auto adjust to the varied tyre sizes placed by an operator and also rotate so that internal and external walls of the tyre are captured in one rotation.
• Capturing images at an optimum speed of rotation, processing, analysing and detecting the faults. 
• Stopping at each faulty location and marking the defects on the tyre
• Easy and safe release of the tyre

Type of defects in tyres

 Out of the 80+ classes of possible defects commonly occurring during tyre manufacturing, segregation is done to create severity sets. Various tyre manufacturers have their own specific categorisation methods based on their unique chemical recipes and mechanical manufacturing processes. Usually around 30+ of these classes are considered to be of high severity. A limited sampling of these defect classes are depicted in the images below:

Fig 1: Tyre with Defects: Left to Right: SWC – Side Wall Crack, RLC – Rim Line Crack, SWB – Side Wall Blister, Stencil number offset, Chamfer defect

Tyre Defect Detection

  A typical method of tyre quality inspection is to use X-ray machines and to use techniques like Weighted Texture Dissimilarity. However, X-ray machines require a lot of space and are very expensive. Instead, we used a RGB camera to capture images of the tyre. To solve tyre defect detection problems, instance segmentation (combination of object detection and segmentation) was chosen, so that each instance of each defect featured in the image could be identified, instead of categorizing each pixel like in semantic segmentation.

Data Generation

For data generation, images captured from multiple tyres with already identified and marked defects were used. A 1080p resolution camera was used to capture videos of faulty and ‘good’ images. Image capture was done from various angles to improve generalisation.

Data Preparation

Data preparation is the process of cleaning and transforming raw data prior to processing and analysis. It is an important step prior to processing and often involves labelling data, reformatting data and making corrections to data. The data was collected by dumping frames from several generated videos. Then the defect portions within the region of interest (ROI) were labelled using labelling tools like LabelMe.

Instance Segmentation

Instance segmentation is a technique in which an object is detected and then masked at an individual pixel level. It combines 1) Object detection, in which each individual object in a frame is classified and localized and 2. Semantic segmentation in which each pixel is segregated into pre-defined classes. Instance segmentation allows a label to be attached to each pixel of an image.

Mask-RCNN

Mask-RCNN is a deep neural network that addresses instance segmentation problems in computer vision. Mask R-CNN is a combination of two well-known network topologies –  Faster R-CNN and a Fully Connected Network (FCN). Mask-RCNN follows a 2-step process. In the first step, for each input image, proposals are generated about the regions where an object might exist. In the second step, the class of the object is predicted and a pixel level mask is generated for the object based on the proposal generated from the first step. 

[Source Link]

Implementation

We build our inference pipeline using a customized variant of Mask-RCNN. The base Keras implementation can be found at https://github.com/matterport/Mask_RCNN.git and the corresponding reference paper can be found at https://arxiv.org/pdf/1703.06870.pdf 

Inference Time

Inference time of the un-optimized model is listed in the following table. 

Architecture	Environment	CPUTimings	GPU Timings(Quadro P4000)
Mask-RCNN	Keras-TF (640*480)	5 sec/image	3sec/image

With model optimization techniques, the per image latency can be reduced to less than 150ms.

Accuracy

Our customized model was able to achieve a detection accuracy of 96%.

Conclusion

This R&D effort to detect defects in tyres indicates that it is possible to achieve significant automation of the tyre quality inspection process. In a practical system, where high inspection throughput is mandatory, an automated inspection jig would be called for. This would include multiple cameras to simultaneously view the outer and inner surfaces of the tyre, a motor assembly to rotate the tyre 360 degrees and a control system that can synchronize the tyre movement, image capture and detection.