By Swapnali Bagal, Associate Data Scientist at AlgoAnalytics
With the recent advancements in Deep Learning, new applications of object detection in image processing are emerging in multiple domains. One such novel application of image analysis is detecting external damages on vehicles, for insurance as well as repair purposes. Various techniques in Deep Learning can be used to not only detect damages on automobiles (such as scratches, dents, broken glass, damaged body panels) but also to estimate the severity of damage and estimate the repair costs. The cost estimation is a critical part for the insurance industry. Such automated damage detection and cost estimation techniques can both save time (these can generate results in an instance — much faster than a manual survey and inspection process), and costs in the insurance survey process. Since one needs only the images of the damaged vehicles, such techniques can easily be used remotely, without having the surveyor visit the location of the vehicle. At AlgoAnalytics, we have our own AI model and tool developed for automated and fast damage detection.
The Two-Step AI behind Damage Detection:
To find the damaged parts of the vehicle, we use a custom trained MaskRCNN model using transfer learning on the COCO dataset model. Our model covers 5 types of damages : Scratch, Bumper Dent, Door Dent, Glass Broken and Smash.
1. Object Detection:
Object localization is done to identify the location of damaged parts in an image and draw a bounding box around each detected object. For object detection, MaskRCNN uses FasterRCNN which combines object localization and classification for generating bounding box and class for each object. In FasterRCNN, an image is inputted to CNN to get a convolutional features map. Region Proposal Networks (RPN) are used on feature map to predict regions proposals to be used by RoI Pooling for predicting class and bounding box of object.
Other techniques for object detection range from You Only Look Once (YOLO) [3], Single Shot MultiBox Detector (SSD) [4] and Fully Convolutional One-Stage Object Detection (FCOS) [5]. YOLO is based on the CNN approach and eliminating background detection. SSD takes only one shot to detect multiple objects in an image similar to YOLO, but it uses multiple sized feature maps. The above two are object detection methods based on anchor boxes. In contrast, FCOS does not use any predefined anchors. It is a one stage object detector, leading to avoid complex computations.
2. Instance Segmentation:
Along with bounding box generation and classification, MaskRCNN also predicts segmentation masks in pixel to pixel manner using Fully Convolutional Network (FCN) on each Region Of Interest (RoI). Binary segmentation is performed in each bounding box to separate the object as foreground from background.
Some researchers also use the technique of focusing on a pixel by formulating instance segmentation as a bottom up pixel assignment problem[6], instead of the top-down approach of first detect and then segment.
Some example cases with their outputs given by our model are shown in Figure 1
Figure 1 Car Damage Detection example cases
Challenges in Vehicle Damage Detection:
Because the real-world vehicle damage photographs are taken in an uncontrolled environment and vehicles have reflective metallic surfaces, images may have reflections which may be misclassified as damage. One research [7] uses a joint approach of semantic and instance segmentation to handle mirror reflections. Two stage object detectors such as MaskRCNN are susceptible to the loss of broader contextual features. This joint approach reintroduces broader context to instance segmentations. It identifies reflections automatically without explicitly labeling reflective regions.
Another challenge is an occasional false positive, which would mean an undamaged vehicle identified as damaged by the AI model. The number of false positives can be reduced to some extent by a two-tier model. The first tier model will only do binary classification on image. If the image is predicted as damaged, then the second tier model will identify the type of damage.
Use Cases by Vehicle Insurance Companies: Image based vehicle insurance processing is an important business use case with good potential for automation. When a vehicle gets damaged in an accident, the insurance company has to cover the repair cost. However, the estimation for the damage and the costs involved happens in the automobile workshop. This process is manual, it is slow, and depends on the subjective analysis of the surveyor. This leads to several delays as well as errors in the process. These issues can be eliminated by using AI based automatic damage detection. Automatically detecting car damage using photographs taken at the accident scene can reduce the cost and time of processing insurance claims. Such automated AI based systems can be used by the insurance companies for quick estimations as well as quick claim processing, benefiting both the insurance companies and their customers.
As mentioned, we at AlgoAnalytics have developed our own AI model and a system for vehicle damage detection. Our model outputs can be seen in the link below
For demo visit our link https://algoanalytics.com/demoapp
For further information, please contact: info@algoanalytics.com
References:
[1] Mask R-CNN URL: https://arxiv.org/pdf/1703.06870.pdf
[2] Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks URL: https://arxiv.org/pdf/1506.01497.pdf
[3] You Only Look Once:Unified, Real-Time Object Detection URL:https://arxiv.org/pdf/1506.02640.pdf
[4] SSD: Single Shot MultiBox Detector URL: https://arxiv.org/pdf/1512.02325.pdf
[5] FCOS: Fully Convolutional One-Stage Object Detection URL: https://arxiv.org/abs/1904.01355
[6] Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth URL: https://arxiv.org/pdf/1906.11109.pdf
[7] Detecting Reflections by Combining Semantic and Instance Segmentation URL: https://arxiv.org/pdf/1904.13273.pdf
