We automated an insurance damage report process with Machine Learning and Machine Vision solution
We had the opportunity to demonstrate image processing automation and combine it with a response system (chatbot). Our client is a multinational insurance company and the endproduct is a software which makes damage reporting a smoother process. The solution could be used in a wide range of cases within the insurance industry, and anywhere else where there is a steady stream of incoming digital images.
The insurance company knocking on our door identified a problem. Their clients upload the images of the damage on their cars, but these photos need to be checked and filtered by human experts, who obviously are way too skilled to waste time with something so menial. A huge amount of high quality human capacity could be freed up here, if they could somehow automate this task.
Needs and requirements
The software they had in mind needs to be able to identify if a photo is not meeting the requirements.
The images need to be uploaded in a certain sequence:
Right Front / Left Front / Right Rear / Left Rear / Damaged Area / Chassis Number / Dashboard (mileage)
This where the clients can make an easy mistake, and upload the images in an incorrect order.
There are some other criteria:
- the complete vehicle must be visible on every image
- licence plate has to be legible when it is visible
- the same car has to be on all the images, obviously
When these parameters aren’t met, the app needs to ask the client to re-upload the correct photo.
The system needs to be able to scan and register the licence plate number automatically and the vehicle should get a unique ID number for the insurance database.
After identifying the challenges, we came to the conclusion that an image analytical algorithm prepared with machine learning would be able to perform these tasks without any problems.
We add a UI complete with a chatbot and there we have it: a complex damage report application.
The company told us that identifying the brand, model and color of the vehicles would be a welcome addition to the features of the software. There are working solutions for car brand identification available already, and for colors as well, however, only down to basic tones like ‘light blue’ or ‘dark green’. So we responded with telling them that we see no trouble with adding these features as well.
So we started taking pictures of cars. A lot of them.
We wanted to build a database and tag the images to start the training of our machine learning algorithm: identifying the angle, confirming that the car on the image is a match, and reading the licence plate. We shot multiple pictures of 850 vehicles, and successfully achieved validation capability, with close to 100% efficiency.
Our software rotates the licence plate in 3D, crops it, saves it, reads the numerical data and files it in the appropriate spreadsheet of the insurance company. It does the same with the mileage photo.
The brand / model / color identification module has also been integrated and we created a UI with a chatbot which leads the user through the whole process, step-by-step.
The demo presented to the client was able to:
1. Find and identify the car on the photo
2. Check if the full body of the vehicle is visible, ask for a new photo if not
3. Identify the angle from which the photo was done and ask for a new photo if another angle was required at this step
4. Find the licence plate on the image, crop it, rotate it for a full frontal view and read it with an OCR solution
5. Do the same with mileage and the vehicle ID number
6. Check if the car on the image is the same as the one on the previous images.
7. Identify brand, model and color (basic color names).
8. Add a unique ID to the vehicle
The app performs these tasks automatically, and it’s possible to add new features later on.
This use case is a good example of deploying AI for filtering, checking and data gathering where user generated images are part of the full process.
The possibilities are limitless. This method could be used in any case where buying and selling used products or real estate comes into play, or where photo ID is an important requirement, like, when we want to rent a car or electric scooter. As we mentioned in our previous articles about machine vision, AI can be a useful tool in every process which involves humans checking images against a set criteria.
Read more about Machine Vision:
