News

Professor Hideyuki Sawada (Faculty of Science and Engineering)

“Before the Ukraine conflict, it was said that there were more than 100 million landmines remaining in the world, and that number is now believed to have increased. The ICRC, which has been working on removal, began research into an efficient, easy-to-use mine detection system using ICT in collaboration with the Waseda Research Institute for Science and Engineering, which specializes in AI and robotics. I had been researching drones and image recognition, so I decided to cooperate with them because I wanted to apply this research to support mine removal.”

One of the reasons why traditional removal methods are risky is that obstacles such as terrain, vegetation, and scattered metal fragments make it difficult to accurately detect landmines. In order to improve safety and efficiency, a method that uses drones to take images of the ground and detect the location of landmines using AI has attracted attention, but it is not easy to reduce non-detection and false detection. Professor Sawada focused on cameras mounted on drones.

“Today’s machine learning mainly uses RGB cameras to recognize images, which capture color information. However, landmines buried underground cannot be identified as images, so RGB cameras are ineffective. The same is true for landmines scattered on the surface if they are covered with sand, dust, or vegetation. To solve this problem, we approached a method using an IR (infrared) camera.”

Thermal sensing and machine learning improve accuracy of landmine detection

By equipping a drone with an IR camera that uses the properties of infrared light to image thermal energy, it becomes possible to take IR (infrared) images. Professor Sawada is developing a system that uses machine learning to recognize the location and type of landmines based on the captured IR images.

“The technology we use is thermal sensing, which detects and visualizes the thermal energy radiated from an object. Differences in the thermal energy of objects appear as differences in brightness values in IR images. An object’s emissivity changes depending on physical properties such as the material and temperature, while its thermal conductivity changes depending on the humidity of the object and its surrounding environment. If we can distinguish between landmines and soil or vegetation based on differences in brightness values obtained from IR images, we can estimate their location even if they are buried somewhat underground, and also distinguish the type of landmine.”

The research is also being carried out on-site, with experiments in northern Denmark, where landmines were buried about 1cm underground and images were taken from the air by a drone equipped with an IR and RGB camera. A dataset was then created from the collected images, and machine learning was carried out.

An experiment using a drone

“Using machine learning to quickly and accurately process large amounts of data has the potential to significantly improve mine clearance operations. We believe that a convolutional neural network (CNN), which excels at recognizing the characteristics of objects in images, would be effective, and we will continue to build a recognition system using deep learning (DL).”

DL is composed of three layers: “Backbone,” “Neck,” and “Head,” each of which performs a specific function, enabling it to quickly analyze and extract the characteristics of objects in images.

“In the Danish experiment, a dataset was created from 398 images and machine learning was performed. 81 images were then input and the system was asked to predict the location of landmines. The system was able to make correct predictions for all of the images. Similar experiments were also conducted in Jordan, with similar results. However, some results were found to have low confidence scores and surrounding vegetation was mistakenly detected as landmines. The assumed cause of this is that emissivity and thermal conductivity vary depending on soil conditions such as surface temperature and humidity, resulting in differences in brightness values in the IR images.”

Analyzing IR images to identify mine locations

In order to improve the reliability score and reduce false positives, it is necessary to increase the amount of image data used for training and the types of landmines to be labeled.

“In machine learning, performance depends on how much data is collected and used for learning. In the future, it will be necessary to acquire IR images taken under various conditions, such as burial depth, shooting altitude, weather, and temperature, and add them to the learning dataset. In addition, we will need to label objects other than mines and environmental variables that can cause false recognition. As a first step, we have been able to demonstrate the effectiveness of installing an IR camera on a drone and machine learning based on IR images, so in the future we would like to work on improving accuracy.”

Drone and AI technology that can be applied to a variety of sites

Professor Sawada’s research project requires immediate implementation. What kind of research activities will he pursue as the next step?

“The first step is to collect data. We are currently working with the ICRC to prepare to conduct on-site testing once the conflict in Ukraine ends and safety is secured. In areas where the conflict is over, it will be necessary to restore infrastructure and housing, but in order to do this, we must first remove landmines and prepare an environment in which work can be carried out. We will need to pool together technology from around the world, including ours.”

Professor Sawada hopes that the knowledge gained from this project can be applied to solving other social issues.

“This technology can be useful not only in removing weapons such as unexploded ordnance, which has become a hot topic in Japan, but also in times of disaster. For example, when rescuing people from houses that have collapsed in an earthquake, being able to detect the temperature emitted by the human body would enable a swift response. The key to removing weapons and saving lives is the initial ‘discovery’ process. We will continue to conduct research and development in the future, integrating robotics and AI technologies to provide solutions tailored to each problem.”

Photo taken at Nishi-Waseda Campus, Building 55