1. Introduction

Humanitarian demining continues to pose a growing challenge to communities in emerging or underdeveloped countries. Various anti-personnel (AP) landmine detection techniques have been developed. This paper briefly reviews handheld sensor fusion for landmine detection with a focus on handheld sensor systems that combine electromagnetic induction (EMI), which is the metal detector (MD) and ground penetrating radar (GPR). These sensors are the most cited and widely used. The review supplements the review in [1] and proceeds further to focus on research and development efforts made in the previously mentioned aspect in the last several years with an emphasis on handheld dual or multiple sensor systems that are matured technologically or possess a high technology readiness level (TRL) according to [2] and are commercially available. Table 1 summarises some of the major detection technologies [3] and indicates handheld portability and commercial availability. Other handheld sensor fusion methods that may be currently under development or alternatives with confidential technical information are not considered. Section two provides an outline of sensor fusion for landmine detection and a review of MD and GPR sensor fusion is presented in section three. A discussion of the review findings is presented in section four followed by a conclusion in section five.

Table 1. Landmine Detection Methods

2. Sensor Fusion for Landmine Detection

The limitations of individual or single sensor systems in the detection of mines and the need to increase the detection metric (POD) and reduce false positives (FAR) especially for humanitarian demining has led to the exploration of data fusion [4]. Data fusion generally refers to the combination of different sources of information with the aim of system performance optimization. In landmine detection application this is commonly referred to as sensor fusion which is the deployment of two or more sensing techniques in the same location or area of interest and the data from all the sensors are considered sequentially or concurrently for target discrimination. The fusion component specifies the combination of the different data acquired from a number of sensors also referred to as the abstraction level [5]. However data fusion could also refer to the combination of multiple interpretations of data from a single sensor by operators or experts that culminates in a single result [6].

2.1. Types of Sensor Fusion

Sensor fusion can be classified into three levels which include the data level, feature level and decision level [7].

Data level fusion refers to the combination of different aspects of data from a single or set of similar sensors. This type of fusion in landmine detection applications typically occurs with several sensors that produce identical data with variations only in aspects such as wavelength range and polarization [8, 9]. An example is that of camera images which requires obtaining different data or images and converting such into a single image for interpretation.

Feature level fusion combines features of a target obtained from different sensors such as dimensions, shape, texture and material. This method fuses all the features data by linking them or choosing features from individual sensors. The features are either subsequently formed into a single result (feature extraction) or used to determine detection through classification (feature classification).

Decision level fusion combines all the detection data or decisions obtained from multiple sensors. This typically occurs when feature extraction or classification is limited or not possible. The data considered from the sensors is raw data or other outputs such as waveforms or quantitative data that can be used by operators to determine the POD. There are various methods of decision level fusion with differences also at the input or output sub-levels of these methods. Examples are the Bayesian approach, fuzzy logic, rules and voting methods [4]. Decision level fusion also includes the fusion of multiple signal processing schemes applied to data obtained from a single sensor [6].

The aforementioned sensor fusion levels are generally implemented based on the type of sensor in question or under test. Qualitative data or other detailed information obtained from different sensors which can be merged into a single data set that enables the determination of target or object features would be most suited to feature level fusion whereas data fusion would be best applicable to cases with data obtained from similar types of sensors. On the other hand, different types of data obtained from equally different sensors could require the application of decision fusion. Decision fusion enables different sensing techniques and the data they produce to be combined separately or simultaneously by multiple operators or technical experts.

2.2. Challenges of Sensor Fusion

There are a number of factors or technical considerations that make accurate sensor fusion difficult to achieve [10].

1. Sensor positional data must be available with high confidence levels to ensure that data capture of the same area or target of interest is definitive. Data obtained from the sensors must correspond to equivalent physical points.

2. Sample data points in many cases are not congruent. This is largely caused by the deployment of sensors on different platforms as harmonisation into a single system can be non-viable.

3. Mobile sensor platforms are common sources of positional errors and hence fusion algorithms must possess the capability to sufficiently compensate for such errors.

4. Mine detection sensors typically produce distorted target position approximations which could be multi-dimensional. Earth observation sensors provide accurate surface data and unfocused depth data. Distortions arising from differences in sensor data sampling can lead to errors in data fusion. For instance GPR sensors sample depth and along track simultaneously but MD sensors are incapable of sampling in both dimensions.

3. MD and GPR Sensor Fusion for Landmine Detection

Sensor fusion methods have been well researched for landmine detection in the last few decades in order to improve demining operations. The most widely studied and cited are systems combining GPR and a MD. Few exceptions such as heated waterjets and infrared (IR) sensors [11] and MD and gas fused sensor [12] are available theoretically. As previously outlined in the introduction, the two aforementioned sensors are the most cited and also the most technologically mature and commercially available for use particularly in humanitarian demining. Therefore majority of sensor fusion methods for landmine detection utilise these sensors, with the possible inclusion of other sensors. Typically, the MD measures returns from any metallic objects while the GPR is able to detect non-metallic objects or other materials, compensating for the former’s inability to detect non-metallic mines. Here we present a review of some of the research into handheld sensor fusion for landmine detection based on the MD and GPR in the last couple of decades. Our review focus is on the type of fusion level utilised or method of sensor fusion employed and the final outcome. Specific design parameters or details of the mine, soil or datasets are not considered.

3.1. MD and GPR Sensor Fusion Review

A dual MD and impulse GPR sensor fused system that builds images also from the MD is presented in [13]. For the imaging aspect, the metal detector is scanned horizontally in two different orientations of the detector head to obtain approximations of a horizontal and vertical gradient of the received magnetic field. The data is used to produce 2D magnetic gradient field images of an AP mine, rock and metallic debris. The GPR, operating at 800MHz to 2.5 GHz, is used to produce two-dimensional (2D) images of the same objects as that of the metal detector and the images from the two sensors are fused and analysed. The results demonstrated improved feature classification in comparison to the individual sensor images. In [10] sensor fusion that comprises a frequency-stepped continuous wave (FCW) GPR with a frequency range of 1-6 GHz, a MD and an IR camera is presented. The different sensor data are acquired and processed separately before fusion is performed. The GPR data is processed and converted to time domain using inverse Fourier transformation (IFT) and the energy and time spectra used for mine feature extraction. The time-domain waveforms of the MD are used for the data fusion. The IR camera is placed atop a nearby building. A feature level sensor fusion algorithm was developed and applied which yielded a high tolerance to unaligned data samples from the sensors. Decision level and feature level schemes are tested with the HSTAMID in [14]. The sensors fused are GPR, MD and IR sensors. The GPR consists of a bistatic antenna operating at a 500MHz to 3 GHz frequency range. Thermal images from the IR camera and the output signal waveforms from the MD and GPR are fused sequentially and also concurrently in a staged approach in pairs. The results showed that FAR was improved distinctly with combined sensor data. Decision level fusion is also experimented with the Handheld Standoff Mine Detection System (HSTAMID) in a different study to demonstrate the achievement of enhanced detection performance using multi-sensor algorithms as opposed to single sensor algorithms [15].

A study [16] is presented that compares the performance of feature level fusion and decision level for data obtained from a MD and GPR dual sensor fused system. The two sensors are collocated on a platform and a scanning procedure is used to acquire raw data from each sensor separately. Feature information of surrogate mines buried in the ground is extracted from each of the sensors and processed. Different feature level fusion and decision level fusion algorithms are applied to the sensor data and the results are compared. Overall, the feature level fusion was found to yield a better detection performance than the decision level fusion scheme. Similarly, experiments for several data sets for a stepped frequency, continuous wave (SFCW) GPR and a MD are conducted in [17] to compare the performance of feature level fusion and decision level fusion. The results showed that feature level fusion performed better than the latter as well as for individual sensor feature extraction. Additionally, the feature level fusion also produced a lower FAR than any of the sensors in isolation. Generally this conclusion is corroborated by the work of [18] that also compares different feature level fusion and decision level fusion algorithms for a MD, GPR and IR fused system. The decision level algorithms are further classified into hard-decision level fusion and soft-level decision fusion. Hard-decision level fusion produces a global and absolute declaration on detection whereas soft-decision level fusion produces declarations local to each sensor performance which render a global decision subjective. The work concludes that the latter performs better and that along with the feature level fusion, significant enhancements in mine detection are achieved. Nevertheless, a further conclusion is that the feature level fusion achieves a higher FAR reduction than the soft-decision level fusion.

Sensor fusion for two GPR systems, one frequency domain radar and the other a time domain radar and a MD are investigated in [19]. Features are extracted from each of the sensors using a statistical approach for the same locations and then compared in isolation as well as with sensor fusion of the MD and each of the two radars separately. The results are compared simultaneously with POD estimates for a human operator who fuses the data of the sensors in isolation to determine detection. The human operator POD performs better than the sensor data results however the fused sensor data is seen to clearly possess higher POD than the single sensor data results. Comparisons between automated sensor fusion POD and human operator decision POD are also studied by [20]. He proposes data fusion of novel discrimination mode processing schemes for a MD and stepped frequency GPR to reduce the FAR. The sensor data obtained from the discrimination mode search are processed separately for each sensor and then fused by determining the geometric mean with a constant weighting function. This resulted in a significantly larger POD and reduced FAR in comparison to the individual sensor data. Additionally, a plot of the receiver operation characteristics (ROC), which plots the POD versus the FAR, showed that the data fusion detection performance was better than that of a human expert for the same test conditions.

The Advanced Landmine Imaging System (ALIS) fused sensor system presented in [21] comprises a MD and GPR in addition to a camera that is used to acquire sensor positional data during scanning. Visual images are obtained from the MD and GPR and transferred to a computer unit that applies a software based detection algorithm to produce an output that is used by the operator to determine the location of buried mines. The MINEHOUND [22] is a handheld MD and GPR dual sensor system that was designed for humanitarian demining. Signal responses from both sensors are obtained in the form of audio signals. Proprietary software controls the GPR and processes parameters that can be selected by the user based on the conditions of the demining operation. The MD acquires information on metal content and positioning while the GPR provides information on target position, depth and radar cross section (RCS). Additional signal processing to the MINEHOUND audio output based detection using full wave inversion (FWI) to image the ground is proposed by Watson [23]. The work in [24] provides the results of MD and GPR sensor data fusion experiments. Here the MD and GPR are collocated on an unmanned ground vehicle (UGV) and experiments are performed for an anti-tank (AT) mine only buried in stony sand with rocks and a brick. Different antenna heights and soil moisture contents are tested. The sensor data are processed separately with one-dimensional (1D) time domain waveforms and 2D images generated for the MD and GPR respectively in each test and compared. The GPR images are subsequently combined with the MD signal responses with algorithms that utilise linear, square root and logarithmic enhancement functions comparatively. These yielded better images generally with the best image among the three produced with the linear enhancement function. Feature level fusion for a combined MD and GPR sensor system is studied in [25] to compare detection performance through feature extraction for the individual sensors against the fused system. Feature extraction algorithms are applied in each case after the prior application of pre-processing and pre-screening algorithms. The work concluded that the feature extraction for the fused sensory yielded a higher POD than for any of the individual sensors.

Novel methods of regularization and classification of features for data obtained from a MD and GPR on a robotic platform are proposed in [26] which reduce the number of false positives in a demining operation considerably. The method ensures that the number of points or locations revisited after an initial detection by the MD to verify a positive detection or otherwise is reduced considerably and accurately thereby speeding up the operation in a safe and reliable manner. Furthermore robotic based detection with two MD sensors and two GPR sensors are tested in [27]. A global positioning system (GPS) on the robotic cart ensures that acquisition of measurements by the sensors is stable, regular and over precise locations throughout the lane of search. Feature based linear and non-linear statistical fusion algorithms are tested for the fused and individual sensor data. Both fusion algorithms yielded higher ROC curves than for the MD and GPR alone.

4. Discussion

The foregoing overview reinforces the fact that most sensor fused systems for AP landmine detection comprise the MD and GPR and in some cases consist of a third sensor which is commonly an IR imaging sensor or some form of camera. The MD and GPR sensors are the most mature in terms of technology development are complimentary in addressing the detection of metallic and non-metallic mines. Majority of the research also focuses on the development of novel, and in many instances, multiple or multi-staged processing and fusion algorithms aimed at improving detection particularly through the GPR sensor data which provides multi-dimensional data. Specifically, the reduction of the FAR or false positives is the goal as POD results are typically higher using these sensors. Furthermore, the feature level fusion and decision level fusion algorithms are the most popular among the references cited. Finally, feature level fusion is adjudged to yield the best results in term of the reduction of false positives.

Nevertheless only two of the fused sensor systems mentioned in the overview currently possess the feature of being used in successful mine field trials or evaluation as well commercially availability for humanitarian demining. These include the HSTAMID, used in Namibia [28] and Thailand [29] and the MINEHOUND, used in Cambodia, Bosnia and Angola [30, 31]. A third system, the ALIS has also undergone field evaluations in Afghanistan [21, 32] and Croatia [33]. However it is still not currently available commercially. This state of current handheld fused sensor systems for landmine detection remains largely unchanged from a review of landmine detection review published in 2008 [34].

Our focus is on research which contributes to the improvement of humanitarian demining through the production of affordable handheld mine detection systems that are easy to operate, considering that most mined areas are found in emerging or under-developed countries with conditions unsuitable for vehicular demining systems. This includes the enhancement of demining procedures based on current handheld dual or multiple sensor systems with an emphasis on the GPR sensor data which provides multi-dimensional data. A few examples are found in our overview. We outlined a couple of experiments aimed at improving POD with the HSTAMID and the FWI imaging procedure outlined in [23] to demining operations using the MINEHOUND is one of the more recent studies in this subject matter. Watson performed a theoretical FWI imaging for 3D GPR data and argues that more information can be obtained about the subsurface which would reduce the FAR. He essentially implements a decision fusion level procedure as the reconstructed images are combined with the original audio signals produced by the system indicating detection. However feature based information acquired from GPR 3D imaging is used to improve distinguishability of targets. This corroborates our previously outlined deduction of the importance of feature based signal or data processing in achieving a reduction of false positives in demining operations.

5. Conclusions

This paper has considered the topic of sensor fusion for handheld systems and identified the major types of sensor fusion methods. The paper focuses on a brief review of handheld sensor fused systems for landmine detection that utilise the MD and GPR sensors which are two of the few handheld portable technologies currently available for landmine detection. These sensors represent the most cited and actively researched for handheld fused systems intended for humanitarian demining. Most of the fused systems utilise decision level fusion and feature level fusion algorithms. The latter have been demonstrated to achieve a higher performance in the reduction of false positives or the FAR.

Current developments reveal that there are three highly matured handheld MD and GPR fused systems that have undergone real field evaluations for humanitarian demining. These are the HSTAMID, MINEHOUND and ALIS. However only the HSTAMID and MINEHOUND are commercially available while the ALIS is still at a lower TRL. There are still ongoing research into the improvement of mine detection POD and FAR using different fusion algorithms for MD and GPR handheld fused systems which have not attained a very high TRL. Nevertheless mature decision fusion level and especially feature level fusion based enhancements on the currently available systems possess the best prospects of delivering the much needed reduction of false positives in humanitarian demining operations.

ACKNOWLEDGEMENTS

The author wishes to acknowledge the Petroleum Technology Development Fund (PTDF), Abuja, Nigeria.