A CNN model for predicting size of buried objects from GPR B-Scans

Introduction

Ground penetrating radar (GPR) is a nondestructive technique (NDT) that can be used for various applications like archaeology, geological surveys, landmine detection, detection and mapping of subsurface utilities, detection of subsurface structures such as tunnels and boreholes, etc.

Ever since the GPR technology came to force, the basic principles used in creating GPR images have remained more or less the same. A GPR transmits pulses of electromagnetic waves at various frequencies depending on the choice of the user and the type of application. Due to different dielectric properties, the boundaries of buried objects and soil layers reflect radar waves back to a receiving antenna. The resulting radargram (GPR image) is an image of the subsurface that shows the various regions of heterogeneity. The data collected during the survey needs to be processed afterwards to improve and interpret the assessment. The hyperbolic feature is the most common shape found in B-Scan images of GPR. It is a challenging task and requires highly skilled people to properly identify and classify the various hyperbolic features in a GPR image. Additionally, corruption of GPR data due to noise creates added hurdles in the interpretation of GPR data. Therefore, GPR data processing continues to be a challenge to the scientific community.

Literature Review

In recent years, a variety of methods have been proposed by researchers for the interpretation of GPR data. [1] successfully applied the Histograms of Oriented Gradients (HOG) feature extraction technique to landmine detection in GPR data to detect landmines and proposed that other image processing techniques may also be applied for detection of targets in GPR data. A hyperbola fitting technique was used by [2] to estimate the radius of buried pipes and cables. [3] used Fourier Transform and Principal Component Analysis to detect landmines in various burial conditions.

Considerable research has been done in the detection of hyperbolic features and determination of regions of interest in GPR images for further interpretation of various parameters. Different strategies have been used on the topic of hyperbola detection. Fitting algorithms like Radon ([4]), wavelet ([5]) and Hough ([6], [7]) have been used for feature extraction in GPR images. Other feature extractors like Sobel operators ([8]) and Canny operators ([9], [10]) are also used as a pre-processing step. Statistical approaches like least square ([6]), fuzzy shell clustering ([11]), moment of inertia optimization ([12]) etc., have also been used to extract hyperbolic features. These methods usually involve multi-stage techniques. Initially, a region of interest is determined and then a hyperbola fitting algorithm is applied to it. All these methods are computationally slow but perform well from the point of view of accuracy and sensitivity.

[13] proposed a multi-stage technique which could recognise and fit hyperbolae in real time. They adapted OpenCV library to localise reflection hyperbolas in GPR B-Scans. The developed algorithm showed promising results when applied on using realistic data. They used Canny operator for edge detection and Hough transform for hyperbola fitting. Later, [14] working on a similar problem, proposed a novel technique for real-time hyperbola recognition by using a neural network classification algorithm. They used synthetic input images of size 100 \(\times\) 100 pixels. Due to the fast computation speed, both methods proposed by [13] and [14] are suitable for real-time applications.

However, studies by [15] have shown that under certain conditions, Common Midpoint (CMP) method appeared to be a more accurate way to determine GPR subsurface velocity than hyperbola fitting. [16] applied CMP method when estimating the thickness of asphalt.

Machine learning algorithms can be used for meaningful interpretation of GPR data. Machine learning in GPR data interpretation has many applications, some of which include detection of landmines ([17], [18]), predicting geometry and material classification of buried objects ([19], [20]), automatic detection and interpretation of GPR data ([21]), etc. However, with the increase in amount of data, detection of target features takes a long time. Other approaches use machine learning methods to concentrate on hyperbola detection as a first step, followed by a fitting algorithm. [22] used an artificial neural network in extraction of hyperbolic features and segregation of segments from a radargram.

Neural networks are gradually becoming popular since they were first introduced in the 1980s. Machine learning algorithms look at the data as a whole and draw decision boundaries around the samples. Since they treat all given features in a similar way, any change in even an insignificant feature may cause degradation in the model’s performance. A deep learning model, on the other hand, contains multiple layers of neural networks which work on different aspects of the input data. It even checks the significance of different features and tries to give lower weightage to insignificant ones.

For automatic detection of characteristic signatures of buried objects, neural network (NN)-based methodologies were developed by [23] and [24]. The application of multi-layer perceptron (MLP) neural network was effective in automatic identification and location of embedded steel reinforcing bars in concrete ([25]).

In recent years, extensive studies have been carried out on interpretation techniques based on deep-learning-based models. This has brought to light the many advantages of deep learning techniques. With increase in GPR data, the performance of traditional machine learning techniques suffer. This is where deep learning techniques have an edge over the traditional machine learning techniques ([26]). Moreover, since GPR data contains considerable environmental and experimental noise, deep learning techniques are generally better in data interpretation. Over time, researchers have started using CNN-based hyperbolic pattern recognition methods ([27], [28], [29]) for classification of objects.

With the advent of simulation tools like gprMAX ([30]), researchers have started using artificially generated GPR data for machine learning based classification of underground objects ([20], [31], [32], [33]).

Recently, models based on deep CNN were used in pattern recognition ([34]) and classification ([35]) of radar targets. Compared to a conventional neural network, the architecture of the deep CNN consists of 2 or more pairs of convolution and pooling layers. Deep CNN improves the accuracy of the target detection in GPR data. [36] employed multi-stage CNNs to automatically classifying subgrade defects from X-ray computed tomography images, while [37] used a multi-stage CNN to identify asphalt mixtures from GPR data. Although, various techniques have reported improved performance of CNNs, they have not been extensively used in interpretation of GPR data.

[38] proposed a new method to automatically locate rebars in reinforced concrete structures. They used a processing technique to locate the rebar and to estimate the wave propagation velocity using hyperbolic signature from GPR date. The proposed method could estimate both the cover thickness and locate the buried objects with a high degree of accuracy. Later, [39] worked on similar lines and proposed a method which used deep learning to automatically detect and localise buried targets. Single Shot Multibox Detector (SSD) was used to identify the regions of interest in GPR images which consisted hyperbolic signatures. This study concluded that the proposed approach could detect buried rebars in real time with a high degree of accuracy.

Most of the above methods concentrate on the detection and the localisation of buried objects, and the classification of such objects based on their shapes and compositions. However, very few researchers have worked on prediction of size of underground objects. Those that have dealt with prediction of size have used extrapolation ([19]) and gaussian regression techniques ([40]).

Aim of Present Work

In this paper, a feature-based algorithm for object size prediction in GPR data is proposed. It uses deep Convolutional Neural Network (CNN) estimator to decrease error and increase speed of prediction of size of targets from GPR images. Feature extraction is performed on the original pattern of GPR images prior to training the dataset in the proposed CNN model. Many training images are required for successful training of the model and to improve accuracy. To the best knowledge of the authors, no prior work has been done using hyperbolic feature extraction as a pre-processing step followed by a CNN-based estimator to predict size of buried objects.

GPR Data

The electromagnetic simulator gprMax is used to generate simulated data for training the proposed model. Further, acceleration of the simulations is achieved by using NVIDIA’s CUDA framework ([41]). To create a model for simulation, several parameters need to be set. Fig. 1 shows one of the models created using gprMax.

The dimensions of the model are \(1000 \times 400 \times 600~(X \times Y \times Z)~mm^3\). There is a 210 mm layer of air at the top and a soil layer of 390 mm below it. In Fig. 1, a cylindrical object, made of aluminium (\(\sigma = 3.5 \times 10^7\) Siemens/metre, \(\epsilon_r\) = 10.8), is located 213 mm below the ground. The signal source along with the transmitter (Tx) and receiver (Rx) antennas are placed above the soil layer.

1000 GPR B-Scans are created for 7 soil types with random values each for object depths and radii. The dielectric parameters for the soil types are given in Table 1 ([42]).

Sl. No.	Soil Type	Conductivity \(\sigma\)(S/m)	Relative permittivity \(\epsilon_{r}\)
1	Dry and sandy	0.002	10
2	Marshy land, densely wooded	0.0075	12
3	Agricultural land	0.01	15
4	Rocky land, steep hills	0.002	12.5
5	Hilly terrain (to about 1000 m)	0.001	5
6	Pastoral Hills, rich soil	0.007	17
7	Pastoral medium hills and forestation	0.005	13

Depth values are taken randomly between 10 mm and 215 mm below the soil surface. As for radius of the target, random integer values between 5 mm and 20 mm are selected. Care is taken to automatically adjust the depth of the target for larger values of radius, so that the target is always below the soil surface. The parameters used in the simulation are given in Table 2.

Parameters	Parameter values
Source Frequency	400 MHz
Source Waveform	Gaussian
Resolution (Spatial)	2 mm
A-Scan intervals	6 mm
No of A-Scans	108
Time Window	14 ns

The simulations are accelerated using NVIDIA RTX3090 GPU. For each model, 108 A-Scans are taken, sufficient to capture the hyperbola created by the EM wave reflection from the target. Fig. 2 shows a sample B-Scan image which is composed of 108 A-Scans. An object of radius 18 mm (36 mm diameter) is buried 213 mm below the soil surface. Each B-Scan file has 3636 \(\times\) 108 data points. All 1000 B-Scan files are stored in a three dimensional (3D) numpy array along with their corresponding attributes (object radius). A representation image of the array of stacked B-Scan files is shown in Fig. 3.

GPR Principle

As can be seen in Fig. 4, the GPR receives multiple reflections from a buried target as it moves across the scanning direction.

The larger the size of the buried object, the greater will be the area from which reflections occur. When presenting the effect of the diameter of buried object on hyperbolic reflections, [38] observed that the amplitude of the peak hyperbolic reflection has a good correlation with the diameter of the buried objects. Later, in the findings reported by [39], it was seen that the shape of the hyperbolic curve changed slightly with different rebar diameters, even though it is more sensitive to the rebar depth than the rebar diameter. The input image of the proposed model contains 108 A-Scans taken at 6 mm intervals. This indirectly contains size related information regarding the hyperbola. This should help in extracting different characteristic features from the hyperbolic image.

Convolution Neural Networks

Conventional computer vision methods have been applied for structural analysis ([1], [2], [43], [44]), detection of concrete cracks ([45]), recognition and visualization of cracks ([46]), measurement of road surface distresses ([47]), geological parameterization ([48]), etc. The techniques applied in the studies mentioned above require large amount of image pre-processing when the training dataset is being prepared. The training process is tedious as well. CNN models typically perform feature extraction through a series of convolutional layers, pooling layers, and fully connected hidden layers. CNN has shown exceptional performance in various domains. Its use in computer vision is well known with much work being done on face recognition with varying illuminations and facial expressions among others. CNN architectures have also been implemented on portable devices like FPGAs for various applications ([49], [50]). The pre-processing required in a CNN model is lower than that of algorithms ([27], [51]). The architecture of a typical CNN binary classifier with an input image size of 128\(\times\)128 is shown in Fig. 5.

CNNs used in practical purposes have multiple pairs of convolutional and pooling layers. The output of the final pooling layer is connected to a hidden layer, the output of which is passed to a final classification/estimation layer.

Proposed CNN Architecture

A CNN extracts high-level features such as edges from the input image and reduces the dataset to a form which is easier to process. The spatial size of the convolved feature is reduced by the pooling layer. This decreases the computational requirements for processing the data. It is also useful for extracting dominant features. Before feeding it to a neural network, the output from the final pooling layer is flattened into a column vector.

Feature Extraction

The overall GPR data processing pipeline in this work consists of feature extraction, splitting of dataset into train-test sets, normalising the data, training of CNN model, and validation.

As a means of feature extraction, edge detection is used. There are several operators for edge detection that are used in image processing. In this work, Sobel, Laplacian, Scharr, and Canny operators are used for comparing the performance of the proposed CNN architecture. Hyperbolic curves are one of the distinguishing features in GPR images. The parameters of a hyperbolic signature can be used to estimate the size and location of the related target object in addition to the characteristics of the medium (e.g.: soil). In edge detection we try to find the regions in an image where there is a sharp change in intensity or colour. The higher the value, steeper the change and vice versa.

Sobel Operator

A very common operator for edge detection is a Sobel operator. Sobel is a very popular operator and is widely used by researchers as an edge detection tool ([52]). The Sobel operator is a gradient based method and it calculates the first derivatives of the image separately for the X and Y axes. The derivative in X direction is computed by convolving the image \(f(x,y)\) with a kernel \(G_x\). For a kernel size of 3, \(G_x\) is computed as shown in equation (1). The derivative in Y direction is computed as shown in equation (2).

\[G_{x} = \begin{bmatrix} -1 & 0 & +1 \\ -2 & 0 & +2 \\ -1 & 0 & +1 \end{bmatrix} * f(x,y) \tag{1}\]

\[G_{y} = \begin{bmatrix} -1 & -2 & -1 \\ 0 & 0 & 0 \\ +1 & +2 & +1 \end{bmatrix} * f(x,y) \tag{2}\]

From equations (1) and (2), the gradient of an image \(f(x,y)\) with respect to x and y directions can be shown as given in equation (1). The gradient will be in whichever direction the change will be more intense. This gradient direction is calculated as given by equation (2). Finally, at each pixel of the image, the approximation of the gradient is calculated by equation (3).

\[\nabla f = [G_x, G_y] = \left [ \frac{\partial f}{\partial x},\frac{\partial f}{\partial y} \right ] \tag{3}\]

\[\theta = tan^{-1}\left ( \frac{G_x}{G_y} \right ) = tan^{-1}\left ( \frac{\partial f}{\partial y}/\frac{\partial f}{\partial x} \right ) \tag{4}\]

\[G = \left \| \nabla f \right \| = \sqrt{ G_{x}^{2} + G_{y}^{2} } = \sqrt{\left ( \frac{\partial f}{\partial x} \right )^{2}+\left ( \frac{\partial f}{\partial y} \right )^{2}} \tag{5}\]

In this work, 7\(\times\)7 Sobel kernels are implemented. By increasing the size of the convolution matrix, the edge detector becomes less sensitive to local noise. This is useful for low contrast images where edges might not be easy to detect. Sobel operator has the advantage of providing edge detection with simultaneous reduction in noise (gaussian smoothing).

Laplacian Operator

The second edge detection method used to compare the CNN performance is the Laplacian operator ([53]). However, unlike the Sobel operator, the Laplacian uses only one kernel. Laplacian operator is computationally faster than Sobel operator and it calculates second order derivatives in a single pass.

In this work, 5\(\times\)5 Laplacian kernels are implemented due to the advantages mentioned previously.

Canny Operator

The fourth method used for CNN performance comparison is the Canny operator. It has been used as a pre-processing step for automated detection of hyperbolic features ([10]). [54] demonstrated the use of Canny edge detection for 3D seismic discontinuity enhancement. It is a multi-stage edge detector algorithm which goes through the following steps:

1. Noise reduction using a Gaussian filter.
2. Calculate intensity gradient using Sobel filter, using equations (4) and (5).
3. Remove unwanted pixels which may not be part of the edges (non-maximum suppression).
4. Using high and low threshold intensity values, focus on the edges which are important for the present application and filter out the unwanted pixels which non-maximum suppression could not.

In this work, a Canny operator having 20 as low threshold and 120 as high threshold is used. The threshold values are determined using trial-and-error method for best validation loss during CNN training.

The fundamental concept in all the four feature extraction techniques is that the targets (buried object) produce hyperbolic signatures in GPR B-Scans which can be summarised in terms of size and orientation of the hyperbolic shapes.

Training of CNN model

The dataset containing 1000 B-Scans (108,000 A-Scans) is split so that 900 files are used for training and 100 for validation. Care is taken so that we get the same split for each edge detector used. The data is then normalised separately for train and test sets and reshaped to add a \(4^{th}\) dimension for colour channel (1 in this case since it is greyscale).

Convolution Layer

A convolution layer applies a filter to an input image which results in an activation. The corresponding activation function decides which nodes get activated. Repeated application of a filter results in a a feature map. Multiple filters are used in parallel for each convolutional layer. This helps it to extract multiple features in parallel. The Rectified Linear Unit (ReLU) activation function is used in both the convolutional and hidden layers. ReLU is a non-linear activation function used in deep or multi-layer neural networks. It can be represented as:

\[f(x) = max(0,x) ~~~~where, ~x ~is ~an ~input ~value. \tag{6}\]

The output of ReLU is equal to 0 when the input value is less than 0 and equal to the input value when the input is positive. Thus, equation (6) can be re-written as:

\[f(x) = \begin{cases} 0, & \text{if x $<$ 0}\\ x, & \text{if x $\geq$ 0} \end{cases} \tag{7}\]

Pooling Layer

Of the two types of pooling available, Max Pooling downsamples the input image by computing the maximum value from the portion covered by the kernel. On the other hand, Average Pooling computes the average from the portion covered by the kernel. In most cases, Max Pooling has a better performance than Average Pooling. The convolutional and pooling layers, together form an i-th layer of the CNN.

Fully Connected Layer

A fully connected layer is an important final stage of any convolution network. Here, the output of the network’s last convolution layer are flattened to a single 1 dimensional vector. The input nodes of the fully connected layer have a complete connection to all the activations from the previous flattened layer.

Loss Function

The mean absolute percentage error (MAPE) is used as a loss function for training the CNN model. It is the mean of the absolute differences between our target and predicted values. It is defined as the mean of absolute relative errors:

\[MAPE = {\frac{1}{N}}\sum_{t=1}^{N}\left\lvert{\frac{P_t-A_t}{A_t}}\right\rvert \tag{8}\]

where N is the number of samples, (\(P_t\)) is the predicted data value and (\(A_t\)) is the actual data value.

Optimization

The use of an appropriate optimizer is essential in improving the model’s performance. Optimizers minimise losses by adjusting the hyper-parameters of the network like weight and learning rate. Stochastic Gradient Descent (SGD) was perhaps the most popular optimizer during the initial days of deep learning. Over the years, many improvements on SGD like momentum and Nestorov ([55]) have been reported. Adaptive optimizations like AdaDelta, RMSProp, and Adam (Adaptive Moment Estimation) have become popular recently as they adjust the learning rates of the parameters in different proportions, resulting in faster convergence and smoother training process. Adam has become quite popular because it converges faster than the other adaptive optimizers. Hence, it is used in optimizing the proposed CNN model’s training process with an initial learning rate of 0.0001.

Evaluation Metrics

MAPE is used as a metric to measure the performance of the CNN training and validation. It is a measure of prediction accuracy in statistics. MAPE is commonly used for checking the performance of regression models. MAPE can be expressed as a percentage, making it understandable to all types of audiences. By contrast, other metrics that are not expressed in relative terms or as percentages usually require domain expertise and context to understand the significance of their numerical values.

Final Architecture

After multiple training sessions, it is found that 4 filters of 5\(\times\)5 kernel size in the first Convolutional layer gives the best MAPE, irrespective of the number of hidden layers in the model. Hence, this is taken as the base for the model, after which several hidden layers are added. Performance is evaluated for each addition of hidden layer. Table 3 shows the validation losses corresponding to different numbers of hidden layers used, for various edge detectors.

Sl. No.	Number of Hidden Layers	Validation Loss (MAPE)
		No Edge Detector	Sobel	Laplacian	Scharr	Canny
1	1 Layer	28.99	33.1	30.41	8.70	18.51
2	2 Layers	29.59	30.40	31.91	9.55	19.96
3	3 Layers	30.84	34.46	34.27	7.72	20.16
4	4 Layers	30.32	35.68	32.99	7.65	21.31
5	5 Layers	16.39	31.23	29.29	6.94	23.97
6	6 Layers	16.34	30.56	29.25	8.71	25.58
7	7 Layers	16.14	30.24	–	8.35	–
8	8 Layers	16.55	–	–	8.36	–
9	9 Layers	16.11	–	–	8.50	–
10	10 Layers	20.49	–	–	9.87	–

The architecture of the best CNN model is shown in Fig. 7. The input convolution layer is followed by a MaxPool layer of pool size \(2\times2\). 32 filters of kernel size \(3\times3\) is used in the 1st hidden layer followed again by a MaxPool layer of pool size \(2\times2\). Another similar pair of convolution-pooling layer is placed after that, with the number of filters being 64 in this case. Three more convolution layers are used after that with filter numbers 64 (\(3\times3\)), 64 (\(3\times3\)), and 66 (\(2\times2\)) respectively. At this point, the outputs are flattened to a 1D matrix and a fully connected hidden layer of 64 neurons is added. A 1-way ReLU regression layer is used after the hidden layer, to estimate the required parameter from each input image. Mini batch gradient descent method of batch size 32 is used for training the CNN model. The CNN model trains over a series of epochs and makes an estimation of the required parameter, i.e., diameter (2\(\times\)radius) of the buried object. MAPE (loss function) during validation is monitored and early stopping callback API of Keras is used to stop the training if the model performance does not improve after 100 successive epochs.

NVIDIA RTX3090 GPU was used in CNN training. Using a GPU, the CNN training time reduced from several hours on CPU to less than 1 hour.

The whole GPR data processing pipeline, including the CNN and its optimisation algorithms, is written in Python, along with its image processing extension OpenCV and its deep learning extensions, Keras and TensorFlow.

Results and Discussion

It is observed that the model with Scharr edge detector followed by CNN model with 5 hidden layers performs the best with a validation loss of 6.94 (MAPE). However, Sobel, Laplacian, and Canny edge detectors did not give favourable results with the current dataset. Best MAPE (validation) for the CNN model with Sobel edge detector is 30.24 (7 hidden layers), Laplacian edge detector is 29.25 (6 hidden layers), and Canny edge detector is 18.51 (1 hidden layer).

Since the CNN model with Scharr edge detector shows the best validation loss, its model is saved in HDF5 format to evaluate its performance on other datasets. For a final performance evaluation of the model, another set of 100 B-Scans is created using gprMax. The MAPE of the model for this new and unseen dataset is found to be 7.49. Table 4 shows the performance of the model on the new dataset.

Sl. No.	True value	Best Prediction	Worst Prediction	Median
1	10	9.58	7.27	9.51
2	12	13.16	17.48	14.59
3	14	14.18	17.18	14.18
4	16	15.33	17.12	15.33
5	18	19.06	21.39	20.11
6	20	19.53	23.45	20.95
7	22	22.30	24.57	21.80
8	23	24.51	24.85	24.68
9	24	23.93	30.25	24.51
10	26	26.05	23.70	25.73
11	27	32.06	32.06	32.06
12	28	30.59	23.46	27.32
13	30	29.47	23.56	30.81
14	31	32.68	35.58	34.13
15	32	33.33	24.89	33.73
16	34	35.09	29.33	35.09
17	36	36.30	39.74	36.30
18	38	39.70	27.72	34.86
19	39	36.28	36.28	36.28
20	40	40.40	37.80	39.24

A violin plot showing the distribution of predicted values and their probability density is shown in Fig. 8. It divides the data points (predicted values) into four equal portions (quartiles). The white dot in the middle denotes the median value and the thick black bar denotes the interquartile range. The thin black line extending on both sides represents the upper and lower adjacent values in the predicted data. The kernel density estimation on both sides of the black line shows the distribution of the data.

In Figure 8, the plot for diameter 24 mm shows that the median value of the predicted values is 24.51 mm. For an actual value of 24 mm, the highest predicted value of 30.25 mm and the lowest predicted value of 23.93 mm create the boundaries of the interval space which contains all the data points. The interquartile range (IQR) shows the middle 50% of the four quartiles. In this case, the IQR ranges from 22.5 mm (first quartile) to 27.5 mm (third quartile) which means the middle 50% of the predicted values lie between 22.5 mm to 27.5 mm. The kernel density estimation (KDE) plot shows the probability distribution of the predicted values on both the sides of the vertical black line. The KDE plot goes to a maxima at around 24.51 mm which signifies that the probability of finding a predicted value near to 24.51 mm is maximum. This is in rough agreement with the fact that the actual value is 24 mm.

A comparison of the model’s performance shows that Scharr edge detector (SED) followed by a 6 convolution layer CNN model boosts the overall performance over the other methods used. The SED approach extracts sufficient signature information related to the size of the object to raise the system’s performance and reduce its validation MAPE to 6.74 and a MAPE of 7.49 on new, unseen data.

Testing on noisy data

GPR images are seldom noise-free and ideal hyperbolic reflections are not obtained in most cases. Hence, it is imperative to test any feature extraction model on noisy data for proper evaluation of its performance in realistic scenarios.

For this, a new dataset is generated by adding noise randomly. Random number of pixels are selected for each GPR B-Scan and replaced with white and black pixels. The total number of pixels in each B-Scan being 3,92,688 (108 \(\times\) 3636), the range of the randomly selected pixels is randomly varied from  0.3% (1400) to  25% (98000). Before applying the proposed model for diameter prediction, the image is first passed through a median filter which is a widely used non-linear filter to remove noise in GPR images. A Scharr filter is then applied on the image, after which the CNN model is used for diameter prediction. 9 shows a GPR B-Scan with 10% noise added to it. Below it is the noise-free image after a median filter is used.

Several images containing various amounts of noise are evaluated using the proposed CNN model. Table 5 shows the performance of the model for different noise levels in the input image. As seen from the table, the model is able to predict the target’s diameter with low error rates at low noise levels. However, as the noise level increases beyond 20%, it is seen that model’s performance degrades with error rates of 22.26% at 20% noise level and upto 26.88% at 25% noise level.

Sl. No.	Noise Level	Diameter of target object (mm)		Error (mm)
		True values	Predicted values
1	01%	36	36.92	0.92
2	02%	14	14.30	0.30
3	03%	36	34.03	1.97
4	04%	18	17.75	0.25
5	04%	17	17.70	0.70
6	04%	26	24.58	1.42
7	05%	22	23.58	1.58
8	05%	38	34.43	3.57
9	05%	24	20.84	3.16
10	10%	18	17.35	0.65
11	10%	26	24.85	1.15
12	10%	16	17.02	1.02
13	10%	28	25.47	2.53
14	10%	36	32.60	3.40
15	15%	28	26.47	1.53
16	15%	30	27.66	2.34
17	15%	16	14.03	1.97
18	15%	14	12.23	1.77
19	15%	26	29.80	3.80
20	20%	22	20.82	1.18
21	20%	14	14.91	0.91
22	20%	36	39.37	3.37
23	20%	28	21.77	6.23
24	20%	14	10.40	3.60
25	25%	20	15.58	4.42
26	25%	14	10.80	3.20
27	25%	26	19.01	6.99

Testing on non-metallic buried objects

A CNN model for detecting and predicting the size of buried objects will not be considered for practical application if it can be applied to metallic targets only. Hence, its performance evaluation is needed for non-metallic targets as well. For this purpose, a dataset consisting of PVC (\(\sigma\) = 0.001 Siemens/metre, \(\epsilon_r\) = 4) and concrete (\(\sigma\) = 0.01 Siemens/metre, \(\epsilon_r\) = 7.3) is generated. 10 shows a GPR B-Scans after a application of Scharr filter for PVC and concrete targets.

On evaluation of the CNN model of this dataset, it is seen that it can predict the target’s size with very low error rates, irrespective of the target material. The dataset is generated using the methodology as previously described in section GPR Data. Table 6 lists the values predicted by the model for different target materials having different sizes.

Sl. No.	Object Material	Diameter of target object (mm)		Error (mm)
		True values	Predicted values
1	PVC	34	32.90	1.10
2	Concrete	18	19.28	1.28
3	Concrete	32	30.12	1.88
4	PVC	16	14.91	1.09
5	Concrete	26	28.39	2.39
6	PVC	20	21.83	1.83
7	PVC	30	31.36	1.36
8	PVC	10	9.79	0.21
9	Concrete	18	19.38	1.38
10	PVC	22	20.12	1.88
11	Concrete	34	32.11	1.89
12	Concrete	26	23.72	2.28
13	Concrete	20	18.44	1.56
14	Concrete	24	24.29	0.29
15	PVC	16	16.54	0.54
16	PVC	16	14.76	1.24
17	Concrete	38	34.59	3.41
18	Concrete	28	25.87	2.13
19	Concrete	28	25.41	2.59
20	Concrete	30	27.29	2.71

Testing on multiple buried objects

To test the efficacy of the model when multiple buried objects are present, few B-Scans are generated containing upto 3 objects of different sizes and at different depths. All the objects are considered to be cylindrical in shape. One such B-Scan image is shown in 11 where multiple hyperbolas with overlapping edges can be seen. It contains 3 metallic cylinders at depths of diameters 30 mm, 20 mm, and 40 mm from left-to-right.

In the case of multiple target objects, the hyperbolas will have to be isolated from the image before the model can be used for estimation of target size. The hyperbolas can be isolated using template matching, which is a commonly used algorithm in image processing. They can also be extracted manually by cropping the image. Next, each image containing the individual hyperbola is reshaped to have the same size as required by the CNN model, in this case 108 \(\times\) 3636. This is done by padding around the edges of each image with linearly varying values from the pixels at the edge of the image to the 3636th / 108th pixel.

As shown in 12, a Scharr filter is finally applied to each image after which the CNN model is used to estimate the size of the targets. The model shows decent performance in estimating the size of the targets. It predicted a size of 34.46 mm for the object with 34 mm actual diameter, 22.86 mm for 20 mm (actual), and 44.15 mm for an actual diameter of 40 mm. However, those performances in which each image recorded a single hyperbola were better than this result.

Conclusion

A novel approach of using CNN for predicting size of buried objects from GPR B-Scans was presented. The proposed CNN architecture extracts high level signatures in the first stage of feature extraction and learns additional low level features when the input data passes through the neural network to finally make an estimation of the required parameter. A comparison with other techniques is shown in Table 7 from which it can be concluded that the application of CNN in this problem has demonstrated a good performance in predicting size of buried objects. Moreover, it can be used in real time applications in GPR data interpretation since such a model can be implemented on embedded devices like ARM, FPGA, etc.

Sl. No.	Author	Technique used	Application	Real time / Offline
1	Chaudhuri and Samanta (1991) ([12])	Hyperbolic feature extraction using Moment of inertia optimisation	Extraction of hyperbolic features	Offline
2	Delbo et al. (2000) [11]	Hyperbolic feature extraction using Fuzzy shell clustering	Extraction of hyperbolic features	Offline
3	Ko et al. (2012) [3]	PCA and Fourier Transform	Detection of landmines	Offline
4	Wahab et al. (2013) [2]	Hyperbola fitting technique	Estimation of radius of pipes and cables	Offline
5	Maas and Schmalzl (2013) [13]	Canny operator for edge detection and Hough transform for hyperbola fitting	Automatically localize reflection hyperbola	Real time
6	Besaw and Stimac (2015) [28]	ANN based Deep Belief Network (DBN)	Detection of explosive hazards	Offline (in situ)
7	Mertens et al. (2015) [9]	Hyperbolic feature extraction using Canny operator	Automated detection of reflection hyperbola	Offline
8	Dou et al. (2016) [14]	Canny operator for edge detection and Hough transform for hyperbola fitting	Automatically localize reflection hyperbola	Real time
9	Li et al. (2016) [8]	Hyperbolic feature extraction using Sobel operator followed by Hough Transform	Automatic recognition of tree roots	Offline
10	Lameri et al. (2017) [29]	Deep Convolutional Neural Network	Detection of landmines	Real Time
11	Todkar et al. (2017) [56]	Machine learning using Support Vector Machines (SVM)	Detection of debondings	Offline (in situ)
12	Kafedziski et al. (2018) [32]	Faster Region Convolutional Neural Network (Faster R-CNN)	Detection and classification of landmines	Real Time
13	Sonoda and Kimoto (2018) [57]	Deep CNN	Object identification	Real Time
14	Kang et al. (2020) [58]	Basis pursuit-based background filtering algorithm followed by deep CNN	Underground cavity detection	Real Time
15	Bugarinović et al. (2020) [10]	Hyperbolic feature extraction using Canny operator	Automated detection of hyperbolic features	Offline
16	Giannakis et al. (2021) [33]	Autoencoder as a pre processing step followed by ANN based regressor	Estimate the diameter of reinforcement bars in concrete	Real Time
17	Present work	Hyperbolic feature detection followed by deep CNN	Estimation of radius/ diameter of buried objects	Real time

Future work of this approach will concentrate on experimentally verifying the model with actual data taken from an FPGA-based GPR system being developed in-house as well as from commercial GPR systems. The present model was trained with an object which is a metallic cylinder made of aluminium. Even though the model was tested on PVC and concrete objects with good results, its performance can still be improved by training it using greater number of datasets containing objects of a variety of materials. Different types of materials will have different dielectric properties and hence will generate different scattering and dispersion characteristics. Similarly, training the model for different shapes such as rectangular and triangular forms give make it more robust. It remains to be seen how these will affect the robustness of the proposed approach.

Acknowledgements

The authors would also like to thank Dr. Manoj Kumar Phukan (Sr. Scientist, Geo Sciences and Technology Division, CSIR-NEIST, Jorhat) who gave detailed insights on interpretation of GPR data and Ms. Sharmistha Mazumdar for her help in visualising the results. Finally, the authors appreciate the help of Mr. Arnob Doloi for his inputs during database creation from time to time. The authors would also like to thank Ms. Sweta Kumar Gaur for her insightful suggestions in linguistic polish of the manuscript.

Data Availability

All the data, models, or codes generated during this study are available from the corresponding author upon request.

References

1. P. A. Torrione, K. D. Morton, R. Sakaguchi, and L. M. Collins, “Histograms of oriented gradients for landmine detection in ground-penetrating radar data,” IEEE transactions on geoscience and remote sensing, vol. 52, no. 3, pp. 1539–1550, Jan. 2013.
2. W. A. Wahab, J. Jaafar, I. M. Yassin, and M. R. Ibrahim, “Interpretation of ground penetrating radar (GPR) image for detecting and estimating buried pipes and cables,” in 2013 IEEE international conference on control system, computing and engineering, IEEE, Jan. 2013, pp. 361–364.
3. K. H. Ko, G. Jang, K. Park, and K. Kim, “GPR-based landmine detection and identification using multiple features,” International journal of antennas and propagation, vol. 2012, Jan. 2012.
4. A. DellÁcqua, A. Sarti, S. Tubaro, and L. Zanzi, “Detection of linear objects in GPR data,” Signal Processing, vol. 84, no. 4, pp. 785–799, Jan. 2004.
5. C. Windsor, L. Capineri, P. Falorni, S. Matucci, and G. Borgioli, “The estimation of buried pipe diameters using ground penetrating radar,” Insight-Non-Destructive Testing and Condition Monitoring, vol. 47, no. 7, pp. 394–399, Jan. 2005.
6. H. Chen and A. G. Cohn, “Probabilistic robust hyperbola mixture model for interpreting ground penetrating radar data,” pp. 1–8, Jan. 2010.
7. X. Liu et al., “Non-invasive estimation of root zone soil moisture from coarse root reflections in ground-penetrating radar images,” Plant and Soil, vol. 436, no. 1–2, pp. 623–639, Jan. 2019.
8. W. Li, X. Cui, L. Guo, J. Chen, X. Chen, and X. Cao, “Tree root automatic recognition in ground penetrating radar profiles based on randomized hough transform,” Remote Sensing, vol. 8, no. 5, p. 430, Jan. 2016.
9. L. Mertens, R. Persico, L. Matera, and S. Lambot, “Automated detection of reflection hyperbolas in complex GPR images with no a priori knowledge on the medium,” IEEE Transactions on Geoscience and Remote Sensing, vol. 54, no. 1, pp. 580–596, Jan. 2015.
10. Ž. Bugarinovi’c, L. Pajewski, A. Risti’c, M. Vrtunski, M. Govedarica, and M. Borisov, “On the introduction of canny operator in an advanced imaging algorithm for real-time detection of hyperbolas in ground-penetrating radar data,” Electronics, vol. 9, no. 3, p. 541, Jan. 2020.
11. S. Delbo, P. Gamba, and D. Roccato, “A fuzzy shell clustering approach to recognize hyperbolic signatures in subsurface radar images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 38, no. 3, pp. 1447–1451, Jan. 2000.
12. B. B. Chaudhuri and G. Samanta, “Elliptic fit of objects in two and three dimensions by moment of inertia optimization,” Pattern Recognition Letters, vol. 12, no. 1, pp. 1–7, Jan. 1991.
13. C. Maas and J. Schmalzl, “Using pattern recognition to automatically localize reflection hyperbolas in data from ground penetrating radar,” Computers & geosciences, vol. 58, pp. 116–125, Jan. 2013.
14. Q. Dou, L. Wei, D. R. Magee, and A. G. Cohn, “Real-time hyperbola recognition and fitting in GPR data,” IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 1, pp. 51–62, Jan. 2016.
15. R. W. Jacob and T. Urban, “Ground-penetrating radar velocity determination and precision estimates using common-midpoint (CMP) collection with hand-picking, semblance analysis and cross-correlation analysis: A case study and tutorial for archaeologists,” Archaeometry, vol. 58, no. 6, pp. 987–1002, Jan. 2016.
16. S. Zhao and I. L. Al-Qadi, “Development of an analytic approach utilizing the extended common midpoint method to estimate asphalt pavement thickness with 3-d ground-penetrating radar,” NDT & E International, vol. 78, pp. 29–36, Jan. 2016.
17. N. Smitha and V. Singh, “Target detection using supervised machine learning algorithms for GPR data,” Sensing and Imaging, vol. 21, no. 1, pp. 1–15, Jan. 2020.
18. H. Frigui and P. Gader, “Detection and discrimination of land mines in ground-penetrating radar based on edge histogram descriptors and a possibilistic \(k\)-nearest neighbor classifier,” IEEE Transactions on Fuzzy Systems, vol. 17, no. 1, pp. 185–199, Jan. 2008.
19. N. R. Syambas, “An approach for predicting the shape and size of a buried basic object on surface ground penetrating radar system,” International Journal of Antennas and Propagation, vol. 2012, pp. 1–13, Jan. 2012.
20. Q. Lu, J. Pu, and Z. Liu, “Feature extraction and automatic material classification of underground objects from ground penetrating radar data,” Journal of Electrical and Computer Engineering, vol. 2014, Jan. 2014.
21. S. Shihab, W. Al-Nuaimy, and A. Eriksen, “Image processing and neural network techniques for automatic detection and interpretation of ground penetrating radar data,” Proceedings of the 6th International Multi-Conference on Circuits, Systems, Communications and Computers, Cancun, Mexico, pp. 12–16, Jan. 2002.
22. A. Risti’c, Ž. Bugarinovi’c, M. Vrtunski, and M. Govedarica, “Point coordinates extraction from localized hyperbolic reflections in GPR data,” Journal of Applied Geophysics, vol. 144, pp. 1–17, Jan. 2017.
23. W. Al-Nuaimy, Y. Huang, M. Nakhkash, M. Fang, V. Nguyen, and A. Eriksen, “Automatic detection of buried utilities and solid objects with GPR using neural networks and pattern recognition,” Journal of applied Geophysics, vol. 43, no. 2–4, pp. 157–165, Jan. 2000.
24. P. Gamba and S. Lossani, “Neural detection of pipe signatures in ground penetrating radar images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 38, no. 2, pp. 790–797, Jan. 2000.
25. M. Shaw, S. Millard, T. Molyneaux, M. Taylor, and J. Bungey, “Location of steel reinforcement in concrete using ground penetrating radar and neural networks,” Ndt & E International, vol. 38, no. 3, pp. 203–212, Jan. 2005.
26. M. M. Najafabadi, F. Villanustre, T. M. Khoshgoftaar, N. Seliya, R. Wald, and E. Muharemagic, “Deep learning applications and challenges in big data analytics,” Journal of Big Data, vol. 2, no. 1, p. 1, Jan. 2015.
27. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, Neural Information Processing Systems Foundation, Jan. 2012, pp. 1097–1105.
28. L. E. Besaw and P. J. Stimac, “Deep convolutional neural networks for classifying GPR b-scans,” in Detection and sensing of mines, explosive objects, and obscured targets XX, International Society for Optics; Photonics, Jan. 2015, p. 945413.
29. S. Lameri, F. Lombardi, P. Bestagini, M. Lualdi, and S. Tubaro, “Landmine detection from GPR data using convolutional neural networks,” in 2017 25th european signal processing conference (EUSIPCO), IEEE, Jan. 2017, pp. 508–512.
30. C. Warren, A. Giannopoulos, and I. Giannakis, “gprMax: Open source software to simulate electromagnetic wave propagation for ground penetrating radar,” Computer Physics Communications, vol. 209, pp. 163–170, Jan. 2016.
31. Y. Zhang, D. Huston, and T. Xia, “Underground object characterization based on neural networks for ground penetrating radar data,” in Nondestructive characterization and monitoring of advanced materials, aerospace, and civil infrastructure 2016, T. Yu, A. L. Gyekenyesi, P. J. Shull, and H. F. Wu, Eds., International Society for Optics; Photonics; SPIE, Jan. 2016, pp. 10–18. doi: 10.1117/12.2219345.
32. V. Kafedziski, S. Pecov, and D. Tanevski, “Detection and classification of land mines from ground penetrating radar data using faster r-CNN,” in 2018 26th telecommunications forum (TELFOR), IEEE, Jan. 2018, pp. 1–4.
33. I. Giannakis, A. Giannopoulos, and C. Warren, “A machine learning scheme for estimating the diameter of reinforcing bars using ground penetrating radar,” IEEE Geoscience and Remote Sensing Letters, Jan. 2020, Published.
34. J. Ding, B. Chen, H. Liu, and M. Huang, “Convolutional neural network with data augmentation for SAR target recognition,” IEEE Geoscience and remote sensing letters, vol. 13, no. 3, pp. 364–368, Jan. 2016.
35. S. Chen, H. Wang, F. Xu, and Y.-Q. Jin, “Target classification using the deep convolutional networks for SAR images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 54, no. 8, pp. 4806–4817, Jan. 2016.
36. Z. Tong, J. Gao, and H. Zhang, “Innovative method for recognizing subgrade defects based on a convolutional neural network,” Construction and Building Materials, vol. 169, pp. 69–82, Jan. 2018.
37. J. Jiang, Z. Zhang, Q. Dong, and F. Ni, “Characterization and identification of asphalt mixtures based on convolutional neural network methods using x-ray scanning images,” Construction and Building Materials, vol. 174, pp. 72–80, Jan. 2018.
38. P. Wiwatrojanagul, R. Sahamitmongkol, S. Tangtermsirikul, and N. Khamsemanan, “A new method to determine locations of rebars and estimate cover thickness of RC structures using GPR data,” Construction and Building Materials, vol. 140, pp. 257–273, Jan. 2017.
39. H. Liu, C. Lin, J. Cui, L. Fan, X. Xie, and B. F. Spencer, “Detection and localization of rebar in concrete by deep learning using ground penetrating radar,” Automation in Construction, vol. 118, p. 103279, Jan. 2020.
40. E. Pasolli, F. Melgani, and M. Donelli, “Gaussian process approach to buried object size estimation in GPR images,” IEEE Geoscience and Remote Sensing Letters, vol. 7, no. 1, pp. 141–145, Jan. 2009.
41. C. Warren et al., “A CUDA-based GPU engine for gprMax: Open source FDTD electromagnetic simulation software,” Computer Physics Communications, vol. 237, pp. 208–218, Jan. 2019.
42. A. R. V. Hippel, Dielectric materials and applications; papers by twenty-two contributors. published jointly by the Technology Press of M.I.T.; Wiley, 1954.
43. C. Koch, K. Georgieva, V. Kasireddy, B. Akinci, and P. Fieguth, “A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure,” Advanced Engineering Informatics, vol. 29, no. 2, pp. 196–210, Jan. 2015.
44. H. Zakeri, F. M. Nejad, and A. Fahimifar, “Image based techniques for crack detection, classification and quantification in asphalt pavement: A review,” Archives of Computational Methods in Engineering, vol. 24, no. 4, pp. 935–977, Jan. 2017.
45. C. Liu, C.-S. Tang, B. Shi, and W.-B. Suo, “Automatic quantification of crack patterns by image processing,” Computers & Geosciences, vol. 57, pp. 77–80, Jan. 2013.
46. R. Adhikari, O. Moselhi, and A. Bagchi, “Image-based retrieval of concrete crack properties for bridge inspection,” Automation in construction, vol. 39, pp. 180–194, Jan. 2014.
47. C. Zhang and A. Elaksher, “An unmanned aerial vehicle-based imaging system for 3D measurement of unpaved road surface distresses 1,” Computer-Aided Civil and Infrastructure Engineering, vol. 27, no. 2, pp. 118–129, Jan. 2012.
48. Y. Liu and L. J. Durlofsky, “3D CNN-PCA: A deep-learning-based parameterization for complex geomodels,” Computers & Geosciences, p. 104676, Jan. 2020.
49. K. Guo et al., “Angel-eye: A complete design flow for mapping CNN onto embedded FPGA,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 37, no. 1, pp. 35–47, Jan. 2017.
50. D. T. Nguyen, T. N. Nguyen, H. Kim, and H.-J. Lee, “A high-throughput and power-efficient FPGA implementation of YOLO CNN for object detection,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 27, no. 8, pp. 1861–1873, Jan. 2019.
51. J. C. Cheng and M. Wang, “Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques,” Automation in Construction, vol. 95, pp. 155–171, Jan. 2018.
52. G. Chen, Z. Jiang, and M. Kamruzzaman, “Radar remote sensing image retrieval algorithm based on improved sobel operator,” Journal of Visual Communication and Image Representation, vol. 71, p. 102720, Jan. 2020.
53. F. Bergholm, “Edge focusing,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI–9, no. 6, pp. 726–741, Jan. 1987, doi: 10.1109/TPAMI.1987.4767980.
54. H. Di and D. Gao, “Gray-level transformation and canny edge detection for 3D seismic discontinuity enhancement,” Computers & Geosciences, vol. 72, pp. 192–200, Jan. 2014.
55. A. Botev, G. Lever, and D. Barber, “Nesterov’s accelerated gradient and momentum as approximations to regularised update descent,” in 2017 international joint conference on neural networks (IJCNN), IEEE, Jan. 2017, pp. 1899–1903.
56. S. S. Todkar et al., “Detection of debondings with ground penetrating radar using a machine learning method,” in 2017 9th international workshop on advanced ground penetrating radar (IWAGPR), IEEE, Jan. 2017, pp. 1–6.
57. J. Sonoda and T. Kimoto, “Object identification form GPR images by deep learning,” in 2018 asia-pacific microwave conference (APMC), IEEE, Jan. 2018, pp. 1298–1300.
58. M.-S. Kang, N. Kim, J. J. Lee, and Y.-K. An, “Deep learning-based automated underground cavity detection using three-dimensional ground penetrating radar,” Structural Health Monitoring, vol. 19, no. 1, pp. 173–185, Jan. 2020.

© 2022 Elsevier B.V.