Mechanical Performance Prediction for Sustainable High-Strength Concrete Using Bio-Inspired Neural Network

Abstract

High-strength concrete (HSC) is a functional material possessing superior mechanical performance and considerable durability, which has been widely used in long-span bridges and high-rise buildings. Unconfined compressive strength (UCS) is one of the most crucial parameters for evaluating HSC performance. Previously, the mix design of HSC is based on the laboratory test results which is time and money consuming. Nowadays, the UCS can be predicted based on the existing database to guide the mix design with the development of machine learning (ML) such as back-propagation neural network (BPNN). However, the BPNN’s hyperparameters (the number of hidden layers, the number of neurons in each layer), which is commonly adjusted by the traditional trial and error method, usually influence the prediction accuracy. Therefore, in this study, BPNN is utilised to predict the UCS of HSC with the hyperparameters tuned by a bio-inspired beetle antennae search (BAS) algorithm. The database is established based on the results of 324 HSC samples from previous literature. The established BAS-BPNN model possesses excellent prediction reliability and accuracy as shown in the high correlation coefficient (R = 0.9893) and low Root-mean-square error (RMSE = 1.5158 MPa). By introducing the BAS algorithm, the prediction process can be totally automatical since the optimal hyperparameters of BPNN are obtained automatically. The established BPNN model has the benefit of being applied in practice to support the HSC mix design. In addition, sensitivity analysis is conducted to investigate the significance of input variables. Cement content is proved to influence the UCS most significantly while superplasticizer content has the least significance. However, owing to the dataset limitation and limited performance of ML models which affect the UCS prediction accuracy, further data collection and model update must be implemented.

1. Introduction

High-strength concrete (HSC) is a type of cementitious material that has uniaxial compressive strength (UCS) larger than 40 MPa [1,2,3]. The HSC composite exhibit outstanding mechanical strength, considerable durability, low permeability, and compact density. In addition, it satisfies special uniformity and performance requirements, which is superior to ordinary fabricated concrete [4,5,6]. HSC has been widely applied in long-span bridges because it sustains superior dead and live loading with fewer bridge piers and thus prolongs the service lifespan [7,8,9]. Meanwhile, HSC is also promising in high buildings because it enables oversized columns to yield more floor space and larger column spacing without detracting from lower floors [10,11,12]. The behaviour of connectors when embedded in the HSC, such as the shear resistance and ductility, has been investigated by several researchers [13,14]. The general behaviour of HSC beams was investigated based on the mid span deflection, failure mode, and crack growth [15]. HSC can also be served as the main construction material accompanied by several advanced technologies, such as building information modelling, 3D printing technology, etc. [16,17,18,19,20,21,22,23]. Besides, solid waste materials, such as waste glass and recycled aggregate, have the potential to be applied in HSC to overcome the strength shortcomings of the waste itself [24,25,26,27,28,29,30,31]. Therefore, HSC incorporated with solid wastes has the benefits of both strength enhancement and sustainable prospect [32,33,34].

For HSC composites, the uniaxial compressive strength (UCS) is the most significant factor in the design procedure before application. Numerous experiments of HSC by the research facilities have been carried out to investigate the relationship between UCS and its composite constituents. However, the progress is costly and lengthy because too many trial batches have to be prepared to explore desirable mechanical performance with a large number of influencing variables. The pre-configuration of equipment also consumes time and resources. Some conventional evaluation strategies have been used to predict the UCS of HSC composites, such as non-linear regression and linear regression. However, it is still challenging to conduct accurate prediction by applying simple regression models and advanced techniques are in great demand [35,36].

To overcome the above difficulties, machine learning (ML) algorithms have been developed rapidly for predicting the USC of concrete materials. ML models make predictions and decisions by building a mathematical model without being explicit programming based on sample data [37,38,39]. Many ML models have been used to predict concrete strength, such as neural networks, support vector regression (SVR), and tree-based models [40,41,42,43]. For instance, Huynh et al., (2020) [44] utilised artificial neural network (ANN), deep neural network (DNN), and deep residual network (ResNet) to predict the compressive strength of fly ash-based geopolymer concrete. Besides, the deep neural network (DNN) has been applied to perform structural reliability analysis and structural damage detection of truss structures [45,46]. The Extreme Learning Machine (ELM) and ANN were applied and compared to predict the compressive strength of concrete containing fly ash and silica fume [47]. The estimations of moment and rotation in steel rack connections and beam-to-column connections were implemented through ELM [48,49]. Mohammadhassani et al., (2014) [50] used an Adaptive Neuro-Fuzzy Inference System (ANFIS) to predict the shear strength of high strength concrete (HSC) beams without stirrups. The ML models can also be used to further propose multi-objective optimisation design [51,52,53]. Among most ML models, the back-propagation neural network (BPNN) demonstrates superior predicting capacity for solving engineering problems. The main reason is that BPNN is fast and easy to program without parameters to tune apart from the number of neurons in the hidden layer [54,55,56]. Therefore, BPNN is chosen as the prediction ML model in this study.

Generally, the number of hidden layers and the optimal number of neurons in each hidden layer are two parameters which significantly affect the performance of BPNN. To determine the two values, traditional trial and error methods are widely used, which is a waste of effort and time. To overcome the shortcoming, some meta-heuristic algorithms were developed for ML model optimisation. Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) accompanied with ANN were applied for properties prediction [57,58]. These meta-heuristic algorithms also have extensive use in other ML models. For instance, Sharafati et al., (2020) [59,60] developed a combination of adaptive neuro-fuzzy inference system (ANFIS) with several meta-heuristic algorithms (e.g., PSO) to predict the shear strength of HSC slender beam and compressive strength of foamed concrete. A SVR-GA was employed to predict the shear strength of reinforced concrete (RC) deep beams [61]. Multivariate Adaptive Regression Splines optimized using Water Cycle Algorithm (MARS-WCA) was developed for the prediction of the compressive strength of concrete [62]. The grey wolf optimizer (GWO) was implemented with ELM to predict the compressive strength of concrete with partial replacements for cement [63]. It also successfully predicts the behaviour of channel shear connectors in composite floor systems at different temperatures [64]. A Support Vector Machine (SVM) coupled with Firefly Algorithm (FFA) was performed for the shear capacity estimation of angle shear connectors [65]. The Beetle Antennae Search (BAS) is another feasible meta-heuristic algorithm to tune BPNN architecture with fast convergence, stability in local optimization and uncomplicated implementation [41,66,67]. Therefore, BAS algorithm is chosen to tune the hyperparameters of BPNN. Some robust optimisers are also proposed recently such as adaptive hybrid evolutionary firefly algorithm (AHEFA), hybrid differential evolution and symbiotic organisms search (HDS), and evolutionary symbiotic organisms search algorithm (ESOS) [68,69,70].

In this study, the focus is on predicting the UCS of HSC using BAS-BPNN and understanding the sensitivity ranking of varying influencing factors upon the strength performance of HSC. Different from the traditional ML models, this study develops a novel ML model comprising BPNN and BAS architectures based on a total of 324 experiment data from the literature. The BAS algorithm possesses fast convergence which is beneficial to analysis on the basis of a large database. This pioneering research supplies a novel method to predict the mechanical strength of HSC for advanced engineering construction and application.

2. Dataset

A total of 324 HSC data samples are collected from previous literature [71] (listed in the Appendix A). Type 1 ordinary Portland cement (OPC) is used as binder material. Silica sand is incorporated as fine aggregate (FA) and the gravel with the size less than 20 mm is served as coarse gravel aggregate (CA). A polycarboxylate-based superplasticizer (SP) with a density of 1.06 g/cm³ is also introduced for adjusting the cement fluidity and segregation performance.

The specific statistics of the input and output variables are summarised in

Table 1. Chart of input and output statistics.

ID	Data	Unit	Minimum	Maximum	Mean Value
Cement	Input	kg/m3	284	600	417
Water	Input	kg/m3	160	180	170
Coarse Aggregate	Input	kg/m3	845	989	899
Fine Aggregate	Input	kg/m3	552	951	768
SP	Input	kg/m3	0	2	0.95
UCS	Output	MPa	37.5	73.6	52

based on the database (Appendix A). All the five influencing variables comprise the content of cement, fine and coarse aggregates, water, and SP. The correlation coefficient distribution is computed, as shown demonstrated in Figure 1. According to the result, the UCS is highly correlated with cement. For input variables, most of the correlations are relatively low (less than 0.5), suggesting that these variables will not produce multicollinearity problems [72,73,74].

3. Methodology

3.1. BPNN

The artificial neural network (ANN) is one of the commonly used machine learning models, which comprises many categories such as recurrent neural networks (RNN) and feedforward neural network (FFNN). The FFNN includes the Back-propagation neural network (BPNN), which is widely employed to solve problems in the field of building materials and construction [42,75,76]. Back propagation (BP) is a popular approach to adjust the weights and bias of the model, which is composed of an input layer, one or more hidden layers, and one output layer. The BP process will compare the actual outputs and predicted outputs to obtain the optimal weight and threshold values of the network. The output (O) of a neuron is computed as follow

$$\mathrm{O}=\mathrm{f}\left({\displaystyle \sum }_{\mathrm{j}=1}^{\mathrm{n}}\left({\mathrm{w}}_{\mathrm{j}}{\mathrm{x}}_{\mathrm{j}}\right)+\mathrm{b}\right), \mathrm{UCS} \left(\mathrm{MPa}\right)$$

(1)

where ${\mathrm{w}}_{\mathrm{j}}$ represents the weight value of the jth input neuron (${\mathrm{x}}_{\mathrm{j}}$) in the previous layer; b is the bias value of the output neuron; f denotes the activation function. In this study, the following active function was used mainly due to its superior performance [75]:

$$\mathrm{f}\left(\mathrm{x}\right)=\frac{2}{1+\exp \left(-\mathrm{x}\right)}-1$$

(2)

In the backpropagation process, the method computes the gradient of the error function with respect to the weights of the neural networks. The training iteration will stop when the mean square error (MSE) between the actual and predicted outputs become smaller than a defined threshold. The topology of the backpropagation process is shown in Figure 2.

3.2. BAS

The BAS algorithm is a recently proposed metaheuristic optimization algorithm [77]. It is inspired by the hunting behavior of the longhorn beetle with its two long antennae. The beetle gradually moves to the food source (the global optimum). Therefore, the concentration of odour is represented by the objective function at position x. In a multi-dimensional space, the global optimum (source point) lies in the position with the best objective value. The beetle’s searching behaviour is given by:

$$x_r=x^i+d^{i}b$$

(3)

$$x_l=x^i-d^{i}b$$

(4)

where $x_r$and $x_l$ represent the areas in the right-hand side and left-hand side, respectively; $x^i$is the position at an $i$th time instant.$d^i$ denotes the length of the beetle’s antennae at ith iteration.$b$denotes a unit vector that is randomly normalized, which is expressed as

$$\mathrm{b}=\frac{\mathrm{rnd}\left(\mathrm{k},1\right)}{\Vert \mathrm{rnd}\left(\mathrm{k},1\right)\Vert }$$

(5)

where k denotes the dimensionality of the position; $\mathrm{rnd}\left(·\right)$ is a random function.

The beetle’s detecting behaviour is determined using the following equation:

$$x^{i+1}=x^i+{\delta }^{i}b·\mathrm{sign}\left(f\left(x_r\right)-f\left(x_l\right)\right)$$

(6)

where$\mathrm{sign}\left(·\right)$ is the sign function; ${\delta }^i$ represents the step size at the ith iteration, which is updated using the following formula:

$${\delta }^{i+1}=\eta {\delta }^i$$

(7)

where $\eta$ is the attenuation coefficient of the step size.

The flowchart of BAS is shown in Figure 3 and the pseudocode of tuning hyperparameters of BPNN using BAS is presented in Figure 4.

3.3. Performance Evaluation

In this study, Root-mean-square error (RMSE) and Correlation coefficient (R) are used to evaluate the performance of the proposed model. RMSE and R are calculated as follows

$$\mathrm{RMSE}=\sqrt{\frac{1}{n}{{\displaystyle \sum }}_{i=1}^n{\left(y_i^{*}-y_i\right)}^2}, \mathrm{MPa}$$

(8)

where n denotes the number of data samples; $y_i^{*}$ is the predicted value; $y_i$ represents the actual value;

$$R=\frac{{{\displaystyle \sum }}_{i=1}^n\left(y_i^{*}-\overline{y^{*}}\right)\left(y_i-\overline{y}\right)}{\sqrt{{{\displaystyle \sum }}_{i=1}^n{\left(y_i^{*}-\overline{y^{*}}\right)}^2}\sqrt{{{\displaystyle \sum }}_{i=1}^n{\left(y_i-\overline{y}\right)}^2}}, \mathrm{d}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{l}\mathrm{e}\mathrm{s}\mathrm{s}$$

(9)

where and are the mean value of predicted and observed values, respectively.

3.4. Determination of Architecture of BPNN

The hidden layer and the number of neurons in each hidden layer are optimised using BAS in this study. To tune these hyperparameters, 10-fold cross validation (CV) was performed in the training set (Figure 5). The training set is divided into 10 folds, in which 9 folds are used to tune the number of neurons by BAS, and the performance of the BPNN model with the optimal architecture is validated in the remaining fold. After repeating 10 times (for each time, a different fold is selected as the validation fold), the average neuron number is selected as the final neuron number used in this study. Finally, 30% of the data in the test set are used to test the performance of the BPNN with optimal architecture.

4. Results and Discussion

4.1. Results of Hyperparameter Tuning

In this study, the number of neurons in each layer is tuned using the BAS algorithm. In each fold, the RMSE obtained by the BPNN (with optimal neuron number of this fold) is plotted in Figure 6. The smallest RMSE values versus iterations corresponding to varying hidden layers are shown in Figure 7, which presents the process of neuron number tuning. It can be seen that the RMSE decreases to its minimum value within 40 iterations, suggesting that BAS has high efficiency in finding the optimal number of neurons. Ultimately, the final hidden layer is 1 and the corresponding optimal neuron number is 24.

4.2. Performance of the BAS-BPNN Model

Figure 8 shows the actual values (blue line), predicted values (red point), and errors between the actual and predicted UCS (yellow bar graph). It can be observed that although several large noises are observed, most of the errors are pretty small on the training set (Figure 8a) and test set (Figure 8b). This result indicates that the BAS-BPNN model is highly accurate. The correlation between the actual and predicted UCS is visualized in Figure 9. High prediction accuracy is observed on the training set (Figure 9a) and test set (Figure 9b), as indicated by the high R values (0.9971 and 0.9893 on the training and test sets, respectively) and low RMSE values (0.7167 MPa and 1.5158 MPa on the training and test sets, respectively). Compared with previously published papers [42,51], the obtained results show much higher accuracy (R is around 0.99), which might be attributed to the model performance or the accuracy and size of the database. Furthermore, no overfitting problems take place as the test set RMSE (and R) is close to that on the training set. Owing to the inherent stochastic properties of the BAS algorithm, the statistical outcomes of extra 20 run times are also reported in

Table 2. Statistical outcomes (RMSE, R) of the BAS-BPNN for the extra 20 run times.

Run Time	RMSE of Training and Test Sets (MPa)	R of Training and Test Sets	Run Time	RMSE of Training and Test Sets (MPa)	R of Training and Test Sets
1	1.0036, 1.1543	0.9945, 0.9922	11	1.3392, 1.1305	0.9893, 0.9942
2	1.0879, 1.3876	0.9933, 0.9894	12	0.9725, 1.0086	0.9949, 0.9937
3	0.8851, 1.2874	0.9956, 0.9906	13	1.1523, 1.1125	0.9929, 0.9919
4	1.0482, 1.0455	0.9935, 0.9945	14	1.1654, 1.3617	0.9925, 0.9898
5	1.3044, 1.6762	0.9914, 0.9844	15	0.9353, 1.0637	0.9951, 0.9940
6	0.8887, 1.1730	0.9955, 0.9927	16	1.1210, 1.3923	0.9929, 0.9891
7	1.0657, 1.2421	0.9940, 0.9919	17	1.1539, 1.3263	0.9928, 0.9896
8	0.9304, 1.8270	0.9953, 0.9804	18	1.2006, 1.2839	0.9925, 0.9901
9	0.9678, 1.3371	0.9948, 0.9898	19	0.9878, 1.2744	0.9945, 0.9913
10	1.2053, 1.5601	0.9921, 0.9875	20	0.8121, 1.1521	0.9960, 0.9945

to verify the robustness of the introduced ML model.

4.3. Variable Importance

Global sensitivity analysis (GSA) is combined with the developed BPNN model to analyse the variable importance (Figure 10). It can measure the impact on the proposed BAS-BPNN output when the input value changes within its value range [79]. The data sample is represented as $\mathrm{x}$, and ${\mathrm{x}}_a, a\in \left\{1, \dots , M\right\}$ denotes an input variable through its range with L levels ($M$ is the number of input variables). And $y$ represents the UCS value which is predicted by the BPNN. According to the range of ${\mathrm{x}}_a$ and L levels, the input variable ${\mathrm{x}}_a$ can be divided into $i$ values, namely, ${\mathrm{x}}_{ai}, i=\left\{1, \dots , L\right\}$. The respective sensitivity response of each input variable is calculated by Equation (10). Afterward, the relative importance of each variable is calculated by Equation (11).

$$g_a={\displaystyle \sum }_{i=2}^L\frac{\left|\widehat{y_{a,i}}-\widehat{y_{a,i-1}}\right|}{L-1}$$

(10)

$$R_a=g_a/{\displaystyle \sum }_{i=1}^L{g}_i$$

(11)

where $a$ is the input variable that needs to be analysed; $\widehat{y_{a,i}}, i=\left\{1, \dots , L\right\}$ stands for the sensitivity response indicator for ${\mathrm{x}}_{ai}, i=\left\{1, \dots , L\right\}$; $R_a$ is the relative importance of the variable.

It can be observed that UCS of HSC is the most sensitive to contents of cement and water with importance ratios of 44.9% and 34.9%, respectively. This is mainly due to the water-to-cement ratio, which is crucial to the development of concrete strength. It is interesting to note that superplasticiser (importance ratio = 2.7%) is not as important as other influencing variables. This may be caused by insufficient content of superplasticiser in the concrete mixtures. It is worthwhile to note that the importance of input variables is calculated on the basis of the data set collected in this study, as listed in the Appendix A.

4.4. Comparison of the BAS-BPNN Model with Other ML Models

To seek the optimal ML model and further verify the strength of the established BAS-BPNN model in the prediction of UCS of HSC, its prediction performance is compared with several widely used ML models [80]: Support vector machine (SVM), random forest (RF), K-nearest neighbours (KNN), logistic regression (LR), and multiple-linear regression (MLR). Among these models, the hyperparameters of SVM, RF, and KNN are also tuned by BAS. The tuned hyperparameters with their empirical scopes, initial values, and final values are listed in

Table 3. Hyperparameters of different models.

Classifier	Hyperparameter	Empirical Scope	Initial Value	Final Value
SVM	Coefficient of the penalty term	[1,1000]	16	18.73
	Gamma value of gaussian kernel	[0.1,10]	16	34.88
RF	The minimum number of samples required to split an internal node	[1,10]	40	1
	The total number of trees	[2,100]	40	83
KNN	Number of neighbor samples	[1,10]	30	2

. The hyperparameter tuning process of these models is shown in Figure 11. It can be seen that all RMSE curves can converge within 50 iterations, indicating the high searching efficiency of the BAS algorithm. In the first 20 iterations, the RMSE obtained by SVM decreases less significantly in comparison with that obtained by other ML models. This implies the initial hyperparameters of SVM are close to the optimal hyperparameters.

The prediction errors of different ML models are compared on the test set using a boxplot, as shown in Figure 12. The lower edge of the box represents the first quartile, and the upper edge is the third quartile. The median is demonstrated as a red line in the box. The lower and upper whiskers are the 1.5 IQR minus the first quartile and 1.5 IQR above the third quartile, respectively (IQR is the interquartile range). All the other data points are defined as outliers in this study. It can be observed that BPNN has the smallest third quartile, indicating that most of the errors obtained by BPNN are relatively small. Although few outliers were observed in BPNN, the general prediction performance was the best among these ML models. The advantage of BAS-BPNN is also verified by comparing different ML models using a Taylor plot that shows in Figure 13, indicating three model evaluation indices (standard deviation, RMSE, and R). The ML model will be the most realistic if the distance between the ML model and the point labelled “Actual” is the shortest. It can be seen that BPNN is the closest to the “Actual point”, suggesting BPNN performs better in terms of standard deviation, correlation coefficient, and RMSE. Generally, the boxplot and Taylor plot present a similar phenomenon, ranking the accuracy of ML models as BP, SVM, RF, MLR, LR, and KNN. This is controlled according to the model complexity and database suitability. According to the “no free lunch” (NFL) theorem of machine learning, there is no single model that performs universally superior to other models for any dataset. Therefore, based on the dataset used in this study, BPNN is the optimal prediction model.

5. Conclusions

In this study, the BPNN model with the BAS algorithm being used to tune the hyperparameters was established to predict the UCS of HSC. The proposed BAS-BPNN model was developed based on a collected dataset containing over 300 HSC samples with different mixtures. The BPNNs with 1, 2, and 3 hidden layers were compared and the ultimate optimal architecture is one hidden layer with 24 neurons. The results show that BAS has high efficiency in tuning hyperparameters of BPNN and the obtained BAS-BPNN model is highly accurate (R = 0.9893, RMSE = 1.5158 MPa on the test set). Besides, the BAS-BPNN is superior by comparing its prediction performance with other widely used ML models (SVM, RF, KNN, LR, and MLR). In addition, the importance ranking of the input variables through GSA was implemented showing that cement and water are the most significant variables to the UCS of HSC. Generally, the findings in this study can be used in practice to support the HSC mix design.

It is noted that only five input variables are considered in this study, which inevitably influences the diversity and size of the database. Therefore, more samples containing varying raw materials such as fly ash, slags, and other solid wastes will be incorporated in the future to further improve the generalisation ability of the BAS-BPNN model. Also, other active functions, advanced machine learning models, and optimization algorithms (e.g., AHEFA, HDS, and ESOS) can be applied for performance comparison. An Adaptive Neuro-Fuzzy Inference System can be used to determine the most influencing parameters to further verify the findings in this study [81,82].