Integration of Functional Link Neural Networks into a Parameter Estimation Methodology

Abstract

In the field of robust design, most estimation methods for output responses of input factors are based on the response surface methodology (RSM), which makes several assumptions regarding the input data. However, these assumptions may not consistently hold in real-world industrial problems. Recent studies using artificial neural networks (ANNs) indicate that input–output relationships can be effectively estimated without the assumptions mentioned above. The primary objective of this research is to generate a new, robust design dual-response estimation method based on ANNs. First, a second-order functional-link-NN-based robust design estimation approach has been proposed for the process mean and standard deviation (i.e., the dual-response model). Second, the optimal structure of the proposed network is defined based on the Bayesian information criterion. Finally, the estimated response functions of the proposed functional-link-NN-based estimation method are applied and compared with that obtained using the conventional least squares method (LSM)-based RSM. The numerical example results imply that the proposed functional-link-NN-based dual-response robust design estimation model can provide more effective optimal solutions than the LSM-based RSM, according to the expected quality loss criteria.

1. Introduction

Over the past two decades, robust parameter design (RPD), also known as robust design (RD), has been widely applied to improve the quality of products in the offline stage of practical manufacturing processes. Bendell et al. [1] and Dehnad [2] discussed the applications of RD to various engineering concerns in the process, automobile, information technology, and plastic technology industries. The primary purpose of RD is to obtain the optimal settings of control factors by minimizing the process variability and deviation based on the mean value and the presupposed target, which are often referred to as the process bias. As Taguchi explored [3], RD includes two main stages: design of experiments and two-step modeling. However, orthogonal arrays, statistical analyses, and signal-to-noise ratios used in conventional techniques to solve RD problems have been questioned by engineers and statisticians, such as León et al. [4], Box [5], Box et al. [6], and Nair et al. [7]. As a result, to resolve these shortcomings, several advanced studies have been proposed.

The most significant alternative to Taguchi’s approach is the dual-response model approach based on the response surface methodology (RSM) [8]. In this approach, the process mean and variance (or standard deviations) are approximated as two separate functions of input factors based on the LSM. In addition, the dual-response model approach provides an RD optimization model that minimizes the process variability while the process mean is assigned equal to the target value. However, the dual-response approach in Vining and Myers [8] may not always provide efficient optimal RD solutions, which have been discussed in Del Castillo and Montgomery [9] and Copeland and Nelson [10]. Instead, they employed the standard nonlinear programming techniques of the generalized reduced gradient method and the Nelder–Mead simplex method to provide better RD solutions. Subsequently, Lin and Tu [11] identified a drawback in the dual-response model approach whereby the process bias and variance are not simultaneously minimized. To overcome this issue, they proposed a mean square error (MSE) model.

The RSM comprises statistical and mathematical techniques to develop, improve, and optimize processes. It helps design, develop, and formulate new products, as well as improve the existing product designs [12]. The unidentified relationship between input factors and output responses can be investigated using the RSM. To define the input–output functional relationship, the conventional LSM is used to estimate unknown model coefficients. The LSM-based RSM assumes that the sample data follow a normal distribution, and the error terms hold a fixed variance with zero mean. Unfortunately, the Gauss–Markov theorem is not applicable in several practical situations, which implies that those assumptions are not valid. Therefore, weighted least squares, maximum likelihood estimation (MLE), and Bayesian estimation methods can be used as alternatives to determine model parameters. Pertaining to MLE, the unknown parameters are considered as constant, and the observed data are treated as random variables [13]. The MLE approach with abnormal distributed data was implemented in Lee and Park [14], Cho et al. [15], and Cho and Shin [16], whereas Luner [17] and Cho and Park [18] proposed the weighted least squares methods to estimate the model coefficients in the case of unbalanced data. Most estimation methods based on the RSM consider several assumptions or require specific data types to determine functions between the process mean or variance and input factors.

Over the past two decades, artificial neural networks (ANNs), often known as neural networks (NNs), have been widely used to classify, cluster, approximate, forecast, and optimize datasets in the fields of biology, medicine, industrial engineering, control engineering, software engineering, environmental science, economics, and sociology. An ANN is a quantitative numerical model that originates from the organization and operation of the neural networks of the biological brain. The basic building blocks of every ANN are artificial neurons, i.e., simple mathematical models (functions). Typical ANNs comprise thousands or millions of artificial neurons (i.e., nonlinear processing units) connected via (synaptic) weights. ANNs can “learn” a task by adjusting these weights. Neurons receive inputs with their associated weights, transform those inputs using activation functions, and pass the transformed information as outputs. It has been theoretically proved that ANNs can approximate any continuous mapping to arbitrary precision without any assumptions [19,20,21,22]. Furthermore, without any knowledge of underlying principles, ANNs can determine unknown interactions between the input and output performances of a process because of their data-driven and self-adaptive properties. Accordingly, the functional correlation between the input and output quality characteristics in RD can be modeled and analyzed by NNs without any assumptions. The integration of an NN into the experiment design procedure of an RD model has been mentioned in Rowlands et al. [23] and Shin et al. [24]. In recent times, Arungpadang and Kim [25] presented a feed-forward NN-based RSM that improved the precision of estimations without additional experiments. Le et al. [26] proposed an NN-based estimation method that identified a new screening procedure to determine the optimum transfer function, so that a more accurate solution can be obtained. A genetic algorithm with NNs has been executed in Su and Hsieh [27], Cook et al. [28], Chow et al. [29], Chang [30], Chang and Chen [31], Arungpadang et al. [32], and Villa-Murillo et al. [33] as an estimation method to investigate the optimal quality characteristics with associated control factor settings in the RD model without the use of estimation formulas. Winiczenko et al. [34] introduced an efficient optimization method by combining the RSM and a genetic algorithm (GA) to find the optimal topology of ANNs for predicting color changes in rehydrated apple cubes.

Therefore, the main objective is to propose a new dual-response estimation approach based on NNs. First, the normal quadratic process mean and standard deviation functions in RD are estimated using the proposed functional-link-NN-based estimation method. Second, the Bayesian information criterion (BIC) is used to quantify the magnitude of neurons in the hidden layer, which affects the coefficients in the estimated input–output equations. Finally, the results of the case study are presented to verify the effectiveness of the proposed NN-based estimation method compared with the LSM-based RSM. The graphical overview of the proposed NN-based estimation method is demonstrated in Figure 1.

The remainder of the study is organized as follows: Section 2 introduces the conventional LSM-based RSM. Section 3 describes the functional-link-NN-based dual-response estimation model. Section 4 explains the outputs generated by the proposed NN-based estimation method and analyzes the LSM-based RSM based on the results of the case study. Finally, Section 5 concludes the study and describes further studies.

2. Conventional LSM-Based RSM

The RSM was first introduced by Box and Wilson [35] and is used to model empirical relationships between output and input variables. Myers [36] and Khuri and Mukhopadhyay [37] present insightful commentaries on the different development phases of the RSM. When the exact functional relationship is very complicated or is unknown, the conventional LSM is used to estimate the input–response functional relationships of the output [38,39]. In general, the estimated second-order response surface functions are used to analyze RD problems. The estimated process mean and standard deviation functions proposed by Vining and Myers [8] can be defined as follows:

$${\widehat{\mu }}_{LSM}\left(x\right)={\widehat{\beta }}_0+{\displaystyle \sum }_{i=1}^p{\widehat{\beta }}_i{x}_i+{\displaystyle \sum }_{i=1}^p{\widehat{\beta }}_i{x}_i^2+{\displaystyle \sum }_{\begin{array}{c}i=1\\ i<j\end{array}}^p{\displaystyle \sum }_{j=1}^p{\widehat{\beta }}_{ij}{x}_i{x}_j$$

(1)

$${\widehat{\sigma }}_{LSM}\left(x\right)={\widehat{\delta }}_0+{\displaystyle \sum }_{i=1}^p{\widehat{\delta }}_i{x}_i+{\displaystyle \sum }_{i=1}^p{\widehat{\delta }}_i{x}_i^2+{\displaystyle \sum }_{\begin{array}{c}i=1\\ i<j\end{array}}^p{\displaystyle \sum }_{j=1}^p{\widehat{\delta }}_{ij}{x}_i{x}_j$$

(2)

where $x=\left(x_1, \dots ,x_i,\dots ,x_j,\dots ,x_p\right)$ is a vector of input variables, and $\widehat{\beta }$ and $\widehat{\delta }$ are the estimated coefficients in the mean and standard deviation functions, respectively. These regression coefficients can be estimated using the LSM as

$$\widehat{\beta }={\left(X^{\mathrm{T}}X\right)}^{-1}{X}^{\mathrm{T}}\overline{y}$$

(3)

$$\widehat{\delta }={\left(X^{\mathrm{T}}X\right)}^{-1}{X}^{\mathrm{T}}s$$

(4)

where $X$ is the design matrix of control factors, and $\overline{y}$ and $s$ are the mean and standard deviation of the observed responses, respectively.

3. Proposed Dual-Response Model Based on Functional Link NNs

3.1. Proposed Functional Link NN

ANNs are comprised of various processing elements known as artificial neurons or nodes, which are interconnected. Each neuron obtains input signals from the previous nodes, aggregates these signals with the related weights, and generates output signals through a transfer function (or activation function). This output signal forms the input signal for other nodes. The multilayer feed-forward back-propagation NN is the most influential model applied to various practical problems. A multilayer feed-forward back-propagation NN model comprises several layers; each layer contains several nodes. The first and last layers inside the network are considered as input and output layers, respectively, as the input and output units in the NN system are involved. Various hidden layers are located between the input and output layers. A multilayer feed-forward back-propagation NN comprises one input layer, one output layer, and various hidden layers between the input and output layer. The overall structure of a multilayer feed-forward back-propagation NN is illustrated in Figure 2.

The universal approximation theorem for multilayer feed-forward networks was proposed by Cybenko [21] and Hornik et al. [20]. A multilayer feed-forward NN with a hidden layer can approximate a multidimensional, continuous, and arbitrary nonlinear function with any desired accuracy, as mentioned in Funahashi [22] and Hartman et al. [40], based on the theorem stated by Hornik et al. [20] and Cybenko [21]. In the hidden area, the transfer function is used to figure out the functional formation between the input and output factors. Popular transfer functions used in ANNs include step-like, hard limit, sigmoidal, tan sigmoid, log sigmoid, hyperbolic tangent sigmoid, linear, radial basis, saturating linear, multivariate, softmax, competitive, symmetric saturating linear, universal, generalized universal, and triangular basis transfer functions [41,42]. In RD, there are two characteristics of the output responses that are of particular interest: the mean and standard deviation. Each output performance can be separately analyzed and computed in a single NN structure based on the dual-response estimation framework. Figure 3 illustrates the proposed functional-link-NN-based dual-response estimation approach.

As shown in Figure 3, $x_1, x_2, \dots , x_k$ denote $k$ control variables in the input layer. The weighted sum of the $k$ factors with their corresponding biases $b_1, b_2, \dots , b_h$ can represent the input for each hidden neuron. This weighted sum is transformed by the activation function $x+x^2$, also known as the transfer function. The transformed combination is the output of the hidden layer $y_{hid}$ and refers to the input of one output layer as well. Analogously, the integration of the transformed combination of inputs with their relevant biases can represent the output neuron ($\widehat{y}$ or $\widehat{s}$). The linear activation function $x$ can represent the output neuron transfer function. In an h-hidden-node NN system, $1, \dots , j, \dots , h$, are denoted as the hidden layer, and $w$ and $b$ represent the weight term and process bias, separately. In particular, the weight connection between the input factor $x_i$ and hidden node $j$ is written as $w_{ji}$, while $w_j$ is the weight connection between the hidden node $j$ and the output. In addition, $b_j^{hid}$ and $b^{out}$ represent deviations at $j$ and the output, respectively. The output performance of the layers in the hidden neuron can be represented in mathematical formulas as:

$$y_j^{hid}\left(x\right)=\left({\displaystyle \sum }_{i=1}^k{w}_{ji}{x}_i+b_j^{hid}\right)+{\left({\displaystyle \sum }_{i=1}^k{w}_{ji}{x}_i+b_j^{hid}\right)}^2$$

(5)

The outcome of the functional-link-NN-based RD estimation model can be written as:

$${\widehat{y}}_{out}\left(x\right)={\displaystyle \sum }_{j=1}^h{w}_j\left[\left({\displaystyle \sum }_{i=1}^k{w}_{ji}{x}_i+b_j^{hid}\right)+{\left({\displaystyle \sum }_{i=1}^k{w}_{ji}{x}_i+b_j^{hid}\right)}^2\right]+b^{out}$$

(6)

Hence, the regressed formulas for the estimated mean and standard deviation are given as:

$${\widehat{\mu }}_{NN}\left(x\right)={\displaystyle \sum }_{j=1}^{h\_mean}{w}_j\left[\left({\displaystyle \sum }_{i=1}^k{w}_{ji}{x}_i+b_j^{hid\_mean}\right)+{\left({\displaystyle \sum }_{i=1}^k{w}_{ji}{x}_i+b_j^{hid\_mean}\right)}^2\right]+b_{mean}^{out}$$

(7)

$${\widehat{\sigma }}_{NN}\left(x\right)={\displaystyle \sum }_{j=1}^{h\_std}{w}_j\left[\left({\displaystyle \sum }_{i=1}^k{w}_{ji}{x}_i+b_j^{hid\_std}\right)+{\left({\displaystyle \sum }_{i=1}^k{w}_{ji}{x}_i+b_j^{hid\_std}\right)}^2\right]+b_{std}^{out}$$

(8)

where $h\_mean$ and $h\_std$ denote the quantity of the hidden neurons of the h-hidden-node NN for the mean and standard deviation functions, respectively.

3.2. Learning Algorithm

The learning or training process in NNs helps determine suitable weight values. The learning algorithm back-propagation is implemented in training feed-forward NNs. Back-propagation means that the errors are transmitted backward from the output to the hidden layer. First, the weights of the neural network are randomly initialized. Next, based on presetting weight terms, the NN solution can be computed and compared with the desired output target. The goal is to minimize the error term E between the estimated output ${\widehat{y}}_{out}$ and the desired output $y_{out}$, where:

$$E=\frac{1}{2}\sum {\left(y_{out}-{\widehat{y}}_{out}\right)}^2$$

(9)

Finally, the iterative step of the gradient descent algorithm modifies $w_j$ refers to:

$$w_j\to w_j+\Delta w_j$$

(10)

where

$$\Delta w_j=-\eta \frac{\partial E\left(w\right)}{\partial w_j}$$

(11)

The parameter $\eta (>0)$ is known as the learning rate. While using the steepest descent approach to train a multilayer network, the magnitude of the gradient may be minimal, resulting in small changes to weights and biases regardless of the distance between the actual and optimal values of weights and biases. The harmful effects of these small-magnitude partial derivatives can be eliminated using the resilient back-propagation training algorithm (trainrp), in which the weight updating direction is only affected by the sign of the derivative. In addition, the Marquardt–Levenberg algorithm (trainlm), an approximation to Newton’s method, is defined such that the second-order training speed is almost achieved without estimating the Hessian matrix.

One problem with the NN training process is overfitting. This is characterized by large errors when new data are presented to the network, despite the errors on the training set being very small. This implies that the training examples have been stored and memorized in the network, but the training experiences cannot generalize new situations. To avoid the overfitting problem, efficient researchers can use an additional technique of “early stopping” to improve the generalization ability. In this model, the dataset is separated into three subsets, which are specialized to train, validate, and test the database. The process weight and bias terms of the network can be updated in the training set, in which the gradient is estimated as well. Then, the error, which is supervised during the training process, has to be evaluated in the validation set. While in the testing set, the capability to generalize the supposedly trained network can be examined. The accurate proportion of the learning algorithm among training, testing, validation data is determined by the designer; typically, the ratios of training:testing:validation are 50:25:25, 60:20:20, or 70:15:15.

3.3. Number of Hidden Neurons

Based on the number of layers in the hidden neuron, the optimal NN structure can be decided. A random selection of the number of hidden neurons can cause overfitting or underfitting problems. Several approaches can determine the number of hidden neurons in NNs—a literature review can be found in Sheela and Deepa [43]. However, no single method is effective and accurate considering various circumstances. In this study, Schwartz’s Bayesian criterion, known as BIC, can help determine the number of hidden neurons. The BIC is given by:

$$BIC=n\ast ln\left(\frac{1}{n}{\displaystyle \sum }_{i=1}^n{E}^2\right)+p\ast \ln \left(n\right)$$

(12)

where $n$ and $p$ represent the magnitude of the sample data and the number of variables in the mathematical formula, respectively. $\ln \left(n\right)$ in BIC tends to significantly penalize complex models. Moreover, while the size of the dataset $n$ increases, the BIC will be more likely to decide matched-model approaches.

4. Case Study

The printing data proposed by Box and Draper [38] are discussed in this study for comparative analysis; these data had been used by Vining and Myers [8] and Lin and Tu [11] as well. Three experimental parameters, $x_1$, $x_2$, and $x_3$ (speed, pressure, and distance), of a printing machine are treated as input variables to examine the ability to apply colored inks to package levels (y). These three control factors are assumed to be examined in three levels (−1, 0, +1), so that there are 27 runs in total. Based on the general full factorial design in the design of experiments, it contains 27 experimental runs considering all combinations of three levels of three factors. The order of the experiment was set in the standard order, and three repeated experiments were performed for each run. Experimental data (Box and Draper [38]) lists the experimental configurations, which include process mean, standard deviation, and variability, with their corresponding design points.

A variety of criteria have been used to analyze RD solutions. Among them, the expected quality loss (EQL) is widely used as a critical optimization criterion. The expectation of the loss function can be expressed as

$$\mathrm{EQL}=\theta \left[{\left(\widehat{\mu }\left(x\right)-\tau \right)}^2+{\widehat{\sigma }}^2\left(x\right)\right]$$

(13)

where $\theta$ signifies a positive loss coefficient, $\theta =1$, and $\widehat{\mu }\left(x\right)$, $\tau$, and $\widehat{\sigma }\left(x\right)$ are the estimated mean function, desirable target value, and estimated standard deviation function, respectively. In this example, the target value is $\tau =500$.

As this model does not exhibit the unrealistic constraint of forcing the estimated mean response to a specific target value, it avoids misleading the zero-bias logic. The main objective of minimizing process bias and variability to obtain efficient solutions has allowed a slight bias between the estimated mean and its assigned target. For this reason, the EQL is selected as an identification and comparison tool to evaluate optimal solutions obtained from each model.

MATLAB is used in this study to perform the estimated regression functions of mean and standard deviation using the proposed dual-response approach and conventional LSM-based RSM, respectively. The correlation coefficients of the estimated response functions based on Vining and Myers’ [8] dual-response approach are listed in

Table 1. Coefficients of the estimated response functions using LSM.

Treatment Combinations	Coefficients
	$\mathbf{Mean}{\widehat{\mathit{\mu }}}_{\mathit{L}\mathit{S}\mathit{M}}$	$\mathbf{Standard}\mathbf{Deviation}{\widehat{\mathit{\sigma }}}_{\mathit{L}\mathit{S}\mathit{M}}$
Constant	327.630	34.883
$x_1$	177.000	11.527
$x_2$	109.430	15.323
$x_3$	131.460	29.190
$x_1^2$	32.000	4.204
$x_2^2$	−22.389	−1.316
$x_3^2$	−29.056	16.778
$x_1{x}_2$	66.028	7.720
$x_1{x}_3$	75.472	5.109
$x_2{x}_3$	43.583	14.082

.

Table 2. Parameters of NN-based estimation method.

Objective	Response Function	Training Algorithm	Structure	No. of Epoch
Mean	mse	Trainlm	3-21-1	13
Std	mse	Trainlm	3-2-1	12

lists the proposed NN-functional-link-based dual-response RD estimation model after the training procedure.

The weights and biases of the NN for the estimated mean and standard deviation functions are listed in

Table 3. Weight and bias terms of the NN for the estimated process mean.

Weight				Bias
$W_{in\_hid}^{mean}$			${\left(w_{hid\_out}^{mean}\right)}^T$	$b_{hid}^{mean}$	$b_{out}^{mean}$
0.96075	0.11736	1.54028	2.10096	3.63174	1.00092
0.75123	0.38223	0.73934	1.62200	0.77913
−0.28537	−0.34012	−0.80124	2.30133	3.88614
1.17461	0.63199	1.11264	1.73056	1.68918
0.27560	0.60510	−0.26521	−0.48992	−0.70557
−0.72625	0.41018	0.21240	−0.11370	−0.84332
−0.45138	−0.37180	0.56006	−1.03860	−0.39605
−0.40578	−0.11631	−0.02559	−0.09612	−0.44870
0.75884	−0.59636	−0.37276	−0.29991	−0.43415
2.86524	1.95064	1.96605	4.72650	5.36510
−1.13144	−0.73588	−1.17218	0.84079	−1.47882
−0.06226	−0.41228	−0.58818	0.40969	0.05234
0.32760	−0.75682	−0.67588	−0.11504	−0.02238
−0.01851	−0.81573	0.01320	−0.27318	−0.58988
0.11633	0.16928	0.17376	−0.45037	−0.88337
−0.68532	0.37096	−0.27889	−0.27210	0.04470
−0.27500	−0.52907	0.34659	−0.85252	−0.31859
0.91857	0.59698	0.76126	0.59614	0.80572
0.29861	−0.39570	0.10545	0.28709	0.51167
0.56297	−0.03477	−0.09037	0.43088	0.67887
0.10126	−0.37625	−0.01066	−0.64268	−0.36799

and

Table 4. Weight and bias terms of the NN for the estimated process standard deviation.

Weight				Bias
$W_{in\_hid}^{std}$			${\left(w_{hid\_out}^{std}\right)}^T$	$b_{hid}^{std}$	$b_{out}^{std}$
−2.04505	−3.02946	−4.90330	0.86246	−4.32652	−0.01267
−0.38368	−1.22474	−0.62419	−2.32404	−2.08030

, respectively. In these tables, $W_{in\_hid}^{mean}$, ${\left(w_{hid\_out}^{mean}\right)}^T$, $b_{hid}^{mean}$, and $b_{out}^{mean}$ represent the weight connection from the input to the hidden layers, the weight connection from the hidden layers to the output, the process bias in the hidden layers, and the process bias in the output layer of the observed mean formula, respectively. Similarly, $W_{in\_hid}^{std}$, ${\left(w_{hid\_out}^{std}\right)}^T$, $b_{hid}^{std}$, and $b_{out}^{std}$ represent the weight connection from the input to the hidden layers, the weight connection from the hidden layers to the output, the process bias in the hidden layers, and the process bias in the output layer of the observed standard deviation formula, respectively.

According to the estimated regression formulas of the process mean and standard deviation, the response functions of the dual-response models between parameters $x_1$ and $x_2$ for two estimation methods (i.e., LSM and NN) are illustrated in Figure 4 and Figure 5, including statistical indexes such as coefficients of determination $(R^2)$ and root-mean-square error (RMSE). The rest of parameters $x_1$ and $x_3$, $x_2$ and $x_3$ are additionally provided as figures in Appendix A. The process mean, bias, and variance obtained from the proposed functional-link-NN-based dual-response estimation approach and the conventional LSM-based RSM are compared in

Table 5. Comparative analysis results.

Estimation Model	Optimal Factor Settings			Process Mean	Process Bias	Process Variance	EQL
	${\mathit{x}}_1$	${\mathit{x}}_2$	${\mathit{x}}_3$
LSM	1.0000	0.07153	−0.25028	494.67226	5.32773	1977.53266	2005.91742
Proposed NN	0.99999	0.99999	−0.79327	496.39189	3.60810	450.64771	463.66611

based on the corresponding optimal input variable settings and the EQL results. Table 5 implies that the proposed RD dual-response estimation approach based on the functional link NN produced a significantly smaller EQL value than the conventional LSM-based RSM. In addition, compared with the conventional LSM, the proposed method can provide lower process variance. The process mean and the squared bias with regard to the variability of the standard LSM-based RSM and proposed NN-based estimation model are shown in Figure 6. The criterion spaces of the estimated regression functions are denoted by red stars. The optimal settings are marked as green star in each figure.

5. Conclusions and Further Studies

There have been many studies to improve the RSM by combining statistical and mathematical techniques, but there are cases where particular data types are required or assumptions are made to define functions between mean, variance, and input elements. NNs, which have recently been widely applied in artificial intelligence, can present simple mathematical models (functions) using artificial neurons and determine unknown interactions between the input and output performance of a process without any knowledge of the principle.

This study has described a functional-link-NN-based estimation method that offers an alternative RD technique without assumptions inherent in the conventional LSM-based RSM. Compared with the existing RD dual-response estimation approach, the proposed method provides significant advantages in determining the functional relationship between the control factors and output performances and the optimal solutions. The proposed dual-response estimation approach can be quickly and efficiently implemented using MATLAB (see Appendix B). Experimental results show that the proposed NN-based estimation method can achieve better solutions than the conventional LSM-based RSM in the EQL metric.

In the future, the proposed functional-link-NN-based dual-response RD estimation approach will be extended to time series data and multiple-response optimization problems. In addition, we plan to search for the optimal structure by binary coding the neural network structure with a genetic algorithm and conducting research on optimizing the weights of the neural network.