Fourth-Order Comprehensive Adjoint Sensitivity Analysis (4th-CASAM) of Response-Coupled Linear Forward/Adjoint Systems: I. Theoretical Framework

Abstract

The most general quantities of interest (called “responses”) produced by the computational model of a linear physical system can depend on both the forward and adjoint state functions that describe the respective system. This work presents the Fourth-Order Comprehensive Adjoint Sensitivity Analysis Methodology (4th-CASAM) for linear systems, which enables the efficient computation of the exact expressions of the 1st-, 2nd-, 3rd- and 4th-order sensitivities of a generic system response, which can depend on both the forward and adjoint state functions, with respect to all of the parameters underlying the respective forward/adjoint systems. Among the best known such system responses are various Lagrangians, including the Schwinger and Roussopoulos functionals, for analyzing ratios of reaction rates, the Rayleigh quotient for analyzing eigenvalues and/or separation constants, etc., which require the simultaneous consideration of both the forward and adjoint systems when computing them and/or their sensitivities (i.e., functional derivatives) with respect to the model parameters. Evidently, such responses encompass, as particular cases, responses that may depend just on the forward or just on the adjoint state functions pertaining to the linear system under consideration. This work also compares the CPU-times needed by the 4th-CASAM versus other deterministic methods (e.g., finite-difference schemes) for computing response sensitivities These comparisons underscore the fact that the 4th-CASAM is the only practically implementable methodology for obtaining and subsequently computing the exact expressions (i.e., free of methodologically-introduced approximations) of the 1st-, 2nd, 3rd- and 4th-order sensitivities (i.e., functional derivatives) of responses to system parameters, for coupled forward/adjoint linear systems. By enabling the practical computation of any and all of the 1st-, 2nd, 3rd- and 4th-order response sensitivities to model parameters, the 4th-CASAM makes it possible to compare the relative values of the sensitivities of various order, in order to assess which sensitivities are important and which may actually be neglected, thus enabling future investigations of the convergence of the (multivariate) Taylor series expansion of the response in terms of parameter variations, as well as investigating the range of validity of other important quantities (e.g., response variances/covariance, skewness, kurtosis, etc.) that are derived from Taylor-expansion of the response as a function of the model’s parameters. The 4th-CASAM presented in this work provides the basis for significant future advances towards overcoming the “curse of dimensionality” in sensitivity analysis, uncertainty quantification and predictive modeling.

1. Introduction

The functional derivatives of results (customarily called “responses”) produced by computational models of physical systems with respect to the model’s parameters are customarily called the “response sensitivities.” The first-order sensitivities have been used for a variety of purposes, including: (i) understanding the model by ranking the importance of the various parameters; (ii) performing “reduced-order modeling” by eliminating unimportant parameters and/or processes; (iii) quantifying the uncertainties induced in a model response due to model parameter uncertainties; (iv) performing “model validation,” by comparing computations to experiments to address the question “does the model represent reality?” (v) prioritizing improvements in the model; (vi) performing data assimilation and model calibration as part of forward “predictive modeling” to obtain best-estimate predicted results with reduced predicted uncertainties; (vii) performing inverse “predictive modeling”; and (viii) designing and optimizing the system.

As is well known, non-linear operators do not admit adjoint operators; only linear operators admit corresponding adjoint operators. For this reason, many of the most important responses for linear systems involve the solutions of both the forward and the adjoint linear models that correspond to the respective physical system. Included among the widest used system responses that involve both the forward and adjoint functions are the various forms of Lagrangian functionals, the Raleigh quotient for computing eigenvalues and/or separation constants when solving partial differential equations, the Schwinger functional for first-order “normalization-free” solutions, and many others (see, e.g., [1,2]). These functionals play a fundamental role in optimization and control procedures, derivation of numerical methods for solving equations (differential, integral, integro-differential), etc. The analysis of responses that simultaneously involve both forward and adjoint functions makes it necessary to treat linear systems in their own right, rather than treating them as particular cases of nonlinear systems. This is in contradistinction to responses for a nonlinear system, which can depend only on the forward functions, since nonlinear operators do not admit bona-fide adjoint operators; only a linearized form of a nonlinear operator admits an adjoint operator.

As is well known, even the approximate determination of the first-order sensitivities $\partial R/\partial {\mathit{\alpha }}_i,i=1,\dots ,N_{\mathit{\alpha }}$ of a model response $R$ to $N_{\mathit{\alpha }}$ parameters ${\mathit{\alpha }}_i$ using conventional finite-difference methods would require at least $N_{\mathit{\alpha }}$ large-scale computations with altered parameter values. The computation of the distinct second-order response sensitivities would require $N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)/2$ large-scale computations, which rapidly becomes unfeasible for large-scale models comprising many parameters, even using supercomputers. The computation of higher-order sensitivities by conventional methods is limited in practice by the so-called “curse of dimensionality” [3] since the number of large-scale computations needed for computing higher-order response sensitivities increases exponentially with the order of the response sensitivities. Already the First-Order Adjoint Sensitivity Analysis Methodology for Nonlinear Systems, conceived and developed by Cacuci [4,5,6], provides a considerable step forward in the direction of overcoming the “curse of dimensionality”. For the exact computation of the first- and second-order response sensitivities to parameters, the “curse of dimensionality” has been overcome by the Second-Order Adjoint Sensitivity Analysis Methodology conceived and developed by Cacuci [7,8,9], as was demonstrated by the application of this methodology to compute [10,11,12,13,14,15], comprehensively and efficiently, the exact expression of the first- and second-order sensitivities of a the leakage response (which has also been measured experimentally) to the model parameters of an OECD/NEA reactor physics benchmark [16]. The neutron transport code PARTISN [17] has been used to computationally model this reactor physics benchmark, which comprises 21,976 model parameters. Hence, there are 21,976 first-order sensitivities (of which 7477 have nonzero values) and 241,483,276 second-order sensitivities (of which 27,956,503 have nonzero values) of the benchmark’s leakage response to the model parameters. The work presented in [10,11,12,13,14,15] has been extended to third-order in [18], which has subsequently been applied in [19,20,21] to the computation of the third-order sensitivities of the leakage response of the OECD/NEA benchmark [16] to the benchmark’s total microscopic nuclear cross sections, which turned out to be the most important model parameters.

This work presents the Fourth-Order Comprehensive Adjoint Sensitivity Analysis Methodology (4th-CASAM) for general response-coupled forward/adjoint systems. The 4th-CASAM enables the efficient computation of the exact expressions of the 1st-, 2nd-, 3rd- and 4th-order sensitivities of a generic system response, which depends on both the forward and adjoint state functions with respect to all of the parameters underlying the respective systems. The qualifier “comprehensive” is used because this methodology provides exact expressions for the sensitivities of a system response not only to the system’s internal parameters, but also to its (possibly uncertain) boundaries and internal interfaces in phase-space. The development of the 4th-CASAM for coupled forward/adjoint linear system will provide the basis for significant advances towards overcoming the “curse of dimensionality” in sensitivity analysis, uncertainty quantification, as well as forward and inverse predictive modeling [22,23,24].

This work is structured as follows: Section 2 presents the generic mathematical formulation of the forward and adjoint equations that underlies the computational model of a linear physical system, having a response that depends nonlinearly on the forward and adjoint state functions and parameters. Section 3 presents the development of the novel 4th-CASAM, which is developed successively from the 1st-, 2nd- and 3rd-CASAM. Section 3 also presents, for comparison, the expressions and number of large-scale computations that would be needed if the 1st-, 2nd-, 3rd-, and 4th-order response sensitivities were computed by using finite-difference formulas (which would not only be impractical for large-scale systems, but would provide only approximate values for the respective sensitivities) and the Forward Sensitivity Analysis Methodology (FSAM), which would provide exact expressions for the respective sensitivities, but would also be prohibitively expensive in terms of computational costs for large-scale systems. Finally, Section 4 offers conclusions regarding the significance of this work’s novel results in the quest to overcome the curse of dimensionality in sensitivity analysis, uncertainty quantification and predictive modeling.

2. Background: Mathematical Description of the Physical System

A physical system is modeled by using independent variables, dependent variables (“state functions”), as well as parameters which are seldom, if ever, known precisely. Without loss of generality, the model parameters can be considered to be real scalar quantities, having known nominal (or mean) values and, possibly, known higher-order moments or cumulants (i.e., variance/covariances, skewness, kurtosis), which are determined outside the model, e.g., from experimental data. These imprecisely known model parameters will be denoted as ${\mathit{\alpha }}_1$,…,${\mathit{\alpha }}_{N_{\mathit{\alpha }}}$, where $N_{\mathit{\alpha }}$ denotes the total number of imprecisely known parameters underlying the model under consideration. These model parameters are considered to include imprecisely known geometrical parameters that characterize the physical system’s boundaries in the phase-space of the model’s independent variables. For subsequent developments, it is convenient to consider that these parameters are components of a “vector of parameters” denoted as $\mathit{\alpha }\triangleq {\left({\mathit{\alpha }}_1,\dots ,{\mathit{\alpha }}_{N_{\mathit{\alpha }}}\right)}^{†}$; $\mathit{\alpha }\in {ℝ}^{N_{\mathit{\alpha }}}$, where ${ℝ}^{N_{\mathit{\alpha }}}$ denotes the $N_{\mathit{\alpha }}$-dimensional subset of the set of real scalars. The vector $\mathit{\alpha }\in {ℝ}^{N_{\mathit{\alpha }}}$ is considered to include any imprecisely known model parameters that may enter into defining the system’s boundary in the phase-space of independent variables. The symbol “$\triangleq$” will be used to denote “is defined as” or “is by definition equal to.” Matrices and vectors will be denoted using bold letters. All vectors in this work are considered to be column vectors, and transposition will be indicated by a dagger $(†)$ superscript.

The model is considered to comprise $N_{\mathit{x}}$ independent variables which are denoted as ${\mathit{x}}_i,i=1,\dots ,N_{\mathit{x}}$, and are considered to be components of the vector $\mathit{x}\triangleq {\left({\mathit{x}}_1,\dots ,{\mathit{x}}_{N_{\mathit{x}}}\right)}^{†}\in {ℝ}^{N_{\mathit{x}}}$.The vector $\mathit{x}\in {ℝ}^{N_{\mathit{x}}}$ of independent variables is considered to be defined on a phase-space domain, denoted as ${\Omega }_{\mathit{x}}$, and defined as follows: ${\Omega }_{\mathit{x}}\triangleq \left\{-\infty \le {\lambda }_i\left(\mathit{\alpha }\right)\le {\mathit{x}}_i\le {\omega }_i\left(\mathit{\alpha }\right)\le \infty ;i=1,\dots ,N_{\mathit{x}}\right\}$. The lower boundary-point of an independent variable is denoted as ${\lambda }_i\left(\mathit{\alpha }\right)$ (e.g., the inner radius of a sphere or cylinder, the lower range of an energy-variable, etc.), while the corresponding upper boundary-point is denoted as ${\omega }_i\left(\mathit{\alpha }\right)$ (e.g., the outer radius of a sphere or cylinder, the upper range of an energy-variable, etc.). A typical example of boundaries that depend on imprecisely known parameters is provided by the boundary conditions needed for models based on diffusion theory, in which the respective “flux and/or current conditions” for the “boundaries facing vacuum” are imposed on the “extrapolated boundary” of the respective spatial domain. As is well known, the “extrapolated boundary” depends not only on the imprecisely known physical dimensions of the problem’s domain, but also on the medium’s microscopic transport cross sections and atomic number densities. The boundary of ${\Omega }_{\mathit{x}}$, which will be denoted as $\partial {\Omega }_{\mathit{x}}\left(\mathit{\alpha }\right)$, comprises the set of all of the endpoints ${\lambda }_i\left(\mathit{\alpha }\right),{\omega }_i\left(\mathit{\alpha }\right),i=1,\dots ,N_{\mathit{x}},$ of the respective intervals on which the components of $\mathit{x}$ are defined, i.e., $\partial {\Omega }_{\mathit{x}}\left(\mathit{\alpha }\right)\triangleq \left\{{\lambda }_i\left(\mathit{\alpha }\right){{\displaystyle \cup }}^{}{\omega }_i\left(\mathit{\alpha }\right),i=1,\dots ,N_{\mathit{x}}\right\}$.

The model is considered to comprise $N_{\mathit{\phi }}$ dependent variables (also called “state functions”), denoted as ${\mathit{\phi }}_i\left(\mathit{x}\right),i=1,\dots N_{\mathit{\phi }}$, which are considered to be the components of the “vector of dependent variables” defined as $\mathit{\phi }\left(\mathit{x}\right)\triangleq {\left[{\mathit{\phi }}_1\left(\mathit{x}\right),\dots ,{\mathit{\phi }}_{N_{\mathit{\phi }}}\left(\mathit{x}\right)\right]}^{†}$.

A linear physical system is generally modeled by a system of $N_{\mathit{\phi }}$ linear operator-equations which can be generally represented as follows:

$$\mathit{L}\left(\mathit{x};\mathit{\alpha }\right)\mathit{\phi }\left(\mathit{x}\right)={\mathit{q}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right),\mathit{x}\in {\Omega }_{\mathit{x}},$$

(1)

where $\left(\mathit{x};\mathit{\alpha }\right)\triangleq \left[{\mathit{L}}_{ij}\left(\mathit{x};\mathit{\alpha }\right)\right],$ $i,j=1,\dots ,N_{\mathit{\phi }}$, is a matrix of dimensions $N_{\mathit{\phi }}\times N_{\mathit{\phi }}$, while ${\mathit{q}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)\triangleq {\left[{\mathit{q}}_{\mathit{\phi },1}\left(\mathit{x};\mathit{\alpha }\right),\dots .,{\mathit{q}}_{\mathit{\phi },N_{\mathit{\phi }}}\left(\mathit{x};\mathit{\alpha }\right)\right]}^{†}$ is a column vector of dimension $N_{\mathit{\phi }}$. The components ${\mathit{L}}_{ij}\left(\mathit{x};\mathit{\alpha }\right)$ are operators that act linearly on the dependent variables ${\mathit{\phi }}_j\left(\mathit{x}\right)$ and are, in general, nonlinear functions of the imprecisely known parameters $\mathit{\alpha }\in {ℝ}^{N_{\mathit{\alpha }}}$. The components ${\mathit{L}}_{ij}\left(\mathit{\alpha }\right)$ are operators that act linearly on the dependent variables ${\mathit{\phi }}_j\left(\mathit{x}\right)$ and are, in general, nonlinear functions of the imprecisely known parameters $\mathit{\alpha }\in {ℝ}^{N_{\mathit{\alpha }}}$. The components ${\mathit{q}}_{\mathit{\phi },i}\left(\mathit{x};\mathit{\alpha }\right)$ of ${\mathit{q}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)$, where the subscript “$\mathit{\phi }$” indicates sources associated with the “forward” system of equations, are also nonlinear functions of $\mathit{\alpha }\in {ℝ}^{N_{\mathit{\alpha }}}$. Since the right-side of Equation (1) may contain distributions, the equality in this equation is considered to hold in the weak (“distributional”) sense. Similarly, all of the equalities that involve differential equations in this work will be considered to hold in the weak/distributional sense.

When $\mathit{L}\left(\mathit{x};\mathit{\alpha }\right)$ contains differential operators, a set of boundary and/or initial conditions which define the domain of $\mathit{L}\left(\mathit{x};\mathit{\alpha }\right)$ must also be given. Since the complete mathematical model is considered to be linear in $\mathit{\phi }\left(\mathit{x}\right)$, the boundary and/or initial conditions needed to define the domain of $\mathit{L}\left(\mathit{x};\mathit{\alpha }\right)$ must also be linear in $\mathit{\phi }\left(\mathit{x}\right)$. Such a linear boundary and/or initial conditions are represented in the following operator form:

$${\mathit{B}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)\mathit{\phi }\left(\mathit{x}\right)-{\mathit{c}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)=0,\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left(\mathit{\alpha }\right).$$

(2)

In Equation (2), the operator ${\mathit{B}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)\triangleq \left[{\mathit{b}}_{ij}\left(\mathit{x};\mathit{\alpha }\right)\right];i=1,\dots ,N_{\mathit{b}};j=1,\dots ,N_{\mathit{\phi }}$ is a matrix comprising, as components, operators that act linearly on $\mathit{\phi }\left(\mathit{x}\right)$ and nonlinearly on $\mathit{\alpha }$; the quantity $N_{\mathit{B}}$ denotes the total number of boundary and initial conditions. The operator ${\mathit{c}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)\triangleq {\left[c_{\mathit{\phi },1}\left(\mathit{x};\mathit{\alpha }\right),\dots ,c_{\mathit{\phi },N_{\mathit{B}}}\left(\mathit{x};\mathit{\alpha }\right)\right]}^{†}$ is a $N_{\mathit{B}}$-dimensional vector comprising components that are operators acting, in general, nonlinearly on $\mathit{\alpha }$. The subscript “$\mathit{\phi }$” in Equation (2) indicates boundary conditions associated with the forward state function $\mathit{\phi }\left(\mathit{x}\right)$. In this work, capital bold letters will be used to denote matrices (whose components may be operators rather than just functions) while lower case bold letter will be used to denote vectors.

The nominal solution of Equations (1) and (2) is denoted as ${\mathit{\phi }}^0\left(\mathit{x}\right)$, and is obtained by solving these equations at the nominal (or mean) values of the model parameter ${\mathit{\alpha }}^0$. The superscript “zero” will henceforth be used to denote “nominal” (or, equivalently, “expected” or “mean” values). Thus, the vectors ${\mathit{\phi }}^0\left(\mathit{x}\right)$ and ${\mathit{\alpha }}^0$ satisfy the following equations:

$$\mathit{L}\left(\mathit{x};{\mathit{\alpha }}^0\right){\mathit{\phi }}^0\left(\mathit{x}\right)={\mathit{q}}_{\mathit{\phi }}\left(\mathit{x};{\mathit{\alpha }}^0\right),\mathit{x}\in {\Omega }_{\mathit{x}},$$

(3)

$${\mathit{B}}_{\mathit{\phi }}\left(\mathit{x};{\mathit{\alpha }}^0\right){\mathit{\phi }}^0\left(\mathit{x}\right)-{\mathit{c}}_{\mathit{\phi }}\left(\mathit{x};{\mathit{\alpha }}^0\right)=0,\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right).$$

(4)

Linear systems differ fundamentally from nonlinear systems in that linear operators admit adjoint operators, whereas nonlinear operators do not admit adjoint operators. Furthermore, important model responses of linear systems can be functions of both the forward and the adjoint state functions, a situation that cannot occur for nonlinear problems. Therefore, linear physical systems cannot be simply considered to be particular cases of linear systems (although they may be just that in particular cases), but need to be treated comprehensively in their own right.

Physical problems modeled by linear systems and/or operators are naturally defined in Hilbert spaces. Thus, for physical systems represented by Equations (1) and (2), the components ${\mathit{\phi }}_i\left(\mathit{x}\right),i=1,\dots ,N_{\mathit{\phi }}$ are considered to be square-integrable functions and $\mathit{\phi }\left(\mathit{x}\right)\in H_0$, where $H_0$ is the “original”, or “zeroth-level” Hilbert space, as denoted by the subscript “zero”. Subsequently in this work, higher-level Hilbert spaces, which will be denoted as $H_1,H_2,$ etc., will also be introduced. Evidently, all of the elements of $H_0$ are $N_{\mathit{\phi }}$-dimensional vectors that are functions of the independent variables $\mathit{x}$. For two elements $\mathit{\phi }\left(\mathit{x}\right)\in H_0$ and $\mathit{\psi }\left(\mathit{x}\right)\in H_0$, the Hilbert space $H_0$ is endowed with an inner product that will be denoted as $\langle \mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right){\rangle }_0$ and which is defined as follows:

$$\begin{array}{l}\langle \mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right){\rangle }_0\triangleq {{\displaystyle \int }}_{{\Omega }_{\mathit{x}}}\mathit{\phi }\left(\mathit{x}\right)·\mathit{\psi }\left(\mathit{x}\right)d\mathit{x}\triangleq {\displaystyle \prod }_{i=1}^{N_{\mathit{x}}}{{\displaystyle \int }}_{{\lambda }_i\left(\mathit{\alpha }\right)}^{{\omega }_i\left(\mathit{\alpha }\right)}\mathit{\phi }\left(\mathit{x}\right)\mathit{\psi }\left(\mathit{x}\right)d\mathit{x}\\ ={\displaystyle \sum }_{i=1}^{N_{\mathit{\phi }}}{{\displaystyle \int }}_{{\lambda }_1\left(\mathit{\alpha }\right)}^{{\omega }_1\left(\mathit{\alpha }\right)}\dots {{\displaystyle \int }}_{{\lambda }_i\left(\mathit{\alpha }\right)}^{{\omega }_i\left(\mathit{\alpha }\right)}\dots {{\displaystyle \int }}_{{\lambda }_{N_{\mathit{x}}}\left(\mathit{\alpha }\right)}^{{\omega }_{N_{\mathit{x}}}\left(\mathit{\alpha }\right)}{\mathit{\phi }}_i\left(\mathit{x}\right){\mathit{\psi }}_i\left(\mathit{x}\right)d{\mathit{x}}_{1}d{\mathit{x}}_2\dots d{\mathit{x}}_i\dots d{\mathit{x}}_{N_{\mathit{x}}}.\end{array}$$

(5)

In Equation (5), the product-notation ${\displaystyle \prod _{i=1}^{N_{\mathit{x}}}{\displaystyle \underset{{\lambda }_i\left(\mathit{\alpha }\right)}{\overset{{\omega }_i\left(\mathit{\alpha }\right)}{\int }}\left[\right]d{\mathit{x}}_i}}$ compactly denotes the respective multiple integrals, while the dot indicates the “scalar product of two vectors” defined as follows:

$$\mathit{\phi }\left(\mathit{x}\right)·\mathit{\psi }\left(\mathit{x}\right)\triangleq {\sum }_{i=1}^{N_{\mathit{u}}}{\mathit{\phi }}_i\left(\mathit{x}\right){\mathit{\psi }}_i\left(\mathit{x}\right).$$

(6)

In most practical situations the Hilbert space $H_0$ is self-dual. The operator $\mathit{L}\left(\mathit{x};\mathit{\alpha }\right)$ admits an adjoint (operator), which will be denoted as ${\mathit{L}}^{*}\left(\mathit{x};\mathit{\alpha }\right)$, and which is defined through the following relation for an arbitrary vector $\mathit{\psi }\left(\mathit{x}\right)\in H_{\mathit{\phi }}$:

$$\langle \mathit{\psi }\left(\mathit{x}\right),\mathit{L}\left(\mathit{x};\mathit{\alpha }\right)\mathit{\phi }\left(\mathit{x}\right){\rangle }_0=\langle {\mathit{L}}^{*}\left(\mathit{x};\mathit{\alpha }\right)\mathit{\psi }\left(\mathit{x}\right),\mathit{\phi }\left(\mathit{x}\right){\rangle }_0.$$

(7)

In Equation (7), the formal adjoint operator ${\mathit{L}}^{*}\left(\mathit{x};\mathit{\alpha }\right)$ is the $N_{\mathit{\phi }}\times N_{\mathit{\phi }}$ matrix

$${\mathit{L}}^{*}\left(\mathit{x};\mathit{\alpha }\right)\triangleq \left[{\mathit{L}}_{ji}^{*}\left(\mathit{x};\mathit{\alpha }\right)\right],i,j=1,\dots ,N_{\mathit{\phi }},$$

(8)

comprising elements ${\mathit{L}}_{ji}^{*}\left(\mathit{x};\mathit{\alpha }\right)$ which are obtained by transposing the formal adjoints of the operators ${\mathit{L}}_{ij}\left(\mathit{x};\mathit{\alpha }\right)$. Thus, the system adjoint to the linear system represented by Equations (1) and (2) has the following general representation in operator form:

$${\mathit{L}}^{*}\left(\mathit{x};\mathit{\alpha }\right)\mathit{\psi }\left(\mathit{x}\right)={\mathit{q}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right),\mathit{x}\in {\Omega }_{\mathit{x}},$$

(9)

$${\mathit{B}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right)\mathit{\psi }\left(\mathit{x}\right)-{\mathit{c}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right)=0,\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left(\mathit{\alpha }\right).$$

(10)

The domain of ${\mathit{L}}^{*}\left(\mathit{x};\mathit{\alpha }\right)$ is determined by selecting the adjoint boundary and/or initial conditions represented in operator form in Equation (10), where the subscript “$\mathit{\psi }$” indicates adjoint boundary and/or initial conditions associated with the adjoint state function $\mathit{\psi }\left(\mathit{x}\right)$. These adjoint boundary and/or initial conditions are selected so as to ensure that the boundary terms that arise in the so-called “bilinear concomitant” when going from the left-side to the right side of Equation (7) vanish, in conjunction with the forward boundary conditions given in Equation (2).

The nominal solution of Equations (9) and (10) is denoted as ${\mathit{\psi }}^0\left(\mathit{x}\right)$, and is obtained by solving these equations at the nominal parameter values ${\mathit{\alpha }}^0$, i.e.,

$${\mathit{L}}^{*}\left(\mathit{x};{\mathit{\alpha }}^0\right){\mathit{\psi }}^0\left(\mathit{x}\right)={\mathit{q}}_{\mathit{\psi }}\left(\mathit{x};{\mathit{\alpha }}^0\right),\mathit{x}\in {\Omega }_{\mathit{x}},$$

(11)

$${\mathit{B}}_{\mathit{\psi }}\left(\mathit{x};{\mathit{\alpha }}^0\right){\mathit{\psi }}^0\left(\mathit{x};{\mathit{\alpha }}^0\right)-{\mathit{c}}_{\mathit{\psi }}\left(\mathit{x};{\mathit{\alpha }}^0\right)=0,\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right).$$

(12)

In view of Equations (1) and (9), the relationship shown in Equation (7), which is the basis for defining the adjoint operator, also provides the following fundamental “reciprocity-like” relation between the sources of the forward and the adjoint equations, respectively:

$$\langle \mathit{\psi }\left(\mathit{x}\right),{\mathit{q}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right){\rangle }_0=\langle {\mathit{q}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right),\mathit{\phi }\left(\mathit{x}\right){\rangle }_0.$$

(13)

The functional on the right-side of Equation (13) represents a “detector response”, i.e., a particle reaction-rate representative of the “count” of particles incident on a detector of particles, measuring the respective particle flux $\mathit{\phi }\left(\mathit{x}\right)$. Thus, the source term ${\mathit{q}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right)\triangleq {\left[{\mathit{q}}_{\mathit{\psi },1}\left(\mathit{x};\mathit{\alpha }\right),\dots .,{\mathit{q}}_{\mathit{\psi },N_{\mathit{\phi }}}\left(\mathit{x};\mathit{\alpha }\right)\right]}^{†}$ in Equation (11) is usually associated with the “result of interest” to be measured and/or computed, which is customarily called the system’s “response”.

The system response generally depends on the model’s state-functions and on the system parameters, which are considered to also include parameters that may specifically appear only in the definition of the response under consideration (but which may not appear in the definition of the model). Thus, the (physical) “system” is defined in this work to comprise both the system’s computational model and the system’s response. The system’s response will be denoted as $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ and, in the most general case, is a nonlinear operator acting on the model’s forward and adjoint state functions, as well as on imprecisely known parameters, both directly and indirectly through the state functions. The nominal value of the response, $R\left[{\mathit{\phi }}^0\left(\mathit{x}\right),{\mathit{\psi }}^0\left(\mathit{x}\right);{\mathit{\alpha }}^0\right]$, is determined by using the nominal parameter values ${\mathit{\alpha }}^0$, the nominal value ${\mathit{\phi }}^0\left(\mathit{x}\right)$ of the forward state function [obtained by solving Equations (3) and (4)] and the nominal value ${\mathit{\psi }}^0\left(\mathit{x}\right)$ of the adjoint function [obtained by solving Equations (11) and (12)].

A particularly important class of system responses comprises (scalar-valued) functionals of the forward and adjoint state functions. Such responses occur in many fields, including optimization, control, model verification, data assimilation, model validation and calibration, predictive modeling, etc. For example, the well-known Lagrangian functional is usually represented in the following form:

$${\prod }_{i=1}^{N_{\mathit{x}}}{\int }_{{\lambda }_i\left(\mathit{\alpha }\right)}^{{\omega }_i\left(\mathit{\alpha }\right)}\left\{\mathit{\phi }\left(\mathit{x}\right){\mathit{q}}_{\mathit{\psi }}\left(\mathit{\alpha }\right)+\mathit{\psi }\left(\mathit{x}\right)\left[\mathit{L}\left(\mathit{\alpha }\right)\mathit{\phi }\left(\mathit{x}\right)-{\mathit{q}}_{\mathit{\phi }}\left(\mathit{\alpha }\right)\right]\right\}d{\mathit{x}}_i.$$

(14)

In particular, the Schwinger “normalization-free Lagrangian” is usually represented in the following form [1,2]:

$$\frac{\left\{{\prod }_{i=1}^{N_{\mathit{x}}}{\int }_{{\lambda }_i\left(\mathit{\alpha }\right)}^{{\omega }_i\left(\mathit{\alpha }\right)}\mathit{\phi }\left(\mathit{x}\right){\mathit{q}}_{\mathit{\psi }}\left(\mathit{\alpha }\right)d{\mathit{x}}_i\right\}\left\{{\prod }_{i=1}^{N_{\mathit{x}}}{\int }_{{\lambda }_i\left(\mathit{\alpha }\right)}^{{\omega }_i\left(\mathit{\alpha }\right)}\mathit{\psi }\left(\mathit{x}\right){\mathit{q}}_{\mathit{\phi }}\left(\mathit{\alpha }\right)d{\mathit{x}}_i\right\}}{{\prod }_{i=1}^{N_{\mathit{x}}}{\int }_{{\lambda }_i\left(\mathit{\alpha }\right)}^{{\omega }_i\left(\mathit{\alpha }\right)}\mathit{\psi }\left(\mathit{x}\right)\mathit{L}\left(\mathit{\alpha }\right)\mathit{\phi }\left(\mathit{x}\right)d{\mathit{x}}_i}.$$

(15)

When Equation (1) takes on the eigenvalue (or “separated”) form $\mathit{M}\left[\mathit{\alpha }\left(\mathit{x}\right)\right]\mathit{\phi }\left(\mathit{x}\right)=\mu \mathit{N}\left[\mathit{\alpha }\left(\mathit{x}\right)\right]\mathit{\phi }\left(\mathit{x}\right)$, which implies that the adjoint function $\mathit{\psi }\left(\mathit{x}\right)$ is the solution of the corresponding adjoint equation ${\mathit{M}}^{*}\left[\mathit{\alpha }\left(\mathit{x}\right)\right]\mathit{\psi }\left(\mathit{x}\right)=\mu {\mathit{N}}^{*}\left[\mathit{\alpha }\left(\mathit{x}\right)\right]\mathit{\psi }\left(\mathit{x}\right)$, the well-known Raleigh quotient is usually represented in the following form:

$$\frac{{\prod }_{i=1}^{N_{\mathit{x}}}{\int }_{{\lambda }_i\left(\mathit{\alpha }\right)}^{{\omega }_i\left(\mathit{\alpha }\right)}\mathit{\psi }\left(\mathit{x}\right)\mathit{M}\left[\mathit{\alpha }\left(\mathit{x}\right)\right]\mathit{\phi }\left(\mathit{x}\right)d{\mathit{x}}_i}{{\prod }_{i=1}^{N_{\mathit{x}}}{\int }_{{\lambda }_i\left(\mathit{\alpha }\right)}^{{\omega }_i\left(\mathit{\alpha }\right)}\mathit{\psi }\left(\mathit{x}\right)\mathit{N}\left[\mathit{\alpha }\left(\mathit{x}\right)\right]\mathit{\phi }\left(\mathit{x}\right)d{\mathit{x}}_i}.$$

(16)

Actually, any response can be represented in terms of functionals using the inner product underlying the Hilbert space in which the physical problem is formulated (in this case, $H_0$). For example, a measurement of a physical quantity can be represented as a response $R_p\left[\mathit{\phi }\left({\mathit{x}}_p\right),\mathit{\psi }\left({\mathit{x}}_p\right);\mathit{\alpha }\right]$ which is located at a specific point, ${\mathit{x}}_p$, in phase-space. Such a response can be represented mathematically as a functional of the following form:

$$\begin{array}{l}{R}_p\left[\mathit{\phi }\left({\mathit{x}}_p\right),\mathit{\psi }\left({\mathit{x}}_p\right);\mathit{\alpha }\right]\\ \triangleq {{\displaystyle \int }}_{{\lambda }_1\left(\mathit{\alpha }\right)}^{{\omega }_1\left(\mathit{\alpha }\right)}\dots {{\displaystyle \int }}_{{\lambda }_i\left(\mathit{\alpha }\right)}^{{\omega }_i\left(\mathit{\alpha }\right)}\dots {{\displaystyle \int }}_{{\lambda }_{N_{\mathit{x}}}\left(\mathit{\alpha }\right)}^{{\omega }_{N_{\mathit{x}}}\left(\mathit{\alpha }\right)}R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]\delta \left(\mathit{x}-{\mathit{x}}_p\right)d{\mathit{x}}_{1}d{\mathit{x}}_2\dots d{\mathit{x}}_i\dots d{\mathit{x}}_{N_{\mathit{x}}}.\end{array}$$

(17)

where $\delta \left(\mathit{x}-{\mathit{x}}_p\right)$ denotes the multidimensional Dirac-delta functional. Furthermore, a function-valued (operator) response $R_{op}\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ can be represented by a spectral (in multidimensional orthogonal polynomials or Fourier series) expansion of the form:

$$R_{op}\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]={\displaystyle \sum }_{m_1}\dots {\displaystyle \sum }_{m_{N_{\mathit{x}}}}{c}_{m_1\dots m_{N_{\mathit{x}}}}{P}_{m_1}\left({\mathit{x}}_1\right)P_{m_2}\left({\mathit{x}}_2\right)\dots P_{m_{N_{\mathit{x}}}}\left({\mathit{x}}_{N_{\mathit{x}}}\right),$$

(18)

where the quantities $P_{m_i}\left({\mathit{x}}_i\right)$, $i=1,\dots ,N_{\mathit{x}}$, denote the corresponding spectral functions (e.g., orthogonal polynomials or Fourier exponential/trigonometric functions) and where the spectral Fourier) coefficients $c_{m_1\dots m_{N_{\mathit{x}}}}$ are defined as follows:

$$\begin{array}{l}{c}_{m_1\dots m_{N_{\mathit{x}}}}\triangleq \\ {{\displaystyle \int }}_{{\lambda }_1}^{{\omega }_1}\dots {{\displaystyle \int }}_{{\lambda }_i}^{{\omega }_i}\dots {{\displaystyle \int }}_{{\iota }_{N_{\mathit{x}}}}^{{\omega }_{N_{\mathit{x}}}}{R}_{op}\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]P_{m_1}\left({\mathit{x}}_1\right)\dots P_{m_i}\left({\mathit{x}}_i\right)\dots P_{m_{N_{\mathit{x}}}}\left({\mathit{x}}_{N_{\mathit{x}}}\right)d{\mathit{x}}_1\dots d{\mathit{x}}_i\dots d{\mathit{x}}_{N_{\mathit{x}}}.\end{array}$$

(19)

The coefficients $c_{m_1\dots m_{N_{\mathit{x}}}}$ can themselves be considered as system responses since the spectral polynomials $P_{m_i}\left({\mathit{x}}_i\right)$ are perfectly well known while the expansion coefficients will contain all of the dependencies of the respective response on the imprecisely known model and response parameters. Consequently, the sensitivity analysis of operator-valued responses can be reduced to the sensitivity analysis of scalar-valued responses. The expressions in both Equations (17) and (19) are functionals of the forward and adjoint state functions and can be represented in the following general form:

$$\begin{array}{l}R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]\\ \triangleq {{\displaystyle \int }}_{{\lambda }_1\left(\mathit{\alpha }\right)}^{{\omega }_1\left(\mathit{\alpha }\right)}\dots {{\displaystyle \int }}_{{\lambda }_i\left(\mathit{\alpha }\right)}^{{\omega }_i\left(\mathit{\alpha }\right)}\dots {{\displaystyle \int }}_{{\lambda }_{N_{\mathit{x}}}\left(\mathit{\alpha }\right)}^{{\omega }_{N_{\mathit{x}}}\left(\mathit{\alpha }\right)}S\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]d{\mathit{x}}_{1}d{\mathit{x}}_2\dots d{\mathit{x}}_i\dots d{\mathit{x}}_{N_{\mathit{x}}},\end{array}$$

(20)

where $S\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ denotes a suitably Gateaux- (G-) differentiable function of the indicated arguments. Specific examples that illustrate the effects of the first- and second-order sensitivities of responses of the form shown in Equations (14)–(20) to imprecisely known model and domain-boundary parameters are provided in [25,26,27,28,29,30,31,32,33,34].

The generic response defined in Equation (20) provides the basis for constructing any other responses of specific interest, and will therefore be used for the generic “Fourth-Order Comprehensive Sensitivity Analysis Methodology (4th-CASAM) for Linear Systems” to be developed in the remainder of this work. Note that the generic response defined in Equation (20) is, in general, a nonlinear function of all of its arguments, i.e., $S\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ is nonlinear in $\mathit{\phi }\left(\mathit{x}\right)$, $\mathit{\psi }\left(\mathit{x}\right)$, and $\mathit{\alpha }$.

3. The Fourth-Order Comprehensive Adjoint Sensitivity Analysis Methodology (4th-CASAM) for Linear Systems

The model parameters ${\mathit{\alpha }}_i$ are imprecisely known quantities, so their actual values may differ from their nominal values by quantities denoted as $\delta {\mathit{\alpha }}_i\triangleq {\mathit{\alpha }}_i-{\mathit{\alpha }}_i^0,i=1,\dots ,N_{\mathit{\alpha }}.$ Since the model parameters $\mathit{\alpha }$ and the state functions are related to each other through the forward and adjoint systems, it follows that variations $\delta \mathit{\alpha }\triangleq \left(\delta {\mathit{\alpha }}_1,\dots ,\delta {\mathit{\alpha }}_{N_{\mathit{\alpha }}}\right)$ in the model parameters will cause corresponding variations $\delta \mathit{\phi }\triangleq \left(\delta {\mathit{\phi }}_1,\dots ,\delta {\mathit{\phi }}_{N_{\mathit{\phi }}}\right), \delta {\mathit{\phi }}_i\triangleq {\mathit{\phi }}_i-{\mathit{\phi }}_i^0,i=1,\dots ,N_{\mathit{u}}$ and $\delta \mathit{\psi }\triangleq \left(\delta {\mathit{\psi }}_1,\dots ,\delta {\mathit{\psi }}_{N_{\mathit{\phi }}}\right)$, $\delta {\mathit{\psi }}_i\triangleq {\mathit{\psi }}_i-{\mathit{\psi }}_i^0,i=1,\dots ,N_{\mathit{u}}$ in the forward and, respectively, adjoint state functions. In turn, the variations $\delta \mathit{\alpha }$, $\delta \mathit{\phi }$, and $\delta \mathit{\psi }$ cause a response variation $R\left({\mathit{\phi }}^0+\delta \mathit{\phi };{\mathit{\psi }}^0+\delta \mathit{\psi };{\mathit{\alpha }}^0+\delta \mathit{\alpha }\right)$ around the nominal response value $R\left[{\mathit{\phi }}^0\left(\mathit{x}\right),{\mathit{\psi }}^0\left(\mathit{x}\right);{\mathit{\alpha }}^0\right]$.

3.1. Computation of the First-Order Sensitivities of $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$

Since the closed-form analytical expression of the response $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ is not available in practice, the first-order sensitivities of the response cannot be computed directly from Equation (20), but need to be computed by other means. There are three methods for computing response sensitivities: (i) by using finite-difference formulas or statistical procedures to approximate Equation (20) in conjunction with “brute-force” re-computations using altered parameter values in Equations (1) and (2); (ii) by using the “forward sensitivity analysis methodology (FSAM),” which requires solving the differentiated forms of Equations (1), (2), (9) and (10), evaluated at the known nominal and response parameter values; (iii) by applying the “comprehensive adjoint sensitivity analysis methodology” which has been developed based on the pioneering work of Cacuci [4,5,6]. These methods will be discussed in Section 3.1.1, Section 3.1.2 and Section 3.1.3 below.

3.1.1. Finite-Difference Approximation Using Re-Computations with User-Modified Parameters

The first-order sensitivities of the response, $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$, can be computed approximately using the well-known finite-difference formula presented below:

$$\frac{\partial R\left(\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_j}\approx \frac{1}{2{\mathit{h}}_j}\left(R_{j+1}-R_{j-1}\right)+O\left({\mathit{h}}_j{}^2\right),$$

(21)

where $R_{j+1}\triangleq R\left[\mathit{\phi }\left({\mathit{\alpha }}_j+{\mathit{h}}_j;\mathit{x}\right),\mathit{\psi }\left({\mathit{\alpha }}_j+{\mathit{h}}_j;\mathit{x}\right);{\mathit{\alpha }}_j+{\mathit{h}}_j\right]$, $R_{j-1}\triangleq R\left[\mathit{\phi }\left({\mathit{\alpha }}_j-{\mathit{h}}_j;\mathit{x}\right),\mathit{\psi }\left({\mathit{\alpha }}_j-{\mathit{h}}_j;\mathit{x}\right);{\mathit{\alpha }}_j-{\mathit{h}}_j\right]$, and where ${\mathit{h}}_j$ denotes a “judiciously-chosen” variation in the parameter ${\mathit{\alpha }}_j$ around its nominal value ${\mathit{\alpha }}_j^0$. The values $R_{i+1}$ and $R_{i-1}$ are obtained by re-solving the original forward and adjoint systems of equations repeatedly, using the changed parameter values $\left({\mathit{\alpha }}_i\pm {\mathit{h}}_i\right)$. The value of the variation ${\mathit{h}}_j$ is chosen by “trial and error” for each parameter ${\mathit{\alpha }}_i$, and varies from parameter to parameter. If the value of ${\mathit{h}}_j$ is too large or too small, the result produced by Equation (21) will be in considerable error from the exact value of the derivative $\partial R\left(\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_i$, and this negative outcome may be exacerbated by the fact that the user may not be aware of this error since the exact result for the respective sensitivity can computed only by using the forward or the adjoint sensitivity analysis methods (to be presented in Section 3.1.2 and Section 3.1.3). It is important to note that finite difference formulas introduce their intrinsic “methodological errors,” such as the error $\mathsf{O}\left({\mathit{h}}_j{}^2\right)$ indicated in Equation (21), which are in addition to, and independent of, the errors that might be incurred in the computation of $R\left[{\mathit{\phi }}^0\left(\mathit{x}\right),{\mathit{\psi }}^0\left(\mathit{x}\right);{\mathit{\alpha }}^0\right]$. In other words, even if the computation of $R\left[{\mathit{\phi }}^0\left(\mathit{x}\right),{\mathit{\psi }}^0\left(\mathit{x}\right);{\mathit{\alpha }}^0\right]$ were perfect (error-free), the finite-difference formulas nevertheless introduce their own, intrinsic, numerical errors into the computation of the sensitivity ${\left\{dR\left(\mathit{\alpha }\right)/d{\mathit{\alpha }}_j\right\}}_{{\mathit{\alpha }}^0}$. This statement is also valid for any of the statistical methods (e.g., Latin-hypercubes, etc.) that might be used in conjunction with re-computations.

3.1.2. Forward Sensitivity Analysis Methodology (FSAM)

The aim of the Forward Sensitivity Analysis Methodology (FSAM) is to compute the response sensitivities without introducing approximations of their own (i.e., methodological errors), as was the case when using methods that need brute-force re-computations (e.g., finite-difference errors, statistical errors). The mathematical framework of the FSAM is set in the vector-space of the model parameters $\mathit{\alpha }$. In this vector-space, the first-order partial sensitivity of $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ to a generic model parameter, ${\mathit{\alpha }}_i$, can in principle be obtained by taking the first-order partial of the response and equations underlying the model with respect to a generic model/respond parameter ${\mathit{\alpha }}_i$, evaluated at the nominal parameter values ${\mathit{\alpha }}^0$. Recall that in a linear vector space comprising function-valued elements of the form $\mathit{\eta }\left(\mathit{x}\right)\triangleq {\left[{\mathit{\eta }}_1\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_K\left(\mathit{x}\right)\right]}^{†}$, the most comprehensive—and practically computable—concept of a “partial derivative” of a nonlinear operator $F\left[{\mathit{\eta }}_1\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_K\left(\mathit{x}\right)\right]$ with respect to any one of its arguments, ${\mathit{\eta }}_i$, stems from the properties of the 1st-order partial Gateaux- (G-) variation, $\delta F_i\left[{\mathit{\eta }}_1\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_K\left(\mathit{x}\right);{\mathit{h}}_i\left(\mathit{x}\right)\right]$, of $F\left[{\mathit{\eta }}_1\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_K\left(\mathit{x}\right)\right]$, which is defined as follows:

$$\delta F_i\left[{\mathit{\eta }}_1\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_K\left(\mathit{x}\right);{\mathit{h}}_i\left(\mathit{x}\right)\right]\triangleq {\left\{\frac{d}{d\epsilon }F\left[{\mathit{\eta }}_1\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_i\left(\mathit{x}\right)+\epsilon {\mathit{h}}_i\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_K\left(\mathit{x}\right)\right]\right\}}_{s\epsilon 0},$$

(22)

where $\epsilon$ is a scalar (i.e., $\epsilon \in F$, where $F$ denotes the underlying field of real scalars) and where ${\mathit{h}}_i\left(\mathit{x}\right)$ denotes an arbitrary variation in ${\mathit{\eta }}_i\left(\mathit{x}\right)$.

The total G-variation $\delta F_i\left[{\mathit{\eta }}_1\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_K\left(\mathit{x}\right);{\mathit{h}}_1\left(\mathit{x}\right),\dots ,{\mathit{h}}_K\left(\mathit{x}\right)\right]$ of $F\left[{\mathit{\eta }}_1\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_K\left(\mathit{x}\right)\right]$ for arbitrary variations $\mathit{h}\triangleq {\left({\mathit{h}}_1,\dots ,{\mathit{h}}_K\right)}^{†}$ is defined as follows:

$$\begin{array}{l}\delta F_i\left[{\mathit{\eta }}_1\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_K\left(\mathit{x}\right);{\mathit{h}}_1\left(\mathit{x}\right),\dots ,{\mathit{h}}_K\left(\mathit{x}\right)\right]\\ \triangleq {\left\{\frac{d}{d\epsilon }F\left[{\mathit{\eta }}_1\left(\mathit{x}\right)+\epsilon {\mathit{h}}_1\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_k\left(\mathit{x}\right)+\epsilon {\mathit{h}}_K\left(\mathit{x}\right)\right]\right\}}_{\epsilon =0}.\end{array}$$

(23)

The first-order G-variation $\delta F\left(\mathit{\eta };\mathit{h}\right)$ is an operator defined on the same domain as $F\left(\mathit{\eta }\right)$, and has the same range as $F\left(\mathit{\eta }\right)$. The G-variation $\delta F\left(\mathit{\eta };\mathit{h}\right)$ satisfies the relation $F\left(\mathit{\eta }+\epsilon \mathit{h}\right)-F\left(\mathit{\eta }\right)=\delta F\left(\mathit{\eta };\mathit{h}\right)+\Delta \left(\mathit{h}\right),$ with $\underset{\epsilon \to 0}{\mathit{L}im}\frac{\left[\Delta \left(\epsilon \mathit{h}\right)\right]}{\epsilon }=0.$ The existence of the G-variation $\delta F\left(\mathit{\eta };\mathit{h}\right)$ does not guarantee its numerical computability. Numerical methods most often require that $\delta F\left(\mathit{\eta };\mathit{h}\right)$ be linear in the variations $\mathit{h}$ in a neighborhood $\left(\mathit{\eta }+\epsilon \mathit{h}\right)$ around $\mathit{\eta }$. The necessary and sufficient conditions for the G-differential $\delta F\left(\mathit{\eta };\mathit{h}\right)$ of a nonlinear operator $F\left(\mathit{\eta }\right)$ to be linear in the variations $\mathit{h}$ in a neighborhood $\left(\mathit{\eta }+\epsilon \mathit{h}\right)$ around $\mathit{\eta }$ are as follows:

$$\left(\mathrm{i}\right) F\left(\mathit{\eta }\right)\mathrm{satisfies}\mathrm{a}\mathrm{weak}\mathrm{Lipschitz}\mathrm{condition}\mathrm{at}\mathit{\eta };$$

(24)

(ii) For two arbitrary vectors of variations ${\mathit{h}}_1$ and ${\mathit{h}}_2$, the operator $F\left(\mathit{\eta }\right)$ satisfies the relation

$$F\left(\mathit{\eta }+\epsilon {\mathit{h}}_1+\epsilon {\mathit{h}}_2\right)-F\left(\mathit{\eta }+\epsilon {\mathit{h}}_1\right)-F\left(\mathit{\eta }+\epsilon {\mathit{h}}_2\right)+F\left(\mathit{\eta }\right)=o\left(\epsilon \right).$$

(25)

In practice, it is not necessary to investigate if $F\left(\mathit{\eta }\right)$ satisfies the conditions presented in Equations (24) and (25) since it is usually evident if the right-side of Equation (23) is linear (or not) in the variations $\mathit{h}$. When the total variation $\delta F\left(\mathit{\eta };\mathit{h}\right)$ is linear in $\mathit{h}$, it is called the “total differential of $F\left(\mathit{u}\right)$” and is denoted as $DF\left(\mathit{\eta };\mathit{h}\right)$. In this case, the partial variation $\delta F_i\left({\mathit{\eta }}_1,\dots ,{\mathit{\eta }}_K;{\mathit{h}}_i\right)$ is called the “partial differential $\delta F_i\left({\mathit{\eta }}_1,\dots ,{\mathit{\eta }}_K;{\mathit{h}}_i\right)$ of $F\left(\mathit{\eta }\right)$ with respect to ${\mathit{\eta }}_i$” and the following representation holds:

$$DF\left({\mathit{\eta }}_1,\dots ,{\mathit{\eta }}_K;{\mathit{h}}_1,\dots ,{\mathit{h}}_K\right)={\displaystyle \sum }_{k=1}^K\frac{\partial F\left({\mathit{\eta }}_1,\dots ,{\mathit{\eta }}_K\right)}{\partial {\mathit{\eta }}_i}{\mathit{h}}_i,$$

(26)

where the quantities $\partial F\left({\mathit{\eta }}_1,\dots ,{\mathit{\eta }}_K\right)/\partial {\mathit{\eta }}_i$ denote the partial derivatives of $F\left(\mathit{u}\right)$ with respect to its arguments ${\mathit{\eta }}_i$. It will henceforth be assumed that all of the operators considered in this work satisfy the conditions presented in Equations (24) and (25), so that they admit partial G-derivatives such that the representation shown in Equation (26) exists.

The sensitivity ${\left\{\partial R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]/\partial {\mathit{\alpha }}_j\right\}}_{\left({\mathit{\alpha }}^0\right)}$ of $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ to a model parameter ${\mathit{\alpha }}_j$, evaluated at the nominal parameter values ${\mathit{\alpha }}^0$ is obtained by determining the partial G-differential $\delta R\left[{\mathit{\phi }}^0\left(\mathit{x}\right),{\mathit{\psi }}^0\left(\mathit{x}\right);{\mathit{\alpha }}^0;\delta {\mathit{\alpha }}_j\right]={\left\{\partial R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]/\partial {\mathit{\alpha }}_j\right\}}_{\left({\mathit{\alpha }}^0\right)}\delta {\mathit{\alpha }}_j$ of $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ with respect to the respective parameter ${\mathit{\alpha }}_j$, for each $j=1,\dots ,N_{\mathit{\alpha }}$. Since the forward state function $\mathit{\phi }\left(\mathit{x}\right)$ is the solution of Equations (1) and (2), while the adjoint state function $\mathit{\psi }\left(\mathit{x}\right)$ is the solution of Equations (9) and (10), it follows that both $\mathit{\phi }\left(\mathit{x}\right)$ and $\mathit{\psi }\left(\mathit{x}\right)$ are implicit functions of the model parameters $\mathit{\alpha }\triangleq {\left({\mathit{\alpha }}_1,\dots ,{\mathit{\alpha }}_{N_{\mathit{\alpha }}}\right)}^{†}$. Hence, considering that $\mathit{\phi }\left(\mathit{x}\right)=\mathit{\phi }\left(\mathit{x};\mathit{\alpha }\right)$, $\mathit{\psi }\left(\mathit{x}\right)=\mathit{\psi }\left(\mathit{x};\mathit{\alpha }\right)$, and applying the definition provided in Equations (20)–(22) yields the following expression for the 1st-order partial sensitivity ${\left\{\partial R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]/\partial {\mathit{\alpha }}_j\right\}}_{\left({\mathit{\alpha }}^0\right)}$, for each $j=1,\dots ,N_{\mathit{\alpha }}$:

$$\begin{array}{l}{\left\{\frac{\partial R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{\alpha }}_j}\right\}}_{{\mathit{\alpha }}^0}\triangleq \frac{1}{\delta {\mathit{\alpha }}_j}{\left\{\frac{d}{d\epsilon }R\left[\mathit{\phi }\left({\mathit{\alpha }}_j^0+\epsilon \delta {\mathit{\alpha }}_j\right);\mathit{\psi }\left({\mathit{\alpha }}_j^0+\epsilon \delta {\mathit{\alpha }}_j\right);\left({\mathit{\alpha }}_j^0+\epsilon \delta {\mathit{\alpha }}_j\right)\right]\right\}}_{\epsilon =0}\\ ={\displaystyle {\displaystyle \sum }_{j=1}^{N_{\mathit{\alpha }}}{\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{\left\{{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{\alpha }}_j}\right\}}_{{\mathit{\alpha }}^0}\\ +{\displaystyle {\displaystyle \sum }_{j=1}^{N_{\mathit{\alpha }}}{\displaystyle \sum }_{k=1}^{N_{\mathit{x}}}{\displaystyle \prod }_{m=1,k\ne j}^{N_{\mathit{x}}}}\{{{\displaystyle \int }}_{{\lambda }_m\left(\mathit{\alpha }\right)}^{{\omega }_m\left(\mathit{\alpha }\right)}d{\mathit{x}}_{m}S\left[\mathit{\phi }\left({\mathit{x}}_1,\dots ,{\mathit{b}}_k,\dots ,{\mathit{x}}_{N_{\mathit{x}}}\right),\mathit{\psi }\left({\mathit{x}}_1,\dots ,{\mathit{b}}_k,\dots ,{\mathit{x}}_{N_{\mathit{x}}}\right);\mathit{\alpha }\right]\frac{\partial {\omega }_k\left(\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_j}\\ -S\left[\mathit{\phi }\left({\mathit{x}}_1,\dots ,{\mathit{a}}_k,\dots ,{\mathit{x}}_{N_{\mathit{x}}}\right),\mathit{\psi }\left({\mathit{x}}_1,\dots ,{\mathit{a}}_k,\dots ,{\mathit{x}}_{N_{\mathit{x}}}\right);\mathit{\alpha }\right]{\frac{\partial {\lambda }_k\left(\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_j}\}}_{{\mathit{\alpha }}^0}\\ +{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{\left\{{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]}{\partial \mathit{\phi }}\frac{\partial \mathit{\phi }}{\partial {\mathit{\alpha }}_j}+\frac{\partial S\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]}{\partial \mathit{\psi }}\frac{\partial \mathit{\psi }}{\partial {\mathit{\alpha }}_j}\right\}}_{{\mathit{\alpha }}^0},j=1,\dots ,N_{\mathit{\alpha }}.\end{array}$$

(27)

The functions $\partial \mathit{\phi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_j$ and $\partial \mathit{\psi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_j$ are obtained by solving the system of equations obtained by taking the first partial G-derivative of Equations (1), (2), (9) and (10) with respect to the model parameter ${\mathit{\alpha }}_j$. Applying the definition provided in Equation (22) to Equations (1), (2), (9) and (10) yields, after cancelling the scalar parameter variation $\delta {\mathit{\alpha }}_j$ on both sides of the equal signs in resulting expressions, the following First-Order Forward Sensitivity System (1st-OFSS):

$$\begin{array}{ll}\mathit{L}\left(\mathit{x};{\mathit{\alpha }}^0\right)\frac{\partial \mathit{\phi }}{\partial {\mathit{\alpha }}_j}& |={\left\{f^{\left(1\right)}\left(j;\mathit{\phi };\mathit{\alpha }\right)\right\}}_{\left({\mathit{\alpha }}^0\right)};f^{\left(1\right)}\left(j;\mathit{\phi };\mathit{\alpha }\right)\\ & |\triangleq \frac{\partial {\mathit{q}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_j}-\frac{\partial \mathit{L}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_j}\mathit{\phi }\left(\mathit{x}\right);\mathit{x}\in {\Omega }_{\mathit{x}};j=1,\dots ,N_{\mathit{\alpha }};\end{array}$$

(28)

$$\begin{array}{l}{\left\{{\mathit{B}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)\right\}}_{\left({\mathit{\alpha }}^0\right)}\frac{\partial \mathit{\phi }}{\partial {\mathit{\alpha }}_j}={\left\{c_{\mathit{\phi }}^{\left(1\right)}\left(j;\mathit{\phi };\mathit{\alpha }\right)\right\}}_{\left({\mathit{\alpha }}^0\right)};\\ c_{\mathit{\phi }}^{\left(1\right)}\left(j;\mathit{\phi };\mathit{\alpha }\right)\triangleq \frac{\partial \left[{\mathit{b}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)\mathit{\phi }-c_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)\right]}{\partial {\mathit{\alpha }}_j};\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right);\end{array}$$

(29)

$$\begin{array}{l}{\mathit{L}}^{*}\left(\mathit{x};{\mathit{\alpha }}^0\right)\frac{\partial \mathit{\psi }\left(\mathit{x}\right)}{\partial {\mathit{\alpha }}_j}={\left\{g^{\left(1\right)}\left(j;\mathit{\psi };\mathit{\alpha }\right)\right\}}_{\left({\mathit{\alpha }}^0\right)};\\ {\mathit{g}}^{\left(1\right)}\left(j;\mathit{\psi };\mathit{\alpha }\right)\triangleq \frac{\partial {\mathit{q}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_j}-\frac{\partial {\mathit{L}}^{*}\left(\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_j}\mathit{\psi }\left(\mathit{x}\right);\mathit{x}\in {\Omega }_{\mathit{x}};j=1,\dots ,N_{\mathit{\alpha }};\end{array}$$

(30)

$$\begin{array}{l}{\left\{{\mathit{B}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right)\right\}}_{\left({\mathit{\alpha }}^0\right)}\frac{\partial \mathit{\psi }\left(\mathit{x}\right)}{\partial {\mathit{\alpha }}_j}={\left\{c_{\mathit{\psi }}^{\left(1\right)}\left(j;\mathit{\psi };\mathit{\alpha }\right)\right\}}_{\left({\mathit{\alpha }}^0\right)};\\ c_{\mathit{\psi }}^{\left(1\right)}\left(j;\mathit{\psi };\mathit{\alpha }\right)\triangleq \frac{\partial {\mathit{b}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_j}\mathit{\psi }\left(\mathit{x}\right)-\frac{\partial c_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_j};\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right).\end{array}$$

(31)

The system of equations comprising Equations (28) through (31) is called the 1st-OFSS because its solutions are the first-order derivatives of the forward and adjoint state functions $\partial \mathit{\phi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_j$ and $\partial \mathit{\psi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_j$, respectively. Evidently, the 1st-OFSS would need to be solved $2N_{\mathit{\alpha }}$-times, with different right-sides in order to compute the 1st-order derivatives of the forward and, respectively, adjoint state functions with respect to all model parameters ${\mathit{\alpha }}_j$, $j=1,\dots ,N_{\mathit{\alpha }}$. Subsequently, the solutions $\partial \mathit{\phi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_j$ and $\partial \mathit{\psi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_j$ of the 1st-OFSS would be used in Equation (27) to compute the first-order response sensitivity ${\left\{\partial R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]/\partial {\mathit{\alpha }}_j\right\}}_{\left({\mathit{\alpha }}^0\right)}$, for each $j=1,\dots ,N_{\mathit{\alpha }}$, using quadrature formulas. The quadrature computations are small-scale inexpensive computations but solving the 1st-OFSS entails large-scale computations, even if the same linear operators, namely $\mathit{L}\left(\mathit{x};\mathit{\alpha }\right)$ and ${\mathit{L}}^{*}\left(\mathit{x};\mathit{\alpha }\right)$, would need to be inverted each time (which means that the same code/solvers would be used, without needing additional programming for inverting this operator). Thus, the FSAM is advantageous to employ only if, in the problem under consideration, the number $N_{\mathit{\alpha }}$ of model parameters is considerably less than the number of responses of interest. This is rarely the case in practice, however, since most problems of practical interest are characterized by many model parameters and comparatively few responses.

It is evident from Equations (27)–(31) that the 1st-order partial sensitivities ${\left\{\partial R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]/\partial {\mathit{\alpha }}_j\right\}}_{\left({\mathit{\alpha }}^0\right)}$ are functions of the quantities $\partial \mathit{\phi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_j$ and $\partial \mathit{\psi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_j$, as well as of the index $j=1,\dots ,N_{\mathit{\alpha }}$. Therefore, for the subsequent derivation of the 2nd- and higher-order sensitivities of the response with respect to the model parameters, it is convenient to introduce the following notation for the 1st-order response sensitivities obtained by using the FSAM:

$$R^{\left(1\right)}\left(j;\mathit{\phi };\mathit{\psi };\mathit{\alpha };\frac{\partial \mathit{\phi }}{\partial {\mathit{\alpha }}_j};\frac{\partial \mathit{\psi }}{\partial {\mathit{\alpha }}_j}\right)\triangleq \frac{\partial R\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_j}.$$

(32)

3.1.3. First-Order Comprehensive Adjoint Sensitivity Analysis Methodology (1st-CASAM)

The aim of the First-Order Comprehensive Adjoint Sensitivity Analysis Methodology (1st-CASAM) is to find an alternative way for expressing the contribution of the flux variation (the so-called “indirect-effect term” contribution) to the total response sensitivity, so as to avoid the need for having to compute the derivatives $\partial {\mathit{\phi }}^g\left(r,\Omega \right)/\partial {\mathit{\alpha }}_i$ of the state functions with respect to the model’s parameters, as is the case when using the FSAM. In contradistinction to the FSAM, which is framed in the vector-space of the parameters $\mathit{\alpha }$, the 1st-CASAM is framed in the combined phase-space of the parameters $\mathit{\alpha }$ and state-functions $\mathit{\phi }\left(\mathit{x}\right)$ and $\mathit{\psi }\left(\mathit{x}\right)$.

The 1st-order Gateaux- (G-) variation, denoted as $\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\phi },\delta \mathit{\psi },\delta \mathit{\alpha }\right)$, of the response $R\left(e\right)$ for arbitrary variations $\delta \mathit{\phi }\left(\mathit{x}\right),\delta \mathit{\psi }\left(\mathit{x}\right),\delta \mathit{\alpha }$ in the model parameters and state functions, in a neighborhood $\left[{\mathit{\phi }}^0\left(\mathit{x}\right)+\epsilon \delta \mathit{\phi }\left(\mathit{x}\right),{\mathit{\psi }}^0\left(\mathit{x}\right)+\epsilon \delta \mathit{\psi }\left(\mathit{x}\right);{\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }\right]$ around $\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0\right)$, where $\epsilon \in F$ is a real scalar ($F$ denotes the underlying field of scalars), is defined as follows:

$$\begin{array}{l}\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\phi },\delta \mathit{\psi },\delta \mathit{\alpha }\right)\\ |\triangleq {\left\{\frac{d}{d\epsilon }R\left[{\mathit{\phi }}^0\left(\mathit{x}\right)+\epsilon \delta \mathit{\phi }\left(\mathit{x}\right),{\mathit{\psi }}^0\left(\mathit{x}\right)+\epsilon \delta \mathit{\psi }\left(\mathit{x}\right);{\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }\right]\right\}}_{\epsilon =0}\\ |={\left\{\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\phi },\delta \mathit{\psi }\right)\right\}}_{dir}+{\left\{\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\alpha }\right)\right\}}_{ind},\end{array}$$

(33)

where the “direct-effect” term ${\left\{\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\alpha }\right)\right\}}_{dir}$ depends only on the parameter variations $\delta \mathit{\alpha }$ and is defined as follows:

$$\begin{array}{l}{\left\{\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\alpha }\right)\right\}}_{dir}\triangleq {\left\{\frac{\partial }{\partial \mathit{\alpha }}{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_{k}S\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\alpha }\\ ={\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{\left\{{{\displaystyle \int }}_{{\kappa }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]}{\partial \mathit{\alpha }}\delta \mathit{\alpha }\right\}}_{{\mathit{\alpha }}^0}\\ +{\displaystyle {\displaystyle \sum }_{k=1}^{N_{\mathit{x}}}}{\displaystyle {\displaystyle \prod }_{m=1,k\ne j}^{N_{\mathit{x}}}}{\left\{{{\displaystyle \int }}_{{\lambda }_m\left(\mathit{\alpha }\right)}^{{\omega }_m\left(\mathit{\alpha }\right)}d{\mathit{x}}_{m}S\left[\mathit{\phi }\left({\mathit{x}}_1,\dots ,{\mathit{b}}_k,\dots ,{\mathit{x}}_{N_{\mathit{x}}}\right),\mathit{\psi }\left({\mathit{x}}_1,\dots ,{\mathit{b}}_k,\dots ,{\mathit{x}}_{N_{\mathit{x}}}\right);\mathit{\alpha }\right]\frac{\partial {\omega }_k\left(\mathit{\alpha }\right)}{\partial \mathit{\alpha }}\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\alpha }\\ -{\displaystyle {\displaystyle \sum }_{k=1}^{N_{\mathit{x}}}}{\displaystyle {\displaystyle \prod }_{m=1,k\ne j}^{N_{\mathit{x}}}}{\left\{{{\displaystyle \int }}_{{\lambda }_m\left(\mathit{\alpha }\right)}^{{\omega }_m\left(\mathit{\alpha }\right)}d{\mathit{x}}_{m}S\left[\mathit{\phi }\left({\mathit{x}}_1,\dots ,{\mathit{a}}_k,\dots ,{\mathit{x}}_{N_{\mathit{x}}}\right),\mathit{\psi }\left({\mathit{x}}_1,\dots ,{\mathit{a}}_k,\dots ,{\mathit{x}}_{N_{\mathit{x}}}\right);\mathit{\alpha }\right]\frac{\partial {\lambda }_k\left(\mathit{\alpha }\right)}{\partial \mathit{\alpha }}\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\alpha },\end{array}$$

(34)

while the “indirect-effect” term ${\left\{\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\phi },\delta \mathit{\psi }\right)\right\}}_{ind}$ depends only on the variations $\delta \mathit{\phi }$ and $\delta \mathit{\psi }$ in the state functions, and is defined as follows:

$$\begin{array}{l}{\left\{\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\phi },\delta \mathit{\psi }\right)\right\}}_{ind}\\ ={\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{\left\{{{\displaystyle \int }}_{{\ell }_k\left(\mathit{\alpha }\right)}^{{\mathit{u}}_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\left[\frac{\partial S\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]}{\partial \mathit{\phi }}\delta \mathit{\phi }+\frac{\partial S\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]}{\partial \mathit{\psi }}\delta \mathit{\psi }\right]\right\}}_{{\mathit{\alpha }}^0}.\end{array}$$

(35)

In Equations (34) and (35), the notation ${\left\{\right\}}_{{\mathit{\alpha }}^0}$ has been used to indicate that the quantity within the brackets is to be evaluated at the nominal values of the parameters and state functions. This simplified notation is justified by the fact that when the parameters take on their nominal values, it implicitly means that the corresponding state functions also take on their corresponding nominal values, in view of Equations (3), (4), (11) and (12). This simplified notation will be used throughout this work.

The “direct effect” term ${\left\{\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\alpha }\right)\right\}}_{dir}$ defined in Equation (34) depends directly on the parameter variations $\delta \mathit{\alpha }$, and can be computed immediately since it does not depend on the variations $\delta \mathit{\phi }$ and $\delta \mathit{\psi }\left(\mathit{x}\right)$. On the other hand, the “indirect effect” term ${\left\{\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\phi },\delta \mathit{\psi }\right)\right\}}_{ind}$ defined in Equation (35) depends indirectly on the parameter variations $\delta \mathit{\alpha }$, through the variations $\delta \mathit{\phi }\left(\mathit{x}\right)$ and $\delta \mathit{\psi }\left(\mathit{x}\right)$ in the forward state functions $\mathit{\phi }\left(\mathit{x}\right)$ and $\mathit{\psi }\left(\mathit{x}\right)$, and can be computed only after having computed the values of the variations $\delta \mathit{\phi }\left(\mathit{x}\right)$ and $\delta \mathit{\psi }\left(\mathit{x}\right)$.

The variations $\delta \mathit{\phi }\left(\mathit{x}\right)$ and $\delta \mathit{\psi }\left(\mathit{x}\right)$ are the solutions of the system of equations obtained by taking the G-differentials of Equations (1), (2), (9) and (10), which can be written in the following matrix-vector form:

$${\left\{{\mathit{V}}^{\left(1\right)}\left(\mathit{\alpha }\right)\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right)\right\}}_{{\mathit{\alpha }}^0}={\left\{{\mathit{q}}^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0},\mathit{x}\in {\Omega }_{\mathit{x}},$$

(36)

$${\left\{{\mathit{b}}_{\mathit{V}}^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha };\delta {\mathit{u}}^{\left(1\right)};\delta \mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0}=0,\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right),$$

(37)

where

$$\begin{array}{l}{\mathit{V}}^{\left(1\right)}\left(\mathit{\alpha }\right)\triangleq \left(\begin{array}{cc}\mathit{L}\left(\mathit{\alpha }\right)& 0\\ 0& {\mathit{L}}^{*}\left(\mathit{\alpha }\right)\end{array}\right);{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right)\triangleq \left(\begin{array}{c}\mathit{\phi }\left(\mathit{x}\right)\\ \mathit{\psi }\left(\mathit{x}\right)\end{array}\right);\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right)\triangleq \left(\begin{array}{c}\delta \mathit{\phi }\left(\mathit{x}\right)\\ \delta \mathit{\psi }\left(\mathit{x}\right)\end{array}\right);\\ {\mathit{q}}^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right)\triangleq \left(\begin{array}{c}{\mathit{q}}_1^{\left(1\right)}\left(\mathit{\phi };\mathit{\alpha };\delta \mathit{\alpha }\right)\\ {\mathit{q}}_2^{\left(1\right)}\left(\mathit{\psi };\mathit{\alpha };\delta \mathit{\alpha }\right)\end{array}\right);\\ {\mathit{b}}_{\mathit{V}}^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};\delta {\mathit{u}}^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right)\triangleq \left(\begin{array}{c}{\mathit{b}}_1^{\left(1\right)}\left(\mathit{\phi };\mathit{\alpha };\delta \mathit{\alpha }\right)\\ {\mathit{b}}_2^{\left(1\right)}\left(\mathit{\psi };\mathit{\alpha };\delta \mathit{\alpha }\right)\end{array}\right).\end{array}$$

(38)

$${\left\{{\mathit{q}}_1^{\left(1\right)}\left(\mathit{\phi },\mathit{\alpha };\delta \mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0}\triangleq {\left\{\frac{\partial \left[{\mathit{q}}_{\mathit{\phi }}\left(\mathit{\alpha }\right)-\mathit{L}\left(\mathit{\alpha }\right)\mathit{\phi }\right]}{\partial \mathit{\alpha }}\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\alpha },$$

(39)

$${\left\{{\mathit{q}}_2^{\left(1\right)}\left(\mathit{\psi },\mathit{\alpha };\delta \mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0}\triangleq {\left\{\frac{\partial \left[{\mathit{q}}_{\mathit{\psi }}\left(\mathit{\alpha }\right)-{\mathit{L}}^{*}\left(\mathit{\alpha }\right)\mathit{\psi }\left(\mathit{x}\right)\right]}{\partial \mathit{\alpha }}\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\alpha },$$

(40)

The matrices $\partial {\mathit{q}}_{\mathit{\phi }}\left(\mathit{\alpha }\right)/\partial \mathit{\alpha }$ and $\partial \left[\mathit{L}\left(\mathit{\alpha }\right)\mathit{\phi }\right]/\partial \mathit{\alpha }$, which appear on the right-side of Equation (36), are defined as follows:

$$\begin{array}{l}\frac{\partial {\mathit{q}}_{\mathit{\phi }}\left(\mathit{\alpha }\right)}{\partial \mathit{\alpha }}\triangleq \left(\begin{array}{ccc}\frac{\partial {\mathit{q}}_{\mathit{\phi },1}}{\partial {\mathit{\alpha }}_1}& \dots & \frac{\partial {\mathit{q}}_{\mathit{\phi },1}}{\partial {\mathit{\alpha }}_{N_{\mathit{\alpha }}}}\\ ⋮& \ddots & ⋮\\ \frac{\partial {\mathit{q}}_{\mathit{\phi },N_{\mathit{\phi }}}}{\partial {\mathit{\alpha }}_1}& \cdots & \frac{\partial {\mathit{q}}_{\mathit{\phi },N_{\mathit{\phi }}}}{\partial {\mathit{\alpha }}_{N_{\mathit{\alpha }}}}\end{array}\right),\\ \frac{\partial \left[\mathit{L}\left(\mathit{\alpha }\right)\mathit{\phi }\right]}{\partial \mathit{\alpha }}\triangleq \left(\begin{array}{ccc}\frac{\partial \left[{{\displaystyle \sum }}_{j=1}^{N_{\mathit{\phi }}}{\mathit{L}}_{1,j}\left(\mathit{\alpha }\right){\mathit{\phi }}_j\right]}{\partial {\mathit{\alpha }}_1}& \dots & \frac{\partial \left[{{\displaystyle \sum }}_{j=1}^{N_{\mathit{\phi }}}{\mathit{L}}_{1,j}\left(\mathit{\alpha }\right){\mathit{\phi }}_j\right]}{\partial {\mathit{\alpha }}_{N_{\mathit{\alpha }}}}\\ ⋮& \ddots & ⋮\\ \frac{\partial \left[{{\displaystyle \sum }}_{j=1}^{N_{\mathit{\phi }}}{\mathit{L}}_{N_{\mathit{\phi }},j}\left(\mathit{\alpha }\right){\mathit{\phi }}_j\right]}{\partial {\mathit{\alpha }}_1}& \cdots & \frac{\partial \left[{{\displaystyle \sum }}_{j=1}^{N_{\mathit{\phi }}}{\mathit{L}}_{N_{\mathit{\phi }},j}\left(\mathit{\alpha }\right){\mathit{\phi }}_j\right]}{\partial {\mathit{\alpha }}_{N_{\mathit{\alpha }}}}\end{array}\right).\end{array}$$

(41)

The system of equations comprising Equations (36) and (37) will be called the “1st-Level Variational Sensitivity System” (1st-LVSS) and its solution, $\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right)$, will be called the “1st-Level variational sensitivity function.” The superscript “(1)” used in Equations (36) and (37) indicates “1st-Level” (as opposed to “first-order”) because the 1st-LVSS differs from the 1st-OFSS in that the 1st-LVSS involves the total first-order differential $\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right)$ of the state functions whereas the 1st-OFSS involves the first-order partial derivatives of the state functions with respect to a single model parameter ${\mathit{\alpha }}_j$. In principle, the 1st-LVSS could be solved for each possible component of the vectors of parameter variations $\delta \mathit{\alpha }$ to obtain the functions $\delta \mathit{\phi }\left(\mathit{x}\right)$ and $\delta \mathit{\psi }\left(\mathit{x}\right)$. Subsequently, the solutions $\delta \mathit{\phi }\left(\mathit{x}\right)$ and $\delta \mathit{\psi }\left(\mathit{x}\right)$ of the 1st-LVSS could be used together with the known parameter variations $\delta \mathit{\alpha }$ in Equation (35) to compute the indirect-effect term ${\left\{\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\phi },\delta \mathit{\psi }\right)\right\}}_{ind}$. Computing the indirect-effect term ${\left\{\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\phi },\delta \mathit{\psi }\right)\right\}}_{ind}$ by solving the 1st-LVSS would require at least $2N_{\mathit{\alpha }}$ large-scale computations (to solve the 1st-LVSS) for every independent component of the vectors of parameter variations $\delta \mathit{\alpha }$. Therefore, solving the 1st-LVSS is advantageous to employ only if, in the problem under consideration, the number $N_{\mathit{\alpha }}$ of model and boundary parameters is considerably less than the number of responses of interest. This is rarely the case in practice, however, since most problems of practical interest are characterized by many model parameters and comparatively few responses.

The indirect-effect term ${\left\{\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\phi },\delta \mathit{\psi }\right)\right\}}_{ind}$ defined in Equation (35) can be computed by applying of the First-Order Comprehensive Adjoint Sensitivity Analysis Methodology (1st-CASAM), which avoids the need for computing the functions $\delta \mathit{\phi }\left(\mathit{x}\right)$ and $\delta \mathit{\psi }\left(\mathit{x}\right)$. The principles underlying the 1st-CASAM are as follows:

1

Introduce a Hilbert space, denoted as ${\mathrm{H}}_1$, comprising square-integrable functions vector-valued elements of the form ${\mathit{\eta }}^{\left(1\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{\eta }}_1^{\left(1\right)}\left(\mathit{x}\right),{\mathit{\eta }}_2^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†}$, with ${\mathit{\eta }}_i^{\left(1\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{\eta }}_{i,1}^{\left(1\right)}\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_{i,j}^{\left(1\right)}\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_{i,N_{\mathit{\phi }}}^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†},i=1,2,$ and endowed with an inner product between two elements, ${\mathit{\eta }}^{\left(1\right)}\left(\mathit{x}\right)\in {\mathrm{H}}_1$, ${\xi }^{\left(1\right)}\left(\mathit{x}\right)\in {\mathrm{H}}_1$, of this Hilbert space, which will be denoted as $\langle {\mathit{\eta }}^{\left(1\right)}\left(\mathit{x}\right),{\xi }^{\left(1\right)}\left(\mathit{x}\right){\rangle }_1$ and defined as follows:

$$\langle {\mathit{\eta }}^{\left(1\right)}\left(\mathit{x}\right),{\xi }^{\left(1\right)}\left(\mathit{x}\right){\rangle }_1\triangleq {\sum }_{i=1}^2\langle {\mathit{\eta }}_i^{\left(1\right)}\left(\mathit{x}\right),{\xi }_i^{\left(1\right)}\left(\mathit{x}\right){\rangle }_0.$$

(42)

2

In the Hilbert $H_1$, form the inner product of Equation (36) with a yet undefined vector-valued function ${\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right),{\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†}\in H_1$ to obtain the following relation:

$${\{\langle {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right),{\mathit{V}}^{\left(1\right)}\left(\mathit{\alpha }\right)\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right){\rangle }_1\}}_{{\mathit{\alpha }}^0}={\left\{\langle {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right),{\mathit{q}}^{\left(1\right)}\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_1\right\}}_{{\mathit{\alpha }}^0}.$$

(43)

3

Using the definition of the adjoint operator in the Hilbert space $H_1$, recast the left-side of Equation (43) as follows:

$$\begin{array}{l}{\{\langle {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right),{\mathit{V}}^{\left(1\right)}\left(\mathit{\alpha }\right)\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right){\rangle }_1\}}_{{\mathit{\alpha }}^0}\\ ={\left\{\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right),A^{\left(1\right)}\left(\mathit{\alpha }\right){\mathit{a}}^{\left(1\right)}{\left(\mathit{x}\right)}_1\right\}}_{{\mathit{\alpha }}^0}+{\left\{P^{\left(1\right)}\left[\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0},\end{array}$$

(44)

where ${\left\{P^{\left(1\right)}\left[\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}$ denotes the bilinear concomitant defined on the phase-space boundary $\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right)$, and where $A^{\left(1\right)}\left(\mathit{\alpha }\right)$ is the operator formally adjoint to ${\mathit{V}}^{\left(1\right)}\left(\mathit{\alpha }\right)$, i.e.,

$$A^{\left(1\right)}\left(\mathit{\alpha }\right)\triangleq {\left[{\mathit{V}}^{\left(1\right)}\left(\mathit{\alpha }\right)\right]}^{*}=\left(\begin{array}{cc}{\mathit{L}}^{*}\left(\mathit{\alpha }\right)& 0\\ 0& \mathit{L}\left(\mathit{\alpha }\right)\end{array}\right).$$

(45)

4

Require the first term on right-side of Equation (44) to represent the indirect-effect term defined in Equation (35), to obtain the following relation:

$$A^{\left(1\right)}\left(\mathit{\alpha }\right){\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)=s^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right),$$

(46)

where

$$s^{\left(1\right)}\left[{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right]\triangleq {\left[s_1^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right),s_2^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)\right]}^{†}\triangleq \left(\begin{array}{c}{\left[\frac{\partial S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }}\right]}^{†}\\ {\left[\frac{\partial S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }}\right]}^{†}\end{array}\right).$$

(47)

5

Implement the boundary conditions given in Equation (37) into Equation (44) and eliminate the remaining unknown boundary-values of the functions $\delta \mathit{\phi }$ and $\delta \mathit{\psi }$ from the expression of the bilinear concomitant ${\left\{P^{\left(1\right)}\left[\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}$ by selecting appropriate boundary conditions for the function ${\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right),{\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†}$, to ensure that Equation (46) is well-posed while being independent of unknown values of $\delta \mathit{\phi }$, $\delta \mathit{\psi }$, and $\delta \mathit{\alpha }$. The boundary conditions thus chosen for the function ${\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right),{\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†}$ can be represented in operator form as follow

$${\left\{{\mathit{b}}_A^{\left(1\right)}\left[{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\right]\right\}}_{{\mathit{\alpha }}^0}=0,\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right).$$

(48)

6

The selection of the boundary conditions for the adjoint function ${\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right),{\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†}$ represented by Equation (48) eliminates the appearance of the unknown values of $\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right)$ in ${\left\{P^{\left(1\right)}\left[\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}$ and reduces this bilinear concomitant to a residual quantity that contains boundary terms involving only known values of $\mathit{\phi }\left(\mathit{x}\right)$, $\mathit{\psi }\left(\mathit{x}\right)$, ${\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)$, $\mathit{\alpha }$ and $\delta \mathit{\alpha }$. This residual quantity will be denoted as ${\left\{{\widehat{P}}^{\left(1\right)}\left[{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}$. In general, this residual quantity does not automatically vanish, although it may do so occasionally.

7

The system of equations comprising Equation (46) together with the boundary conditions represented by Equation (48) constitute the 1st-Level Adjoint Sensitivity System (1st-LASS). the solution ${\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right),{\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†}$ of the 1st-LASS will be called the 1st-level adjoint function. The 1st-LASS is called “first-level” (as opposed to “first-order”) because it does not contain any differential or functional-derivatives, but its solution ${\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)$ will be used below to compute the first-order sensitivities of the response with respect to the model parameters. This terminology will be also used in the sequel, when deriving the expressions for the 2nd- and 3rd-order sensitivities.

8

It follows from Equations (43) and (44) that the following relation holds:

$$\begin{array}{l}{\{\langle {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right),{\mathit{q}}^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_1\}}_{{\mathit{\alpha }}^0}\\ ={\{\langle \delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right),A^{\left(1\right)}\left(\mathit{\alpha }\right){\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right){\rangle }_1\}}_{{\mathit{\alpha }}^0}+{\left\{{\widehat{P}}^{\left(1\right)}\left[{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}.\end{array}$$

(49)

9

Recalling that the first term on the right-side of Equation (49) is, in view of Equation (46), the indirect-effect term ${\left\{\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\phi },\delta \mathit{\psi }\right)\right\}}_{ind}$, it follows from Equation (49) that the indirect-effect term can be expressed in terms of the 1st-level adjoint function ${\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right),{\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†}$ as follows:

$$\begin{array}{l}{\left\{\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0,{\mathit{\alpha }}^0;\delta \mathit{\phi },\delta \mathit{\psi }\right)\right\}}_{ind}\\ ={\{\langle {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right),{\mathit{q}}^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_1\}}_{{\mathit{\alpha }}^0}-{\left\{{\widehat{P}}^{\left(1\right)}\left[{\mathit{u}}^{\left(1\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\\ ={\left\{{\displaystyle {\displaystyle \sum }_{i=1}^2}\langle {\mathit{a}}_i^{\left(1\right)}\left(\mathit{x}\right),{\mathit{q}}_i^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_0\right\}}_{{\mathit{\alpha }}^0}-{\left\{{\widehat{P}}^{\left(1\right)}\left[{\mathit{u}}^{\left(1\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\\ \equiv {\left\{\delta R\left({\mathit{u}}^{\left(1\right),0};{\mathit{a}}^{\left(1\right),0};{\mathit{\alpha }}^0;\delta \mathit{\alpha }\right)\right\}}_{ind}.\end{array}$$

(50)

As has been indicated by the last equality in Equation (50), the variations $\delta \mathit{\phi }$ and $\delta \mathit{\psi }$ have been eliminated from the original expression of the indirect-effect term, which now instead depends on the 1st-level adjoint function ${\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right),{\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†}$. As indicated in Equation (46), solving the 1st-Level Adjoint Sensitivity System (1st-LASS) entails the following operations: (i) inverting (i.e., solving) the original left-side of the adjoint equation with the source ${\left[\frac{\partial S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }}\right]}^{†}$ to obtain the 1st-level adjoint function ${\mathit{\psi }}_1^{\left(1\right)}\left(\mathit{x}\right)$; and (ii) inverting the original left-side of the forward equation with the source ${\left[\frac{\partial S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }}\right]}^{†}$ to obtain the 1st-level adjoint function ${\mathit{\psi }}_2^{\left(1\right)}\left(\mathit{x}\right)$. It is very important to note that the 1st-LASS is independent of parameter variations $\delta \mathit{\alpha }$. Hence, the 1st-LASS needs to be solved only once (as opposed to the 1st-LFSS, which would need to be solved anew for each parameter variation) to determine the 1st-level adjoint function ${\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right),{\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†}$. Subsequently, the “indirect-effect term” ${\left\{\delta R\left({\mathit{u}}^{\left(1\right),}{}^0;{\mathit{a}}^{\left(1\right),0};{\mathit{\alpha }}^0;\delta \mathit{\alpha }\right)\right\}}_{ind}$ is computed efficiently and exactly by simply performing the integrations over the adjoint function ${\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right),{\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†}$, as indicated on the right-side of Equation (50).

As indicated in Equation (33), the total 1st-order sensitivity of the response $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ to the model parameters is obtained by adding the expressions of the direct-effect term defined in Equation (34) and indirect-effect term as obtained in Equation (50), which yields the following expression:

$$\begin{array}{l}{\left\{\delta R\left[{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right),\mathit{\alpha };\delta \mathit{\phi },\delta \mathit{\psi },\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}={\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{\left\{{{\displaystyle \int }}_{{\kappa }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S\left[{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right]}{\partial \mathit{\alpha }}\delta \mathit{\alpha }\right\}}_{{\mathit{\alpha }}^0}\\ +{\displaystyle {\displaystyle \sum }_{k=1}^{N_{\mathit{x}}}{\displaystyle \prod }_{m=1,k\ne j}^{N_{\mathit{x}}}}{\left\{{{\displaystyle \int }}_{{\lambda }_m\left(\mathit{\alpha }\right)}^{{\omega }_m\left(\mathit{\alpha }\right)}d{\mathit{x}}_{m}S\left[\mathit{\phi }\left({\mathit{x}}_1,\dots ,{\mathit{b}}_k,\dots ,{\mathit{x}}_{N_{\mathit{x}}}\right),\mathit{\psi }\left({\mathit{x}}_1,\dots ,{\mathit{b}}_k,\dots ,{\mathit{x}}_{N_{\mathit{x}}}\right);\mathit{\alpha }\right]\frac{\partial {\omega }_k\left(\mathit{\alpha }\right)}{\partial \mathit{\alpha }}\delta \mathit{\alpha }\right\}}_{{\mathit{\alpha }}^0}\\ -{\displaystyle {\displaystyle \sum }_{k=1}^{N_{\mathit{x}}}{\displaystyle \prod }_{m=1,k\ne j}^{N_{\mathit{x}}}}{\left\{{{\displaystyle \int }}_{{\lambda }_m\left(\mathit{\alpha }\right)}^{{\omega }_m\left(\mathit{\alpha }\right)}d{\mathit{x}}_{m}S\left[\mathit{\phi }\left({\mathit{x}}_1,\dots ,{\mathit{a}}_k,\dots ,{\mathit{x}}_{N_{\mathit{x}}}\right),\mathit{\psi }\left({\mathit{x}}_1,\dots ,{\mathit{a}}_k,\dots ,{\mathit{x}}_{N_{\mathit{x}}}\right);\mathit{\alpha }\right]\frac{\partial {\lambda }_k\left(\mathit{\alpha }\right)}{\partial \mathit{\alpha }}\}\delta \mathit{\alpha }\right\}}_{{\mathit{\alpha }}^0}\\ +{\left\{{\displaystyle {\displaystyle \sum }_{i=1}^2}{\mathit{a}}_i^{\left(1\right)}\left(\mathit{x}\right),{\mathit{q}}_i^{\left(1\right)}{\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right)}_0\right\}}_{{\mathit{\alpha }}^0}-{\left\{{\widehat{P}}^{\left(1\right)}\left[{\mathit{u}}^{\left(1\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\\ \equiv {\left\{\delta R\left[{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}.\end{array}$$

(51)

The expression in Equation (51) no longer depends on the variations the variations $\delta \mathit{\phi }$ and $\delta \mathit{\psi }$, which are expensive to compute, but instead depends on the 1st-level adjoint function ${\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right),{\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†}$. In particular, this expression also reveals that the sensitivities of the response $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ to parameters that characterize the system’s boundary and/or internal interfaces can arise both from the direct-effect and indirect-effect terms. It also follows from Equation (51) that the total 1st-order sensitivity $\delta R\left({\mathit{\phi }}^0,{\mathit{\psi }}^0;{\mathit{a}}^{\left(1\right)};{\mathit{\alpha }}^0;\delta \mathit{\alpha }\right)$ can be expressed in terms of the 1st-level adjoint functions as follows:

$${\left\{\delta R\left[{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}={\sum }_{j=1}^{N_{\mathit{\alpha }}}{\left\{\frac{\partial R\left[{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{\alpha }}_j}\right\}}_{{\mathit{\alpha }}^0}\delta {\mathit{\alpha }}_j,$$

(52)

where the quantities

$$\begin{array}{l}\frac{\partial R\left[j;{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{\alpha }}_j}={\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}{{\displaystyle \int }}_{{\ell }_k\left(\mathit{\alpha }\right)}^{{\mathit{u}}_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k{S}^{\left(1\right)}\left[j;{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right]\\ \triangleq R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right], j=1,\dots ,N_{\mathit{\alpha }}.\end{array}$$

(53)

Represent the 1st-order response sensitivities, evaluated at the nominal values of the state functions and model parameters. In particular, if the residual bilinear concomitant is non-zero, the functions $S^{\left(1\right)}\left[j;\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right]$ would contain suitably defined Dirac delta-functionals for expressing the respective non-zero boundary terms as volume-integrals over the phase-space of the independent variables. Dirac-delta functionals would also be needed to represent, within $S^{\left(1\right)}\left[j;\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right]$, the terms containing the derivatives of the boundary end-points with respect to the model and/or response parameters.

In the particular case in which the model’s response depended only on the forward state function $\mathit{\phi }\left(\mathit{x}\right)$, the sensitivities $\partial R\left[\mathit{\phi }\left(\mathit{x}\right);{\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right]/\partial {\mathit{\alpha }}_j$,$j=1,\dots ,N_{\mathit{\alpha }},$ would depend only on the 1st-level adjoint function ${\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right)$, which is the solution of an adjoint-like 1st-LASS. Conversely, if the model’s response depended only on the adjoint state function $\mathit{\psi }\left(\mathit{x}\right)$, then the sensitivities $\partial R\left[\mathit{\psi }\left(\mathit{x}\right);{\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right]/\partial {\mathit{\alpha }}_j$,$j=1,\dots ,N_{\mathit{\alpha }},$ would depend only on the 1st-level adjoint function ${\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)$, which is the solution of a forward-like 1st-LASS.

3.1.4. Comparison of Computational Requirements for Computing the First-Order Response Sensitivities with Respect to the Model Parameters

From the derivations presented in Section 3.1.1, Section 3.1.2 and Section 3.1.3, it is evident that the computational requirements for the three deterministic methods discussed therein for computing the sensitivities of a scalar valued response $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ with respect to the $N_{\mathit{\alpha }}$ model parameters $\mathit{\alpha }\triangleq {\left({\mathit{\alpha }}_1,\dots ,{\mathit{\alpha }}_{N_{\mathit{\alpha }}}\right)}^{†}$ are as follows:

• The 1st-Order Comprehensive Adjoint Sensitivity Analysis Methodology (1st-CASAM) requires $2$ large-scale computations: one large-scale computation for computing the first-level adjoint state function ${\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right)$ and one large-scale computation for computing the first-level adjoint state function ${\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)$.
• The Forward Sensitivity Analysis Method (FSAM) requires $2N_{\mathit{\alpha }}$ large-scale computations: $N_{\mathit{\alpha }}$ large-scale computations for computing the first-order derivatives of the forward state function $\mathit{\phi }\left(\mathit{\alpha };\mathit{x}\right)$ and $N_{\mathit{\alpha }}$ large-scale computations for computing the first-order derivatives of the adjoint state function $\mathit{\psi }\left(\mathit{\alpha };\mathit{x}\right)$.
• The finite-difference (FD) method requires $4N_{\mathit{\alpha }}$ large-scale computations: $2N_{\mathit{\alpha }}$ large-scale computations for computing the forward state functions $\mathit{\phi }\left({\mathit{\alpha }}_j\pm {\mathit{h}}_j;\mathit{x}\right)$ and $2N_{\mathit{\alpha }}$ large-scale computations for computing the adjoint state functions $\mathit{\psi }\left({\mathit{\alpha }}_j\pm {\mathit{h}}_j;\mathit{x}\right)$.

If the response depends only on the forward state function, the following simplifications occur:

$${\mathit{V}}^{\left(1\right)}\left(\mathit{\alpha }\right)=\mathit{L}\left(\mathit{\alpha }\right);A^{\left(1\right)}\left(\mathit{\alpha }\right)={\mathit{L}}^{*}\left(\mathit{\alpha }\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)={\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right).$$

(54)

A similar simplification occurs in the boundary conditions represented by Equation (48).

If the response depends only on the adjoint state function, the following simplifications occur:

$${\mathit{V}}^{\left(1\right)}\left(\mathit{\alpha }\right)={\mathit{L}}^{*}\left(\mathit{\alpha }\right);A^{\left(1\right)}\left(\mathit{\alpha }\right)=\mathit{L}\left(\mathit{\alpha }\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)={\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right).$$

(55)

A similar simplification occurs in the boundary conditions represented by Equation (48). Hence, if the response depends only on the forward or only on the adjoint state function, then only half of the number of large-scale computations discussed in points A–C, above, would be needed.

It is evident that the 1st-CASAM is the most efficient of all of the above methods for computing the first-order sensitivities of scalar-valued responses to parameters, and the relative superiority of the 1st-CASAM over the other methods increases as the number of model parameters increases. This efficiency was evidently illustrated in [10,11,12,13,14,15], which presented the CPU times needed to compute the first-order sensitivities of the response to the model parameters for an OECD/NEA reactor physics benchmark [16], which consists of a plutonium metal sphere surrounded by a spherical shell made of polyethylene. The benchmark’s response of interest was the experimentally measured leakage of particles (neutrons) out of the sphere’s outer boundary, which depends only on the forward state function, but does not depend on the adjoint state-function (adjoint flux), so this response was modeled using a particular form of the right-side of Equation (13) within the PARTISN neutron transport code [17]. To compute the 7477 non-zero 1st-order sensitivities of the benchmark’s neutron leakage response to the benchmark’s parameters, the 1st-CASAM needed 1 large-scale adjoint computation for computing the 1st-level adjoint function [the equivalent of the function ${\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right)$], which required 85 CPU-seconds. In addition, the 1st-CASAM required 10 CPU-seconds for computing the 7477 integrals over the adjoint functions [i.e., the equivalent of computing Equation (50)] for computing the numerical values of all of the 7477 1st-order sensitivities. In contradistinction, the FSAM required 7477 large-scale computations for solving the respective 1st-OFSS [i.e., the equivalent of solving Equations (28) and (29)], which required 694-CPU Hours. The finite-difference (FD) approximation would have required 1388-CPU Hours for obtaining approximate values for the 7477 1st-order sensitivities.

3.2. Computation of the Second-Order Sensitivities of $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$

Since this work ultimately aims at deriving the explicit expressions of the 4th-order sensitivities of the response $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ with respect to the model parameters, the proliferation of indices, superscripts and subscripts is unavoidable. Nevertheless, “subscripted-subscripts” can be avoided by using subscripts of the form $j1=1,\dots ,N_{\mathit{\alpha }};j2=1,\dots ,j1$, where the index $j1$ will replace the index $j$ (which was used in the Section 3.1) to index the 1st-order sensitivities, and where the index $j2$ will be used (in addition to $j1$) to index the 2nd-order sensitivities. Furthermore, anticipating the notation to be used in subsequent Subsections, the index $j3$ will be used (in addition to the indices $j1$ and $j2$) to index the 3rd-order sensitivities and the index $j4$ will be used (in addition to $j1$, $j2$ and $j3$) to index the 4th-order sensitivities.

There are $N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)/2$ distinct second-order sensitivities of the response with respect to the model and response parameters. Section 3.2.1, below, summarizes the computational aspects of using finite-difference formulas and Section 3.2.2 describes the “forward sensitivity analysis methodology” for computing the 2nd-order response sensitivities to the model parameters. Section 3.2.3 presents the 2nd-Order Comprehensive Adjoint Sensitivity Analysis Methodology (2nd-CASAM), for computing the 2nd-order response sensitivities along the concepts originally developed by Cacuci [7,8,9].

3.2.1. Finite-Difference Approximation Using Re-Computations with User-Modified Parameters

The second-order responses sensitivities could be computed approximately by “brute force” re-computations using standard forward or backward differences. Thus, the second-order unmixed sensitivities can be calculated using the following formula:

$$\frac{{\partial }^{2}R\left(\mathit{\alpha }\right)}{{\left(\partial {\mathit{\alpha }}_{j1}\right)}^2}\approx \frac{1}{{\mathit{h}}_{j1}{}^2}\left(R_{j1+1}-R_{j1}+R_{j1-1}\right)+O\left({\mathit{h}}_{j1}{}^2\right),j1=1,\dots ,N_{\mathit{\alpha }},$$

(56)

where $R_{j1+1}\triangleq R\left({\mathit{\alpha }}_{j1}+{\mathit{h}}_{j1}\right)$, $R_{j1}\triangleq R\left({\mathit{\alpha }}_{j1}\right)$, $R_{j1-1}\triangleq R\left({\mathit{\alpha }}_{j1}-{\mathit{h}}_{j1}\right)$ and ${\mathit{h}}_{j1}$ denotes a “judiciously-chosen” variation in the parameter ${\mathit{\alpha }}_{j1}$ around its nominal value ${\mathit{\alpha }}_{j1}^0$.

The 2nd-order mixed sensitivities, ${\partial }^{2}R\left(\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$, can be calculated by using the following finite-difference formula:

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}}\approx \frac{1}{4{\mathit{h}}_{j1}{\mathit{h}}_{j2}}\left(R_{j1+1,j2+1}-R_{j1-1,j2+1}-R_{j1+1,j2-1}+R_{j1-1,j2-1}\right)\\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+O\left({\mathit{h}}_{j1}{}^2,{\mathit{h}}_{j2}{}^2\right),\\ for j1=1,\dots ,N_{\mathit{\alpha }};j2=1,\dots ,j1,\end{array}$$

(57)

where ${\mathit{h}}_{j1}$ and ${\mathit{h}}_{j2}$ denote the variations in the parameters ${\mathit{\alpha }}_{j1}$ and ${\mathit{\alpha }}_{j2}$, respectively. The values of the quantities $R_{j1+1,j2+1}\triangleq R\left({\mathit{\alpha }}_{j1}+{\mathit{h}}_{j1},{\mathit{\alpha }}_{j2}+{\mathit{h}}_{j2}\right)$, $R_{j1-1,j2+1}\triangleq R\left({\mathit{\alpha }}_{j1}-{\mathit{h}}_{j1},{\mathit{\alpha }}_{j2}+{\mathit{h}}_{j2}\right)$, $R_{j1+1,j2-1}\triangleq R\left({\mathit{\alpha }}_{j1}+{\mathit{h}}_{j1},{\mathit{\alpha }}_{j2}-{\mathit{h}}_{j2}\right)$, and $R_{j1-1,j2-1}\triangleq R\left({\mathit{\alpha }}_{j1}-{\mathit{h}}_{j1},{\mathit{\alpha }}_{j2}-{\mathit{h}}_{j2}\right)$ are obtained by repeatedly re-solving Equations (1), (2), (9) and (10), using the changed parameter values $\left({\mathit{\alpha }}_{j1}\pm {\mathit{h}}_{j1}\right)$ and $\left({\mathit{\alpha }}_{j2}\pm {\mathit{h}}_{j2}\right)$. The values of ${\mathit{h}}_{j1}$ and ${\mathit{h}}_{j2}$, respectively, must be chosen “judiciously” by “trial and error” for each of the parameters ${\mathit{\alpha }}_{j1}$ and ${\mathit{\alpha }}_{j2}$; otherwise, the result produced by Equation (57) will be erroneous, significantly removed from the exact value of the derivative ${\partial }^{2}R\left(\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$. These finite-difference formulas introduce their intrinsic “methodological errors” of order $\mathsf{O}\left({\mathit{h}}_{j1}{}^2\right)$ and/or $\mathsf{O}\left({\mathit{h}}_{j1}{}^2,{\mathit{h}}_{j2}{}^2\right)$, as indicated in Equations (56) and (57), which are in addition to, and independent of, the errors that might be incurred in the computation of $\mathit{L}\left(\mathit{\alpha }\right)$. In other words, even if the computation of $R\left(\mathit{\alpha }\right)$ were perfect (error-free), the finite-difference formulas nevertheless introduce their own, intrinsic, numerical errors into the computation of ${\partial }^{2}R\left(\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$.

3.2.2. Forward Sensitivity Analysis Methodology (FSAM)

Within the framework of the FSAM, the second-order sensitivities ${\partial }^{2}R\left(\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$,$j1,j2=1,\dots ,N_{\mathit{\alpha }}$, of the response to the model and response parameters are obtained by computing the partial G-derivatives of the first-order sensitivities represented by Equation (32), which yields the following result:

$$\begin{array}{l}\frac{{\partial }^{2}R\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}}\triangleq \frac{1}{\delta {\mathit{\alpha }}_{j2}}\{\frac{d}{d\epsilon }{R}^{\left(1\right)}[j1;\mathit{\phi }\left({\mathit{\alpha }}_{j2}^0+\epsilon \delta {\mathit{\alpha }}_{j2}\right);\mathit{\psi }\left({\mathit{\alpha }}_{j2}^0+\epsilon \delta {\mathit{\alpha }}_{j2}\right);{\mathit{\alpha }}_{j2}^0+\epsilon \delta {\mathit{\alpha }}_{j2};\\ {\frac{\partial \mathit{\phi }\left({\mathit{\alpha }}_{j2}^0+\epsilon \delta {\mathit{\alpha }}_{j2}\right)}{\partial {\mathit{\alpha }}_{j1}};\frac{\partial \mathit{\psi }\left({\mathit{\alpha }}_{j2}^0+\epsilon \delta {\mathit{\alpha }}_{j2}\right)}{\partial {\mathit{\alpha }}_{j1}}]\}}_{\epsilon =0}\\ =R^{\left(2\right)}\left[j2;j1;\mathit{\phi };\mathit{\psi };\mathit{\alpha };\frac{\partial \mathit{\phi }}{\partial {\mathit{\alpha }}_{j2}};\frac{\partial \mathit{\psi }}{\partial {\mathit{\alpha }}_{j2}};\frac{{\partial }^2\mathit{\phi }}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}};\frac{{\partial }^2\mathit{\psi }}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}}\right];j1,j2=1,\dots ,N_{\mathit{\alpha }}.\end{array}$$

(58)

As indicated by the functional dependence of the last term in Equation (58), the evaluation of the second-order response sensitivities requires prior knowledge of the 1st- and 2nd-order derivatives of the state functions with respect to the model parameters. The functions $\partial \mathit{\phi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_{j1}$ and $\partial \mathit{\psi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_{j1}$ are obtained by solving the 1st-OFSS. The functions ${\partial }^2\mathit{\phi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$ and ${\partial }^2\mathit{\psi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$ are the solutions of the following Second-Order Forward Sensitivity System (2nd-OFSS), which is obtained by taking the partial G-derivative of the 1st-OFSS with respect to a generic model parameter ${\mathit{\alpha }}_{j2}$:

$$\begin{gathered} \mathit{L}\left(\mathit{x};{\mathit{\alpha }}^0\right)\frac{{\partial }^2\mathit{\phi }}{\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}={\left\{f^{\left(2\right)}\left(j2;j1;\mathit{\phi };\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0};\mathit{x}\in {\Omega }_{\mathit{x}}; \\ j1=1,\dots ,N_{\mathit{\alpha }};j2=1,\dots ,j1; \end{gathered}$$

(59)

$$\begin{gathered} {\left\{{\mathit{B}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)\right\}}_{\left(e^0\right)}\frac{{\partial }^2\mathit{\phi }}{\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}={\left\{c_{\mathit{\phi }}^{\left(2\right)}\left(j2;j1;\mathit{\phi };\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0};\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right); \\ j1=1,\dots ,N_{\mathit{\alpha }};j2=1,\dots ,j1; \end{gathered}$$

(60)

$$\begin{gathered} {\mathit{L}}^{*}\left(\mathit{x};{\mathit{\alpha }}^0\right)\frac{{\partial }^2\mathit{\psi }}{\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}={\left\{g^{\left(2\right)}\left(j2;j1;\mathit{\psi };\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0};\mathit{x}\in {\Omega }_{\mathit{x}}; \\ j1=1,\dots ,N_{\mathit{\alpha }};j2=1,\dots ,j1; \end{gathered}$$

(61)

$$\begin{gathered} {\left\{{\mathit{B}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right)\right\}}_{\left(e^0\right)}\frac{{\partial }^2\mathit{\psi }}{\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}={\left\{c_{\mathit{\psi }}^{\left(2\right)}\left(j2;j1;\mathit{\psi };\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0};\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right); \\ j1=1,\dots ,N_{\mathit{\alpha }};j2=1,\dots ,j1; \end{gathered}$$

(62)

where

$$\begin{gathered} f^{\left(2\right)}\left(j2;j1;\mathit{\phi };\mathit{\alpha }\right)\triangleq \frac{\partial f^{\left(1\right)}\left(j1;\mathit{\phi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j2}}-\frac{\partial \mathit{L}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j2}}\frac{\partial \mathit{\phi }}{\partial {\mathit{\alpha }}_{j1}}; \\ j1=1,\dots ,N_{\mathit{\alpha }};j2=1,\dots ,j1; \end{gathered}$$

(63)

$$\begin{gathered} g^{\left(2\right)}\left(j2;j1;\mathit{\psi };\mathit{\alpha }\right)\triangleq \frac{\partial g^{\left(1\right)}\left(j1;\mathit{\psi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j2}}-\frac{\partial {\mathit{L}}^{*}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j2}}\frac{\partial \mathit{\psi }}{\partial {\mathit{\alpha }}_{j1}}; \\ j1=1,\dots ,N_{\mathit{\alpha }};j2=1,\dots ,j1; \end{gathered}$$

(64)

$$\begin{gathered} c_{\mathit{\phi }}^{\left(2\right)}\left(j2;j1;\mathit{\phi };\mathit{\alpha }\right)\triangleq \frac{\partial c_{\mathit{\phi }}^{\left(1\right)}\left(j1;\mathit{\phi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_2}-\frac{\partial {\mathit{B}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_2}\frac{\partial \mathit{\phi }}{\partial {\mathit{\alpha }}_{j1}};\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right); \\ j1=1,\dots ,N_{\mathit{\alpha }};j2=1,\dots ,j1; \end{gathered}$$

(65)

$$\begin{gathered} c_{\mathit{\psi }}^{\left(2\right)}\left(j2;j1;\mathit{\psi };\mathit{\alpha }\right)\triangleq \frac{\partial c_{\mathit{\psi }}^{\left(1\right)}\left(j1;\mathit{\psi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j2}}-\frac{\partial {\mathit{B}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j2}}\frac{\partial \mathit{\psi }\left(\mathit{x}\right)}{\partial {\mathit{\alpha }}_{j1}};\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right); \\ j1=1,\dots ,N_{\mathit{\alpha }};j2=1,\dots ,j1. \end{gathered}$$

(66)

Evidently, the 1st-OFSS would need to be solved $2N_{\mathit{\alpha }}$-times, with different right-sides in order to compute the 1st-order derivatives $\partial \mathit{\phi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_{j1}$ and $\partial \mathit{\psi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_{j1}$ with respect to all model parameters, which will entail large-scale computations. Furthermore, there are $N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)/2$ distinct 2nd-order functions ${\partial }^2\mathit{\phi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$ and just as many distinct 2nd-order functions ${\partial }^2\mathit{\psi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$. This means that the 2nd-OFSS would need to be solved $N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)$ times to compute these 2nd-order functions. Hence, the computation of the $N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)/2$ distinct 2nd-order response sensitivities to the model parameters using the FSAM would necessitate a total number of $N_{\mathit{\alpha }}^2+3N_{\mathit{\alpha }}$ large-scale computations, evidently highlighting the impact of the “curse of dimensionality” [Bellman, 1957].

3.2.3. The 2nd-Order Comprehensive Adjoint Sensitivity Analysis Methodology (2nd-CASAM)

The starting point for the development of the 2nd-CASAM is the expressions of the sensitivities provided in terms of the 1st-level adjoint functions, namely Equation (53), rather that expression of the 1st-order sensitivities produced by the FSAM [cf. Equation (32)]. Thus, the 2nd-order total sensitivity of the model response is obtained by applying the definition of the 1st-order G-differential to Equation (53), which yields the following result:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\delta {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\\ \triangleq {\left\{\frac{d}{d\epsilon }\left[{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left({\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }\right)}^{{\omega }_k\left({\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }\right)}d{\mathit{x}}_k{S}^{\left(1\right)}\left[\begin{array}{c}j1;{\mathit{\phi }}^0+\epsilon \delta \mathit{\phi };\\ {\mathit{\psi }}^0+\epsilon \delta \mathit{\psi };{\mathit{a}}^{\left(1\right),0}\left(\mathit{x}\right)+\epsilon \delta {\mathit{a}}^{\left(1\right)};{\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }\end{array}\right]\right]\right\}}_{\epsilon =0}\\ ={\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{dir}\\ +{\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\delta {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\right]\right\}}_{ind},j1=1,\dots ,N_{\mathit{\alpha }},\end{array}$$

(67)

where the direct-effect term ${\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{dir}$ is defined as follows:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{dir}\\ \triangleq {\left\{\frac{\partial }{\partial \mathit{\alpha }}{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k{S}^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\alpha },\end{array}$$

(68)

while the “indirect-effect term” ${\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\delta {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\right]\right\}}_{ind}$ is defined as follows:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\delta {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\right]\right\}}_{ind}\\ \triangleq {\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(1\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }}\delta \mathit{\phi }\left(\mathit{x}\right)\\ +{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(1\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }}\delta \mathit{\psi }\left(\mathit{x}\right)\\ +{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(1\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_1^{\left(1\right)}}\delta {\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right)\\ +{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(1\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_2^{\left(1\right)}}\delta {\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right).\end{array}$$

(69)

Note that the left-side of Equation (69) is to be computed at nominal parameter and state function values but the corresponding indication (namely, the superscript “zero”) has been omitted in Equation (69) in order to simplify the notation.

The direct-effect term ${\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{dir}$ can be computed immediately, while the “indirect-effect term” ${\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\delta {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\right]\right\}}_{ind}$ depends on the variations $\delta \mathit{\phi }$, $\delta \mathit{\psi }\left(\mathit{x}\right)$ and $\delta {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)$ in the forward state functions and, respectively, the components of the first-level adjoint function. Recall that the functions $\delta \mathit{\phi }$ and $\delta \mathit{\psi }\left(\mathit{x}\right)$ are the solutions of the 1st-LVSS defined by Equations (36) and (37). On the other hand, the function $\delta {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)$ is the solution of the system of equations obtained by G-differentiating the 1st-LASS. By definition, the first G-differential of the 1st-LASS is obtained as follows:

$${\left\{\frac{d}{d\epsilon }\left[{\mathit{L}}^{*}\left({\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }\right)\left({\mathit{a}}_1^{\left(1\right),0}+\epsilon \delta {\mathit{a}}_1^{\left(1\right)}\right)-\frac{\partial S\left({\mathit{\phi }}^0+\epsilon \delta \mathit{\phi };{\mathit{\psi }}^0+\epsilon \delta \mathit{\psi };{\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }\right)}{\partial \mathit{\phi }}\right]\right\}}_{\epsilon =0}=0,$$

(70)

$${\left\{\frac{d}{d\epsilon }\left[\mathit{L}\left({\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }\right)\left({\mathit{a}}_2^{\left(1\right),0}+\epsilon \delta {\mathit{a}}_2^{\left(1\right)}\right)-\frac{\partial S\left({\mathit{\phi }}^0+\epsilon \delta \mathit{\phi };{\mathit{\psi }}^0+\epsilon \delta \mathit{\psi };{\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }\right)}{\partial \mathit{\psi }}\right]\right\}}_{\epsilon =0}=0,$$

(71)

$${\left\{\frac{d}{d\epsilon }{\mathit{b}}_A^{\left(1\right)}\left[{\mathit{\phi }}^0+\epsilon \delta \mathit{\phi };{\mathit{\psi }}^0+\epsilon \delta \mathit{\psi };{\mathit{a}}^{\left(1\right),0}+\epsilon \delta {\mathit{a}}^{\left(1\right)};{\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }\right]\right\}}_{\epsilon =0}=0,\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right).$$

(72)

Carrying out the operations in Equations (70)–(72) yields the following equations:

$$\begin{gathered} {\mathit{L}}^{*}\left({\mathit{\alpha }}^0\right)\delta {\mathit{a}}_1^{\left(1\right)}-{\left\{\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }}\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\phi }-{\left\{\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }}\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\psi } \\ ={\left\{p_1^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};{\mathit{a}}_1^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0}, \\ {p}_1^{\left(2\right)}\left(\mathit{\phi };\mathit{\psi };{\mathit{a}}_1^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right)\triangleq \frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\alpha }\partial \mathit{\phi }}\delta \mathit{\alpha }-\frac{\partial \left[{\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_1^{\left(1\right)}\right]}{\partial \mathit{\alpha }}\delta \mathit{\alpha }, \end{gathered}$$

(73)

$$\begin{gathered} \mathit{L}\left({\mathit{\alpha }}^0\right)\delta {\mathit{a}}_2^{\left(1\right)}-{\left\{\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }}\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\phi }-{\left\{\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }}\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\psi } \\ ={\left\{p_2^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};{\mathit{a}}_2^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0}, \\ {p}_2^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};{\mathit{a}}_2^{\left(1\right)};\mathit{\alpha };\delta \mathit{\alpha }\right)\triangleq \frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\alpha }\partial \mathit{\psi }}\delta \mathit{\alpha }-\frac{\partial \left[\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_2^{\left(1\right)}\right]}{\partial \mathit{\alpha }}\delta \mathit{\alpha }, \end{gathered}$$

(74)

$$\begin{array}{l}{\left\{\delta {\mathit{b}}_A^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0}\\ \triangleq {\left\{\frac{\partial {\mathit{b}}_A^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }}\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\phi }+{\left\{\frac{\partial {\mathit{b}}_A^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }}\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\psi }\\ +{\left\{\frac{\partial {\mathit{b}}_A^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_1^{\left(1\right)}}\right\}}_{{\mathit{\alpha }}^0}\delta {\mathit{a}}_1^{\left(1\right)}+{\left\{\frac{\partial {\mathit{b}}_A^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_2^{\left(1\right)}}\right\}}_{{\mathit{\alpha }}^0}\delta {\mathit{a}}_2^{\left(1\right)}\\ +{\left\{\frac{\partial {\mathit{b}}_A^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\alpha }}\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\alpha }=0.\end{array}$$

(75)

Recalling the 1st-LVSS defined in Equations (36) and (37), and considering Equations (73) and (74), it follows that the functions $\delta \mathit{\phi }$, $\delta \mathit{\psi }\left(\mathit{x}\right)$ and $\delta {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\triangleq {\left[\delta {\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right),\delta {\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†}$ are the solution of the following system of equations:

$${\left\{{\mathit{V}}^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\right\}}_{{\mathit{\alpha }}^0}={\left\{{\mathit{q}}^{\left(2\right)}\left({\mathit{u}}^{\left(2\right)};\mathit{\alpha };\delta \mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0},\mathit{x}\in {\Omega }_{\mathit{x}},$$

(76)

$$\begin{gathered} {\left\{{\mathit{b}}_{\mathit{v}}^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)};\mathit{\alpha };\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\triangleq {\left\{\left(\begin{array}{c}{\mathit{b}}_{\mathit{v}}^{\left(1\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha };\delta {\mathit{u}}^{\left(1\right)};\delta \mathit{\alpha }\right)\\ \delta {\mathit{b}}_A^{\left(1\right)}\left({\mathit{u}}^{\left(2\right)};\mathit{\alpha };\delta {\mathit{u}}^{\left(2\right)};\delta \mathit{\alpha }\right)\end{array}\right)\right\}}_{{\mathit{\alpha }}^0}=\left(\begin{array}{c}0\\ 0\end{array}\right), \\ \mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right), \end{gathered}$$

(77)

where

$${\mathit{V}}^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)\triangleq \left(\begin{array}{cccc}\mathit{L}\left(\mathit{\alpha }\right)& 0& 0& 0\\ 0& {\mathit{L}}^{*}\left(\mathit{\alpha }\right)& 0& 0\\ -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }}& -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }}& {\mathit{L}}^{*}\left(\mathit{\alpha }\right)& 0\\ -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\phi }}& -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }}& 0& \mathit{L}\left(\mathit{\alpha }\right)\end{array}\right);$$

(78)

$$\begin{gathered} {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\triangleq \left(\begin{array}{c}{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right)\\ {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\end{array}\right)=\left(\begin{array}{c}\mathit{\phi }\left(\mathit{x}\right)\\ \mathit{\psi }\left(\mathit{x}\right)\\ {\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right)\\ {\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\end{array}\right);\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\triangleq \left(\begin{array}{c}\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right)\\ \delta {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\end{array}\right)=\left(\begin{array}{c}\delta \mathit{\phi }\left(\mathit{x}\right)\\ \delta \mathit{\psi }\left(\mathit{x}\right)\\ \delta {\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right)\\ \delta {\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\end{array}\right); \\ {q}^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\triangleq \left(\begin{array}{c}{q}_1^{\left(2\right)}\left(u^{\left(1\right)};\alpha ;\delta \alpha \right)\\ q_2^{\left(2\right)}\left(u^{\left(1\right)};\alpha ;\delta \alpha \right)\\ q_3^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\\ q_4^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\end{array}\right)\triangleq \left(\begin{array}{c}{q}_1^{\left(1\right)}\left(\phi ;\alpha ;\delta \alpha \right)\\ q_2^{\left(1\right)}\left(\psi ;\alpha ;\delta \alpha \right)\\ p_1^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\\ p_2^{\left(2\right)}\left(u^{\left(2\right)};\alpha ;\delta \alpha \right)\end{array}\right). \end{gathered}$$

(79)

All of the components of the matrices and vectors defined in Equations (76) and (77) are to be computed at nominal parameter and state function values. The matrix ${\mathit{V}}^{\left(2\right)}\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)$, which is defined in Equation (78), has the following structure:

$$\begin{array}{l}{\mathit{V}}^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)=\left(\begin{array}{cc}{\mathit{V}}^{\left(1\right)}& {\left[0\right]}_{2\times 2}\\ {\mathit{V}}_{21}^{\left(2\right)}& {\mathit{V}}_{22}^{\left(2\right)}\end{array}\right);\\ {\mathit{V}}_{21}^{\left(2\right)}\triangleq \left(\begin{array}{cc}-\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }}& -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }}\\ -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\phi }}& -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }}\end{array}\right),{\mathit{V}}_{22}^{\left(2\right)}=A^{\left(1\right)}\left(\mathit{\alpha }\right).\end{array}$$

(80)

The matrix ${\mathit{V}}_{21}^{\left(2\right)}$ depends only the system’s response and is responsible for coupling the forward and adjoint systems, even though they could still be solved successively rather than simultaneously, because the matrix ${\mathit{V}}^{\left(2\right)}\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)$ is block-diagonal.

The system of equations represented by Equation (76) together with the boundary conditions represented by Equation (77) will be called the “2nd-Level Variational Sensitivity System” (2nd-LVSS). The solution, $\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)$, of the 2nd-LVSS will be called the “2nd-level variational sensitivity function”. The 2nd-LVSS differs from the 2nd-OFSS in that the 2nd-LVSS involves first-order differentials of the state functions, whereas the 2nd-OFSS involves the second-order partial derivatives of the state functions with respect to model parameters ${\mathit{\alpha }}_{j2}$ and ${\mathit{\alpha }}_{j1}$, $j1=1,\dots ,N_{\mathit{\alpha }};$ $j2=1,\dots ,j1$. In principle, the 2nd-LVSS could be solved for each possible component of the vectors of parameter variations $\delta \mathit{\alpha }$ to obtain the 2nd-level variational vector $\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)$. Subsequently, $\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)$ could be used together with the known parameter variations $\delta \mathit{\alpha }$ in Equation (69) to compute the indirect-effect term ${\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\delta {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\right]\right\}}_{ind}$. Computing ${\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\delta {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\right]\right\}}_{ind}$ by solving the 2nd-LVSS would require at least $2N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)$ large-scale computations (to solve the 2nd-LVSS) for every independent component of the vectors of parameter variations $\delta \mathit{\alpha }$. Therefore, solving the 2nd-LVSS is advantageous to employ only if, in the problem under consideration, the number $N_{\mathit{\alpha }}$ of model and boundary parameters is considerably less than the number of responses of interest. This is rarely the case in practice, however, since most problems of practical interest are characterized by many model parameters and comparatively few responses.

Following the principles introduced by Cacuci [7,8,9], the application of the 2nd-CASAM avoids the need for solving the 2nd-LVSS, by expressing the indirect-effect term ${\left\{\delta R^{\left(1\right)}\left(j1\right)\right\}}_{ind}$ defined in Equation (69) in an alternative manner, in terms of the solution of a 2nd-Level Adjoint Sensitivity System (2nd-LASS), so as to eliminate the appearance of the 2nd-level variational vector $\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)$ in the alternative expression of the indirect-effect term ${\left\{\delta R^{\left(1\right)}\left(j1\right)\right\}}_{ind}$. The construction of the requisite 2nd-LASS commences by considering a Hilbert space, denoted as $H_2$, comprising square-integrable vector-valued elements of the form ${\mathit{\eta }}^{\left(2\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{\eta }}_1^{\left(2\right)}\left(\mathit{x}\right),{\mathit{\eta }}_2^{\left(2\right)}\left(\mathit{x}\right),{\mathit{\eta }}_3^{\left(2\right)}\left(\mathit{x}\right),{\mathit{\eta }}_4^{\left(2\right)}\left(\mathit{x}\right)\right]}^{†}\in H_2$, with ${\mathit{\eta }}_i^{\left(2\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{\eta }}_{i,1}^{\left(2\right)}\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_{i,j}^{\left(2\right)}\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_{i,N_{\mathit{\phi }}}^{\left(2\right)}\left(\mathit{x}\right)\right]}^{†},i=1,2,3,4,$. The inner product between two elements, ${\mathit{\eta }}^{\left(2\right)}\left(\mathit{x}\right)\in H_2$ and ${\xi }^{\left(2\right)}\left(\mathit{x}\right)\in H_2$, of this Hilbert space, will be denoted as ${\mathit{\eta }}^{\left(2\right)}\left(\mathit{x}\right),{\xi }^{\left(2\right)}{\left(\mathit{x}\right)}_2$ and is defined as follows:

$$\langle {\mathit{\eta }}^{\left(2\right)}\left(\mathit{x}\right),{\xi }^{\left(2\right)}\left(\mathit{x}\right){\rangle }_2\triangleq {\sum }_{i=1}^2\langle {\mathit{\eta }}_i^{\left(2\right)}\left(\mathit{x}\right),{\xi }_i^{\left(2\right)}\left(\mathit{x}\right){\rangle }_2={\sum }_{i=1}^4\langle {\mathit{\eta }}_i^{\left(2\right)}\left(\mathit{x}\right),{\xi }_i^{\left(2\right)}\left(\mathit{x}\right){\rangle }_0.$$

(81)

In the Hilbert $H_2$, form the inner product of Equation (76) with a set of yet undefined vector-valued functions ${\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{a}}_2^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}^{†}\in H_2$,$j1=1,\dots ,N_{\mathit{\alpha }}$, to obtain the following relation:

$${\{\langle {\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{V}}^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right){\rangle }_2\}}_{{\mathit{\alpha }}^0}={\{\langle {\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{q}}^{\left(2\right)}\left({\mathit{u}}^{\left(2\right)};\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_2\}}_{{\mathit{\alpha }}^0}.$$

(82)

Using the definition of the adjoint operator in the Hilbert space $H_2$, recast the left-side of Equation (82) as follows:

$$\begin{array}{l}{\left\{{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{V}}^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)\delta {\mathit{u}}^{\left(2\right)}{\left(\mathit{x}\right)}_2\right\}}_{{\mathit{\alpha }}^0}\\ ={\left\{\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right),A^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right){\mathit{a}}^{\left(2\right)}{\left(j1;\mathit{x}\right)}_2\right\}}_{{\mathit{\alpha }}^0}\\ +{\left\{P^{\left(2\right)}\left[\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0},\end{array}$$

(83)

where ${\left\{P^{\left(2\right)}\left[\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}$ denotes the bilinear concomitant defined on the phase-space boundary $\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right)$ and where $A^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)\triangleq {\left[{\mathit{V}}^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)\right]}^{*}$ is the operator formally adjoint to ${\mathit{V}}^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)$ and has therefore the following form:

$$\begin{array}{l}{A}^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)=\left(\begin{array}{cc}{A}^{\left(1\right)}\left(\mathit{\alpha }\right)& {\left[{\mathit{V}}_{21}^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)\right]}^{*}\\ {\left[0\right]}_{\left(2\times 2\right)}& {\mathit{V}}^{\left(1\right)}\left(\mathit{\alpha }\right)\end{array}\right)\\ =\left(\begin{array}{cccc}{\mathit{L}}^{*}\left(\mathit{\alpha }\right)& 0& -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }}& -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\phi }}\\ 0& \mathit{L}\left(\mathit{\alpha }\right)& -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }}& -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }}\\ 0& 0& \mathit{L}\left(\mathit{\alpha }\right)& 0\\ 0& 0& 0& {\mathit{L}}^{*}\left(\mathit{\alpha }\right)\end{array}\right).\end{array}$$

(84)

The first term on right-side of Equation (83) is now required to represent the indirect-effect term ${\left\{\delta R^{\left(1\right)}\left(j1\right)\right\}}_{ind}$ defined in Equation (69). This requirement is satisfied by imposing the following relation on each element ${\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{a}}_2^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}^{†}\in H_2$,$j1=1,\dots ,N_{\mathit{\alpha }}$:

$${\left\{A^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right){\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right)\right\}}_{{\mathit{\alpha }}^0}={\left\{s^{\left(2\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0},j1=1,\dots ,N_{\mathit{\alpha }},$$

(85)

where the block-vector $s^{\left(2\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)$ comprises, for each $j1=1,\dots ,N_{\mathit{\alpha }}$, four vector-components which are defined as follows:

$$\begin{gathered} \begin{array}{l}{s}^{\left(2\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right) \\ \triangleq {\left[s_1^{\left(2\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{x}\right),s_2^{\left(2\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{x}\right),s_3^{\left(2\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{x}\right),s_4^{\left(2\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{x}\right)\right]}^{†}\\ \triangleq [\frac{\partial S^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }},\frac{\partial S^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }},{\frac{\partial S^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_1^{\left(1\right)}},\frac{\partial S^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_2^{\left(1\right)}}]}^{†}.\end{array} \end{gathered}$$

(86)

The definition of the set of vectors ${\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{a}}_2^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}^{†}$ will be completed by selecting boundary conditions for this set of vectors, which will be represented in operator form as follows:

$${\left\{{\mathit{b}}_A^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}=0,\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right),j1=1,\dots ,N_{\mathit{\alpha }}.$$

(87)

The boundary conditions represented by Equation (87) are selected so as to satisfy the following requirements:

(i)

The boundary conditions Equation (87) together with the operator Equation (85) constitute a well posed problem for the functions ${\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right)$.

(ii)

The implementation in Equation (83) of the boundary conditions provided in Equation (77) together with those provided in Equation (87) eliminates all of the unknown values of the functions $\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)$ and ${\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right)$ in the expression of the bilinear concomitant ${\left\{P^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}$. The bilinear concomitant may vanish after these boundary conditions are implemented, but if it does not, it will be reduced to a residual quantity which will be denoted as ${\widehat{P}}^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]$ and which will comprise only known values of ${\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)$, ${\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right)$, $\mathit{\alpha }$ and $\delta \mathit{\alpha }$. In principle, the bilinear concomitant can always be “forced” to vanish by introducing delta-function terms in the definition of the adjoint operator, but such a practice could lead to severe (and usually unnecessary) difficulties when attempting to solve the equations that involve such extended adjoint operators.

The system of equations represented by Equation (85) together with the boundary conditions represented by Equation (87) constitute the 2nd-Level Adjoint Sensitivity System (2nd-LASS). The solution ${\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{a}}_2^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}^{†}\in H_2$,$j1=1,\dots ,N_{\mathit{\alpha }}$, of the 2nd-LASS will be called the 2nd-level adjoint function. The results provided in Equations (82), (83) and (85) are employed in Equation (69) to obtain the following expression for the indirect-effect term ${\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\delta {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\right]\right\}}_{ind}$ in terms of the 2nd-level adjoint functions ${\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right)$, for $j1=1,\dots ,N_{\mathit{\alpha }}$:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\delta {\mathit{a}}^{\left(1\right)}\left(\mathit{x}\right)\right]\right\}}_{ind}\\ ={\{\langle {\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{q}}^{\left(2\right)}\left({\mathit{u}}^{\left(2\right)};\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_2\}}_{{\mathit{\alpha }}^0}\\ -{\left\{{\widehat{P}}^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0},j1=1,\dots ,N_{\mathit{\alpha }}.\end{array}$$

(88)

Inserting the expressions that define the vector ${\mathit{q}}^{\left(2\right)}\left({\mathit{u}}^{\left(2\right)};\mathit{\alpha };\delta \mathit{\alpha }\right)$ from Equation (78) into Equation (88) and adding the resulting expression for the indirect-effect term with the expression of the direct-effect term given in Equation (68) yields the following expression for the total second-order G-differential of the response $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$:

$$\begin{array}{l}{\left\{\delta R^{\left(1\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\\ ={\left\{\frac{\partial }{\partial \mathit{\alpha }}{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\alpha }\\ +{\{\langle {\mathit{a}}_1^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{q}}_1^{\left(1\right)}\left(\mathit{\phi };\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_0\}}_{{\mathit{\alpha }}^0}+{\{\langle {\mathit{a}}_2^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{q}}_2^{\left(1\right)}\left(\mathit{\psi };\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +{\{\langle {\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{q}}_3^{\left(2\right)}\left({\mathit{u}}^{\left(2\right)};\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_0\}}_{{\mathit{\alpha }}^0}+{\{\langle {\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right),{\mathit{q}}_4^{\left(2\right)}\left({\mathit{u}}^{\left(2\right)};\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ -{\left\{{\widehat{P}}^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\\ \equiv {\displaystyle {\displaystyle \sum }_{j2=1}^{N_{\mathit{\alpha }}}}{\left\{R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\delta {\mathit{\alpha }}_{j2},\end{array}$$

(89)

where $R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]$ denotes the 2nd-order partial sensitivity of the response with respect to the model parameters, evaluated at the nominal parameter values ${\mathit{\alpha }}^0$, and has the following expression for $j1,j2=1,\dots ,N_{\mathit{\alpha }}$:

$$\begin{array}{l}{R}^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\equiv \frac{\partial R^{\left(1\right)}\left[j1;\mathit{\phi },\mathit{\psi };{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right]}{\partial {\mathit{\alpha }}_{j2}}\equiv \frac{{\partial }^{2}R\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}\\ =\frac{\partial }{\partial {\mathit{\alpha }}_{j2}}{\left\{{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0}\\ +{\{\langle {\mathit{a}}_1^{\left(2\right)}\left(j1;\mathit{x}\right),\frac{\partial \left[{\mathit{q}}_{\mathit{\phi }}\left(\mathit{\alpha }\right)-\mathit{L}\left(\mathit{\alpha }\right)\mathit{\phi }\left(\mathit{x}\right)\right]}{\partial {\mathit{\alpha }}_{j2}}{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +{\{\langle {\mathit{a}}_2^{\left(2\right)}\left(j1;\mathit{x}\right),\frac{\partial \left[{\mathit{q}}_{\mathit{\psi }}\left(\mathit{\alpha }\right)-{\mathit{L}}^{*}\left(\mathit{\alpha }\right)\mathit{\psi }\left(\mathit{x}\right)\right]}{\partial {\mathit{\alpha }}_{j2}}{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +{\{\langle {\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right),\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j2}\partial \mathit{\phi }}-\frac{\partial \left[{\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_1^{\left(1\right)}\right]}{\partial {\mathit{\alpha }}_{j2}}{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +{\{\langle {\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right),\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j2}\partial \mathit{\psi }}-\frac{\partial \left[\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_2^{\left(1\right)}\right]}{\partial {\mathit{\alpha }}_{j2}}{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ -{\left\{\frac{\partial }{\partial {\mathit{\alpha }}_{j2}}{\widehat{P}}^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}.\end{array}$$

(90)

Since the 2nd-LASS is independent of parameter variations $\delta \mathit{\alpha }$, the exact computation of all of the partial second-order sensitivities $R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]$ requires at most $N_{\mathit{\alpha }}$ large-scale (adjoint) computations using the 2nd-LASS, rather than $O\left(N_{\mathit{\alpha }}^2\right)$ large-scale computations as would be required by forward methods. In component form, the equations comprising the 2nd-LASS are as solved for each $j1=1,\dots ,N_{\mathit{\alpha }}$ in the following order:

$$\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)=\frac{\partial S^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_1^{\left(1\right)}},$$

(91)

$${\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)=\frac{\partial S^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_2^{\left(1\right)}},$$

(92)

$${\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_1^{\left(2\right)}\left(j1;\mathit{x}\right)=\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }}{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)+\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\phi }}{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)+\frac{\partial S^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }},$$

(93)

$$\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_2^{\left(2\right)}\left(j1;\mathit{x}\right)=\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }}{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)+\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }}{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)+\frac{\partial S^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }}.$$

(94)

It is important to note that by solving the 2nd-LASS ${\mathrm{N}}_{\mathsf{\alpha }}$-times, the “off-diagonal” 2nd-order mixed sensitivities ${\partial }^{2}R/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$ will be computed twice, in two different ways (i.e., using distinct 2nd-level adjoint functions), thereby providing an independent intrinsic (numerical) verification that the 1st- and 2nd-order response sensitivities are computed accurately. The information provided by the 1st-order sensitivities usually indicates which 2nd-order sensitivities are important and which could be neglected. Therefore, it is useful to prioritize the computation of the 2nd-order sensitivities by using the rankings of the relative magnitudes of the 1st-order sensitivities as a “priority indicator”: the larger the magnitude of the relative 1st-order sensitivity, the higher the priority for computing the corresponding 2nd-order sensitivities. Also, since vanishing 1st-order sensitivities may indicate critical points of the response in the phase-space of model parameters, it is also of interest to compute the 2nd-order sensitivities that correspond to vanishing 1st-order sensitivities. In practice, only those 2nd-order partial sensitivities which are deemed important would need to be computed.

Dirac delta-functionals in Equation (90) may need to be used for expressing the non-zero residual terms in the respective residual bilinear concomitant and/or the terms containing derivatives with respect to the lower- and upper-boundary points, so that the expression of the partial second-order sensitivities $R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]$ can be written in the following form, in preparation for computing the 3rd-order sensitivities:

$$\begin{array}{l}{R}^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\\ ={\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k{S}^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right].\end{array}$$

(95)

3.2.4. Comparison of Computational Requirements for Computing the Second-Order Response Sensitivities with Respect to the Model Parameters

Numerical results have been presented in [10,11,12,13,14,15] for the 27,956,503 nonzero 2nd-order sensitivities that correspond to the 7477 non-zero first-order sensitivities of the PERP benchmark’s leakage response to the benchmark’s parameters. These numerical results were computed using the PARTISN neutron transport code [17] running on a DELL desktop computer (AMD FX-8350) with an 8-core processor. A typical large-scale (forward or adjoint) computation required ca. 160 s CPU-time while the computation of a single 2nd-order sensitivity, including reading the adjoint functions and the integration over the adjoint functions, required ca. 0.046 s CPU-time. Altogether, the computation of all of the 27,956,503 nonzero 2nd-order sensitivities of the PERP leakage response using the 2nd-CASAM required ca. 929 CPU-hours, comprising 735 CPU-hours used for the 14, 843 large-scale computations (needed to compute the 2nd-level adjoint functions) and 194 CPU-hours used for performing the integrations needed to compute the respective unmixed and mixed second-order sensitivities.

In contradistinction, the FSAM would have required ca. 38,000 CPU-hours; using finite-differences (FD) would have required over 147,000 CPU-hours to compute the 27,956,503 nonzero 2nd-order sensitivities. Evidently, the 2nd-CASAM is the only method that can be used in practice for computing accurately and efficiently the 2nd-order response sensitivities for large-scale systems involving many model parameters.

3.3. Computation of the Third-Order Sensitivities of $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$

There are $N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)\left(N_{\mathit{\alpha }}+2\right)/3!$ distinct 3rd-order sensitivities of the response $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ with respect to the model and response parameters. The finite-difference formulas for computing these sensitivities are presented in Section 3.3.1. The Forward Sensitivity Analysis Methodology (FSAM) for computing the 3rd-order response sensitivities are presented in Section 3.3.2 and the 3rd-Order Comprehensive Adjoint Sensitivity Analysis Methodology (3rd-CASAM) is presented in Section 3.3.3. Section 3.3.4 presents a comparison of computational requirements for computing the third-order response sensitivities with respect to model parameters in the case of the OECD/NEA benchmark measured in [16].

3.3.1. Finite-Difference Approximation Using Re-Computations with User-Modified Parameters

The 3rd-order unmixed sensitivities, ${\partial }^{3}R\left(\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_j^3$, of the response can be approximately computed by using the well-known finite-difference formula presented below:

$$\frac{{\partial }^{3}R\left(\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j1}{}^3}\approx \frac{1}{2{\mathit{h}}_{j1}{}^3}\left(R_{j1+2}-2R_{j1+1}+2R_{j1-1}-R_{j1-2}\right)+O\left({\mathit{h}}_{j1}{}^2\right),j1=1,\dots ,N_{\mathit{\alpha }},$$

(96)

where $R_{j1+2}\triangleq R\left({\mathit{\alpha }}_{j1}+2{\mathit{h}}_{j1}\right)$, $R_{j1+1}\triangleq R\left({\mathit{\alpha }}_{j1}+{\mathit{h}}_{j1}\right)$, $R_{j1-1}\triangleq R\left({\mathit{\alpha }}_{j1}-{\mathit{h}}_{j1}\right)$, $R_{j1-2}\triangleq R\left({\mathit{\alpha }}_{j1}-2{\mathit{h}}_{j1}\right)$ and ${\mathit{h}}_{j1}$ denotes a “judiciously-chosen” variation in the parameter ${\mathit{\alpha }}_{j1}$ around its nominal value ${\mathit{\alpha }}_{j1}^0$. The 3rd-order mixed sensitivities, ${\partial }^{3}R\left(\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}$, can be calculated by using the following finite-difference formula:

$$\begin{array}{ll}\frac{{\partial }^{3}R\left(\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}}\approx & \frac{1}{8{\mathit{h}}_{j1}{\mathit{h}}_{j2}{\mathit{h}}_{j3}}(R_{j1+1,j2+1,j3+1}-R_{j1+1,j2+1,j3-1}-R_{j1+1,j2-1,j3+1}\\ & +R_{j1+1,j2-1,j3-1}-R_{j1-1,j2+1,j3+1}+R_{j1-1,j2+1,j3-1}\\ & +R_{j1-1,j2-1,j3+1}-R_{j1-1,j2-1,j3-1})+O\left({\mathit{h}}_{j1}{}^2,{\mathit{h}}_{j2}{}^2,{\mathit{h}}_{j3}{}^2\right),\end{array}$$

(97)

where $R_{j1+1,j2+1,j3+1}=R\left({\mathit{\alpha }}_{j1}+{\mathit{h}}_{j1},{\mathit{\alpha }}_{j2}+{\mathit{h}}_{j2},{\mathit{\alpha }}_{j3}+{\mathit{h}}_{j3}\right)$, $R_{j1-1,j2+1,j3+1}=R\left({\mathit{\alpha }}_{j1}-{\mathit{h}}_{j1},{\mathit{\alpha }}_{j2}+{\mathit{h}}_{j2},{\mathit{\alpha }}_{j3}+{\mathit{h}}_{j3}\right)$, etc. The values of the quantities on the rights-sides of the expressions shown in Equations (69) and (70) are obtained by re-solving Equations (1) and (2) repeatedly, using the changed parameter values $\left({\mathit{\alpha }}_{j1}\pm {\mathit{h}}_{j1}\right)$, $\left({\mathit{\alpha }}_{j1}\pm {\mathit{h}}_{j2}\right)$ and $\left({\mathit{\alpha }}_{j3}\pm {\mathit{h}}_{j3}\right)$. As has also been previously discussed, the values of ${\mathit{h}}_{j1}$, ${\mathit{h}}_{j2}$ and ${\mathit{h}}_{j3}$, respectively, must be chosen judiciously by trial and error for each of the parameters ${\mathit{\alpha }}_{j1}$, ${\mathit{\alpha }}_{j2}$ and ${\mathit{\alpha }}_{j3}$. The finite difference formulas introduce their intrinsic “methodological errors” of order $\mathsf{O}\left({\mathit{h}}_{j1}{}^2,{\mathit{h}}_{j2}{}^2,{\mathit{h}}_{j3}{}^2\right)$ which are in addition to, and independent of, the errors that might be incurred in the computation of ${\partial }^{3}R\left(\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}$.

3.3.2. Forward Sensitivity Analysis Methodology

Within the framework of the FSAM, the third-order sensitivities ${\partial }^{3}R\left(\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}$,$j1,j2,j3=1,\dots ,N_{\mathit{\alpha }}$, of the response to the model parameters are obtained by computing the partial G-derivatives of the second-order sensitivities represented by Equation (58), which yields the following result:

$$\begin{array}{l}\frac{{\partial }^{3}R\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}}\triangleq \frac{1}{\delta {\mathit{\alpha }}_{j3}}\{\frac{d}{d\epsilon }{R}^{\left(2\right)}[j2;j1;\mathit{\phi }\left({\mathit{\alpha }}_{j3}^0+\epsilon \delta {\mathit{\alpha }}_{j3}\right);\mathit{\psi }\left({\mathit{\alpha }}_{j3}^0+\epsilon \delta {\mathit{\alpha }}_{j3}\right);{\mathit{\alpha }}_{j3}^0+\epsilon \delta {\mathit{\alpha }}_{j3};\\ {\frac{\partial \mathit{\phi }\left({\mathit{\alpha }}_{j3}^0+\epsilon \delta {\mathit{\alpha }}_{j3}\right)}{\partial {\mathit{\alpha }}_{j1}};\frac{\partial \mathit{\psi }\left({\mathit{\alpha }}_{j3}^0+\epsilon \delta {\mathit{\alpha }}_{j3}\right)}{\partial {\mathit{\alpha }}_{j1}};\frac{{\partial }^2\mathit{\phi }\left({\mathit{\alpha }}_{j3}^0+\epsilon \delta {\mathit{\alpha }}_{j3}\right)}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}};\frac{{\partial }^2\mathit{\psi }\left({\mathit{\alpha }}_{j3}^0+\epsilon \delta {\mathit{\alpha }}_{j3}\right)}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}}]\}}_{\epsilon =0}\\ =R^{\left(3\right)}\left[j2;j1;\mathit{\phi };\mathit{\psi };\mathit{\alpha };\frac{\partial \mathit{\phi }}{\partial {\mathit{\alpha }}_{j2}};\frac{\partial \mathit{\psi }}{\partial {\mathit{\alpha }}_{j2}};\frac{{\partial }^2\mathit{\phi }}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}};\frac{{\partial }^2\mathit{\psi }}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}};\frac{{\partial }^3\mathit{\phi }}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}};\frac{{\partial }^3\mathit{\psi }}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}}\right].\end{array}$$

(98)

As indicated by the functional dependence of the last term in Equation (98), the evaluation of the 3rd-order response sensitivities requires prior knowledge of the 1st- and 2nd-order derivatives of the state functions with respect to the model parameters. The functions $\partial \mathit{\phi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_{j2}$ and $\partial \mathit{\psi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_{j2}$ are obtained by solving the 1st-OFSS. The functions ${\partial }^2\mathit{\phi }/{\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$ and ${\partial }^2\mathit{\psi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$ are the solutions of the 2nd-OFSS. The functions $\partial {\mathit{\alpha }}_{j3}{\partial }^3\mathit{\phi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}$ and ${\partial }^3\mathit{\psi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}$ are the solutions of the following “Third-Order Forward Sensitivity System” (3rd-OFSS), which is obtained by taking the partial G-derivative of the 2nd-OFSS with respect to a generic model parameter ${\mathit{\alpha }}_{j3}$, which yields the following system:

$$\mathit{L}\left(\mathit{x};{\mathit{\alpha }}^0\right)\frac{{\partial }^3\mathit{\phi }}{\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}={\left\{f^{\left(3\right)}\left(j3;j2;j1;\mathit{\phi };\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0};j1,j2,j3=1,\dots ,N_{\mathit{\alpha }};\mathit{x}\in {\Omega }_{\mathit{x}};$$

(99)

$${\left\{{\mathit{b}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0}\frac{{\partial }^3\mathit{\phi }}{\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}={\left\{c_{\mathit{\phi }}^{\left(3\right)}\left(j3;j2;j1;\mathit{\phi };\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0};j1,j2,j3=1,\dots ,N_{\mathit{\alpha }};\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right);$$

(100)

$${\mathit{L}}^{*}\left(\mathit{x};{\mathit{\alpha }}^0\right)\frac{{\partial }^3\mathit{\psi }}{\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}={\left\{g^{\left(3\right)}\left(j3;j2;j1;\mathit{\psi };\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0};j1,j2,j3=1,\dots ,N_{\mathit{\alpha }};\mathit{x}\in {\Omega }_{\mathit{x}};$$

(101)

$$\begin{gathered} {\left\{{\mathit{b}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0}\frac{{\partial }^3\mathit{\psi }}{\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}={\left\{c_{\mathit{\psi }}^{\left(3\right)}\left(j3;j2;j1;\mathit{\psi };\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0};j1,j2,j3=1,\dots ,N_{\mathit{\alpha }}; \\ \mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right); \end{gathered}$$

(102)

where

$$f^{\left(3\right)}\left(j3;j2;j1;\mathit{\phi };\mathit{\alpha }\right)\triangleq \frac{\partial f^{\left(2\right)}\left(j2;j1;\mathit{\phi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}}-\frac{\partial \mathit{L}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}}\frac{{\partial }^2\mathit{\phi }}{\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}};$$

(103)

$$g^{\left(3\right)}\left(j3;j2;j1;\mathit{\psi };\mathit{\alpha }\right)\triangleq \frac{\partial g^{\left(2\right)}\left(j2;j1;\mathit{\psi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}}-\frac{\partial {\mathit{L}}^{*}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}}\frac{{\partial }^2\mathit{\psi }}{\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}};$$

(104)

$$c_{\mathit{\phi }}^{\left(3\right)}\left(j3;j2;j1;\mathit{\phi };\mathit{\alpha }\right)\triangleq \frac{\partial c_{\mathit{\phi }}^{\left(2\right)}\left(j2;j1;\mathit{\phi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}}-\frac{\partial {\mathit{b}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}}\frac{{\partial }^2\mathit{\phi }}{\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}};\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right).$$

(105)

$$c_{\mathit{\psi }}^{\left(3\right)}\left(j3;j2;j1;\mathit{\psi };\mathit{\alpha }\right)\triangleq \frac{\partial c_{\mathit{\psi }}^{\left(2\right)}\left(j2;j1;\mathit{\psi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}}-\frac{\partial {\mathit{b}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}}\frac{{\partial }^2\mathit{\psi }}{\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}};\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right).$$

(106)

Evidently, the computations of the third-order sensitivities ${\partial }^{3}R\left(\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}$ require $2N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)\left(N_{\mathit{\alpha }}+2\right)/3!$ large-scale computations to solve Equations (99)–(102), followed by $2N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)\left(N_{\mathit{\alpha }}+2\right)/3!$ small-scale computations for performing the integrations that define the various sensitivities. These computations would be in addition to the $2N_{\mathit{\alpha }}$ large-scale computations needed to determine the functions $\partial \mathit{\phi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_{j2}$ and $\partial \mathit{\psi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_{j2}$ by solving the 1st-OFSS, and the $N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)$ large-scale computations needed to determine the functions ${\partial }^2\mathit{\phi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$ and ${\partial }^2\mathit{\psi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$ by solving the 2nd-OFSS.

3.3.3. The 3rd-Order Comprehensive Adjoint Sensitivity Analysis Methodology (3rd-CASAM)

The third-order sensitivities of the response $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ with respect to the model parameters are obtained by determining the first-order G-differential of the 2nd-order sensitivities $R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]={\partial }^{2}R\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$, which were computed in Equation (90) in Section 3.2.3. By definition, the total G-differential of $R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]$ is obtained as follows:

$$\begin{array}{l}{\left\{\delta R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)};{\mathit{a}}^{\left(2\right)}\left(j1\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\\ \triangleq {\left\{\frac{d}{d\epsilon }\left[{\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}{{\displaystyle \int }}_{{\lambda }_k\left({\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }\right)}^{{\omega }_k\left({\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }\right)}d{\mathit{x}}_k{S}^{\left(2\right)}[\begin{array}{c}j2;j1;{\mathit{u}}^{\left(2\right),0}+\epsilon \delta {\mathit{u}}^{\left(2\right)};\\ {\mathit{a}}^{\left(2\right),0}\left(j1\right)+\epsilon \delta {\mathit{a}}^{\left(2\right)}\left(j1\right);{\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }\end{array}\right]\right\}}_{\epsilon =0}\\ ={\left\{\delta R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right),0};{\mathit{a}}^{\left(2\right),0}\left(j1\right);{\mathit{\alpha }}^0;\delta \mathit{\alpha }\right]\right\}}_{dir}\\ +{\left\{\delta R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right),0};{\mathit{a}}^{\left(2\right),0}\left(j1\right);{\mathit{\alpha }}^0;\delta {\mathit{u}}^{\left(2\right)};\delta {\mathit{a}}^{\left(2\right)}\left(j1\right)\right]\right\}}_{ind},\end{array}$$

(107)

where the direct-effect term ${\left\{\delta R^{\left(2\right)}\left(j2;j1\right)\right\}}_{dir}$ depends directly on the parameter variations and is defined as follows:

$$\begin{array}{l}{\left\{\delta R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right),0};{\mathit{a}}^{\left(2\right),0}\left(j1\right);{\mathit{\alpha }}^0;\delta \mathit{\alpha }\right]\right\}}_{dir}\\ \triangleq {\left\{\frac{\partial }{\partial \mathit{\alpha }}{\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}\underset{{\lambda }_k\left(\mathit{\alpha }\right)}{\overset{{\omega }_k\left(\mathit{\alpha }\right)}{{\displaystyle \int }}}d{\mathit{x}}_k{S}^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\alpha },\end{array}$$

(108)

while the “indirect-effect term” ${\left\{\delta R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right),0};{\mathit{a}}^{\left(2\right),0}\left(j1\right);{\mathit{\alpha }}^0;\delta {\mathit{u}}^{\left(2\right)};\delta {\mathit{a}}^{\left(2\right)}\left(j1\right)\right]\right\}}_{ind}$ depends indirectly on the parameter variations through the variations in the forward and adjoint state functions and is defined as follows:

$$\begin{array}{l}{\left\{\delta R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right),0};{\mathit{a}}^{\left(2\right),0}\left(j1\right);{\mathit{\alpha }}^0;\delta {\mathit{u}}^{\left(2\right)};\delta {\mathit{a}}^{\left(2\right)}\left(j1\right)\right]\right\}}_{ind}\\ \triangleq {\left\{{\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial \mathit{\phi }}\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\phi }\left(\mathit{x}\right)\\ +{\left\{{\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial \mathit{\psi }}\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\psi }\left(\mathit{x}\right)\\ +{\left\{{\displaystyle \sum }_{m=1}^2{\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{a}}_m^{\left(1\right)}}\right\}}_{{\mathit{\alpha }}^0}\delta {\mathit{a}}_m^{\left(1\right)}\left(\mathit{x}\right)\\ +{\left\{{\displaystyle \sum }_{m=1}^4{\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{a}}_m^{\left(2\right)}\left(j1\right)}\right\}}_{{\mathit{\alpha }}^0}\delta {\mathit{a}}_m^{\left(2\right)}\left(j1;\mathit{x}\right).\end{array}$$

(109)

The indirect-effect term defined in Equation (109) can be computed only after having computed the functions $\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)$ and $\delta {\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right)\triangleq {\left[\delta {\mathit{a}}_1^{\left(2\right)}\left(j1;\mathit{x}\right),\delta {\mathit{a}}_2^{\left(2\right)}\left(j1;\mathit{x}\right),\delta {\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right),\delta {\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}^{†}$. The vector of variations $\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)$ is the solution of the 2nd-LVSS while the vector $\delta {\mathit{a}}^{\left(2\right)}\left(\mathit{x}\right)$ is the solution of the system of equations obtained by G-differentiating the 2nd-LASS defined by Equations (85) and (87).

Applying the definition of the total G-differential, cf. Equation (23), to the 2nd-LASS yields the following set of equations for the vector of variations $\delta {\mathit{a}}^{\left(2\right)}\left(\mathit{x}\right)$, all to be computed at the nominal values of the parameters and state functions (although this will not be indicated explicitly below, in order to keep the notation as simple as possible):

$$\begin{array}{l}{\mathit{L}}^{*}\left(\mathit{\alpha }\right)\delta {\mathit{a}}_1^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }}\delta {\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)\\ -\left[\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }\partial \mathit{\phi }}\delta \mathit{\phi }+\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }\partial \mathit{\psi }}\delta \mathit{\psi }\right]{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\phi }}\delta {\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)\\ -\left[\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\phi }\partial \mathit{\phi }}\delta \mathit{\phi }+\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }\partial \mathit{\phi }}\delta \mathit{\psi }\right]{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }}\delta \mathit{\phi }\\ -\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }}\delta \mathit{\psi }-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial {\mathit{a}}_1^{\left(1\right)}}\delta {\mathit{a}}_1^{\left(1\right)}\\ -\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial {\mathit{a}}_2^{\left(1\right)}}\delta {\mathit{a}}_2^{\left(1\right)}=p_1^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)};{\mathit{a}}^{\left(2\right)}\left(j1\right);\mathit{\alpha };\delta \mathit{\alpha }\right],\end{array}$$

(110)

$$\begin{gathered} \begin{array}{l}\mathit{L}\left(\mathit{\alpha }\right)\delta {\mathit{a}}_2^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }}\delta {\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)\\ -\left[\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }\partial \mathit{\psi }}\delta \mathit{\phi }+\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }\partial \mathit{\psi }}\delta \mathit{\psi }\right]{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right) \\ -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }}\delta {\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)\\ -\left[\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }\partial \mathit{\psi }}\delta \mathit{\phi }+\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }\partial \mathit{\psi }}\delta \mathit{\psi }\right]{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right) \\ -\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }}\delta \mathit{\phi }\\ -\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }}\delta \mathit{\psi }-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_1^{\left(1\right)}\partial \mathit{\psi }}\delta {\mathit{a}}_1^{\left(1\right)}-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_2^{\left(1\right)}\partial \mathit{\psi }}\delta {\mathit{a}}_2^{\left(1\right)}\\ =p_2^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)};{\mathit{a}}^{\left(2\right)}\left(j1\right);\mathit{\alpha };\delta \mathit{\alpha }\right],\end{array} \end{gathered}$$

(111)

$$\begin{array}{l}\mathit{L}\left(\mathit{\alpha }\right)\delta {\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial {\mathit{a}}_1^{\left(1\right)}}\delta \mathit{\phi }-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial {\mathit{a}}_1^{\left(1\right)}}\delta \mathit{\psi }\\ -\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_1^{\left(1\right)}\partial {\mathit{a}}_1^{\left(1\right)}}\delta {\mathit{a}}_1^{\left(1\right)}-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_1^{\left(1\right)}\partial {\mathit{a}}_2^{\left(1\right)}}\delta {\mathit{a}}_2^{\left(1\right)}\\ =p_3^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)};{\mathit{a}}^{\left(2\right)}\left(j1\right);\mathit{\alpha };\delta \mathit{\alpha }\right],\end{array}$$

(112)

$$\begin{array}{c}{\mathit{L}}^{*}\left(\mathit{\alpha }\right)\delta {\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial {\mathit{a}}_2^{\left(1\right)}}\delta \mathit{\phi }-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial {\mathit{a}}_2^{\left(1\right)}}\delta \mathit{\psi }\\ -\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_1^{\left(1\right)}\partial {\mathit{a}}_2^{\left(1\right)}}\delta {\mathit{a}}_1^{\left(1\right)}-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_2^{\left(1\right)}\partial {\mathit{a}}_2^{\left(1\right)}}\delta {\mathit{a}}_2^{\left(1\right)}\\ =p_4^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)};{\mathit{a}}^{\left(2\right)}\left(j1\right);\mathit{\alpha };\delta \mathit{\alpha }\right],\end{array}$$

(113)

where:

$$\begin{array}{l}{p}_1^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)};{\mathit{a}}^{\left(2\right)}\left(j1\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\\ \triangleq -\frac{\partial \left[{\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_1^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}{\partial \mathit{\alpha }}\delta \mathit{\alpha }+\left[\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\alpha }\partial \mathit{\phi }\partial \mathit{\phi }}\delta \mathit{\alpha }\right]{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)\\ +\left[\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\alpha }\partial \mathit{\psi }\partial \mathit{\phi }}\delta \mathit{\alpha }\right]{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)+\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\alpha }\partial \mathit{\phi }}\delta \mathit{\alpha },\end{array}$$

(114)

$$\begin{array}{l}{p}_2^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)};{\mathit{a}}^{\left(2\right)}\left(j1\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\triangleq -\frac{\partial \left[\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_2^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}{\partial \mathit{\alpha }}\delta \mathit{\alpha }\\ +\left[\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\alpha }\partial \mathit{\phi }\partial \mathit{\psi }}\delta \mathit{\alpha }\right]{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)+\left[\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\alpha }\partial \mathit{\psi }\partial \mathit{\psi }}\delta \mathit{\alpha }\right]{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)\\ +\frac{\partial S^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\alpha }\partial \mathit{\psi }}\delta \mathit{\alpha },\end{array}$$

(115)

$$p_3^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)};{\mathit{a}}^{\left(2\right)}\left(j1\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\triangleq -\frac{\partial \left[\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}{\partial \mathit{\alpha }}\delta \mathit{\alpha }+\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\alpha }\partial {\mathit{a}}_1^{\left(1\right)}}\delta \mathit{\alpha },$$

(116)

$$p_4^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)};{\mathit{a}}^{\left(2\right)}\left(j1\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\triangleq -\frac{\partial \left[{\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}{\partial \mathit{\alpha }}\delta \mathit{\alpha }+\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\alpha }\partial {\mathit{a}}_2^{\left(1\right)}}\delta \mathit{\alpha }.$$

(117)

The vector of variations $\delta {\mathit{a}}^{\left(2\right)}\left(\mathit{x}\right)$ is subject to the boundary conditions obtained by G-differentiating Equation (87), which will be represented in operator form as follows:

$$\begin{gathered} {\left\{\delta {\mathit{b}}_A^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)};{\mathit{a}}^{\left(2\right)}\left(j1\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(1\right)};\delta {\mathit{a}}^{\left(2\right)}\left(j1\right);\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}=0, \\ \mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right),j1=1,\dots ,N_{\mathit{\alpha }}. \end{gathered}$$

(118)

As Equations (110) through (118) indicate, the vector of variations $\delta {\mathit{a}}^{\left(2\right)}\left(\mathit{x}\right)$ and $\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\triangleq {\left[\delta {\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\delta {\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right),\delta {\mathit{a}}_2^{\left(1\right)}\left(\mathit{x}\right)\right]}^{†}$ are related to each other, and must be determined by solving simultaneously the coupled system of equations obtained by concatenating Equations (110) through (117) to the 2nd-LVSS, while being subject to the boundary conditions obtained by concatenating the boundary conditions which belong to the 2nd-LVSS with the boundary conditions represented by Equation (118). This coupled system will comprise 8 (coupled) equations, which can be represented in the following matrix-form:

$$\begin{array}{l}{\mathit{V}}^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\right]\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)={\mathit{q}}^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right],\\ \mathit{x}\in {\Omega }_{\mathit{x}},j1=1,\dots ,N_{\mathit{\alpha }},\end{array}$$

(119)

$$\begin{array}{l}{\mathit{b}}_{\mathit{v}}^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\delta \mathit{\alpha }\right]\\ \triangleq \left(\begin{array}{c}{\mathit{b}}_{\mathit{V}}^{\left(2\right)}\left[{\mathit{u}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(2\right)}\left(j1;\mathit{x}\right);\delta \mathit{\alpha }\right]\\ \delta {\mathit{b}}_A^{\left(2\right)}\left[j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\delta \mathit{\alpha }\right]\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right),\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right),\end{array}$$

(120)

where all of the components of the matrices and vectors are to be computed at nominal parameter and state function values, although the corresponding indication (namely, the superscript “zero”) has been omitted in order to simplify the notation. The 8 × 8 block-matrix ${\mathit{V}}^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\right]$, and the 8-component block-vectors $\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)$ and ${\mathit{q}}^{\left(3\right)}\left[{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]$ have the following structures:

$$\begin{array}{l}{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)={\left[{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\right]}^{†};\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)={\left[\delta {\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right);\delta {\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}^{†};\\ {\mathit{q}}^{\left(3\right)}\left[{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\triangleq \left(\begin{array}{c}{\mathit{q}}^{\left(2\right)}\left({\mathit{u}}^{\left(2\right)};\mathit{\alpha };\delta \mathit{\alpha }\right)\\ p^{\left(2\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\end{array}\right)\\ \triangleq {\left[{\mathit{q}}_1^{\left(3\right)},{\mathit{q}}_2^{\left(3\right)},{\mathit{q}}_3^{\left(3\right)},{\mathit{q}}_4^{\left(3\right)},{\mathit{q}}_5^{\left(3\right)}\left(j1\right),{\mathit{q}}_6^{\left(3\right)}\left(j1\right),{\mathit{q}}_7^{\left(3\right)}\left(j1\right),{\mathit{q}}_8^{\left(3\right)}\left(j1\right)\right]}^{†};\end{array}$$

(121)

$$\begin{array}{l}{\mathit{V}}^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\right]\triangleq \left(\begin{array}{cc}{\mathit{V}}^{\left(2\right)}\left({\mathit{u}}^{\left(1\right)}\right)& {\left[0\right]}_{\left(4\times 4\right)}\\ {\mathit{V}}_{21}^{\left(3\right)}\left(j1;{\mathit{u}}^{\left(2\right)}\right)& {\mathit{V}}_{22}^{\left(3\right)}\left({\mathit{u}}^{\left(1\right)}\right)\end{array}\right);\\ {\mathit{V}}_{22}^{\left(3\right)}\left({\mathit{u}}^{\left(1\right)}\right)\triangleq \left(\begin{array}{cccc}{\mathit{L}}^{*}\left(\mathit{\alpha }\right)& 0& -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }}& -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\phi }}\\ 0& \mathit{L}\left(\mathit{\alpha }\right)& -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }}& -\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }}\\ 0& 0& \mathit{L}\left(\mathit{\alpha }\right)& 0\\ 0& 0& 0& {\mathit{L}}^{*}\left(\mathit{\alpha }\right)\end{array}\right);\end{array}$$

(122)

and where the components ${\mathit{v}}_{21}^{\left(3\right)}\left(i,j\right)$ of the matrix ${\mathit{V}}_{21}^{\left(3\right)}\left(j1\right)\triangleq {\left[{\mathit{v}}_{21}^{\left(3\right)}\left(i,j\right)\right]}_{\left(4\times 4\right)};i,j=1,\dots ,4$ are defined as follows:

$${\mathit{v}}_{21}^{\left(3\right)}\left(1,1\right)=-\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }\partial \mathit{\phi }}{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\phi }\partial \mathit{\phi }}{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }},$$

(123)

$${\mathit{v}}_{21}^{\left(3\right)}\left(1,2\right)=-\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }\partial \mathit{\psi }}{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }\partial \mathit{\phi }}{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }},$$

(124)

$${\mathit{v}}_{21}^{\left(3\right)}\left(1,3\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial {\mathit{a}}_1^{\left(1\right)}},{\mathit{v}}_{21}^{\left(3\right)}\left(1,4\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial {\mathit{a}}_2^{\left(1\right)}},$$

(125)

$${\mathit{v}}_{21}^{\left(3\right)}\left(2,1\right)=-\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }\partial \mathit{\psi }}{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }\partial \mathit{\psi }}{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }},$$

(126)

$${\mathit{v}}_{21}^{\left(3\right)}\left(2,2\right)=-\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }\partial \mathit{\psi }}{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }\partial \mathit{\psi }}{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }},$$

(127)

$${\mathit{v}}_{21}^{\left(3\right)}\left(2,3\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_1^{\left(1\right)}\partial \mathit{\psi }},{\mathit{v}}_{21}^{\left(3\right)}\left(2,4\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_2^{\left(1\right)}\partial \mathit{\psi }},$$

(128)

$${\mathit{v}}_{21}^{\left(3\right)}\left(3,1\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial {\mathit{a}}_1^{\left(1\right)}},{\mathit{v}}_{21}^{\left(3\right)}\left(3,2\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial {\mathit{a}}_1^{\left(1\right)}},$$

(129)

$${\mathit{v}}_{21}^{\left(3\right)}\left(3,3\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_1^{\left(1\right)}\partial {\mathit{a}}_1^{\left(1\right)}},{\mathit{v}}_{21}^{\left(3\right)}\left(3,4\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_2^{\left(1\right)}\partial {\mathit{a}}_1^{\left(1\right)}},$$

(130)

$${\mathit{v}}_{21}^{\left(3\right)}\left(4,1\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial {\mathit{a}}_2^{\left(1\right)}},{\mathit{v}}_{21}^{\left(3\right)}\left(4,2\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial {\mathit{a}}_2^{\left(1\right)}},$$

(131)

$${\mathit{v}}_{21}^{\left(3\right)}\left(4,3\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_1^{\left(1\right)}\partial {\mathit{a}}_2^{\left(1\right)}},{\mathit{v}}_{21}^{\left(3\right)}\left(4,4\right)=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_2^{\left(1\right)}\partial {\mathit{a}}_2^{\left(1\right)}}.$$

(132)

The matrix-operator equations represented by Equation (119) together with the boundary conditions represented by Equation (120) will be called the 3rd-Level Variational Sensitivity System (3rd-LVSS). The solution, $\delta {\mathit{u}}^{\left(3\right)}\left(\mathit{x}\right)$, of the 3rd-LVSS will be called the “3rd-level variational sensitivity function”.

In the particular case when the response depends only on the forward function $\mathit{\phi }\left(\mathit{x}\right)$, i.e., if the response has the functional dependence $R\left[\mathit{\phi }\left(\mathit{x}\right);\mathit{\alpha }\right]$, the quantities $\mathit{\psi }$, ${\mathit{a}}_2^{\left(1\right)}$, ${\mathit{a}}_2^{\left(2\right)}\left(j1;\mathit{x}\right)$, ${\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)$ and related quantities would not exist. Consequently, the vector $w^{\left(3\right)}\left(j1;\mathit{x}\right)$ would reduce to the following 4-component vector: $\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)={\left[\delta \mathit{\phi }\left(\mathit{x}\right),\delta {\mathit{a}}_1^{\left(1\right)}\left(\mathit{x}\right),\delta {\mathit{a}}_1^{\left(2\right)},\delta {\mathit{a}}_3^{\left(2\right)}\right]}^{†}$ and the 3rd-LVSS would reduce correspondingly to the following system of 4x4 matrix equations:

$$\left(\begin{array}{cccc}\mathit{L}\left(\mathit{\alpha }\right)& 0& 0& 0\\ -\frac{{\partial }^{2}S\left(\mathit{\phi };{\mathit{\alpha }}^0\right)}{\partial \mathit{\phi }\partial \mathit{\phi }}& {\mathit{L}}^{*}\left(\mathit{\alpha }\right)& 0& 0\\ F_{31}^{\left(3\right)}\left[j_1\right]& F_{32}^{\left(3\right)}\left[j_1\right]& {\mathit{L}}^{*}\left(\mathit{\alpha }\right)& -\frac{{\partial }^{2}S\left(\mathit{\phi };\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }}\\ F_{41}^{\left(3\right)}\left[j_1\right]& F_{42}^{\left(3\right)}\left[j_1\right]& 0& \mathit{L}\left(\mathit{\alpha }\right)\end{array}\right)\left(\begin{array}{c}\delta \mathit{\phi }\left(\mathit{x}\right)\\ \delta {\mathit{a}}_1^{\left(1\right)}\\ \delta {\mathit{a}}_1^{\left(2\right)}\left(j1;\mathit{x}\right)\\ \delta {\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)\end{array}\right)=\left(1\right)\left(1\right)=\left(\begin{array}{c}{\mathit{q}}_1^{\left(1\right)}\\ {\mathit{q}}_3^{\left(2\right)}\\ {\mathit{q}}_5^{\left(3\right)}\\ {\mathit{q}}_7^{\left(3\right)}\end{array}\right),$$

(133)

where

$$\begin{array}{l}{F}_{31}^{\left(3\right)}\left[j_1\right]=-\frac{{\partial }^{3}S\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }\partial \mathit{\phi }}{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;\mathit{\phi };\mathit{\psi };{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }};F_{32}^{\left(3\right)}\left[i_1\right]=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;\mathit{\phi };{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial {\mathit{a}}_1^{\left(1\right)}};\\ F_{41}^{\left(3\right)}\left[j_1\right]=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;\mathit{\phi };\mathit{\psi };{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial {\mathit{a}}_1^{\left(1\right)}};F_{42}^{\left(3\right)}\left[i_1\right]=-\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;\mathit{\phi };\mathit{\psi };{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial {\mathit{a}}_1^{\left(1\right)}\partial {\mathit{a}}_1^{\left(1\right)}};\\ {\mathit{q}}_1^{\left(1\right)}=\frac{\partial \left[{\mathit{q}}_{\mathit{\phi }}\left(\mathit{\alpha }\right)-\mathit{L}\left(\mathit{\alpha }\right)\mathit{\phi }\right]}{\partial \mathit{\alpha }};{\mathit{q}}_3^{\left(2\right)}=\frac{{\partial }^{2}S\left(\mathit{\phi };\mathit{\psi };{\mathit{\alpha }}^0\right)}{\partial \mathit{\alpha }\partial \mathit{\phi }}\delta \mathit{\alpha }-\frac{\partial \left[{\mathit{L}}^{*}\left({\mathit{\alpha }}^0\right){\mathit{a}}_1^{\left(1\right)}\right]}{\partial \mathit{\alpha }}\delta \mathit{\alpha };\\ {\mathit{q}}_5^{\left(3\right)}=-\frac{\partial \left[{\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_1^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}{\partial \mathit{\alpha }}\delta \mathit{\alpha }+\frac{{\partial }^2{S}^{\left(1\right)}\left(j1\right)}{\partial \mathit{\alpha }\partial \mathit{\phi }}\delta \mathit{\alpha }+\left[\frac{{\partial }^{3}S\left(\mathit{\phi };\mathit{\alpha }\right)}{\partial \mathit{\alpha }\partial \mathit{\phi }\partial \mathit{\phi }}\delta \mathit{\alpha }\right]{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right);\\ {\mathit{q}}_7^{\left(3\right)}=-\frac{\partial \left[\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}{\partial \mathit{\alpha }}\delta \mathit{\alpha }+\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;\mathit{\phi };{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\alpha }\partial {\mathit{a}}_1^{\left(1\right)}}\delta \mathit{\alpha }.\end{array}$$

A reduction of the 3rd-LVSS to a 4 × 4 matrix-equation, which would be similar to the reduction shown in Equation (133), would also occur for a response $R\left[\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ which depended only on the adjoint function $\mathit{\psi }\left(\mathit{x}\right)$.

Since the 3rd-LVSS equations depend on the parameter variations $\delta {\mathit{\alpha }}_i$, solving them is prohibitively expensive computationally for large-scale systems involving many parameters. The need for solving the 3rd-LVSS can be avoided by expressing the indirect-effect term ${\left\{\delta R^{\left(1\right)}\left(j2;j1\right)\right\}}_{ind}$ defined in Equation (109) in an alternative way, namely in terms of the solutions of a 3rd-Level Adjoint Sensitivity System (3rd-LASS), which will not involve any variations in the state functions. This 3rd-LASS is constructed by implementing the same sequence of logical steps as used to construct the 1st-LASS and the 2nd-LASS, namely:

1

Define a Hilbert space, denoted as ${\mathrm{H}}_3$, comprising vector-valued elements of the form ${\mathit{\eta }}^{\left(3\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{\eta }}_1^{\left(3\right)}\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_8^{\left(3\right)}\left(\mathit{x}\right)\right]}^{†}\in {\mathrm{H}}_3$, with ${\mathit{\eta }}_i^{\left(3\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{\eta }}_{i,1}^{\left(3\right)}\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_{i,j}^{\left(3\right)}\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_{i,N_{\mathit{\phi }}}^{\left(3\right)}\left(\mathit{x}\right)\right]}^{†},i=1,..,8.$ The inner product between two elements, ${\mathit{\eta }}^{\left(3\right)}\left(\mathit{x}\right)\in {\mathrm{H}}_3$ and ${\xi }^{\left(3\right)}\left(\mathit{x}\right)\in {\mathrm{H}}_3$, of this Hilbert space, will be denoted as ${\mathit{\eta }}^{\left(3\right)}\left(\mathit{x}\right),{\xi }^{\left(3\right)}{\left(\mathit{x}\right)}_3$ and is defined as follows:

$$\langle {\mathit{\eta }}^{\left(3\right)}\left(\mathit{x}\right),{\xi }^{\left(3\right)}\left(\mathit{x}\right){\rangle }_3\triangleq {\sum }_{i=1}^8\langle {\mathit{\eta }}_i^{\left(3\right)}\left(\mathit{x}\right),{\xi }_i^{\left(3\right)}\left(\mathit{x}\right){\rangle }_0.$$

(134)

2

In the Hilbert $H_3$, form the inner product of Equation (119) with a set of yet undefined vector-valued functions ${\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\dots ,{\mathit{a}}_8^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\right]}^{†}\in H_3$,$j1,j2=1,\dots ,N_{\mathit{\alpha }}$, to obtain the following relation:

$$\begin{array}{l}{\{\langle {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right),{\mathit{V}}^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\right]\delta {\mathit{u}}^{\left(3\right)}\left(\mathit{x}\right){\rangle }_3\}}_{{\mathit{\alpha }}^0}\\ ={\{\langle {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right),{\mathit{q}}^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]{\rangle }_3\}}_{{\mathit{\alpha }}^0}.\end{array}$$

(135)

3

Using the definition of the adjoint operator in the Hilbert space $H_3$, recast the left-side of Equation (135) as follows:

$$\begin{array}{l}{\left\{{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right),{\mathit{V}}^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\right]\delta {\mathit{u}}^{\left(3\right)}{\left(\mathit{x}\right)}_3\right\}}_{{\mathit{\alpha }}^0}\\ ={\left\{\delta {\mathit{u}}^{\left(3\right)}\left(\mathit{x}\right),A^{\left(3\right)}\left(j1\right){\mathit{a}}^{\left(3\right)}{\left(j2;j1;\mathit{x}\right)}_3\right\}}_{{\mathit{\alpha }}^0}\\ +{\left\{P^{\left(3\right)}\left[\delta {\mathit{u}}^{\left(3\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0},\end{array}$$

(136)

where ${\left\{P^{\left(3\right)}\left[\delta {\mathit{u}}^{\left(3\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}$ denotes the bilinear concomitant defined on the phase-space boundary $\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right)$ and where $A^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\right]\triangleq {\left[{\mathit{V}}^{\left(3\right)}\left(j1;{\mathit{u}}^{\left(2\right)}\right)\right]}^{*}$ is the operator formally adjoint to ${\mathit{V}}^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\right]$ and has therefore the following form:

$$A^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\right]=\left(\begin{array}{cc}{A}^{\left(2\right)}& {\left[{\mathit{V}}_{21}^{\left(3\right)}\left(j1;{\mathit{u}}^{\left(2\right)}\right)\right]}^{*}\\ {\left[0\right]}_{\left(4\times 4\right)}& {\left[{\mathit{V}}_{22}^{\left(3\right)}\right]}^{*}\end{array}\right).$$

(137)

4

The first term on right-side of Equation (136) is now required to represent the indirect-effect term ${\left\{\delta R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right),0};{\mathit{a}}^{\left(2\right),0}\left(j1\right);{\mathit{\alpha }}^0;\delta {\mathit{u}}^{\left(2\right)};\delta {\mathit{a}}^{\left(2\right)}\left(j1\right)\right]\right\}}_{ind}$ defined in Equation (109). This requirement is satisfied by imposing the following relation on each element ${\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\dots ,{\mathit{a}}_8^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\right]}^{†}\in H_3$, $j1,j2=1,\dots ,N_{\mathit{\alpha }}$:

$$\begin{array}{l}{\left\{A^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(j1;\mathit{x}\right)\right]{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\right\}}_{{\mathit{\alpha }}^0}={\left\{s^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0},\\ j1,j2=1,\dots ,N_{\mathit{\alpha }},\end{array}$$

(138)

where the block-vector $s^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]$ comprises, for each $j1,j2=1,\dots ,N_{\mathit{\alpha }}$, eight components (which are themselves block-vectors) defined as follows:

$$\begin{array}{l}{s}^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\\ \triangleq {\left[s_1^{\left(3\right)}\left(j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right),\dots ,s_8^{\left(3\right)}\left(j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right)\right]}^{†},\end{array}$$

(139)

where

$$s_1^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial \mathit{\phi }},$$

(140)

$$s_2^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial \mathit{\psi }},$$

(141)

$$s_3^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{a}}_1^{\left(1\right)}},$$

(142)

$$s_4^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{a}}_2^{\left(1\right)}\left(j1\right)},$$

(143)

$$s_5^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{a}}_1^{\left(2\right)}},$$

(144)

$$s_6^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{a}}_2^{\left(2\right)}\left(j1\right)},$$

(145)

$$s_7^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{a}}_3^{\left(2\right)}\left(j1\right)},$$

(146)

$$s_8^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\triangleq \frac{\partial S^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{a}}_4^{\left(2\right)}\left(j1\right)}.$$

(147)

5

The definition of the set of vectors ${\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\dots ,{\mathit{a}}_8^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\right]}^{†}\in H_3$ will now be completed by selecting boundary conditions for this set of vectors, which will be represented in operator form as follows:

$${\left\{{\mathit{b}}_A^{\left(3\right)}\left[{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}=0,\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right),j1,j2=1,\dots ,N_{\mathit{\alpha }}.$$

(148)

The boundary conditions represented by Equation (148) are selected so as to satisfy the following requirements:

(i)

The boundary conditions Equation (148) together with the operator Equation (138) constitute a well posed problem for the functions ${\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$.

(ii)

Implementation in Equation (136) of the boundary conditions provided in Equation (120) together with those provided in Equation (148) eliminates all of the unknown values of the functions $\delta {\mathit{u}}^{\left(3\right)}\left(\mathit{x}\right)$ and ${\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$ in the expression of the bilinear concomitant ${\left\{P^{\left(3\right)}\left[\delta {\mathit{u}}^{\left(3\right)}\left(\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{\left({\mathit{\alpha }}^0\right)}$. This bilinear concomitant may vanish after implementing the boundary conditions represented by Equation (148), but if it does not, it will be reduced to a residual quantity which will be denoted as ${\widehat{P}}^{\left(3\right)}\left[{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]$ and which will comprise only known values of ${\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$, ${\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)$, $\mathit{\alpha }$ and $\delta \mathit{\alpha }$.

The system of equations represented by Equation (138) together with the boundary conditions represented by Equation (148) constitute the 3rd-Level Adjoint Sensitivity System (3rd-LASS). The solution ${\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\dots ,{\mathit{a}}_8^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\right]}^{†}\in H_3$, $j1,j2=1,\dots ,N_{\mathit{\alpha }}$, of the 3rd-LASS will be called the 3rd-level adjoint function.

The 3rd-LASS together with the results provided in Equations (135) and (136) are employed in Equation (109) to obtain the following expression for the indirect-effect term ${\left\{\delta R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(2\right)};\delta {\mathit{a}}^{\left(2\right)}\left(j1\right)\right]\right\}}_{ind}$ in terms of the 3rd-level adjoint functions ${\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$, for $j1,j2=1,\dots ,N_{\mathit{\alpha }}$:

$$\begin{array}{l}{\left\{\delta R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)\right]\right\}}_{ind}\\ ={\{\langle {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right),{\mathit{q}}^{\left(3\right)}\left[{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]{\langle }_3\}}_{{\mathit{\alpha }}^0}\\ -{\left\{{\widehat{P}}^{\left(3\right)}\left[{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\\ \equiv {\left\{\delta R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{ind}.\end{array}$$

(149)

As the identity in the last line of Equation (149) indicates, the dependence of the indirect-effect term on the variations $\delta {\mathit{u}}^{\left(2\right)}$ and $\delta {\mathit{a}}^{\left(2\right)}\left(j1\right)$ in the state functions has been eliminated, by having replaced this dependence by the dependence on the 3rd-level adjoint function ${\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$.

Inserting the expressions of the components of the vector ${\mathit{q}}^{\left(3\right)}\left[{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)\mathit{\alpha };\delta \mathit{\alpha }\right]$, cf. Equation (122), into Equation (149) and adding the resulting expression for the indirect-effect term ${\left\{\delta R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(2\right)};\delta {\mathit{a}}^{\left(2\right)}\left(j1\right)\right]\right\}}_{ind}$ to the expression of the direct-effect term given in Equation (108) yields the following expression for the total third-order G-differential of the response $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$:

$$\begin{array}{l}{\left\{\delta R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\\ ={\left\{\frac{\partial }{\partial \mathit{\alpha }}{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k{S}^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\alpha }\\ +{\{\langle {\mathit{a}}_1^{\left(3\right)}\left(j2;j1;\mathit{x}\right),{\mathit{q}}_1^{\left(1\right)}\left(\mathit{\phi };\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_0\}}_{{\mathit{\alpha }}^0}+{\{\langle {\mathit{a}}_2^{\left(3\right)}\left(j2;j1;\mathit{x}\right),{\mathit{q}}_2^{\left(1\right)}\left(\mathit{\psi };\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +{\{\langle {\mathit{a}}_3^{\left(3\right)}\left(j2;j1;\mathit{x}\right),{\mathit{q}}_3^{\left(2\right)}\left({\mathit{u}}^{\left(2\right)};\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +{\{\langle {\mathit{a}}_4^{\left(3\right)}\left(j2;j1;\mathit{x}\right),{\mathit{q}}_4^{\left(2\right)}\left({\mathit{u}}^{\left(2\right)};\mathit{\alpha };\delta \mathit{\alpha }\right){\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +{\{\langle {\mathit{a}}_5^{\left(3\right)}\left(j2;j1;\mathit{x}\right),{\mathit{q}}_5^{\left(3\right)}\left[{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +{\{\langle {\mathit{a}}_6^{\left(3\right)}\left(j2;j1;\mathit{x}\right),{\mathit{q}}_6^{\left(3\right)}\left[{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +{\{\langle {\mathit{a}}_7^{\left(3\right)}\left(j2;j1;\mathit{x}\right),{\mathit{q}}_7^{\left(3\right)}\left[{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +{\{\langle {\mathit{a}}_8^{\left(3\right)}\left(j2;j1;\mathit{x}\right),{\mathit{q}}_8^{\left(3\right)}\left[{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ -{\left\{{\widehat{P}}^{\left(3\right)}\left[{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0},forj1,j2=1,\dots ,N_{\mathit{\alpha }}.\end{array}$$

(150)

Inserting the explicit expressions for the components of ${\mathit{q}}^{\left(3\right)}\left[{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]$ into Equation (150) leads to the following expression of the total third-order G-differential ${\left\{\delta R^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}$ in terms of the third-order partial sensitivities (G-derivatives) of the response with respect to the model parameters:

$$\begin{gathered} {\left\{\delta R^{\left(2\right)}\left[{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0} \\ ={\sum }_{j3=1}^{N_{\mathit{\alpha }}}{\left\{R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\delta {\mathit{\alpha }}_{j3}, \end{gathered}$$

(151)

where $R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]$ denotes the 3rd-order partial sensitivity of the response with respect to the model parameters, evaluated at the nominal parameter values ${\mathit{\alpha }}^0$. The explicit expression of $R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]$ is obtained by using Equation (150) in Equation (151) and subsequently identifying the expression that multiplies the parameter $\delta {\mathit{\alpha }}_{j3}$. This process indicates that the 3rd-order partial sensitivity $R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]$ has the following explicit expression, for each $j1,j2,j3=1,\dots ,N_{\mathit{\alpha }}$:

$$\begin{array}{l}{R}^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]\equiv \frac{{\partial }^{3}R\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}\\ =-{\left\{\frac{\partial }{\partial {\mathit{\alpha }}_{j3}}{\widehat{P}}^{\left(3\right)}\left[{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\\ +\frac{\partial }{\partial {\mathit{\alpha }}_{j3}}{\left\{{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k{S}^{\left(2\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\\ +\{\langle {\mathit{a}}_1^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\frac{\partial \left[{\mathit{q}}_{\mathit{\phi }}\left(\mathit{\alpha }\right)-\mathit{L}\left(\mathit{\alpha }\right)\mathit{\phi }\left(\mathit{x}\right)\right]}{\partial {\mathit{\alpha }}_{j3}}{\rangle }_0\\ +\langle {{\mathit{a}}_2^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\frac{\partial \left[{\mathit{q}}_{\mathit{\psi }}\left(\mathit{\alpha }\right)-{\mathit{L}}^{*}\left(\mathit{\alpha }\right)\mathit{\psi }\left(\mathit{x}\right)\right]}{\partial {\mathit{\alpha }}_{j3}}{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +\{\langle {\mathit{a}}_3^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}\partial \mathit{\phi }}\mathit{\alpha }-\frac{\partial \left[{\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_1^{\left(1\right)}\right]}{\partial {\mathit{\alpha }}_{j3}}{\rangle }_0\\ +\langle {{\mathit{a}}_4^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}\partial \mathit{\psi }}-\frac{\partial \left[\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_2^{\left(1\right)}\right]}{\partial {\mathit{\alpha }}_{j3}}{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +\{\langle {\mathit{a}}_5^{\left(3\right)}\left(j2;j1;\mathit{x}\right),-\frac{\partial \left[{\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_1^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}{\partial {\mathit{\alpha }}_{j3}}+\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}\partial \mathit{\phi }\partial \mathit{\phi }}{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)\\ {+\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}\partial \mathit{\psi }\partial \mathit{\phi }}{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)+\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}\partial \mathit{\phi }}{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +\{\langle {\mathit{a}}_6^{\left(3\right)}\left(j2;j1;\mathit{x}\right),-\frac{\partial \left[\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_2^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}{\partial {\mathit{\alpha }}_{j3}}+\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}\partial \mathit{\phi }\partial \mathit{\psi }}{\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)\\ {+\frac{{\partial }^{3}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}\partial \mathit{\psi }\partial \mathit{\psi }}{\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)+\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};{\mathit{a}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}\partial \mathit{\psi }}{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +{\{\langle {\mathit{a}}_7^{\left(3\right)}\left(j2;j1;\mathit{x}\right),-\frac{\partial \left[\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_3^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}{\partial {\mathit{\alpha }}_{j3}}+\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}\partial {\mathit{a}}_1^{\left(1\right)}}{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ +{\{\langle {\mathit{a}}_8^{\left(3\right)}\left(j2;j1;\mathit{x}\right),-\frac{\partial \left[{\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_4^{\left(2\right)}\left(j1;\mathit{x}\right)\right]}{\partial {\mathit{\alpha }}_{j3}}+\frac{{\partial }^2{S}^{\left(1\right)}\left(j1;{\mathit{u}}^{\left(2\right)};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j3}\partial {\mathit{a}}_2^{\left(1\right)}}{\rangle }_0\}}_{{\mathit{\alpha }}^0}\\ \triangleq {\left\{{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k{S}^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}.\end{array}$$

(152)

The expression in Equation (152) can be computed after having obtained the 3rd-level adjoint function ${\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\dots ,{\mathit{a}}_8^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\right]}^{†}$. Since the 3rd-LASS is independent of parameter variations $\delta \mathit{\alpha }$, the exact computation of all of the partial third-order sensitivities $R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]$ requires at most $O\left(N_{\mathit{\alpha }}^2\right)$ large-scale (adjoint) computations using the 3rd-LASS, rather than $O\left(N_{\mathit{\alpha }}^3\right)$ large-scale computations as would be required by forward methods. In component form, the equations comprising the 3rd-LASS are as solved (for each $j1=1,\dots ,N_{\mathit{\alpha }};j2=1,\dots ,j1$) at the nominal values for the model parameters and state functions in the following order:

$$\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_5^{\left(3\right)}\left(j2;j1;\mathit{x}\right)=s_5^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right],$$

(153)

$${\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_6^{\left(3\right)}\left(j2;j1;\mathit{x}\right)=s_6^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right],$$

(154)

$$\begin{array}{l}{\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_7^{\left(3\right)}\left(j2;j1;\mathit{x}\right)=s_7^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]+\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }}{\mathit{a}}_5^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\\ +\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }}{\mathit{a}}_6^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\end{array}$$

(155)

$$\begin{array}{l}\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_8^{\left(3\right)}\left(j2;j1;\mathit{x}\right)=s_8^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]+\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\phi }}{\mathit{a}}_5^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\\ +\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }}{\mathit{a}}_6^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\end{array}$$

(156)

$$\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_3^{\left(3\right)}\left(j2;j1;\mathit{x}\right)=s_3^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]+{\displaystyle \sum }_{k=1}^4{\mathit{v}}_{21}^{\left(3\right)}\left(k,3\right){\mathit{a}}_{k+4}^{\left(3\right)}\left(j2;j1;\mathit{x}\right),$$

(157)

$${\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_4^{\left(3\right)}\left(j2;j1;\mathit{x}\right)=s_4^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]+{\displaystyle \sum }_{k=1}^4{\mathit{v}}_{21}^{\left(3\right)}\left(k,4\right){\mathit{a}}_{k+4}^{\left(3\right)}\left(j2;j1;\mathit{x}\right),$$

(158)

$$\begin{array}{c}{\mathit{L}}^{*}\left(\mathit{\alpha }\right){\mathit{a}}_1^{\left(3\right)}\left(j2;j1;\mathit{x}\right)=s_1^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]+\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\phi }}{\mathit{a}}_3^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\\ +\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\phi }}{\mathit{a}}_4^{\left(3\right)}\left(j2;j1;\mathit{x}\right)+{\displaystyle {\displaystyle \sum }_{k=1}^4}{\mathit{v}}_{21}^{\left(3\right)}\left(k,1\right){\mathit{a}}_{k+4}^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\end{array}$$

(159)

$$\begin{array}{c}\mathit{L}\left(\mathit{\alpha }\right){\mathit{a}}_2^{\left(3\right)}\left(j2;j1;\mathit{x}\right)=s_2^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]+\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\phi }\partial \mathit{\psi }}{\mathit{a}}_3^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\\ +\frac{{\partial }^{2}S\left({\mathit{u}}^{\left(1\right)};\mathit{\alpha }\right)}{\partial \mathit{\psi }\partial \mathit{\psi }}{\mathit{a}}_4^{\left(3\right)}\left(j2;j1;\mathit{x}\right)+{\displaystyle {\displaystyle \sum }_{k=1}^4}{\mathit{v}}_{21}^{\left(3\right)}\left(k,2\right){\mathit{a}}_{k+4}^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\end{array}$$

(160)

The equations underlying the 3rd-LASS are solved successively, starting with Equations (153) and (154) to compute the 3rd-level adjoint functions ${\mathit{a}}_5^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$ and ${\mathit{a}}_6^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$. Note that solving using Equations (153) and (154) would be performed by using the same forward and adjoint solvers (i.e., computer codes) as used for solving the original forward and adjoint systems, namely Equations (3) and (11), respectively. Only the corresponding source terms and boundary conditions would differ. After having obtained the 3rd-level adjoint functions ${\mathit{a}}_5^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$ and ${\mathit{a}}_6^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$, the next round of computations would be to solve Equations (155) and (156) in order to determine the 3rd-level adjoint functions ${\mathit{a}}_7^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$ and ${\mathit{a}}_8^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$, respectively. Solving Equations (155) and (156) would also be performed by using the same forward and adjoint solvers (i.e., computer codes) as used for solving the original forward and adjoint systems. The next round of computations would employ the forward and adjoint solvers for solving Equations (157) and (158) in order to determine the 3rd-level adjoint functions ${\mathit{a}}_3^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$ and ${\mathit{a}}_4^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$, respectively. The final set of computations would require the use of the forward and adjoint solvers to solve Equations (159) and (160) in order to determine the 3rd-level adjoint functions ${\mathit{a}}_1^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$ and ${\mathit{a}}_2^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$, respectively. Thus, solving the 3rd-LASS in order to determine the 3rd-level adjoint functions does not require any significant “code development,” since the original forward and adjoint solvers (codes) do not need to be modified; only the right-sides (i.e., “sources”) and boundary conditions would need to be programmed accordingly. Using the 3rd-LASS enables the specific computation of the 3rd-order sensitivities in the priority order set by the user, so that only the important 3rd-order partial sensitivities $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ would be computed. Information provided by the first- and second-order sensitivities is very likely to indicate which 3rd-order sensitivities could be neglected.

3.3.4. Comparison of Computational Requirements for Computing the Third-Order Response Sensitivities with Respect to Model Parameters

The largest third-order sensitivities of the leakage response of the OECD/NEA reactor physics benchmark measured in [16] have been computed in [19,20,21], where it was shown that these large sensitivities (which reached relative values of 10⁴) involved the benchmark’s 180 imprecisely known microscopic total cross sections. Using a DELL computer (AMD FX-8350) with an 8-core processor, the CPU-time for a typical adjoint computation (which represents a large-scale computation) of one (of the 8) of the adjoint functions needed for computing the 3rd-order response sensitivities to the benchmark’s total microscopic cross sections is ca. 20 s, while the CPU-time computing an integral (which represents a small-scale computation) over the adjoint function(s) which appear(s) in the formula for computing a 3rd-order sensitivity is ca. 0.003 s. By applying the 3rd-CASAM, the computation of the 3rd-order sensitivities of the benchmark’s leakage response to the 180 total microscopic cross sections required 33,301 large-scale adjoint computations, which required 175 CPU-hours. By comparison, computing these 3rd-order sensitivities using the FSAM would have required 1,004,730 large-scale forward computations (which would have required 12,559 CPU-hours). Using finite-differences would have required 7,905,360 large-scale forward computations (which would have required 98,817 CPU-hours) to obtain, at best, approximate values for these 3rd-order sensitivities. Evidently, the 3rd-CASAM is the only practical methodology capable of computing, without introducing methodological errors, 3rd-order sensitivities for large-scale systems involving many parameters.

3.4. Computation of the Fourth-Order Sensitivities of $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$

There are $N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)\left(N_{\mathit{\alpha }}+2\right)\left(N_{\mathit{\alpha }}+3\right)/4!$ distinct 4th-order sensitivities of the response with respect to the model and response parameters. It is not practicable to compute these sensitivities using finite-difference formulas or the forward sensitivity analysis methodology. The finite-difference formulas for computing these sensitivities are presented in Section 3.4.1. The Forward Sensitivity Analysis Methodology (FSAM) for computing the 3rd-order response sensitivities are presented in Section 3.4.2, while the 4th-Order Comprehensive Adjoint Sensitivity Analysis Methodology (4th-CASAM) is presented in Section 3.4.3.

3.4.1. Finite-Difference Approximation Using Re-Computations with User-Modified Parameters

The 4th-order unmixed sensitivities, ${\partial }^{4}R\left(\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j1}^4$, of the leakage response, $R\left(\mathit{\alpha }\right)$, with respect to all the model parameters, can be approximately computed by re-computations using the well-known finite-difference formula presented below:

$$\begin{array}{c}\frac{{\partial }^{4}R\left(\mathit{\alpha }\right)}{{\left(\partial {\mathit{\alpha }}_{j1}\right)}^4}\approx \frac{1}{{\mathit{h}}_{j1}{}^4}\left(R_{j1+2}-4R_{j1+1}+6R_{j1}-4R_{j1-1}+R_{j1-2}\right)+O\left({\mathit{h}}_{j1}{}^2\right),\\ j1=1,\dots ,N_{\mathit{\alpha }},\end{array}$$

(161)

where $R_{j1+2}\triangleq R\left({\mathit{\alpha }}_{j1}+2{\mathit{h}}_{j1}\right)$, $R_{j1+1}\triangleq R\left({\mathit{\alpha }}_{j1}+{\mathit{h}}_{j1}\right)$, $R_{j1-1}\triangleq R\left({\mathit{\alpha }}_{j1}-{\mathit{h}}_{j1}\right)$, $R_{j1-2}\triangleq R\left({\mathit{\alpha }}_{j1}-2{\mathit{h}}_{j1}\right)$ and ${\mathit{h}}_{j1}$ denotes a “judiciously-chosen” variation in the parameter ${\mathit{\alpha }}_{j1}$ around its nominal value ${\mathit{\alpha }}_{j1}^0$. The 4th-order mixed sensitivities, ${\partial }^{4}R\left(\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j4}$, can be calculated by using the following finite-difference formula:

$$\begin{array}{l}\frac{{\partial }^{4}R\left(\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j4}}\approx \frac{1}{16{\mathit{h}}_{j1}{\mathit{h}}_{j2}{\mathit{h}}_{j3}{\mathit{h}}_{j4}}(R_{j1+1,j2+1,j3+1,j4+1}-R_{j1+1,j2+1,j3+1,j4-1}\\ -R_{j1+1,j2+1,j3-1,j4+1}+R_{j1+1,j2+1,j3-1,j4-1}-R_{j1+1,j2-1,j3+1,j4+1}\\ +R_{j1+1,j2-1,j3+1,j4-1}+R_{j1+1,j2-1,j3-1,j4+1}-R_{j1+1,j2-1,j3-1,j4-1}\\ -R_{j1-1,j2+1,j3+1,j4+1}+R_{j1-1,j2+1,j3+1,j4-1}+R_{j1-1,j2+1,j3-1,j4+1}\\ -R_{j1-1,j2+1,j3-1,j4-1}+R_{j1-1,j2-1,j3+1,j4+1}-R_{j1-1,j2-1,j3+1,j4-1}\\ -R_{j1-1,j2-1,j3-1,j4+1}+R_{j1-1,j2-1,j3-1,j4-1})+O\left({\mathit{h}}_{j1}{}^2,{\mathit{h}}_{j2}{}^2,{\mathit{h}}_{j3}{}^2,{\mathit{h}}_{j4}{}^2\right),\end{array}$$

(162)

where $R_{j1+1,j2+1,j3+1,j4+1}\triangleq R\left({\mathit{\alpha }}_{j1}+{\mathit{h}}_{j1},{\mathit{\alpha }}_{j2}+{\mathit{h}}_{j2},{\mathit{\alpha }}_{j3}+{\mathit{h}}_{j3},{\mathit{\alpha }}_{j4}+{\mathit{h}}_{j4}\right)$, etc. The finite difference formulas introduce their intrinsic “methodological errors” of order $\mathsf{O}\left({\mathit{h}}_{j1}{}^2,{\mathit{h}}_{j2}{}^2,{\mathit{h}}_{j3}{}^2,{\mathit{h}}_{j4}{}^2\right)$ which are in addition to, and independent of, the errors that might be incurred in the computation of ${\partial }^{4}R\left(\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j4}$.

3.4.2. Forward Sensitivity Analysis Methodology

Within the framework of the FSAM, the fourth-order sensitivities ${\partial }^{4}R\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j4}$; $j1,j2,j3;j4=1,\dots ,N_{\mathit{\alpha }}$, of the response to the model parameters are obtained by computing the partial G-derivatives of the third-order sensitivities represented by Equation (98), which yields the following result:

$$\begin{array}{l}\frac{{\partial }^{4}R\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j4}}\triangleq \frac{1}{\delta {\mathit{\alpha }}_{j4}}\{\frac{d}{d\epsilon }{R}^{\left(3\right)}[j3;j2;j1;\mathit{\phi }\left({\mathit{\alpha }}_{j4}^0+\epsilon \delta {\mathit{\alpha }}_{j4}\right);\mathit{\psi }\left({\mathit{\alpha }}_{j4}^0+\epsilon \delta {\mathit{\alpha }}_{j4}\right);\\ {\mathit{\alpha }}_{j4}^0+\epsilon \delta {\mathit{\alpha }}_{j4};\frac{\partial \mathit{\phi }\left({\mathit{\alpha }}_{j4}^0+\epsilon \delta {\mathit{\alpha }}_{j4}\right)}{\partial {\mathit{\alpha }}_{j1}};\frac{\partial \mathit{\psi }\left({\mathit{\alpha }}_{j4}^0+\epsilon \delta {\mathit{\alpha }}_{j4}\right)}{\partial {\mathit{\alpha }}_{j1}};\frac{{\partial }^2\mathit{\phi }\left({\mathit{\alpha }}_{j4}^0+\epsilon \delta {\mathit{\alpha }}_{j4}\right)}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}};\\ {\frac{{\partial }^2\mathit{\psi }\left({\mathit{\alpha }}_{j4}^0+\epsilon \delta {\mathit{\alpha }}_{j4}\right)}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}};\frac{{\partial }^3\mathit{\phi }\left({\mathit{\alpha }}_{j4}^0+\epsilon \delta {\mathit{\alpha }}_{j4}\right)}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}};\frac{{\partial }^3\mathit{\psi }\left({\mathit{\alpha }}_{j4}^0+\epsilon \delta {\mathit{\alpha }}_{j4}\right)}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}}]\}}_{\epsilon =0}\\ =R^{\left(4\right)}[j4;j3j2;j1;\mathit{\phi };\mathit{\psi };\mathit{\alpha };\frac{\partial \mathit{\phi }}{\partial {\mathit{\alpha }}_{j2}};\frac{\partial \mathit{\psi }}{\partial {\mathit{\alpha }}_{j2}};\frac{{\partial }^2\mathit{\phi }}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}};\frac{{\partial }^2\mathit{\psi }}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}};\\ \frac{{\partial }^3\mathit{\phi }}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}};\frac{{\partial }^3\mathit{\psi }}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}};\frac{{\partial }^4\mathit{\phi }}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j4}};\frac{{\partial }^4\mathit{\psi }}{\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j4}}].\end{array}$$

(163)

As indicated by the functional dependence of the last term in Equation (163), the evaluation of the 4th-order sensitivities requires prior knowledge of the 1st-, 2nd-, and 3rd-order partial derivatives of the state functions with respect to the model parameters. The functions $\partial \mathit{\phi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_{j1}$ and $\partial \mathit{\psi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_{j1}$ are obtained by solving the 1st-OFSS. The functions ${\partial }^2\mathit{\phi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$ and ${\partial }^2\mathit{\psi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$ are the solutions of the 2nd-OFSS. The functions ${\partial }^3\mathit{\phi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}$ and ${\partial }^3\mathit{\psi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}$ are the solutions of the 3rd-OFSS. The functions ${\partial }^4\mathit{\phi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j4}$ and ${\partial }^4\mathit{\psi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j4}$ are the solutions of the following Fourth-Order Forward Sensitivity System (4th-OFSS), which is obtained by taking the partial G-derivative of the 3rd-OFSS with respect to a generic model parameter ${\mathit{\alpha }}_{j4}$:

$$\begin{array}{c}\mathit{L}\left(\mathit{x};{\mathit{\alpha }}^0\right)\frac{{\partial }^4\mathit{\phi }}{\partial {\mathit{\alpha }}_{j4}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}={\left\{f^{\left(4\right)}\left(j4;j3;j2;j1;\mathit{\phi };\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0};\\ j1,\dots ,j4=1,\dots ,N_{\mathit{\alpha }};\mathit{x}\in {\Omega }_{\mathit{x}};\end{array}$$

(164)

$$\begin{array}{c}{\left\{{\mathit{B}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0}\frac{{\partial }^4\mathit{\phi }}{\partial {\mathit{\alpha }}_{j4}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}={\left\{c_{\mathit{\phi }}^{\left(4\right)}\left(j4;j3;j2;j1;\mathit{\phi };\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0};\\ j1,\dots ,j4=1,\dots ,N_{\mathit{\alpha }};\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right);\end{array}$$

(165)

$$\begin{array}{c}{\mathit{L}}^{*}\left(\mathit{x};{\mathit{\alpha }}^0\right)\frac{{\partial }^4\mathit{\psi }}{\partial {\mathit{\alpha }}_{j4}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}={\left\{g^{\left(4\right)}\left(j4;j3;j2;j1;\mathit{\psi };\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0};\\ j1,\dots ,j4=1,\dots ,N_{\mathit{\alpha }};\mathit{x}\in {\Omega }_{\mathit{x}};\end{array}$$

(166)

$$\begin{array}{c}{\left\{{\mathit{B}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0}\frac{{\partial }^4\mathit{\psi }}{\partial {\mathit{\alpha }}_{j4}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}={\left\{c_{\mathit{\psi }}^{\left(4\right)}\left(j4;j3;j2;j1;\mathit{\psi };\mathit{\alpha }\right)\right\}}_{{\mathit{\alpha }}^0};\\ j1,\dots ,j4=1,\dots ,N_{\mathit{\alpha }};\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right);\end{array}$$

(167)

where

$$f^{\left(4\right)}\left(j4;j3;j2;j1;\mathit{\phi };\mathit{\alpha }\right)\triangleq \frac{\partial f^{\left(3\right)}\left(j3;j2;j1;\mathit{\phi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j4}}-\frac{\partial \mathit{L}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j4}}\frac{{\partial }^3\mathit{\phi }}{\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}};$$

(168)

$$g^{\left(4\right)}\left(j4;j3;j2;j1;\mathit{\psi };\mathit{\alpha }\right)\triangleq \frac{\partial g^{\left(3\right)}\left(j3;j2;j1;\mathit{\psi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j4}}-\frac{\partial {\mathit{L}}^{*}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j4}}\frac{{\partial }^3\mathit{\psi }}{\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}};$$

(169)

$$\begin{array}{c}{c}_{\mathit{\phi }}^{\left(4\right)}\left(j4;j3;j2;j1;\mathit{\phi };\mathit{\alpha }\right)\triangleq \frac{\partial c_{\mathit{\phi }}^{\left(3\right)}\left(j3;j2;j1;\mathit{\phi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_4}-\frac{\partial {\mathit{B}}_{\mathit{\phi }}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_4}\frac{{\partial }^3\mathit{\phi }}{\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}};\\ \mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right);\end{array}$$

(170)

$$\begin{array}{c}{c}_{\mathit{\psi }}^{\left(4\right)}\left(j4;j3;j2;j1;\mathit{\psi };\mathit{\alpha }\right)\triangleq \frac{\partial c_{\mathit{\psi }}^{\left(3\right)}\left(j3;j2;j1;\mathit{\psi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j4}}-\frac{\partial {\mathit{B}}_{\mathit{\psi }}\left(\mathit{x};\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j4}}\frac{{\partial }^3\mathit{\psi }}{\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}};\\ \mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right).\end{array}$$

(171)

Evidently, the computations of the fourth-order sensitivities ${\partial }^{4}R\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j4}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}$ require $2N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)\left(N_{\mathit{\alpha }}+2\right)\left(N_{\mathit{\alpha }}+3\right)/4!$ large-scale computations to solve Equations (164)–(167), followed by just as many small-scale computations for performing the integrations that define the various sensitivities. These computations would be in addition to the $2N_{\mathit{\alpha }}$ large-scale computations needed to determine the functions $\partial \mathit{\phi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_{j1}$ and $\partial \mathit{\psi }\left(\mathit{x}\right)/\partial {\mathit{\alpha }}_{j1}$ by solving the 1st-OFSS, the $N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)$ large-scale computations needed to determine the functions ${\partial }^2\mathit{\phi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$ and ${\partial }^2\mathit{\psi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}$ by solving the 2nd-OFSS, and the $N_{\mathit{\alpha }}\left(N_{\mathit{\alpha }}+1\right)\left(N_{\mathit{\alpha }}+2\right)/3$ large-scale computations needed to determine the functions ${\partial }^3\mathit{\phi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}$ and ${\partial }^3\mathit{\psi }/\partial {\mathit{\alpha }}_{j1}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j3}$ by solving the 3rd-OFSS.

3.4.3. The Fourth-Order Adjoint Sensitivity Analysis Methodology (4th-CASAM)

The fourth-order sensitivities of the response $R\left[\mathit{\phi }\left(\mathit{x}\right),\mathit{\psi }\left(\mathit{x}\right);\mathit{\alpha }\right]$ with respect to the model parameters are obtained by determining the first-order G-differential of the 3rd-order sensitivities $R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]\equiv {\partial }^{3}R\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)/\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}$, which were computed in Section 3.3 and defined in Equation (152). By definition, the total G-differential of $R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]$ is obtained as follows:

$$\begin{array}{c}{\left\{\delta R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(3\right)};\delta {\mathit{a}}^{\left(3\right)};\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\\ \triangleq \{\frac{d}{d\epsilon }{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k{S}^{\left(3\right)}[j3;j2;j1;{\mathit{u}}^{\left(3\right),0}\left(j1;\mathit{x}\right)+\epsilon \delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\\ {{\mathit{a}}^{\left(3\right),0}\left(j2;j1;\mathit{x}\right)+\epsilon \delta {\mathit{a}}^{\left(3\right)}\left(j2;j1\right);{\mathit{\alpha }}^0+\epsilon \delta \mathit{\alpha }]\}}_{\epsilon =0}\\ ={\left\{\delta R^{\left(3\right)}\left(j3;j2;j1;{\mathit{u}}^{\left(3\right)};{\mathit{a}}^{\left(3\right)};\delta \mathit{\alpha }\right)\right\}}_{dir}\\ +{\left\{\delta R^{\left(3\right)}\left(j3;j2;j1;{\mathit{u}}^{\left(3\right)};{\mathit{a}}^{\left(3\right)};\delta {\mathit{u}}^{\left(3\right)};\delta {\mathit{a}}^{\left(3\right)}\right)\right\}}_{ind},j1,j2;j3=1,\dots ,N_{\mathit{\alpha }},\end{array}$$

(172)

where the direct-effect term ${\left\{\delta R^{\left(3\right)}\left(j3;j2;j1;{\mathit{u}}^{\left(3\right)};{\mathit{a}}^{\left(3\right)};\delta \mathit{\alpha }\right)\right\}}_{dir}$ depends directly on the parameter variations and is defined as follows:

$$\begin{array}{c}{\left\{\delta R^{\left(3\right)}\left(j3;j2;j1;{\mathit{u}}^{\left(3\right)};{\mathit{a}}^{\left(3\right)};\delta \mathit{\alpha }\right)\right\}}_{dir}\\ \triangleq {\left\{\frac{\partial }{\partial \mathit{\alpha }}{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k{S}^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\delta \mathit{\alpha },\end{array}$$

(173)

while the indirect-effect term ${\left\{\delta R^{\left(3\right)}\left(j3;j2;j1;{\mathit{u}}^{\left(3\right)};{\mathit{a}}^{\left(3\right)};\delta {\mathit{u}}^{\left(3\right)};\delta {\mathit{a}}^{\left(3\right)}\right)\right\}}_{ind}$ depends indirectly on the parameter variations through the variations in the forward and adjoint state functions and is defined as follows:

$$\begin{array}{c}{\left\{\delta R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\delta {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\right]\right\}}_{ind}\\ \triangleq {\left\{{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)}\right\}}_{{\mathit{\alpha }}^0}\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)\\ +{\left\{{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k\frac{\partial S^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)}\right\}}_{{\mathit{\alpha }}^0}\delta {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right),\end{array}$$

(174)

where

$$\begin{array}{c}\frac{\partial [ ]}{\partial {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)}\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)\equiv \frac{\partial [ ]}{\partial \mathit{\phi }}\delta \mathit{\phi }\left(\mathit{x}\right)+\frac{\partial [ ]}{\partial \mathit{\psi }}\delta \mathit{\psi }\left(\mathit{x}\right)\\ +{\displaystyle {\displaystyle \sum }_{m=1}^2}\frac{\partial S^{\left(3\right)}[ ]}{\partial {\mathit{a}}_m^{\left(1\right)}}\delta {\mathit{a}}_m^{\left(1\right)}\left(\mathit{x}\right)+{\displaystyle {\displaystyle \sum }_{m=1}^4}\frac{\partial [ ]}{\partial {\mathit{a}}_m^{\left(2\right)}\left(j1\right)}\delta {\mathit{a}}_m^{\left(2\right)}\left(j1;\mathit{x}\right),\end{array}$$

(175)

$$\frac{\partial [ ]}{\partial {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)}\delta {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\equiv {\displaystyle {\displaystyle \sum }_{m=1}^8}\frac{\partial [ ]}{\partial {\mathit{a}}_m^{\left(3\right)}\left(j2;j1;\mathit{x}\right)}\delta {\mathit{a}}_m^{\left(3\right)}\left(j2;j1;\mathit{x}\right).$$

(176)

The indirect-effect term ${\left\{\delta R^{\left(3\right)}\left(j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\delta {\mathit{u}}^{\left(3\right)};\delta {\mathit{a}}^{\left(3\right)}\right)\right\}}_{ind}$ defined in Equation (174) can be computed only after having computed the variations $\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)$ and $\delta {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$. Recall that the 3rd-level variational sensitivity function $\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)$ is the solution of the 3rd-LVSS. On the other hand, the vector $\delta {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$ is the solution of the system of equations obtained by G-differentiating the 3rd-LASS. Applying the definition of the total G-differential, cf. Equation (23), to the 3rd-LASS, cf. Equations (138) and (148), yields the following set of equations for the vector of variations $\delta {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$, all to be evaluated at the nominal values for the parameters and state functions, for $j1,j2=1,\dots ,N_{\mathit{\alpha }}$:

$$\begin{array}{c}\frac{\partial \left\{A^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(j1;\mathit{x}\right)\right]{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)-s^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)};\mathit{\alpha }\right]\right\}}{\partial {\mathit{u}}^{\left(3\right)}}\delta {\mathit{u}}^{\left(3\right)}\\ +{\left\{A^{\left(3\right)}\left(j1;{\mathit{u}}^{\left(2\right)}\right)\right\}}_{{\mathit{\alpha }}^0}\delta {\mathit{a}}^{\left(3\right)}=p^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right],\end{array}$$

(177)

$$\begin{array}{c}{\left\{\delta {\mathit{b}}_A^{\left(3\right)}\left[{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta {\mathit{a}}^{\left(3\right)};\delta {\mathit{u}}^{\left(3\right)};\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}=0,\\ \mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right),j1,j2=1,\dots ,N_{\mathit{\alpha }},\end{array}$$

(178)

where

$$\begin{array}{c}{p}^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\\ \triangleq \frac{\partial \left\{s^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]-A^{\left(3\right)}\left({\mathit{u}}^{\left(2\right)}\right){\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\right\}}{\partial \mathit{\alpha }}\delta \mathit{\alpha }.\end{array}$$

(179)

As Equations (177) and (178) indicate, the vectors of variations $\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)$ and $\delta {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$ are related to each other, and must be determined by solving simultaneously the coupled system of equations obtained by concatenating the 3rd-LVSS to Equation (177) while being subject to the boundary conditions obtained by concatenating the boundary conditions which belong to the 3rd-LVSS with the boundary conditions represented by Equation (178). Concatenating Equations (119) and (177) yields the following 16 × 16 block-matrix equation:

$${\mathit{V}}^{\left(4\right)}\left({\mathit{u}}^{\left(3\right)}\right)\delta {\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right)={\mathit{q}}^{\left(4\right)}\left[j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right],$$

(180)

where:

$${\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right)\triangleq \left(\begin{array}{c}{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)\\ {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\end{array}\right);\delta {\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right)\triangleq \left(\begin{array}{c}\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)\\ \delta {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\end{array}\right);$$

(181)

$$\begin{array}{c}{\mathit{V}}^{\left(4\right)}\left({\mathit{u}}^{\left(4\right)}\right)=\left(\begin{array}{cc}{\mathit{V}}^{\left(3\right)}\left(j1;{\mathit{u}}^{\left(2\right)}\right)& {\left[0\right]}_{\left(8\times 8\right)}\\ {\mathit{V}}_{21}^{\left(4\right)}\left(j2;j1\right)& {\mathit{V}}_{22}^{\left(4\right)}\left(j1\right)\end{array}\right);{\mathit{V}}_{22}^{\left(4\right)}\left(j1\right)\triangleq A^{\left(3\right)}\left(j1;{\mathit{u}}^{\left(2\right)}\right);\\ {\mathit{V}}_{21}^{\left(4\right)}\triangleq \frac{\partial \left\{A^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(j1;\mathit{x}\right)\right]{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)-s^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}{\partial {\mathit{u}}^{\left(3\right)}};\end{array}$$

(182)

$$\begin{array}{c}{\mathit{q}}^{\left(4\right)}\left[j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\triangleq \left(\begin{array}{c}{\mathit{q}}^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\\ p^{\left(3\right)}\left[j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\end{array}\right)\\ \equiv \left(\begin{array}{c}{\mathit{q}}_1^{\left(4\right)}\left[j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\\ \\ {\mathit{q}}_{16}^{\left(4\right)}\left[j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\end{array}\right).\end{array}$$

(183)

Concatenating the boundary conditions which belong to the 3rd-LVSS, cf. Equation (120), with the boundary conditions represented by Equation (178) yields the following boundary conditions for the function $\delta {\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right)$:

$$\begin{array}{c}{\mathit{b}}_{\mathit{v}}^{\left(4\right)}\left[\mathit{u};{\mathit{a}}^{\left(1\right)};{\mathit{a}}^{\left(2\right)}\left(j1\right);\mathit{\alpha };\delta \mathit{\phi }\left(\mathit{x}\right);\delta \mathit{\psi }\left(\mathit{x}\right);\delta {\mathit{a}}^{\left(2\right)}\left(j1;\mathit{x}\right);\delta \mathit{\alpha }\right]\\ \triangleq \left(\begin{array}{c}{\mathit{b}}_{\mathit{v}}^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\delta \mathit{\alpha }\right]\\ \delta {\mathit{b}}_A^{\left(3\right)}\left[{\mathit{u}}^{\left(2\right)}\left(j1;\mathit{x}\right);\mathit{\alpha };\delta {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\delta \mathit{\alpha }\right]\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right),\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right).\end{array}$$

(184)

The matrix-operator equations represented by Equation (180) together with the boundary conditions represented by Equation (184) will be called the 4th-Level Variational Sensitivity System (4th-LVSS). The solution, $\delta {\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right)$, of the 4th-LVSS will be called the “4th-level variational sensitivity function”.

Since the 4th-LVSS equations depend on the parameter variations $\delta {\mathit{\alpha }}_i$, solving them is prohibitively expensive computationally for large-scale systems involving many parameters. The need for solving the 4th-LVSS can be avoided by expressing the indirect-effect term ${\left\{\delta R^{\left(3\right)}\left(j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\delta {\mathit{u}}^{\left(3\right)};\delta {\mathit{a}}^{\left(3\right)}\right)\right\}}_{ind}$ defined in Equation (174) in an alternative way, which does not involve the function $\delta {\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right)$. This alternative expression will be derived by using the solution of a 4th-Level Adjoint Sensitivity Systems (4th-LASS), which will be constructed by implementing the same sequence of logical steps as were followed for constructing the 1st, 2nd, and 3rd-LASS. The first step is to define a Hilbert space, denoted as $H_4$, comprising block-vector elements of the form ${\mathit{\eta }}^{\left(4\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{\eta }}_1^{\left(4\right)}\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_{16}^{\left(4\right)}\left(\mathit{x}\right)\right]}^{†}\in H_4$, with ${\mathit{\eta }}_i^{\left(4\right)}\left(\mathit{x}\right)\triangleq {\left[{\mathit{\eta }}_{i,1}^{\left(4\right)}\left(\mathit{x}\right),\dots ,{\mathit{\eta }}_{i,N_{\mathit{\phi }}}^{\left(4\right)}\left(\mathit{x}\right)\right]}^{†},i=1,..,16.$ The inner product between two elements, ${\mathit{\eta }}^{\left(4\right)}\left(\mathit{x}\right)\in H_4$ and ${\xi }^{\left(4\right)}\left(\mathit{x}\right)\in H_4$, of this Hilbert space, will be denoted as $\langle {\mathit{\eta }}^{\left(4\right)}\left(\mathit{x}\right),{\xi }^{\left(4\right)}{\left(\mathit{x}\right)\rangle }_4$ and is defined as follows:

$$\langle {\mathit{\eta }}^{\left(4\right)}\left(\mathit{x}\right),{\xi }^{\left(4\right)}{\left(\mathit{x}\right)\rangle }_4\triangleq {\sum }_{i=1}^{16}\langle {\mathit{\eta }}_i^{\left(4\right)}\left(\mathit{x}\right),{\xi }_i^{\left(4\right)}{\left(\mathit{x}\right)\rangle }_0.$$

(185)

Using the definition provided in Equation (185), form the inner product in $H_4$ of Equation (180) with a set of yet undefined vector-valued functions ${\mathit{a}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(4\right)}\left(j2;j1;\mathit{x}\right),\dots ,{\mathit{a}}_{16}^{\left(4\right)}\left(j2;j1;\mathit{x}\right)\right]}^{†}\in {\mathit{h}}_4$,$j1,j2=1,\dots ,N_{\mathit{\alpha }}$, to obtain the following relation:

$$\begin{array}{c}{\{\langle {\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right),{\mathit{V}}^{\left(4\right)}\left({\mathit{u}}^{\left(3\right)}\right)\delta {\mathit{u}}^{\left(4\right)}{\left(j2;j1;\mathit{x}\right)\rangle }_4\}}_{{\mathit{\alpha }}^0}\\ ={\{\langle {\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right),{\mathit{q}}^{\left(4\right)}{\left[j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\rangle }_4\}}_{{\mathit{\alpha }}^0}.\end{array}$$

(186)

Using the definition of the adjoint operator in the Hilbert space $H_4$, recast the left-side of Equation (186) as follows:

$$\begin{array}{c}{\{\langle {\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right),{\mathit{V}}^{\left(4\right)}\left({\mathit{u}}^{\left(3\right)}\right)\delta {\mathit{u}}^{\left(4\right)}{\left(j2;j1;\mathit{x}\right)\rangle }_4\}}_{{\mathit{\alpha }}^0}\\ ={\{\langle \delta {\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right),A^{\left(4\right)}\left({\mathit{u}}^{\left(3\right)}\right){\mathit{a}}^{\left(4\right)}{\left(j3;j2;j1;\mathit{x}\right)\rangle }_4\}}_{{\mathit{\alpha }}^0}\\ +{\left\{P^{\left(4\right)}\left[\delta {\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);{\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0},\end{array}$$

(187)

where ${\left\{P^{\left(4\right)}\left[\delta {\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);{\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}$ denotes the bilinear concomitant defined on the phase-space boundary $\mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right)$ and where $A^{\left(4\right)}\left(j2;j1;{\mathit{u}}^{\left(3\right)}\right)={\left\{{\mathit{V}}^{\left(4\right)}\left[j2;j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\right]\right\}}^{*}$ is the operator formally adjoint to ${\mathit{V}}^{\left(3\right)}\left[j1;{\mathit{u}}^{\left(2\right)}\left(\mathit{x}\right)\right]$ and has therefore the following form:

$$A^{\left(4\right)}\left(j2;j1;{\mathit{u}}^{\left(3\right)}\right)={\left(\begin{array}{cc}{\mathit{V}}^{\left(3\right)}\left({\mathit{u}}^{\left(1\right)}\right)& {\left[0\right]}_{\left(4\times 4\right)}\\ {\mathit{V}}_{21}^{\left(4\right)}\left(j1;{\mathit{u}}^{\left(2\right)}\right)& {\mathit{V}}_{22}^{\left(4\right)}\left({\mathit{u}}^{\left(1\right)}\right)\end{array}\right)}^{*}=\left(\begin{array}{cc}{A}^{\left(3\right)}& {\left[{\mathit{V}}_{21}^{\left(4\right)}\right]}^{*}\\ {\left[0\right]}_{\left(4\times 4\right)}& {\left[{\mathit{V}}_{22}^{\left(4\right)}\right]}^{*}\end{array}\right).$$

(188)

The first term on right-side of Equation (187) is now required to represent the indirect-effect term ${\left\{\delta R^{\left(3\right)}\left(j3;j2;j1;{\mathit{u}}^{\left(3\right)};{\mathit{a}}^{\left(3\right)};\delta {\mathit{u}}^{\left(3\right)};\delta {\mathit{a}}^{\left(3\right)}\right)\right\}}_{ind}$ defined in Equation (184). This requirement is satisfied by imposing the following relation on each element ${\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right),\dots ,{\mathit{a}}_{16}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right)\right]}^{†},j3,j2,j1=1,\dots ,N_{\mathit{\alpha }}$:

$$\begin{array}{c}{A}^{\left(4\right)}\left({\mathit{u}}^{\left(3\right)}\right){\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right)=s^{\left(4\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right],\\ j1,j2,j3=1,\dots ,N_{\mathit{\alpha }},\end{array}$$

(189)

where the block-vector

$$s^{\left(4\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]\triangleq {\left[s_1^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right),\dots ,s_{16}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right)\right]}^{†},$$

(190)

comprises, for each $j3,j2,j1=1,\dots ,N_{\mathit{\alpha }}$, sixteen components defined as follows:

$$\begin{array}{c}{s}_1^{\left(4\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]\triangleq \frac{\partial S^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial \mathit{\phi }},\\ s_2^{\left(4\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]=\frac{\partial S^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial \mathit{\psi }},\\ s_{2+m}^{\left(4\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]=\frac{\partial S^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{a}}_m^{\left(1\right)}};m=1,2;\\ s_{4+m}^{\left(4\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]=\frac{\partial S^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{a}}_m^{\left(2\right)}\left(j1\right)};\\ m=1,..,4;\\ s_{8+m}^{\left(4\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]=\frac{\partial S^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{a}}_m^{\left(3\right)}\left(j2;j1\right)};\\ m=1,..,8.\end{array}$$

(191)

The definition of the set of vectors ${\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right),\dots ,{\mathit{a}}_{16}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right)\right]}^{†}$ will now be completed by selecting boundary conditions for this set of vectors, which will be represented in operator form as follows:

$$\begin{array}{c}{\left\{{\mathit{b}}_A^{\left(4\right)}\left[{\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right);{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}=0,\\ \mathit{x}\in \partial {\Omega }_{\mathit{x}}\left({\mathit{\alpha }}^0\right),j1,j2,j3=1,\dots ,N_{\mathit{\alpha }}.\end{array}$$

(192)

The boundary conditions represented by Equation (148) are selected so as to satisfy the following requirements:

(i)

The boundary conditions Equation (192) together with the operator Equation (189) constitute a well posed problem for the functions ${\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$.

(ii)

Implementation in Equation (187) of the boundary conditions (for the 3rd-LVSSS) provided in Equation (148) together with those provided in Equation (192) eliminates all of the unknown values of the functions $\delta {\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right)$ and ${\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right)$ in the expression of the bilinear concomitant ${\left\{P^{\left(4\right)}\left[\delta {\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);{\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}$. This bilinear concomitant may vanish after implementing the boundary conditions represented by Equation (148), but if it does not, it will be reduced to a residual quantity which will be denoted as ${\left\{{\widehat{P}}^{\left(4\right)}\left[{\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right);{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}$ and which will comprise only known values of ${\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right)$, ${\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right)$, $\mathit{\alpha }$ and $\delta \mathit{\alpha }$.

The system of equations represented by Equation (189) together with the boundary conditions represented by Equation (192) constitute the 4th-Level Adjoint Sensitivity System (4th-LASS). The solution ${\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right)\triangleq {\left[{\mathit{a}}_1^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right),\dots ,{\mathit{a}}_{16}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right)\right]}^{†}$, $j1,j2,j3=1,\dots ,N_{\mathit{\alpha }}$, of the 4th-LASS will be called the 4th-level adjoint function.

The 4th-LASS together with the results provided in Equations (186) and (187) are employed in Equation (174) to obtain the following expression for the indirect-effect term ${\left\{\delta R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\delta {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\right]\right\}}_{ind}$ in terms of the 4th-level adjoint functions ${\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right)$, for $j1,j2,j3=1,\dots ,N_{\mathit{\alpha }}$:

$$\begin{array}{c}{\left\{\delta R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);\delta {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)\right]\right\}}_{ind}\\ ={\left\{{\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right),{\mathit{q}}^{\left(4\right)}{\left[j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]}_4\right\}}_{{\mathit{\alpha }}^0}\\ -{\left\{{\widehat{P}}^{\left(4\right)}\left[{\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right);{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}\\ \equiv {\left\{\delta R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]\right\}}_{ind}.\end{array}$$

(193)

As the identity in the last line of Equation (193) indicates, the dependence of the indirect-effect term on the variations $\delta {\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right)$ and $\delta {\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right)$ has been eliminated by having replaced these functional-dependences by the dependence on the 4th-level adjoint function ${\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right)$.

Representing the components of ${\mathit{q}}^{\left(4\right)}\left[j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha }\right]$ in the following form:

$$\begin{array}{c}{\mathit{q}}_m^{\left(4\right)}\left(j2;j1;{\mathit{u}}^{\left(4\right)};\mathit{\alpha };\delta \mathit{\alpha }\right)\\ \equiv {\displaystyle {\displaystyle \sum }_{j4=1}^{N_{\mathit{\alpha }}}}{\left\{\frac{\partial {\mathit{q}}_m^{\left(4\right)}\left[j4;j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]}{\partial {\mathit{\alpha }}_{j4}}\right\}}_{\left({\mathit{\alpha }}^0\right)}\delta {\mathit{\alpha }}_{j4},m=1,\dots ,16.\end{array}$$

(194)

and adding the expression obtained in Equation (193) for the indirect-effect term with the expression of the direct-effect term given in Equation (173) yields the following expression for the 4th-order total G-differential ${\left\{\delta R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta {\mathit{u}}^{\left(3\right)};\delta {\mathit{a}}^{\left(3\right)};\delta \mathit{\alpha }\right]\right\}}_{{\mathit{\alpha }}^0}$ of the response with respect to the model parameters, for $j1,j2,j3=1,\dots ,N_{\mathit{\alpha }}$:

$$\begin{array}{c}\delta R^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha };\delta \mathit{\alpha };\right]\\ ={\displaystyle {\displaystyle \sum }_{j4=1}^{N_{\mathit{\alpha }}}}{\left\{R^{\left(4\right)}\left[j4;j3;j2;j1;{\mathit{u}}^{\left(4\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{\left({\mathit{\alpha }}^0\right)}\delta {\mathit{\alpha }}_{j4},\end{array}$$

(195)

where $R^{\left(4\right)}\left[j4;j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]$ denotes the 4th-order partial sensitivity of the response $R\left[{\mathit{u}}^{\left(1\right)}\left(\mathit{x}\right);\mathit{\alpha }\right]$ with respect to the model parameters, evaluated at the nominal parameter values ${\mathit{\alpha }}^0$, and can be represented as follows:

$$\begin{array}{c}{R}^{\left(4\right)}\left[j4;j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]=\frac{{\partial }^{4}R\left(\mathit{\phi };\mathit{\psi };\mathit{\alpha }\right)}{\partial {\mathit{\alpha }}_{j4}\partial {\mathit{\alpha }}_{j3}\partial {\mathit{\alpha }}_{j2}\partial {\mathit{\alpha }}_{j1}}\\ =-{\left\{\frac{\partial }{\partial {\mathit{\alpha }}_{j4}}{\widehat{P}}^{\left(4\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{\left({\mathit{\alpha }}^0\right)}\\ +\frac{\partial }{\partial {\mathit{\alpha }}_{j4}}{\left\{{\displaystyle {\displaystyle \prod }_{k=1}^{N_{\mathit{x}}}}{{\displaystyle \int }}_{{\lambda }_k\left(\mathit{\alpha }\right)}^{{\omega }_k\left(\mathit{\alpha }\right)}d{\mathit{x}}_k{S}^{\left(3\right)}\left[j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]\right\}}_{\left({\mathit{\alpha }}^0\right)}\\ +\frac{\partial }{\partial {\mathit{\alpha }}_{j4}}{\left\{{\displaystyle {\displaystyle \sum }_{m=1}^{16}}\langle {\mathit{a}}_m^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right),Q_m^{\left(4\right)}{\left[j4;j3;j2;j1;{\mathit{u}}^{\left(3\right)}\left(j1;\mathit{x}\right);{\mathit{a}}^{\left(3\right)}\left(j2;j1;\mathit{x}\right);\mathit{\alpha }\right]\rangle }_0\right\}}_{\left({\mathit{\alpha }}^0\right)}.\end{array}$$

(196)

After obtaining the 4th-level adjoint function ${\mathit{a}}^{\left(4\right)}\left(j3;j2;j1;\mathit{x}\right)$ by performing at most $O\left(N_{\mathit{\alpha }}^3\right)$ large-scale computations, the 4th-order response sensitivities to model parameters are obtained using the expression provided in Equation (196).

4. Discussion and Conclusions

This work has presented the 4th-Order Comprehensive Adjoint Sensitivity Analysis Methodology (4th-CASAM), which enables the efficient computation of the exact expressions of the 4th-order functional derivatives (“sensitivities”) of a general system response, which depends on both the forward and adjoint state functions, with respect to all of the parameters underlying the respective forward and adjoint systems. Since nonlinear operators do not admit adjoint operators (only linearized versions of nonlinear operators can admit a bona-fide adjoint operator), responses that simultaneously depend on forward and adjoint functions can arise only in conjunction with linear systems, thus providing the fundamental motivation for treating linear systems in their own right rather than just as particular cases of nonlinear systems. Very importantly, the computation of the 2nd-, 3rd-, and 4th-level adjoint functions uses the same forward and adjoint solvers (i.e., computer codes) as used for solving the original forward and adjoint systems, thus requiring relatively minor additional software development for computing the various-order sensitivities.

The 4th-CASAM presented in this work is the only practically implementable methodology for obtaining and subsequently computing the exact expressions (i.e., free of methodologically-introduced approximations) of the 1st-, 2nd-, 3rd-, and 4th-order sensitivities (i.e., functional derivatives) of responses to system parameters, for coupled forward/adjoint linear systems. By enabling the practical computation of any and all of the 1st-, 2nd-, 3rd-, and 4th-order response sensitivities to model parameters, the 4th-CASAM makes it possible to compare the relative values of the sensitivities of various order, in order to assess which sensitivities are important and which may actually be neglected, thus enabling future investigations of the convergence of the (multivariate) Taylor series expansion of the response in terms of parameter variations, as well as investigating the actual validity of expressions that are derived from Taylor-expansion of the response (e.g., response variances/covariance, skewness, kurtosis, etc.) as a function of the model’s parameters.

The 4th-CASAM presented in this work provides a fundamentally important step in the quest to overcome the “curse of dimensionality” in sensitivity analysis, uncertainty quantification, and predictive modeling. Ongoing work aims at generalizing the 4th-CASAM, attempting to provide a general methodology for the practical, efficient, and exact computation of arbitrarily-high order sensitivities of responses to model parameters, both for linear and nonlinear systems.