Prediction of Lower Limb Kinetics and Kinematics during Walking by a Single IMU on the Lower Back Using Machine Learning

Abstract

Recent studies have reported the application of artificial neural network (ANN) techniques on data of inertial measurement units (IMUs) to predict ground reaction forces (GRFs), which could serve as quantitative indicators of sports performance or rehabilitation. The number of IMUs and their measurement locations are often determined heuristically, and the rationale underlying the selection of these parameter values is not discussed. Using the dynamic relationship between the center of mass (CoM), the GRFs and joint kinetics, we propose the CoM as a single measurement location with which to predict the dynamic data of the lower limbs, using an ANN. Data from seven subjects walking on a treadmill at various speeds were collected from a single IMU worn near the sacrum. The data was segmented by step and numerically processed for integration. Six segment angles of the stance and swing leg, three joint torques, and two GRFs were estimated from the kinematics of the CoM measured from a single IMU sensor, with fair accuracy. These results indicate the importance of the CoM as a dynamic determinant of multi-segment kinetics during walking. The tradeoff between data quantity and wearable convenience can be solved by utilizing a machine learning algorithm based on the dynamic characteristics of human walking.

1. Introduction

Kinetic data of human motion, such as ground reaction forces (GRFs) and joint torques, serve as a quantitative indicator of sports performance or the effect of rehabilitation. GRFs and joint torques have served as indicators of injury risks and pain during running [1,2,3,4,5]. The progress of gait rehabilitation of hemiparetic patients has been monitored by the magnitude and the degree of asymmetry of the GRF [6,7,8]. The motion analysis used for these studies is performed in a laboratory with accurate motion trackers and force transducers, which often impose spatial and temporal constraints on subjects.

With the development of sensor technology, the market for wearable motion monitoring systems has rapidly grown. Wearable motion monitoring products such as Galaxy, Garmin, Fitbit, and Apple watches monitor the overall rough motion information, such as the steps, cadence, distance, etc. That rough motion information, however, makes it difficult to estimate the kinetic information for the analysis of injury risks and rehabilitation. One of the common issues of current wearables is how to increase the quantity and quality of the measured data with minimal increase in the system complexity. In particular, obtaining force information using wearables is very challenging due to the size and weight of the force transducers. Instead of direct measurement, researchers have attempted to predict unmeasured GRF information from joint kinematics information. Oh et al. predicted 3D GRFs based on the whole body motion capture data, but the motion capture system still imposes spatial and temporal constraints on subjects [9]. To overcome the constraints of the motion capture system, Karatsidis et al. predicted GRFs using inertial measurement units (IMUs) on multiple limb segments, and fairly accurately estimated the 3D GRFs from walking speeds of 0.9 to 1.6 m/s [9,10].

The number of IMUs measuring whole body motion, however, greatly limits the applicability of the IMU based prediction method for wearable devices. To reduce the number of IMUs used for GRF prediction, recent studies used the artificial neural network (ANN) with a limited amount of measurement information. Using the ANN, researchers have utilized multi-IMU data located on the shank or foot of both legs to predict the 3D GRFs in walking and running [11,12]. Using a single IMU measurement taken at the sacrum, researchers predicted the vertical GRF [13] or the 3D GRF and used additional sensor information to detect gait events such as heel strike or toe-off [14]. The ANN could predict unmeasured force information from the IMUs, find the global optima of the loss function during backpropagation [15,16,17,18], generalize without overfitting [19,20], and select appropriate input types and size [10,21,22]. Selecting an appropriate type and size of the data plays a key role in improving training efficiency and test accuracy [21,22], and dimension reduction techniques or the employment of large datasets are often used for this purpose [10,23]. Oh et al. used self-organizing maps (SOM) and a generic algorithm-general regression neural network (GA-GRNN) to reduce the dimensionality of the input from 90 kinematic data points to 14, to predict 3D GRFs [10]. Johnson et al. collected kinematic and kinetic data from over 400 subjects, and used eight trajectories of the kinematic marker data, body parameters of mass and height, and gender information as inputs to the network [23]. Although a sufficiently large input data size may help with the search for appropriate input combinations, the search process often involves optimization and the use of heuristics.

To resolve some issues of the ANN listed above, one may use knowledge of the dynamic characteristics of human walking. Whereas the multiple limbs used during walking are coordinated and actuated in a complicated manner by multiple muscles, the resultant motion of the body CoM and related forces are known to be describable by simple spring mechanics [24,25,26,27,28]. The experimental GRFs at various walking conditions are well emulated by the springy mechanics of the CoM [25,26], leading to the CoM motion being easily estimated by the GRF, and vice versa. By adding spring components and geometrical constraints to the spring-loaded inverted pendulum (SLIP) model, recent studies have shown the extended dynamic association of the CoM with the ankle joint torque and multiple joint kinetics [29], as well as swing leg kinetics [30]. These findings highlight the importance of the CoM as a dynamic determinant of multi-segment kinetics during gait. Noting that the ANN generates the output from the weighted sum of the inputs, the mechanical coupling between the CoM and the joint kinetics could be implemented in the ANN to efficiently and accurately predict the unmeasured joint kinetics.

In this study, using the dynamic relationship between the CoM, GRFs, and joint kinetics, we propose the CoM as a single measurement location which can be used to predict unmeasured dynamic data of the lower limbs using an ANN. Data from seven subjects walking on a treadmill at various speeds were collected from a single IMU worn near the sacrum. The data was segmented by step and numerically processed for integration. After training the ANN with the processed IMU data, lower limb kinematics and kinetics at various gait speeds were predicted and tested by the leave-one-subject-out (LOO) validation method.

2. Methods

We proposed a method for the prediction of six lower limb kinematics and five lower limb kinetics data points from a single IMU measurement using the ANN, based on the biomechanical characteristics of the CoM and the GRF during human walking (Figure 1). Mechanical coupling between the CoM, the GRF, and joint kinematics were used to approximate the GRF and joint motion as a weighted sum of the CoM, which is the basic input–output relationship of an ANN. We hypothesized that the CoM kinematics measured from a single IMU, used as an input node to the ANN, could predict the unmeasured GRF, joint kinematics and kinetics. The IMU measurement obtained from the sacrum was pre-processed to identify each step and to obtain position and velocity information. The processed IMU data were then fed into the input node of the ANN, which consists of a single hidden layer of 20 nodes with sigmoid activation and eleven output nodes for the segment angles of the foot, shank, and thigh of the stance and swing leg, the joint torques of the hip, knee, and ankle of the stance leg, and the horizontal and vertical GRF, respectively. The details of the experimental protocols, the signal processing of the IMU data, and the prediction algorithm are presented in Figure 1.

2.1. Finding the Location of a Single Measurement by the IMU

To examine which part of body motion should be measured to best estimate the lower limb joint kinetics, we used the dynamic characteristics of human walking. Since the outputs of the ANN are obtained from the weighted sum of the inputs passed through a nonlinear activation function, the equations of motion of the CoM during walking obtained from the recently published SLIP model [29] were approximated, to estimate the GRF and joint dynamics in terms of the weighted sum of the CoM. An approximation of the relationship between the CoM position and the state variables of the mass–spring model of human walking was formulated (Equation (1)). The GRFs and angles of multiple joints were formulated in terms of state variables (Equations (2) and (3)). Finally, the GRFs, ankle joint torque, and angles of the foot, shank, and thigh were formulated as the weighted sum of the CoM positions. These calculations led us to propose the CoM as the best measurement position for prediction of the unmeasured GRF and joint dynamics. A brief summary follows, the details of which can be found in the Appendix A.

The position of the CoM, x = (x, y), is expressed as model parameters of the SLIP, such as the leg angle θ, the leg length l, the radius of the curvy foot R, the offset of the ankle joint with respect to the foot d, and the ground contact position c_hs, with the subscript hs denoting values at the onset of the stance phase (Figure 2), as follows:

By linearization, we can obtain the mapping equation from the CoM position x to the augmented state z = [l, θ, lθ]^Τ, which includes the state parameters, l and θ of the SLIP model, and its nonlinear term lθ as follows:

$$\begin{array}{l}z=W_{1}x+b_1\\ \mathrm{where}{W}_1=\left[\begin{array}{cc}0& 1\\ y_{hs}^{-1}& 0\\ 1& 0\end{array}\right],b_1=\left[\begin{array}{c}0\\ \alpha y_{hs}^{-1}\\ \alpha \end{array}\right]\end{array}$$

(1)

where α = Rθ_hs − c_hs + d.

The relationship between the augmented state z and the augmented kinetics f = [f_x, f_y, T_a]^T such as GRF and ankle torque are obtained as follows:

$$\begin{array}{c}f=W_{2}z+b_2\\ \mathrm{where}{W}_2=\left[\begin{array}{ccc}kd{l}_0{}^{-1}& k\left(l_0-R\right)& k\left(R{l}_0{}^{-1}-1\right)\\ -k& kd& -kd{l}_0{}^{-1}\\ -kd{c}_l& kR{l}_0{c}_l& -kR{c}_l\end{array}\right],b_2=\left[\begin{array}{c}-kd\\ k{l}_0\\ kd{l}_0{c}_l\end{array}\right]\end{array}$$

(2)

The relationship between the augmented state z and the augmented segment angles θ = [θ₁, θ₂, θ₃]^T can be rewritten as follows:

$$\begin{array}{l}\mathsf{\theta }=W_{3}z+b_3\\ \mathrm{where}{W}_3=\left[\begin{array}{ccc}\left(1-c_k\right)l_1{}^{-1}& 1& 0\\ \mp 0.25c_l{c}_k{l}_0{l}_2{}^{-2}& 0.5\left(c_l-c_k\right)l_0{l}_2{}^{-1}& 0.5c_k{l}_2{}^{-1}\\ \pm 0.25c_l{c}_k{l}_0{l}_2{}^{-2}& 0.5\left(c_l-c_k\right)l_0{l}_2{}^{-1}& 0.5c_k{l}_2{}^{-1}\end{array}\right],\\ b_3=\left[\begin{array}{c}-\left(1-c_k\right)l_0{l}_1{}^{-1}\\ \pm \left(1-0.125c_l\left(c_l-2c_k\right)l_0{}^2{l}_2{}^{-2}\right)\\ \mp \left(1-0.125c_l\left(c_l-2c_k\right)l_0{}^2{l}_2{}^{-2}\right)\end{array}\right]\end{array}$$

(3)

In summary, the kinematics and kinetics of the lower limb can be approximated as a weighted sum of augmented states z of the spring mechanics and the bias as follows:

$$\left[\begin{array}{c}f\\ \mathsf{\theta }\end{array}\right]=\left[\begin{array}{c}{W}_2{W}_1\\ W_3{W}_1\end{array}\right]x+\left[\begin{array}{c}{W}_1{b}_1+b_2\\ W_3{b}_1+b_3\end{array}\right]$$

(4)

From the approximated relationship between the CoM position x and the lower limb kinetics and kinematics [f^T, θ^T] as a form of weighted sum of x and bias, we tried to predict the unmeasured lower limb kinematics and kinetics using the sacrum motion profiles, the measurable approximation of the CoM. We may use the terms ‘CoM’ and ‘sacrum’ interchangeably throughout the manuscript (see also the Appendix A for the formulation of the lower limb kinematics and kinetics as an approximation of the weighted sum of the CoM states).

2.2. Experimental Protocols and Data Collection

Seven young (25.0 ± 2.9 years) healthy male subjects with average heights and weights of 168.8 ± 7.5 cm and 72.1 ± 7.7 kg, respectively, participated in the data collection, after signing fully informed consent forms approved by KAIST Institutional Review Board on 22 January 2018 (IRB-18-023). The subjects participated in one-minute treadmill walking trials at three customized gait speeds, ranging from 1.0 m/s to 2.3 m/s. The test speeds were determined by each subject’s preferred gait speed, the midpoint between his preferred and maximum speeds, and a speed slower than the preferred speed by an amount equal to approximately 20% of the difference between the preferred and maximum speeds. The label ‘slow’, ‘moderate’, and ‘fast’ indicates slow speed, preferred speed, and fast speed, respectively. The speed of each subject is in

Table 1. Speed of each subject’s slow, moderate, and fast speed used for data collection.

Subject	1	2	3	4	5	6	7	Average
Slow (m/s)	1.07	1.71	1.38	1.22	1.29	1.00	1.20	1.27 ± 0.23
Moderate (m/s)	1.21	1.87	1.49	1.37	1.51	1.14	1.36	1.42 ± 0.24
Fast (m/s)	1.54	2.26	1.78	1.76	2.07	1.49	1.76	1.81 ± 0.27

. Three one-minute gait trials per test speed were randomly ordered. The kinematics of the lower back, which is assumed to be close to the CoM, were measured by an IMU (EBIMU-9DOFV4) with an acceleration sensitivity of $\pm$16 g, and resolution of 0.001 g. The width of the IMU was 15.6 mm, and the length was 18.6 mm. The IMU data were collected at a sampling frequency of 100 Hz, and used the universal asynchronous receiver/transmitter (UART) protocol for communication with a baud rate of 921,600 bps. The IMU was aligned with the subject’s heading direction to set the local frame of the IMU to the body frame. The 2D acceleration data from the IMU in the sagittal plane were used for the analysis. The lower limb kinematics and ground reaction forces under each foot were measured by an optical motion capture system (Motion Analysis Hawk^®) and force plates (Bertec, FP 6012^®), respectively. For the segment angle calculation, 12 markers were located at the toe, heel, ankle, knee, and anterior and posterior superior iliac spine (ASI and PSI) of both legs. The midpoint of the PSIs was used as the reference position for the IMU. Motion data and force data were collected at the sampling frequency of 100 Hz and 200 Hz, respectively, and filtered with a fifth-order low-pass Butterworth filter with a cutoff frequency of 10 Hz. The joint torque was calculated by the inverse dynamics of the seven-segmental rigid body model, which includes the hip, knee, and ankle. The mass distribution of the rigid body were based on the anatomical reports [31].

2.3. Preprocessing of IMU Data for ANN Input

The IMU measurement data that contain the double support phase must be segmented into one stance phase per leg before feeding into the ANN for GRF prediction. The stance phase is defined based on the GRF as the duration from the onset of heel strike (HS) to toe off (TO) (Figure 3A,C). To identify the characteristic features of the IMU data which indicate the gait events of HS and TO, the IMU acceleration and GRF profiles were aligned over time, and three characteristic features were chosen: the CoM apex, HS, and TO (Figure 3). The filter frequency of IMU acceleration was selected heuristically to best extract the characteristic features, and was chosen to be 40 Hz for identifying HS and 10 Hz for identifying TO. The CoM apex is the point at which the height of CoM is maximal in sagittal plane around the mid-stance phase, and the corresponding vertical GRF has a local minimum (Figure 3B). The CoM apex was identified as the local minimum of the vertical IMU acceleration data filtered with a cutoff frequency of 10 Hz (Figure 3B). The HS was identified as the first local minimum of the vertical acceleration filtered by a 40 Hz cutoff frequency, following the positive local maximum after CoM apex (Figure 3B). The TO was identified as the first local minimum of the horizontal acceleration data filtered by a 10 Hz cutoff frequency after CoM apex (Figure 3D). The detailed process of IMU data segmentation is shown in Figure 3E. The precedence of HS over TO was checked for validation. When HS estimation follows the TO, the HS was re-identified as the preceding local minimum of vertical acceleration compared with the one originally found. After completing segmentation of the full stance, the consecutive segmentation process was performed by identifying the subsequent CoM apex based on the acceleration profiles from the TO event, the last event of the previous step.

Another step of preprocessing the IMU data is to calculate the velocity and position of the CoM, i.e., the sacrum, from the acceleration, which includes drift removal and identification of the integral constant. To eliminate the integration drift, steady walking and random error were assumed. Then, the drift was removed linearly with time over the duration from one apex to the following apex. The drift-removed velocity of the CoM $\widehat{v}(t)$ could be obtained as follows:

$$\widehat{v}(t)=v(t)-\left(v(T)-v(0)\right)\cdot \frac{t}{T}$$

(5)

where v (t) is the time integration of the acceleration data for time t over the duration of the stance phase T. To satisfy the steady walking assumption, the mean velocity change was set to zero so that position drift due to the random error was simultaneously removed.

The velocity and the position offsets were estimated in a heuristic manner to obtain the velocity and the position from the acceleration integration. From the observed proportionality of the gait speed v with the average magnitude of the acceleration, A, and the gait frequency, f, (0.79 and 0.73 Pearson correlation coefficients, respectively), the gait speed offset v₀ was estimated as follows:

$$v_0=\frac{aA+bf+c}{2}\cdot \sqrt{h\cdot g}$$

(6)

where a, b, and c are the regression coefficients obtained in a least square manner. To minimize the size effect of subjects, we normalized the speed with $\sqrt{h\cdot g}$, where h and g are the height of the subject and gravitational acceleration, respectively. The position of the sacrum was obtained by integrating the compensated velocity with the position offset, defined as the subject’s leg length. All data processing was conducted by MATLAB R2018a (Mathworks, Inc., Natick, MA, USA).

2.4. Structure of the ANN and Its Training and Test Procedures

The GRFs and segment angles of the lower limbs were approximated using the weighted sum of CoM position (Equation (4)), so we hypothesized that a simple ANN that generates its output from the weighted sum of inputs could be a good candidate for a prediction network. We used a fully connected feed-forward ANN with one input layer, one hidden layer and one output layer. The seven input nodes consist of a 7 × 1 column vector consisting of a time t, and the corresponding CoM kinematics (t, x, y, v_x, v_y, a_x, a_y), where x, y are horizontal and vertical position, respectively, and v and a are velocity and acceleration measured at a specific time, such as t = t₀. The position of the CoM was reset to be zero at every HS, and we defined ‘displacement’ as the reset position. The eleven output nodes constitute an 11 × 1 column vector consisting of the GRFs and the joint kinetics and kinematics corresponding to t = t₀, such as (GRF_x, GRF_y, T_ank, T_knee, T_hip, θ_foot,stance, θ_shank,stance, θ_thigh,stance, θ_foot,swing, θ_shank,swing, θ_thigh,swing), where T, and θ are joint torque and segment angle, respectively, with subscripts showing the names of the lower limb segment. All kinematic data of the sacrum, such as the acceleration, velocity and displacement, were normalized by its maximum norm such that the values ranged from 0 to 1. The hidden layer has 20 nodes, and node weights are iteratively updated to best match the data. The sigmoid and linear activation transfer functions were used in the hidden and output layers. The loss function of the network was set as the mean square error of the predicted angles of both the stance and swing legs, the joint torque of the stance leg and the GRF with respect to the observed data. The network employed an Adam optimizer for the back-propagation process. To match the size of the training dataset collected at various gait speeds of many subjects, gait data of one stance phase were interpolated into 200 points. As a quantitative measure of prediction performance, we used the normalized root mean square error (NRMSE), which was normalized by the difference between the maximum and minimum values of the outputs of each subject at each speed. The NRMSE value of each subject represents the error of a total of 30 trials at one speed for one output. The neural network programming was conducted using PyTorch 0.4.1 [32].

To test the proposed prediction methods, we used LOO cross validation. A total of 540 stance phase trials obtained from six subjects was used as the training dataset, and the results were validated using data from the remaining subject. Validation was performed for each of seven subjects. To evaluate which kinematic information contributed most to the prediction accuracy, various combinations of kinematic inputs, such as CoM kinematics with or without position, velocity, and acceleration, were used as the inputs to the ANN. To examine whether the walking speed at which the training datasets were collected affected the prediction accuracy of the GRFs at other speeds, the data collected at various walking speeds were used as training datasets of the ANN.

3. Results

From the single IMU measurement at the lower back, the practical approximate of the CoM, eleven joint kinetics data points were predicted by the ANN over walking speeds ranging from 1.0 to 2.3 m/s. The predicted joint data are the thigh, shank and foot angles of the swing and stance leg, the hip, knee, and ankle joint torques of the stance leg, and the vertical and A–P ground reaction forces.

Gait event detection from the IMU data showed a reasonable match with those defined from the GRF, with average detection errors of the HS and TO of 0.025 s and 0.014 s, respectively (

Table 2. Mean absolute error of heel-strike (HS) and toe-off (TO) timings with gait speed.

MAE (s)	Slow	Moderate	Fast	Total
HS	0.030 ± 0.022	0.025 ± 0.016	0.021 ± 0.017	0.025 ± 0.019
TO	0.016 ± 0.010	0.016 ± 0.011	0.010 ± 0.009	0.014 ± 0.010

). The detection algorithm showed a more accurate performance for TO than that for HS, with the accuracy increasing with the gait speed. With drift removal and the estimation of the integral constant of the IMU measured acceleration, the velocity and displacement trajectories of the CoM were obtained by integrating the IMU data based on the detected gait events (Figure 4). The calculated sacrum speed and the displacement segmented by the stance phase matched well with the data obtained from optical markers, with NRMSEs of approximately 25% and 12%, respectively (

Table 3. Mean absolute error and normalized root mean square error (NRMSE) of displacement and velocity of sacrum.

	Error	Slow	Moderate	Fast	Total
Vertical displacement	MAE (m)	0.006 ± 0.003	0.005 ± 0.002	0.006 ± 0.003	0.006 ± 0.003
	NRMSE (%)	20.12 ± 10.67	16.83 ± 7.14	16.82 ± 7.48	17.93 ± 8.71
A–P displacement	MAE (m)	0.06 ± 0.03	0.04 ± 0.02	0.04 ± 0.02	0.04 ± 0.03
	NRMSE (%)	8.76 ± 4.72	5.96 ± 3.75	4.69 ± 2.90	6.47 ± 4.21
Vertical velocity	MAE (m/s)	0.06 ± 0.02	0.06 ± 0.02	0.06 ± 0.03	0.06 ± 0.02
	NRMSE (%)	15.89 ± 5.82	14.21 ± 4.84	12.77 ± 4.43	14.29 ± 5.22
A–P velocity	MAE (m/s)	0.14 ± 0.079	0.12 ± 0.07	0.16 ± 0.08	0.14 ± 0.08
	NRMSE (%)	43.98 ± 24.94	34.87 ± 20.82	43.96 ± 23.40	40.94 ± 23.48

). No significant tendencies were observed for the estimation errors of the sacrum kinematics as a function of the gait speed.

The prediction results (NRMSE and ${\mathrm{R}}^2$) of each subjects are presented in

Table A1. Percentage of NRMSE values of each subjects.

Subject	1	2	3	4	5	6	7
Segment angles of stance leg
Slow
Thigh	8.81	9.38	8.34	5.27	8.58	9.32	8.03
Shank	3.17	4.06	3.01	2.12	3.25	3.98	6.24
Foot	3.49	3.84	3.48	4.03	3.02	4.24	6.31
Moderate
Thigh	5.45	10.04	8.19	5.26	9.48	7.64	7.86
Shank	2.58	4.25	2.94	3.16	3.10	2.12	7.48
Foot	4.06	3.82	4.21	3.80	3.31	3.47	7.04
Fast
Thigh	3.81	7.27	5.82	6.87	11.96	7.47	9.87
Shank	2.42	3.71	2.76	3.70	3.60	3.61	4.40
Foot	2.94	3.65	3.59	5.25	4.46	5.31	4.43
Segment angles of swing leg
Slow
Thigh	9.91	8.99	10.55	5.57	8.24	10.36	12.33
Shank	7.92	5.25	5.97	4.43	3.85	6.78	6.51
Foot	5.99	5.07	5.29	5.58	4.71	8.85	9.53
Moderate
Thigh	6.41	10.39	10.31	6.17	8.38	7.48	13.13
Shank	5.39	5.05	6.50	4.12	6.00	3.90	6.83
Foot	5.02	5.41	5.63	4.83	5.97	5.93	8.43
Fast
Thigh	4.21	8.00	8.31	7.55	9.53	6.45	11.53
Shank	4.15	4.94	7.89	3.72	4.64	3.99	5.63
Foot	4.23	5.20	5.65	5.56	7.46	6.16	5.22
Joint torques of stance leg
Slow
Hip	11.27	8.45	13.55	12.65	9.44	13.46	12.85
Knee	10.81	11.34	8.70	9.51	10.47	13.43	9.78
Ankle	8.58	8.90	7.65	8.33	6.44	15.62	11.88
Moderate
Hip	11.67	8.28	12.12	11.39	11.00	10.30	10.44
Knee	11.48	9.44	8.41	8.10	11.43	8.59	10.00
Ankle	7.96	8.99	8.33	8.11	7.41	11.94	11.97
Fast
Hip	11.67	9.79	10.75	11.80	11.59	9.52	9.44
Knee	7.67	8.28	10.04	7.36	16.50	6.06	9.39
Ankle	9.51	8.53	7.35	8.32	11.81	8.58	11.49
Ground reaction forces
Slow
Vertical	8.02	6.02	7.66	4.49	5.30	12.24	3.87
A–P	11.00	4.57	4.95	5.14	6.56	8.79	4.38
Moderate
Vertical	7.65	6.63	7.29	4.22	6.82	6.21	5.01
A–P	8.95	4.92	4.98	5.38	8.41	5.78	4.70
Fast
Vertical	6.10	6.75	8.55	5.19	15.77	5.91	9.21
A–P	5.22	5.80	5.90	5.93	13.43	4.95	5.64

and

Table A2. R² values of each subjects.

Subject	1	2	3	4	5	6	7
Angles of stance leg
Slow
Thigh	0.92	0.89	0.93	0.97	0.92	0.91	0.94
Shank	0.98	0.97	0.98	0.99	0.98	0.97	0.91
Foot	0.96	0.96	0.96	0.95	0.98	0.93	0.88
Moderate
Thigh	0.97	0.88	0.93	0.94	0.91	0.94	0.94
Shank	0.99	0.97	0.98	0.98	0.98	0.99	0.88
Foot	0.95	0.96	0.95	0.96	0.97	0.96	0.86
Fast
Thigh	0.99	0.95	0.97	0.96	0.85	0.95	0.91
Shank	0.99	0.98	0.99	0.98	0.98	0.97	0.96
Foot	0.98	0.97	0.97	0.94	0.96	0.94	0.95
Angles of swing leg
Slow
Thigh	0.90	0.91	0.90	0.97	0.93	0.90	0.87
Shank	0.95	0.98	0.97	0.98	0.99	0.96	0.96
Foot	0.96	0.97	0.97	0.97	0.98	0.91	0.91
Moderate
Thigh	0.96	0.88	0.89	0.97	0.93	0.95	0.85
Shank	0.98	0.98	0.96	0.99	0.97	0.99	0.96
Foot	0.97	0.97	0.97	0.98	0.97	0.96	0.93
Fast
Thigh	0.98	0.94	0.93	0.95	0.89	0.96	0.87
Shank	0.99	0.98	0.95	0.99	0.98	0.99	0.97
Foot	0.98	0.98	0.97	0.97	0.95	0.97	0.98
Joint torques of stance leg
Slow
Hip	0.42	0.76	0.31	0.45	0.69	0.22	0.42
Knee	0.80	0.73	0.85	0.84	0.79	0.70	0.81
Ankle	0.91	0.90	0.93	0.91	0.96	0.71	0.78
Moderate
Hip	0.41	0.75	0.37	0.56	0.56	0.54	0.58
Knee	0.78	0.72	0.86	0.88	0.71	0.87	0.75
Ankle	0.92	0.88	0.92	0.90	0.93	0.85	0.72
Fast
Hip	0.50	0.72	0.67	0.56	0.68	0.68	0.71
Knee	0.91	0.87	0.82	0.92	0.48	0.94	0.81
Ankle	0.87	0.90	0.92	0.88	0.83	0.90	0.76
Ground reaction forces
Slow
Vertical	0.88	0.82	0.87	0.95	0.94	0.70	0.96
A–P	0.79	0.96	0.95	0.95	0.92	0.84	0.96
Moderate
Vertical	0.87	0.90	0.88	0.95	0.90	0.82	0.93
A–P	0.86	0.95	0.96	0.95	0.87	0.94	0.96
Fast
Vertical	0.91	0.91	0.83	0.93	0.49	0.91	0.82
A–P	0.95	0.94	0.94	0.93	0.68	0.96	0.93

in the Appendix A. The lower-limb joint kinematics and kinetics at various gait speeds for all seven subjects were fairly well predicted from the kinematics of the sacrum, such as the displacement, velocity, and acceleration, with an average NRMSE of approximately 7% (Figure 5 and Figure 6,

Table 4. Percentage of NRMSE values of all subjects as a function of the gait speed.

	Speed	Slow	Moderate	Fast
Segment angles of stance leg	Thigh	8.25 ± 1.40	7.70 ± 1.82	7.58 ± 2.66
	Shank	3.69 ± 1.30	3.66 ± 1.81	3.46 ± 0.66
	Foot	4.06 ± 1.07	4.24 ± 1.27	4.23 ± 0.89
Segment angles of swing leg	Thigh	9.42 ± 2.13	8.89 ± 2.52	7.94 ± 2.31
	Shank	5.82 ± 1.41	5.40 ± 1.13	4.99 ± 1.43
	Foot	6.43 ± 1.93	5.89 ± 1.20	5.64 ± 1.00
Joint torques of stance leg	Hip	11.67 ± 2.03	10.74 ± 1.26	10.65 ± 1.06
	Knee	10.58 ± 1.53	9.63 ± 1.40	9.33 ± 3.42
	Ankle	9.63 ± 3.12	9.24 ± 1.91	9.37 ± 1.68
Ground reaction forces	Vertical	6.80 ± 2.85	6.26 ± 1.24	8.21 ± 3.63
	A–P	6.49 ± 2.51	6.16 ± 1.76	6.70 ± 2.99

). The segment angles of the stance and swing leg showed a reasonably good match between the prediction and experimental data, even for the worst case test set (Figure 6A,B,

Table 5. Percentage of NRMSE values of specific subjects with different error levels.

	Error	Minimum	Median	Maximum
Segment angles of stance leg	Thigh	5.80 ± 0.93	7.45 ± 1.41	8.59 ± 1.11
	Shank	2.99 ± 0.80	2.90 ± 0.13	6.04 ± 1.55
	Foot	4.36 ± 0.78	3.76 ± 0.39	5.93 ± 1.35
Segment angles of swing leg	Thigh	6.43 ± 1.02	9.72 ± 1.23	12.33 ± 0.80
	Shank	4.09 ± 0.36	6.79 ± 0.99	6.32 ± 0.62
	Foot	5.33 ± 0.43	5.53 ± 0.20	7.73 ± 2.24
Joint torques of stance leg	Hip	11.95 ± 0.64	12.14 ± 1.40	10.91 ± 1.75
	Knee	8.32 ± 1.09	9.05 ± 0.87	9.72 ± 0.31
	Ankle	8.25 ± 0.12	7.78 ± 0.50	11.78 ± 0.25
Ground reaction forces	Vertical	4.63 ± 0.50	7.84 ± 0.65	6.03 ± 2.82
	A–P	5.49 ± 0.41	5.28 ± 0.54	4.91 ± 0.65

), whereas the joint torques and GRFs showed larger error, especially for the estimation of the high-frequency component of the hip joint torques (Figure 6C,D, Table 5). The average prediction errors also showed a similar trend in the prediction accuracy (Figure 5, Table 4). As long as the displacement information of the sacrum is provided, the velocity and acceleration input to the ANN do not significantly improve the prediction accuracy, whereas the omission of the displacement input significantly (p < 0.05) increases the prediction errors of the segment angles, torques and GRFs up to 3% (Figure 7,

Table 6. Percentage of NRMSE values of different kinematics combinations.

Inputs	x, t	x, v, t	v, t	x, v, a, t	v, a, t
Segment angles of stance leg	5.20 ± 2.42	5.23 ± 2.69	7.62 ± 2.71	5.21 ± 2.38	6.34 ± 2.71
Segment angles of swing leg	7.22 ± 2.19	6.76 ± 2.30	8.44 ± 2.81	6.71 ± 2.25	8.37 ± 2.88
Joint torques of stance leg	10.04 ± 2.64	10.44 ± 2.63	12.25 ± 3.57	10.09 ± 2.10	11.56 ± 3.91
GRF	6.95 ± 2.58	6.76 ± 2.31	10.18 ± 3.17	6.77 ± 2.55	8.06 ± 2.34
Total	7.39 ± 3.04	7.35 ± 3.19	9.57 ± 3.57	7.24 ± 2.95	8.63 ± 3.63

). These results imply that the position of the sacrum contributes most to the predictions of the lower-limb joint kinetics.

The ANN predicted the kinematics and kinetics of the lower limb at various gait speeds that were not used for the training datasets. The joint kinematics and kinetics are known to vary as a function of the walking speed. To examine the effect of the walking speed at which the training data sets are collected to the prediction accuracy, the ANN was trained by the data collected at three different speed conditions: at moderate speed only, at slow and moderate speeds, and at slow, moderate and fast speeds (Figure 8). The results showed that the inclusion of the data of a specific gait speed, which are to be predicted through the ANN, would slightly but not significantly increase the prediction performance. More importantly, the sacrum kinematics measured at the moderate speed could predict the eleven unmeasured joint kinetics at various walking speeds, which were not included in the training of the ANN (Figure 8). The prediction results of the joint torques and GRFs at the various gait speeds obtained from the ANN trained by trials of moderate speed showed a reasonable match with the data (Figure 9,

Table 7. Percentage of NRMSE values of ANN trained with moderate speed only.

	Speed	Slow	Moderate	Fast
Joint torques of stance leg	Hip	13.41 ± 2.91	11.42 ± 0.65	11.07 ± 1.41
	Knee	11.35 ± 2.64	10.11 ± 2.23	9.92 ± 2.94
	Ankle	10.38 ± 3.94	9.30 ± 2.29	9.41 ± 1.95
Ground reaction forces	Vertical	6.64 ± 2.00	6.84 ± 1.30	7.93 ± 1.61
	A–P	6.24 ± 2.05	5.93 ± 0.85	7.09 ± 2.83

).

4. Discussion

Biomechanical knowledge about walking dynamics was used to design an ANN to predict unmeasured motion data from a single IMU measurement. Equations related to CoM dynamics were reformulated to approximate the GRF and lower limb joint dynamics as the weighted sum of the CoM, the basic input–output relationship of the ANN. Based on this approximation, the ANN was designed to have input nodes receiving the kinematics of the CoM, with the layers being fully connected. Using this ANN, eleven lower limb joint dynamics and the GRF data at various walking speeds were predicted from a single and only IMU measurement near the sacrum. The predicted segment angles, joint torques, and GRF data showed a reasonably good match with the experimental data despite the very simple network structure (Figure 1) with a relatively small number of training data sets (Figure 5, Figure 6, Figure 7 and Figure 8). Although trained with the CoM data measured at a specific gait speed, the ANN could predict multiple joint kinetics at various gait speeds that have not been used for network training. These results imply that the CoM is important as a dynamic determinant of multi-segment kinetics during walking, and that the tradeoff between data quantity and wearable convenience can be solved by utilizing a machine learning algorithm based on the dynamic characteristics of human walking.

Based on a biomechanical analysis using 2D spring mechanics, we showed that the 2D acceleration data of the CoM could predict the GRFs and joint dynamics in 2D fairly well. Due to the point mass assumption of the CoM, no rotational motion of the CoM was assumed, and no gyroscope data were used for prediction. We may improve the quality of the prediction heuristically with the full usage of IMU sensors, such as 3D accelerations and 3D angular rates from the gyroscope. We do not, however, have sound explanations as to why the addition of 3D data would help in obtaining better prediction. We are investigating whether the 3D GRF could be well formulated using 3D CoM kinematics as a form of simple mechanics, as was the case for 2D. Despite the lack of quantitative formulation, there is considerable experimental evidence to show that full usage of the IMU data could predict the unmeasured GRFs, metabolic cost, and joint angles [9,33,34]. Three dimensional acceleration and angular rate measured from the 17 IMUs attached to the whole body have been used to successfully predict 3D GRF data [9]. Similar to our study, the 2D joint angles of lower limbs have been estimated from IMU measurement at the foot, shank, and thigh [34]. The 3D acceleration and gyroscope data were processed using general regression neural networks and the authors found that foot acceleration was sufficient to predict the other joint angles. Despite the larger number of sensors, the prediction errors were slightly larger (4~9 degrees) than ours (2~3 degrees), implying that the CoM kinematics are closely related to lower limb kinematics. The predicted quantities were not limited to the GRF or joint kinematics. Metabolic cost is one of the key determinants for human motion [35,36,37,38]. Measurements of energy expenditure, however, are limited to indoor settings, and the measurement procedures are often tedious. Researchers attempted to estimate the metabolic cost during walking from IMU data measured at the hip joint using Bayesian linear regression [33]. With the support of quantitative biomechanical analysis of the target motion to be predicted, the IMU data could be used more effectively and efficiently. There are studies that use machine learning technology to predict human motion, independent of gait biomechanics, but often with the cost of increased measurement complexity of the prediction errors. Oh et al. successfully predicted the joint torques and GRFs of human walking using a general regression neural network (GRNN), with the cost of increased measurement of the whole body joint kinematics using an optical marker system [10], which is therefore limited for on-field applications. The errors of the vertical and A–P GRF were comparable to ours (

Table 8. Prediction errors of ground reaction forces (GRF) (previous studies and proposed method).

	S. E. Oh et al. (2013)		G. Leporace et al. (2018)		Proposed Method
Number of subjects	48		17		7
Measurement	11 Optical markers		2 IMUs at each shank		1 IMU at sacrum
Prediction method	GRNN		FFNN		FFNN
Prediction parameters	rRMSE (%)	ρ	MAD (%)	ρ	NRMSE (%)	ρ
Vertical GRF	5.8 ± 1.0	0.98	4.6 ± 0.7	0.97	6.26 ± 1.24	0.96 ± 0.03
A–P GRF	7.3 ± 0.8	0.97	4.0 ± 0.8	0.98	6.16 ± 1.76	0.98 ± 0.01
ML GRF	19.8 ± 2.2	0.92	10.5 ± 3.3	0.80	-	-

GRNN: General-regression neural network; FFNN: Feedforward neural network.

). Recently, without a great loss of prediction accuracy, the number of IMUs were reduced to two, attached at each shank of the subject [11]. To complement the reduced sensor information, a feed-forward neural network was used to produce the best estimate of the GRFs. In this recent study, however, the training datasets included part of the data from the test subject [11], which may have decreased the prediction accuracy when LOO validation was performed. Because the GRF pattern could differ between subjects [39,40], we used a conservative validation method, by excluding the test subjects’ data from the training dataset, and following the LOO validation method to guarantee generalization.

Machine learning techniques have been used to estimate joint angles and joint torques (

Table 9. Prediction errors of stance leg angle (previous studies and proposed method).

	A Findlow et al. (2008)		T.P. Luu et al. (2014)		Proposed Method
Number of subjects	8		17		7
Measurement	4 IMUs at each shank and feet		2 Gait parameters + 4 Anthropometric data		1 IMU at sacrum
Prediction method	GRNN		GRNN		FFNN
Prediction parameters (proposed)	MAD (degree)	ρ	MAD (degree)	ρ	RMSE (degree)	ρ
Hip (thigh)	8.64 ± 1.45	0.80 ± 0.05	3.73 ± 1.64	0.98 ± 0.03	3.14 ± 1.49	0.99 ± 0.03
Knee (shank)	7.14 ± 1.33	0.89 ± 0.05	5.41 ± 2.01	0.97 ± 0.04	2.17 ± 1.23	0.99 ± 0.00
Ankle (foot)	4.91 ± 0.76	0.75 ± 0.06	3.58 ± 1.44	0.92 ± 0.07	3.35 ± 1.58	0.99 ± 0.01

) [34]. Luu et al. used GRNN and inverse fast Fourier transforms [41]. The predicted joint angles of lower limbs showed relatively small errors, in the range of 3~5 degrees, compared to others, but the test data were included in the training data set, which would produce less-accurate predictions when LOO validation is performed. Joint torque predictions have been performed using joint angle data [42] as well as surface electromyography (sEMG) [43]. The errors of the joint torque predictions were larger than those of GRFs and/or joint angles, as was observed in our study (

Table 10. Prediction errors of joint torques (previous studies and proposed method).

	M. M. Ardestani et al. (2014)		M. Mundt et al. (2018)		Proposed Method
Number of subjects	4		12		7
Measurement	14 sEMGs		3D joint angles		1 IMU at sacrum
Prediction method	WNN		LSTM		FFNN
Prediction parameters	rRMSE (%)	ρ	MAD (%)	ρ	NRMSE (%)	ρ
Hip	6.42	0.93	18.15	0.97	10.74 ± 1.26	0.90 ± 0.04
Knee	4.30	0.98	13.50	0.93	9.63 ± 1.40	0.96 ± 0.03
Ankle	4.20	0.98	6.41	0.98	9.24 ± 1.91	0.98 ± 0.01

WNN: Wavelet neural network; LSTM: Long-short term memory recurrent neural network.

). Ardestani et al. used 14 sEMG data and showed relatively high accuracy in joint torque prediction [43], but, again, the inclusion of test data in the training dataset of the four subjects may contribute to a misleadingly high prediction accuracy. Compared to those of related works, we predicted more data at various walking speeds with fewer measurements from the complementary combination of the biomechanics and the ANN.

The proposed prediction method using a combination of biomechanics and machine learning could be used to resolve the tradeoff between data richness and measurement convenience inherent in wearable technologies. The rehabilitation success of stroke patients can be assessed using the A–P direction of the GRF of the nonparametric leg [6,7,8], which would be challenging to measure if patients have to regularly visit a motion analysis laboratory and undergo almost over an hour of data collection procedures. Motion monitoring of those patients using simple wearables during their clinical visits or even at home in daily life would greatly increase patients’ convenience, if the data from wearables could produce data of quality and quantity comparable to those obtained in the lab. A recent study demonstrated successful long-term monitoring of the range of motion (ROM) of the shoulders of elderly people using the IMU, which showed a fairly good match with data from optical motion capture systems [44]. Similarly, onsite monitoring of lower limb joint loading could be estimated from a single IMU sensor for patients with rheumatoid arthritis or anterior cruciate ligament recovery. As a quantitative comparison, the asymmetry of the GRFs of mild hemiparesis patients seemed to be approximated as 50 N [7]. Kesar et al. proposed minimal detectable changes of the kinematic and kinetic indicators of rehabilitation assessment of post-stroke patients to be about 3–10 degrees and 20–33 N, respectively [45]. In our study, the average prediction errors of segment angles are about (3.1 degree) thigh, (2.2 degree) shank and (3.4 degree) foot, respectively. The approximate errors of vertical and AP GRF are 58 N and 23 N, and those of hip, knee, ankle joint torques are 16.7 Nm, 11.4 Nm, and 15.3 Nm, respectively. Considering the observed changes of GRFs and joint kinematics of stroke patients, the proposed method could serve well for kinematics monitoring, whereas the GRF monitoring seemed to be feasible for A–P GRF but need improvement for vertical GRF. In addition to the rehabilitation research, the number of studies monitoring sports activity is rapidly increasing. Wearable monitoring systems are widely used for monitoring the effect of posture or loading conditions on kinetics [46]. Data from sacrum-mounted IMUs were used to analyze the effect of load carriage and fatigue on jumping motions. The change of kinematics due to fatigue was similar to that under load conditions. IMUs on the lower body were used to estimate joint kinematics and GRF during running [47,48]. Combining single IMU data from the sacrum with a convolutional neural network, Johnson et al. estimated 3D GRF with the error of 4–9% during running and side-stepping [14]. A leg-mounted IMU combined with a feed forward neural network predicted knee joint forces with an error rate of 17% during running, changing direction, and jumping [49]. Issues to be addressed for sports wearables include reducing the number of sensors and enhancing prediction accuracy. The difference in kinetics with different foot strike patterns in running is about 10%–20% of joint torque and 30% of GRF. [50]. A recent study showed that a magnitude of about 30% of GRF in response to the breaking force could serve as an indicator for injury risk [51]. By assuming reference force and length to be the average weight and height of our test subjects, the observed kinetic differences range approximately 200 N and 18 Nm, respectively. Considering the error range of our study, the proposed prediction method can be applicable for monitoring running.

The limitations of our study, and technical issues to be resolved, lie in the main components of the prediction method, such as the biomechanical model, the processing of the IMU data, and the design of the ANN. First, the proposed prediction method utilized a spring-based biomechanical model of CoM dynamics during steady gait trials in 2D. Pathological gaits, however, often show increased asymmetry in the medial–lateral (M–L) as well as the A–P direction [52]. Thus, further research is required into applying the proposed method to the monitoring of unsteady or abnormal walking in 3D. Biomechanical modeling of human walking with a passive spring and asymmetric foot could well emulate the human-like ankle joint torque [29], implying that the CoM and ankle joint torque are mechanically coupled. The biomechanical model of the point mass CoM, however, could not represent the activation of hip joint torques, so there is no mechanical and kinematical correlation between the CoM motion and the hip joint torques. Since the ANN used for joint torque prediction was designed based on the biomechanics of the CoM, the larger prediction errors in hip joint torque presented in Figure 9A could be attributed to the limited representability of the hip joint torque by the CoM. Secondly, one of technical challenges of handling IMU data is the determination of appropriate and accurate segmentation. For long-term IMU data from motion monitoring, the very first step to perform in signal processing is to extract meaningful data from a whole data collection. A recent study used GPS information to specify the data segment of the action of interest out of approximately two weeks of IMU data in daily life [24]. To expand the proposed prediction method to gait monitoring during daily life, an appropriate activity recognition and extraction process must be undertaken. Once the data were roughly segmented for a specific action, such as walking in the hallway, further segmentation was carried out based on specific gait events, such as HS or TO. IMU data gathered near a ground contact such as the shank or foot had more accurate event detection results than when the IMU was mounted near the sacrum or lower back [53,54] possibly due to the distal measurement of the indicating signal around the foot [54,55,56]. Similar to previous studies, the identification of gait events and segmentation of the data were performed in a heuristic manner, but in a simpler way than those presented in other studies, such as those using wavelet transforms [57]. For example, the filtering frequency of the IMU data was selected heuristically by choosing the one that did not smooth the indicating signal of the gait event, while not including unnecessary peaks. The proposed event detection method produced errors comparable to those of previous studies, ranging from approximately 0.01–0.03 seconds [53,54]. The detection errors were reduced with gait speed due to the highlighted singular motion of the sacrum at faster speeds [28]. The gait event detection mostly follows the real gait event due to the delayed sacrum acceleration profile compared to that of the true CoM acceleration, which is obtained from the GRF, and resulted in shifted estimation of sacrum trajectory. Likewise, although small in magnitude, event detection errors could greatly affect the prediction results when erroneously segmented IMU data are used for training an ANN. An ANN is highly dependent on its input information, so poorly-estimated sacrum kinematics cause a deterioration in prediction accuracy. Thus, the prediction performance is sensitively dependent on the signal quality of the IMU and the following signal processing. Considering that supervised learning is performed between the segmented IMU data and the segmented GRF, erroneously segmented IMU data distort the coupling relationship between the CoM and the GRF, and the network will be trained inaccurately and inefficiently. Further, there was a relatively large error (~25%) in the velocity estimation (Figure 4). The integration of the acceleration had drift removal and offset compensation issues, which were resolved in a heuristic manner. Thorough examination of the best estimation of the gait speed was not performed in this study, so the error rate could potentially be reduced by employing other estimation algorithms or machine learning techniques. The effects of erroneous velocity estimation on the prediction results, however, are reduced when the velocity is fed into the ANN after normalization. In addition, the validation conducted depends on the location of IMU. Since the trunk has rotational motion with respect to the vertical axis, the location of IMU in the horizontal axis can affect the prediction performance. Thus, the sensitivity analysis about the location of IMU should be performed for validation of the reliable wearable motion monitoring system. Lastly, the proposed ANN has much room for improvement of the prediction accuracy. The proposed ANN is a very simple network, designed using a simplified mathematical approximation between the GRFs and the CoM. A more realistic approximation could be achieved by adding complexity to the network, such as additional input nodes or hidden layers, or a change of activation function. A relatively small number of training datasets, and reasonably good test results compared to previous studies with more sensor data could support an efficiently trained ANN. However, a small training data set could result in the limited performance accuracy of the network, which was observed by an increase error in response to variations of the input. Since the GRF data are known to have high intra-subject variability, training with a larger dataset should produce improved prediction results for diverse test subjects.

Biomechanical knowledge about walking dynamics was used to design the ANN that predicts the unmeasured motion data from only a single IMU measurement. From the sacrum-mounted IMU data, the motion and force information of multiple joints were well predicted at various gait speeds by a simple ANN with a relatively small number of training datasets. This is attributed to the role of the CoM as a key dynamic descriptor of multi-segment lower limb coordination during the human walking behavior. The reliable prediction performance presented in this study implies the complementary roles of biomechanics domain knowledge and machine learning technology in predicting motion kinetics.