1. Introduction
In recent years, human activity recognition and pedestrian dead reckoning using inertial sensor-based wearable devices have received much attention of researchers to support human life [
1,
2,
3,
4,
5,
6,
7,
8,
9,
10]. Using more wearable devices is synonymous with deploying more sensors. However, this is inconvenient for users if they simultaneously use many devices and perform various daily activities [
11]. The main problem is that researchers must find a robust algorithm with high performance using the smallest number of sensors to provide the most convenience for users.
Some studies estimated the walking distance by considering a small number of user’s walking modes and device poses [
5,
10,
12]. Phuc et al. [
12] presented the precise stride counting-based method to estimate the walking distance using insole sensors. The insole sensors consisted of a triaxial inertial sensor and eight pressure sensors. The authors estimated the traveling distance based on the number of strides extracted from the phase information. However, they only considered the walking distance estimation of normal walking on flat ground. Lee et al. [
5] introduced a robust step detection algorithm for three step modes and seven device poses of the smartphone. The step detection used an adaptive magnitude and temporal thresholds which addressed the transition among step modes or device poses and the time-varying pace of human walking or running problems. The developed method can detect the number of steps for any combination of step mode and device pose. Ho et al. [
10] developed a method of walking distance estimation based on an adaptive estimator of the step length and robust step detection. The presented method successfully estimated the traveling distance at three speed levels and four different distances. Furthermore, the step-length estimator, which was an improvement of Weinberg equation [
13], used an adaptive
K-value as a linear regression model.
In common approaches, all processed activities data are directly fed to an adaptive step detector without classifying the performing activities [
5,
6]. However, it is more effective if the activities are classified because the thresholds of the acceleration values depend on the type of activities. To achieve high accuracy in estimating the traveling distance with various actions, e.g., texting, calling, and swinging, some studies [
9,
14,
15] addressed the problem by implementing classifiers or improving the step detection algorithms before estimating the distance. Susi et al. [
9] proposed an adaptive step detection by analyzing the characteristics of the gait cycle, which included the hand motion and carrying-mode difference of a pedestrian using a smartphone. The authors detected the motion modes, e.g., swinging, texting, phoning, bag and irregular motion, before applying the step detection algorithm on the collected inertial signals. Renaudin et al. [
14] estimated the step length using a handheld sensor, which was an extended idea from [
9]. The presented step detection algorithm used the step frequency, height of the pedestrian, and three variables to estimate the step frequency of non-body fixed sensors. Zhang et al. [
15] designed an inertial pedestrian navigation system (IPNS) based on the improvement of the step mode and device pose algorithm using a low cost hand-held device. The step detection algorithm addressed the over-counting and under-counting errors by implementing a support vector machine that was used to recognize step modes and device poses.
The aforementioned studies do not account for the errors caused by the classifiers and step detectors. Therefore, the step detector can make a serious error, where it attempts to detect the steps of a calling or texting activity, which is classified as hand swinging and vice versa. The system can give an exception message when we do not handle these errors. Specifically, the accuracy rates of the walking distance estimator significantly decrease. Thus, in the paper, we propose a new method that uses a smart band to estimate the walking distance based on a robust step detection and an adaptive step length estimation for five daily wrist activities during walking: phone texting, phone calling, hand in pocket, suitcase carrying and hand swinging. The performance of step detection and traveling distance estimation can be improved by applying classifiers, robust step detectors, and the error feedback technique. The activity samples are classified and labeled by support vector machine (SVM) classifiers. A 2-s window of the preprocessed data is used to obtain features that are fed to the classifiers. The step detector used adaptive thresholds each activity. Basically, activity samples can be classified two times before being fed into the step detectors. The movement distances are estimated by summing the length of all walking steps. Furthermore, the step length equation is constructed based on a non-parametric regression of an average magnitude of tri-axial velocities and a set of variables. The contributions of this paper are as follows:
Developing a hierarchy framework of the walking distance estimation for five daily living activities: phone texting; phone calling; hand in pocket; suitcase carrying; hand swinging.
Proposing a robust step detection algorithms using an adaptive threshold.
Improving the step detectors and traveling distance estimators using error feedback.
Developing the step-length estimation based on non-parametric regression.
Estimating and comparing the performance of each walking distance estimator with various activities and speed levels.
This paper is organized as follows. In
Section 2, we describe the hierarchical framework of the walking distance estimation in details.
Section 3 shows the results of our method in three parts: activity classification, step detection, and walking distance estimation. Finally, in
Section 4, we conclude the paper and provide directions for future works.
3. Experimental Results
3.1. Activity Classification
As mentioned, to collect sufficient data to assess the performance of our proposed method, ten participants were requested to perform five daily wrist activities in 20 m of walking at different levels of speed. We used a confusion matrix to estimate the performance of the classifiers in
Table 3 (classifier 1) and
Table 4 (classifier 2). As described in these tables, the first column lists the performed activities by the participants, and the first row lists the predicted activities by the classifiers.
In
Table 3, the swing activity is 100% correctly predicted. As mentioned, the swing acceleration data is significantly different from other cases. In addition to the up and down actions of the hip, the forward and backward actions of the arms also affect the acceleration data. This characteristic makes the swinging activity different from the other activities. The accuracy of predicting texting/calling/hand in pocket/suitcase carrying is 99%. The first classifier incorrectly predicted 1% of them as swinging. Texting, calling, hand in pocket, and suitcase carrying are center-of-mass motions, and the acceleration is generated by the up and down actions of the hip, but, in some cases, the arm slightly moves because of the inertia of fast walking. In this situation, texting/calling/hand in pocket/suitcase carrying is identical to swinging at a slow speed, so the classifier failed to classify these activities. All activities that are predicted as swinging are the input of the swinging step detector. The 1% incorrectly predicted activity is rechecked in the step detector and returned to the second classifier.
The classes (texting, calling, hand in pocket and suitcase carrying) from the first classifier are the input of the second classifier. The confusion matrix is provided in
Table 4. The hand in pocket is perfectly predicted. The calling and suitcase carrying are 2% incorrectly predicted as texting and calling, respectively. The texting is 1% incorrectly classified as hand in pocket. Those errors affect the performance of the step detector but in acceptable amounts because all activities are one peak between two valleys.
3.2. Step Detection
Classified data are fed to the step detector, which has five different reference and adaptive thresholds for five walking activities. The misclassifications of the first classifier are returned and corrected. The step detection algorithm is affected by the wrong classification of the first and second classifier.
Figure 9 illustrates the accuracy and standard variance of the step detection between with and without misclassification correction for each walking activity. As shown in the figure, the accuracy of the step detection algorithm with misclassification correction is higher than without misclassification correction in the calling, suitcase carrying and swinging. This is because the 1% error of predicted swinging activity in the first classifier is returned to the second classifier. For step detection with misclassification correction, the accuracy of each walking activity is higher than 98% and the highest standard deviation is 3%.
We must emphasize that, for each walking activity, the vertical acceleration data change among the x, y and z-axes. It is difficult for the step detection and distance estimation algorithm when we use the vertical acceleration data as a fixed axis. The classification data solve these problems by classifying the activity and using corresponding vertical data of that activity. In addition, the adaptive threshold also renders the step detection performance.
3.3. Walking Distance Estimation
To evaluate the performance of the proposed method, the Leave-One-Sample-Out technique, which makes one trial a test set and the remaining trials the training set in each epoch, was applied to the classified activity data. This technique is commonly used for small datasets [
35]. We derived the
K-factor as a
p-degree polynomial function of the velocity feature. In the experiment, we examined various values of
p to minimize the estimation error. The polynomial degree
p of the
K-factor, which was implemented for three methods [
13,
29,
30], was four.
The walking speeds are: low speed (
), normal speed (
) and high speed (
). Here,
is the average speeds and
is the deviation of human walking speed [
10]. The performance (accuracy, standard deviation (Std) and normalized mean square error (NMSE)) of each traveling distance estimator considering the activities and walking speed is presented in
Table 5.
All three proposed methods estimate the walking distance in the texting activity efficiently, and the performance is best when the person walks at high speed (the accuracy is more than 97.91%). The average distance accuracy is higher than 97% for low, normal and high speeds, the normalized mean square error is 1.19, and the standard variance is acceptable (below 0.92 m). Otherwise, with the calling case, the estimated distance at high walking speed is lower than that at normal and low walking speeds. For the other case, the accuracy does not depend on the walking speed but on the deployed method. For example, in the hand-in-pocket case, the highest accuracy is 97.89% using the non-parametric Kim method at high walking speed, and the worst accuracy is 93.33% using the non-parametric Tian method at low speed. The swinging case has a larger standard variance than the other activities due to the change in vertical acceleration as a result of both arm swinging and hip moving during walking.
One of our objectives is to select the best distance estimators that can be stable and achieve high accuracy for each hand daily activity using a smart band. The non-parametric Tian estimator was implemented to estimate the walking distance of texting, calling, and suitcase-carrying activities. In addition, the non-parametric Kim estimator was used for the hand-in-pocket and swinging activities.
According to
Table 6, the proposed method achieved an average accuracy of 96.9%, whereas that of the reference method is 95.1%. The calling-during-walking experiment is the most unstable because of different arm gestures of phone call and arm fatigue during the experiments. The smallest and largest gaps of accuracy between the proposed method and reference method were found for the texting and hand in pocket activities, respectively.
According to
Figure 10, the proposed and reference methods suffer low accuracy and high standard deviation for the suitcase-carrying and calling activities, respectively. Overall, an aspect of the proposed method that used walking distance estimators with each activity can surpass the reference method in terms of accuracy and standard deviation.
4. Conclusions
In this paper, a step detection algorithm and a walking distance estimation based on daily hand activity recognition using the smart band have been presented and experimentally evaluated. Five daily hand activities during walking were considered: phone calling, phone texting, hand in pocket, suitcase carrying and swinging. Each hand activity has different vertical acceleration data, and changing the vertical acceleration data of the smart band is the main challenge of the distance estimation. Therefore, two SVM classifiers are used to classify and let the step detector and distance estimator know the activity that is processed. In addition, the classification is processed in two steps to improve the robustness of the step detection and walking distance estimation by feedback data of the wrong candidates. The new step detection and distance estimation algorithm using the smart band have been presented. To evaluate the performance of the proposed method, experiments of 20-m walking while performing daily hand activities using the Microsoft smart band were conducted with ten participants. The accuracy of this classification was above 99% for all activities of both classifiers. With prior knowledge about the data being processed, the adaptive threshold strategy of the step detection algorithm is effectively performed. The error of misstep detection is approximately 2%. The experiment results also show the performance of three non-parametric methods, and we compared the performance of the walking distance estimation algorithm with the reference method. The result shows that the proposed method has outstanding accuracy and robustness.
In the proposed method, a post-hoc analysis has been applied. For real applications, real-time processing algorithms will be required. Also, to enhance the performance of estimation, we should consider other daily living activities. These remain future work.