Next Article in Journal
Characterization of Biodegraded Ignitable Liquids by Headspace–Ion Mobility Spectrometry
Next Article in Special Issue
Feasibility of Smartphone-Based Badminton Footwork Performance Assessment System
Previous Article in Journal
A 5.86 Million Quality Factor Cylindrical Resonator with Improved Structural Design Based on Thermoelastic Dissipation Analysis
Previous Article in Special Issue
Does the Position of Foot-Mounted IMU Sensors Influence the Accuracy of Spatio-Temporal Parameters in Endurance Running?
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Energy-Efficient Wearable EPTS Device Using On-Device DCNN Processing for Football Activity Classification

1
Department of Electrical Engineering, Pohang University of Science and Technology (POSTECH), 77 Cheongam-ro, Pohang 37673, Korea
2
Institute of Artificial Intelligence, Pohang University of Science and Technology (POSTECH), 77 Cheongam-ro, Pohang 37673, Korea
3
Sports AIX Graduate Program, Pohang University of Science and Technology (POSTECH), 77 Cheongam-ro, Pohang 37673, Korea
*
Author to whom correspondence should be addressed.
Submission received: 16 September 2020 / Revised: 13 October 2020 / Accepted: 20 October 2020 / Published: 22 October 2020
(This article belongs to the Special Issue Technologies for Sports Engineering and Analytics)

Abstract

:
This paper presents an energy-optimized electronic performance tracking system (EPTS) device for analyzing the athletic movements of football players. We first develop a tiny battery-operated wearable device that can be attached to the backside of field players. In order to analyze the strategic performance, the proposed wearable EPTS device utilizes the GNSS-based positioning solution, the IMU-based movement sensing system, and the real-time data acquisition protocol. As the life-time of the EPTS device is in general limited due to the energy-hungry GNSS sensing operations, for the energy-efficient solution extending the operating time, in this work, we newly develop the advanced optimization methods that can reduce the number of GNSS accesses without degrading the data quality. The proposed method basically identifies football activities during the match time, and the sampling rate of the GNSS module is dynamically relaxed when the player performs static movements. A novel deep convolution neural network (DCNN) is newly developed to provide the accurate classification of human activities, and various compression techniques are applied to reduce the model size of the DCNN algorithm, allowing the on-device DCNN processing even at the memory-limited EPTS device. Experimental results show that the proposed DCNN-assisted sensing control can reduce the active power by 28%, consequently extending the life-time of the EPTS device more than 1.3 times.

1. Introduction

Since the German national football (The term football in this work refers to soccer or association football.) team dominated the World Cup in 2014 with various IT technologies [1], the use of an electronic performance tracking system (EPTS) has been gaining huge popularity in the football industry and now has been standardized by FIFA to be used even at the international matches [2]. Basically, an EPTS device can be attached to the body of football players during the match (or training) time, sensing a number of data related to the athletic and strategic performances. As exemplified in Figure 1, the EPTS device may have a positioning system assisted by numerous calibration sensors [3,4,5], collecting quantitative data such as total distance covered, peak/average speed, or other physiological data. These on-site measurements can be used for analyzing the physical workloads of each player, providing valuable insights to optimize the performance [6,7]. Considering the agile movements of football players, the accurate positioning solution is one of the key technologies, which can be realized by an optical-based tracking system (OTS), a local positioning system (LPS), or a global navigation satellite system (GNSS) [8]. While the OTS and LPS show a high sampling rate and measurement accuracy, both systems typically require high-cost calibration infrastructures installed around the stadium [8]. On the other hand, the GNSS-based EPTS device is usable in any open field, whereas the position and speed are directly measured using satellite signals, as shown in Figure 1. Due to the intrinsic errors of the GNSS module [9,10], the MEMS-based inertia measurement unit (IMU) module is in general integrated into the GNSS-based EPTS device to provide more accurate information [11,12,13].
Even though the calibration schemes are essentially applied for minimizing the existing measurement errors, many researches have revealed that the accessing rate of GNSS dominantly determines the measurement accuracy of wearable EPTS devices [7,14,15]. Therefore, the current wearable EPTS devices frequently perform the GNSS sampling operations to retain the validity and the reliability of measurements. For example, the state-of-the-art wearable EPTS device, which delivers comparable accuracy to the LPS (or OTS) solutions, includes a 10 Hz GNSS receiver for the main localizing operation and also utilizes the IMU module running at a speed of 100 Hz for the proprietary filter algorithm [15,16]. As activating the GNSS module, as well as other assisting sensors consumes a huge amount of energy, the contemporary approaches typically suffer from the battery overheads to support the required operating time, increasing the device form factor accordingly. Note that the related standard clearly indicates the minimum safety requirement of wearable EPTS devices [17], and it is widely reported that the small form factor of the EPTS device is essential for preventing unwanted injuries [18]. Despite the quantitative feedback to players, as a result, many players still disapprove of using this technology at the important matches due to the uncomfortable experience. Hence, improving the energy efficiency of wearable EPTS devices without degrading the accuracy performance is still an open problem to reduce the required battery size and will play an important role in overcoming the limitations of current approaches in real environments.
In this paper, we propose a novel approach to reduce the overall energy consumption of the wearable EPTS device. To realize the experimental environments, we first design a prototype of the GNSS-based EPTS device, and then, the adaptive GNSS control method is newly introduced by categorizing the football movement types of players using a deep convolutional neural network (DCNN). Considering the tight memory size of the EPTS device, the previous DCNN model for recognizing the human activities is compressed to remove as many of the redundant parameters as possible, allowing the on-device DCNN processing by consuming negligible energy at the embedded processor. Experimental results show that the proposed idea significantly reduces the number of GNSS accesses while supporting similar distance and speed measurement errors compared to the baseline operations, consequently saving the overall energy consumption of the EPTS device by 28%.
The rest of this paper is organized as follows. Section 2 presents the detailed architecture of the prototype EPTS device, and the proposed energy-aware sensing-rate control method is introduced in Section 3. The experimental results under the field-level tests are shown and compared to the other works in Section 4. Finally, concluding remarks are made in Section 5.

2. Proposed Wearable EPTS Device

2.1. System Architecture

Figure 2 illustrates the block diagram of the proposed wearable EPTS device prototype that includes a GNSS module [19], an IMU module [20], a micro-controller unit (MCU) [21], NAND flash memory, and wireless communication modules using BLE5protocols, which is generally adapted by commercial EPTS devices [22,23,24]. For the baseline EPTS operation, the accessing frequency of GNSS module was set to 10Hz, measuring the position, as well as the speed of players every 100 ms by receiving the global positioning system (GPS) signal and the global navigation satellite system (GLONASS) signal concurrently. The nine axis MEMS-based IMU device consists of an accelerometer, a gyroscope, and a magnetometer, where the first two internal sensors generate the measurement data every 10 ms. The magnetometer in the IMU module is reserved for the future extension, and thus, we disabled the corresponding parts for low-energy operations. From the IMU measurements, the firmware program basically computes the total amount of acceleration by performing the attitude estimation filter [25]. Then, the linear Kalman filter is applied to compensate for the intrinsic speed errors by combining the measured speed from the GNSS unit with the estimated net acceleration [26]. Note that the sampling time of sensor measurements are synchronized at the firmware level to provide accurate filtering operations. In addition to the sensor-related EPTS operations, in the proposed EPTS device, the BLE5-based radio transceiver was adopted to realize the stable real-time tracking and the efficient remote control of the EPTS device with low energy consumption. When the EPTS cannot use wireless communication, the sensing data are temporally stored in the NAND flash memory of 512 MB, which is enough to keep all the data for the match time. To manage the overall EPTS operations, we also implemented the control firmware on the ARM Cortex-M4 MCU, which disables the inactivated components to reduce the power consumption in standby mode.

2.2. Evaluation of Baseline EPTS operations

Based on the system architecture shown in Figure 2, we realized a prototype EPTS device having a form factor of 53 mm × 32 mm × 9 mm , where the top and bottom views of the prototype are shown in Figure 3. As discussed in the prior subsection, the basic EPTS operation sets the GNSS and the IMU sensing rates to be 10 Hz and 100 Hz, respectively, whereas the embedded MCU operates at a speed of 80 MHz for performing the overall firmware programs managing sensors and data flows. Compared to our original EPTS solution described in [27], note that the proposed version reduces the device size by 40% through the compact design of the circuit board while providing measurements at the same sensing rates. The BLE5 unit continuously transfers the position, speed, and IMU measurements at a speed of 125 Kbps. Using the coin-sized 550 mAh battery, the baseline operation on the prototype EPTS device lasts about five hours, which is comparable to the commercialized products having similar form factors [22,23,24].
In order to evaluate the accuracy of the baseline EPTS operations, we performed the standardized testing scenario from [28]. As depicted in Figure 4a, more precisely, this trajectory testing includes different movements that occurred in football matches such as walking, running, and turning. For the given trajectory, Figure 4b shows the sampling positions from the prototype EPTS device by activating the GNSS module every 100 ms. The quality of the acquired data from the EPTS device was investigated by comparing with the ground truth using two metrics. To analyze the quality of the acquired information from the EPTS device, it is necessary to prepare the ground truth information of the location and speed at each sampling point, and then, two error metrics E d and E s are investigated to represent the errors of distance and speed, respectively [15,28,29]. For the sake of simplicity, we define d E and s E to denote the measured information of the distance between two adjacent samples and the speed value at each sample, respectively. Then, the error metrics E d and E s are calculated as follows:
E d = 1 n i = 1 n ( d E ( i ) d G ( i ) ) 2 , E s = 1 n i = 1 n ( s E ( i ) s G ( i ) ) 2 ,
where n is the total number of samples and d G and s G indicate the distance and the speed of the pre-defined ground truth, respectively. Similarly, d E and s E represent the measured data of the distance and speed from the EPTS device, respectively. As depicted in Figure 4, note that our prototype EPTS device supports quite accurate measurements of athletic movements, i.e., E d = 0.076 m and E s = 0.47 m · s 1 , which are comparable to the contemporary EPTS products targeting national-level football matches [22,23,24].
As the energy consumption of the EPTS device is directly related to the operating time, as well as the form factor, it is necessary to deeply understand the power consumption of each building block. For the quantitative analysis, we evaluated the energy consumption of the prototype EPTS device performing the baseline operations. Table 1 summarizes how the three major components dissipate the power, where the effects of other modules are negligible in terms of power consumption. It is clear that the GNSS module is the most power-hungry unit due to the frequent activation of RFIC circuits followed by the antenna component, consuming more than half of the total power budget. In order to develop the energy-efficient EPTS device, therefore, it is necessary to develop the advanced control scheme of GNSS operations, which reduces the number of GNSS samples without degrading the error metrics, i.e., E d and E s .

3. Proposed Power Optimization Method

3.1. Activity-Aware GNSS Control

It is well known that the energy consumption of sensor-based mobile devices can be remarkably saved by introducing the adaptive sensor control strategy associated with domain-specific knowledge [30,31]. For example, Krause et al. [32] investigated the trade-off of adaptive sensor control strategies between the human-motion classification performance and the sampling rate, which can be specialized to handle the support vector machine (SVM) and Markov chain model. Andersson et al. [33] introduced the two step control method for activating the power-hungry sensors based on the sensing results of low-power but less-accurate sensors. Considering the movements of football players as described for the tracking test scenario in [28], inspired by the prior works, we designed an advanced firmware-level optimization to reduce the activation frequency of the GNSS module depending on the athletic actions of players. By categorizing the movement types using the current sensing data, as conceptually illustrated in Figure 5, more precisely, we can disable the GNSS receiver for a moment when the player stays from a certain position during the match time. For the case study, in this work, we define six activities that frequently occur at football matches, as summarized in Table 2. To minimize the measurement errors caused by the reduced number of sensing samples, therefore, it is important to develop a simple but accurate algorithm on the embedded MCU for categorizing these pre-defined football activities.
For the straight-forward classification, similar to the prior work from [34], we may directly use the speed measurements from the GNSS unit to set the intuitive thresholds to classify some human activities. For example, the athlete’s movement can be determined based on the fixed speed criteria defined in related papers [29,35], and then, the GNSS sampling rate for each activity can be adjusted to eliminate over-sampling data, as described in Table 2. However, the speed value can only provide hints for categorizing the linear movements, and this approach cannot capture the rotation-related motions. Therefore, the energy-reduction from the straight-forward recognition is marginal due to the limited capability for categorizing the movement types of football players. In order to provide precise control of sensor modules, by using additional sensing signals from the IMU module, we develop the DCNN-based recognition of football activities, further reducing the energy consumption of the EPTS device.

3.2. Proposed DCNN-Based Classification of Football Activities

Recently, like the other classification issues [36,37,38,39], the algorithm-level performance of the human activity recognition (HAR) problem from IMU data has been remarkably improved by accepting DCNN approaches [40,41,42,43]. For example, the 1D CNN from [44] utilizes the sensing data of a three axis accelerometer, finding the temporal features along each channel. Extending the dimension of the CNN architecture can capture the correlated features in multiple channel domains, improving the recognition accuracy of the HAR problem [41]. Applying the pre-processing for multiple sensor measurements, furthermore, the quality of the CNN-based HAR system can be further improved as reported in [42,43]. For open-source datasets of motions in daily life [40,45,46,47], Table 3 summarizes the performance of different CNN-based HAR systems in terms of the recognition accuracy, as well as the required memory size for storing the trained network. Note that the recent work from [43] offers the smallest memory footprint among the existing works. Hence, we design a compact DCNN model based on the work from [43], which is dedicated to recognizing the football activities at the resource-limited device.
Instead of using the open-source dataset directly [40,45,46,47], in order to generate a compact DCNN design, we collected actual data samples using the prototype EPTS devices. The custom dataset contains total 28,712 one second long samples of IMU measurements from five male subjects (age = 24.6 ± 2.2 years; height = 175.8 ± 3.7 cm; weight = 68.6 ± 3.6 kg). For the balanced data acquisition, in addition to using the standardized trajectory shown in Figure 4a, we used two more tracks for collecting samples, which are more focused on the straight and rotation activities, as illustrated in Figure 6a and Figure 6b, respectively. Note that the acquired IMU samples were divided into training and testing sets by randomly selecting 400 samples from the collected dataset, as described in Table 4.
Figure 7 depicts the processing sequence of the proposed firmware-level EPTS sensor control that includes the low-cost CNN architecture for on-device processing. Similar to the prior work [43], in every time window, we first gather measurement data from the multi-channel IMU sensor shown in Figure 2 and then make a signal image by rearranging the sensor outputs in the 2D domain. More precisely, the signal image allows making all the sensor channels appear adjacent to each other. Then, as illustrated in Figure 7, we deploy three convolution layers (CONVs) and two max pooling layers (POOLs) to find the temporal features for characterizing the target football activities. For the classifier, we simply introduce an global average pooling layer (GAP) rather than utilizing the computation-intensive fully-connected layers [43], which can provide enough recognition accuracy to be used for categorizing the football activities. Targeting the football activity datasets from the real experimental environments, as a result, the baseline CNN architecture in this work achieves a recognition accuracy of 98.29% while requiring 7.56 KB for storing a whole network model based on 32 bit floating-point numbers.
To make the lightweight processing suitable for the on-device DCNN processing, the proposed DCNN architecture is further optimized by applying the quantization method [48] to represent each network parameter with an 8 bit fixed-point number. In addition, the layer-fusing method from [49] is utilized to merge two adjacent processing layers: one convolution layer and the following pooling layer, making a single processing layer associated with fewer parameters. Table 5 compares the proposed DCNN architecture with the previous method from [43], which provides the smallest model size among existing works, as summarized in Table 3. For fair comparisons, we newly trained the prior work [43] by using the custom dataset of football activities shown in Table 4. Applying the post-training of 8 bit fixed-point parameters, note that we can remarkably compress the model size by 76.3% without degrading the algorithm-level performance. By only using the simple integer-based operations, note that the proposed quantized network even reduces the computing time for recognizing the football activities by 58.3% using the commercialized MCU module when compared to the previous state-of-the-art work using floating-point numbers. It is also possible to reduce the complexity of the prior design by exploiting the fixed-point number system. As depicted in Table 5, however, the aggressive quantization severely degrades the algorithm-level performance of the DCNN model in [43] as it necessitates intensive accumulations for realizing the fully-connected layer. Therefore, the proposed DCNN solution adopting cost-aware optimization schemes is a suitable option to recognize football activities at the resource-limited EPTS device.

3.3. DCNN-Based Sensing Rate Control for Energy-Optimized EPTS Operations

After categorizing the current football activity with the proposed lightweight DCNN architecture accepting the IMU measurements, it is possible to adjust the sampling frequency of the GNSS module for reducing the overall energy consumption of the EPTS device, which is the most energy-consuming component, as described in Table 1. Due to the accurate classification results shown in Table 5, it is easy to expect that the proposed activity-based control successfully maintains the amount of position errors with fewer GNSS samples when compared to the baseline EPTS operations shown in Figure 4b. Compared to the straight-forward GNSS control that exploits the speed information only, the proposed technique obviously offers a better control option that further reduces the number of redundant GNSS samples, consequently relaxing the overheads to activate the power-hungry satellite accessing.
It is also possible to reduce the number of sensing operations on the IMU unit, which are used to make the signal image for CNN-based football activity recognition. Therefore, it is required to consider two sensing rates of the GNSS and IMU modules at the same time to find the optimal configuration in terms of energy consumption. For the sake of simplicity, we define f G and f I to denote the sampling rates of the GNSS and IMU units, respectively, which are used for developing the practical algorithm for developing the energy-optimized EPTS operations. Considering the allowable rates for each sensing device, the candidate set S is defined by including the possible pairs of f G and f I , where f G { 1   Hz , 2   Hz , 4   Hz , 10   Hz } and f I { 10   Hz , 20   Hz , 50   Hz , 100   Hz } . For each football activity, we can find the optimal sensing-rate pair ( f ^ G , f ^ I ) , which can be obtained by solving the following problem:
( f ^ G , f ^ I ) = arg min ( f G , f I ) S G ( f G , f I )
Note that the cost function G ( · ) is newly introduced as follows:
G ( f G , f I ) = α × P ( f G , f I ) + ( 1 α ) × E ( f G , f I ) ,
In the proposed cost function, P ( f G , f I ) and E ( f G , f I ) reflect the power consumption and the sensing error, respectively, where the hyper parameter α provides the scaling factor between two metrics. More specifically, P ( f G , f I ) is simply calculated as:
P ( f G , f I ) = P G ( f G ) + P I ( f I ) ,
where P G ( f G ) and P I ( f I ) denote the power consumption of the GNSS and IMU modules running at the given sampling rates, respectively. On the other hand, the second part of the cost function in Equation (3), i.e., E ( f G , f I ) , can be formulated as follows:
E ( f G , f I ) = β × E d ( f G , f I ) + ( 1 β ) × E s ( f G , f I )
For the given configuration of the sampling rates, note that E d ( · ) and E s ( · ) are calculated by using Equation (1), evaluating the errors in the distance and speed, respectively. We also define the second hyper parameter β that can adjust the ratio of the contributions between two errors.
Due to the impractical number of configurations defined by the candidate set S, it is impractical to find the optimal set ( f ^ G , f ^ I ) by solving Equation (3) directly. For the practical solution, as described in Algorithm 1, we propose an iterative way to find a near-optimal configuration ( f ˜ G , f ˜ I ) , which still provides the energy-efficient, yet accurate EPTS operation. Starting from the initial configuration ( f G ( 0 ) , f I ( 0 ) ) = ( 10 Hz , 100 Hz ) , for each football activity described in Table 2, we first reduce the sampling rate of the GNSS module, which minimizes the proposed cost function. Then, the IMU module is adjusted to find the better configuration. As described in Algorithm 1, this process is repeated until there is no change for the two sensing rates. Note that we always find the change of f G first, as the energy consumption caused by the GNSS module is much larger that that of the IMU device, as reported in Table 1. Without evaluating the complex cost function in the excessive number of times, as a result, we can simply get all the practical near-optimal configurations of sensing options for the target football activities, which is described in Table 6. Compared to the threshold-based approach shown in Table 2, which is a straight-forward way of using the speed information from the GNSS module directly, note that the proposed scheme provides a very aggressive control strategy with the high-quality DCNN-based activity classification, leading to the energy-optimized EPTS operations.
As we actively reduce the sensing rate of the IMU module, it is necessary to design an alternative way to construct the signal map, especially for the reduced number of IMU samples. In this work, we apply the linear interpolation to fill the IMU sensing data at the disabled time positions. By preserving the size of the input feature maps for different sensing rates, as a result, we can reuse the proposed DCNN architecture trained for the IMU sampling rate of 100 Hz, when the IMU module even goes to the power-saving mode measuring fewer samples. In other words, the proposed pre-processing always generates the same input format to the pre-designed network for the initial configuration of the sampling rates, i.e., ( f G ( 0 ) , f I ( 0 ) ) , reducing the training overheads to consider the different configurations.
Algorithm 1: Iterative method for finding the near-optimal sensing-rate configurations.
Sensors 20 06004 i001

4. Experimental Results

To verify the effectiveness of the proposed DCNN-based energy-efficient EPTS operations, we installed several firmware programs at the prototype EPTS device: (1) performing the baseline EPTS operations, (2) reducing the GNSS sampling rate with the straight-forward speed checking, and (3) reducing both f / G and f I by utilizing the proposed lightweight DCNN algorithm. Based on the standardized trajectory test shown in Figure 4a, the recovered paths from the reduced GNSS samples are shown in Figure 8. In both approaches adjusting the sampling rates, we utilized fewer points to yield similar paths when compared to the results of the baseline operations shown in Figure 4b. As we removed only the redundant sensing operations, which is supported by the accurate recognition of football activities with the proposed DCNN architecture, note that the measured trajectory in Figure 8b uses the fewest sampling points compared to the other approaches shown in Figure 4b and Figure 8a.
In order to show the quantitative analysis of the proposed DCNN-based EPTS operation, for different sampling rates of the GNSS module, the amount of distance and speed errors is detailed in Figure 9a and Figure 9b, respectively. Note that the optimal f G can be different for each football activity, and the proposed sensor control strategy shown in Table 6 successfully selects the valid option, reducing the measurement errors for the standardized trajectory test. For example, the proposed approach automatically sets f G = 2   Hz when the CNN operation detects the turning_slow activity, which consumes the minimum energy without increasing the distance errors, as shown in Figure 10. On the other hand, the straight-forward method reduces the sampling rates depending only on speed, which requires more samples for turning movements as shown in Figure 8a. Moreover, the proposed DCNN-based control method allows turning off the IMU operations for a while, further reducing the energy consumption of the EPTS device.
Table 7 finally compares the different firmware solutions on the prototype EPTS device in terms of the energy consumption and the sensing errors. Note that the additional firmware overheads to activate the on-device DCNN recognition can be negligible due to the simplified network architecture, increasing the power consumption of the MCU by only less than 1%. Considering the acceptable amount of errors in the practical match scenarios [15,29], as a result, the proposed DCNN-based sensing-rate adjustment saves the power consumption of the GNSS module by 38% when compared to the baseline EPTS operation. Moreover, we can save the IMU power by 34%, whereas the straight-forward way cannot reduce the power consumption of the IMU module. Decreasing the sample rates of the GNSS and IMU modules on demand, as a result, the proposed approach saves the overall power consumption of EPTS operations by 28% and 15% when compared to the baseline system and the straight-forward method, respectively. Note that we still provide attractive data quality, achieving the distance and the speed errors of 0.071 m and 0.657 m · s 1 , respectively. Therefore, the proposed on-device DCNN processing offers a promising solution for reducing the energy consumption of the EPTS device without degrading the quality of the acquired data.

5. Conclusions

In this work, we presented a DCNN-based EPTS device that classifies the movement of football players for reducing the power consumption caused by sensing operations. Compared to the baseline EPTS operations, which fully activate the GNSS and IMU modules, the proposed method dynamically selects the optimal sampling rates of these major sensing components. The novel DCNN architecture is newly introduced for providing the accurate classification of football activities, and several optimization schemes are proposed to enable the on-device DCNN processing with the compressed DCNN structure. Experimental results with the prototype EPTS device reveal that the proposed schemes can drastically reduce the amount of sensing data without degrading the measurement errors, accordingly reducing the power consumption by 28%.

Author Contributions

Conceptualization, H.K. and Y.L.; methodology, H.K. and J.K.; software, H.K. and J.K.; validation, H.K. and J.K.; formal analysis, H.K. and J.K.; investigation, H.K. and J.K.; resources, Y.-S.K. and Y.L.; data curation, H.K. and J.K.; writing, original draft preparation, H.K.; writing, review and editing, Y.L., H.K., and J.K.; visualization, H.K.; supervision, Y.L., Y.-S.K., and M.K.; project administration, H.K. and Y.L.; funding acquisition, Y.L., Y.-S.K., and M.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the Sports Promotion Fund of Seoul Olympic Sports Promotion Foundation from the Ministry of Culture, Sports and Tourism, Korea, and also supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2020-2020-0-01461) supervised by the IITP (Institute for Information & Communication Technology Planning & Evaluation).

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Bojanova, I. It enhances football at world cup 2014. IT Prof. 2014, 16, 12–17. [Google Scholar] [CrossRef]
  2. International Football Association Board (IFAB). Laws of the Game 2020-21. Available online: https://www.the-afc.com/documents/ifab-laws-of-the-game-2020-21 (accessed on 10 August 2020).
  3. Perse, M.; Pers, J.; Kristan, M.; Kovacic, S.; Vuckovic, G. Physics-based modelling of human motion using kalman filter and collision avoidance algorithm. In Proceedings of the ISPA 2005, 4th International Symposium on Image and Signal Processing and Analysis, Zagreb, Croatia, 15–17 September 2005; pp. 328–333. [Google Scholar]
  4. Ogris, G.; Leser, R.; Horsak, B.; Kornfeind, P.; Heller, M.; Baca, A. Accuracy of the LPM tracking system considering dynamic position changes. J. Sport. Sci. 2012, 30, 1503–1511. [Google Scholar] [CrossRef] [PubMed]
  5. Seidl, T.; Linke, D.; Lames, M. Estimation and validation of spatio-temporal parameters for sprint running using a radio-based tracking system. J. Biomech. 2017, 65, 89–95. [Google Scholar] [CrossRef]
  6. Rossi, A.; Pappalardo, L.; Cintia, P.; Iaia, F.M.; Fernández, J.; Medina, D. Effective injury forecasting in soccer with GPS training data and machine learning. PLoS ONE 2018, 13, e0201264. [Google Scholar] [CrossRef] [Green Version]
  7. Cummins, C.; Orr, R.; O’Connor, H.; West, C. Global positioning systems (GPS) and microtechnology sensors in team sports: A systematic review. Sport. Med. 2013, 43, 1025–1042. [Google Scholar] [CrossRef]
  8. International Federation of Association Football (FIFA). Electronic Performance & Tracking Systems. Available online: https://football-technology.fifa.com/en/media-tiles/epts-1/ (accessed on 10 August 2020).
  9. Zavorotny, V.U.; Larson, K.M.; Braun, J.J.; Small, E.E.; Gutmann, E.D.; Bilich, A.L. A physical model for GPS multipath caused by land reflections: Toward bare soil moisture retrievals. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 2009, 3, 100–110. [Google Scholar] [CrossRef]
  10. Conker, R.S.; El-Arini, M.B.; Hegarty, C.J.; Hsiao, T. Modeling the effects of ionospheric scintillation on GPS/satellite-based augmentation system availability. Radio Sci. 2003, 38, 1-1–1-23. [Google Scholar] [CrossRef]
  11. Caron, F.; Duflos, E.; Pomorski, D.; Vanheeghe, P. GPS/IMU data fusion using multisensor Kalman filtering: Introduction of contextual aspects. Inf. Fusion 2006, 7, 221–230. [Google Scholar] [CrossRef]
  12. Qi, H.; Moore, J.B. Direct Kalman filtering approach for GPS/INS integration. IEEE Trans. Aerosp. Electron. Syst. 2002, 38, 687–693. [Google Scholar]
  13. Chen, X.; Wang, X.; Xu, Y. Performance enhancement for a GPS vector-tracking loop utilizing an adaptive iterated extended Kalman filter. Sensors 2014, 14, 23630–23649. [Google Scholar] [CrossRef] [Green Version]
  14. Jennings, D.; Cormack, S.; Coutts, A.J.; Boyd, L.; Aughey, R.J. The validity and reliability of GPS units for measuring distance in team sport specific running patterns. Int. J. Sport. Physiol. Perform. 2010, 5, 328–341. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Scott, M.T.; Scott, T.J.; Kelly, V.G. The validity and reliability of global positioning systems in team sport: A brief review. J. Strength Cond. Res. 2016, 30, 1470–1490. [Google Scholar] [CrossRef] [PubMed]
  16. Chambers, R.; Gabbett, T.J.; Cole, M.H.; Beard, A. The use of wearable microsensors to quantify sport-specific movements. Sport. Med. 2015, 45, 1065–1081. [Google Scholar] [CrossRef] [PubMed]
  17. International Federation of Association Football (FIFA). About the IMS Standard for Wearable Tracking Devices. Available online: https://football-technology.fifa.com/en/media-tiles/about-the-ims-standard-for-wearable-tracking-devices/ (accessed on 7 October 2020).
  18. Dunn, M.; Hart, J.; James, D. Wearing electronic performance and tracking system devices in association football: Potential injury scenarios and associated impact energies. Proceedings 2018, 2, 232. [Google Scholar] [CrossRef] [Green Version]
  19. U-Blox. Ultra-Small u-Blox M8 GNSS SiP Modules Datasheet. Available online: https://www.u-blox.com/ (accessed on 7 October 2020).
  20. CEVA. BNO08X Datasheet. Available online: https://www.ceva-dsp.com/ (accessed on 7 October 2020).
  21. STMicroelectronics. Ultra-Low-Power Arm Cortex-M4 32-Bit MCU+FPU Datasheet. Available online: https://www.st.com/ (accessed on 7 October 2020).
  22. STATSports. Apex Athlete Series Fifa Approved GPS Player Tracker. Available online: https://statsports.com/apex-athlete-series/ (accessed on 8 September 2020).
  23. CatapultSports. OptimEye S5. Available online: https://www.catapultsports.com/products/optimeye-s5 (accessed on 8 September 2020).
  24. CatapultSports. Vector. Available online: https://www.catapultsports.com/products/vector (accessed on 8 September 2020).
  25. Madgwick, S.O.; Harrison, A.J.; Vaidyanathan, R. Estimation of IMU and MARG orientation using a gradient descent algorithm. In Proceedings of the 2011 IEEE international conference on rehabilitation robotics, Zurich, Switzerland, 29 June–1 July 2011; pp. 1–7. [Google Scholar]
  26. Mohinder, S.G.; Lawrence, R.W.; Angus, P.A. Global Positioning Systems, Inertial Navigation, and Integration, 2nd ed.; John Wiley & Sons: Hoboken, NJ, USA, 2007; Chapter 10. [Google Scholar]
  27. Kim, H.; Lim, J.; Hong, W.; Park, J.; Kim, Y.S.; Kim, M.; Lee, Y. Design of a Low-Power BLE5-Based Wearable Device for Tracking Movements of Football Players. In Proceedings of the 2019 International SoC Design Conference (ISOCC), Jeju, Korea, 6–9 October 2019; pp. 11–12. [Google Scholar]
  28. International Federation of Association Football (FIFA). Stay informed about the latest Football Technology and Innovations. Available online: https://football-technology.fifa.com/media/172234/2019-handbook-of-test-methods-epts.pdf (accessed on 10 August 2020).
  29. Linke, D.; Link, D.; Lames, M. Validation of electronic performance and tracking systems EPTS under field conditions. PLoS ONE 2018, 13, e0199519. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Rault, T.; Bouabdallah, A.; Challal, Y.; Marin, F. A survey of energy-efficient context recognition systems using wearable sensors for healthcare applications. Pervasive Mob. Comput. 2017, 37, 23–44. [Google Scholar] [CrossRef] [Green Version]
  31. Oletic, D.; Arsenali, B.; Bilas, V. Low-power wearable respiratory sound sensing. Sensors 2014, 14, 6535–6566. [Google Scholar] [CrossRef]
  32. Krause, A.; Ihmig, M.; Rankin, E.; Leong, D.; Gupta, S.; Siewiorek, D.; Smailagic, A.; Deisher, M.; Sengupta, U. Trading off prediction accuracy and power consumption for context-aware wearable computing. In Proceedings of the Ninth IEEE International Symposium on Wearable Computers (ISWC’05), Osaka, Japan, 18–21 October 2005; pp. 20–26. [Google Scholar]
  33. Ben Abdesslem, F.; Phillips, A.; Henderson, T. Less is more: Energy-efficient mobile sensing with senseless. In Proceedings of the 1st ACM Workshop on Networking, Systems, and Applications for Mobile Handhelds; Association for Computing Machinery: New York, NY, USA, 2009; pp. 61–62. [Google Scholar]
  34. Reddy, S.; Mun, M.; Burke, J.; Estrin, D.; Hansen, M.; Srivastava, M. Using mobile phones to determine transportation modes. ACM Trans. Sens. Netw. (TOSN) 2010, 6, 1–27. [Google Scholar] [CrossRef]
  35. Pons, E.; García-Calvo, T.; Resta, R.; Blanco, H.; López del Campo, R.; Díaz García, J.; Pulido, J.J. A comparison of a GPS device and a multi-camera video technology during official soccer matches: Agreement between systems. PLoS ONE 2019, 14, e0220729. [Google Scholar] [CrossRef] [Green Version]
  36. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems; Neural Information Processing Systems Conference: Lake Tahoe, Spain, 2012; pp. 1097–1105. [Google Scholar]
  37. Hara, K.; Kataoka, H.; Satoh, Y. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet? In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 6546–6555. [Google Scholar]
  38. Kutlu, H.; Avcı, E. A novel method for classifying liver and brain tumors using convolutional neural networks, discrete wavelet transform and long short-term memory networks. Sensors 2019, 19, 1992. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  39. Zhang, H.B.; Zhang, Y.X.; Zhong, B.; Lei, Q.; Yang, L.; Du, J.X.; Chen, D.S. A comprehensive survey of vision-based human action recognition methods. Sensors 2019, 19, 1005. [Google Scholar] [CrossRef] [Green Version]
  40. Zeng, M.; Nguyen, L.T.; Yu, B.; Mengshoel, O.J.; Zhu, J.; Wu, P.; Zhang, J. Convolutional neural networks for human activity recognition using mobile sensors. In Proceedings of the 6th International Conference on Mobile Computing, Applications and Services, Austin, TX, USA, 6–7 November 2014; pp. 197–205. [Google Scholar]
  41. Chen, Y.; Xue, Y. A deep learning approach to human activity recognition based on single accelerometer. In Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics, Kowloon, China, 9–12 October 2015; pp. 1488–1492. [Google Scholar]
  42. Ha, S.; Yun, J.M.; Choi, S. Multi-modal convolutional neural networks for activity recognition. In Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics, Kowloon, China, 9–12 October 2015; pp. 3017–3022. [Google Scholar]
  43. Jiang, W.; Yin, Z. Human activity recognition using wearable sensors by deep convolutional neural networks. In Proceedings of the 23rd ACM International Conference on Multimedia, Brisbane, Australia, October 2015; ACM: New York, NY, USA, 2015; pp. 1307–1310. [Google Scholar]
  44. Lee, S.M.; Yoon, S.M.; Cho, H. Human activity recognition from accelerometer data using Convolutional Neural Network. In Proceedings of the 2017 IEEE International Conference on Big Data and Smart Computing (Bigcomp), Jeju, Korea, 13–16 February 2017; pp. 131–134. [Google Scholar]
  45. Zhang, M.; Sawchuk, A.A. USC-HAD: A daily activity dataset for ubiquitous activity recognition using wearable sensors. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, Pittsburgh, PA, USA, 5–8 September 2012; pp. 1036–1043. [Google Scholar]
  46. Lockhart, J.W.; Weiss, G.M.; Xue, J.C.; Gallagher, S.T.; Grosner, A.B.; Pulickal, T.T. Design considerations for the WISDM smart phone-based sensor mining architecture. In Proceedings of the Fifth International Workshop on Knowledge Discovery from Sensor Data, San Diego, CA, USA, 21 August 2011; pp. 25–33. [Google Scholar]
  47. Zappi, P.; Lombriser, C.; Stiefmeier, T.; Farella, E.; Roggen, D.; Benini, L.; Tröster, G. Activity recognition from on-body sensors: Accuracy-power trade-off by dynamic sensor selection. In Proceedings of the European Conference on Wireless Sensor Networks, Bologna, Italy, 30 January–1 February 2008; Springer: Berlin/Heidelberg, Germany, 2008; pp. 17–33. [Google Scholar]
  48. Jacob, B.; Kligys, S.; Chen, B.; Zhu, M.; Tang, M.; Howard, A.; Adam, H.; Kalenichenko, D. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 2704–2713. [Google Scholar]
  49. Xing, Y.; Liang, S.; Sui, L.; Jia, X.; Qiu, J.; Liu, X.; Wang, Y.; Shan, Y.; Wang, Y. Dnnvm: End-to-end compiler leveraging heterogeneous optimizations on fpga-based cnn accelerators. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 2019, 39, 2668–2681. [Google Scholar] [CrossRef] [Green Version]
Figure 1. A wearable electrical performance tracking system (EPTS) in football matches.
Figure 1. A wearable electrical performance tracking system (EPTS) in football matches.
Sensors 20 06004 g001
Figure 2. A block diagram of the proposed wearable EPTS device.
Figure 2. A block diagram of the proposed wearable EPTS device.
Sensors 20 06004 g002
Figure 3. A circuit board of the proposed wearable EPTS device: (a) front-side and (b) back-side.
Figure 3. A circuit board of the proposed wearable EPTS device: (a) front-side and (b) back-side.
Sensors 20 06004 g003
Figure 4. (a) Standardized testing scenario. (b) The measured trajectory using the proposed device.
Figure 4. (a) Standardized testing scenario. (b) The measured trajectory using the proposed device.
Sensors 20 06004 g004
Figure 5. (a) A trajectory recovered by the baseline sampling associated with the fully-activated GNSS module and (b) a trajectory recovered by the activity-aware sampling with the reduced number of GNSS accesses.
Figure 5. (a) A trajectory recovered by the baseline sampling associated with the fully-activated GNSS module and (b) a trajectory recovered by the activity-aware sampling with the reduced number of GNSS accesses.
Sensors 20 06004 g005
Figure 6. Circuit diagrams for collecting samples of (a) linear movements and (b) turning movements.
Figure 6. Circuit diagrams for collecting samples of (a) linear movements and (b) turning movements.
Sensors 20 06004 g006
Figure 7. Processing sequence of the proposed DCNN-based sensor control strategy.
Figure 7. Processing sequence of the proposed DCNN-based sensor control strategy.
Sensors 20 06004 g007
Figure 8. The recovered trajectories derived from (a) the straight-forward method and (b) the proposed DCNN-based scheme.
Figure 8. The recovered trajectories derived from (a) the straight-forward method and (b) the proposed DCNN-based scheme.
Sensors 20 06004 g008
Figure 9. Measurement errors for different football activities in terms of (a) distance and (b) speed ( f I = 100   Hz ).
Figure 9. Measurement errors for different football activities in terms of (a) distance and (b) speed ( f I = 100   Hz ).
Sensors 20 06004 g009
Figure 10. Evaluating the objective function using Algorithm 1 to find the near-optimal sensing-rate configuration of the turning_slow activity.
Figure 10. Evaluating the objective function using Algorithm 1 to find the near-optimal sensing-rate configuration of the turning_slow activity.
Sensors 20 06004 g010
Table 1. Power consumption of each component for performing the baseline operation.
Table 1. Power consumption of each component for performing the baseline operation.
GNSSIMUMCU
Current58.0 mA22.0 mA25.0 mA
Power191.4 mW72.6 mW82.5 mW
Proportion55.2%21.0%23.8%
Table 2. Football activity types in this work.
Table 2. Football activity types in this work.
ClassificationDefinitionSpeed RangeGNSS Rate
(Straight-Forward)
StationaryStaying in one spot<0.27 m · s 1 1 Hz
WalkingMoving forward by stepping0.27–1.67 m · s 1 2 Hz
JoggingMoving forward at a slow, monotonous pace1.67–3.32 m · s 1 4 Hz
RunningMoving forward at a high intensity>3.32 m · s 1 10 Hz
Turning_slowArc or semicircular motion at a slow pace
Turning_fastArc or semicircular motion at a fast pace
Table 3. Summary of the previous DCNN-based human activity classifiers.
Table 3. Summary of the previous DCNN-based human activity classifiers.
MethodDatasetAccuracySize
Zeng et al. [40][46]96.88%173.56 KB
Chen and Xue [41][41]93.80%100.17 KB
Ha et al. [42][47]97.92%211.40 KB
Jiang and Yin [43][45]97.01%8.27 KB
Table 4. Summary of acquired samples for training the proposed DCNN.
Table 4. Summary of acquired samples for training the proposed DCNN.
DatasetStationaryWalkingJoggingRunningTurning_slowTurning_fast
Training set502852884988371635483744
Testing set400400400400400400
Table 5. Implementation results of DCNN operations at the prototype EPTS device.
Table 5. Implementation results of DCNN operations at the prototype EPTS device.
ModelAccuracyLatencySize
Jiang and Yin [43], 32b floating-point98.13%115 ms8.27 KB
Jiang and Yin [43], 8b fixed-point89.42%55 ms2.60 KB
Proposed, 32b floating-point98.29%59 ms7.56 KB
Proposed, 8b fixed-point98.12%48 ms1.96 KB
Table 6. Near-optimal sensing-rate configurations from Algorithm 1.
Table 6. Near-optimal sensing-rate configurations from Algorithm 1.
Classification f ˜ G f ˜ I
Stationary1 Hz10 Hz
Walking1 Hz20 Hz
Jogging2 Hz10 Hz
Running2 Hz10 Hz
Turning_slow2 Hz20 Hz
Turning_fast4 Hz10 Hz
Table 7. Performance of different firmware solutions.
Table 7. Performance of different firmware solutions.
SchemePower ConsumptionMeasurement Error
GNSSIMUMCUTotal E d (m) E s ( m · s 1 )
Baseline191.4 mW72.6 mW82.5 mW346.5 mW0.0760.679
Straight-forward137.5 mW72.6 mW82.5 mW292.6 mW0.0730.669
Proposed117.8 mW48.2 mW82.5 mW248.4 mW0.0710.657
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Kim, H.; Kim, J.; Kim, Y.-S.; Kim, M.; Lee, Y. Energy-Efficient Wearable EPTS Device Using On-Device DCNN Processing for Football Activity Classification. Sensors 2020, 20, 6004. https://0-doi-org.brum.beds.ac.uk/10.3390/s20216004

AMA Style

Kim H, Kim J, Kim Y-S, Kim M, Lee Y. Energy-Efficient Wearable EPTS Device Using On-Device DCNN Processing for Football Activity Classification. Sensors. 2020; 20(21):6004. https://0-doi-org.brum.beds.ac.uk/10.3390/s20216004

Chicago/Turabian Style

Kim, Hyunsung, Jaehee Kim, Young-Seok Kim, Mijung Kim, and Youngjoo Lee. 2020. "Energy-Efficient Wearable EPTS Device Using On-Device DCNN Processing for Football Activity Classification" Sensors 20, no. 21: 6004. https://0-doi-org.brum.beds.ac.uk/10.3390/s20216004

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop