0 Introduction

Physical inactivity is now considered as one of the major risk factors accounts for global mortality^[1]. Consequently, World Health Organisation^[2]recommended that general adults should engage in aerobic physical activity moderately for at least 150 min, or vigorously for 75 min per week. Hospital ward is one of the most frequent places where inpatients suffer directly from physical inactivity or indirectly from complications. A certain amount of physical activity will be also beneficial to the blood circulation of the bedridden patients to prevent hospital complications such as pressure ulcers and constipation^[3].

Previous public health studies on the assessments of physical activity and sedentary behaviour items tend to use self-reported questionnaires, for these were the only feasible and available method of the period^[4]. Now qualitative measurement of physical activities seems to be feasible as human motion tracking system is available in mass market. Data collection of physical activity data would much benefit from cost-effective motion tracking system. However, such empirical evidence will be provided to inform evidence-based practises in promoting public health, only if its validity and reliability in measuring physical activity has been verified^[5].

Currently the majority of human motion tracking systems used for clinical purpose are marker-based^[6]. Marker-based systems (MBS) are considered to be valid and accurate in occasional clinical assessment like kinesiological evaluations, despite the instrumental error^[7]. However, the disadvantages of MBS, e.g. high cost, high maintenance and poor portability, make that the procedure can only be done in specialized department rather than hospital ward or patient's home where actual activities take place^[8]. As setting up the markers causes awareness to the participants, naturalistic observation of physical activities can hardly be realised by MBS. Application of marker-free system (MFS) has been motivated of circumventing the technical flaws of MBS, which were summarised by Zhou and Hu^[9]: (1) standard bony landmarks identification could be unreliable; (2) soft tissue overlying bony landmarks moves thus introduces noise to data; (3) the marker itself will wobble due to its own inertia; (4) markers become adrift and cause greater error as time goes.

Another highly promising type of motion tracking system is based on e-skin (electronic skin) wearable tactile sensors ^[10]. Cao et al ^[11] reviewed the recent progress of wearable tactile sensors for e-skin. However stretchable or flexible, the entire sensor placed on human skin can be considered as a big marker. In the application of passively monitoring the physical activity of inpatients for the sake of rehabilitation, any factor that may introduce involuntary use should be taken as a risk. Thus in this paper we will focus on marker-free motion tracking system in strict sense.

Past studies have brought promising results on the validity and reliability of using MFS such as Kinect^TM in functional assessment^{[8, 12]}, joint angle measurement^[13] and static foot posture evaluation^[14]. MFS hence seems to be feasible in assessing physical activity in a more cost-effective, less obtrusive way, with the recent availability of inexpensive motion sensors such as Microsoft Kinect^TM, Nintendo Wii^TM, Asus Xtion^TM, and so on. Application potentials of MFS as ergonomics monitoring system at occupational work for observational purposes were explored^[15-17].

One drawback of MFS is that the capacity of joint detection decreases when subject is not directly facing to the sensor. For example, when MFS is used as freehand gesture detector, strong directional effects on its performance and usability were revealed by Ren and O'Neill^[18]. When MFS is intended to be applied to measure physical activity in an observational or passive manner, it will be unrealistic to assume that the subject(s) being observed will directly face to the sensor at all time. Thus observing angle would be a major factor affecting the reliability of MFS and multiple sensors maybe required. So far little is known on how reliable/unreliable the MFS is in motion tracking when observing from different viewing angles. The results of a recent study carried out by Diego-Mas and Alcaide-Marzal ^[15] demonstrated a trend that the quality of posture assessment data became worse as viewing angle increased from 0° to 80° in 20° interval. The trials however were performed in sequence rather than concurrently so that the investigation was not based on simultaneous data.

The main aim of this work is to investigate the reliability of MFS in measuring physical activity from different viewing angles, since angular reliability needs to be explored before implementation of MFS as non-obstructive activity monitors in hospital or patient's home. The results are expected to inform the circumstances under which MFS is effective and specifications of optimal system settings.

1 Methods 1.1 Participants

Ten healthy adults (27±2 years old, 173±8 cm height, 64±2.1 kg weight, 21± 2 kg/m² body mass index, 4 females) were recruited to participant the experiment. This work was approved by the Ethical Committee of the University. Written consent forms were collected from all participants prior to the experiment and information sheet were provided. A small fee was given to the participants for their contribution.

1.2 Experiment Setting

The experiment setting is shown in Fig. 1. The participants were asked to lay on a sickbed with an angle of 45°. One pair of Kinect^TM sensors (30 Hz sample rate) were placed at a height of 1.0 m, which were approximately level with the central trunk of an average participant. There is a horizontal 45-degree angle between the optic axes of two sensors. The distance between the sensor and the participant was 2.0 m, recommended by the manufacturer ^[19]. The experiment room was well-lit by fluorescent lamps and the average horizontal illuminance on sickbed was measured to be 350 lux.

A data collection program based on the Microsoft Kinect^TM SDK (v1.8) was developed to capture the spatiotemporal locations using 'seated mode', which tracks the ten upper-body skeleton joints (head, central shoulder, two shoulders, two elbows, two wrists, and two hands) from both sensors simultaneously. See Fig. 2 for definition of the ten skeleton joints. It was reported that depth signal will considerably degrade when multiple sensors are pointing to the same scene simultaneously, as the sensors continuously project a structured light dot pattern onto the scene without modulation ^[20]. To reduce this interference, a non-destructive Shake'n'Sense approach (motion blur brought by rapid but small vibration) proposed by Butler et al.^[20] was adopted. An offset-weight vibration motor with a rotation frequency of 50 Hz (3000 revolutions per minute) was firmly adhered to one of the two sensors throughout the recording period.

1.3 Procedure

All participants were asked to perform four types of flexion/extension movement: elbow along frontal plane (EF), elbow along median plane (EM), shoulder along frontal plane (SF) and shoulder along median plane (SM). The frontal plane (EF and SF) corresponds to the medial-lateral plane, while median plane (EM and SM) corresponds to anterior-posterior plane. See Fig. 3 for illustration of the four movements at typical moment. For each flexion/extension movement there were ten repetitions. Participants were instructed to perform the movement as consistently as they can, and to follow the pace indicated by a visual metronome (1 s for flexion, 1 s for extension, 1 s for flexion again, and so on). Thus one trial was about 20 s, leaving out the preparatory and closure phases. There was no particular instruction about the amplitude of reaching. A second block of these four movements was performed while the alternative sensor was set to be vibrating, since one of the two sensors need to vibrate to reduce interference in the Shake'n'Sense approach. This was to counter-balance the potential impact of vibration on the data that may confound the analyses.

1.4 Data Processing

The obtained raw data for each trial were a series of x, y, and z coordinates of the ten joints. Data collected from the two sensors were synchronized to be comparable within an error interval of 17 ms (constrained by the 30 Hz sampling rate). Then a fourth order zero lag low-pass Butterworth filter with a cut-off frequency of 6 Hz was applied on all data^{[8, 21]}. Data of each trial was cut by a windows of 6000-15999 ms with an exact 10000 millisecond sample period that was supposed to include five complete cycles of flexion/extension movement, to confidently avoid the unstable preparatory and closure phases.

The amount of activity can be expressed as the summation of difference (integral of variation if in ideal sampling condition) between two contiguous joint positions or angles. For elbow flexion/extension movement this was expected to be dominated by joint of 'elbow', whilst joint 'shoulder' dominated shoulder movement. Displacements of wrist and elbow were used for elbow and shoulder movement respectively. The elbow angle was defined by positions of shoulder (S), elbow (E), and wrist (W): ∠SEW. The shoulder angle was defined by positions of central shoulder (C), shoulder (S), and elbow (E): ∠CSE. The value of angle was worked out based on the Law of Cosines.

There were in total 80 trials (10 participants, 4 movements, and 2 blocks). For each sensor and each trial, accumulative displacement and angle change within each 2 s and 10 s time intervals were calculated as metrics of amount of activity. The accumulative displacements and changes in angle within 10 s time interval were converted pro rata to 2 s to ensure the comparability, i.e. one data point of 10 s interval can be seen as the mean of the five data points of 2 s interval. This introduced repetition that enables validity check when data is analysed in 2 s/10 s interval. Data of the two blocks, that either sensor #1 or #2 was set to vibrate, were put together to be analysed. Data collected from each single participant was kept as separate data points. All statistical analyses were carried out using PASW Statistics 18 (a.k.a. SPSS 18). Results of Kolmogorov-Smirnov and Shapiro-Wilk tests suggested that data was normally distributed therefore parametric tests were applicable. Statistical significant level was set as two-tailed 0.05. Pearson correlation coefficient (R), intra-class correlation coefficients (ICC), 95% limits of agreement analysis (LOA) based on Bland & Altman plots^[22], and coefficients of variation (CV) were used to characterise the reliability between two sensors with 0° and 45° viewing angles.

2 Results

The results of reliability measures (R, ICC, LOA and |CV|) are shown in Table 1. These are listed separately by four movements, displacement/angle, and 2 s/10 s time intervals. The unit of LOAs for displacement is centimetre (cm) while for angles it is degree (°), so it should be noted that the comparison between them are not meaningful. In general, there is little difference between R and ICC for the same groups of trials.

For elbow flexion/extension along frontal plane (EF), highest reliability measures are found in displacement of left elbow. Lowest reliability measures are found in displacement of right elbow. Reliability measures of angle (both left and right) lay in between. The LOAs and |CV|s of left elbow are smaller than those of right elbow. For elbow flexion/extension along median plane (EM), there is hardly any sign of fair reliability. The general reliability is clearly found to be worse than in EF. Negative values of R and ICC, indicating poor reliability, are found in both displacement and angle measures of EM movement.

For shoulder flexion/extension along frontal plane (SF), it is found that the R and ICC for the displacement of left elbow are greater than 0.40, while for right shoulder these are greater than 0.25. However for angles, R and ICC of less than 0.20 show poor reliability. LOA and |CV| of right shoulder are extremely higher than those of left shoulder. For shoulder flexion/extension along median plane (SM), only the measures displacement of right shoulder demonstrate reliability to an extent (R>0.22, ICC>0.22). The general reliability is clearly found to be worse than in EF. Negative values of R and ICC, indicating poor reliability, are found in both displacement and angle measures.

It can be seen that level of reliability varies in different groups of data. Wilcoxon Signed Rank Tests on reliability measures were performed to check the preference of reliability as if it is affected by grouping variables such as time interval (2 s/10 s), type of movement (elbow/shoulder, frontal/median), type of data (displacement/angle), or left/right. Note that R² and |ICC| were used instead of R and ICC respectively, to retain only the information of extent. The results are shown in Table 2.

All reliability measures based on 10 s time interval is found to be significantly 'better' or 'preferred' than those based on 2 s: greater R² and |ICC|, narrower LOA and smaller |CV|. No difference of R², ICC, or |CV| is found when comparing elbow and shoulder movement. When comparing reliability measures between movements, there is a marginal trend of significance toward greater R² and ICC for movements along frontal plane than median plane. Data collected from the left-hand side shows a significantly smaller |CV| and a slightly narrower LOA than data obtained from right-hand side. Comparison on LOA is not applicable to either elbow/shoulder or displacement/angle comparison due to the different natures.

3 Discussion

The aim of this study is to investigate the potential application scope as well as limits of MFS as physical activity monitor. The results show that the overall reliability between 0° and 45° clearly varies among the different types of physical activity measurement (Table 1). Pearson correlation coefficient R and ICC are ranging from -0.3 to 0.5. LOA of displacement and angle reached to approximately ±100 cm and ±200 degree in some of the conditions. The |CV| varies from 1.80 to 17.6 for displacement and 2.0 to 418.2 for angle. These reliability measures are much worse than those found in studies compared MFS with MBS^{[8, 12, 23]}.

Significantly better reliability is found in all metrics when activity data is calculated in 10 s interval than 2 s. This demonstrates a trend that MFS has a higher tolerance of unreliability in activity monitoring when longer observation interval (10 s) was used as elementary measuring unit. In real-life scenario, it may be that the level of activity is not required to be measured as precisely and instantly as in functional assessment, but needed in rougher and less strict applications.

Reliability are poorer for movements along the median plane than frontal plane. This reduction of reliability might be explained by occlusion, i.e., part of the movement is not within the 'interaction space', which is defined as 'the area in front of the Kinect sensor where the infrared and colour sensors have an unblocked view of everything in front of the sensor'^[19]. During the experiment, it was observed from the point of view of sensor #2 (45°) that part of the right arm was covered by the left arm when performing movements along the median plane. However, occlusion was not the case for either sensor or movements along the frontal plane.

Similarly, poorer reliability is also found in data of right-hand side, which was likely to be blocked by the left-hand side rather than the other way round. There are two possible ways to compensate the loss of reliability cause by observing angle. First, use more sensors to achieve a less dispersed thus more intensive coverage of a given range of viewing angles (e.g., 180° or 360°), assuming that better reliability comes with smaller angle discrepancy. Second, screen out the optimum set of data collected by the best positioned sensor for a given period of observation time.

The results do not reveal any significant superiority for either displacements or changes in angle as outcome measures of the amount of physical activity. However, the two highest reliabilities are found in movements of left arm along frontal plane when data is calculated as displacement. See Fig. 4 for Bland-Altman plots on displacement data of these two movements in 2 s. Note that Bland-Altman plots with 10 s interval were not presented as these contain only 1/5 data points as plots with 2 s interval do. No specific relationship of the discrepancies between the measurements of 0° and 45° was observed. The ranges of LOA that cover dozens of centimetre with occasional outliers by no means present a reliable measurement.

There are three limitations of this study. First, the discrepancy of two observation angles was 45° in the horizontal direction as preliminary parameters. Further investigations on vertical and smaller angular discrepancy (e.g., 30°) would be helpful. Second, complex movements crossing multi anatomical planes were not investigated thus extrapolation of current results to real-life application will be less effective. As to further work, reliability with wider range of observation interval (from seconds, minutes to hours) could be evaluated to facilitate better understanding on application scope of the MFS as activity monitor described in this study. Furthermore, with the recent release of Kinect^TM v2 featured with higher number of joints being tracked, better algorithm identifies body, hand and joint orientation, wider field of view, that enable greater resolution, precision and responsiveness. The occlusion issue might also be improved by reckoning algorithm that provides "better estimation of the joint centres, even if they are not visible in the image" ^[8]. There are already a few studies published lately reporting the extent of improvements brought by Kinect^TM v2 comparing with v1^{[13, 24-25]}.

4 Conclusions

This study aims to investigate the reliability of MFS such as Microsoft Kinect in measuring upper extremity motion from different angles. The results show that the reliability of MFS between 0° and 45° observation angles is not satisfactory when measuring physical activity of human upper extremity. Relatively better reliability was observed with less presence of occlusion when movements on left-hand side or along frontal plane were examined. However, the results suggest that longer observation interval as elementary unit of measurement increases the angular reliability. The results are sought to provide better understanding on the effectiveness of MFS and to facilitate the future implementation of MFS as non-obstructive physical activity monitors in either hospital or patient's home.

Acknowledgement

The authors would like to thank Qun Li for the permission to use the illustrative photos.