Abstract
Gait recognition, a promising biometric technology, relies on analyzing individuals' walking patterns and offers a non-intrusive and convenient approach to identity verification. However, gait recognition accuracy is often compromised by external factors such as changes in viewpoint and attire, which present substantial challenges in practical applications. To enhance gait recognition performance under diverse viewpoints and complex conditions, a global-local part-shift network is proposed in this paper. This framework integrates two novel modules: the part-shift feature extractor and the dynamic feature aggregator. The part-shift feature extractor strategically shifts body parts to capture the intrinsic relationships between non-adjacent regions, enriching the recognition process with both global and local spatial features. The dynamic feature aggregator addresses long-range dependency issues by incorporating multi-range temporal modeling, effectively aggregating information across parts and time steps to achieve a more robust recognition outcome. Comprehensive experiments on the CASIA-B dataset demonstrate that the proposed global-local part-shift network delivers superior performance compared with state-of-the-art methods, highlighting its potential for practical deployment.
Keywords
0 Introduction
As a technique used to recognize a person based on the distinctive way they walk, Gait recognition has gained increasing interest recently because of its possible uses across various domains such as security, forensics, and healthcare. From an individual's gait, it is possible to identify them from a distance and without requiring any physical contact.
While gait recognition has many potential benefits, it also poses some challenges. Gait patterns can be affected by camera viewpoints and clothing, making it difficult to identify individuals accurately.
In order to solve these problems, many methods based on deep learning have been proposed. He et al.[1] proposed to use a multi-task generative adversarial learning network to learn the feature representation of a specific perspective, and improve the representation ability of features. GaitSet[2] regards the gait sequence as an unordered set, and obtains more discriminative features through the designed global and local feature extraction dual-branch structure, and finally uses pyramid pooling for feature mapping to get the final representation. In order to obtain more fine-grained spatial information, GaitPart[3]was proposed to divide the gait picture into different parts and perform feature extraction separately, and then model the temporal features through the proposed micro-motion capture module, and uses the attention mechanism to obtain short-term temporal features and eliminate redundant long-range information.
These methods only focus on the local information of the specific receptive field of the convolution kernel size on a single image, ignoring the internal feature relations between different parts. Therefore, to mine the internal relationship between different parts, improve the global perception ability while maintaining the local perception ability of the feature, a global-local part-shift network is proposed in this study to improve the discrimination ability of the feature:
1) We propose a part-shift module to recombine the features between different parts, and then perform feature extraction to improve the global perception ability of the module.
2) For the global-local features obtained in different parts, we propose a dynamic temporal aggregator to model temporal features. This module can not only model short-range features, but also obtain long-range features. Which can alleviate the long-range dependency problem and improve the ability of temporal modeling.
3) Extensive experiments on the CASIA-B dataset demonstrate the superiority of our method.
1 Related Work
Current gait recognition methods based on deep learning can broadly be categorized into model-based methods[4-6] and appearance-based methods[2-3, 7-8]. Below, we review and critically analyze representative methods from both categories, explicitly highlighting their limitations and clearly positioning our method in relation to them.
1.1 Model-Based Methods
Model-based approaches primarily rely on extracting structured features such as human pose or body joints to achieve gait recognition. For instance, PoseGait[5] leverages 3D human pose estimation to effectively mitigate the impact of viewpoint and clothing variations. It combines CNN and LSTM to model spatial and temporal information jointly and employs a multi-loss strategy for optimization. GaitGraph[6] utilizes 2D pose estimation and extracts spatial features via graph convolutional neural networks (GCNs) , thus effectively capturing the structural dependencies among body joints. Furthermore, the approach proposed in Ref.[4]uses a multi-linear human model and fine-tunes a pre-trained human mesh recovery (HMR) network for gait recognition. Despite the effectiveness of these methods in handling pose variations, they are highly dependent on accurate pose estimation. Pose inaccuracies or occlusions can severely deteriorate their performance. Additionally, these methods usually focus primarily on structural information while neglecting richer appearance-based cues.
1.2 Appearance-Based Methods
Appearance-based methods directly extract features from silhouette images or gait sequences without explicitly modeling the human pose[9-16]. A representative work, Gait Energy Image (GEI) [17], compresses temporal gait information into a single template image, effectively representing individual walking patterns. However, the primary drawback of GEI-based methods, as pointed out in Ref.[18], is the substantial loss of temporal dynamics. GEnI [18] addresses robustness concerns by proposing gait entropy images, but temporal information loss persists. Extending GEI, Periodic Energy Image (PEI) [19] proposes a multi-channel gait template for capturing more discriminative features via adversarial training, yet this method remains view-dependent and still compromises temporal granularity[20-24]. To alleviate temporal information loss, recent methods utilize raw gait frame sequences as network inputs. The gait lateral network structure in Ref.[7]effectively captures discriminative compact features directly from gait contour sequences, significantly reducing feature redundancy without accuracy loss. Meanwhile, GaitPart[3] emphasizes local spatial-temporal characteristics by segmenting gait sequences into distinct body parts to extract fine-grained features. GaitSlice[8] further enhances subtle spatial interactions among adjacent gait parts through its SED module, incorporating a frame attention mechanism for temporal feature selection. Although these appearance-based methods have made considerable progress, a common limitation is their predominant reliance on local receptive fields via convolution operations, which restricts them to localized spatial-temporal interactions. Consequently, they fail to explicitly model the inherent global relationship among different gait parts.
Different from the aforementioned methods, our proposed global-local part-shift network explicitly addresses these limitations. Specifically, our part-shift module strategically recombines features across distinct gait parts, capturing intrinsic relationships between non-adjacent regions. This approach enhances global spatial perception while simultaneously preserving local feature integrity. To further mitigate limitations in temporal modeling, particularly concerning long-range dependencies, we introduce a dynamic feature aggregator. This module employs multi-range temporal modeling to effectively integrate both short-range and long-range temporal interactions within gait sequences, thus significantly improving the robustness and discriminative power of the recognition process.
2 Method
In this section, we first provide a brief introduction to our proposed approach, followed by a detailed description of the Part-Shift Feature Extractor (PSFE) module, and finally present the Dynamic Feature Aggregator (DFA) module.
2.1 Overview
As shown in Fig.1, our network receives a sequence of gait contours xi as input, where i∈1, 2, ···, T, T denotes the number of frames.xi first is input to the PSFE, the PSFE enhances the richness of features by mining the internal relations of different body parts, and obtains the global part-aware vector Pg and local part-aware vector Pl :
(1)
Afterwards, through the Horizontal Pooling (HP) operation (it is worth noting that the maximum function is used as the horizontal pooling strategy in this study) , the features are mapped to obtain the decisive spatial feature vector Dg and Dl :
(2)
The decisive spatial features are sent to the DFA. The DFA contains two sub-modules, Multi-Range Modeling (MRM) and Feature Aggregator (FA) . MRM maps features into short-range temporal features Ts and long-range temporal features Tl in the temporal dimension:
(3)
where m=g, l, that is, the global salient spatial features Dgi and local salient spatial features Dli will be sent to the MRM for temporal modeling, but they do not share parameters.
Then, the global-local multi-range temporal information is sent to the FA for mapping to obtain the fusion feature Fi:
(4)
where m=s, l; n=g, l, the global-local long-range and short-range four types of features are used as input.
Finally, map Fi through seperate Fully Connected Layer (FC) to get the final feature representation Xi:
(5)
Fig.1Overview of the proposed method
2.2 PSFE
The part-shift extractor aims to improve the perception of long-range spatial information while retaining the ability to extract fine-grained spatial features. It consists of two Part-Shift Convolution (PSConv) modules. The PSConv is introduced in detail below.
There are primarily two approaches to frame-level spatial feature extraction: the ordinary 2D convolution method and the part-based method. The part-based approach involves segmenting the input frame into distinct regions, followed by the application of 2D convolution within each region. However, both methods are limited to capturing local spatial features, lacking the capacity to model comprehensive spatial relationships across the entire frame. To address this limitation, we propose an apart-shift convolution module grounded in an apart-aware approach, which facilitates the exchange of positions across segmented regions. As illustrated in Fig.2, part-shift enables previously non-adjacent body parts to become adjacent, allowing convolutional operations to explicitly capture interconnections among these parts and enhance global spatial modeling.
The specific operation is illustrated in Fig.3. The input feature map is divided into distinct segments, referred to as gait stripes. Initially, the sequentially arranged raw feature maps are processed by a convolutional neural network to model features within each segment. Subsequently, the part-shift operation reorders the feature maps across different segments.
Fig.2After the part-shift change, the comparison in the change in the perception range
Fig.3The detailed structure of PSConv module
After exchanging the feature map, we model the original non-adjacent feature points through 2D convolution to obtain the intrinsic connection between long-distance spatial feature points.
A part-shift convolution module can be expressed as follows :
(6)
where i∈1, 2, ···, T, T denotes the frame number of input gait sequence.
2.3 Dynamic Temporal Feature Aggregator
The dynamic feature aggregator further performs temporal modeling of global-local, frame-level spatial features across various granularities to address challenges related to long-range dependencies. This module comprises two independent multi-range feature aggregators, which do not share parameters, as well as the final feature aggregator. A detailed introduction to these modules will be provided subsequently.
Unlike other biometric recognition techniques, gait recognition uniquely requires feature modeling across two domains: temporal and spatial. Thus, after extracting frame-level spatial features through the PSFE, additional temporal feature modeling is necessary to capture dynamic characteristics.
2.3.1 Multi-range temporal modeling
Feature modeling in the temporal dimension is often susceptible to long-range dependency issues. To enhance the expressiveness of these features, we employ a multi-scale approach to capture features at distinct scales. Specifically, convolutional kernels of varying sizes are utilized to convolve features within the temporal domain. As convolutional kernels of different sizes correspond to different receptive fields, they influence the convolution operation's perception range uniquely. Larger convolutional kernels yield a broader receptive field, which helps mitigate long-range dependency issues. Accordingly, we apply convolution operations across two scales to capture features at varying temporal resolutions, as shown below:
(7)
(8)
where xi refers to feature map sequence, i∈1, 2, ···, T, T denotes the number of frames.
2.3.2 Feature aggregator
This module weighted the vector of global-local features for aggregation. As shown in Fig.4, the feature aggregator consists of two parallel branches, the part-score branch and the weighted function branch. The former branch further models each spatio-temporal feature vector to highlight the most discriminative spatio-temporal features, specifically for each feature, the vector calculates the importance score through the attention mechanism, and the specific operation is shown as follows:
(9)
In the part-selection module, the original feature vector is mapped to a scoring feature vector using a max function operation. This transformation can be formally expressed by the following equation:
(10)
The weighted function is used to aggregate prominent temporal features and process the features to eliminate possible temporal domain aliasing. The specific operation of the weighted function is shown as follows:
(11)
Through the above formula, we obtain the weight vector for calculation. The decisive spatio-temporal feature vector obtained through the dot product operation of the weighted vector and the scoring vector is shown as follows:
(12)
Fig.4The abstract structure of dynamic feature aggregator in practice
3 Experiments
A large number of experiments were conducted on two commonly used datasets to evaluate the proposed algorithm in this section. The relevant details of the CASIA-B dataset[25] are introduced in Section 3.1. In Section 3.2, the experimental settings of the proposed algorithm are described, and in Section 3.3, a comparison of experimental results between the proposed method and current state-of-the-art methods on the CASIA-B dataset [25] is presented.
3.1 Datasets
As a popular benchmark dataset for evaluating gait recognition, CASIA-B[25] contained 124 subjects' gait sequences. The subject ( from 001 to 124) has 110 sequences in totally, which are captured from 11 different viewpoints with gaps of 18° ( 0° to 180°) . For every viewpoint, 10 sequences are collected under three scenarios: six sequences with normal walking, two sequences carrying a backpack, and two sequences wearing a coat.
We follow dataset partitioning strategies from Refs.[3]and [2], dividing CASIA-B into Large-sample training (LT) , Medium-sample training (MT) , and Small-sample training (ST) setups, with specified training/testing splits, using NM (#01–04) sequences as gallery sets and NM (#05–06) , BG (#01–02) , CL (#01–02) sequences as probe subsets.
3.2 Implementation Details
Experiments were conducted on NVIDIA 3090 GPUs using silhouette alignment, Adam optimizer (learning rate10-4, momentum 0.2) , batch-all separate triplet loss (margin 0.2) , and a batch size of (8, 12) trained for 100000 iterations.
3.3 Comparison to State-of-the-Art Methods
To assess the performance of our approach under cross-view conditions, we conducted a detailed comparison to several leading methods. As presented in Table1, our method achieves higher recognition accuracy across most viewpoints compared with the seven reference methods. Specifically, for NM/BG/CL conditions, the average recognition accuracy of our method surpasses that of CaitSlice [8] by 0.6%, 1.7%, and 2.3%, respectively. Although our proposed global-local part-shift network significantly improves upon previous single-modal gait recognition methods, it achieves slightly lower average recognition accuracy compared with recent multimodal approaches such as AttnGait[26] and GMSN[27], specifically 1.9% and 4% lower, respectively. This performance gap is understandable given that multimodal methods leverage complementary information from multiple modalities (e.g., appearance, pose, or depth data) , naturally granting them higher discriminative capabilities.
However, the method proposed in this paper utilizes only single-modal gait silhouettes and still achieves competitive accuracy. Moreover, our approach introduces notable improvements in modeling internal relationships among non-adjacent body parts and effectively addresses long-range temporal dependencies within gait sequences. These enhancements offer substantial value, especially in scenarios where multimodal data collection may be constrained by privacy concerns, deployment costs, or hardware limitations. Future research could explore integrating our global-local part-shift network with complementary modalities to further bridge this performance gap.
Table1Rank-1 accuracy averaged across three experimental settings on CASIA-B, excluding same-view comparisons
Additionally, we evaluated the performance of our approach in data-limited scenarios, with experimental results illustrated in Fig.5. These results indicate that our method significantly outperforms both GaitSlice [8] and MT3D[30]. In particular, the average recognition accuracy under ST and MT settings is 1.8% and 1.4% higher than GaitSlice[8], and 0.8% and 1.1% higher than MT3D[30], respectively. The superior accuracy under MT further underscores the robustness and efficiency of our method compared with GaitSlice[8].
Fig.5Benchmarking against leading approaches under ST/MT configurations
4 Conclusions
In this work, we introduce a distinct global-local representation learning framework that leverages a part-shift mechanism specifically designed to overcome constraints from insufficient visual information and external interferences in gait recognition. The proposed framework comprises two specialized modules: the PSFE and the DFA. Specifically, the PSFE strategically shifts and recombines posture segments to capture intrinsic connections between non-adjacent body parts, thereby significantly enriching both global and local spatial features. Concurrently, the DFA employs multi-range temporal modeling, effectively preserving short-range spatio-temporal patterns and addressing long-range temporal dependencies. Extensive experiments conducted on the widely used CASIA-B dataset confirm the effectiveness and competitive performance of our method, highlighting its practical applicability and robustness against challenging variations.
