Gait Activity Classification Using CNN-LSTM Networks

Objective

The primary objective of this research is to develop and evaluate a hybrid deep learning architecture that effectively classifies human gait activities using multi-modal sensor data. Specifically, this study aims to:

• Leverage convolutional neural networks (CNN) for automatic feature extraction from sequential sensor data
• Utilize Long Short-Term Memory (LSTM) networks to capture temporal dependencies in gait patterns
• Investigate the performance improvement gained by fusing inertial measurement unit (IMU) data with GPS information
• Compare classification accuracy across subject-specific and multi-subject training scenarios
• Classify gait activities based on terrain inclination and surface characteristics

Approach

Model Architecture

This study employs a hybrid CNN-LSTM architecture that combines the strengths of both neural network types. The CNN component processes subsequences of input data to automatically extract relevant features, while the LSTM backend interprets these features as temporal sequences to capture long-term dependencies in gait patterns.

Data Processing Pipeline

• Sliding Window Segmentation: Time-series data is partitioned into overlapping windows for activity classification
• Data Augmentation: Gaussian noise is added to the training set to improve model robustness and generalization
• Train-Test Split: A balanced 50-50 split ensures comprehensive coverage of all gait activities in both training and testing sets
• Feature Normalization: Sensor readings are standardized to facilitate efficient model training

Experimental Design

Multiple experimental trials were conducted to evaluate different configurations:

• Subject-Specific Models: Trained and tested on individual subjects to maximize personalization
• Multi-Subject Models: Trained across multiple subjects to assess generalization capabilities
• Sensor Modalities: Compared IMU-only versus IMU+GPS data fusion
• Classification Targets: Two complexity levels - 4-class (inclination only) and 7-class (inclination + terrain)

Dataset

Data Collection

The dataset comprises synchronized IMU and GPS sensor recordings captured during various gait activities across different terrains and inclinations. Data was collected from multiple subjects performing standardized locomotion tasks in controlled and natural environments.

Sensor Configuration

• IMU Sensors: Tri-axial accelerometer and gyroscope measurements capturing linear and angular motion
• GPS Data: Location coordinates, altitude, and velocity information providing spatial context
• Sampling Rate: High-frequency data collection to capture subtle gait variations

Activity Classes

Data Characteristics

Results & Discussion

Performance Improvements with GPS Integration

The experimental results demonstrate substantial performance gains when GPS data is incorporated alongside IMU measurements in the CNN-LSTM architecture. The improvements are particularly pronounced in subject-specific models, though multi-subject models also show meaningful enhancements.

Key Findings

• Multi-modal Sensor Fusion Effectiveness: The integration of GPS data consistently improved classification performance across all experimental conditions, validating the hypothesis that spatial context enhances gait activity recognition
• Subject-Specific vs. Multi-Subject Trade-offs: While subject-specific models achieved higher accuracy gains (+23.8% to +30.4%), they may be more susceptible to overfitting. Multi-subject models showed more modest improvements (+9.1% to +13.4%) but demonstrate better generalization potential
• Classification Complexity Impact: The 7-class problem (inclination + terrain) showed higher improvement rates for subject-specific models, suggesting that GPS data is particularly valuable for disambiguating complex activity patterns
• CNN-LSTM Architecture Suitability: The hybrid architecture effectively leverages both spatial feature extraction (CNN) and temporal pattern recognition (LSTM) capabilities, making it well-suited for time-series gait data

Discussion

The substantial performance improvements observed with GPS integration can be attributed to several factors. GPS data provides complementary spatial information that helps distinguish between activities that may produce similar IMU patterns but occur in different environmental contexts. For instance, walking on level ground versus a slight incline might generate comparable acceleration patterns, but GPS altitude data can disambiguate these scenarios.

The higher improvement rates in subject-specific models suggest that individual gait patterns interact uniquely with environmental factors. This finding has important implications for personalized activity monitoring systems, where user-specific calibration could significantly enhance accuracy. However, the risk of overfitting in subject-specific models must be carefully managed through appropriate regularization and validation strategies.

The positive results for multi-subject models, while more modest, are encouraging for developing generalized activity recognition systems. These models could be deployed without extensive per-user calibration while still benefiting from multi-modal sensor fusion. The 9.1% to 13.4% improvements represent meaningful gains that could translate to more reliable activity tracking in practical applications.

Future Directions

Future research could explore several promising avenues:

• Investigating attention mechanisms to automatically weight the contribution of different sensor modalities
• Exploring transfer learning approaches to leverage subject-specific gains while maintaining generalization
• Evaluating performance with additional sensor types (e.g., barometric pressure, heart rate)
• Testing real-time implementation feasibility on resource-constrained wearable devices
• Extending classification to more granular activity types and transition states