Deep Learning vs. Machine Learning for Predictive Maintenance under Limited Industrial Data: A Cross-Machine Validation Case Study

Abstract:

Predictive maintenance (PdM) has become a strategic factor of modern manufacturing systems. While deep learning (DL) architectures such as Long Short-Term Memory (LSTM) models have demonstrated strong performance in large-scale industrial datasets, their effectiveness in PdM under limited data conditions remains unclear. This study investigates whether LSTM provides advantages over classical machine learning (ML) model called Random Forest (RF) in small-sample industrial environments.

This study contributes to the literature by systematically evaluating cross-machine generalization under limited failure data conditions and provides managerial implications for ML and DL adoption in PdM. The empirical analysis focuses on five CNC machines. The analyzed data is fraught with challenges common in production systems: irregular time intervals, missing observations, and machine downtime. A Leave-One-Machine-Out cross-validation strategy was implemented to evaluate models’ performance.

Results indicate that while LSTM model capture temporal patterns in sensor data, its performance advantage over RF diminishes in data-constrained settings. The findings suggest that in small and medium-sized manufacturing enterprises, where large labeled datasets are rarely available, classical ML approaches may offer more robust and cost-effective solutions than DL architectures.