Abstract:

This paper presents a methodological analysis of the Regularized Self-Attention Regression (RSAR) model and its extension with multi-objective hyperparameter optimization using the NSGA-II algorithm. The RSAR architecture, originally introduced by Zhou et al. (2020), integrates LSTM, self-attention, and CNN components with L1/L2 regularization to enhance stability and generalization in financial time-series forecasting. The original implementation evaluates only a limited set of manually selected configurations, which restricts the model’s adaptability to diverse datasets. To overcome this limitation, a framework based on NSGA-II is proposed, enabling automated, multi-objective optimization that simultaneously minimizes validation error and the train-validation gap to mitigate overfitting. The paper provides a detailed description of the RSAR architecture, mathematical foundations of the attention mechanism and regularization, as well as a complete formulation of the NSGA-II optimization pipeline. Experimental evaluation is intentionally omitted and will be presented in a subsequent study. The presented framework establishes a foundation for fully automated adaptation of RSAR models, improving predictive accuracy and model robustness across various financial markets.