Integrating causal discovery and machine learning for enhanced financial forecasting

Abstract

Financial market forecasting relies heavily on selecting relevant input features, often based on correlations that may include redundant or irrelevant information, thus reducing predictive accuracy. This thesis explores how integrating causal discovery techniques with machine learning models can enhance forecasting reliability. Specifically, the causal discovery algorithm NTS-NOTEARS is applied to identify causal relationships within financial data. The focus is on forecasting NASDAQ-100 closing prices using an enriched dataset that includes technical indicators and macroeconomic variables over a ten-year period. Causally selected features are used as inputs to Long Short-Term Memory (LSTM) models and Ordinary Least Squares (OLS) regression models. The models are evaluated using standard financial forecasting metrics such as Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE). The results indicate that features selected via causal discovery significantly improve the accuracy and robustness of LSTM-based forecasts compared to traditional correlation-based feature selection. Furthermore, LSTM models incorporating causally selected features outperform OLS regression models. These findings demonstrate that combining causal discovery with machine learning offers a promising path toward more reliable financial forecasting.