YIC2025

Intertwining Deep Autoencoders and Singular Value Decomposition: Beyond Efficiency, Towards Explainability

  • Brivio, Simone (Politecnico di Milano)
  • Franco, Nicola Rares (Politecnico di Milano)
  • Fresca, Stefania (Politecnico di Milano)
  • Manzoni, Andrea (Politecnico di Milano)

Please login to view abstract download link

Deep autoencoders have grown into increasingly crucial architectures in the context of dimensionality reduction for parameterized PDEs. Indeed, many Deep Learning-based Reduced Order Models (DL-ROMs) leverage the compressed representation entailed by autoencoders to design accurate yet lightweight surrogate models for complex parameterized physical systems [3]. Still, high-dimensional data usually pose a challenge to autoencoders, leading to large deep learning architectures that are computationally demanding to train. It is possible to solve the latter computational bottleneck by enhancing autoencoders with proper orthogonal decomposition (POD), leveraging singular value decomposition (SVD) or its randomized version. The resulting architectures are equipped with rigorous error estimates highlighting the contribution of the POD projection, the data sampling procedure, and the neural network approximation error [1,2]. While the contributions of data sampling and POD projection are glaring, the impact of the autoencoder core is more obscure. Indeed, despite being central in many frameworks, the approximation properties of deep autoencoders remain evasive to this day. To fill this gap, we propose deep symmetric autoencoders, whose peculiar architecture enables us to draw a parallel with SVD, which now becomes crucial in the theoretical characterization of the autoencoders' expressivity. Under this fresh perspective, (i) we derive informative error bounds to the approximation error, and (ii) we design a novel training algorithm specific for deep symmetric autoencoders. Finally, we equip our analysis with suitable numerical experiments that also demonstrate the practical relevance of our approach. References [1] Simone Brivio et al. “Error estimates for POD-DL-ROMs: a deep learning framework for reduced order modeling of nonlinear parametrized PDEs enhanced by proper orthogonal decomposition”. In: Adv. Comput. Math. 50.33 (2024). [2] Nicola Rares Franco, Andrea Manzoni, and Paolo Zunino. “A deep learning approach to reduced order modelling of parameter dependent partial differential equations”. In: Math. Comp. 92.340 (2023), pp. 483–524. [3] Stefania Fresca, Luca Dede’, and Andrea Manzoni. “A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs”. In: Journal of Scientific Computing 87 (2021), pp. 1–36.