Improving the robustness of neural ODEs with minimal weight perturbation
Please login to view abstract download link
We design deep neural networks as discretizations of neural ODEs that are inherently robust against data perturbations, including adversarial attacks, carefully crafted perturbations in the input designed to mislead the network into making incorrect predictions. Neural ODEs model the forward pass of a neural network as the evolution of a feature vector through a continuous-time dynamical system. We propose a two-level optimization strategy that optimizes the network parameters to balance accuracy and stability. Specifically, we formulate an optimization problem where we seek the closest perturbation of the weight matrices that enforces a desired stability condition. This approach allows us to stabilize a neural network while maintaining high predictive performance. We validate our method through a range of numerical experiments against existing baseline methods to stabilize neural ODEs on standard classification benchmarks, demonstrating that our approach effectively improves robustness while preserving competitive accuracy, outperforming existing approaches.