Theory-Guided Machine Learning for Inverse Problems

Abstract

This dissertation studies two machine learning methods for ill-posed inverse problems. The first concerns the adaptive moment estimation (ADAM) algorithm, a widely used adaptive stochastic optimization method in machine learning, and establishes its convergence for linear inverse problems in both the absence and presence of data noise. The second develops a neural-network-based learning framework for inverse scattering problems with multi-frequency data. Both parts combine computational methodology with theoretical analysis, aiming to clarify the convergence, stability, and reconstruction mechanisms of machine learning methods for inverse problems.

For the first part, we apply the ADAM algorithm to finite-dimensional linear inverse problems, with motivation from large-scale computation, and establish convergence rates for the iterated solutions. In the absence of noise, a sub-exponential convergence rate is obtained. In the presence of noisy data, the error analysis reveals the interaction between optimization error and noise amplification, leading to a stopping criterion for the ADAM iterations. The analysis is based on suitable Lyapunov functions constructed by viewing the ADAM iteration as a stochastic dynamical system with respect to the iteration number. Numerical experiments support the theoretical findings and compare the performance of ADAM with stochastic gradient descent.

For the second part, we propose a multi-level neural network framework for solving inverse scattering problems with multi-frequency data. The neural network is built along the frequency axis of the scattering problem: at each fixed frequency, a new level is added to the existing architecture to update the reconstruction. By marching through the frequency levels, the proposed framework progressively recovers higher-order Fourier modes of the imaging target as the network depth grows and higher-frequency data are used. Moreover, the overall learning problem is formulated as a sequence of simpler local tasks at each frequency, reducing the optimization difficulty and the risk of being trapped in undesirable local minima. Numerical experiments for inverse source and inverse medium scattering problems illustrate the effectiveness and robustness of the method. Theoretical analysis in the neural tangent kernel regime further explains this progressive recovery mechanism.

In summary, these two parts show how machine learning methods, including adaptive stochastic optimization and neural network architectures, can be analyzed and designed to develop reliable computational methods for inverse problems.