When
Where
NOTE SPECIAL DAY: Friday, March 7
TIME CHANGE! Talk from 1:00 to 1:50 pm (same location)
Available in person in Harshbarger 110 or via zoom (see email link)
Abstract
Big data and artificial intelligence (AI) methods are revolutionizing how knowledge is gained and predictions are made for sustainability sciences and the global environment. However, purely data-driven AI models often suffer from performance penalties when applied to data-scarce regions or extreme regimes, besides lacking interpretability. Here we show that differentiable models (a genre of physics-informed machine learning where gradients can be rapidly computed via a range of methods, allowing process-based equations to be seamlessly trained together with neural networks, https://t.co/qyuAzYPA6Y) are well-suited to capture unseen extremes because they utilize physical principles like mass balances and first-order exchanges to restrict the role of neural networks. They offer highly competitive performance and simultaneously provide physical process clarity. We further consider solving process-rich partial differential equation efficiently using AI methods (neural operators) with the correct sensitivity. Recent advances in Fourier Neural Operator (FNO), have demonstrated significant efficiency in approximating solution paths u. However, these approximations result in inaccurate solutions to inverse problems due to inaccurate sensitivities and are highly sensitive to concept drift. We propose Sensitivity-Constrained Fourier Neural Operators (SC-FNO, to be presented in ICLR 2025). SC-FNO ensures accuracy in the solution paths, inverse problems, and sensitivity calculations, even under sparse training data or concept drift scenarios. Our approach significantly outperforms both the original FNO and FNO combined with Physics-Informed Neural Network regularization (FNO-PINN) on multiple tasks. Differentiable modeling together with sensitivity-constrained neural operators are posed to drastically improve our simulation and learning capabilities for a wide range of engineering and geoscientific problems.
Nature Reviews/Earth and Environment (11 July 2023): Differentiable modelling to unify machine learning and physical models for geosciences
Bio
Chaopeng Shen Multi-Scale Hydrology, Processes, and Intelligence Group at Pennsylvania State University
Our research focuses on advancing the fundamental understanding of the interactions between hydrology and other subsystems (e.g., ecosystems, energy and carbon cycles, solid earth and channels). Water scarcity and excess create varied conflicts and competitions in different parts of the world, and drastic changes in the water cycle put stress on natural and societal systems. Importantly, the changes in water states and flows are a significant driver for changes in other systems. We strive to provide sound physical science, produced by data, data-driven, and process-based models, to support decision-making from catchment to global scales. Meanwhile, our fundamental understanding of the hydrologic cycle, despite decades of research, still has much to be improved. We strive to identify commonalities and learn underlying principles.
Our methods include state-of-the-art deep learning (DL) and physics-based hydrologic models. Recently, we demonstrated a genre of physics-informed machine learning called "differentiable modeling". Differentiable models mix process-based equations (called priors) and neural networks (NNs) at a fundamental level, so they can be trained together in one stage (called "end-to-end"). This way, the network components can be supervised indirectly by outputs of the combined system, and do not necessarily need training data for their direct outputs (such as model parameters). Differentiability can be supported by automatic differentiation, adjoints, or any other method that can efficiently produce gradients of loss with respect to large amounts of parameters. Such models can train a neural network using big data while respecting physical laws, improving the generalizability, robustness, and complexity of the learned relationships. We have found massive scale, efficiency, and performance advantages with differentiable models in rainfall-runoff, routing, ecosystem, and water quality modeling. Our newest version even surpasses state-of-the-art LSTM models in data-dense regions. See our benchmarks for more information.
Our hydrologic deep learning and differentiable model codes are open source and publicly available.
Besides machine learning, another tool we used in the past is the Process-based Adaptive Watershed Simulator (PAWS), a comprehensive, computationally-efficient parallel hydrologic model designed for large-scale simulation. The model is coupled to the Community Land Model (CLM), and therefore is able to simulate carbon/nitrogen cycling, ecosystem dynamics, and their interactions with the water cycle. PAWS is now open to all users. READ repo access will be granted to anyone who requests it by sending an email to cshen@engr.psu.edu.