Hybrid Physical Models

Blog Posts

Discussion Seminars and Webinars

Innovation Grants

Talks

Workshop Papers

Venue Title
NeurIPS 2024 AQ-PINNs: Attention-Enhanced Quantum Physics-Informed Neural Networks for Carbon-Efficient Climate Modeling (Papers Track)
Abstract and authors: (click to expand)

Abstract: The growing computational demands of artificial intelligence (AI) in addressing climate change raise significant concerns about inefficiencies and environmental impact, as highlighted by Jevons' paradox. We propose an attention-enhanced quantum physics-informed neural network (AQ-PINN) to tackle these challenges. This approach integrates quantum computing techniques into physics-informed neural networks (PINNs) for climate modeling, aiming to enhance predictive accuracy in fluid dynamics governed by the Navier-Stokes equations while reducing the computational burden and carbon footprint. By harnessing variational quantum multi-head self-attention mechanisms, our AQ-PINN achieves a 51.51% reduction in model parameters compared to classical multi-head self-attention methods while maintaining comparable convergence and loss. It also employs quantum tensor networks to enhance representational capacity, which can lead to more efficient gradient computations and reduced susceptibility to barren plateaus. Our AQ-PINN represents a crucial step towards more sustainable and effective climate modeling solutions.

Authors: Siddhant Dutta (SVKM's Dwarkadas J. Sanghvi College of Engineering); Nouhaila Innan (New York University Abu Dhabi); Sadok Ben Yahia (Southern Denmark University); Muhammad Shafique (New York University Abu Dhabi)
NeurIPS 2024 Towards turbine-location-aware multi-decadal wind power predictions with CMIP6 (Papers Track)
Abstract and authors: (click to expand)

Abstract: With the increasing amount of renewable energy in the grid, long-term wind power forecasting for multiple decades becomes more critical. In these long-term forecasts, climate data is essential as it allows us to account for climate change. Yet the resolution of climate models is often very coarse. In this paper, we show that by including turbine locations when downscaling with Gaussian Processes, we can generate valuable aggregate wind power predictions despite the low resolution of the CMIP6 climate models. This work is a first step towards multi-decadal turbine-location-aware wind power forecasting using global climate model output.

Authors: Nina Effenberger (University of Tübingen); Nicole Ludwig (University of Tübingen)
NeurIPS 2024 RAIN: Reinforcement Algorithms for Improving Numerical Weather and Climate Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: This study explores integrating reinforcement learning (RL) with idealised climate models to address key parameterisation challenges in climate science. Current climate models rely on complex mathematical parameterisations to represent sub-grid scale processes, which can introduce substantial uncertainties. RL offers capabilities to enhance these parameterisation schemes, including direct interaction, handling sparse or delayed feedback, continuous online learning, and long-term optimisation. We evaluate the performance of eight RL algorithms on two idealised environments: one for temperature bias correction, another for radiative-convective equilibrium (RCE) imitating real-world computational constraints. Results show different RL approaches excel in different climate scenarios with exploration algorithms performing better in bias correction, while exploitation algorithms proving more effective for RCE. These findings support the potential of RL-based parameterisation schemes to be integrated into global climate models, improving accuracy and efficiency in capturing complex climate dynamics. Overall, this work represents an important first step towards leveraging RL to enhance climate model accuracy, critical for improving climate understanding and predictions. Code accessible at https://github.com/p3jitnath/climate-rl.

Authors: Pritthijit Nath (University Of Cambridge); Henry Moss (Secondmind); Emily Shuckburgh (University Of Cambridge); Mark Webb (Met Office)
NeurIPS 2024 Exploring Physics-Informed Neural Networks for Crop Yield Loss Forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: In response to climate change, assessing crop productivity under extreme weather conditions is essential to enhance food security. Crop simulation models, which align with physical processes, offer explainability but often perform poorly. Conversely, machine learning (ML) models for crop modeling are powerful and scalable yet operate as black boxes and lack adherence to crop growth’s physical principles. To bridge this gap, we propose a novel method that combines the strengths of both approaches by estimating the water use and the crop sensitivity to water scarcity at the pixel level. This approach enables yield loss estimation grounded in physical principles by sequentially solving the equation for crop yield response to water scarcity, using an enhanced loss function. Leveraging Sentinel-2 satellite imagery, climate data, simulated water use data, and pixel-level yield data, our model demonstrates high accuracy, achieving an $R^2$ of up to 0.77—matching or surpassing state-of-the-art models like RNNs and Transformers. Additionally, it provides interpretable and physical consistent outputs, supporting industry, policymakers, and farmers in adapting to extreme weather conditions.

Authors: Miro Miranda (University of Kaiserslautern-Landau); Marcela Charfuelan (German Research Center for Artificial Intelligence); Andreas Dengel (DFKI GmbH)
NeurIPS 2024 Modeling Pollution Spread with Obstructions using Physics-Informed Neural Networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: Pollution modeling plays a crucial role in combating climate change and protecting public health. Scientists rely on traditional numerical methods, such as finite difference and computational fluid dynamics, for accurate pollution simulations; however, they are resource-intensive and time-consuming often taking days to execute. Physics-Informed Neural Networks (PINNs) present a promising alternative, capable of significantly improving speed while maintaining high accuracy. In our research, we developed a PINN to model pollution spread under laminar flow conditions in a two-dimensional environment with obstructions and reflecting boundaries. Our model integrates the Navier-Stokes and advection-diffusion partial differential equations (PDEs) and enforces zero-flux Neumann boundaries at obstruction surfaces and simulation edges. We used a hybrid learning approach, generating a dataset of 6.1 million colocation points for supervised learning and embedding physical laws into the loss function for unsupervised learning. In our experiments, the PINN was over 2520 times faster than the traditional numerical solver, returning results in under 2190 milliseconds and achieving a mean squared error below 3E-5. Our findings demonstrate that PINNs not only offer a drastic reduction in computational time but also scale favorably with both the time domain and spatial resolution, making them a viable solution for real-time pollution monitoring and emergency response planning. Future work will focus on extending our model to dynamic obstructions and arbitrary grid geometries.

Authors: Yash Ranjith (Westmont)
NeurIPS 2024 RL for Mitigating Cascading Failures: Targeted Exploration via Sensitivity Factors (Papers Track)
Abstract and authors: (click to expand)

Abstract: Electricity grid's resiliency and climate change strongly impact one another due to an array of technical and policy-related decisions that impact both. This paper introduces a physics-informed machine learning-based framework to enhance grid's resiliency. Specifically, when encountering disruptive events, this paper designs remedial control actions to prevent blackouts. The proposed~\textbf{P}hysics-\textbf{G}uided \textbf{R}einforcement \textbf{L}earning (PG-RL) framework determines effective real-time remedial line-switching actions, considering their impact on power balance, system security, and grid reliability. To identify an effective blackout mitigation policy, PG-RL leverages power-flow sensitivity factors to guide the RL exploration during agent training. Comprehensive evaluations using the Grid2Op platform demonstrate that incorporating physical signals into RL significantly improves resource utilization within electric grids and achieves better blackout mitigation policies -- both of which are critical in addressing climate change.

Authors: Anmol Dwivedi (RPI); Ali Tajer (RPI); Santiago Paternain (Rensselaer Polytechnic Institute); Nurali Virani (GE Research)
NeurIPS 2024 A Hybrid Machine Learning Model For Ship Speed ThroughWater: Solve And Predict (Proposals Track)
Abstract and authors: (click to expand)

Abstract: This research proposes a hybrid model for predicting ship speed through water, addressing challenges in estimating GHG emissions from shipping while contributing to climate change mitigation. Predicting ship speed through water is a key element in calculating GHG emissions. However, few models address this prediction in a way that integrates both physical principles and machine learning. Our approach combines physical modeling with data-driven techniques to predict real ship speed through water in two key steps: "Solve" and "Predict". In the first step “Solve”, a differential equation is resolved to estimate speed through calm water. "Predict" step uses a machine learning model that incorporates maritime and meteorological conditions and historical data to improve speed predictions in real-world conditions. This hybrid approach leverages both physics-based knowledge and machine learning models to provide a more comprehensive solution for accurately predicting ship speed through water.

Authors: Zakarya ELMIMOUNI (ENSAE); Ayoub Atanane (UQAR); Loubna Benabbou (UQAR)
NeurIPS 2024 Enhanced PINNs for high-order power grid dynamics (Proposals Track)
Abstract and authors: (click to expand)

Abstract: We develop improved physics-informed neural networks (PINNs) for high-order and high-dimensional power system models described by nonlinear ordinary differential equations. We propose some novel enhancements to improve PINN training and accuracy and also implement several other recently proposed ideas from the literature. We successfully apply these to study the transient dynamics of synchronous generators. We also make progress towards applying PINNs to advanced inverter models. Such enhanced PINNs can allow us to accelerate high-fidelity simulations needed to ensure a stable and reliable renewables-rich future grid.

Authors: Vineet Jagadeesan Nair (MIT)
ICLR 2024 A Deep Learning Technology Suite for Cost-Effective Sequestered CO2 Monitoring (Papers Track)
Abstract and authors: (click to expand)

Abstract: Carbon capture and storage (CCS) is a way of reducing carbon emissions to help tackle global warming. Injecting CO2 into rock formations and preventing it from escaping to the surface is a main step in a CCS project. Therefore, monitoring of geologically sequestered CO2 is important for CCS security assessment. Time-lapse seismic (4D seismic) is one of the most effective tools for CO2 monitoring. Unfortunately, the main challenge of 4D seismic is the high cost due to repeated monitoring seismic data acquisition surveys and the subsequent time-consuming data processing that involves imaging and inversion. To address this, we developed a technology suite powered by deep learning engines that significantly reduces the cost by (1) acquiring very sparse monitoring data; (2) firing multiple seismic sources simultaneously; (3) converting 2D images to 3D volume; (4) enforcing repeatability between baseline data and monitoring data; and (5) nonlinearly mapping seismic data to subsurface property model to bypass complex wave-equation-based seismic data processing procedures.

Authors: Wenyi Hu (SLB); Son Phan (SLB); Cen Li (SLB); Aria Abubakar (SLB)
ICLR 2024 Generalizable Temperature Nowcasting with Physics-Constrained RNNs for Predictive Maintenance of Wind Turbine Components (Papers Track)
Abstract and authors: (click to expand)

Abstract: Machine learning plays an important role in the operation of current wind energy production systems. One central application is predictive maintenance to increase efficiency and lower electricity costs by reducing downtimes. Integrating physics-based knowledge in neural networks to enforce their physical plausibilty is a promising method to improve current approaches, but incomplete system information often impedes their application in real world scenarios. We describe a simple and efficient way for physics-constrained deep learning-based predictive maintenance for wind turbine gearbox bearings with partial system knowledge. The approach is based on temperature nowcasting constrained by physics, where unknown system coefficients are treated as learnable neural network parameters. Results show improved generalization performance to unseen environments compared to a baseline neural network, which is especially important in low data scenarios often encountered in real-world applications.

Authors: Johannes Exenberger (Graz University of Technology); Matteo Di Salvo (Sirius Energy Automation); Thomas Hirsch (Graz University of Technology); Franz Wotawa (Graz University of Technology, Institute for Software Technology); Gerald Schweiger (Graz University of Technology)
ICLR 2024 A cautionary tale about deep learning-based climate emulators (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate models are computationally too expensive for many tasks, such as, rapidly exploring future impacts of climate policies. Thus, since the 1980s scientists have been developing lightweight approximations or emulators of climate models. Recently, deep learning has been proposed for this task and most commonly been evaluated on the benchmark ClimateBenchv1.0. We implemented a linear regression-based model from the 1990s with 30K parameters, called linear pattern scaling, that is now the 'best' model on ClimateBenchv1.0 -- outperforming the incumbent 100M-parameter foundation model, ClimaX, on the spatial error of 3 out of the 4 variables. Nevertheless, climate emulation might benefit from innovations in machine learning and we analyse two aspects that need to be addressed in future emulators: First, the data complexity depends strongly on the climate variable of interest and the chosen spatiotemporal resolution. Second, current benchmarks do not sufficiently address the large impact of interannual variability in the climate system. We have published our analysis as an interactive tutorial at github.com/ygaxolotl/tags-linear-pattern-scaling.

Authors: Björn Lütjens (Massachussets Institute of Technology); Raffaele Ferrari (Massachusetts Institute of Technology); Paolo Giani (Massachusetts Institute of Technology); Dava Newman (MIT); Andre Souza (Massachusetts Institute of Technology); Duncan Watson-Parris (University of California San Diego); Noelle Selin (Massachusetts Institute of Technology)
ICLR 2024 Severe Wind Event Prediction with Multivariate Physics-Informed Deep Learning (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Wind turbines play a crucial role in combating climate change by harnessing the force of the wind to generate clean and renewable energy. One key factor in ensuring the long-term effectiveness of wind turbines is the reduction of operating costs due to maintenance. Severe weather events, such as extreme changes in wind, can damage turbines, resulting in costly maintenance and economic losses in power production. We propose a preliminary physics-informed deep learning model to improve predictions of severe wind events and a multivariate time series extension for this work.

Authors: Willa Potosnak (Carnegie Mellon University); Cristian I Challu (Carnegie Mellon University); Kin G. Olivares (Carnegie Mellon University); James K Miller (Carnegie Mellon University); Artur Dubrawski (Carnegie Mellon University)
NeurIPS 2023 Physics-Informed Domain-Aware Atmospheric Radiative Transfer Emulator for All Sky Conditions (Papers Track)
Abstract and authors: (click to expand)

Abstract: Radiative transfer modeling is a complex and computationally expensive process that is used to predict how radiation interacts with the atmosphere and Earth's surface. The Rapid Radiation Transfer Model (RRTM) is one such process model that is used in many Earth system models. In recent years, there has been a growing interest in using machine learning (ML) to speed up radiative transfer modeling. ML algorithms can be trained on large datasets of existing RRTM simulations to learn how to predict the results of new simulations without having to run the full RRTM model so one can use the algorithm for new simulations with very light computational demand. This study developed a new physics-based ML emulator for RRTM that is built on a convolutional neural network (CNN) where we trained the CNN on a dataset of 28 years of RRTM simulations. We built a custom loss function, which incorporates information on how radiation interacts with clouds at day- and night-time. The emulator was able to learn how to predict the vertical heating rates in the atmosphere with a high degree of accuracy (RMSE of less than 2% and Pearson's correlation above 0.9). The new ML emulator is over 56 times faster than the original RRTM model on traditional multi-CPU machines. This speedup could allow scientists to call the RRTM much more frequently in atmosphere models, which may improve the accuracy of climate models and reduce the uncertainty in the future climate projections.

Authors: Piyush Garg (Argonne National Laboratory); Emil Constantinescu (Argonne National Laboratory); Bethany Lusch (Argonne National Lab); Troy Arcomano (Argonne National Laboratory); Jiali Wang (Argonne National Laboratory); Rao Kotamarthi (Argonne National Laboratory)
NeurIPS 2023 Extreme Event Prediction with Multi-agent Reinforcement Learning-based Parametrization of Atmospheric and Oceanic Turbulence (Papers Track)
Abstract and authors: (click to expand)

Abstract: Global climate models (GCMs) are the main tools for understanding and predicting climate change. However, due to limited numerical resolutions, these models suffer from major structural uncertainties; e.g., they cannot resolve critical processes such as small-scale eddies in atmospheric and oceanic turbulence. Thus, such small-scale processes have to be represented as a function of the resolved scales via closures (parametrization). The accuracy of these closures is particularly important for capturing climate extremes. Traditionally, such closures are based on heuristics and simplifying assumptions about the unresolved physics. Recently, supervised-learned closures, trained offline on high-fidelity data, have been shown to outperform the classical physics-based closures. However, this approach requires a significant amount of high-fidelity training data and can also lead to instabilities. Reinforcement learning is emerging as a potent alternative for developing such closures as it requires only low-order statistics and leads to stable closures. In Scientific Multi-Agent Reinforcement Learning (SMARL) computational elements serve a dual role of discretization points and learning agents. Here, we leverage SMARL and fundamentals of turbulence physics to learn closures for canonical prototypes of atmospheric and oceanic turbulence. The policy is trained using only the enstrophy spectrum, which is nearly invariant and can be estimated from a few high-fidelity samples. We show that these closures lead to stable low-resolution simulations that, at a fraction of the cost, can reproduce the high-fidelity simulations' statistics, including the tails of the probability density functions (PDFs). These results demonstrate the high potential of SMARL for closure modeling for GCMs, especially in the regime of scarce data and indirect observations.

Authors: Rambod Mojgani (Rice University); Daniel Waelchli (ETHZ); Yifei Guan (Rice University); Petros Koumoutsakos (Harvard); Pedram Hassanzadeh (Rice University)
NeurIPS 2023 Hybridizing Physics and Neural ODEs for Predicting Plasma Inductance Dynamics in Tokamak Fusion Reactors (Papers Track)
Abstract and authors: (click to expand)

Abstract: While fusion reactors known as tokamaks hold promise as a firm energy source, advances in plasma control, and handling of events where control of plasmas is lost, are needed for them to be economical. A significant bottleneck towards applying more advanced control algorithms is the need for better plasma simulation, where both physics-based and data-driven approaches currently fall short. The former is bottle-necked by both computational cost and the difficulty of modelling plasmas, and the latter is bottle-necked by the relative paucity of data. To address this issue, this work applies the neural ordinary differential equations (ODE) framework to the problem of predicting a subset of plasma dynamics, namely the coupled plasma current and internal inductance dynamics. As the neural ODE framework allows for the natural inclusion of physics-based inductive biases, we train both physics-based and neural network models on data from the Alcator C-Mod fusion reactor and find that a model that combines physics-based equations with a neural ODE performs better than both existing physics-motivated ODEs and a pure neural ODE model.

Authors: Allen Wang (MIT); Cristina Rea (MIT); Darren Garnier (MIT)
NeurIPS 2023 Explainable Offline-online Training of Neural Networks for Multi-scale Climate Modeling (Papers Track)
Abstract and authors: (click to expand)

Abstract: In global climate models, small-scale physical processes are represented using subgrid-scale (SGS) models known as parameterizations, and these parameterizations contribute substantially to uncertainties in climate projections. Recently, machine learning techniques, particularly deep neural networks (NNs), have emerged as novel tools for developing SGS parameterizations. Different strategies exist for training these NN-based SGS models. Here, we use a 1D model of the quasi-biennial oscillation (QBO) and atmospheric gravity wave (GW) parameterizations as testbeds to explore various learning strategies and challenges due to scarcity of high-fidelity training data. We show that a 12-layer convolutional NN that predicts GW forcings for given wind profiles, when trained offline in a big-data regime (100-years), produces realistic QBOs once coupled to the 1D model. In contrast, offline training of this NN in a small-data regime (18-months) yields unrealistic QBOs. However, online re-training of just two layers of this NN using ensemble Kalman inversion and only time-averaged QBO statistics leads to parameterizations that yield realistic QBOs. Fourier analysis of these three NNs’ kernels suggests how/why re-training works and reveals that these NNs primarily learn low-pass, high-pass, and a combination of band-pass Gabor filters, consistent with the importance of both local and non-local dynamics in GW propagation/dissipation. These strategies/findings apply to data-driven parameterizations of other climate processes generally.

Authors: Hamid Alizadeh Pahlavan (Rice University); Pedram Hassanzadeh (Rice University); M. Joan Alexander (NorthWest Research Associates)
NeurIPS 2023 Stress-testing the coupled behavior of hybrid physics-machine learning climate simulations on an unseen, warmer climate (Papers Track)
Abstract and authors: (click to expand)

Abstract: Accurate and computationally-viable representations of clouds and turbulence are a long-standing challenge for climate model development. Traditional parameterizations that crudely but efficiently approximate these processes are a leading source of uncertainty in long-term projected warming and precipitation patterns. Machine Learning (ML)-based parameterizations have long been hailed as a promising alternative with the potential to yield higher accuracy at a fraction of the cost of more explicit simulations. However, these ML variants are often unpredictably unstable and inaccurate in online testing (i.e. in a downstream hybrid simulation task where they are dynamically coupled to the large-scale climate model). These issues are exacerbated in out-of-distribution climates. Certain design decisions such as ``climate-invariant" feature transformation, input vector expansion, and temporal history incorporation have been shown to improve online performance, but they may be insufficient for the mission-critical task of online out-of-distribution generalization. If feature selection and transformations can inoculate hybrid physics-ML climate models from non-physical out-of-distribution extrapolation in a changing climate, there is far greater potential in extrapolating from observational data. Otherwise, training on multiple simulated climates becomes an inevitable necessity. While our results show generalization benefits from these design decisions, such benefits do not sufficiently preclude the necessity of using multi-climate simulated training data.

Authors: Jerry Lin (University of California, Irvine); Mohamed Aziz Bhouri (Columbia University); Tom G Beucler (Columbia University & UCI); Sungduk Yu (University of California, Irvine); Michael Pritchard (UCI)
NeurIPS 2023 Continuous Convolutional Neural Networks for Disruption Prediction in Nuclear Fusion Plasmas (Papers Track)
Abstract and authors: (click to expand)

Abstract: Grid decarbonization for climate change requires dispatchable carbon-free energy like nuclear fusion. The tokamak concept offers a promising path for fusion, but one of the foremost challenges in implementation is the occurrence of energetic plasma disruptions. In this study, we delve into Machine Learning approaches to predict plasma state outcomes. Our contributions are twofold: (1) We present a novel application of Continuous Convolutional Neural Networks for disruption prediction and (2) We examine the advantages and disadvantages of continuous models over discrete models for disruption prediction by comparing our model with the previous, discrete state of the art, and show that continuous models offer significantly better performance (Area Under the Receiver Operating Characteristic Curve = 0.974 v.s. 0.799) with fewer parameters.

Authors: William F Arnold (KAIST); Lucas Spangher (MIT PSFC); Cristina Rea (MIT PSFC)
NeurIPS 2023 Learning to forecast diagnostic parameters using pre-trained weather embedding (Papers Track)
Abstract and authors: (click to expand)

Abstract: Data-driven weather prediction (DDWP) models are increasingly becoming popular for weather forecasting. However, while operational weather forecasts predict a wide variety of weather variables, DDWPs currently forecast a specific set of key prognostic variables. Non-prognostic ("diagnostic") variables are sometimes modeled separately as dependent variables of the prognostic variables (c.f. FourCastNet \cite{pathak2022fourcastnet}), or by including the diagnostic variable as a target in the DDWP. However, the cost of training and deploying bespoke models for each diagnostic variable can increase dramatically with more diagnostic variables, and limit the operational use of such models. Likewise, retraining an entire DDWP each time a new diagnostic variable is added is also cost-prohibitive. We present an two-stage approach that allows new diagnostic variables to be added to an end-to-end DDWP model without the expensive retraining. In the first stage, we train an autoencoder that learns to embed prognostic variables into a latent space. In the second stage, the autoencoder is frozen and "downstream" models are trained to predict diagnostic variables using only the latent representations of prognostic variables as input. Our experiments indicate that models trained using the two-stage approach offer accuracy comparable to training bespoke models, while leading to significant reduction in resource utilization during training and inference. This approach allows for new "downstream" models to be developed as needed, without affecting existing models and thus reducing the friction in operationalizing new models.

Authors: Peetak Mitra (Excarta); Vivek Ramavajjala (Excarta)
NeurIPS 2023 Physics-informed DeepONet for battery state prediction (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Electrification has emerged as a pivotal trend in the energy transition to address climate change, leading to a substantial surge in the demand for batteries. Accurately predicting the internal states and performance of batteries assumes paramount significance, as it ensures the safe and stable operation of batteries and informs decision-making processes, such as optimizing battery operation for arbitrage opportunities. However, current models struggle to strike a balance between precision and computational efficiency or are limited in their applicability to specific scenarios. We aim to adopt a physics-informed deep operator network (PI-DeepONet) for internal battery state estimation based on the rigorous P2D model, which can simultaneously achieve high precision and computational efficiency. Furthermore, it exhibits promising prospects for extension beyond lithium-ion batteries to encompass various battery technologies.

Authors: Keyan Guo (Peking University)
ICLR 2023 Improving global high-resolution Earth system model simulations of precipitation with generative adversarial networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: Precipitation extremes are expected to become stronger and more frequent in response to anthropogenic global warming. Accurately projecting the ecological and socioeconomic impacts is an urgent task. Impact models are developed and calibrated with observation-based data but rely on Earth system model (ESM) output for future scenarios. ESMs, however, exhibit significant biases in their output because they cannot fully resolve complex cross-scale interactions of processes that produce precipitation cannot. State-of-the-art bias correction methods only address errors in the simulated frequency distributions, locally at every individual grid cell. Improving unrealistic spatial patterns of the ESM output, which would require spatial context, has not been possible so far. Here we show that a post-processing method based on physically constrained generative adversarial networks (GANs) can correct biases of a state-of-the-art global ESM both in local frequency distributions and in the spatial patterns at once. While our method improves local frequency distributions similarly well to a gold-standard ESM bias-adjustment framework, it strongly outperforms existing methods in correcting spatial patterns. Our study highlights the importance of physical constraints in neural networks for out-of-sample predictions in the context of climate change.

Authors: Philipp Hess (Technical University of Munich)
ICLR 2023 Improving a Shoreline Forecasting Model with Symbolic Regression (Papers Track)
Abstract and authors: (click to expand)

Abstract: Given the current context of climate change and the increasing population densities at coastal zones around the globe, there is an increasing need to be able to predict the development of our coasts. Recent advances in artificial intelligence allow for automatic analysis of observational data. Symbolic Regression (SR) is a type of Machine Learning algorithm that aims to find interpretable symbolic expressions that can explain relations in the data. In this work, we aim to study the problem of forecasting shoreline change using SR. We make use of Cartesian Genetic Programming (CGP) in order to encode and improve upon ShoreFor, a physical shoreline prediction model. During training, CGP individuals are evaluated and selected according to their predictive score at five different coastal sites. This work presents a comparison between a CGP-evolved model and the base ShoreFor model. In addition to evolution's ability to produce well-performing models, it demonstrates the usefulness of SR as a research tool to gain insight into the behaviors of shorelines in various geographical zones.

Authors: Mahmoud AL NAJAR (Laboratory of Spatial Geophysics and Oceanography Studies); Rafael ALMAR (Laboratory of Spatial Geophysics and Oceanography Studies); Erwin BERGSMA (CNES); Jean-Marc DELVIT (CNES); Dennis Wilson (ISAE)
ICLR 2023 Disentangling observation biases to monitor spatio-temporal shifts in species distributions (Proposals Track)
Abstract and authors: (click to expand)

Abstract: The accelerated pace of environmental change due to anthropogenic activities makes it more important than ever to understand current and future ecosystem dynamics at a global scale. Species observations stemming from citizen science platforms are increasingly leveraged to gather information about the geographic distributions of many species. However, their usability is limited by the strong biases inherent to these community-driven efforts. These biases in the sampling effort are often treated as noise that has to be compensated for. In this project, we posit that better modelling the sampling effort (including the usage of the different platforms across countries, local accessibility, attractiveness of the location for platform users, affinity of different user groups for different species, etc.) is the key towards improving Species Distribution Models (SDM) using observations from citizen science platforms, thus opening up the possibility of leveraging them to monitor changes in species distributions and population densities.

Authors: Diego Marcos (Inria); Christophe Botella (); Ilan Havinga (Wageningen University); Dino Ienco (INRAE); Cassio F. Dantas (TETIS, INRAE, Univ Montpellier); Pierre Alliez (INRIA Sophie-Antipolis, France); Alexis Joly (INRIA, FR)
ICLR 2023 A High-Resolution, Data-Driven Model of Urban Carbon Emissions (Papers Track) Best Pathway to Impact
Abstract and authors: (click to expand)

Abstract: Cities represent both a fundamental contributor to greenhouse (GHG) emissions and a catalyst for climate action. Many global cities have outlined sustainability and climate change mitigation plans, focusing on energy efficiency, shifting away from fossil fuels, and prioritizing environmental and social justice. To achieve broad-based and equitable carbon emissions reductions and sustainability goals, new data-driven methodologies are needed to identify and target efficiency and carbon reduction opportunities in the built environment at the building, neighborhood, and city-scale. Our methodology integrates data from numerous data sources and develops data-driven and physical models of energy use and carbon emissions from buildings and transportation to generate a high spatiotemporal resolution model of urban greenhouse gas emissions. The method and data tool are designed to support city leaders and urban policymakers with an unprecedented view of localized carbon emissions to enable data-driven and evidenced-based climate action.

Authors: Bartosz Bonczak (New York University); Boyeong Hong (New York University); Constantine E. Kontokosta (New York University)
ICLR 2023 Modelling Atmospheric Dynamics with Spherical Fourier Neural Operators (Papers Track)
Abstract and authors: (click to expand)

Abstract: Fourier Neural Operators (FNOs) have established themselves as an efficient method for learning resolution-independent operators in a wide range of scientific machine learning applications. This can be attributed to their ability to effectively model long-range dependencies in spatio-temporal data through computationally ef- ficient global convolutions. However, the use of discrete Fourier transforms (DFTs) in FNOs leads to spurious artifacts and pronounced dissipation when applied to spherical coordinates, due to the incorrect assumption of flat geometry. To ad- dress the issue, we introduce Spherical FNOs (SFNOs), which use the generalized Fourier transform for learning operators on spherical geometries. We demonstrate the effectiveness of the method for forecasting atmospheric dynamics, producing stable auto-regressive results for a simulated time of one year (1,460 steps) while retaining physically plausible dynamics. This development has significant implica- tions for machine learning-based climate dynamics emulation, which could play a crucial role in accelerating our response to climate change.

Authors: Boris Bonev (NVIDIA); Thorsten Kurth (Nvidia); Christian Hundt (NVIDIA AI Technology Center); Jaideep Pathak (NVIDIA Corporation); Maximilian Baust (NVIDIA); Karthik Kashinath (NVIDIA); Anima Anandkumar (NVIDIA/Caltech)
ICLR 2023 Data-driven multiscale modeling of subgrid parameterizations in climate models (Papers Track) Best ML Innovation
Abstract and authors: (click to expand)

Abstract: Subgrid parameterizations that represent physical processes occurring below the resolution of current climate models are an important component in producing accurate, long-term predictions for the climate. A variety of approaches have been tested to design these components, including deep learning methods. In this work, we evaluate a proof of concept illustrating a multiscale approach to this prediction problem. We train neural networks to predict subgrid forcing values on a testbed model and examine improvements in prediction accuracy which can be obtained by using additional information in both fine-to-coarse and coarse-to-fine directions.

Authors: Karl Otness (New York University); Laure Zanna (NYU); Joan Bruna (NYU)
NeurIPS 2022 Aboveground carbon biomass estimate with Physics-informed deep network (Papers Track)
Abstract and authors: (click to expand)

Abstract: The global carbon cycle is a key process to understand how our climate is changing. However, monitoring the dynamics is difficult because a high-resolution robust measurement of key state parameters including the aboveground carbon biomass (AGB) is required. We use deep neural network to generate a wall-to-wall map of AGB within the Continental USA (CONUS) with 30-meter spatial resolution for the year 2021. We combine radar and optical hyperspectral imagery, with a physical climate parameter of solar-induced chlorophyll fluorescence (SIF)-based gross primary productivity (GPP). Validation results show that a masked variation of UNet has the lowest validation RMSE of 37.93 ± 1.36 Mg C/ha, as compared to 52.30 ± 0.03 Mg C/ha for random forest algorithm. Furthermore, models that learn from SIF-based GPP in addition to radar and optical imagery reduce validation RMSE by almost 10% and the standard deviation by 40%. Finally, we apply our model to measure losses in AGB from the recent 2021 Caldor wildfire in California, and validate our analysis with Sentinel-based burn index.

Authors: Juan Nathaniel (Columbia University); Levente Klein (IBM Research); Campbell D Watson (IBM Reserch); Gabrielle Nyirjesy (Columbia University); Conrad M Albrecht (IBM Research)
NeurIPS 2022 Improving the predictions of ML-corrected climate models with novelty detection (Papers Track)
Abstract and authors: (click to expand)

Abstract: While previous works have shown that machine learning (ML) can improve the prediction accuracy of coarse-grid climate models, these ML-augmented methods are more vulnerable to irregular inputs than the traditional physics-based models they rely on. Because ML-predicted corrections feed back into the climate model’s base physics, the ML-corrected model regularly produces out of sample data, which can cause model instability and frequent crashes. This work shows that adding semi-supervised novelty detection to identify out-of-sample data and disable the ML-correction accordingly stabilizes simulations and sharply improves the quality of predictions. We design an augmented climate model with a one-class support vector machine (OCSVM) novelty detector that provides better temperature and precipitation forecasts in a year-long simulation than either a baseline (no-ML) or a standard ML-corrected run. By improving the accuracy of coarse-grid climate models, this work helps make accurate climate models accessible to researchers without massive computational resources.

Authors: Clayton H Sanford (Columbia); Anna Kwa (Allen Institute for Artificial Intelligence); Oliver Watt-Meyer (Allen Institute for AI); Spencer Clark (Allen Institute for AI); Noah Brenowitz (Allen Institute for AI); Jeremy McGibbon (Allen Institute for AI); Christopher Bretherton (Allen Institute for AI)
NeurIPS 2022 Deep Hydrology: Hourly, Gap-Free Flood Maps Through Joint Satellite and Hydrologic Modelling (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change-driven weather disasters are rapidly increasing in both frequency and magnitude. Floods are the most damaging of these disasters, with approximately 1.46 billion people exposed to inundation depths of over 0.15m, a significant life and livelihood risk. Accurate knowledge of flood-extent for ongoing and historical events facilitates climate adaptation in flood-prone communities by enabling near real-time disaster monitoring to support planning, response, and relief during these extreme events. Satellite observations can be used to derive flood-extent maps directly; however, these observations are impeded by cloud and canopy cover, and can be very infrequent and hence miss the flood completely. In contrast, physically-based inundation models can produce spatially complete event maps but suffer from high uncertainty if not frequently calibrated with expensive land and infrastructure surveys. In this study, we propose a deep learning approach to reproduce satellite-observed fractional flood-extent maps given dynamic state variables from hydrologic models, fusing information contained within the states with direct observations from satellites. Our model has an hourly temporal resolution, contains no cloud-gaps, and generalizes to watersheds across the continental United States with a 6% error on held-out areas that never flooded before. We further demonstrate through a case study in Houston, Texas that our model can distinguish tropical cyclones that caused flooding from those that did not within two days of landfall, thereby providing a reliable source for flood-extent maps that can be used by disaster monitoring services.

Authors: Tanya Nair (Cloud To Street); Veda Sunkara (Cloud to Street); Jonathan Frame (Cloud to Street); Philip Popien (Cloud to Street); Subit Chakrabarti (Cloud To Street)
NeurIPS 2022 Hybrid Recurrent Neural Network for Drought Monitoring (Papers Track)
Abstract and authors: (click to expand)

Abstract: Droughts are pervasive hydrometeorological phenomena and global hazards, whose frequency and intensity are expected to increase in the context of climate change. Drought monitoring is of paramount relevance. Here we propose a hybrid model for drought detection that integrates both climatic indices and data-driven models in a hybrid deep learning approach. We exploit time-series of multi-scale Standardized Precipitation Evapotranspiration Index together with precipitation and temperature as inputs. We introduce a dual-branch recurrent neural network with convolutional lateral connections for blending the data. Experimental and ablative results show that the proposed system outperforms both the considered drought index and purely data-driven deep learning models. Our results suggest the potential of hybrid models for drought monitoring and open the door to synergistic systems that learn from data and domain knowledge altogether.

Authors: Mengxue Zhang (Universitat de València); Miguel-Ángel Fernández-Torres (Universitat de València); Gustau Camps-Valls (Universitat de València)
NeurIPS 2022 Short-range forecasts of global precipitation using deep learning-augmented numerical weather prediction (Papers Track)
Abstract and authors: (click to expand)

Abstract: Precipitation drives the hydroclimate of Earth and its spatiotemporal changes on a day to day basis have one of the most notable socioeconomic impacts. The success of numerical weather prediction (NWP) is measured by the improvement of forecasts for various physical fields such as temperature and pressure. Large biases however exist in the precipitation predictions. Pure deep learning based approaches lack the advancements acheived by NWP in the past two to three decades. Hybrid methodology using NWP outputs as inputs to the deep learning based refinement tool offer an attractive means taking advantage of both NWP and state of the art deep learning algorithms. Augmenting the output from a well-known NWP model: Coupled Forecast System ver.2 (CFSv2) with deep learning for the first time, we demonstrate a hybrid model capability (DeepNWP) which shows substantial skill improvements for short-range global precipitation at 1-, 2- and 3-days lead time. To achieve this hybridization, we address the sphericity of the global data by using modified DLWP-CS architecture which transforms all the fields to cubed-sphere projection. The dynamical model outputs corresponding to precipitation and surface temperature are ingested to a UNET for predicting the target ground truth precipitation. While the dynamical model CFSv2 shows a bias in the range of +5 to +7 mm/day over land, the multivariate deep learning model reduces it to -1 to +1 mm/day over global land areas. We validate the results by taking examples from Hurricane Katrina in 2005, Hurricane Ivan in 2004, Central European floods in 2010, China floods in 2010, India floods in 2005 and the Myanmar cyclone Nargis in 2008.

Authors: Manmeet Singh (The University of Texas at Austin); Vaisakh SB (Indian Institute of Tropical Meteorology); Nachiketa Acharya (Department of Meteorology and Atmospheric Science,Pennsylvania State University); Aditya Grover (UCLA); Suryachandra A. Rao (Indian Institute of Tropical Meteorology); Bipin Kumar (Indian Institute of Tropical Meteorology); Zong-Liang Yang (The University of Texas at Austin); Dev Niyogi (The University of Texas at Austin)
NeurIPS 2022 A Multi-Scale Deep Learning Framework for Projecting Weather Extremes (Papers Track) Best Paper: ML Innovation
Abstract and authors: (click to expand)

Abstract: Weather extremes are a major societal and economic hazard, claiming thousands of lives and causing billions of dollars in damage every year. Under climate change, their impact and intensity are expected to worsen significantly. Unfortunately, general circulation models (GCMs), which are currently the primary tool for climate projections, cannot characterize weather extremes accurately. To address this, we present a multi-resolution deep-learning framework that, firstly, corrects a GCM's biases by matching low-order and tail statistics of its output with observations at coarse scales; and secondly, increases the level of detail of the debiased GCM output by reconstructing the finer scales as a function of the coarse scales. We use the proposed framework to generate statistically realistic realizations of the climate over Western Europe from a simple GCM corrected using observational atmospheric reanalysis. We also discuss implications for probabilistic risk assessment of natural disasters in a changing climate.

Authors: Antoine Blanchard (MIT); Nishant Parashar (Verisk Analytics); Boyko Dodov (Verisk Analytics); Christian Lessig (Otto-von-Guericke-Universitat Magdeburg); Themis Sapsis (MIT)
NeurIPS 2022 Transformers for Fast Emulation of Atmospheric Chemistry Box Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: When modeling atmospheric chemistry, concentrations are determined by numerically solving large systems of ordinary differential equations that represent a set of chemical reactions. These solvers can be very computationally intensive, particularly those with the thousands or tens of thousands of chemical species and reactions that make up the most accurate models. We demonstrate the application of a deep learning transformer architecture to emulate an atmospheric chemistry box model, and show that this attention-based model outperforms LSTM and autoencoder baselines while providing interpretable predictions that are more than 2 orders of magnitude faster than a numerical solver. This work is part of a larger study to replace the numerical solver in a 3D global chemical model with a machine learned emulator and achieve significant speedups for global climate simulations.

Authors: Herbie Bradley (University of Cambridge); Nathan Luke Abraham (National Centre for Atmospheric Science, UK); Peer Nowack (Imperial College London); Doug McNeall (Met Office Hadley Centre, UK)
NeurIPS 2022 Dynamic weights enabled Physics-Informed Neural Network for simulating the mobility of Engineered Nano Particles in a contaminated aquifer (Papers Track)
Abstract and authors: (click to expand)

Abstract: Numerous polluted groundwater sites across the globe require an active remediation strategy for the restoration of natural environmental conditions and local ecosystem. The Engineered Nanoparticles (ENPs) has emerged as an efficient reactive agent for the in-situ degradation of groundwater contaminants. While the performance of these ENPs has been highly promising on the laboratory scale, their application in a real field case conditions is still limited. The optimized injection of the ENPs in the contaminated aquifer and its subsequent monitoring are hindered by the complex transport and retention mechanisms of ENPs. Therefore, a predictive tool for understanding the transport and retention behavior of ENPs becomes highly important. The existing tools in the literature are dominated with numerical simulators, which have limited flexibility and accuracy in the presence of sparse dataset. In this work, a dynamic weights enabled Physics-Informed Neural network (dw-PINN) framework is applied to model the nano-particle´s behavior within an aquifer. The result from the forward model demonstrates the effective capability of dw-PINN in accurately predicting the ENPs mobility. The model verification step shows that the mean squared error of the predicted ENPs concentration using dw-PINN converges to a minimum value of 1.3e-5. In the subsequent step, the result from the inverse model estimates the governing parameters of ENPs mobility with reasonable accuracy. The research work demonstrates the tool´s capability in providing predictive insights for the development of an efficient groundwater remediation strategy.

Authors: Shikhar Nilabh (Amphos21)
NeurIPS 2022 An Inversion Algorithm of Ice Thickness and InSAR Data for the State of Friction at the Base of the Greenland Ice Sheet (Proposals Track)
Abstract and authors: (click to expand)

Abstract: With the advent of climate change and global warming, the Greenland Ice Sheet (GrIS) has been melting at an alarming rate, losing over 215 Gt per yr, and accounting for 10% of mean global sea level rise since the 1990s. It is imperative to understand what dynamics are causing ice loss and influencing ice flow in order to successfully project mass changes of ice sheets and associated sea level rise. This work applies machine learning, ice thickness data, and horizontal ice velocity measurements from satellite radar data to quantify the magnitudes and distributions of the basal traction forces that are holding the GrIS back from flowing into the ocean. Our approach uses a hybrid model: InSAR velocity data trains a linear regression model, and these model coefficients are fed into a geophysical algorithm to estimate basal tractions that capture relationships between the ice motion and physical variables. Results indicate promising model performance and reveal significant presence of large basal traction forces around the coastline of the GrIS.

Authors: Aryan Jain (Amador Valley High School); Jeonghyeop Kim (Stony Brook University); William Holt (Stony Brook University)
NeurIPS 2022 Machine Learning for Predicting Climate Extremes (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: Climate change has led to a rapid increase in the occurrence of extreme weather events globally, including floods, droughts, and wildfires. In the longer term, some regions will experience aridification while others will risk sinking due to rising sea levels. Typically, such predictions are done via weather and climate models that simulate the physical interactions between the atmospheric, oceanic, and land surface processes that operate at different scales. Due to the inherent complexity, these climate models can be inaccurate or computationally expensive to run, especially for detecting climate extremes at high spatiotemporal resolutions. In this tutorial, we aim to introduce the participants to machine learning approaches for addressing two fundamental challenges. We will walk the participants through a hands-on tutorial for predicting climate extremes relating to temperature and precipitation in 2 setups: (1) temporal forecasting: the goal is to predict climate variables into the future (both direct single step approaches and iterative approaches that roll out the model for several timesteps), and (2) spatial downscaling: the goal is to learn a mapping that transforms low-resolution outputs of climate models into high-resolution regional forecasts. Through introductory presentations and colab notebooks, we aim to expose the participants to (a) APIs for accessing and navigating popular repositories that host global climate data, such as the Copernicus data store, (b) identifying relevant datasets, including auxiliary data (e.g., other climate variables such as geopotential), (c) scripts for downloading and preprocessing relevant datasets, (d) algorithms for training machine learning models, (d) metrics for evaluating model performance, and (e) visualization tools for both the dataset and predicted outputs. The coding notebooks will be in Python. No prior knowledge of climate science is required.

Authors: Hritik Bansal (UCLA); Shashank Goel (University of California Los Angeles); Tung Nguyen (University of California, Los Angeles); Aditya Grover (UCLA)
AAAI FSS 2022 Generating physically-consistent high-resolution climate data with hard-constrained neural networks
Abstract and authors: (click to expand)

Abstract: The availability of reliable, high-resolution climate and weather data is important to inform long-term decisions on climate adaptation and mitigation and to guide rapid responses to extreme events. Forecasting models are limited by computational costs and therefore often predict quantities at a coarse spatial resolution. Statistical downscaling can provide an efficient method of upsampling low-resolution data. In this field, deep learning has been applied successfully, often using methods from the super-resolution domain in computer vision. Despite often achieving visually compelling results, such models often violate conservation laws when predicting physical variables. In order to conserve important physical quantities, we develop methods that guarantee physical constraints are satisfied by a deep downscaling model while also increasing their performance according to traditional metrics. We introduce two ways of constraining the network: a renormalization layer added to the end of the neural network and a successive approach that scales with increasing upsampling factors. We show the applicability of our methods across different popular architectures and upsampling factors using ERA5 reanalysis data.

Authors: Paula Harder (Fraunhofer Institute ITWM, Mila Quebec AI Institute), Qidong Yang (Mila Quebec AI Institute, New York University), Venkatesh Ramesh (Mila Quebec AI Institute, University of Montreal), Alex Hernandez-Garcia (Mila Quebec AI Institute, University of Montreal), Prasanna Sattigeri (IBM Research), Campbell D. Watson (IBM Research), Daniela Szwarcman (IBM Research) and David Rolnick (Mila Quebec AI Institute, McGill University).
NeurIPS 2021 Hurricane Forecasting: A Novel Multimodal Machine Learning Framework (Papers Track)
Abstract and authors: (click to expand)

Abstract: This paper describes a machine learning (ML) framework for tropical cyclone intensity and track forecasting, combining multiple distinct ML techniques and utilizing diverse data sources. Our framework, which we refer to as Hurricast (HURR), is built upon the combination of distinct data processing techniques using gradient-boosted trees and novel encoder-decoder architectures, including CNN, GRU and Transformers components. We propose a deep-learning feature extractor methodology to mix spatial-temporal data with statistical data efficiently. Our multimodal framework unleashes the potential of making forecasts based on a wide range of data sources, including historical storm data, and visual data such as reanalysis atmospheric images. We evaluate our models with current operational forecasts in North Atlantic (NA) and Eastern Pacific (EP) basins on 2016-2019 for 24-hour lead time, and show our models consistently outperform statistical-dynamical models and compete with the best dynamical models. Furthermore, the inclusion of Hurricast into an operational forecast consensus model leads to a significant improvement of 5% - 15% over NHC's official forecast, thus highlighting the complementary properties with existing approaches.

Authors: Léonard Boussioux (MIT, CentraleSupélec); Cynthia Zeng (MIT); Dimitris Bertsimas (MIT); Théo J Guenais (Harvard University)
NeurIPS 2021 MS-nowcasting: Operational Precipitation Nowcasting with Convolutional LSTMs at Microsoft Weather (Papers Track)
Abstract and authors: (click to expand)

Abstract: We present the encoder-forecaster convolutional long short-term memory (LSTM) deep-learning model that powers Microsoft Weather's operational precipitation nowcasting product. This model takes as input a sequence of weather radar mosaics and deterministically predicts future radar reflectivity at lead times up to 6 hours. By stacking a large input receptive field along the feature dimension and conditioning the model's forecaster with predictions from the physics-based High Resolution Rapid Refresh (HRRR) model, we are able to outperform optical flow and HRRR baselines by 20-25% on multiple metrics averaged over all lead times.

Authors: Sylwester Klocek (Microsoft Corporation); Haiyu Dong (Microsoft); Matthew Dixon (Microsoft Corporation); Panashe Kanengoni (Microsoft Corporation); Najeeb Kazmi (Microsoft); Pete Luferenko (Microsoft Corporation); Zhongjian Lv (Microsoft Corporation); Shikhar Sharma (); Jonathan Weyn (Microsoft); Siqi Xiang (Microsoft Corporation)
NeurIPS 2021 Capturing Electricity Market Dynamics in the Optimal Trading of Strategic Agents using Neural Network Constrained Optimization (Papers Track)
Abstract and authors: (click to expand)

Abstract: In competitive electricity markets the optimal trading problem of an electricity market agent is commonly formulated as a bi-level program, and solved as mathematical program with equilibrium constraints (MPEC). In this paper, an alternative paradigm, labeled as mathematical program with neural network constraint (MPNNC), is developed to incorporate complex market dynamics in the optimal bidding strategy. This method uses input-convex neural networks (ICNNs) to represent the mapping between the upper-level (agent) decisions and the lower-level (market) outcomes, i.e., to replace the lower-level problem by a neural network. In a comparative analysis, the optimal bidding problem of a load agent is formulated via the proposed MPNNC and via the classical bi-level programming method, and compared against each other.

Authors: Mihály Dolányi (KU Leuven); Kenneth Bruninx (KU Leuven); Jean-François Toubeau (Faculté Polytechnique (FPMs), Université de Mons (UMONS)); Erik Delaue (KU Leuven)
NeurIPS 2021 Hybrid physics-based and data-driven modeling with calibrated uncertainty for lithium-ion battery degradation diagnosis and prognosis (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Advancing lithium-ion batteries (LIBs) in both design and usage is key to promoting electrification in the coming decades to mitigate human-caused climate change. Inadequate understanding of LIB degradation is an important bottleneck that limits battery durability and safety. Here, we propose hybrid physics-based and data-driven modeling for online diagnosis and prognosis of battery degradation. Compared to existing battery modeling efforts, we aim to build a model with physics as its backbone and statistical learning techniques as enhancements. Such a hybrid model has better generalizability and interpretability together with a well-calibrated uncertainty associated with its prediction, rendering it more valuable and relevant to safety-critical applications under realistic usage scenarios.

Authors: Jing Lin (Institute for Infocomm Research); Yu Zhang (I2R); Edwin Khoo (Institute for Infocomm Research)
ICML 2021 Physics-Informed Graph Neural Networks for Robust Fault Location in Power Grids (Papers Track) Best Paper: ML Innovation
Abstract and authors: (click to expand)

Abstract: The reducing cost of renewable energy resources, such as solar photovoltaics (PV) and wind farms, is accelerating global energy transformation to mitigate climate change. However, a high level of intermittent renewable energy causes power grids to have more stability issues. This accentuates the need for quick location of system failures and follow-up control actions. In recent events such as in California, line failures have resulted in large-scale wildfires leading to loss of life and property. In this article, we propose a two-stage graph learning framework to locate power grid faults in the challenging but practical regime characterized by (a) sparse observations, (b) low label rates, and (c) system variability. Our approach embeds the geometrical structure of power grids into the graph neural networks (GNN) in stage I for fast fault location, and then stage II further enhances the location accuracy by employing the physical similarity of the labeled and unlabeled data samples. We compare our approach with three baselines in the IEEE 123-node benchmark system and show that it outperforms the others by significant margins in various scenarios.

Authors: Wenting Li (Los Alamos National Laboratory); Deepjyoti Deka (Los Alamos National Laboratory)
NeurIPS 2020 Learning the distribution of extreme precipitation from atmospheric general circulation model variables (Papers Track)
Abstract and authors: (click to expand)

Abstract: Precipitation extremes are projected to become more frequent and severe in a warming atmosphere over the coming decades. However, the accurate prediction of precipitation, in particular of extremes, remains a challenge for numerical weather prediction models. A large source of error are subgrid-scale parameterizations of processes that play a crucial role in the complex, multi-scale dynamics of precipitation, but are not explicitly resolved in the model formulation. Here we follow a hybrid, data-driven approach, in which atmospheric variables such as wind fields are forecast in time by a general circulation model (GCM) ensemble and then mapped to precipitation using a deep convolutional autoencoder. A frequency-based weighting of the loss function is introduced to improve the learning with regard to extreme values. Our results show an improved representation of extreme precipitation frequencies, as well as better error and correlation statistics compared to a state-of-the-art GCM ensemble.

Authors: Philipp Hess (Free University Berlin); Niklas Boers (Free University Berlin)
NeurIPS 2020 Context-Aware Urban Energy Efficiency Optimization Using Hybrid Physical Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: Buildings produce more U.S. greenhouse gas emissions through electricity generation than any other economic sector. To improve the energy efficiency of buildings, engineers often rely on physics-based building simulations to predict the impacts of retrofits in individual buildings. In dense urban areas, these models suffer from inaccuracy due to imprecise parameterization or external, unmodeled urban context factors such as inter-building effects and urban microclimates. In a case study of approximately 30 buildings in Sacramento, California, we demonstrate how our hybrid physics-driven deep learning framework can use these external factors advantageously to identify a more optimal energy efficiency retrofit installation strategy and achieve significant savings in both energy and cost.

Authors: Benjamin Choi (Stanford University); Alex Nutkiewicz (Stanford University); Rishee Jain (Stanford University)
NeurIPS 2020 Physics-constrained Deep Recurrent Neural Models of Building Thermal Dynamics (Papers Track)
Abstract and authors: (click to expand)

Abstract: We develop physics-constrained and control-oriented predictive deep learning models for the thermal dynamics of a real-world commercial office building. The proposed method is based on the systematic encoding of physics-based prior knowledge into a structured recurrent neural architecture. Specifically, our model mimics the structure of the building thermal dynamics model and leverages penalty methods to model inequality constraints. Additionally, we use constrained matrix parameterization based on the Perron-Frobenius theorem to bound the eigenvalues of the learned network weights. We interpret the stable eigenvalues as dissipativeness of the learned building thermal model. We demonstrate the effectiveness of the proposed approach on a dataset obtained from an office building with 20 thermal zones.

Authors: Jan Drgona (Pacific Northwest National Laboratory); Aaron R Tuor (Pacific Northwest National Laboratory); Vikas Chandan (PNNL); Draguna Vrabie (PNNL)
NeurIPS 2020 EarthNet2021: A novel large-scale dataset and challenge for forecasting localized climate impacts (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change is global, yet its concrete impacts can strongly vary between different locations in the same region. Seasonal weather forecasts currently operate at the mesoscale (> 1 km). For more targeted mitigation and adaptation, modelling impacts to < 100 m is needed. Yet, the relationship between driving variables and Earth’s surface at such local scales remains unresolved by current physical models. Large Earth observation datasets now enable us to create machine learning models capable of translating coarse weather information into high-resolution Earth surface forecasts encompassing localized climate impacts. Here, we define high-resolution Earth surface forecasting as video prediction of satellite imagery conditional on mesoscale weather forecasts. Video prediction has been tackled with deep learning models. Developing such models requires analysis-ready datasets. We introduce EarthNet2021, a new, curated dataset containing target spatio-temporal Sentinel 2 satellite imagery at 20 m resolution, matched with high-resolution topography and mesoscale (1.28 km) weather variables. With over 32000 samples it is suitable for training deep neural networks. Comparing multiple Earth surface forecasts is not trivial. Hence, we define the EarthNetScore, a novel ranking criterion for models forecasting Earth surface reflectance. For model intercomparison we frame EarthNet2021 as a challenge with four tracks based on different test sets. These allow evaluation of model validity and robustness as well as model applicability to extreme events and the complete annual vegetation cycle. In addition to forecasting directly observable weather impacts through satellite-derived vegetation indices, capable Earth surface models will enable downstream applications such as crop yield prediction, forest health assessments, coastline management, or biodiversity monitoring. Find data, code, and how to participate at www.earthnet.tech .

Authors: Christian Requena-Mesa (Computer Vision Group, Friedrich Schiller University Jena; DLR Institute of Data Science, Jena; Max Planck Institute for Biogeochemistry, Jena); Vitus Benson (Max-Planck-Institute for Biogeochemistry); Jakob Runge (Institute of Data Science, German Aerospace Center (DLR)); Joachim Denzler (Computer Vision Group, Friedrich Schiller University Jena, Germany); Markus Reichstein (Max Planck Institute for Biogeochemistry, Jena; Michael Stifel Center Jena for Data-Driven and Simulation Science, Jena)
NeurIPS 2020 Machine Learning Climate Model Dynamics: Offline versus Online Performance (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate models are complicated software systems that approximate atmospheric and oceanic fluid mechanics at a coarse spatial resolution. Typical climate forecasts only explicitly resolve processes larger than 100 km and approximate any process occurring below this scale (e.g. thunderstorms) using so-called parametrizations. Machine learning could improve upon the accuracy of some traditional physical parametrizations by learning from so-called global cloud-resolving models. We compare the performance of two machine learning models, random forests (RF) and neural networks (NNs), at parametrizing the aggregate effect of moist physics in a 3 km resolution global simulation with an atmospheric model. The NN outperforms the RF when evaluated offline on a testing dataset. However, when the ML models are coupled to an atmospheric model run at 200 km resolution, the NN-assisted simulation crashes with 7 days, while the RF-assisted simulations remain stable. Both runs produce more accurate weather forecasts than a baseline configuration, but globally averaged climate variables drift over longer timescales.

Authors: Noah D Brenowitz (Vulcan Inc.); Brian Henn (Vulcan, Inc.); Spencer Clark (Vulcan, Inc.); Anna Kwa (Vulcan, Inc.); Jeremy McGibbon (Vulcan, Inc.); W. Andre Perkins (Vulcan, Inc.); Oliver Watt-Meyer (Vulcan, Inc.); Christopher S. Bretherton (Vulcan, Inc.)
NeurIPS 2020 Data-driven modeling of cooling demand in a commercial building (Papers Track)
Abstract and authors: (click to expand)

Abstract: Heating, ventilation, and air conditioning (HVAC) systems account for 30% of the total energy consumption in buildings. Design and implementation of energy-efficient schemes can play a pivotal role in minimizing energy usage. As an important first step towards improved HVAC system controls, this study proposes a new framework for modeling the thermal response of buildings by leveraging data measurements and formulating a data-driven system identification model. The proposed method combines principal component analysis (PCA) to identify the most significant predictors that influence the cooling demand of a building with an auto-regressive integrated moving average with exogenous variables (ARIMAX) model. The performance of the developed model was evaluated both analytically and visually. It was found that our PCA-based ARIMAX (2-0-5) model was able to accurately forecast the cooling demand for the prediction horizon of 7 days. In this work, the actual measurements from a university campus building are used for model development and validation.

Authors: Aqsa Naeem (Stanford University); Sally Benson (Stanford University); Jacques de Chalendar (Stanford University)
NeurIPS 2020 ACED: Accelerated Computational Electrochemical systems Discovery (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Large-scale electrification is vital to addressing the climate crisis, but many engineering challenges remain to fully electrifying both the chemical industry and transportation. In both of these areas, new electrochemical materials and systems will be critical, but developing these systems currently relies heavily on computationally expensive first-principles simulations as well as human-time-intensive experimental trial and error. We propose to develop an automated workflow that accelerates these computational steps by introducing both automated error handling in generating the first-principles training data as well as physics-informed machine learning surrogates to further reduce computational cost. It will also have the capacity to include automated experiments ``in the loop'' in order to dramatically accelerate the overall materials discovery pipeline.

Authors: Rachel C Kurchin (CMU); Eric Muckley (Citrine Informatics); Lance Kavalsky (CMU); Vinay Hegde (Citrine Informatics); Dhairya Gandhi (Julia Computing); Xiaoyu Sun (CMU); Matthew Johnson (MIT); Alan Edelman (MIT); James Saal (Citrine Informatics); Christopher V Rackauckas (Massachusetts Institute of Technology); Bryce Meredig (Citrine Informatics); Viral Shah (Julia Computing); Venkat Viswanathan (Carnegie Mellon University)
NeurIPS 2020 Forecasting Marginal Emissions Factors in PJM (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Many climate change applications rely on accurate forecasts of power grid emissions, but many forecasting methods can be expensive, sensitive to input errors, or lacking in domain knowledge. Motivated by initial experiments using deep learning and power system modeling techniques, we propose a method that combines the strengths of both of these approaches to forecast hourly day-ahead MEFs for the PJM region of the United States.

Authors: Amy H Wang (Western University); Priya L Donti (Carnegie Mellon University)
NeurIPS 2020 Artificial Intelligence, Machine Learning and Modeling for Understanding the Oceans and Climate Change (Proposals Track)
Abstract and authors: (click to expand)

Abstract: These changes will have a drastic impact on almost all forms of life in the ocean with further consequences on food security, ecosystem services in coastal and inland communities. Despite these impacts, scientific data and infrastructures are still lacking to understand and quantify the consequences of these perturbations on the marine ecosystem. Understanding this phenomenon is not only an urgent but also a scientifically demanding task. Consequently, it is a problem that must be addressed with a scientific cohort approach, where multi-disciplinary teams collaborate to bring the best of different scientific areas. In this proposal paper, we describe our newly launched four-years project focused on developing new artificial intelligence, machine learning, and mathematical modeling tools to contribute to the understanding of the structure, functioning, and underlying mechanisms and dynamics of the global ocean symbiome and its relation with climate change. These actions should enable the understanding of our oceans and predict and mitigate the consequences of climate change.

Authors: Nayat Sánchez Pi (Inria); Luis Martí (Inria); André Abreu (Fountation Tara Océans); Olivier Bernard (Inria); Colomban de Vargas (CNRS); Damien Eveillard (Univ. Nantes); Alejandro Maass (CMM, U. Chile); Pablo Marquet (PUC); Jacques Sainte-Marie (Inria); Julien Salomin (Inria); Marc Schoenauer (INRIA); Michele Sebag (LRI, CNRS, France)