Uncertainty Quantification & Robustness

Discussion Seminars and Webinars

Talks

Workshop Papers

Venue Title
NeurIPS 2024 Using Convolutional Neural Processes to Produce High-Resolution Weather Datasets Over New Zealand (Papers Track)
Abstract and authors: (click to expand)

Abstract: In recent years, there has been a surge in the development and success of artificial intelligence methods for global weather forecasting and climate modelling. For regional models, however, there is a lack of high-resolution datasets that can be used to train data-driven forecasting models. In this work, we showcase the use of convolutional neural processes (ConvNP) to generate hourly high-resolution (1km) weather datasets over New Zealand for temperature and precipitation. ConvNP models allow us to produce datasets that are enhanced by station observations and provide uncertainty estimates in their predictions. The generated datasets have applications in AI weather and climate models, model verification, and broader environmental research.

Authors: Emily O'Riordan (Bodeker Scientific)
NeurIPS 2024 AI-Driven Predictive Modeling of PFAS Contamination in Aquatic Ecosystems: Exploring A Geospatial Approach (Papers Track)
Abstract and authors: (click to expand)

Abstract: Per- and polyfluoroalkyl substances (PFAS), a class of synthetic fluorinated compounds termed “forever chemicals”, have garnered significant attention due to their persistence, widespread environmental presence, bioaccumulative properties, and associated risks for human health. Their presence in aquatic ecosystems highlights the link between human activity and the hydrological cycle. They also disrupt aquatic life, interfere with gas exchange, and disturb the carbon cycle, contributing to greenhouse gas emissions and exacerbating climate change. Federal agencies, state governments and non-government research and public interest organizations have emphasized the need for documenting the sites and the extent of PFAS contamination. However, the time-consuming and expensive nature of data collection and analysis poses challenges. It hinders the rapid identification of locations at high risk of PFAS contamination, which may then require further sampling or remediation. To address this data limitation, our study leverages a novel geospatial dataset, machine learning models including frameworks such as Random Forest, IBM-NASA's Prithvi and UNet, and geospatial analysis to predict regions with high PFAS concentrations in surface water. Using fish data from the National Rivers and Streams Assessment (NRSA) dataset by the Environmental Protection Agency (EPA), our analysis suggests the potential value of machine learning based models for targeted deployment of sampling investigations and remediation efforts.

Authors: Jowaria Khan (University of Michigan); David Andrews (Environmental Working Group); Kaley Beins (Environmental Working Group); Sydney Evans (Environmental Working Group); Alexa Friedman (Environmental Working Group); Elizabeth Bondi-Kelly (MIT)
NeurIPS 2024 End-to-End Conformal Calibration for Robust Grid-Scale Battery Storage Optimization (Papers Track)
Abstract and authors: (click to expand)

Abstract: The rapid proliferation of intermittent renewable electricity generation demands a corresponding growth in grid-scale energy storage systems to enable grid decarbonization. To encourage investment in energy storage infrastructure, storage operators rely on forecasts of electricity prices along with uncertainty estimates to maximize profit while managing risk. However, well-calibrated uncertainty estimates can be difficult to obtain in high-capacity prediction models such as deep neural networks. Moreover, in high-dimensional settings, there may be many valid uncertainty estimates with varied performance profiles—i.e., not all uncertainty is equally valuable for downstream decision-making. To address this challenge, this paper develops an end-to-end framework for conditional robust optimization, with robustness and calibration guarantees provided by conformal prediction. We represent arbitrary convex uncertainty sets with sublevel sets of partially input-convex neural networks, which are learned as part of our framework. We demonstrate the value of our approach for robust decision-making on a battery storage arbitrage application.

Authors: Christopher Yeh (California Institute of Technology); Nicolas Christianson (California Institute of Technology); Adam Wierman (California Institute of Technology); Yisong Yue (Caltech)
NeurIPS 2024 Climate Impact Assessment Requires Weighting: Introducing the Weighted Climate Dataset (Papers Track)
Abstract and authors: (click to expand)

Abstract: High-resolution gridded climate data are readily available from multiple sources, yet climate research and decision-making increasingly require country and region-specific climate information weighted by socio-economic factors. Moreover, the current landscape of disparate data sources and inconsistent weighting methodologies exacerbates the reproducibility crisis and undermines scientific integrity. To address these issues, we have developed a globally comprehensive dataset at both country (GADM0) and region (GADM1) levels, encompassing various climate indicators (precipitation, temperature, SPEI, wind gust). Our methodology involves weighting gridded climate data by population density, night-time light intensity, cropland area, and concurrent population count – all proxies for socio-economic activity – before aggregation. We process data from multiple sources, offering daily, monthly, and annual climate variables spanning from 1900 to 2023. A unified framework streamlines our preprocessing steps, and rigorous validation against leading climate impact studies ensures data reliability. The resulting Weighted Climate Dataset is publicly accessible through an online dashboard at https://weightedclimatedata.streamlit.app/.

Authors: Marco Gortan (University of Basel); Lorenzo Testa (Carnegie Mellon University); Giorgio Fagiolo (Sant'Anna School of Advanced Studies); Francesco Lamperti (Sant'Anna School of Advanced Studies)
ICLR 2024 Calibrating Bayesian UNet++ for Sub-seasonal Forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: Seasonal forecasting is a crucial task when it comes to detecting the extreme heat and colds that occur due to climate change. Confidence in the predictions should be reliable since a small increase in the temperatures in a year has a big impact on the world. Calibration of the neural networks provides a way to ensure our confidence in the predictions. However, calibrating regression models is an under-researched topic, especially in forecasters. We calibrate a UNet++ based architecture, which was shown to outperform physics-based models in temperature anomalies. We show that with a slight trade-off between prediction error and calibration error, it is possible to get more reliable and sharper forecasts. We believe that calibration should be an important part of safety-critical machine learning applications such as weather forecasters.

Authors: Büşra Asan (Istanbul Technical University); Abdullah Akgül (University of Southern Denmark); Alper Unal (Istanbul Technical University); Melih Kandemir (University of Southern Denmark); Gozde Unal (Istanbul Technical University)
ICLR 2024 Using expired weather forecasts to supply 10 000y of data for accurate planning of a renewable European energy system (Papers Track)
Abstract and authors: (click to expand)

Abstract: Expanding renewable energy generation and electrifying heating to address climate change will heighten the exposure of our power systems to the variability of weather. Planning and assessing these future systems typically lean on past weather data. We spotlight the pitfalls of this approach---chiefly its reliance on what we claim is a limited weather record---and propose a novel approach: to evaluate these systems on two orders of magnitude more weather scenarios. By repurposing past ensemble weather predictions, we not only drastically expand the known weather distribution---notably its extreme tails---for traditional power system modeling but also unveil its potential to enable data-intensive self-supervised, diffusion-based and optimization ML techniques. Building on our methodology, we introduce a **dataset** collected from ECMWF ENS forecasts, encompassing power-system relevant variables over Europe, and detail the intricate process behind its assembly.

Authors: Petr Dolezal (AI4ER CDT, University of Cambridge); Emily Shuckburgh (University of Cambridge)
ICLR 2024 Advancing Earth System Model Calibration: A Diffusion-Based Method (Papers Track) Honorable Mention
Abstract and authors: (click to expand)

Abstract: Understanding of climate impact on ecosystems globally requires site-specific model calibration. Here we introduce a novel diffusion-based uncertainty quantification (DBUQ) method for efficient model calibration. DBUQ is a score-based diffusion model that leverages Monte Carlo simulation to estimate the score function and evaluates a simple neural network to quickly generate samples for approximating parameter posterior distributions. DBUQ is stable, efficient, and can effectively calibrate the model given diverse observations, thereby enabling rapid and site-specific model calibration on a global scale. This capability significantly advances Earth system modeling and our understanding of climate impacts on Earth systems. We demonstrate DBUQ's capability in E3SM land model calibration at the Missouri Ozark AmeriFlux forest site. Both synthetic and real-data applications indicate that DBUQ produces accurate parameter posterior distributions similar to those generated by Markov Chain Monte Carlo sampling but with 30X less computing time. This efficiency marks a significant stride in model calibration, paving the way for more effective and timely climate impact analyses.

Authors: Yanfang Liu (Oak Ridge National Laboratory); Dan Lu (Oak Ridge National Laboratory); Zezhong Zhang (Oak Ridge National Laboratory); Feng Bao (Florida State University); Guannan Zhang (Oak Ridge National Laboratory)
ICLR 2024 Deep Gaussian Processes and inversion for decision support in model-based climate change mitigation and adaptation problems (Papers Track)
Abstract and authors: (click to expand)

Abstract: To inform their decisions, policy makers often rely on models developed by researchers that are computationally intensive and complex and that frequently run on High Performance Computers (HPC). These decision-support models are not used directly by deciders and the results of these models tend to be presented by experts as a limited number of potential scenarios that would result from a limited number of potential policy choices. Machine learning models such as Deep Gaussian Processes (DGPs) can be used to radically re-define how decision makers can use models by creating a ‘surrogate model’ or ‘emulator’ of the original model. Surrogate models can then be embedded into apps that decisions makers can use to directly explore a vast array of policy options corresponding to potential target outcomes (model inversion). To illustrate the mechanism, we give an example of application that is envisaged as part of the UK government’s Net Zero strategy. To achieve Net Zero CO2 emissions by 2050, the UK government is considering multiple options that include planting trees to capture carbon. However, the amount of CO2 captured by the trees depend on a large number of factors that include climate conditions, soil type, soil carbon, tree type, ... Depending on these factors the net balance of carbon removal after planting trees may not necessarily be positive. Hence, choosing the right place to plant the right tree is very important. A decision-helping model has been developed to tackle this problem. For a given policy input, the model outputs its impact in terms of CO2 sequestration, biodiversity and other ecosystem services. We show how DGPs can be used to create a surrogate model of this original afforestation model and how these can be embedded into an R shiny app that can then be directly used by decision makers.

Authors: bertrand nortier (University of Exeter); daniel williamson (University of Exeter); mattia mancini (University of Exeter); amy binner (University of Exeter); brett day (University of Exeter); ian bateman (University of Exeter)
NeurIPS 2023 Uncertainty Quantification of the Madden–Julian Oscillation with Gaussian Processes (Papers Track)
Abstract and authors: (click to expand)

Abstract: The Madden–Julian Oscillation (MJO) is an influential climate phenomenon that plays a vital role in modulating global weather patterns. In spite of the improvement in MJO predictions made by machine learning algorithms, such as neural networks, most of them cannot provide the uncertainty levels in the MJO forecasts directly. To address this problem, we develop a nonparametric strategy based on Gaussian process (GP) models. We calibrate GPs using empirical correlations. Furthermore, we propose a posteriori covariance correction that extends the probabilistic coverage by more than three weeks.

Authors: Haoyuan Chen (Texas A&M University); Emil Constantinescu (Argonne National Laboratory); Vishwas Rao (Argonne National Laboratory); Cristiana Stan (George Mason University)
NeurIPS 2023 Uncertainty Quantified Machine Learning for Street Level Flooding Predictions in Norfolk, Virginia (Papers Track)
Abstract and authors: (click to expand)

Abstract: Everyday citizens, emergency responders, and critical infrastructure can be dramatically affected by the flooding of streets and roads. Climate change exacerbates these floods through sea level rise and more frequent major storm events. Low-level flooding, such as nuisance flooding, continues to increase in frequency, especially in cities like Norfolk, Virginia, which can expect nearly 200 flooding events by 2050 [1]. Recently, machine learning (ML) models have been leveraged to produce real-time predictions based on local weather and geographic conditions. However, ML models are known to produce unusual results when presented with data that varies from their training set. For decision-makers to determine the trustworthiness of the model's predictions, ML models need to quantify their prediction uncertainty. This study applies Deep Quantile Regression to a previously published, Long Short-Term Memory-based model for hourly water depth predictions [2], and analyzes its out-of-distribution performance.

Authors: Steven Goldenberg (Thomas Jefferson National Accelerator Facility); Diana McSpadden (Thomas Jefferson National Accelerator Facility); Binata Roy (University of Virginia); Malachi Schram (Thomas Jefferson National Accelerator Facility); Jonathan Goodall (University of Virginia); Heather Richter (Old Dominion University)
NeurIPS 2023 Unleashing the Autoconversion Rates Forecasting: Evidential Regression from Satellite Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: High-resolution simulations such as the ICOsahedral Non-hydrostatic Large-Eddy Model (ICON-LEM) can be used to understand the interactions between aerosols, clouds, and precipitation processes that currently represent the largest source of uncertainty involved in determining the radiative forcing of climate change. Nevertheless, due to the exceptionally high computing cost required, this simulation-based approach can only be employed for a short period of time within a limited area. Despite the fact that machine learning can mitigate this problem, the related model uncertainties may make it less reliable. To address this, we developed a neural network (NN) model powered with evidential learning to assess the data and model uncertainties applied to satellite observation data. Our study focuses on estimating the rate at which small droplets (cloud droplets) collide and coalesce to become larger droplets (raindrops) – autoconversion rates -- since this is one of the key processes in the precipitation formation of liquid clouds, hence crucial to better understanding cloud responses to anthropogenic aerosols. The results of estimating the autoconversion rates demonstrate that the model performs reasonably well, with the inclusion of both aleatoric and epistemic uncertainty estimation, which improves the credibility of the model and provides useful insights for future improvement.

Authors: Maria C Novitasari (University College London); Johannes Quaas (Universität Leipzig); Miguel Rodrigues (University College London)
NeurIPS 2023 Data Assimilation using ERA5, ASOS, and the U-STN model for Weather Forecasting over the UK (Papers Track)
Abstract and authors: (click to expand)

Abstract: In recent years, the convergence of data-driven machine learning models with Data Assimilation (DA) offers a promising avenue for enhancing weather forecasting. This study delves into this emerging trend, presenting our methodologies and outcomes. We harnessed the UK's local ERA5 850 hPa temperature data and refined the U-STN12 global weather forecasting model, tailoring its predictions to the UK's climate nuances. From the ASOS network, we sourced t2m data, representing ground observations across the UK. We employed the advanced kriging method with a polynomial drift term for consistent spatial resolution. Furthermore, Gaussian noise was superimposed on the ERA5 T850 data, setting the stage for ensuing multi-time step virtual observations. Probing into the assimilation impacts, the ASOS t2m data was integrated with the ERA5 T850 dataset. Our insights reveal that while global forecast models can adapt to specific regions, incorporating atmospheric data in DA significantly bolsters model accuracy. Conversely, the direct assimilation of surface temperature data tends to mitigate this enhancement, tempering the model's predictive prowess.

Authors: WENQI WANG (Imperial College London); Jacob Bieker (Open Climate Fix); Rossella Arcucci (Imperial College London); Cesar Quilodran-Casas (Imperial College London)
NeurIPS 2023 Probabilistic land cover modeling via deep autoregressive models (Papers Track)
Abstract and authors: (click to expand)

Abstract: Land use and land cover (LULC) modeling is a challenging task due to long-range dependencies between geographic features and distinct spatial patterns related in topography, ecology, and human development. We explore the usage of a modified Pixel Constrained CNN as applied to inpainting for categorical image data from the National Land Cover Database for producing a diverse set of land use counterfactual scenarios. We find that this approach is effective for producing a distribution of realistic image completions in certain masking configurations. However, the resulting distribution is not well-calibrated in terms of spatial summary statistics commonly used with LULC data and exhibits substantial underdispersion.

Authors: Christopher Krapu (Duke University); Ryan Calder (Virginia Tech); Mark Borsuk (Duke University)
NeurIPS 2023 Hyperspectral shadow removal with iterative logistic regression and latent Parametric Linear Combination of Gaussians (Papers Track)
Abstract and authors: (click to expand)

Abstract: Shadow detection and removal is a challenging problem in the analysis of hyperspectral images. Yet, this step is crucial for analyzing data for remote sensing applications like methane detection. In this work, we develop a shadow detection and removal method only based on the spectrum of each pixel and the overall distribution of spectral values. We first introduce Iterative Logistic Regression(ILR) to learn a spectral basis in which shadows can be linearly classified. We then model the joint distribution of the mean radiance and the projection coefficients of the spectra onto the above basis as a parametric linear combination of Gaussians. We can then extract the maximum likelihood mixing parameter of the Gaussians to estimate the shadow coverage and to correct the shadowed spectra. Our correction scheme reduces correction artefacts at shadow borders. The shadow detection and removal method is applied to hyperspectral images from MethaneAIR, a precursor to the satellite MethaneSAT.

Authors: Core Francisco Park (Harvard University); Maya Nasr (Harvard University); Manuel Pérez-Carrasco (University of Concepcion); Eleanor Walker (Harvard University); Douglas Finkbeiner (Harvard University); Cecilia Garraffo (AstroAI at the Center for Astrophysics, Harvard & Smitnsonian)
NeurIPS 2023 Data-Driven Traffic Reconstruction and Kernel Methods for Identifying Stop-and-Go Congestion (Papers Track)
Abstract and authors: (click to expand)

Abstract: Identifying stop-and-go events (SAGs) in traffic flow presents an important avenue for advancing data-driven research for climate change mitigation and sustainability, owing to their substantial impact on carbon emissions, travel time, fuel consumption, and roadway safety. In fact, SAGs are estimated to account for 33-50\% of highway driving externalities. However, insufficient attention has been paid to precisely quantifying where, when, and how much these SAGs take place– necessary for downstream decision making, such as intervention design and policy analysis. A key challenge is that the data available to researchers and governments are typically sparse and aggregated to a granularity that obscures SAGs. To overcome such data limitations, this study thus explores the use of traffic reconstruction techniques for SAG identification. In particular, we introduce a kernel-based method for identifying spatio-temporal features in traffic and leverage bootstrapping to quantify the uncertainty of the reconstruction process. Experimental results on California highway data demonstrate the promise of the method for capturing SAGs. This work contributes to a foundation for data-driven decision making to advance sustainability of traffic systems.

Authors: Edgar Ramirez Sanchez (MIT); Shreyaa Raghavan (MIT); Cathy Wu ()
ICLR 2023 Deep ensembles to improve uncertainty quantification of statistical downscaling models under climate change conditions (Papers Track)
Abstract and authors: (click to expand)

Abstract: Recently, deep learning has emerged as a promising tool for statistical downscaling, the set of methods for generating high-resolution climate fields from coarse low-resolution variables. Nevertheless, their ability to generalize to climate change conditions remains questionable, mainly due to the stationarity assumption. We propose deep ensembles as a simple method to improve the uncertainty quantification of statistical downscaling models. By better capturing uncertainty, statistical downscaling models allow for superior planning against extreme weather events, a source of various negative social and economic impacts. Since no observational future data exists, we rely on a pseudo reality experiment to assess the suitability of deep ensembles for quantifying the uncertainty of climate change projections. Deep ensembles allow for a better risk assessment, highly demanded by sectoral applications to tackle climate change.

Authors: Jose González-Abad (Instituto de Fı́sica de Cantabria (IFCA), CSIC-Universidad de Cantabria); Jorge Baño-Medina (Institute of Physics of Cantabria)
ICLR 2023 Bayesian Inference of Severe Hail in Australia (Papers Track)
Abstract and authors: (click to expand)

Abstract: Severe hailstorms are responsible for some of the most costly insured weather events in Australia and can cause significant damage to homes, businesses, and agriculture. However their response to climate change remains uncertain, in large part due to the challenges of observing severe hailstorms. We propose a novel Bayesian approach which explicitly models known biases and uncertainties of current hail observations to produce more realistic estimates of severe hail risk from existing observations. Training this model on data from south-east Queensland, Australia, suggests that previous analyses of severe hail that did not account for this uncertainty may produce poorly calibrated risk estimates. Preliminary evaluation on withheld data confirms that our model produces well-calibrated probabilities and is applicable out of sample. Whilst developed for hail, we highlight also the generality of our model and its potential applications to other severe weather phenomena and areas of climate change adaptation and mitigation.

Authors: Isabelle C Greco (University of New South Wales); Steven Sherwood (University of New South Wales); Timothy Raupach (University of New South Wales); Gab Abramowitz (University of New South Wales)
ICLR 2023 On the impact of small-data diversity on forecasts: evidence from meteorologically-driven electricity demand in Mediterranean zones. (Papers Track)
Abstract and authors: (click to expand)

Abstract: In this paper, we compare the improvement of probabilistic electricity demand forecasts for three specific coastal and island regions using raw and pre-computed meteorological features based on empirically-tested formulations drawn from climate science literature. Typically for the general task of time-series forecasting with strong weather/climate drivers, go-to models like the Autoregressive Integrated Moving Average (ARIMA) model are built with assumptions of how independent variables will affect a dependent one and are at best encoded with a handful of exogenous features with known impact. Depending on the geographical region and/or cultural practices of a population, such a selection process may yield a non-optimal feature set which would ultimately drive a weak impact on underline demand forecasts. The aim of this work is to assess the impact of a documented set of meteorological features on electricity demand using deep learning models in comparative studies. Leveraging the defining computational architecture of the Temporal Fusion Transformer (TFT), we discover the unimportance of weather features for improving probabilistic forecasts for the targeted regions. However, through experimentation, we discover that the more stable electricity demand of the coastal Mediterranean regions, the Ceuta and Melilla autonomous cities in Morocco, improved the forecast accuracy of the strongly tourist-driven electricity demand for the Balearic islands located in Spain during the time of travel restrictions (i.e., during COVID19 (2020))--a root mean squared error (RMSE) from ~0.090 to ~0.012 with a substantially improved 10th/90th quantile bounding.

Authors: Reginald Bryant (IBM Research - Africa); Julian Kuehnert (IBM Research)
NeurIPS 2022 Improving the predictions of ML-corrected climate models with novelty detection (Papers Track)
Abstract and authors: (click to expand)

Abstract: While previous works have shown that machine learning (ML) can improve the prediction accuracy of coarse-grid climate models, these ML-augmented methods are more vulnerable to irregular inputs than the traditional physics-based models they rely on. Because ML-predicted corrections feed back into the climate model’s base physics, the ML-corrected model regularly produces out of sample data, which can cause model instability and frequent crashes. This work shows that adding semi-supervised novelty detection to identify out-of-sample data and disable the ML-correction accordingly stabilizes simulations and sharply improves the quality of predictions. We design an augmented climate model with a one-class support vector machine (OCSVM) novelty detector that provides better temperature and precipitation forecasts in a year-long simulation than either a baseline (no-ML) or a standard ML-corrected run. By improving the accuracy of coarse-grid climate models, this work helps make accurate climate models accessible to researchers without massive computational resources.

Authors: Clayton H Sanford (Columbia); Anna Kwa (Allen Institute for Artificial Intelligence); Oliver Watt-Meyer (Allen Institute for AI); Spencer Clark (Allen Institute for AI); Noah Brenowitz (Allen Institute for AI); Jeremy McGibbon (Allen Institute for AI); Christopher Bretherton (Allen Institute for AI)
NeurIPS 2022 Short-term Prediction and Filtering of Solar Power Using State-Space Gaussian Processes (Papers Track)
Abstract and authors: (click to expand)

Abstract: Short-term forecasting of solar photovoltaic energy (PV) production is important for powerplant management. Ideally these forecasts are equipped with error bars, so that downstream decisions can account for uncertainty. To produce predictions with error bars in this setting, we consider Gaussian processes (GPs) for modelling and predicting solar photovoltaic energy production in the UK. A standard application of GP regression on the PV timeseries data is infeasible due to the large data size and non-Gaussianity of PV readings. However, this is made possible by leveraging recent advances in scalable GP inference, in particular, by using the state-space form of GPs, combined with modern variational inference techniques. The resulting model is not only scalable to large datasets but can also handle continuous data streams via Kalman filtering.

Authors: So Takao (UCL); Sean Nassimiha (UCL); Peter Dudfield (Open Climate Fix); Jack Kelly (Open Climate Fix); Marc Deisenroth (University College London)
NeurIPS 2022 Identifying latent climate signals using sparse hierarchical Gaussian processes (Papers Track)
Abstract and authors: (click to expand)

Abstract: Extracting latent climate signals from multiple climate model simulations is important to estimate future climate change. To tackle this we develop a sparse hierarchical Gaussian process (SHGP), which probabilistically learns a latent distribution from a set of vectors. We use this to predict the latent surface temperature change globally and for central England from an ensemble of climate models, in a scalable manner and with robust uncertainty propagation.

Authors: Matt Amos (Lancaster University); Thomas Pinder (Lancaster University); Paul Young (Lancaster University)
NeurIPS 2022 Probabilistic forecasting of regional photovoltaic power production based on satellite-derived cloud motion (Papers Track)
Abstract and authors: (click to expand)

Abstract: Solar energy generation drastically increased in the last years, and it is expected to grow even more in the next decades. So, accurate intra-day forecasts are needed to improve the predictability of the photovoltaic power production and associated balancing measures to increase the shares of renewable energy in the power grid. Most forecasting methods require numerical weather predictions, which are slow to compute, or long-term datasets to run the forecast. These issues make the models difficult to implement in an operational setting. To overcome these problems, we propose a novel regional intraday probabilistic PV power forecasting model able to exploit only 2 hours of satellite-derived cloudiness maps to produce the ensemble forecast. The model is easy to implement in an operational setting as it is based on Pysteps, an already-operational Python library for precipitation nowcasting. With few adaptations of the Steps algorithm, we reached state-of-the-art performance, reaching a 71% lower RMSE than the Persistence model and a 50% lower CRPS than the Persistence Ensemble model for forecast lead times up to 4 hours.

Authors: Alberto Carpentieri (Bern University of Applied Science); Doris Folini (Institute for Atmospheric and Climate Science, ETH Zurich); Martin Wild (Institute for Atmospheric and Climate Science, ETH Zurich); Angela Meyer (Bern University of Applied Science)
NeurIPS 2022 Robustifying machine-learned algorithms for efficient grid operation (Papers Track)
Abstract and authors: (click to expand)

Abstract: We propose a learning-augmented algorithm, RobustML, for operation of dispatchable generation that exploits the good performance of a machine-learned algorithm while providing worst-case guarantees on cost. We evaluate the algorithm on a realistic two-generator system, where it exhibits robustness to distribution shift while enabling improved efficiency as renewable penetration increases.

Authors: Nicolas Christianson (California Institute of Technology); Christopher Yeh (California Institute of Technology); Tongxin Li (The Chinese University of Hong Kong (Shenzhen)); Mahdi Torabi Rad (Beyond Limits); Azarang Golmohammadi (Beyond Limits, Inc.); Adam Wierman (California Institute of Technology)
NeurIPS 2022 Bridging the Microwave Data Gap; Using Bayesian Deep Learning to “See” the Unseen (Papers Track)
Abstract and authors: (click to expand)

Abstract: Having microwave data with the spatial and temporal resolution of infrared data would provide a large positive impact on many climate and weather applications. We demonstrate that Bayesian deep learning is a promising technique for both creating and improving synthetic microwave data from infrared data. We report 0.7% mean absolute percentage error for 183+/-3 GHz microwave brightness temperature and uncertainty metrics and find that more training data is needed to achieve improved performance at 166 GHz, 37 GHz, and 23 GHz. Analysis of the spatial distribution of uncertainty reveals that additional cloud data will provide the greatest increase in skill, which will potentially allow for generation of many secondary products derived from microwave data in the future.

Authors: Pedro Ortiz (Naval Postgraduate School); Eleanor Casas (Naval Postgraduate School); Marko Orescanin (Naval Postgraduate School); Scott Powell (Naval Postgraduate School)
NeurIPS 2022 Using uncertainty-aware machine learning models to study aerosol-cloud interactions (Papers Track)
Abstract and authors: (click to expand)

Abstract: Aerosol-cloud interactions (ACI) include various effects that result from aerosols entering a cloud, and affecting cloud properties. In general, an increase in aerosol concentration results in smaller droplet sizes which leads to larger, brighter, longer-lasting clouds that reflect more sunlight and cool the Earth. The strength of the effect is however heterogeneous, meaning it depends on the surrounding environment, making ACI one of the most uncertain effects in our current climate models. In our work, we use causal machine learning to estimate ACI from satellite observations by reframing the problem as a treatment (aerosol) and outcome (change in droplet radius). We predict the causal effect of aerosol on clouds with uncertainty bounds depending on the unknown factors that may be influencing the impact of aerosol. Of the three climate models evaluated, we find that only one plausibly recreates the trend, lending more credence to its estimate cooling due to ACI.

Authors: Maëlys Solal (University of Oxford); Andrew Jesson (University of Oxford); Yarin Gal (University of Oxford); Alyson Douglas (University of Oxford)
NeurIPS 2022 Calibration of Large Neural Weather Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: Uncertainty quantification of weather forecasts is a necessity for reliably planning for and responding to extreme weather events in a warming world. This motivates the need for well-calibrated ensembles in probabilistic weather forecasting. We present initial results for the calibration of large-scale deep neural weather models for data-driven probabilistic weather forecasting. By explicitly accounting for uncertainties about the forecast's initial condition and model parameters, we generate ensemble forecasts that show promising results on standard diagnostics for probabilistic forecasts. Specifically, we are approaching the Integrated Forecasting System (IFS), the gold standard on probabilistic weather forecasting, on: (i) the spread-error agreement; and (ii) the Continuous Ranked Probability Score (CRPS). Our approach scales to state-of-the-art data-driven weather models, enabling cheap post-hoc calibration of pretrained models with tens of millions of parameters and paving the way towards the next generation of well-calibrated data-driven weather models.

Authors: Andre Graubner (Nvidia); Kamyar Kamyar Azizzadenesheli (Nvidia); Jaideep Pathak (NVIDIA Corporation); Morteza Mardani (Nvidia); Mike Pritchard (Nvidia); Karthik Kashinath (Nvidia); Anima Anandkumar (NVIDIA/Caltech)
NeurIPS 2022 Interpretable Spatiotemporal Forecasting of Arctic Sea Ice Concentration at Seasonal Lead Times (Proposals Track)
Abstract and authors: (click to expand)

Abstract: There are many benefits from the accurate forecasting of Arctic sea ice, however existing models struggle to reliably predict sea ice concentration at long lead times. Many numerical models exist but can be sensitive to initial conditions, and while recent deep learning-based methods improve overall robustness, they either do not utilize temporal trends or rely on architectures that are not performant at learning long-term sequential dependencies. We propose a method of forecasting sea ice concentration using neural circuit policies, a form of continuous time recurrent neural architecture, which improve the learning of long-term sequential dependencies compared to existing techniques and offer the added benefits of adaptability to irregular sequence intervals and high interpretability.

Authors: Matthew Beveridge (Independent Researcher); Lucas Pereira (ITI, LARSyS, Técnico Lisboa)
NeurIPS 2022 FourCastNet: A practical introduction to a state-of-the-art deep learning global weather emulator (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: Accurate, reliable, and efficient means of forecasting global weather patterns are of paramount importance to our ability to mitigate and adapt to climate change. Currently, real-time weather forecasting requires repeated numerical simulation and data assimilation cycles on dedicated supercomputers, which restricts the ability to make reliable, high-resolution forecasts to a handful of organizations. However, recent advances in deep learning, specifically the FourCastNet model, have shown that data-driven approaches can forecast important atmospheric variables with excellent skill and comparable accuracy to standard numerical methods, but at orders-of-magnitude lower computational and energy cost during inference, enabling larger ensembles for better probabilistic forecasts. In this tutorial, we demonstrate various applications of FourCastNet for high-resolution global weather forecasting, with examples including real-time forecasts, uncertainty quantification for extreme events, and adaptation to specific variables or localized regions of interest. The tutorial will provide examples that will demonstrate the general workflow for formatting and working with global atmospheric data, running autoregressive inference to obtain daily global forecasts, saving/visualizing model predictions of atmospheric events such as hurricanes and atmospheric rivers, and computing quantitative evaluation metrics for weather models. The exercises will primarily use PyTorch and do not require detailed understanding of the climate and weather system. With this tutorial, we hope to equip attendees with basic knowledge about building deep learning-based weather model surrogates and obtaining forecasts of crucial atmospheric variables using these models.

Authors: Jaideep Pathak (NVIDIA Corporation); Shashank Subramanian (Lawrence Berkeley National Laboratory); Peter Harrington (Lawrence Berkeley National Laboratory (Berkeley Lab)); Thorsten Kurth (Nvidia); Andre Graubner (Nvidia); Morteza Mardani (NVIDIA Corporation); David M. Hall (NVIDIA); Karthik Kashinath (Lawrence Berkeley National Laboratory); Anima Anandkumar (NVIDIA/Caltech)
AAAI FSS 2022 Predicting Wildfire Risk Under Novel 21st-Century Climate Conditions
Abstract and authors: (click to expand)

Abstract: Wildfires are one of the most impactful hazards associated with climate change, and in a hotter, drier world, wildfires will be much more common than they have historically been. However, the exact severity and frequency of future wildfires are difficult to estimate, because climate change will create novel combinations of vegetation and fire weather outside what has been historically observed. This provides a challenge for AI-based approaches to long-term fire risk modeling, as much future fire risk is outside of the available feature space provided by the historical record. Here, we give an overview of this problem that is inherent to many climate change impacts and propose a restricted model form that makes monotonic and interpretable predictions in novel fire weather environments. We then show how our model outperforms other neural networks and logistic regression models when making predictions on unseen data from a decade into the future.

Authors: Matthew Cooper (Sust Global).
AAAI FSS 2022 Probabilistic Machine Learning in Polar Earth and Climate Science: A Review of Applications and Opportunities
Abstract and authors: (click to expand)

Abstract: Our world’s climate future is on thin ice. The study of longterm weather patterns in the polar regions is an important building block in tackling Climate Change. Our understanding of the past, the present and the future of the earth system, and the inherent uncertainty, informs planning, mitigation, and adaptation strategies. In this work we review previous applications of machine learning and statistical computing to polar climate research, and we highlight promising probabilistic machine learning methods that address the modelling needs of climate-related research in the Arctic and the Antarctic. We discuss common challenges in this interdisciplinary field and provide an overview of opportunities for future work in this novel area of research.

Authors: Kim Bente (The University of Sydney), Judy Kay (The University of Sydney) and Roman Marchant (Commonwealth Scientific and Industrial Research Organisation (CSIRO))
NeurIPS 2021 Addressing Deep Learning Model Uncertainty in Long-Range Climate Forecasting with Late Fusion (Papers Track)
Abstract and authors: (click to expand)

Abstract: Global warming leads to the increase in frequency and intensity of climate extremes that cause tremendous loss of lives and property. Accurate long-range climate prediction allows more time for preparation and disaster risk management for such extreme events. Although machine learning approaches have shown promising results in long-range climate forecasting, the associated model uncertainties may reduce their reliability. To address this issue, we propose a late fusion approach that systematically combines the predictions from multiple models to reduce the expected errors of the fused results. We also propose a network architecture with the novel denormalization layer to gain the benefits of data normalization without actually normalizing the data. The experimental results on long-range 2m temperature forecasting show that the framework outperforms the 30-year climate normals, and the accuracy can be improved by increasing the number of models.

Authors: Ken C. L. Wong (IBM Research – Almaden Research Center); Hongzhi Wang (IBM Almaden Research Center); Etienne E Vos (IBM); Bianca Zadrozny (IBM Research); Campbell D Watson (IBM Reserch); Tanveer Syeda-Mahmood (IBM Research)
NeurIPS 2021 Accurate and Timely Forecasts of Geologic Carbon Storage using Machine Learning Methods (Papers Track)
Abstract and authors: (click to expand)

Abstract: Carbon capture and storage is one strategy to reduce greenhouse gas emissions. One approach to storing the captured CO2 is to inject it into deep saline aquifers. However, dynamics of the injected CO2 plume is uncertain and the potential for leakage back to the atmosphere must be assessed. Thus, accurate and timely forecasts of CO2 storage via real-time measurements integration becomes very crucial. This study proposes a learning-based, inverse-free prediction method that can accurately and rapidly forecast CO2 movement and distribution with uncertainty quantification based on limited simulation and observation data. The machine learning techniques include dimension reduction, multivariate data analysis, and Bayesian learning. The outcome is expected to provide CO2 storage site operators with an effective tool for real-time decision making.

Authors: Dan Lu (Oak Ridge National Laboratory); Scott Painter (Oak Ridge National Laboratory); Nicholas Azzolina (University of North Dakota); Matthew Burton-Kelly (University of North Dakota)
NeurIPS 2021 Hybrid physics-based and data-driven modeling with calibrated uncertainty for lithium-ion battery degradation diagnosis and prognosis (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Advancing lithium-ion batteries (LIBs) in both design and usage is key to promoting electrification in the coming decades to mitigate human-caused climate change. Inadequate understanding of LIB degradation is an important bottleneck that limits battery durability and safety. Here, we propose hybrid physics-based and data-driven modeling for online diagnosis and prognosis of battery degradation. Compared to existing battery modeling efforts, we aim to build a model with physics as its backbone and statistical learning techniques as enhancements. Such a hybrid model has better generalizability and interpretability together with a well-calibrated uncertainty associated with its prediction, rendering it more valuable and relevant to safety-critical applications under realistic usage scenarios.

Authors: Jing Lin (Institute for Infocomm Research); Yu Zhang (I2R); Edwin Khoo (Institute for Infocomm Research)
ICML 2021 Estimation of Corporate Greenhouse Gas Emissions via Machine Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: As an important step to fulfill the Paris Agreement and achieve net-zero emissions by 2050, the European Commission adopted the most ambitious package of climate impact measures in April 2021 to improve the flow of capital towards sustainable activities. For these and other international measures to be successful, reliable data is key. The ability to see the carbon footprint of companies around the world will be critical for investors to comply with the measures and hit climate neutrality. However, with only a small portion of companies volunteering to disclose their greenhouse gas (GHG) emissions, it is nearly impossible for investors to align their investment strategies with the measures. By training a machine learning model on disclosed GHG emissions, we are able to estimate the emissions of other companies globally who do not disclose their emissions. In this paper, we show that our model provides accurate estimates of corporate GHG emissions to investors such that they are able to align their investments with the regulatory measures and achieve net-zero goals.

Authors: You Han (Bloomberg L.P.); Achintya Gopal (Bloomberg LP); Liwen Ouyang (Bloomberg L.P.); Aaron Key (Bloomberg LP)
ICML 2021 Physics-Informed Graph Neural Networks for Robust Fault Location in Power Grids (Papers Track) Best Paper: ML Innovation
Abstract and authors: (click to expand)

Abstract: The reducing cost of renewable energy resources, such as solar photovoltaics (PV) and wind farms, is accelerating global energy transformation to mitigate climate change. However, a high level of intermittent renewable energy causes power grids to have more stability issues. This accentuates the need for quick location of system failures and follow-up control actions. In recent events such as in California, line failures have resulted in large-scale wildfires leading to loss of life and property. In this article, we propose a two-stage graph learning framework to locate power grid faults in the challenging but practical regime characterized by (a) sparse observations, (b) low label rates, and (c) system variability. Our approach embeds the geometrical structure of power grids into the graph neural networks (GNN) in stage I for fast fault location, and then stage II further enhances the location accuracy by employing the physical similarity of the labeled and unlabeled data samples. We compare our approach with three baselines in the IEEE 123-node benchmark system and show that it outperforms the others by significant margins in various scenarios.

Authors: Wenting Li (Los Alamos National Laboratory); Deepjyoti Deka (Los Alamos National Laboratory)
ICML 2021 Probabilistic Short-Term Low-Voltage Load Forecasting using Bernstein-Polynomial Normalizing Flows (Papers Track)
Abstract and authors: (click to expand)

Abstract: The transition to a fully renewable energy grid requires better forecasting of demand at the low-voltage level. However, high fluctuations and increasing electrification cause huge forecast errors with traditional point estimates. Probabilistic load forecasts take future uncertainties into account and thus enables various applications in low-carbon energy systems. We propose an approach for flexible conditional density forecasting of short-term load based on Bernstein-Polynomial Normalizing Flows where a neural network controls the parameters of the flow. In an empirical study with 363 smart meter customers, our density predictions compare favorably against Gaussian and Gaussian mixture densities and also outperform a non-parametric approach based on the pinball loss for 24h-ahead load forecasting for two different neural network architectures.

Authors: Marcel Arpogaus (Konstanz University of Applied Sciences); Marcus Voß (Technische Universität Berlin (DAI-Labor)); Beate Sick (ZHAW and University of Zurich); Mark Nigge-Uricher (Bosch.IO GmbH); Oliver Dürr (Konstanz University of Applied Sciences)
ICML 2021 Designing Bounded min-knapsack Bandits algorithm for Sustainable Demand Response (Papers Track)
Abstract and authors: (click to expand)

Abstract: Around 40% of global energy produced is consumed by buildings. By using renewable energy resources we can alleviate the dependence on electrical grids. Recent trends focus on incentivizing consumers to reduce their demand consumption during peak hours for sustainable demand response. To minimize the loss, the distributor companies should target the right set of consumers and demand the right amount of electricity reductions. This paper proposes a novel bounded integer min-knapsack algorithm and shows that the algorithm, while allowing for multiple unit reduction, also optimizes the loss to the distributor company within a factor of two (multiplicative) and a problem-dependent additive constant. Existing CMAB algorithms fail to work in this setting due to non-monotonicity of reward function and time-varying optimal sets. We propose a novel algorithm Twin-MinKPDR-CB to learn these compliance probabilities efficiently. Twin-MinKPDR-CB works for non-monotone reward functions bounded min-knapsack constraints and time-varying optimal sets. We find that Twin-MinKPDR-CB achieves sub-linear regret of O(log T) with T being the number of rounds demand response is run.

Authors: Akansha Singh (Indian Institute of Technology, Ropar); Meghana Reddy (Indian Institute of Technology, Ropar); Zoltan Nagy (University of Texas); Sujit P. Gujar (Machine Learning Laboratory, International Institute of Information Technology, Hyderabad); Shweta Jain (Indian Institute of Technology Ropar)
NeurIPS 2020 Short-Term Solar Irradiance Forecasting Using Calibrated Probabilistic Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: Advancing probabilistic solar forecasting methods is essential to supporting the integration of solar energy into the electricity grid. In this work, we develop a variety of state-of-the-art probabilistic models for forecasting solar irradiance. We investigate the use of post-hoc calibration techniques for ensuring well-calibrated probabilistic predictions. We train and evaluate the models using public data from seven stations in the SURFRAD network, and demonstrate that the best model, NGBoost, achieves higher performance at an intra-hourly resolution than the best benchmark solar irradiance forecasting model across all stations. Further, we show that NGBoost with CRUDE post-hoc calibration achieves comparable performance to a numerical weather prediction model on hourly-resolution forecasting.

Authors: Eric Zelikman (Stanford University); Sharon Zhou (Stanford University); Jeremy A Irvin (Stanford); Cooper Raterink (Stanford University); Hao Sheng (Stanford University); Avati Anand (Stanford University); Jack Kelly (Open Climate Fix); Ram Rajagopal (Stanford University); Andrew Ng (Stanford University); David J Gagne (National Center for Atmospheric Research)
NeurIPS 2020 Revealing the Oil Majors' Adaptive Capacity to the Energy Transition with Deep Multi-Agent Reinforcement Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: A low-carbon energy transition is transpiring to combat climate change, posing an existential threat to oil and gas companies, particularly the Majors. Though Majors yield the resources and expertise to adapt to low-carbon business models, meaningful climate-aligned strategies have yet to be enacted. A 2-degrees pathways (2DP) wargame was developed to assess climate-compatible pathways for the oil Majors. Recent advances in deep multi-agent reinforcement learning (MARL) have achieved superhuman-level performance in solving high-dimensional continuous control problems. Modeling within a Markovian framework, we present the novel 2DP-MARL model which applies deep MARL methods to solve the 2DP wargame across a multitude of transition scenarios. Designed to best mimic Majors in real- life competition, the model reveals all Majors quickly adapt to low-carbon business models to remain robust amidst energy transition uncertainty. The purpose of this work is provide tangible metrics to support the call for oil Majors to diversify into low-carbon business models and, thus, accelerate the energy transition.

Authors: Dylan Radovic (Imperial College London); Lucas Kruitwagen (University of Oxford); Christian Schroeder de Witt (University of Oxford)
NeurIPS 2020 HECT: High-Dimensional Ensemble Consistency Testing for Climate Models (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Climate models play a crucial role in understanding the effect of environmental and man-made changes on climate to help mitigate climate risks and inform governmental decisions. Large global climate models such as the Community Earth System Model (CESM), developed by the National Center for Atmospheric Research, are very complex with millions of lines of code describing interactions of the atmosphere, land, oceans, and ice, among other components. As development of the CESM is constantly ongoing, simulation outputs need to be continuously controlled for quality. To be able to distinguish a ``climate-changing'' modification of the code base from a true climate-changing physical process or intervention, there needs to be a principled way of assessing statistical reproducibility that can handle both spatial and temporal high-dimensional simulation outputs. Our proposed work uses probabilistic classifiers like tree-based algorithms and deep neural networks to perform a statistically rigorous goodness-of-fit test of high-dimensional spatio-temporal data.

Authors: Niccolo Dalmasso (Carnegie Mellon University); Galen Vincent (Carnegie Mellon University); Dorit Hammerling (Colorado School of Mines); Ann Lee (Carnegie Mellon University)
NeurIPS 2020 Expert-in-the-loop Systems Towards Safety-critical Machine Learning Technology in Wildfire Intelligence (Proposals Track)
Abstract and authors: (click to expand)

Abstract: With the advent of climate change, wildfires are becoming more frequent and severe across several regions worldwide. To prevent and mitigate its effects, wildfire intelligence plays a pivotal role, e.g. to monitor the evolution of wildfires and for early detection in high-risk areas such as wildland-urban-interface regions. Recent works have proposed deep learning solutions for fire detection tasks, however the current limited databases prevent reliable real-world deployments. We propose the development of expert-in-the-loop systems that combine the benefits of semi-automated data annotation with relevant domain knowledge expertise. Through this approach we aim to improve the data curation process and contribute to the generation of large-scale image databases for relevant wildfire tasks and empower the application of machine learning techniques in wildfire intelligence in real scenarios.

Authors: Maria João Sousa (IDMEC, Instituto Superior Técnico, Universidade de Lisboa); Alexandra Moutinho (IDMEC, Instituto Superior Técnico, Universidade de Lisboa); Miguel Almeida (ADAI, University of Coimbra)
NeurIPS 2020 ACED: Accelerated Computational Electrochemical systems Discovery (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Large-scale electrification is vital to addressing the climate crisis, but many engineering challenges remain to fully electrifying both the chemical industry and transportation. In both of these areas, new electrochemical materials and systems will be critical, but developing these systems currently relies heavily on computationally expensive first-principles simulations as well as human-time-intensive experimental trial and error. We propose to develop an automated workflow that accelerates these computational steps by introducing both automated error handling in generating the first-principles training data as well as physics-informed machine learning surrogates to further reduce computational cost. It will also have the capacity to include automated experiments ``in the loop'' in order to dramatically accelerate the overall materials discovery pipeline.

Authors: Rachel C Kurchin (CMU); Eric Muckley (Citrine Informatics); Lance Kavalsky (CMU); Vinay Hegde (Citrine Informatics); Dhairya Gandhi (Julia Computing); Xiaoyu Sun (CMU); Matthew Johnson (MIT); Alan Edelman (MIT); James Saal (Citrine Informatics); Christopher V Rackauckas (Massachusetts Institute of Technology); Bryce Meredig (Citrine Informatics); Viral Shah (Julia Computing); Venkat Viswanathan (Carnegie Mellon University)