Climate Science & Modeling

Tutorials

Blog Posts

Discussion Seminars and Webinars

Innovation Grants

Talks

Workshop Papers

Venue Title
NeurIPS 2024 Temporal sequence transformer to advance long-term streamflow prediction (Papers Track) Best ML Innovation
Abstract and authors: (click to expand)

Abstract: Accurate streamflow prediction is crucial for understanding climate change impacts on water resources and for effective management of extreme hydrological events. While Long Short-Term Memory (LSTM) networks have been the dominant data-driven approach for streamflow forecasting, recent advancements in transformer architectures for time series tasks have shown promise in outperforming traditional LSTM models. This study introduces a transformer-based model that integrates historical streamflow data with climatic variables to enhance streamflow prediction accuracy. We evaluated our transformer model against a benchmark LSTM across five diverse basins in the United States. Results demonstrate that the transformer architecture consistently outperforms the LSTM model across all evaluation metrics, highlighting its potential as a more effective tool for hydrological forecasting. This research contributes to the ongoing development of advanced AI techniques for improved water resource management and climate change adaptation strategies.

Authors: Ruhaan Singh (Farragut High School); Dan Lu (Oak Ridge National Laboratory); Kshitij Tayal (Oak Ridge National Laboratory)
NeurIPS 2024 Exploring Vision Transformers for Early Detection of Climate Change Signals (Papers Track)
Abstract and authors: (click to expand)

Abstract: This study evaluates Vision Transformers (ViTs) for detecting anthropogenic climate change signals, crucial for effective policy planning and risk assessment. Compared to previously suggested models like CNN, MLP, and ridge regression, ViTs consistently detect forced climate signals earlier across three reanalysis datasets (ERA5, JRA-3Q, and MERRA-2). Interpretation with Integrated Gradients reveals consistent spatial patterns, suggesting ViTs utilize physically-grounded signals. This work highlights ViTs' potential to advance climate change detection and attribution tasks.

Authors: Sungduk Yu (Intel Labs); Brian White (UNC Chapel Hill); Anahita Bhiwandiwalla (Intel Labs); Yaniv Gurwicz (Intel Labs); Musashi Hinck (Intel Corporation); Matthew Olson (Intel Labs); Raanan Rohekar (Intel Labs); Vasudev Lal (Intel Corp)
NeurIPS 2024 No Location Left Behind: Introducing the Fairness Assessment for Implicit Representations of Earth Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: Encoding and predicting physical measurements such as temperature or carbon dioxide is instrumental to many high-stakes challenges – including climate change. Yet, all recent advances solely assess models’ performances at a global scale. But while models’ predictions are improving on average over the entire globe, performances on sub-groups such as islands or coastal areas are left uncharted. To ensure safe deployment of those models, we thus introduce FAIR-Earth, a fine-grained evaluation suite made of diverse and high-resolution dataset. Our findings are striking–current methods produce highly biased predictions towards specific geospatial locations. The specifics of the biases vary based on the data modality and hyper-parameters of the models. Hence, we hope that FAIR-Earth will enable future research to design solutions aware of those per-group biases.

Authors: Daniel Cai (Brown University); Randall Balestriero (Brown University)
NeurIPS 2024 Using Convolutional Neural Processes to Produce High-Resolution Weather Datasets Over New Zealand (Papers Track)
Abstract and authors: (click to expand)

Abstract: In recent years, there has been a surge in the development and success of artificial intelligence methods for global weather forecasting and climate modelling. For regional models, however, there is a lack of high-resolution datasets that can be used to train data-driven forecasting models. In this work, we showcase the use of convolutional neural processes (ConvNP) to generate hourly high-resolution (1km) weather datasets over New Zealand for temperature and precipitation. ConvNP models allow us to produce datasets that are enhanced by station observations and provide uncertainty estimates in their predictions. The generated datasets have applications in AI weather and climate models, model verification, and broader environmental research.

Authors: Emily O'Riordan (Bodeker Scientific)
NeurIPS 2024 AI-Driven Predictive Modeling of PFAS Contamination in Aquatic Ecosystems: Exploring A Geospatial Approach (Papers Track)
Abstract and authors: (click to expand)

Abstract: Per- and polyfluoroalkyl substances (PFAS), a class of synthetic fluorinated compounds termed “forever chemicals”, have garnered significant attention due to their persistence, widespread environmental presence, bioaccumulative properties, and associated risks for human health. Their presence in aquatic ecosystems highlights the link between human activity and the hydrological cycle. They also disrupt aquatic life, interfere with gas exchange, and disturb the carbon cycle, contributing to greenhouse gas emissions and exacerbating climate change. Federal agencies, state governments and non-government research and public interest organizations have emphasized the need for documenting the sites and the extent of PFAS contamination. However, the time-consuming and expensive nature of data collection and analysis poses challenges. It hinders the rapid identification of locations at high risk of PFAS contamination, which may then require further sampling or remediation. To address this data limitation, our study leverages a novel geospatial dataset, machine learning models including frameworks such as Random Forest, IBM-NASA's Prithvi and UNet, and geospatial analysis to predict regions with high PFAS concentrations in surface water. Using fish data from the National Rivers and Streams Assessment (NRSA) dataset by the Environmental Protection Agency (EPA), our analysis suggests the potential value of machine learning based models for targeted deployment of sampling investigations and remediation efforts.

Authors: Jowaria Khan (University of Michigan); David Andrews (Environmental Working Group); Kaley Beins (Environmental Working Group); Sydney Evans (Environmental Working Group); Alexa Friedman (Environmental Working Group); Elizabeth Bondi-Kelly (MIT)
NeurIPS 2024 AtmosArena: Benchmarking Foundation Models for Atmospheric Sciences (Papers Track)
Abstract and authors: (click to expand)

Abstract: Deep learning has emerged as a powerful tool for atmospheric sciences, showing significant utility across various tasks in weather and climate modeling. In line with recent progress in language and vision foundation models, there are growing efforts to scale and finetune such models for multi-task spatiotemporal reasoning. Despite promising results, existing works often evaluate their model on a small set of non-uniform tasks, which makes it hard to quantify broad generalization across diverse tasks and domains. To address this challenge, we introduce AtmosArena, the first multi-task benchmark dedicated to foundation models in atmospheric sciences. AtmosArena comprises a suite of tasks that cover a broad spectrum of applications in atmospheric physics and atmospheric chemistry. To showcase the capabilities and key features of our benchmark, we conducted extensive experiments to evaluate two state-of-the-art deep learning models, ClimaX and Stormer on AtmosArena, and compare their performance with other deep learning and traditional baselines. By providing a standardized, open-source benchmark, we aim to facilitate further advancements in the field, much like open-source benchmarks have driven the development of foundation models for language and vision.

Authors: Tung Nguyen (University of California, Los Angeles); Prateik Sinha (UCLA); Advit Deepak (University of California, Los Angeles); Karen A. McKinnon (University of California, Los Angeles); Aditya Grover (UCLA)
NeurIPS 2024 Harnessing AI for Wildfire Defense: An approach to Predict and Mitigate Global Fire Risk (Papers Track)
Abstract and authors: (click to expand)

Abstract: Wildfires pose a critical threat to wildlife, economies, properties, and human lives globally, making accurate risk assessment essential for effective management and mitigation. This study introduces a novel machine learning-based approach utilizing a Convolutional Neural Network (CNN) to evaluate wildfire risks across diverse ecosystems. Leveraging a comprehensive dataset of remote-sensed variables—including topography, vegetation health indicators, and climatic conditions—our model operates at a spatial resolution of 1000 meters per pixel, providing enhanced precision in predicting wildfire occurrences. The CNN outperforms state-of-the-art models, achieving a fire detection ratio of 0.82 and a no-fire detection ratio of 0.87. The results demonstrate that most dataset variables are crucial for accurate risk assessment, although some are non-essential. By integrating data from regions around the globe, this study underscores the feasibility and effectiveness of implementing globally scalable wildfire prediction tools.

Authors: Hassan Ashfaq (Ghulam Ishaq Khan Institute of Engineering Sciences and Technology)
NeurIPS 2024 Tree Species Classification using Machine Learning and 3D Tomographic SAR - a case study in Northern Europe (Papers Track)
Abstract and authors: (click to expand)

Abstract: Tree species classification plays an important role in nature conservation, forest inventories, forest management, and the protection of endangered species. Over the past four decades, remote sensing technologies have been extensively utilized for tree species classification, with Synthetic Aperture Radar (SAR) emerging as a key technique. In this study, we employed TomoSense, a 3D tomographic dataset, which utilizes a stack of single-look complex (SLC) images, a byproduct of SAR, captured at different incidence angles to generate a three-dimensional representation of the terrain. Our research focuses on evaluating multiple tabular machine-learning models using the height information derived from the tomographic image intensities to classify eight distinct tree species. The SLC data and tomographic imagery were analyzed across different polarimetric configurations and geosplit configurations. We investigated the impact of these variations on classification accuracy, comparing the performance of various tabular machine-learning models and optimizing them using Bayesian optimization. Additionally, we incorporated a proxy for actual tree height using point cloud data from Light Detection and Ranging (LiDAR) to provide height statistics associated with the model’s predictions. This comparison offers insights into the reliability of tomographic data in predicting tree species classification based on height.

Authors: Jumpei Takami (United Nations Office for Outer Space Affairs); Grace Colverd (University of Cambridge); Laura Schade (Department of Energy Security and Net Zero - UKGOV); Karol Bot (INESCTEC); Joseph Gallego (Drexel University)
NeurIPS 2024 AQ-PINNs: Attention-Enhanced Quantum Physics-Informed Neural Networks for Carbon-Efficient Climate Modeling (Papers Track)
Abstract and authors: (click to expand)

Abstract: The growing computational demands of artificial intelligence (AI) in addressing climate change raise significant concerns about inefficiencies and environmental impact, as highlighted by Jevons' paradox. We propose an attention-enhanced quantum physics-informed neural network (AQ-PINN) to tackle these challenges. This approach integrates quantum computing techniques into physics-informed neural networks (PINNs) for climate modeling, aiming to enhance predictive accuracy in fluid dynamics governed by the Navier-Stokes equations while reducing the computational burden and carbon footprint. By harnessing variational quantum multi-head self-attention mechanisms, our AQ-PINN achieves a 51.51% reduction in model parameters compared to classical multi-head self-attention methods while maintaining comparable convergence and loss. It also employs quantum tensor networks to enhance representational capacity, which can lead to more efficient gradient computations and reduced susceptibility to barren plateaus. Our AQ-PINN represents a crucial step towards more sustainable and effective climate modeling solutions.

Authors: Siddhant Dutta (SVKM's Dwarkadas J. Sanghvi College of Engineering); Nouhaila Innan (New York University Abu Dhabi); Sadok Ben Yahia (Southern Denmark University); Muhammad Shafique (New York University Abu Dhabi)
NeurIPS 2024 Classification of Snow Depth Measurements for tracking plant phenological shifts in Alpine regions (Papers Track)
Abstract and authors: (click to expand)

Abstract: Ground-based snow depth measurements are often realized using ultrasonic or laser technologies, which by their nature measure the height of any underlying object, whether it is snow or vegetation in snow-free periods. We propose a machine learning approach to the automated classification of snow depth measurements into a snow cover class and a class corresponding to everything else, which takes into account both the temporal context and the dependencies between snow depth and other sensor measurements. Through a series of experiments we demonstrate that our approach simplifies the detection of seasonal snowmelt and corresponding onset of plant growth, which we used to assess climate-change related phenological shifts in otherwise rather poorly monitored high alpine regions.

Authors: Jan Svoboda (WSL Institute for Snow and Avalanche Research SLF); Michael Zehnder (WSL Institute for Snow and Avalanche Research SLF); Marc Ruesch (WSL Institute for Snow and Avalanche Research SLF); David Liechti (WSL Institute for Snow and Avalanche Research SLF); Corinne Jones (Swiss Data Science Center); Michele Volpi (Swiss Data Science Center, ETH Zurich); Christian Rixen (WSL Institute for Snow and Avalanche Research SLF); Jürg Schweizer (WSL Institute for Snow and Avalanche Research SLF)
NeurIPS 2024 Climate PAL: Climate Analysis through Conversational AI (Papers Track)
Abstract and authors: (click to expand)

Abstract: To support climate change research and its communication to the public, we propose Climate Projection and Analysis with Language models (Climate PAL). Our system allows users to retrieve and analyze climate projection data through conversational English. Using a crowdsourced evaluation dataset, we demonstrate that Climate PAL's retrieved data are more relevant to user queries, with over 20% higher accuracy than baselines on several key metrics.

Authors: Sonia Cromp (University of Wisconsin-Madison); Behrad Rabiei (University of California San Diego); Maxwell Elling (University of Colorado Boulder); Alexander Herron (National Aeronautics and Space Administration); Michael Hendrickson (National Aeronautics and Space Administration)
NeurIPS 2024 Adaptive Policy Regularization for Offline-to-Online Reinforcement Learning in HVAC Control (Papers Track)
Abstract and authors: (click to expand)

Abstract: Reinforcement learning (RL)-based control methods have been extensively studied to improve building heating, ventilation, and air conditioning (HVAC) efficiency. Data-driven approaches demonstrate better transferability and scalability, making them useful in real-world applications. Most prior works focus on online learning requiring simulators or models of environment dynamics. However, transferring thermal simulators between environments is inefficient. We build on recent works that employ offline training on static datasets from unknown policies. Pure offline RL is constrained by the replay buffer's distribution, we propose using offline-to-online RL to enhance pre-trained offline models through online adaptation to distribution shifts. We show that direct online fine-tuning deteriorates performance on offline policies. To address this, we propose automatically tuning the actor's regularization during training to optimize the exploration-exploitation tradeoff. Specifically, we leverage simple moving averages of mean Q-values sampled throughout training. Simulation experiments demonstrate our method outperforms state-of-the-art approaches under various conditions, improving performance by 32.9% and enhancing pre-trained models' capabilities online.

Authors: Hsin-Yu Liu (University of California San Diego); Bharathan Balaji (Amazon); Rajesh Gupta (UC San Diego); Dezhi Hong (Amazon)
NeurIPS 2024 Towards turbine-location-aware multi-decadal wind power predictions with CMIP6 (Papers Track)
Abstract and authors: (click to expand)

Abstract: With the increasing amount of renewable energy in the grid, long-term wind power forecasting for multiple decades becomes more critical. In these long-term forecasts, climate data is essential as it allows us to account for climate change. Yet the resolution of climate models is often very coarse. In this paper, we show that by including turbine locations when downscaling with Gaussian Processes, we can generate valuable aggregate wind power predictions despite the low resolution of the CMIP6 climate models. This work is a first step towards multi-decadal turbine-location-aware wind power forecasting using global climate model output.

Authors: Nina Effenberger (University of Tübingen); Nicole Ludwig (University of Tübingen)
NeurIPS 2024 Methane SatMapper: Methane Detection from Satellite Imagery Using Hyperspectral Transformer (Papers Track)
Abstract and authors: (click to expand)

Abstract: Methane (CH4) plays a critical role in accelerating global climate change, and recent advancements using Sentinel-2 satellite imagery have demonstrated potential in detecting and quantifying significant methane emissions. However, existing approaches often rely on temporal analysis of shortwave-infrared spectra, assuming consistent ground conditions and prior knowledge of methane-free periods, which can lead to errors and limit scalability. To overcome these challenges, we present Methane SatMapper, an innovative end-to-end spectral transformer model specifically designed to accurately identify and quantify methane plumes. Our model introduces two novel modules: one that identifies potential methane emission sites by analyzing solar radiation absorption in the spectral domain and another that localizes and quantifies methane plumes without the need for temporal data. By utilizing all 12 spectral channels of Sentinel-2 imagery, our architecture effectively estimates ground terrain and detects methane emissions, providing enhanced robustness to variable ground conditions and increased computational efficiency by eliminating the need for historical time-series data. Primary evaluations confirm that Methane SatMapper delivers precise and reliable methane detection, addressing key limitations in scalability and temporal dependence.

Authors: Satish Kumar (University of California, Santa Barbara); ASM Iftekhar (Microsoft); Bowen Zhang (University Of California, Santa Barbara); Richard Sserunjogi (Makerere University); Mehan Jayasuriya (Mozilla Technology Foundation)
NeurIPS 2024 Paraformer: Parameterization of Sub-grid Scale Processes Using Transformers (Papers Track)
Abstract and authors: (click to expand)

Abstract: One of the major sources of uncertainty in the current generation of Global Climate Models (GCMs) is the simulation of sub-grid scale physical processes. Over the years, with significantly improved computational performance, a series of Deep Learning (DL) parameterization schemes have been developed and incorporated into GCMs. However, these schemes use classic architectures whereas the new attention mechanism is not widely investigated. We proposed a “memory-aware” transformer-based model on ClimSim, the largest-ever dataset for climate parameterization. Our results show that the attention mechanism successfully captures the complex non-linear dependencies of sub-grid scale variables and reduces the prediction error.

Authors: Shuochen Wang (Northeastern University); Nishant Yadav (Microsoft); Auroop Ganguly (Northeastern University)
NeurIPS 2024 Efficient Localized Adaptation of Neural Weather Forecasting: A Case Study in the MENA Region (Papers Track)
Abstract and authors: (click to expand)

Abstract: Accurate weather and climate modeling are critical for both scientific advancement and safeguarding communities against environmental risks. Traditional approaches rely heavily on Numerical Weather Prediction (NWP) models, which simulate energy and matter flow across Earth's systems. However, heavy computational requirements and low efficiency restrict the suitability of NWP, leading to a pressing need for enhanced modeling techniques. Neural network-based models have emerged as promising alternatives, leveraging data-driven approaches to forecast atmospheric variables. In this work, we focus on limited-area modeling and train our model specifically for localized region-level downstream tasks. As a case study, we consider the MENA region due to its unique climatic challenges, where accurate localized weather forecasting is crucial for managing water resources, agriculture and mitigating the impacts of extreme weather events. This targeted approach allows us to tailor the model's capabilities to the unique conditions of the region of interest. Our study aims to validate the effectiveness of integrating parameter-efficient fine-tuning (PEFT) methodologies, specifically Low-Rank Adaptation (LoRA) and its variants, to enhance forecast accuracy, as well as training speed, computational resource utilization, and memory efficiency in weather and climate modeling for specific regions. Our codebase and pre-trained models can be accessed at \url{https://github.com/akhtarvision/weather-regional}.

Authors: Muhammad Akhtar Munir (MBZUAI); Fahad Shahbaz (MBZUAI); Salman Khan (MBZUAI)
NeurIPS 2024 RAIN: Reinforcement Algorithms for Improving Numerical Weather and Climate Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: This study explores integrating reinforcement learning (RL) with idealised climate models to address key parameterisation challenges in climate science. Current climate models rely on complex mathematical parameterisations to represent sub-grid scale processes, which can introduce substantial uncertainties. RL offers capabilities to enhance these parameterisation schemes, including direct interaction, handling sparse or delayed feedback, continuous online learning, and long-term optimisation. We evaluate the performance of eight RL algorithms on two idealised environments: one for temperature bias correction, another for radiative-convective equilibrium (RCE) imitating real-world computational constraints. Results show different RL approaches excel in different climate scenarios with exploration algorithms performing better in bias correction, while exploitation algorithms proving more effective for RCE. These findings support the potential of RL-based parameterisation schemes to be integrated into global climate models, improving accuracy and efficiency in capturing complex climate dynamics. Overall, this work represents an important first step towards leveraging RL to enhance climate model accuracy, critical for improving climate understanding and predictions. Code accessible at https://github.com/p3jitnath/climate-rl.

Authors: Pritthijit Nath (University Of Cambridge); Henry Moss (Secondmind); Emily Shuckburgh (University Of Cambridge); Mark Webb (Met Office)
NeurIPS 2024 Modeling Pollution Spread with Obstructions using Physics-Informed Neural Networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: Pollution modeling plays a crucial role in combating climate change and protecting public health. Scientists rely on traditional numerical methods, such as finite difference and computational fluid dynamics, for accurate pollution simulations; however, they are resource-intensive and time-consuming often taking days to execute. Physics-Informed Neural Networks (PINNs) present a promising alternative, capable of significantly improving speed while maintaining high accuracy. In our research, we developed a PINN to model pollution spread under laminar flow conditions in a two-dimensional environment with obstructions and reflecting boundaries. Our model integrates the Navier-Stokes and advection-diffusion partial differential equations (PDEs) and enforces zero-flux Neumann boundaries at obstruction surfaces and simulation edges. We used a hybrid learning approach, generating a dataset of 6.1 million colocation points for supervised learning and embedding physical laws into the loss function for unsupervised learning. In our experiments, the PINN was over 2520 times faster than the traditional numerical solver, returning results in under 2190 milliseconds and achieving a mean squared error below 3E-5. Our findings demonstrate that PINNs not only offer a drastic reduction in computational time but also scale favorably with both the time domain and spatial resolution, making them a viable solution for real-time pollution monitoring and emergency response planning. Future work will focus on extending our model to dynamic obstructions and arbitrary grid geometries.

Authors: Yash Ranjith (Westmont)
NeurIPS 2024 Prediction of PM2.5 concentration based on ARIMA, LSTM, and TCN models in Kigali, Rwanda (Papers Track)
Abstract and authors: (click to expand)

Abstract: PM2.5 pollution is a major health concern, especially in areas lacking robust real-time monitoring and predictive capabilities. This study presents a comparative analysis of three forecasting models—Autoregressive Integrated Moving Average (ARIMA), Long Short-Term Memory (LSTM), and Temporal Convolutional Networks (TCN)—to predict PM2.5 concentrations in four regions of Kigali, Rwanda. Utilizing a dataset spanning from late 2020 to 2024, these models were trained on historical air quality data obtained from sensors. Our findings reveal that the LSTM model consistently outperforms both TCN and ARIMA models, delivering lower Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) in predicting PM2.5 levels. These results underscore the effectiveness of LSTM models in providing more accurate air quality forecasts in complex temporal environments. This research lays the groundwork for enhancing air quality monitoring and public health strategies in Rwanda.

Authors: Yamlak Bogale (Carnegie Mellon University Africa); Choukouriyah Arinloye (Carnegie Mellon University Africa); Joselyne Muragijemariya (Carnegie Mellon University Africa)
NeurIPS 2024 Light-weight geospatial model for global deforestation attribution (Papers Track)
Abstract and authors: (click to expand)

Abstract: Forests are in decline worldwide and it is critical to attribute forest cover loss to its causes. We gathered a curated global dataset of all forest loss drivers and developed a neural network model to recognize the main drivers of deforestation or forest degradation at 1-km scale. Using remote sensing satellite data together with ancillary biophysical and socioeconomic data the model estimates the dominant drivers of forest loss from 2001 to 2022. Using a relatively light-weight geospatial model allowed us to to train a single world-wide model. We generated a global map of drivers of forest loss that is being validated, and present the first insights such data can provide.

Authors: Anton Raichuk (Google); Michelle Sims (WRI); Radost Stanimirova (WRI); Maxim Neumann (Google)
NeurIPS 2024 Regional Ocean Forecasting with Hierarchical Graph Neural Networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: Accurate ocean forecasting systems are vital for understanding marine dynamics, which play a crucial role in environmental management and climate adaptation strategies. Traditional numerical solvers, while effective, are computationally expensive and time-consuming. Recent advancements in machine learning have revolutionized weather forecasting, offering fast and energy-efficient alternatives. Building on these advancements, we introduce SeaCast, a neural network designed for high-resolution, medium-range ocean forecasting. SeaCast employs a graph-based framework to effectively handle the complex geometry of ocean grids and integrates external forcing data tailored to the regional ocean context. Our approach is validated through experiments at a high spatial resolution using the operational numerical model of the Mediterranean Sea provided by the Copernicus Marine Service, along with both numerical and data-driven atmospheric forcings.

Authors: Daniel Holmberg (University of Helsinki); Emanuela Clementi (CMCC Foundation); Teemu Roos (University of Helsinki)
NeurIPS 2024 Emulating the Global Change Analysis Model with Deep Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: The Global Change Analysis Model (GCAM) simulates complex interactions between the coupled Earth and human systems, providing valuable insights into the co-evolution of land, water, and energy sectors under different future scenarios. Understanding the sensitivities and drivers of this multisectoral system can lead to more robust understanding of the different pathways to particular outcomes. The interactions and complexity of the coupled human-Earth systems make GCAM simulations costly to run at scale - a requirement for large ensemble experiments which explore uncertainty in model parameters and outputs. A differentiable emulator with similar predictive power, but greater efficiency, could provide novel scenario discovery and analysis of GCAM and its outputs, requiring fewer runs of GCAM. As a first use case, we train a deep learning model on an existing large ensemble that explores a range of GCAM inputs related to different relative contributions of energy production sources, with a focus on wind and solar. We complement this existing ensemble with interpolated input values and a wider selection of outputs, predicting 22,528 GCAM outputs across time, sectors, and regions. We report a median R^2 score of 0.998 for the emulator's predictions and an R^2 score of 0.812 for its input-output sensitivity.

Authors: Andrew Holmes (Western Washington University); Matt Jensen (Pacific Northwest National Laboratory); Sarah Coffland (Western Washington University); Hidemi Mitani-Shen (Western Washington University); Logan Sizemore (Western Washington University); Seth Bassetti (Utah State University); Brenna Nieva (Western Washington University); Claudia Tebaldi (Joint Global Change Research Institute); Abigail Snyder (Joint Global Change Research Institute); Brian Hutchinson (Western Washington University)
NeurIPS 2024 Operational Wind Speed Forecasts for Chile's Electric Grid Using A Hybrid Machine Learning Model (Papers Track)
Abstract and authors: (click to expand)

Abstract: As Chile's electric power sector advances toward a future powered by renewable energy, accurate forecasting of renewable generation is essential for managing grid operations. The integration of renewable energy sources is particularly challenging due to the operational difficulties of managing their power generation, which is highly variable compared to fossil fuel sources, delaying the availability of clean energy. To mitigate this, we quantify the impact of increasing intermittent generation from wind and solar on thermal power plants in Chile and introduce a hybrid wind speed forecasting methodology which combines two custom ML models for Chile. The first model is based on TiDE, an MLP-based ML model for short-term forecasts, and the second is based on a graph neural network, GraphCast, for medium-term forecasts up to 10 days. Our hybrid approach outperforms the most accurate operational deterministic systems by 4-21% for short-term forecasts and 5-23% for medium-term forecasts and can directly lower the impact of wind generation on thermal ramping, curtailment, and system-level emissions in Chile.

Authors: Dhruv Suri (Stanford University); Praneet Dutta (Google); Flora Xue (Google DeepMind); Ines Azevedo (Stanford University); Ravi Jain (X (formerly Google X))
NeurIPS 2024 Spatio-Temporal Machine Learning Models for Emulation of Global Atmospheric Composition (Papers Track)
Abstract and authors: (click to expand)

Abstract: Interactive atmospheric composition simulations are among the most computationally expensive components in Earth System Models (ESMs) due to the need to transport a large number of gaseous and aerosol tracers at every model time step. This poses a significant limitation for higher-resolution transient climate simulations with current computational resources. In ESMs such as NASA GISS-E2.1 (hereafter referred to as ModelE), pre-computed monthly-averaged atmospheric composition concentrations are often used to reduce computational expenses. This approach is referred to as Non-Interactive Tracers (NINT). In this study, we extend the NINT version of the ModelE using machine learning to emulate the effects of interactive emissions on climate forcing. We use data from a fully interactive composition climate model with surface-driven emissions to develop an ML-based NINT climate model. This version accounts for instantaneous atmospheric conditions, enabling the tracers to respond dynamically to meteorology without the need for explicit calculation of tracer transport. This approach could be applied to any aerosol species and integrated into ESMs to simulate aerosol concentrations interactively. The proposed framework emulates the advection term at the surface pressure level, with a focus on predicting surface-level concentrations of Black Carbon (BC) from biomass burning, which is a contributor to elevated levels of PM2.5 concentrations. Two consecutive years of ModelE simulated data were used as training data. To capture both temporal and spatial dependencies, a Convolutional Long Short-Term Memory (ConvLSTM) model was used. Results show the ConvLSTM achieved an average R-squared of 0.85 (STD = 0.08) on the test set. In contrast, using monthly-averaged atmospheric composition concentrations resulted in an average R-squared of 0.42 (STD = 0.73) for the same period.

Authors: Mohammad Erfani (Columbia University); Kara Lamb (Columbia University); Susanne Bauer (NASA Goddard Institute for Space Studies); Kostas Tsigaridis (Columbia University); Marcus van Lier-Walqui (Columbia University); Gavin Schmidt (NASA Goddard Institute for Space Studies)
NeurIPS 2024 Probabilistic representation learning of subseasonal to seasonal ocean dynamics (Papers Track)
Abstract and authors: (click to expand)

Abstract: Data-driven weather and climate forecasts on subseasonal-to-seasonal (S2S) scales face a two-fold challenge: the observational records are comparatively short and predictability strongly depends on the coupling of earth systems evolving on very different timescales. As a first step towards general subseasonal-to-seasonal predictions, we model the El Niño Southern Oscillation (ENSO), one of the principal sources of predictability on S2S scales. Characterised by anomalously warm sea surface temperatures in the tropical Pacific, an El Niño event arises from a combination of slow heat transfer within the Pacific and between ocean basins, as well as fast atmospheric dynamics, such as westerly wind bursts and convection. Here, we design a deep learning model that can flexibly represent information from modalities on different timescales, trained as a Masked Autoencoder and optimising the Empirical Continuous Ranked Probability Score. We find that the representation learning approach exhibits zero-shot performance that is competitive with task-specific models on S2S of ENSO in terms of correlation skill. To the best of our knowledge, we are the first to predict well-calibrated uncertainty estimates on a 24 month horizon.

Authors: Jannik Thuemmel (University of Tuebingen); Jakob Schlör (ECMWF); Florian Ebmeier (University of Tuebingen); Bedartha Goswami (University of Tübingen)
NeurIPS 2024 Multi-branch Spatio-Temporal Graph Neural Network For Efficient Ice Layer Thickness Prediction (Papers Track) Overall Best Paper
Abstract and authors: (click to expand)

Abstract: Understanding spatio-temporal patterns in polar ice layers is essential for tracking changes in ice sheet balance and assessing ice dynamics. While convolutional neural networks are widely used in learning ice layer patterns from raw echogram images captured by airborne snow radar sensors, noise in the echogram images prevents researchers from getting high-quality results. Instead, we focus on geometric deep learning using graph neural networks, aiming to build a spatio-temporal graph neural network that learns from thickness information of the top ice layers and predicts for deeper layers. In this paper, we developed a novel multi-branch spatio-temporal graph neural network that used the GraphSAGE framework for spatio features learning and a temporal convolution operation to capture temporal changes, enabling different branches of the network to be more specialized and focusing on a single learning task. We found that our proposed multi-branch network can consistently outperform the current fused spatio-temporal graph neural network in both accuracy and efficiency.

Authors: Zesheng Liu (Lehigh University); Maryam Rahnemoonfar (Lehigh University)
NeurIPS 2024 Resolution-Agnostic Transformer-based Climate Downscaling (Papers Track)
Abstract and authors: (click to expand)

Abstract: Understanding future weather changes at regional and local scales is crucial for planning and decision-making, particularly in the context of extreme weather events, as well as for broader applications in agriculture, insurance, and infrastructure development. However, the computational cost of downscaling Global Climate Models (GCMs) to the fine resolutions needed for such applications presents a significant barrier. Drawing on advancements in weather forecasting models, this study introduces a cost-efficient downscaling method using a pretrained Earth Vision Transformer (Earth ViT) model. Initially trained on ERA5 data to downscale from 50 km to 25 km resolution, the model is then tested on the higher resolution BARRA-SY dataset at 3 km resolutions. Remarkably, it performs well without additional training, demonstrating its ability to generalize across different resolutions. This approach holds promise for generating large ensembles of regional climate simulations by downscaling GCMs with varying input resolutions without incurring additional training costs. Ultimately, this method could provide more comprehensive estimates of potential future changes in key climate variables, aiding in effective planning for extreme weather events and climate change adaptation strategies.

Authors: Declan Curran (UNSW); Hira Saleem (UNSW); Sanaa Hobeichi (The University of New South Wales); Flora Salim (University of New South Wales)
NeurIPS 2024 Estimating Greenland Ice Sheet Dynamics Using Neural Operators (Papers Track)
Abstract and authors: (click to expand)

Abstract: This study investigates the application of neural operators, particularly the Fourier Neural Operator (FNO), to enhance the modeling accuracy of the Greenland ice sheet's internal layers. Traditional computational methods often struggle to capture the complex spatial patterns of ice sheets adequately. Our approach leverages a radar image dataset to evaluate the effectiveness of neural operators in comparison to traditional neural network models. The FNO outperformed other models in key performance metrics, such as pixel error and layer accuracy. The results highlight the potential of neural operators to significantly improve the computational efficiency and accuracy of environmental models, paving the way for their broader application in cryospheric science and climate change research.

Authors: Maryam Rahnemoonfar (Lehigh University); Heling Wang (Lehigh University)
NeurIPS 2024 Mamba MethaneMapper: State Space Model for Methane Detection from Hyperspectral Imagery (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Methane (CH4) is the chief contributor to global climate change. Recent advancements in AI-based image processing have paved the way for innovative approaches for the detection of methane using hyper-spectral imagery. Existing methods, while effective, often come with high computational demands and associated costs that can limit their practical applications. Addressing these limitations, we propose the Mamba MethaneMapper (MMM), a cost-effective and efficient AI-driven solution designed to enhance methane detection capabilities in hyper-spectral images. MMM will incorporate two key innovations that collectively improve performance while managing costs. First, we will utilize a gpu-aware state-space encoder, which optimizes the computational resources and efficiency of the system. Second, MMM will use an environment-sensitive module to prioritize image regions likely containing methane emissions, which are then analyzed by our efficient Mamba algorithm. This selective approach not only improves the accuracy of methane detection but also significantly reduces unnecessary computations and memory consumption.

Authors: Satish Kumar (University of California, Santa Barbara); ASM Iftekhar (Microsoft); Kaikai Liu (University of California, Santa Barbara); Bowen Zhang (University Of California, Santa Barbara); Mehan Jayasuriya (Mozilla)
NeurIPS 2024 Flood Prediction in Kenya - Leveraging Pre-Trained Models to Generate More Validation Data in Sparse Observation Settings (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Kenya has lacked a national flood risk management framework and also has sparse flood observation data, which makes developing deep learning flood prediction models on a national scale challenging. Flood prediction models are critical to operationalize Early Warning Systems (EWS). We propose two different models to feed into an EWS. The first model will leverage statistical machine learning approaches to predict flood or no flood events on a 0.25 x 0.25 degree scale (approximately 30 km x 30 km in Kenya) using ERA5 features as well as land cover and Digital Terrain Model (DTM) data. This first model will also be used to create a baseline prediction benchmark across the entire country of Kenya. The second model will leverage pre-trained remote sensing based models to generate segmented flood or no flood data on a fine spatial scale. This will increase the number of validation points by a factor of 1000, which opens the door to deep learning approaches to predict flood or no-flood events on a 30 meter x 30 meter spatial scale. We hope that this approach of leveraging pre-trained models to generate fine scale validation data can eventually be used widely in other extreme climate event forecasting scenarios given the scarcity of historical extreme climate events compared to normal weather events.

Authors: Alim Karimi (Self / Purdue University); David Quispe (University of Toronto); Hammed Akande (Concordia University); Nicole Mong'are (Athi Water Works Development Agency); Valerie Brosnan (Mitga Solutions); Asbina Baral (Ministry of Education, Science and Technology)
ICLR 2024 Scaling Transformers for Skillful and Reliable Medium-range Weather Forecasting (Papers Track) Overall Best Paper
Abstract and authors: (click to expand)

Abstract: Weather forecasting is a fundamental problem for anticipating and mitigating the impacts of climate change. Recently, data-driven approaches for weather forecasting based on deep learning have shown great promise, achieving accuracies that are competitive with operational systems. However, those methods often employ complex, customized architectures without sufficient ablation analysis, making it difficult to understand what truly contributes to their success. Here we introduce Stormer, a simple transformer model that achieves state-of-the-art performance on weather forecasting with minimal changes to the standard transformer backbone. We identify the key components of Stormer through careful empirical analyses, including weather-specific embedding, randomized dynamics forecast, and pressure-weighted loss. At the core of Stormer is a randomized forecasting objective that trains the model to forecast the weather dynamics over varying time intervals. During inference, this allows us to produce multiple forecasts for a target lead time and combine them to obtain better forecast accuracy. On WeatherBench 2, Stormer performs competitively at short to medium-range forecasts and outperforms current methods beyond 7 days, while requiring orders-of-magnitude less training data and compute. Additionally, we demonstrate Stormer’s favorable scaling properties, showing consistent improvements in forecast accuracy with increases in model size and training tokens.

Authors: Tung Nguyen (University of California, Los Angeles); Rohan Shah (Carnegie Mellon University); Hritik Bansal (UCLA); Troy Arcomano (Argonne National Laboratory); Sandeep Madireddy (Argonne National Laboratory); Romit Maulik (Argonne National Laboratory); Veerabhadra Kotamarthi (Argonne National Laboratory); Ian Foster (Computation Institute); Aditya Grover (UCLA)
ICLR 2024 Categorization of Meteorological Data by Contrastive Clustering (Papers Track)
Abstract and authors: (click to expand)

Abstract: Visualized ceilometer backscattering data, displaying meteorological phenomena like clouds, precipitation, and aerosols, is mostly analyzed manually by meteorology experts. In this work, we present an approach for the categorization of backscattering data using a contrastive clustering approach, incorporating image and spatiotemporal information into the model. We show that our approach leads to meteorologically meaningful clusters, opening the door to the automatic categorization of ceilometer data, and how our work could potentially create insights in the field of climate science.

Authors: Michael Dammann (HAW Hamburg); Ina Mattis (DWD); Michael Neitzke (HAW Hamburg); Ralf Möller (University of Lübeck)
ICLR 2024 AI-driven emulation of ocean dynamics on sub-seasonal scales (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate forecasting systems rely on coupling atmospheric models to ocean and sea ice models. However, while there have recently been significant efforts to accelerate atmospheric models using AI, there have been very scarce efforts to accelerate the latter. As a result, climate forecasting systems still rely on expensive numerical simulations, which renders large-scale ensembling and probabilistic prediction cumbersome. To address this issue, we propose a large-scale AI model of ocean dynamics. Our method relies on a spherical neural operator to accurately capture the functional nature of ocean dynamics on the sphere. We empirically demonstrate that our model can accurately predict ocean dynamics for sub-seasonal horizons and outperforms the existing method. It offers a 60x speedup over the fastest numerical solver currently used for the task.

Authors: Suyash Bire (Massachusetts Institute of Technology); Jean Kossaifi (NVIDIA); Simone Silvestri (Massachusetts Institute of Technology); Nikola Kovachki (Nvidia Corp.); Kamyar Azizzadenesheli (Nvidia Corp.); Chris N Hill (MIT); Animashree Anandkumar (Caltech)
ICLR 2024 Calibrating Bayesian UNet++ for Sub-seasonal Forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: Seasonal forecasting is a crucial task when it comes to detecting the extreme heat and colds that occur due to climate change. Confidence in the predictions should be reliable since a small increase in the temperatures in a year has a big impact on the world. Calibration of the neural networks provides a way to ensure our confidence in the predictions. However, calibrating regression models is an under-researched topic, especially in forecasters. We calibrate a UNet++ based architecture, which was shown to outperform physics-based models in temperature anomalies. We show that with a slight trade-off between prediction error and calibration error, it is possible to get more reliable and sharper forecasts. We believe that calibration should be an important part of safety-critical machine learning applications such as weather forecasters.

Authors: Büşra Asan (Istanbul Technical University); Abdullah Akgül (University of Southern Denmark); Alper Unal (Istanbul Technical University); Melih Kandemir (University of Southern Denmark); Gozde Unal (Istanbul Technical University)
ICLR 2024 Extreme Precipitation Nowcasting using Transformer-based generative models (Papers Track)
Abstract and authors: (click to expand)

Abstract: This paper presents an innovative approach to extreme precipitation nowcasting by employing Transformer-based generative models, specifically VideoGPT with Extreme Value Loss (EVL) regularization. Leveraging a comprehensive dataset from the Royal Netherlands Meteorological Institute (KNMI), our study focuses on predicting short-term precipitation with high accuracy. We introduce a novel method for computing EVL without assuming fixed extreme representations, addressing the limitations of current models in capturing extreme weather events. We present both qualitative and quantitative analyses, demonstrating the superior performance of the proposed VideoGPT-EVL in generating accurate precipitation forecasts, especially when dealing with extreme precipitation events.

Authors: Cristian Meo (TUDelft); Mircea T Lica (Delft University of Technology); Ankush Roy (TUDelft); Zeina Boucher (TUDelft); Junzhe Yin (TUDelft); Yanbo Wang (Delft University of Technology); Ruben Imhoff (Deltares); Remko Uijlenhoet (TUDelft); Justin Dauwels (TU Delft)
ICLR 2024 GeoFormer: A Vision and Sequence Transformer-based Approach for Greenhouse Gas Monitoring (Papers Track)
Abstract and authors: (click to expand)

Abstract: Air pollution represents a pivotal environmental challenge globally, playing a major role in climate change via greenhouse gas emissions and negatively affecting the health of billions. However, predicting the spatial and temporal patterns of pollutants remains challenging. The scarcity of ground-based monitoring facilities and the dependency of air pollution modeling on comprehensive datasets, often inaccessible for numerous areas, complicate this issue. In this work, we introduce GeoFormer, a compact model that combines a vision transformer module with a highly efficient time-series transformer module to predict surface-level nitrogen dioxide (NO2) concentrations from Sentinel-5P satellite imagery. We train the proposed model to predict surface-level NO2 measurements using a dataset we constructed with Sentinel-5P images of ground-level monitoring stations, and their corresponding NO2concentration readings. The proposed model attains high accuracy (MAE 5.65), demonstrating the efficacy of combining vision and time-series transformer architectures to harness satellite-derived data for enhanced GHG emission insights, proving instrumental in advancing climate change monitoring and emission regulation efforts globally.

Authors: Madhav Khirwar (Independent)
ICLR 2024 Using expired weather forecasts to supply 10 000y of data for accurate planning of a renewable European energy system (Papers Track)
Abstract and authors: (click to expand)

Abstract: Expanding renewable energy generation and electrifying heating to address climate change will heighten the exposure of our power systems to the variability of weather. Planning and assessing these future systems typically lean on past weather data. We spotlight the pitfalls of this approach---chiefly its reliance on what we claim is a limited weather record---and propose a novel approach: to evaluate these systems on two orders of magnitude more weather scenarios. By repurposing past ensemble weather predictions, we not only drastically expand the known weather distribution---notably its extreme tails---for traditional power system modeling but also unveil its potential to enable data-intensive self-supervised, diffusion-based and optimization ML techniques. Building on our methodology, we introduce a **dataset** collected from ECMWF ENS forecasts, encompassing power-system relevant variables over Europe, and detail the intricate process behind its assembly.

Authors: Petr Dolezal (AI4ER CDT, University of Cambridge); Emily Shuckburgh (University of Cambridge)
ICLR 2024 Towards a Data-Driven Understanding of Cloud Structure Formation (Papers Track)
Abstract and authors: (click to expand)

Abstract: The physics of cloud formation and evolution is still not fully understood and constitutes one of the highest uncertainties in climate modeling. We are working on an approach that aims at improving our understanding of how clouds of different structures form from a data-driven perspective: By predicting the visual appearance of cloud photographs from physical quantities obtained from reanalysis data and subsequently attributing the decisions to physical quantities using ``explainable AI'' methods, we try to identify relevant physical processes. At the current stage, this is just a proof of concept, being at least able to identify basic meteorologically plausible facts from data.

Authors: Ann-Christin Wörl (Johannes Gutenberg University); Michael Wand (University of Mainz); Peter Spichtinger (Johannes Gutenberg University)
ICLR 2024 Towards Downscaling Global AOD with Machine Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Poor air quality represents a significant threat to human health, especially in urban areas. To improve forecasts of air pollutant mass concentrations, there is a need for high-resolution Aerosol Optical Depth (AOD) forecasts as proxy. However, current General Circulation Model (GCM) forecasts of AOD suffer from limited spatial resolution, making it difficult to accurately represent the substantial variability exhibited by AOD at the local scale. To address this, a deep residual convolutional neural network (ResNet) is evaluated for the GCM to local scale downscaling of low-resolution global AOD retrievals, outperforming a non-trainable interpolation baseline. We explore the bias correction potential of our ResNet using global reanalysis data, evaluating it against in-situ AOD observations. The improved resolution from our ResNet can assist in the study of local AOD variations.

Authors: Josh Millar (Imperial College London); Paula Harder (Mila); Lilli J Freischem (University of Oxford); Philipp Weiss (University of Oxford); Philip Stier (University of Oxford)
ICLR 2024 Diffusion-Based Joint Temperature and Precipitation Emulation of Earth System Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: Earth system models (ESMs) are the principal tools used in climate science to generate future climate projections under various atmospheric emissions scenarios on a global or regional scale. Generative deep learning approaches are suitable for emulating these tools due to their computational efficiency and ability, once trained, to generate realizations in a fraction of the time required by ESMs. We extend previous work that used a generative probabilistic diffusion model to emulate ESMs by targeting the joint emulation of multiple variables, temperature and precipitation, by a single diffusion model. Joint generation of multiple variables is critical to generate realistic samples of phenomena resulting from the interplay of multiple variables. The diffusion model emulator takes in the monthly mean-map of temperature and precipitation and produces the daily values of each of these variables that exhibit statistical properties similar to those generated by ESMs. Our results show the outputs from our extended model closely resemble those from ESMs on various climate metrics including dry spells and hot streaks, and that the joint distribution of temperature and precipitation in our sample closely matches those of ESMs.

Authors: Katie Christensen (Western Washington University); Lyric Otto (Western Washington University); Seth Bassetti (Utah State University); Claudia Tebaldi (Joint Global Change Research Institute); Brian Hutchinson (Western Washington University)
ICLR 2024 Near-real-time monitoring of global ocean carbon sink (Papers Track)
Abstract and authors: (click to expand)

Abstract: The ocean, absorbing about 25% of anthropogenic CO2 emissions, plays a crucial role in mitigating climate change. However, the delayed (by one year) traditional estimates of ocean-atmosphere CO2 flux hinder timely understanding and response to the global carbon cycle’s dynamics. Addressing this challenge, we introduce Carbon Monitor Ocean (CMO-NRT), a pioneering dataset providing near-real-time, monthly gridded estimates of global surface ocean fugacity of CO2 (fCO2) and ocean-atmosphere CO2 flux from January 2022 to July 2023. This dataset marks a significant advancement by updating the global carbon budget’s estimates through a fusion of data from 10 Global Ocean Biogeochemical Models (GOBMs) and 8 data products into a near-real-time analysis framework. By harnessing the power of Convolutional Neural Networks (CNNs) and semi-supervised learning techniques, we decode the complex nonlinear relationships between model or product estimates and observed environmental predictors. The predictive models, both for GOBM and data products, exhibit exceptional accuracy, with root mean square errors (RMSEs) maintaining below the 5% threshold. This advancement supports more effective climate change mitigation efforts by providing scientists and policymakers with timely and accurate data.

Authors: Xiaofan Gui (Microsoft Research); Jiang Bian (Microsoft Research)
ICLR 2024 Fast non-stationary geospatial modelling with multiresolution (wavelet) Gaussian processes (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate modelling tasks involve assimilating large amounts of geospatial data from different sources, such as simulators and measurements from weather stations and satellites. These sources of data are weighted according to their uncertainty, so good quality uncertainty estimates are essential. Gaussian processes (GPs) offer flexible models with uncertainty estimates, and have a long track record of use in geospatial modelling. However, much of the research effort, including recent work on scalability, is focused on statistically stationary models, which are not suitable for many climatic variables, such as precipitation. Here we propose a novel, scalable, nonstationary GP model based upon discrete wavelets, and evaluate them on toy and real world data.

Authors: Talay M Cheema (University of Cambridge); Carl Edward Rasmussen (Cambridge University)
ICLR 2024 Physics-informed Machine Learning-based Cloud Microphysics parameterization for Earth System Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: In this study, we develop a physics-informed machine learning (ML)-based cloud microphysics parameterization for the ICON model. By training the ML parameterization on high-resolution simulation data, we aim to improve Earth System Models (ESMs) in comparison to traditional parameterization schemes. We investigate the usage of a multilayer perceptron (MLP) with feature engineering and physics-constraints, and use explainability techniques to understand the relationship between input features and model output. Our novel approach yields promising results, with the physics-informed ML-based cloud microphysics parameterization achieving an R$^2$ score up to 0.777 for an individual feature. Additionally, we demonstrate a notable improvement in the overall performance in comparison to a baseline MLP, increasing its average R$^2$ score from 0.290 to 0.613 across all variables. This approach to improve the representation of cloud microphysics in ESMs promises to enhance climate projections, contributing to a better understanding of climate change.

Authors: Ellen Sarauer (German Aerospace Center (DLR)); Mierk Schwabe (Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Institut für Physik der Atmosphäre, Oberpfaffenhofen, Germany); Philipp Weiss (University of Oxford); Axel Lauer (Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Institut für Physik der Atmosphäre, Oberpfaffenhofen, Germany); Philip Stier (University of Oxford); Veronika Eyring (Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Institut für Physik der Atmosphäre, Oberpfaffenhofen, Germany; University of Bremen, Institute of Environmental Physics (IUP), Bremen, Germany)
ICLR 2024 A Deep Learning Technology Suite for Cost-Effective Sequestered CO2 Monitoring (Papers Track)
Abstract and authors: (click to expand)

Abstract: Carbon capture and storage (CCS) is a way of reducing carbon emissions to help tackle global warming. Injecting CO2 into rock formations and preventing it from escaping to the surface is a main step in a CCS project. Therefore, monitoring of geologically sequestered CO2 is important for CCS security assessment. Time-lapse seismic (4D seismic) is one of the most effective tools for CO2 monitoring. Unfortunately, the main challenge of 4D seismic is the high cost due to repeated monitoring seismic data acquisition surveys and the subsequent time-consuming data processing that involves imaging and inversion. To address this, we developed a technology suite powered by deep learning engines that significantly reduces the cost by (1) acquiring very sparse monitoring data; (2) firing multiple seismic sources simultaneously; (3) converting 2D images to 3D volume; (4) enforcing repeatability between baseline data and monitoring data; and (5) nonlinearly mapping seismic data to subsurface property model to bypass complex wave-equation-based seismic data processing procedures.

Authors: Wenyi Hu (SLB); Son Phan (SLB); Cen Li (SLB); Aria Abubakar (SLB)
ICLR 2024 On the potential of Optimal Transport in Geospatial Data Science (Papers Track)
Abstract and authors: (click to expand)

Abstract: Prediction problems in geographic information science and transportation are often motivated by the possibility to enhance operational efficiency and thereby reduce emissions. Examples range from predicting car sharing demand for relocation planning to forecasting traffic congestion for navigation purposes. However, conventional accuracy metrics ignore the spatial distribution of the errors, despite its relevance for operations. Here, we put forward a spatially aware evaluation metric and loss function based on Optimal Transport (OT). Our framework leverages partial OT and can minimize relocation costs in any spatial prediction problem. We showcase the advantages of OT-based evaluation over conventional metrics and further demonstrate the application of an OT loss function for improving forecasts of bike sharing demand and charging station occupancy. Thus, our framework not only aligns with operational considerations, but also signifies a step forward in refining predictions within geospatial applications. All code is available at https://github.com/mie-lab/geospatial_optimal_transport.

Authors: Nina V Wiedemann (ETH Zurich); Martin Raubal (ETH Zürich)
ICLR 2024 DiffObs: Generative Diffusion for Global Forecasting of Satellite Observations (Papers Track)
Abstract and authors: (click to expand)

Abstract: This work presents an autoregressive generative diffusion model (DiffObs) to predict the global evolution of daily precipitation, trained on a satellite observational product, and assessed with domain-specific diagnostics. The model is trained to probabilistically forecast day-ahead precipitation. Nonetheless, it is stable for multi-month rollouts, which reveal a qualitatively realistic superposition of convectively coupled wave modes in the tropics. Cross-spectral analysis confirms successful generation of low frequency variations associated with the Madden--Julian oscillation, which regulates most subseasonal to seasonal predictability in the observed atmosphere, and convectively coupled moist Kelvin waves with approximately correct dispersion relationships. Despite secondary issues and biases, the results affirm the potential for a next generation of global diffusion models trained on increasingly sparse, and increasingly direct and differentiated observations of the world, for practical applications in subseasonal and climate prediction.

Authors: Jason Stock (Colorado State University); Jaideep Pathak (NVIDIA Corporation); Yair Cohen (NVIDIA Corporation); Mike Pritchard (NVIDIA Corporation); Piyush Garg (NVIDIA); Dale Durran (NVIDIA Corporation); Morteza Mardani (NVIDIA Corporation); Noah D Brenowitz (NVIDIA)
ICLR 2024 A cautionary tale about deep learning-based climate emulators (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate models are computationally too expensive for many tasks, such as, rapidly exploring future impacts of climate policies. Thus, since the 1980s scientists have been developing lightweight approximations or emulators of climate models. Recently, deep learning has been proposed for this task and most commonly been evaluated on the benchmark ClimateBenchv1.0. We implemented a linear regression-based model from the 1990s with 30K parameters, called linear pattern scaling, that is now the 'best' model on ClimateBenchv1.0 -- outperforming the incumbent 100M-parameter foundation model, ClimaX, on the spatial error of 3 out of the 4 variables. Nevertheless, climate emulation might benefit from innovations in machine learning and we analyse two aspects that need to be addressed in future emulators: First, the data complexity depends strongly on the climate variable of interest and the chosen spatiotemporal resolution. Second, current benchmarks do not sufficiently address the large impact of interannual variability in the climate system. We have published our analysis as an interactive tutorial at github.com/ygaxolotl/tags-linear-pattern-scaling.

Authors: Björn Lütjens (Massachussets Institute of Technology); Raffaele Ferrari (Massachusetts Institute of Technology); Paolo Giani (Massachusetts Institute of Technology); Dava Newman (MIT); Andre Souza (Massachusetts Institute of Technology); Duncan Watson-Parris (University of California San Diego); Noelle Selin (Massachusetts Institute of Technology)
ICLR 2024 A Deep Learning Framework to Efficiently Estimate Precipitation at the Convection Permitting Scale (Papers Track)
Abstract and authors: (click to expand)

Abstract: Precipitation-related extreme events are rapidly growing due to climate change, emphasizing the need for accurate hazard projections. To effectively model the convective phenomena driving severe precipitation, high-resolution estimates are crucial. Existing methods struggle with either insufficient expressiveness in capturing complex convective dynamics, due to the low resolution, or excessive computational demands. In response, we propose an innovative deep learning framework that efficiently harnesses available data to yield precise results. This model, based on graph neural networks, utilises two grids with different resolution and two sets of edges to represent spatial relationships. Employing as input ERA5 reanalysis atmospheric variables on an approximately 25 km grid, the framework produces hourly precipitation estimates on a finer 3 km grid. Findings are promising in accurately capturing yearly precipitation distribution and estimating cumulative precipitation during extreme events. Notably, the model demonstrates effectiveness in spatial regions not included in the training, motivating further exploration of its transferability potential.

Authors: Valentina Blasone (University of Trieste); Erika Coppola (Earth System Physics Section, ICTP, Trieste); Guido Sanguinetti (SISSA); Viplove Arora (Theoretical and Scientific Data Science, SISSA, Trieste); Serafina Di Gioia (Earth System Physics Section, ICTP, Trieste); Luca Bortolussi (University of Trieste)
ICLR 2024 Global Vegetation Modeling With Pre-Trained Weather Transformers (Papers Track)
Abstract and authors: (click to expand)

Abstract: Accurate vegetation models can produce further insights into the complex inter-action between vegetation activity and ecosystem processes. Previous research has established that long-term trends and short-term variability of temperature and precipitation affect vegetation activity. Motivated by the recent success of Transformer-based Deep Learning models for medium-range weather forecasting, we adapt the publicly available pre-trained FourCastNet to model vegetation activity while accounting for the short-term dynamics of climate variability. We investigate how the learned global representation of the atmosphere’s state can be transferred to model the normalized difference vegetation index (NDVI). Our model globally estimates vegetation activity at a resolution of 0.25◦ while relying only on meteorological data. We demonstrate that leveraging pre-trained weather models improves the NDVI estimates compared to learning an NDVI model from scratch. Additionally, we compare our results to other recent data-driven NDVI modeling approaches from machine learning and ecology literature. We further provide experimental evidence on how much data and training time is necessary to turn FourCastNet into an effective vegetation model. Code and models are available at https://github.com/LSX-UniWue/Global-Ecosystem-Modeling.

Authors: Pascal Janetzky (University Wuerzburg); Florian Gallusser (Universität Würzburg); Simon Hentschel (Julius-Maximilians-Universität of Würzburg); Andreas Hotho (University of Wuerzburg); Anna Krause (Universität Würzburg, Department of Computer Science, CHair X Data Science)
ICLR 2024 Building Ocean Climate Emulators (Papers Track)
Abstract and authors: (click to expand)

Abstract: The current explosion in machine learning for climate has led to skilled, computationally cheap emulators for the atmosphere. However, the research for ocean emulators remains nascent despite the large potential for accelerating coupled climate simulations and improving ocean forecasts on all timescales. There are several fundamental questions to address that can facilitate the creation of ocean emulators. Here we focus on two questions: 1) the role of the atmosphere in improving the extended skill of the emulator and 2) the representation of variables with distinct timescales (e.g., velocity and temperature) in the design of any emulator. In tackling these questions, we show stable prediction of surface fields for over 8 years, training and testing on data from a high-resolution coupled climate model, using results from four regions of the globe. Our work lays out a set of physically motivated guidelines for building ocean climate emulators.

Authors: Adam Subel (New York University); Laure Zanna (New York University)
ICLR 2024 An Adaptive Hydropower Management Approach for Downstream Ecosystem Preservation (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Hydropower plants play a pivotal role in advancing clean and sustainable energy production, contributing significantly to the global transition towards renewable energy sources. However, hydropower plants are currently perceived both positively as sources of renewable energy and negatively as disruptors of ecosystems. In this work, we highlight the overlooked potential of using hydropower plant as protectors of ecosystems by using adaptive ecological discharges. To advocate for this perspective, we propose using a neural network to predict the minimum ecological discharge value at each desired time. Additionally, we present a novel framework that seamlessly integrates it into hydropower management software, taking advantage of the well-established approach of using traditional constrained optimisation algorithms. This novel approach not only protects the ecosystems from climate change but also contributes to potentially increase the electricity production.

Authors: Cecília Coelho (University of Minho); Ming Jin (Virginia Tech); M. Fernanda P. Costa (Dep. Mathematics, University of Minho); Luís L. Ferrás (University of Porto)
ICLR 2024 A Benchmark Dataset for Meteorological Downscaling (Proposals Track)
Abstract and authors: (click to expand)

Abstract: High spatial resolution in atmospheric representations is crucial across Earth science domains, but global reanalysis datasets like ERA5 often lack the detail to capture local phenomena due to their coarse resolution. Recent efforts have leveraged deep neural networks from computer vision to enhance the spatial resolution of meteorological data, showing promise for statistical downscaling. However, methodological diversity and insufficient comparisons with traditional downscaling techniques challenge these advancements. Our study introduces a benchmark dataset for statistical downscaling, utilizing ERA5 and the finer-resolution COSMO-REA6, to facilitate direct comparisons of downscaling methods for 2m temperature, global (solar) irradiance and 100m wind fields. Accompanying U-Net, GAN, and transformer models with a suite of evaluation metrics aim to standardize assessments and promote transparency and confidence in applying deep learning to meteorological downscaling.

Authors: Michael Langguth (Juelich Supercomputing Centre - Forschungszentrum Juelich); Paula Harder (Mila); Irene Schicker (Geos); Ankit Patnala (Juelich Supercomputing Centre - Forschungszentrum Juelich); Sebastian Lehner (GeoSphere Austria); Konrad Mayer (GeoSphere Austria); Markus Dabernig (GeoSphere Austria)
ICLR 2024 Adjustment of ocean carbon sink predictions with an emission-driven Earth system model using a deep neural network (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Near-term predictions of the Global Carbon Budget (GCB) with Earth system models (ESMs) driven by specified CO2 emissions were used to inform the GCB annual update for the first time in 2023. These predictions are biased because they are initialized indirectly from the ESMs response to physical observational constraints, and because the ESMs themselves are imperfect representations of the climate system. We propose a deep learning-based post-processing method to adjust GCB predictions using an autoencoder, which outperforms standard bias and trend correction methods.

Authors: Reinel Sospedra-Alfonso (Environment and Climate Change Canada); Parsa Gooya (Environment and Climate Change Canada); Johannes Exenberger (Graz University of Technology)
ICLR 2024 Calibrating Earth System Models with Bayesian Optimal Experimental Design (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Earth system models (ESMs) are complex climate simulations that are critical for projecting future climate change and its impacts. However, running ESMs is extremely computationally expensive, limiting the number of simulations that can be performed. This results in significant uncertainty in key climate metrics estimated from ESM ensembles. We propose a Bayesian optimal experimental design (BOED) approach to efficiently calibrate ESM simulations to observational data by actively selecting the most informative input parameters. BOED optimises the expected information gain (EIG) to select the ESM input parameter to reduce the final uncertainty estimates in the climate metrics of interest. Initial results on a synthetic benchmark demonstrate our approach can more efficiently reduce uncertainty compared to common sampling schemes like Latin hypercube sampling.

Authors: Tim Reichelt (University of Oxford); Shahine Bouabid (University of Oxford); Luke Ong (University of Oxford); Duncan Watson-Parris (University of California San Diego); Tom Rainforth (University of Oxford)
ICLR 2024 Planning for Floods & Droughts: Intro to AI-Driven Hydrological Modeling (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: This tutorial presents an AI-driven hydrological modeling approach to advance predictions of extreme hydrological events, including floods and droughts, which are of significant socioeconomic concerns. Traditionally, physics-based hydrological models have been the mainstay for simulating rainfall-runoff dynamics and forecasting streamflow. These models, while effective, are constrained by limitations in our systematic understanding and an inability to incorporate heterogeneous data. Recently, the surge in availability of multi-scale, multi-modal hydrological data has spurred the adoption of data-driven machine learning (ML) techniques. These methods have shown promising predictive performance. However, they often struggle with generalization and reliability, especially under climate change. This tutorial introduces physics-informed ML, by leveraging data and domain knowledge, to improve prediction accuracy and trustworthiness. We will delve into uncertainty quantification methods for probabilistic predictions that are vital for climate-resilient planning in managing floods and droughts. Participants will be guided through a comprehensive workflow, encompassing data analysis, model construction, and model evaluation. This tutorial is designed to elevate researchers’ understanding of hydrological systems and provide practitioners with robust, climate-resilient water management tools. These tools are instrumental in facilitating informed decision-making, crucial in the context of climate adaptation strategies. Participants will learn: ● Heterogeneous climate and hydrology data analysis ● State-of-the-art neural network models for rainfall-runoff modeling. ● ML model construction, training, validating, and testing ● Multiple ways to build a physics-informed ML model ● Uncertainty quantification in ML model predictions. All code and data will be publicly available for researchers/practitioners to build their own models.

Authors: Kshitij Tayal (Oak Ridge National Lab); Arvind Renganathan (University of Minnesota); Dan Lu (Oak Ridge National Laboratory)
ICLR 2024 Understanding drivers of climate extremes using regime-specific causal graphs (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: The climate system is intricate, involving numerous interactions among various components at multiple spatio-temporal scales. This complexity poses a significant challenge in understanding and predicting weather extremes within the Earth's climate system. However, a better understanding of the dynamics of such events is crucial due to their profound impact on ecosystems, economies, and worldwide communities. This tutorial will offer a comprehensive guide on using Regime-PCMCI (Saggioro et al., 2020), a constraint-based causal discovery technique, to uncover the causal relationships governing anomalous climate phenomena. Regime-PCMCI is designed to uncover causal relationships in time-series where transitions between regimes exist, and different causal relationships may govern each regime. In this tutorial, we will first discuss how to frame the problem of understanding climate and weather extremes using regime-specific causal discovery. We will shortly introduce constraint-based causal discovery and present the Regime-PCMCI algorithm. To enable participants to gain hands-on experience with the algorithm, we will apply Regime-PCMCI, implemented in the open-source Python package Tigramite (https://github.com/jakobrunge/tigramite), to a real-world climate science problem. Our example will focus on validating hypothesized regime-specific causal graphs that describe the causal relationship between atmospheric circulation, temperature, rainfall, evaporation, and soil moisture under various moisture regimes. Our tutorial will cover essential steps such as data preprocessing, parameter selection, and interpretation of results, ensuring that all participants with a basic understanding of climate science or data analysis can grasp the presented concepts. With this tutorial, we wish to equip participants with the skills to apply Regime-PCMCI in their research to further uncover complex mechanisms in climate science, as this knowledge is crucial for more informed policy-making.

Authors: Oana-Iuliana Popescu (Institute of Data Science, German Aerospace Center (DLR)); Wiebke Günther (German Aerospace Center); Raed Hamed (Institute for Environmental Studies, VU Amsterdam); Dominik Schumacher (4Institute for Atmospheric and Climate Science, ETH Zürich); Martin Rabel (DLR); Dim Coumou (IVM/VU); Jakob Runge (Institute of Data Science, German Aerospace Center (DLR))
ICLR 2024 1. Building Sustainable Futures: Tutorial on Carbon Footprint Analysis and Mitigation Strategies Using Counter Factual Queries (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: As the sense of urgency regarding climate change continues to mount with growing regulatory pressure across the globe, it has become increasingly crucial for enterprises and governments to align their goals with sustainability values. They face a crucial imperative to act on climate change mitigation by disclosing their GHG emissions and committing to reduction and optimization of emissions from their industrial activities including operations, infrastructure, logistics, and supply chains. The world's largest enterprises have set long-term net-zero targets but lacks an integrated view of how their key business operations and processes contribute to their sustainability journey, which makes it difficult for them to embark on a well-planned journey to achieve their sustainability goals. With the recent advancement, AI intervention becomes imperative to measure, track, and improve ESG performance to achieve sustainability goals. This tutorial aims to provide a comprehensive guide on leveraging advanced AI techniques for analysing and mitigating carbon footprints in various sectors. The tutorial covers the utilization of a generalized framework that integrates sector-specific and cross-sector enterprise data, including assets and operations, to derive actionable insights. The framework also uses additional data such as weather parameters and contextual information to facilitate a holistic approach to carbon footprint analysis and its mitigation strategies. The tutorial will delve into the working of a framework which comprises of an LLM driven carbon accounting engine, predictive models for carbon emissions, anomaly detection models, and counterfactual models. It identifies the emission hotspots, thereafter provides actionable recommendations to mitigate the carbon emission. The proposed tutorial aims to empower participants with the knowledge and skills to make informed decisions towards building a more sustainable future

Authors: Kumar Saurav (IBM); Manikandan Padmanaban (IBM Research India); Ayush Jain (IBM Research); Jagabondhu Hazra (IBM Research India)
NeurIPS 2023 Adaptive-Labeling for Enhancing Remote Sensing Cloud Understanding (Papers Track)
Abstract and authors: (click to expand)

Abstract: Cloud analysis is a critical component of weather and climate science, impacting various sectors like disaster management. However, achieving fine-grained cloud analysis, such as cloud segmentation, in remote sensing remains challenging due to the inherent difficulties in obtaining accurate labels, leading to significant labeling errors in training data. Existing methods often assume the availability of reliable segmentation annotations, limiting their overall performance. To address this inherent limitation, we introduce an innovative model-agnostic Cloud Adaptive-Labeling (CAL) approach, which operates iteratively to enhance the quality of training data annotations and consequently improve the performance of the learned model. Our methodology commences by training a cloud segmentation model using the original annotations. Subsequently, it introduces a trainable pixel intensity threshold for adaptively labeling the cloud training images on-the-fly. The newly generated labels are then employed to fine-tune the model. Extensive experiments conducted on multiple standard cloud segmentation benchmarks demonstrate the effectiveness of our approach in significantly boosting the performance of existing segmentation models. Our CAL method establishes new state-of-the-art results when compared to a wide array of existing alternatives.

Authors: Jay Gala (NMIMS); Sauradip Nag (University of Surrey); Huichou Huang (City University of Hong Kong); Ruirui Liu (Brunel University London); Xiatian Zhu (University of Surrey)
NeurIPS 2023 AtmoRep: A stochastic model of atmosphere dynamics using large scale representation learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: AtmoRep is a novel, task-independent stochastic computer model of atmospheric dynamics inspired by the concept of foundation models in natural language processing, like the GPT line or PalmX, applied in the context of Earth system science. The main innovative aspect consists in the fact that the model can skillfully solve scientific tasks it was not specifically trained on, clearly exhibiting in-context learning capabilities. AtmoRep's skill has been tested on nowcasting, temporal interpolation, model correction, and counterfactuals, demonstrating that large-scale neural networks can provide skillful, task-independent models able to complement the existing numerical approaches in multiple applications. In addition, the authors also demonstrated the possibility to further increase the model accuracy by fine tuning it directly on observational data for tasks such as precipitation corrections or downscaling.

Authors: ilaria luise (CERN)
NeurIPS 2023 Physics-Informed Domain-Aware Atmospheric Radiative Transfer Emulator for All Sky Conditions (Papers Track)
Abstract and authors: (click to expand)

Abstract: Radiative transfer modeling is a complex and computationally expensive process that is used to predict how radiation interacts with the atmosphere and Earth's surface. The Rapid Radiation Transfer Model (RRTM) is one such process model that is used in many Earth system models. In recent years, there has been a growing interest in using machine learning (ML) to speed up radiative transfer modeling. ML algorithms can be trained on large datasets of existing RRTM simulations to learn how to predict the results of new simulations without having to run the full RRTM model so one can use the algorithm for new simulations with very light computational demand. This study developed a new physics-based ML emulator for RRTM that is built on a convolutional neural network (CNN) where we trained the CNN on a dataset of 28 years of RRTM simulations. We built a custom loss function, which incorporates information on how radiation interacts with clouds at day- and night-time. The emulator was able to learn how to predict the vertical heating rates in the atmosphere with a high degree of accuracy (RMSE of less than 2% and Pearson's correlation above 0.9). The new ML emulator is over 56 times faster than the original RRTM model on traditional multi-CPU machines. This speedup could allow scientists to call the RRTM much more frequently in atmosphere models, which may improve the accuracy of climate models and reduce the uncertainty in the future climate projections.

Authors: Piyush Garg (Argonne National Laboratory); Emil Constantinescu (Argonne National Laboratory); Bethany Lusch (Argonne National Lab); Troy Arcomano (Argonne National Laboratory); Jiali Wang (Argonne National Laboratory); Rao Kotamarthi (Argonne National Laboratory)
NeurIPS 2023 Uncertainty Quantification of the Madden–Julian Oscillation with Gaussian Processes (Papers Track)
Abstract and authors: (click to expand)

Abstract: The Madden–Julian Oscillation (MJO) is an influential climate phenomenon that plays a vital role in modulating global weather patterns. In spite of the improvement in MJO predictions made by machine learning algorithms, such as neural networks, most of them cannot provide the uncertainty levels in the MJO forecasts directly. To address this problem, we develop a nonparametric strategy based on Gaussian process (GP) models. We calibrate GPs using empirical correlations. Furthermore, we propose a posteriori covariance correction that extends the probabilistic coverage by more than three weeks.

Authors: Haoyuan Chen (Texas A&M University); Emil Constantinescu (Argonne National Laboratory); Vishwas Rao (Argonne National Laboratory); Cristiana Stan (George Mason University)
NeurIPS 2023 ACE: A fast, skillful learned global atmospheric model for climate prediction (Papers Track)
Abstract and authors: (click to expand)

Abstract: Existing ML-based atmospheric models are not suitable for climate prediction, which requires long-term stability and physical consistency. We present ACE (AI2 Climate Emulator), a 200M-parameter, autoregressive machine learning emulator of an existing comprehensive 100-km resolution global atmospheric model. The formulation of ACE allows evaluation of physical laws such as the conservation of mass and moisture. The emulator is stable for 100 years, nearly conserves column moisture without explicit constraints and faithfully reproduces the reference model's climate, outperforming a challenging baseline on over 90% of tracked variables. ACE requires nearly 100x less wall clock time and is 100x more energy efficient than the reference model using typically available resources. Without fine-tuning, ACE can stably generalize to a previously unseen historical sea surface temperature dataset.

Authors: Oliver Watt-Meyer (Allen Institute for AI); Gideon Dresdner (Allen Institute for AI Climate Science); Jeremy McGibbon (Allen Institute for AI); Spencer K Clark (Allen Institute for Artificial Intelligence); James Duncan (University of California, Berkeley); Brian Henn (Allen Institute for AI); Matthew Peters (AI2); Noah D Brenowitz (NVIDIA); Karthik Kashinath (NVIDIA); Mike Pritchard (NVIDIA); Boris Bonev (NVIDIA); Christopher Bretherton (Allen Institute for AI)
NeurIPS 2023 Graph-based Neural Weather Prediction for Limited Area Modeling (Papers Track)
Abstract and authors: (click to expand)

Abstract: The rise of accurate machine learning methods for weather forecasting is creating radical new possibilities for modeling the atmosphere. In the time of climate change, having access to high-resolution forecasts from models like these is also becoming increasingly vital. While most existing Neural Weather Prediction (NeurWP) methods focus on global forecasting, an important question is how these techniques can be applied to limited area modeling. In this work we adapt the graph-based NeurWP approach to the limited area setting and propose a multi-scale hierarchical model extension. Our approach is validated by experiments with a local model for the Nordic region.

Authors: Joel Oskarsson (Linköping University); Tomas Landelius (SMHI); Fredrik Lindsten (Linköping University)
NeurIPS 2023 A machine learning framework for correcting under-resolved simulations of turbulent systems using nudged datasets (Papers Track)
Abstract and authors: (click to expand)

Abstract: Due to the rapidly changing climate, the frequency and severity of extreme weather, such as storms and heatwaves is expected to increase drastically over the coming decades. Accurately quantifying the risk of such events with high spatial resolution is a critical step in the implementation of strategies to prepare for and mitigate the damages. As fully resolved simulations remain computationally out of reach, policy makers must rely on coarse resolution climate models which either parameterize or completely ignore sub-grid scale dynamics. In this work we propose a machine learning framework to debias under-resolved simulations of complex and chaotic dynamical systems such as atmospheric dynamics. The proposed strategy uses ``nudged'' simulations of the coarse model to generate training data designed to minimize the effects of chaotic divergence. We illustrate through a prototype QG model that the proposed approach allows us to machine learn a map from the chaotic attractor of under-resolved dynamics to that of the fully resolved system. In this way we are able to recover extreme event statistics using a very small training dataset.

Authors: Benedikt Barthel (MIT); Themis Sapsis (MIT)
NeurIPS 2023 Spatially-resolved emulation of climate extremes via machine learning stochastic models (Papers Track)
Abstract and authors: (click to expand)

Abstract: Emulators, or reduced-complexity models, serve as an ideal complement to earth system models (ESM) by providing the climate information under various scenarios at much lower computational costs. We develop an emulator of climate extremes that produce the temporal evolution of probability distributions of local variables on a spatially resolved grid. The representative modes of climate change are identified using principal component analysis (PCA), and the PCA time series are approximated using stochastic models. When applied to ERA5 data, the model accurately reproduces the quantiles of local daily maximum temperature and effectively captures the non-Gaussian statistics. We also discuss potential generalization of our emulator to different climate change scenarios.

Authors: Mengze Wang (Massachusetts Institute of Technology); Andre Souza (Massachusetts Institute of Technology); Raffaele Ferrari (Massachusetts Institute of Technology); Themis Sapsis (MIT)
NeurIPS 2023 Extreme Event Prediction with Multi-agent Reinforcement Learning-based Parametrization of Atmospheric and Oceanic Turbulence (Papers Track)
Abstract and authors: (click to expand)

Abstract: Global climate models (GCMs) are the main tools for understanding and predicting climate change. However, due to limited numerical resolutions, these models suffer from major structural uncertainties; e.g., they cannot resolve critical processes such as small-scale eddies in atmospheric and oceanic turbulence. Thus, such small-scale processes have to be represented as a function of the resolved scales via closures (parametrization). The accuracy of these closures is particularly important for capturing climate extremes. Traditionally, such closures are based on heuristics and simplifying assumptions about the unresolved physics. Recently, supervised-learned closures, trained offline on high-fidelity data, have been shown to outperform the classical physics-based closures. However, this approach requires a significant amount of high-fidelity training data and can also lead to instabilities. Reinforcement learning is emerging as a potent alternative for developing such closures as it requires only low-order statistics and leads to stable closures. In Scientific Multi-Agent Reinforcement Learning (SMARL) computational elements serve a dual role of discretization points and learning agents. Here, we leverage SMARL and fundamentals of turbulence physics to learn closures for canonical prototypes of atmospheric and oceanic turbulence. The policy is trained using only the enstrophy spectrum, which is nearly invariant and can be estimated from a few high-fidelity samples. We show that these closures lead to stable low-resolution simulations that, at a fraction of the cost, can reproduce the high-fidelity simulations' statistics, including the tails of the probability density functions (PDFs). These results demonstrate the high potential of SMARL for closure modeling for GCMs, especially in the regime of scarce data and indirect observations.

Authors: Rambod Mojgani (Rice University); Daniel Waelchli (ETHZ); Yifei Guan (Rice University); Petros Koumoutsakos (Harvard); Pedram Hassanzadeh (Rice University)
NeurIPS 2023 Machine learning applications for weather and climate predictions need greater focus on extremes (Papers Track)
Abstract and authors: (click to expand)

Abstract: Multiple studies have now demonstrated that machine learning (ML) can give improved skill for predicting or simulating fairly typical weather events, for tasks such as short-term and seasonal weather forecasting, downscaling simulations to higher resolution and emulating and speeding up expensive model parameterisations. Many of these used ML methods with very high numbers of parameters, such as neural networks, which are the focus of the discussion here. Not much attention has been given to the performance of these methods for extreme event severities of relevance for many critical weather and climate prediction applications, with return periods of more than a few years. This leaves a lot of uncertainty about the usefulness of these methods, particularly for general purpose prediction systems that must perform reliably in extreme situations. ML models may be expected to struggle to predict extremes due to there usually being few samples of such events. However, there are some studies that do indicate that ML models can have reasonable skill for extreme weather, and that it is not hopeless to use them in situations requiring extrapolation. This paper reviews these studies, updating an earlier review, and argues that this is an area that needs researching more. Ways to get a better understanding of how well ML models perform at predicting extreme weather events are discussed.

Authors: Peter Watson (Bristol)
NeurIPS 2023 Uncertainty Quantified Machine Learning for Street Level Flooding Predictions in Norfolk, Virginia (Papers Track)
Abstract and authors: (click to expand)

Abstract: Everyday citizens, emergency responders, and critical infrastructure can be dramatically affected by the flooding of streets and roads. Climate change exacerbates these floods through sea level rise and more frequent major storm events. Low-level flooding, such as nuisance flooding, continues to increase in frequency, especially in cities like Norfolk, Virginia, which can expect nearly 200 flooding events by 2050 [1]. Recently, machine learning (ML) models have been leveraged to produce real-time predictions based on local weather and geographic conditions. However, ML models are known to produce unusual results when presented with data that varies from their training set. For decision-makers to determine the trustworthiness of the model's predictions, ML models need to quantify their prediction uncertainty. This study applies Deep Quantile Regression to a previously published, Long Short-Term Memory-based model for hourly water depth predictions [2], and analyzes its out-of-distribution performance.

Authors: Steven Goldenberg (Thomas Jefferson National Accelerator Facility); Diana McSpadden (Thomas Jefferson National Accelerator Facility); Binata Roy (University of Virginia); Malachi Schram (Thomas Jefferson National Accelerator Facility); Jonathan Goodall (University of Virginia); Heather Richter (Old Dominion University)
NeurIPS 2023 Zero shot microclimate prediction with deep learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: While weather station data is a valuable resource for climate prediction, its reliability can be limited in remote locations. Furthermore, making local predictions often relies on sensor data that may not be accessible for a new, unmonitored location. In response to these challenges, we introduce a novel zero-shot learning approach designed to forecast various climate measurements at new and unmonitored locations. Our method surpasses conventional weather forecasting techniques in predicting microclimate variables by leveraging knowledge extracted from other geographic locations.

Authors: Iman Deznabi (UMass); Peeyush Kumar (Microsoft Research); Madalina Fiterau (University of Massachusetts Amherst)
NeurIPS 2023 Unleashing the Autoconversion Rates Forecasting: Evidential Regression from Satellite Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: High-resolution simulations such as the ICOsahedral Non-hydrostatic Large-Eddy Model (ICON-LEM) can be used to understand the interactions between aerosols, clouds, and precipitation processes that currently represent the largest source of uncertainty involved in determining the radiative forcing of climate change. Nevertheless, due to the exceptionally high computing cost required, this simulation-based approach can only be employed for a short period of time within a limited area. Despite the fact that machine learning can mitigate this problem, the related model uncertainties may make it less reliable. To address this, we developed a neural network (NN) model powered with evidential learning to assess the data and model uncertainties applied to satellite observation data. Our study focuses on estimating the rate at which small droplets (cloud droplets) collide and coalesce to become larger droplets (raindrops) – autoconversion rates -- since this is one of the key processes in the precipitation formation of liquid clouds, hence crucial to better understanding cloud responses to anthropogenic aerosols. The results of estimating the autoconversion rates demonstrate that the model performs reasonably well, with the inclusion of both aleatoric and epistemic uncertainty estimation, which improves the credibility of the model and provides useful insights for future improvement.

Authors: Maria C Novitasari (University College London); Johannes Quaas (Universität Leipzig); Miguel Rodrigues (University College London)
NeurIPS 2023 Climate Variable Downscaling with Conditional Normalizing Flows (Papers Track)
Abstract and authors: (click to expand)

Abstract: Predictions of global climate models typically operate on coarse spatial scales due to the large computational costs of climate simulations. This has led to a considerable interest in methods for statistical downscaling, a similar process to super-resolution in the computer vision context, to provide more local and regional climate information. In this work, we apply conditional normalizing flows to the task of climate variable downscaling. This approach allows for a probabilistic interpretation of the predictions, while also capturing the stochasticity inherent in the relationships among fine and coarse spatial scales. We showcase its successful performance on an ERA5 water content dataset for different upsampling factors. Additionally, we show that the method allows us to assess the predictive uncertainty in terms of standard deviation from the fitted conditional distribution mean.

Authors: Christina Elisabeth Winkler (Mila); Paula Harder (Mila); David Rolnick (McGill University, Mila)
NeurIPS 2023 IceCloudNet: Cirrus and mixed-phase cloud prediction from SEVIRI input learned from sparse supervision (Papers Track)
Abstract and authors: (click to expand)

Abstract: Clouds containing ice particles play a crucial role in the climate system. Yet they remain a source of great uncertainty in climate models and future climate projections. In this work, we create a new observational constraint of regime-dependent ice microphysical properties at the spatio-temporal coverage of geostationary satellite instruments and the quality of active satellite retrievals. We achieve this by training a convolutional neural network on three years of SEVIRI and DARDAR data sets. This work will enable novel research to improve ice cloud process understanding and hence, reduce uncertainties in a changing climate and help assess geoengineering methods for cirrus clouds.

Authors: Kai Jeggle (ETH Zurich); Mikolaj Czerkawski (ESA); Federico Serva (European Space Agency, Italian Space Agency); Bertrand Le Saux (European Space Agency (ESA)); David Neubauer (ETH Zurich); Ulrike Lohmann (ETH Zurich)
NeurIPS 2023 Surrogate modeling based History Matching for an Earth system model of intermediate complexity (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate General Circulation Models (GCMs) constitute the primary tools for climate projections that inform IPCC Assessment Reports. Calibrating, or tuning the parameters of the models can significantly improve their predictions, thus their scientific and societal impacts. Unfortunately, traditional tuning techniques remain time-consuming and computationally costly, even at coarse resolution. A specific challenge for the tuning of climate models lies in the tuning of both fast and slow climatic features: while atmospheric processes adjust on hourly to weekly timescales, vegetation or ocean dynamics drive mechanisms of variability at decadal to millenial timescales. In this work, we explore whether and how History Matching, which uses machine learning based emulators to accelerate and automate the tuning process, is relevant for tuning climate models with multiple timescales. To facilitate this exploration, we work with a climate model of intermediate complexity, yet test experimental tuning protocols that can be directly applied to more complex GCMs to reduce uncertainty in climate projections.

Authors: Maya Janvier (Centrale Supélec); Redouane Lguensat (IPSL); Julie Deshayes (LOCEAN IPSL); Aurélien Quiquet (LSCE); Didier Roche (LSCE); V. Balaji (Schmidt Futures)
NeurIPS 2023 Typhoon Intensity Prediction with Vision Transformer (Papers Track)
Abstract and authors: (click to expand)

Abstract: Predicting typhoon intensity accurately across space and time is crucial for issuing timely disaster warnings and facilitating emergency response. This has vast potential for minimizing life losses and property damages as well as reducing economic and environmental impacts. Leveraging satellite imagery for scenario analysis is effective but also introduces additional challenges due to the complex relations among clouds and the highly dynamic context. Existing deep learning methods in this domain rely on convolutional neural networks (CNNs), which suffer from limited per-layer receptive fields. This limitation hinders their ability to capture long-range dependencies and global contextual knowledge during inference. In response, we introduce a novel approach, namely "Typhoon Intensity Transformer" (Tint), which leverages self-attention mechanisms with global receptive fields per layer. Tint adopts a sequence-to-sequence feature representation learning perspective. It begins by cutting a given satellite image into a sequence of patches and recursively employs self-attention operations to extract both local and global contextual relations between all patch pairs simultaneously, thereby enhancing per-patch feature representation learning. Extensive experiments on a publicly available typhoon benchmark validate the efficacy of Tint in comparison with both state-of-the-art deep learning and conventional meteorological methods. Our code is available at https://github.com/chen-huanxin/Tint.

Authors: Huanxin Chen (South China University of Technology); Pengshuai Yin (South China University of Technology); Huichou Huang (City University of Hong Kong); Qingyao Wu (South China University of Technology); Ruirui Liu (Brunel University London); Xiatian Zhu (University of Surrey)
NeurIPS 2023 Stress-testing the coupled behavior of hybrid physics-machine learning climate simulations on an unseen, warmer climate (Papers Track)
Abstract and authors: (click to expand)

Abstract: Accurate and computationally-viable representations of clouds and turbulence are a long-standing challenge for climate model development. Traditional parameterizations that crudely but efficiently approximate these processes are a leading source of uncertainty in long-term projected warming and precipitation patterns. Machine Learning (ML)-based parameterizations have long been hailed as a promising alternative with the potential to yield higher accuracy at a fraction of the cost of more explicit simulations. However, these ML variants are often unpredictably unstable and inaccurate in online testing (i.e. in a downstream hybrid simulation task where they are dynamically coupled to the large-scale climate model). These issues are exacerbated in out-of-distribution climates. Certain design decisions such as ``climate-invariant" feature transformation, input vector expansion, and temporal history incorporation have been shown to improve online performance, but they may be insufficient for the mission-critical task of online out-of-distribution generalization. If feature selection and transformations can inoculate hybrid physics-ML climate models from non-physical out-of-distribution extrapolation in a changing climate, there is far greater potential in extrapolating from observational data. Otherwise, training on multiple simulated climates becomes an inevitable necessity. While our results show generalization benefits from these design decisions, such benefits do not sufficiently preclude the necessity of using multi-climate simulated training data.

Authors: Jerry Lin (University of California, Irvine); Mohamed Aziz Bhouri (Columbia University); Tom G Beucler (Columbia University & UCI); Sungduk Yu (University of California, Irvine); Michael Pritchard (UCI)
NeurIPS 2023 Towards a spatio-temporal deep learning approach to predict malaria outbreaks using earth observation measurements in South Asia (Papers Track)
Abstract and authors: (click to expand)

Abstract: Environmental indicators can play a crucial role in forecasting infectious disease outbreaks, holding promise for community-level interventions. Yet, significant gaps exist in the literature regarding the influence of changes in environmental conditions on disease spread over time and across different regions and climates making it challenging to obtain reliable forecasts. This paper aims to propose an approach to predict malaria incidence over time and space by employing a multi-dimensional long short-term memory model (M-LSTM) to simultaneously analyse environmental indicators such as vegetation, temperature, night-time lights, urban/rural settings, and precipitation. We developed and validated a spatio-temporal data fusion approach to predict district-level malaria incidence rates for the year 2017 using spatio-temporal data from 2000 to 2016 across three South Asian countries: Pakistan, India, and Bangladesh. In terms of predictive performance the proposed M-LSTM model results in lower country-specific error rates compared to existing spatio-temporal deep learning models. The data and code have been made publicly available at the study GitHub repository.

Authors: Usman Nazir (Lahore University of Management Sciences); Ahzam Ejaz (Lahore University of Management Sciences); Muhammad Talha Quddoos (Lahore University of Management Sciences); Momin Uppal (Lahore University of Management Sciences); Sara khalid (University of Oxford)
NeurIPS 2023 Comparing Data-Driven and Mechanistic Models for Predicting Phenology in Deciduous Broadleaf Forests (Papers Track)
Abstract and authors: (click to expand)

Abstract: Understanding the future climate is crucial for informed policy decisions on climate change prevention and mitigation. Earth system models play an important role in predicting future climate, requiring accurate representation of complex sub- processes that span multiple time scales and spatial scales. One such process that links seasonal and interannual climate variability to cyclical biological events is tree phenology in deciduous forests. Phenological dates, such as the start and end of the growing season, are critical for understanding the exchange of carbon and water between the biosphere and the atmosphere. Mechanistic prediction of these dates is challenging. Hybrid modelling, which integrates data-driven approaches into complex models, offers a solution. In this work, as a first step towards this goal, train a deep neural network to predict a phenological index from meteorological time series. We find that this approach outperforms traditional process-based models. This highlights the potential of data-driven methods to improve climate predictions. We also analyze which variables and aspects of the time series influence the predicted onset of the season, in order to gain a better understanding of the advantages and limitations of our model.

Authors: Christian Reimers (Department of Biogeochemical Integration, Max Planck Institute for Biogeochemistry); David Hafezi Rachti (Department of Biogeochemical Integration, Max Planck Institute for Biogeochemistry); Guohua Liu (Department of Biogeochemical Integration, Max Planck Institute for Biogeochemistry); Alexander Winkler (Department of Biogeochemical Integration, Max Planck Institute for Biogeochemistry)
NeurIPS 2023 RMM-VAE: a machine learning method for identifying probabilistic weather regimes targeted to a local-scale impact variable (Papers Track)
Abstract and authors: (click to expand)

Abstract: Identifying large-scale atmospheric patterns that modulate extremes in local-scale variables such as precipitation has the potential to improve long-term climate projections as well as extended-range forecasting skill. This paper proposes a novel probabilistic machine learning method, RMM-VAE, based on a variational autoencoder architecture for identifying weather regimes targeted to a local-scale impact variable. The new method is compared to three existing methods in the task of identifying robust weather regimes that are predictive of precipitation over Morocco while capturing the full phase space of atmospheric dynamics over the Mediterranean. RMM-VAE performs well across these different objectives, outperforming linear methods in reconstructing the full phase space and predicting the target variable, highlighting the potential benefit of applying the method to various climate applications such as downscaling and extended-range forecasting.

Authors: Fiona R Spuler (University of Reading); Marlene Kretschmer (Universität Leipzig); Yevgeniya Kovalchuck (University College London); Magdalena Balmaseda (ECMWF); Ted Shepherd (University of Reading)
NeurIPS 2023 Accelerating GHG Emissions Inference: A Lagrangian Particle Dispersion Model Emulator Using Graph Neural Networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: Inverse modelling systems relying on Lagrangian Particle Dispersion Models (LPDMs) are a popular way to quantify greenhouse gas (GHG) emissions using atmospheric observations, providing independent validation to countries' self-reported emissions. However, the increased volume of satellite measurements cannot be fully leveraged due to computational bottlenecks. Here, we propose a data-driven architecture with Graph Neural Networks that emulates the outputs of LPDMs using only meteorological inputs, and demonstrate it in application with preliminary results for satellite measurements over Brazil.

Authors: Elena Fillola (University of Bristol); Raul Santos Rodriguez (University of Bristol); Matt Rigby (University of Bristol)
NeurIPS 2023 Sim2Real for Environmental Neural Processes (Papers Track)
Abstract and authors: (click to expand)

Abstract: Machine learning (ML)-based weather models have recently undergone rapid improvements.These models are typically trained on gridded reanalysis data from numerical data assimilation systems. However, reanalysis data comes with limitations, such as assumptions about physical laws and low spatiotemporal resolution. The gap between reanalysis and reality has sparked growing interest in training ML models directly on observations such as weather stations. Modelling scattered and sparse environmental observations requires scalable and flexible ML architectures, one of which is the convolutional conditional neural process (ConvCNP). ConvCNPs can learn to condition on both gridded and off-the-grid context data to make uncertainty-aware predictions at target locations. However, the sparsity of real observations presents a challenge for data-hungry deep learning models like the ConvCNP. One potential solution is `Sim2Real': pre-training on reanalysis and fine-tuning on observational data. We analyse Sim2Real with a ConvCNP trained to interpolate surface air temperature over Germany, using varying numbers of weather stations for fine-tuning. On held-out weather stations, Sim2Real training substantially outperforms the same model trained only with reanalysis data or only with station data, showing that reanalysis data can serve as a stepping stone for learning from real observations. Sim2Real could enable more accurate models for climate change monitoring and adaptation.

Authors: Jonas Scholz (University of Cambridge)
NeurIPS 2023 Machine Learning Assisted Bayesian Calibration of Model Physics Parameters for Wetland Methane Emissions: A Case Study at a FLUXNET-CH4 Site (Papers Track)
Abstract and authors: (click to expand)

Abstract: Methane (CH4) possesses a notably higher warming potential than carbon dioxide despite its lower atmospheric concentration, making it integral to global climate dynamics. Wetlands stand out as the predominant natural contributor to global methane emissions. Accurate modeling of methane emissions from wetlands is crucial for understanding and predicting climate change dynamics. However, such modeling efforts are often constrained by the inherent uncertainties in model parameters. Our work leverages machine learning (ML) to calibrate five physical parameters of the Energy Exascale Earth System Model (E3SM) land model (ELM) to improve the model’s accuracy in simulating wetland methane emissions. Unlike traditional deterministic calibration methods that target a single set of optimal values for each parameter, Bayesian calibration takes a probabilistic approach and enables capturing the inherent uncertainties in complex systems and providing robust parameter distributions for reliable predictions. However, Bayesian calibration requires numerous model runs and makes it computationally expensive. We employed an ML algorithm, Gaussian process regression (GPR), to emulate the ELM’s methane model, which dramatically reduced the computational time from 6 CPU hours to just 0.72 milliseconds per simulation. We exemplified the procedure at a representative FLUXNET-CH4 site (US-PFa) with the longest continuous methane emission data. Results showed that the default values for two of the five parameters examined were not aligned well with their respective posterior distributions, suggesting that the model’s default parameter values might not always be optimal for all sites, and that site-specific analysis is warranted. In particular, analyses at sites with different vegetation types and wetland characteristics could reveal more useful insights for understanding methane emissions modeling.

Authors: Sandeep Chinta (Massachusetts Institute of Technology); Xiang Gao (Massachusetts Institute of Technology); Qing Zhu (Lawrence Berkeley National Laboratory)
NeurIPS 2023 ALAS: Active Learning for Autoconversion Rates Prediction from Satellite Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: High-resolution simulations, such as the ICOsahedral Non-hydrostatic Large-Eddy Model (ICON-LEM), provide valuable insights into the complex interactions among aerosols, clouds, and precipitation, which are the major contributors to climate change uncertainty. However, due to their exorbitant computational costs, they can only be employed for a limited period and geographical area. To address this, we propose a more cost-effective method powered by an emerging machine learning approach to better understand the intricate dynamics of the climate system. Our approach involves active learning techniques by leveraging high-resolution climate simulation as an oracle that is queried based on an abundant amount of unlabeled data drawn from satellite observations. In particular, we aim to predict autoconversion rates, a crucial step in precipitation formation, while significantly reducing the need for a large number of labeled instances. In this study, we present novel methods: custom query strategy fusion for labeling instances -- weight fusion (WiFi) and merge fusion (MeFi) -- along with active feature selection based on SHAP. These methods are designed to tackle real-world challenges -- in this case, climate change, with a specific focus on the prediction of autoconversion rates -- due to their simplicity and practicality in application.

Authors: Maria C Novitasari (University College London); Johanness Quaas (Universität Leipzig); Miguel Rodrigues (University College London)
NeurIPS 2023 Towards Causal Representations of Climate Model Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate models, such as Earth system models (ESMs), are crucial for simulating future climate change based on projected Shared Socioeconomic Pathways (SSP) greenhouse gas emissions scenarios. While ESMs are sophisticated and invaluable, machine learning-based emulators trained on existing simulation data can project additional climate scenarios much faster and are computationally efficient. However, they often lack generalizability and interpretability. This work delves into the potential of causal representation learning, specifically the Causal Discovery with Single-parent Decoding (CDSD) method, which could render climate model emulation efficient and interpretable. We evaluate CDSD on multiple climate datasets, focusing on emissions, temperature, and precipitation. Our findings shed light on the challenges, limitations, and promise of using CDSD as a stepping stone towards more interpretable and robust climate model emulation.

Authors: Julien Boussard (Columbia University); Chandni Nagda (University of Illinois at Urbana-Champaign); Julia Kaltenborn (McGill University); Charlotte Lange (Mila); Yaniv Gurwicz (Intel Labs); Peer Nowack (Grantham Institute, Imperial College London. Department of Physics, Imperial College. Data Science Institute, Imperial College. School of Environmental Sciences, University of East Anglia); David Rolnick (McGill University, Mila)
NeurIPS 2023 Exploring Causal Relationship between Environment and Drizzle Properties using Machine Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Cloud and precipitation properties are controlled by both local and large-scale forcings. Current weather and climate models represent clouds and precipitation through parameterizations that are based on theoretical relationships between environment, clouds, and precipitation. However, these relationships vary considerably among different weather and cloud conditions, thereby leading to inaccurate simulation of cloud and precipitation properties. In this study, we use observations from a site in the Eastern North Atlantic Ocean (28W, 39.5N) to establish a potential causal relationship between large-scale environment, cloud, and precipitation properties. We estimate the structure of a directed acyclic graph (DAG) with the NOTEARS algorithm (Non-combinatorial Optimization via Trace Exponential and Augmented lagRangian for Structure learning) (Zheng et al., 2018 \cite{Zheng2018DAGsLearning}) with a multi-layer perceptron (MLP) neural network classification architecture. We classify liquid water path (LWP), rain rate, and rain drop diameter in two classes based on lower and upper quantiles to identify the governing mechanisms responsible for the two tails of the distribution. We also invoke Random Forest classification to compare our causal model results with conventional decision tree-based approaches. We hypothesize the dominant role of cloud LWP and net radiative cooling in controlling the cloud and precipitation properties. In this way, this study demonstrates the application of a causal machine learning method to identify which environmental properties potentially control cloud and precipitation development. These results will be extremely valuable to both observational and numerical modeling communities as they could help improve the current parameterizations in the weather and climate models.

Authors: Piyush Garg (Argonne National Laboratory); Virendra Ghate (Argonne National Laboratory); Maria Cadeddu (Argonne National Laboratory); Bethany Lusch (Argonne National Lab)
NeurIPS 2023 GraphTransformers for Geospatial Forecasting of Hurricane Trajectories (Papers Track)
Abstract and authors: (click to expand)

Abstract: In this paper we introduce a novel framework for trajectory prediction of geospatial sequences using GraphTransformers. When viewed across several sequences, we observed that a graph structure automatically emerges between different geospatial points that is often not taken into account for such sequence modeling tasks. We show that by leveraging this graph structure explicitly, geospatial trajectory prediction can be significantly improved. Our GraphTransformer approach improves upon state-of-the-art Transformer based baseline significantly on HURDAT, a dataset where we are interested in predicting the trajectory of a hurricane on a 6 hourly basis. This helps inform evacuation efforts by narrowing down target location by 10 to 20 kilometers along both the north-south and east-west directions.

Authors: Satyaki Chakraborty (Carnegie Mellon University); Pallavi Banerjee (University of Washington)
NeurIPS 2023 A 3D super-resolution of wind fields via physics-informed pixel-wise self-attention generative adversarial network (Papers Track)
Abstract and authors: (click to expand)

Abstract: To mitigate global warming, greenhouse gas sources need to be resolved at a high spatial resolution and monitored in time to ensure the reduction and ultimately elimination of the pollution source. However, the complexity of computation in resolving high-resolution wind fields left the simulations impractical to test different time lengths and model configurations. This study presents a preliminary development of a physics-informed super-resolution (SR) generative adversarial network (GAN) that super-resolves the three-dimensional (3D) low-resolution wind fields by upscaling x9 times. We develop a pixel-wise self-attention (PWA) module that learns 3D weather dynamics via a self-attention computation followed by a 2D convolution. We also employ a loss term that regularizes the self-attention map during pretraining, capturing the vertical convection process from input wind data. The new PWA SR-GAN shows the high-fidelity super-resolved 3D wind data, learns a wind structure at the high-frequency domain, and reduces the computational cost of a high-resolution wind simulation by x 89.7 times.

Authors: Takuya Kurihana (University of Chicago); Levente Klein (IBM Research); Kyongmin Yeo (IBM Research); Daniela Szwarcman (IBM Research); Bruce G Elmegreen (IBM Research); Surya Karthik Mukkavilli (IBM Research, Zurich); Johannes Schmude (IBM)
NeurIPS 2023 Proof-of-concept: Using ChatGPT to Translate and Modernize an Earth System Model from Fortran to Python/JAX (Papers Track)
Abstract and authors: (click to expand)

Abstract: Earth system models (ESMs) are vital for understanding past, present, and future climate, but they suffer from legacy technical infrastructure. ESMs are primarily implemented in Fortran, a language that poses a high barrier of entry for early career scientists and lacks a GPU runtime, which has become essential for continued advancement as GPU power increases and CPU scaling slows. Fortran also lacks differentiability — the capacity to differentiate through numerical code — which enables hybrid models that integrate machine learning methods. Converting an ESM from Fortran to Python/JAX could resolve these issues. This work presents a semi-automated method for translating individual model components from Fortran to Python/JAX using a large language model (GPT-4). By translating the photosynthesis model from the Community Earth System Model (CESM), we demonstrate that the Python/JAX version results in up to 100x faster runtimes using GPU parallelization, and enables parameter estimation via automatic differentiation. The Python code is also easy to read and run and could be used by instructors in the classroom. This work illustrates a path towards the ultimate goal of making climate models fast, inclusive, and differentiable.

Authors: Anthony Zhou (Columbia University), Linnia Hawkins (Columbia University), Pierre Gentine (Columbia University)
NeurIPS 2023 Learning to forecast diagnostic parameters using pre-trained weather embedding (Papers Track)
Abstract and authors: (click to expand)

Abstract: Data-driven weather prediction (DDWP) models are increasingly becoming popular for weather forecasting. However, while operational weather forecasts predict a wide variety of weather variables, DDWPs currently forecast a specific set of key prognostic variables. Non-prognostic ("diagnostic") variables are sometimes modeled separately as dependent variables of the prognostic variables (c.f. FourCastNet \cite{pathak2022fourcastnet}), or by including the diagnostic variable as a target in the DDWP. However, the cost of training and deploying bespoke models for each diagnostic variable can increase dramatically with more diagnostic variables, and limit the operational use of such models. Likewise, retraining an entire DDWP each time a new diagnostic variable is added is also cost-prohibitive. We present an two-stage approach that allows new diagnostic variables to be added to an end-to-end DDWP model without the expensive retraining. In the first stage, we train an autoencoder that learns to embed prognostic variables into a latent space. In the second stage, the autoencoder is frozen and "downstream" models are trained to predict diagnostic variables using only the latent representations of prognostic variables as input. Our experiments indicate that models trained using the two-stage approach offer accuracy comparable to training bespoke models, while leading to significant reduction in resource utilization during training and inference. This approach allows for new "downstream" models to be developed as needed, without affecting existing models and thus reducing the friction in operationalizing new models.

Authors: Peetak Mitra (Excarta); Vivek Ramavajjala (Excarta)
NeurIPS 2023 Generative Nowcasting of Marine Fog Visibility in the Grand Banks area and Sable Island in Canada (Papers Track)
Abstract and authors: (click to expand)

Abstract: This study presents the application of generative deep learning techniques to evaluate marine fog visibility nowcasting using the FATIMA (Fog and turbulence interactions in the marine atmosphere) campaign observations collected during July 2022 in the North Atlantic in the Grand Banks area and vicinity of Sable Island (SI), northeast of Canada. The measurements were collected using the Vaisala Forward Scatter Sensor model FD70 and Weather Transmitter model WXT50, and Gill R3A ultrasonic anemometer mounted on the Research Vessel Atlantic Condor. To perform nowcasting, the time series of fog visibility (Vis), wind speed, dew point depression, and relative humidity with respect to water were preprocessed to have lagged time step features. Generative nowcasting of Vis time series for lead times of 30 and 60 minutes were performed using conditional generative adversarial networks (cGAN) regression at visibility thresholds of Vis < 1 km and < 10 km. Extreme gradient boosting (XGBoost) was used as a baseline method for comparison against cGAN. At the 30 min lead time, Vis was best predicted with cGAN at Vis < 1 km (RMSE = 0.151 km) and with XGBoost at Vis < 10 km (RMSE = 2.821 km). At the 60 min lead time, Vis was best predicted with XGBoost at Vis < 1 km (RMSE = 0.167 km) and Vis < 10 km (RMSE = 3.508 km), but the cGAN RMSE was similar to XGBoost. Despite nowcasting Vis at 30 min being quite difficult, the ability of the cGAN model to track the variation in Vis at 1 km suggests that there is potential for generative analysis of marine fog visibility using observational meteorological parameters.

Authors: Eren Gultepe (Southern Illinois University Edwardsville); Sen Wang (University of Notre Dame); Byron Blomquist (NOAA); Harindra Fernando (University of Notre Dame); Patrick Kreidl (University of North Florida); David Delene (University of North Dakota); Ismail Gultepe (Ontario Tech University)
NeurIPS 2023 Spatially Regularized Graph Attention Autoencoder Framework for Detecting Rainfall Extremes (Papers Track)
Abstract and authors: (click to expand)

Abstract: We introduce a novel Graph Attention Autoencoder (GAE) with spatial regularization to address the challenge of scalable anomaly detection in spatiotemporal rainfall data across India from 1990 to 2015. Our model leverages a Graph Attention Network (GAT) to capture spatial dependencies and temporal dynamics in the data, further enhanced by a spatial regularization term ensuring geographic coherence. We construct two graph datasets employing rainfall, pressure, and temperature attributes from the Indian Meteorological Department and ERA5 Reanalysis on Single Levels, respectively. Our network operates on graph representations of the data, where nodes represent geographic locations, and edges, inferred through event synchronization, denote significant co-occurrences of rainfall events. Through extensive experiments, we demonstrate that our GAE effectively identifies anomalous rainfall patterns across the Indian landscape. Our work paves the way for sophisticated spatiotemporal anomaly detection methodologies in climate science, contributing to better climate change preparedness and response strategies.

Authors: Mihir Agarwal (IIT Gandhinagar); Progyan Das (IIT Gandhinagar); Udit Bhatia (IIT Gandhinagar)
NeurIPS 2023 Antarctic Bed Topography Super-Resolution via Transfer Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: High-fidelity topography models of the bedrock underneath the thick Antarctic ice sheet can improve scientists' understanding of ice flow and its contributions to sea level rise. However, the bed topography of Antarctica is one of the most challenging surfaces on earth to map, requiring aircrafts with ice penetrating radars to survey the vast and remote continent. We propose FROST, Fusion Regression for Optimal Subglacial Topography, a method that leverages readily available surface topography data from satellites as an auxiliary input modality for bed topography super-resolution. FROST uses a non-parametric Gaussian Process model to transfer local, non-stationary covariance patterns from the ice surface to the bedrock. In a controlled topography reconstruction experiment over complex East Antarctic terrain, our proposed method outperforms bicubic interpolation at all five tested magnification factors, reducing RMSE by 67% at x2, and 25% at x6 magnification. This work demonstrates the opportunity for data fusion methods to advance downstream climate modelling and steward climate change adaptation.

Authors: Kim Bente (The University of Sydney); Roman Marchant (University of Technology Sydney); Fabio Ramos (NVIDIA, The University of Sydney)
ICLR 2023 Tutorial: Quantus x Climate - Applying explainable AI evaluation in climate science (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: Explainable artificial intelligence (XAI) methods shed light on the predictions of deep neural networks (DNNs). In the climate context, XAI has been applied to improve and validate deep learning (DL) methods while providing researchers with new insight into physical processes. However, the evaluation, validation and selection of XAI methods are challenging due to often lacking ground truth explanations. In this tutorial, we introduce the XAI evaluation package Quantus to the climate community. We start by providing the users with pre-processed input and output data alongside a convolutional neural network (CNN) trained to assign yearly temperature maps to classes according to their decade. We explain the network prediction of an example temperature map using five different explanation techniques Gradient GradientShap, IntegratedGradients, LRP-z and Occlusion. By visually analyzing each explanation method around the North Atlantic (NA) cooling patch 10-80W, 20-60N, we provide a motivating example that shows that different explanations may disagree in their explained evidence which subsequently can lead to different scientific interpretation and potentially, misleading conclusions. We continue by introducing Quantus including the explanation properties that can be evaluated such as robustness, faithfulness, complexity, localization and randomization. We guide the participants towards a practical understanding of XAI evaluation by demonstrating how metrics differ in their scoring and interpretation. Moreover, we teach the participants to compare and select an appropriate XAI method by performing a comprehensive XAI evaluation. Lastly, we return to the motivating example, highlighting how Quantus can facilitate well-founded XAI research in climate science.

Authors: Philine L Bommer (TU Berlin); Anna Hedström (Technische Universität Berlin); Marlene Kretschmer (University of Reading); Marina M.-C. Höhne (TU Berlin)
ICLR 2023 Improving global high-resolution Earth system model simulations of precipitation with generative adversarial networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: Precipitation extremes are expected to become stronger and more frequent in response to anthropogenic global warming. Accurately projecting the ecological and socioeconomic impacts is an urgent task. Impact models are developed and calibrated with observation-based data but rely on Earth system model (ESM) output for future scenarios. ESMs, however, exhibit significant biases in their output because they cannot fully resolve complex cross-scale interactions of processes that produce precipitation cannot. State-of-the-art bias correction methods only address errors in the simulated frequency distributions, locally at every individual grid cell. Improving unrealistic spatial patterns of the ESM output, which would require spatial context, has not been possible so far. Here we show that a post-processing method based on physically constrained generative adversarial networks (GANs) can correct biases of a state-of-the-art global ESM both in local frequency distributions and in the spatial patterns at once. While our method improves local frequency distributions similarly well to a gold-standard ESM bias-adjustment framework, it strongly outperforms existing methods in correcting spatial patterns. Our study highlights the importance of physical constraints in neural networks for out-of-sample predictions in the context of climate change.

Authors: Philipp Hess (Technical University of Munich)
ICLR 2023 Improving extreme weather events detection with light-weight neural networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: To advance automated detection of extreme weather events, which are increasing in frequency and intensity with climate change, we explore modifications to a novel light-weight Context Guided convolutional neural network architecture trained for semantic segmentation of tropical cyclones and atmospheric rivers in climate data. Our primary focus is on tropical cyclones, the most destructive weather events, for which current models show limited performance. We investigate feature engineering, data augmentation, learning rate modifications, alternative loss functions, and architectural changes. In contrast to previous approaches optimizing for intersection over union, we specifically seek to improve recall to penalize under-counting and prioritize identification of tropical cyclones. We report success through the use of weighted loss functions to counter class imbalance for these rare events. We conclude with directions for future research on extreme weather events detection, a crucial task for prediction, mitigation, and equitable adaptation to the impacts of climate change.

Authors: Romain Lacombe (Stanford University); Hannah Grossman (Stanford); Lucas P Hendren (Stanford University); David Ludeke (Stanford University)
ICLR 2023 Long-lead forecasts of wintertime air stagnation index in southern China using oceanic memory effects (Papers Track)
Abstract and authors: (click to expand)

Abstract: Stagnant weather condition is one of the major contributors to air pollution as it is favorable for the formation and accumulation of pollutants. To measure the atmosphere’s ability to dilute air pollutants, Air Stagnation Index (ASI) has been introduced as an important meteorological index. Therefore, making long-lead ASI forecasts is vital to make plans in advance for air quality management. In this study, we found that autumn Niño indices derived from sea surface temperature (SST) anomalies show a negative correlation with wintertime ASI in southern China, offering prospects for a prewinter forecast. We developed an LSTM-based model to predict the future wintertime ASI. Results demonstrated that multivariate inputs (past ASI and Niño indices) achieve better forecast performance than univariate input (only past ASI). The model achieves a correlation coefficient of 0.778 between the actual and predicted ASI, exhibiting a high degree of consistency.

Authors: Chenhong Zhou (Hong Kong Baptist University); Xiaorui Zhang (Hong Kong Baptist University); Meng Gao (Hong Kong Baptist University); Shanshan Liu (University of science and technology of China); Yike Guo (Hong Kong University of Science and Technology); Jie Chen (Hong Kong Baptist University)
ICLR 2023 Unsupervised machine learning techniques for multi-model comparison: A case study on Antarctic Intermediate Water in CMIP6 models (Papers Track)
Abstract and authors: (click to expand)

Abstract: The Climate Model Intercomparison Project provides access to ensembles of model experiments that are widely used to better understand past, present, and future climate changes. In this study, we use Principal Component Analysis and K-means and hierarchical clustering techniques to guide identification of models in the CMIP6 dataset that are best suited for specific modelling objectives. An example is discussed here that focuses on how CMIP6 models reproduce the physical properties of Antarctic Intermediate Water, a key feature of the global oceanic circulation and of the ocean-climate system, noting that the tools and methods introduced here can readily be extended to the analysis of other features and regions.

Authors: Ophelie Meuriot (Imperial College London); Yves Plancherel (Imperial College London); Veronica Nieves (University of Valencia)
ICLR 2023 An automatic mobile approach for Tree DBH Estimation Using a Depth Map and a Regression Convolutional Neural Network (Papers Track)
Abstract and authors: (click to expand)

Abstract: Carbon credit programs finance projects to reduce emissions, remove pollutants, improve livelihoods, and protect natural ecosystems. Ensuring the quality and integrity of such projects is essential to their success. One of the most important variables used in nature-based solutions to measure carbon sequestration is the diameter at breast height (DBH) of trees. In this paper, we propose an automatic mobile computer vision method to estimate the DBH of a tree using a single depth map on a smartphone, along with our created dataset DepthMapDBH2023. We successfully demonstrated that this dataset paired with a lightweight regression convolutional neural network is able to accurately estimate the DBH of trees distinct in appearance, shape, number of tree forks, tree density and crowding, and vine presence. Automation of these measurements will help crews in the field who are collecting data for forest inventories. Gathering as much on-the-ground data as possible is required to ensure the transparency of carbon credit projects. Access to high-quality datasets of manual measurements helps improve biomass models which are widely used in the field of ecological simulation. The code used in this paper will be publicly available on Github and the dataset on Kaggle.

Authors: Margaux Masson-Forsythe (Earthshot Labs); Margaux Masson-Forsythe (Earthshot Labs)
ICLR 2023 BurnMD: A Fire Projection and Mitigation Modeling Dataset (Papers Track)
Abstract and authors: (click to expand)

Abstract: Today's fire projection modeling tools struggle to keep up with the rapid rate and increasing severity of climate change, leaving disaster managers dependent on tools which are increasingly unrepresentative of complex interactions between fire behavior, environmental conditions, and various mitigation options. This has consequences for equitably minimizing wildfire risks to life, property, ecology, cultural heritage, and public health. Fortunately, decades of data exist for fuel populations, weather conditions, and outcomes of significant fires in the American West and globally. The fire management community faces a lack of data standardization and validation among many competing fire models. Likewise, the machine learning community lacks curated datasets and benchmarks to develop solutions necessary to generate impact in this space. We present a novel dataset composed of 308 medium sized fires from the years 2018-2021, complete with both time series airborne based inference and ground operational estimation of fire extent, and operational mitigation data such as control line construction. As the first large wildfire dataset with mitigation information, Burn Mitigation Dataset (BurnMD) will help advance fire projection modeling, fire risk modeling, and AI generated land management policies.

Authors: Marissa Dotter (MITRE Corporation)
ICLR 2023 A simplified machine learning based wildfire ignition model from insurance perspective (Papers Track)
Abstract and authors: (click to expand)

Abstract: In the context of climate change, wildfires are becoming more frequent, intense, and prolonged in the western US, particularly in California. Wildfires cause catastrophic socio-economic losses and are projected to worsen in the near future. Inaccurate estimates of fire risk put further pressure on wildfire (re)insurance and cause many homes to lose wildfire insurance coverage. Efficient and effective prediction of fire ignition is one step towards better fire risk assessment. Here we present a simplified machine learning-based fire ignition model at yearly scale that is well suited to the use case of one-year term wildfire (re)insurance. Our model yields a recall, precision, and the area under the precision-recall curve of 0.69, 0.86 and 0.81, respectively, for California, and significantly higher values of 0.82, 0.90 and 0.90, respectively, for the populated area, indicating its good performance. In addition, our model feature analysis reveals that power line density, enhanced vegetation index (EVI), vegetation optical depth (VOD), and distance to the wildland-urban interface stand out as the most important features determining ignitions. The framework of this simplified ignition model could easily be applied to other regions or genesis of other perils like hurricane, and it paves the road to a broader and more affordable safety net for homeowners.

Authors: Yaling Liu (OurKettle Inc); Son Le (OurKettle Inc.); Yufei Zou (Our Kettle, Inc.); mojtaba Sadgedhi (OurKettle Inc.); Yang Chen (University of California, Irvine); Niels Andela (BeZero Carbon); Pierre Gentine (Columbia University)
ICLR 2023 Nested Fourier Neural Operator for Basin-Scale 4D CO2 Storage Modeling (Papers Track)
Abstract and authors: (click to expand)

Abstract: Carbon capture and storage (CCS) plays an essential role in global decarbonization. Scaling up CCS requires accurate and high-resolution modeling of the storage reservoir pressure buildup and the gaseous plume migration. However, such modeling is very challenging at scale due to the high computational costs of existing numerical methods. This challenge leads to significant uncertainty in evaluating storage opportunities which can delay the pace of global CCS deployments. We introduce a machine-learning approach for dynamic basin-scale modeling that speeds up flow prediction nearly 700,000 times compared to existing methods. Our framework, Nested Fourier Neural Operator (FNO), provides a general-purpose simulator alternative under diverse reservoir conditions, geological heterogeneity, and injection schemes. It enables unprecedented real-time high-fidelity modeling to support decision-making in basin-scale CCS projects.

Authors: Gege Wen (Stanford University)
ICLR 2023 SEA LEVEL PROJECTIONS WITH MACHINE LEARNING USING ALTIMETRY AND CLIMATE MODEL ENSEMBLES (Papers Track)
Abstract and authors: (click to expand)

Abstract: Satellite altimeter observations retrieved since 1993 show that the global mean sea level is rising at an unprecedented rate (3.4mm/year). With almost three decades of observations, we can now investigate the contributions of anthropogenic climate-change signals such as greenhouse gases, aerosols, and biomass burning in this rising sea level. We use machine learning (ML) to investigate future patterns of sea level change. To understand the extent of contributions from the climate-change signals, and to help in forecasting sea level change in the future, we turn to climate model simulations. This work presents a machine learning framework that exploits both satellite observations and climate model simulations to generate sea level rise projections at a 2-degree resolution spatial grid, 30 years into the future. We train fully connected neural networks (FCNNs) to predict altimeter values through a non-linear fusion of the climate model hindcasts (for 1993-2019). The learned FCNNs are then applied to future climate model projections to predict future sea level patterns. We propose segmenting our spatial dataset into meaningful clusters and show that clustering helps to improve predictions of our ML model.

Authors: Saumya Sinha (University of Colorado, Boulder); John Fasullo (NCAR); R. Steven Nerem (Univesity of Colorado, Boulder); Claire Monteleoni (University of Colorado Boulder)
ICLR 2023 A High-Resolution, Data-Driven Model of Urban Carbon Emissions (Papers Track) Best Pathway to Impact
Abstract and authors: (click to expand)

Abstract: Cities represent both a fundamental contributor to greenhouse (GHG) emissions and a catalyst for climate action. Many global cities have outlined sustainability and climate change mitigation plans, focusing on energy efficiency, shifting away from fossil fuels, and prioritizing environmental and social justice. To achieve broad-based and equitable carbon emissions reductions and sustainability goals, new data-driven methodologies are needed to identify and target efficiency and carbon reduction opportunities in the built environment at the building, neighborhood, and city-scale. Our methodology integrates data from numerous data sources and develops data-driven and physical models of energy use and carbon emissions from buildings and transportation to generate a high spatiotemporal resolution model of urban greenhouse gas emissions. The method and data tool are designed to support city leaders and urban policymakers with an unprecedented view of localized carbon emissions to enable data-driven and evidenced-based climate action.

Authors: Bartosz Bonczak (New York University); Boyeong Hong (New York University); Constantine E. Kontokosta (New York University)
ICLR 2023 ClimaX: A foundation model for weather and climate (Papers Track)
Abstract and authors: (click to expand)

Abstract: Recent data-driven approaches based on machine learning aim to directly solve a downstream forecasting or projection task by learning a data-driven functional mapping using deep neural networks. However, these networks are trained using curated and homogeneous climate datasets for specific spatiotemporal tasks, and thus lack the generality of currently used physics-informed numerical models for weather and climate modeling. We develop and demonstrate ClimaX, a flexible and generalizable deep learning model for weather and climate science that can be trained using heterogeneous datasets spanning different variables, spatiotemporal coverage, and physical groundings. ClimaX extends the Transformer architecture with novel encoding and aggregation blocks that allow effective use of available compute and data while maintaining general utility. ClimaX is pre-trained with a self-supervised learning objective on climate datasets derived from CMIP6. The pretrained ClimaX can then be fine-tuned to address a breadth of climate and weather tasks, including those that involve atmospheric variables and spatiotemporal scales unseen during pretraining. Compared to existing data-driven baselines, we show that this generality in ClimaX results in superior performance on benchmarks for weather forecasting and climate projections.

Authors: Tung Nguyen (University of California, Los Angeles); Johannes Brandstetter (Microsoft Research); Ashish Kapoor (Microsoft); Jayesh Gupta (Microsoft Research); Aditya Grover (UCLA)
ICLR 2023 Sub-seasonal to seasonal forecasts through self-supervised learning (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Sub-seasonal to seasonal (S2S) weather forecasts are an important decision- making tool that informs economical and logistical planning in agriculture, energy management, and disaster mitigation. They are issued on time scales of weeks to months and differ from short-term weather forecasts in two important ways: (i) the dynamics of the atmosphere on these timescales can be described only statistically and (ii) these dynamics are characterized by large-scale phenomena in both space and time. While deep learning (DL) has shown promising results in short-term weather forecasting, DL-based S2S forecasts are challenged by comparatively small volumes of available training data and large fluctuations in predictability due to atmospheric conditions. In order to develop more reliable S2S predictions that leverage current advances in DL, we propose to utilize the masked auto-encoder (MAE) framework to learn generic representations of large-scale atmospheric phenomena from high resolution global data. Besides exploring the suitability of the learned representations for S2S forecasting, we will also examine whether they account for climatic phenomena (e.g., the Madden-Julian Oscillation) that are known to increase predictability on S2S timescales.

Authors: Jannik Thuemmel (University of Tuebingen); Felix Strnad (Potsdam Institute for Climate Impact Research); Jakob Schlör (Eberhard Karls Universität Tübingen); Martin V. Butz (University of Tübingen); Bedartha Goswami (University of Tübingen)
ICLR 2023 Graph-Based Deep Learning for Sea Surface Temperature Forecasts (Papers Track)
Abstract and authors: (click to expand)

Abstract: Sea surface temperature (SST) forecasts help with managing the marine ecosystem and the aquaculture impacted by anthropogenic climate change. Numerical dynamical models are resource intensive for SST forecasts; machine learning (ML) models could reduce high computational requirements and have been in the focus of the research community recently. ML models normally require a large amount of data for training. Environmental data are collected on regularly-spaced grids, so early work mainly used grid-based deep learning (DL) for prediction. However, both grid data and the corresponding DL approaches have inherent problems. As geometric DL has emerged, graphs as a more generalized data structure and graph neural networks (GNNs) have been introduced to the spatiotemporal domains. In this work, we preliminarily explored graph re-sampling and GNNs for global SST forecasts, and GNNs show better one month ahead SST prediction than the persistence model in most oceans in terms of root mean square errors.

Authors: Ding Ning (University of Canterbury); Varvara Vetrova (University of Canterbury); Karin Bryan (University of Waikato)
ICLR 2023 DiffESM: Conditional Emulation of Earth System Models with Diffusion Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: Earth System Models (ESMs) are essential tools for understanding the impact of human actions on Earth's climate. One key application of these models is studying extreme weather events, such as heat waves or dry spells, which have significant socioeconomic and environmental consequences. However, the computational demands of running a sufficient number of simulations to analyze the risks are often prohibitive. In this paper we demonstrate that diffusion models -- a class of generative deep learning models -- can effectively emulate the spatio-temporal trends of ESMs under previously unseen climate scenarios, while only requiring a small fraction of the computational resources. We present a diffusion model that is conditioned on monthly averages of temperature or precipitation on a 96x96 global grid, and produces daily values that are both realistic and consistent with those averages. Our results show that the output from our diffusion model closely matches the spatio-temporal behavior of the ESM it emulates in terms of the frequency of phenomena such as heat waves, dry spells, or rainfall intensity.

Authors: Seth Bassetti (Western Washington University); Brian Hutchinson (Western Washington University); Claudia Tebaldi (Joint Global Change Research Institute); Ben Kravitz (Indiana University)
ICLR 2023 Deep ensembles to improve uncertainty quantification of statistical downscaling models under climate change conditions (Papers Track)
Abstract and authors: (click to expand)

Abstract: Recently, deep learning has emerged as a promising tool for statistical downscaling, the set of methods for generating high-resolution climate fields from coarse low-resolution variables. Nevertheless, their ability to generalize to climate change conditions remains questionable, mainly due to the stationarity assumption. We propose deep ensembles as a simple method to improve the uncertainty quantification of statistical downscaling models. By better capturing uncertainty, statistical downscaling models allow for superior planning against extreme weather events, a source of various negative social and economic impacts. Since no observational future data exists, we rely on a pseudo reality experiment to assess the suitability of deep ensembles for quantifying the uncertainty of climate change projections. Deep ensembles allow for a better risk assessment, highly demanded by sectoral applications to tackle climate change.

Authors: Jose González-Abad (Instituto de Fı́sica de Cantabria (IFCA), CSIC-Universidad de Cantabria); Jorge Baño-Medina (Institute of Physics of Cantabria)
ICLR 2023 Bayesian Inference of Severe Hail in Australia (Papers Track)
Abstract and authors: (click to expand)

Abstract: Severe hailstorms are responsible for some of the most costly insured weather events in Australia and can cause significant damage to homes, businesses, and agriculture. However their response to climate change remains uncertain, in large part due to the challenges of observing severe hailstorms. We propose a novel Bayesian approach which explicitly models known biases and uncertainties of current hail observations to produce more realistic estimates of severe hail risk from existing observations. Training this model on data from south-east Queensland, Australia, suggests that previous analyses of severe hail that did not account for this uncertainty may produce poorly calibrated risk estimates. Preliminary evaluation on withheld data confirms that our model produces well-calibrated probabilities and is applicable out of sample. Whilst developed for hail, we highlight also the generality of our model and its potential applications to other severe weather phenomena and areas of climate change adaptation and mitigation.

Authors: Isabelle C Greco (University of New South Wales); Steven Sherwood (University of New South Wales); Timothy Raupach (University of New South Wales); Gab Abramowitz (University of New South Wales)
ICLR 2023 Modelling Atmospheric Dynamics with Spherical Fourier Neural Operators (Papers Track)
Abstract and authors: (click to expand)

Abstract: Fourier Neural Operators (FNOs) have established themselves as an efficient method for learning resolution-independent operators in a wide range of scientific machine learning applications. This can be attributed to their ability to effectively model long-range dependencies in spatio-temporal data through computationally ef- ficient global convolutions. However, the use of discrete Fourier transforms (DFTs) in FNOs leads to spurious artifacts and pronounced dissipation when applied to spherical coordinates, due to the incorrect assumption of flat geometry. To ad- dress the issue, we introduce Spherical FNOs (SFNOs), which use the generalized Fourier transform for learning operators on spherical geometries. We demonstrate the effectiveness of the method for forecasting atmospheric dynamics, producing stable auto-regressive results for a simulated time of one year (1,460 steps) while retaining physically plausible dynamics. This development has significant implica- tions for machine learning-based climate dynamics emulation, which could play a crucial role in accelerating our response to climate change.

Authors: Boris Bonev (NVIDIA); Thorsten Kurth (Nvidia); Christian Hundt (NVIDIA AI Technology Center); Jaideep Pathak (NVIDIA Corporation); Maximilian Baust (NVIDIA); Karthik Kashinath (NVIDIA); Anima Anandkumar (NVIDIA/Caltech)
ICLR 2023 Uncovering the Spatial and Temporal Variability of Wind Resources in Europe: A Web-Based Data-Mining Tool (Papers Track)
Abstract and authors: (click to expand)

Abstract: We introduce REmap-eu.app, a web-based data-mining visualization tool of the spatial and temporal variability of wind resources. It uses the latest open-access dataset of the daily wind capacity factor in 28 European countries between 1979 and 2019 and proposes several user-configurable visualizations of the temporal and spatial variations of the wind power capacity factor. The platform allows for a deep analysis of the distribution, the cross-country correlation, and the drivers of low wind power events. It offers an easy-to-use interface that makes it suitable for the needs of researchers and stakeholders. The tool is expected to be useful in identifying areas of high wind potential and possible challenges that may impact the large-scale deployment of wind turbines in Europe. Particular importance is given to the visualization of low wind power events and to the potential of cross-border cooperations in mitigating the variability of wind in the context of increasing reliance on weather-sensitive renewable energy sources.

Authors: Alban Puech (École Polytechnique); Jesse Read (Ecole Polytechnique)
ICLR 2023 Understanding forest resilience to drought with Shapley values (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Increases in drought frequency, intensity, and duration due to climate change are threatening forests around the world. Climate-driven tree mortality is associated with devastating ecological and societal consequences, including the loss of carbon sequestration, habitat provisioning, and water filtration services. A spatially fine-grained understanding of the site characteristics making forests more resilient to drought is still lacking. Furthermore, the complexity of drought effects on forests, which can be cumulative and delayed, demands investigation of the most appropriate drought indices. In this study, we aim to gain a better understanding of the temporal and spatial drivers of drought-induced changes in forest vitality using Shapley values, which allow for the relevance of predictors to be quantified locally. A better understanding of the contribution of meteorological and environmental factors to trees’ response to drought can support forest managers aiming to make forests more climate-resilient.

Authors: Stenka Vulova (Technische Universität Berlin); Alby Duarte Rocha (Technische Universität Berlin); Akpona Okujeni (Humboldt-Universität zu Berlin); Johannes Vogel (Freie Universität Berlin); Michael Förster (Technische Universität Berlin); Patrick Hostert (Humboldt-Universität zu Berlin); Birgit Kleinschmit (Technische Universität Berlin)
ICLR 2023 Improving the spatial accuracy of extreme tropical cyclone rainfall in ERA5 using deep learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: We propose a novel method for the bias adjustment and post-processing of gridded rainfall data products. Our method uses U-Net (a deep convolutional neural network) as a backbone, and a novel loss function given by the combination of a pixelwise bias component (Mean Absolute Error) and a spatial accuracy component (Fractions Skill Score). We evaluate the proposed approach by adjusting extreme rainfall from the popular ERA5 reanalysis dataset, using the multi-source observational dataset MSWEP as a target. We focus on a sample of extreme rainfall events induced by tropical cyclones and show that the proposed method significantly reduces both the MAE (by 16\%) and FSS (by 53\%) of ERA5.

Authors: Guido Ascenso (Politecnico di Milano); Andrea Ficchì (Politecnico di Milano); Matteo Giuliani (Politecnico di Milano); Leone Cavicchia (Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici (CMCC)); Enrico Scoccimarro (Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici (CMCC)); Andrea Castelletti (Politecnico di Milano)
ICLR 2023 Robustly modeling the nonlinear impact of climate change on agriculture by combining econometrics and machine learning (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Climate change is expected to have a dramatic impact on agricultural production; however, due to natural complexity, the exact avenues and relative strengths by which this will happen are still unknown. The development of accurate forecasting models is thus of great importance to enable policy makers to design effective interventions. To date, most machine learning methods aimed at tackling this problem lack a consideration of causal structure, thereby making them unreliable for the types of counterfactual analysis necessary when making policy decisions. Econometrics has developed robust techniques for estimating cause-effect relations in time-series, specifically through the use of cointegration analysis and Granger causality. However, these methods are frequently limited in flexibility, especially in the estimation of nonlinear relationships. In this work, we propose to integrate the non-linear function approximators with the robust causal estimation methods to ultimately develop an accurate agricultural forecasting model capable of robust counterfactual analysis. This method would be a valuable new asset for government and industrial stakeholders to understand how climate change impacts agricultural production.

Authors: Benedetta Francesconi (Independent Researcher); Ying-Jung C Deweese (Descartes Labs / Georgia Insititute of Technology)
ICLR 2023 EfficientTempNet: Temporal Super-Resolution of Radar Rainfall (Papers Track)
Abstract and authors: (click to expand)

Abstract: Rainfall data collected by various remote sensing instruments such as radars or satellites has different space-time resolutions. This study aims to improve the temporal resolution of radar rainfall products to help with more accurate climate change modeling and studies. In this direction, we introduce a solution based on EfficientNetV2, namely EfficientTempNet, to increase the temporal resolution of radar-based rainfall products from 10 minutes to 5 minutes. We tested EfficientRainNet over a dataset for the state of Iowa, US, and compared its performance to three different baselines to show that EfficientTempNet presents a viable option for better climate change monitoring.

Authors: Bekir Z Demiray (University of Iowa); Muhammed A Sit (The University of Iowa); Ibrahim Demir (University of Iowa)
ICLR 2023 Fourier Neural Operators for Arbitrary Resolution Climate Data Downscaling (Papers Track)
Abstract and authors: (click to expand)

Abstract: Running climate simulations informs us of future climate change. However, it is computationally expensive to resolve complex climate processes numerically. As one way to speed up climate simulations, neural networks have been used to downscale climate variables from fast-running low-resolution simulations. So far, all neural network downscaling models can only downscale input samples with a pre-defined upsampling factor. In this work, we propose a Fourier neural operator downscaling model. It trains with data of a small upsampling factor and then can zero-shot downscale its input to arbitrary unseen high-resolutions. Evaluated on Navier-Stokes equation solution data and ERA5 water content data, our downscaling model demonstrates better performance than widely used convolutional and adversarial generative super-resolution models in both learned and zero-shot downscaling. Our model's performance is further boosted when a constraint layer is applied. In the end, we show that by combining our downscaling model with a low-resolution numerical PDE solver, the downscaled solution outperforms the solution of the state-of-the-art high-resolution data-driven solver. Our model can be used to cheaply and accurately generate arbitrarily high-resolution climate simulation data with fast-running low-resolution simulation as input.

Authors: Qidong Yang (New York University); Paula Harder (Fraunhofer ITWM); Venkatesh Ramesh (University of Montreal, Mila); Alex Hernandez-Garcia (Mila - Quebec AI Institute); Daniela Szwarcman (IBM Research); Prasanna Sattigeri (IBM Research); Campbell D Watson (IBM Reserch); David Rolnick (McGill University, Mila)
ICLR 2023 Data-driven multiscale modeling of subgrid parameterizations in climate models (Papers Track) Best ML Innovation
Abstract and authors: (click to expand)

Abstract: Subgrid parameterizations that represent physical processes occurring below the resolution of current climate models are an important component in producing accurate, long-term predictions for the climate. A variety of approaches have been tested to design these components, including deep learning methods. In this work, we evaluate a proof of concept illustrating a multiscale approach to this prediction problem. We train neural networks to predict subgrid forcing values on a testbed model and examine improvements in prediction accuracy which can be obtained by using additional information in both fine-to-coarse and coarse-to-fine directions.

Authors: Karl Otness (New York University); Laure Zanna (NYU); Joan Bruna (NYU)
ICLR 2023 On the impact of small-data diversity on forecasts: evidence from meteorologically-driven electricity demand in Mediterranean zones. (Papers Track)
Abstract and authors: (click to expand)

Abstract: In this paper, we compare the improvement of probabilistic electricity demand forecasts for three specific coastal and island regions using raw and pre-computed meteorological features based on empirically-tested formulations drawn from climate science literature. Typically for the general task of time-series forecasting with strong weather/climate drivers, go-to models like the Autoregressive Integrated Moving Average (ARIMA) model are built with assumptions of how independent variables will affect a dependent one and are at best encoded with a handful of exogenous features with known impact. Depending on the geographical region and/or cultural practices of a population, such a selection process may yield a non-optimal feature set which would ultimately drive a weak impact on underline demand forecasts. The aim of this work is to assess the impact of a documented set of meteorological features on electricity demand using deep learning models in comparative studies. Leveraging the defining computational architecture of the Temporal Fusion Transformer (TFT), we discover the unimportance of weather features for improving probabilistic forecasts for the targeted regions. However, through experimentation, we discover that the more stable electricity demand of the coastal Mediterranean regions, the Ceuta and Melilla autonomous cities in Morocco, improved the forecast accuracy of the strongly tourist-driven electricity demand for the Balearic islands located in Spain during the time of travel restrictions (i.e., during COVID19 (2020))--a root mean squared error (RMSE) from ~0.090 to ~0.012 with a substantially improved 10th/90th quantile bounding.

Authors: Reginald Bryant (IBM Research - Africa); Julian Kuehnert (IBM Research)
ICLR 2023 Artificial Intelligence in Tropical Cyclone Forecasting (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Tropical cyclones (TC) in Bangladesh and other developing nations harm property and human lives. Theoretically, artificial intelligence (AI) can anticipate TC using tracking, intensity, and cyclone aftereffect phenomena. Although AI has a significant impact on predicting, poorer nations have struggled to adjust to its real-world applications. The interpretability of such a solution from an AI-based solution is the main factor in that situation, together with the infrastructure. Explainable AI has been extensively employed in the medical field because the outcome is so important. We believe that using explainable AI in TC forecasting is equally important as one large forecast can cause the thought of life loss. Additionally, it will improve the long-term viability of the AI-based weather forecasting system. To be more specific, we want to employ explainable AI in every way feasible in this study to address the problems of TC forecasting, intensity estimate, and tracking. We'll do this by using the graph neural network. The adoption of AI-based solutions in underdeveloped nations will be aided by this solution, which will boost their acceptance. With this effort, we also hope to tackle the challenge of sustainable AI in order to tackle the issue of climate change on a larger scale. However, Cyclone forecasting might be transformed by sustainable AI, guaranteeing precise and early predictions to lessen the effects of these deadly storms. The examination of vast volumes of meteorological data to increase forecasting accuracy is made possible by the combination of AI algorithms and cutting-edge technologies like machine learning and big data analytics. Improved accuracy is one of the main advantages of sustainable AI for cyclone prediction. To provide more precise forecasts, AI systems can evaluate a lot of meteorological data, including satellite imagery and ocean temperature readings.

Authors: Dr. Nusrat Sharmin (Military Institute of Science and Technology); Professor Dr. Md. Mahbubur Rahman Rahman (Military Institute of Science and Technology (MIST)); Sabbir Rahman (Military Institute of Science and Technology); Mokhlesur Rahman (Military Institute of Science and Technology)
NeurIPS 2022 Bayesian inference for aerosol vertical profiles (Papers Track)
Abstract and authors: (click to expand)

Abstract: Aerosol-cloud interactions constitute the largest source of uncertainty in assessments of the anthropogenic climate change. This uncertainty arises in part from the difficulty in measuring the vertical distributions of aerosols. We often have to settle for less informative vertically aggregated proxies such as aerosol optical depth (AOD). In this work, we develop a framework to infer vertical aerosol profiles using AOD and readily available vertically resolved meteorological predictors such as temperature or relative humidity. We devise a simple Gaussian process prior over aerosol vertical profiles and update it with AOD observations. We validate our approach using ECHAM-HAM aerosol-climate model data. Our results show that, while simple, our model is able to reconstruct realistic extinction profiles with well-calibrated uncertainty. In particular, the model demonstrates a faithful reconstruction of extinction patterns arising from aerosol water uptake in the boundary layer.

Authors: Shahine Bouabid (University of Oxford); Duncan Watson-Parris (University of Oxford); Dino Sejdinovic (University of Adelaide)
NeurIPS 2022 Attention-Based Scattering Network for Satellite Imagery (Papers Track)
Abstract and authors: (click to expand)

Abstract: Multi-channel satellite imagery, from stacked spectral bands or spatiotemporal data, have meaningful representations for various atmospheric properties. Combining these features in an effective manner to create a performant and trustworthy model is of utmost importance to forecasters. Neural networks show promise, yet suffer from unintuitive computations, fusion of high-level features, and may be limited by the quantity of available data. In this work, we leverage the scattering transform to extract high-level features without additional trainable parameters and introduce a separation scheme to bring attention to independent input channels. Experiments show promising results on estimating tropical cyclone intensity and predicting the occurrence of lightning from satellite imagery.

Authors: Jason Stock (Colorado State University); Charles Anderson (Colorado State University)
NeurIPS 2022 Discovering Interpretable Structural Model Errors in Climate Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: Inaccuracies in the models of the Earth system, i.e., structural and parametric model errors, lead to inaccurate climate change projections. Errors in the model can originate from unresolved phenomena due to a low numerical resolution, as well as misrepresentations of physical phenomena or boundaries (e.g., orography). Therefore, such models lead to inaccurate short--term forecasts of weather and extreme events, and more importantly, long term climate projections. While calibration methods have been introduced to address for parametric uncertainties, e.g., by better estimation of system parameters from observations, addressing structural uncertainties, especially in an interpretable manner, remains a major challenge. Therefore, with increases in both the amount and frequency of observations of the Earth system, algorithmic innovations are required to identify interpretable representations of the model errors from observations. We introduce a flexible, general-purpose framework to discover interpretable model errors, and show its performance on a canonical prototype of geophysical turbulence, the two--level quasi--geostrophic system. Accordingly, a Bayesian sparsity--promoting regression framework is proposed, that uses a library of kernels for discovery of model errors. As calculating the library from noisy and sparse data (e.g., from observations) using convectional techniques leads to interpolation errors, here we use a coordinate-based multi--layer embedding to impute the sparse observations. We demonstrate the importance of alleviating spectral bias, and propose a random Fourier feature layer to reduce it in the proposed embeddings, and subsequently enable an accurate discovery. Our framework is demonstrated to successfully identify structural model errors due to linear and nonlinear processes (e.g., radiation, surface friction, advection), as well as misrepresented orography.

Authors: Rambod Mojgani (Rice University); Ashesh K Chattopadhyay (Rice University); Pedram Hassanzadeh (Rice University)
NeurIPS 2022 Improving the predictions of ML-corrected climate models with novelty detection (Papers Track)
Abstract and authors: (click to expand)

Abstract: While previous works have shown that machine learning (ML) can improve the prediction accuracy of coarse-grid climate models, these ML-augmented methods are more vulnerable to irregular inputs than the traditional physics-based models they rely on. Because ML-predicted corrections feed back into the climate model’s base physics, the ML-corrected model regularly produces out of sample data, which can cause model instability and frequent crashes. This work shows that adding semi-supervised novelty detection to identify out-of-sample data and disable the ML-correction accordingly stabilizes simulations and sharply improves the quality of predictions. We design an augmented climate model with a one-class support vector machine (OCSVM) novelty detector that provides better temperature and precipitation forecasts in a year-long simulation than either a baseline (no-ML) or a standard ML-corrected run. By improving the accuracy of coarse-grid climate models, this work helps make accurate climate models accessible to researchers without massive computational resources.

Authors: Clayton H Sanford (Columbia); Anna Kwa (Allen Institute for Artificial Intelligence); Oliver Watt-Meyer (Allen Institute for AI); Spencer Clark (Allen Institute for AI); Noah Brenowitz (Allen Institute for AI); Jeremy McGibbon (Allen Institute for AI); Christopher Bretherton (Allen Institute for AI)
NeurIPS 2022 Deep learning for downscaling tropical cyclone rainfall (Papers Track)
Abstract and authors: (click to expand)

Abstract: Flooding is often the leading cause of mortality and damages from tropical cyclones. With rainfall from tropical cyclones set to rise under global warming, better estimates of extreme rainfall are required to better support resilience efforts. While high resolution climate models capture tropical cyclone statistics well, they are computationally expensive leading to a trade-off between accuracy and generating enough ensemble members to generate sufficient high impact, low probability events. Often, downscaling models are used as a computationally cheaper alternative. Here, we develop and evaluate a set of deep learning models for downscaling tropical cyclone rainfall for more robust risk analysis.

Authors: Emily Vosper (University of Bristol); Lucy Harris (University of Oxford); Andrew McRae (University of Oxford); Laurence Aitchison (University of Bristol); Peter Watson (Bristol); Raul Santos Rodriguez (University of Bristol); Dann Mitchell (University of Bristol)
NeurIPS 2022 Short-term Prediction and Filtering of Solar Power Using State-Space Gaussian Processes (Papers Track)
Abstract and authors: (click to expand)

Abstract: Short-term forecasting of solar photovoltaic energy (PV) production is important for powerplant management. Ideally these forecasts are equipped with error bars, so that downstream decisions can account for uncertainty. To produce predictions with error bars in this setting, we consider Gaussian processes (GPs) for modelling and predicting solar photovoltaic energy production in the UK. A standard application of GP regression on the PV timeseries data is infeasible due to the large data size and non-Gaussianity of PV readings. However, this is made possible by leveraging recent advances in scalable GP inference, in particular, by using the state-space form of GPs, combined with modern variational inference techniques. The resulting model is not only scalable to large datasets but can also handle continuous data streams via Kalman filtering.

Authors: So Takao (UCL); Sean Nassimiha (UCL); Peter Dudfield (Open Climate Fix); Jack Kelly (Open Climate Fix); Marc Deisenroth (University College London)
NeurIPS 2022 Identifying latent climate signals using sparse hierarchical Gaussian processes (Papers Track)
Abstract and authors: (click to expand)

Abstract: Extracting latent climate signals from multiple climate model simulations is important to estimate future climate change. To tackle this we develop a sparse hierarchical Gaussian process (SHGP), which probabilistically learns a latent distribution from a set of vectors. We use this to predict the latent surface temperature change globally and for central England from an ensemble of climate models, in a scalable manner and with robust uncertainty propagation.

Authors: Matt Amos (Lancaster University); Thomas Pinder (Lancaster University); Paul Young (Lancaster University)
NeurIPS 2022 Detecting Methane Plumes using PRISMA: Deep Learning Model and Data Augmentation (Papers Track)
Abstract and authors: (click to expand)

Abstract: The new generation of hyperspectral imagers, such as PRISMA, has improved significantly our detection capability of methane (CH4) plumes from space at high spatial resolution (∼30m). We present here a complete framework to identify CH4 plumes using images from the PRISMA satellite mission and a deep learning technique able to automatically detect plumes over large areas. To compensate for the sparse database of PRISMA images, we trained our model by transposing high resolution plumes from Sentinel-2 to PRISMA. Our methodology avoids computationally expensive synthetic plume from Large Eddy Simulations while generating a broad and realistic training database, and paves the way for large-scale detection of methane plumes using future hyperspectral sensors (EnMAP, EMIT, CarbonMapper).

Authors: Alexis Groshenry (Kayrros); Clément Giron (Kayrros); Alexandre d'Aspremont (CNRS, DI, Ecole Normale Supérieure; Kayrros); Thomas Lauvaux (University of Reims Champagne Ardenne, GSMA, UMR 7331); Thibaud Ehret (Centre Borelli)
NeurIPS 2022 Deep Hydrology: Hourly, Gap-Free Flood Maps Through Joint Satellite and Hydrologic Modelling (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change-driven weather disasters are rapidly increasing in both frequency and magnitude. Floods are the most damaging of these disasters, with approximately 1.46 billion people exposed to inundation depths of over 0.15m, a significant life and livelihood risk. Accurate knowledge of flood-extent for ongoing and historical events facilitates climate adaptation in flood-prone communities by enabling near real-time disaster monitoring to support planning, response, and relief during these extreme events. Satellite observations can be used to derive flood-extent maps directly; however, these observations are impeded by cloud and canopy cover, and can be very infrequent and hence miss the flood completely. In contrast, physically-based inundation models can produce spatially complete event maps but suffer from high uncertainty if not frequently calibrated with expensive land and infrastructure surveys. In this study, we propose a deep learning approach to reproduce satellite-observed fractional flood-extent maps given dynamic state variables from hydrologic models, fusing information contained within the states with direct observations from satellites. Our model has an hourly temporal resolution, contains no cloud-gaps, and generalizes to watersheds across the continental United States with a 6% error on held-out areas that never flooded before. We further demonstrate through a case study in Houston, Texas that our model can distinguish tropical cyclones that caused flooding from those that did not within two days of landfall, thereby providing a reliable source for flood-extent maps that can be used by disaster monitoring services.

Authors: Tanya Nair (Cloud To Street); Veda Sunkara (Cloud to Street); Jonathan Frame (Cloud to Street); Philip Popien (Cloud to Street); Subit Chakrabarti (Cloud To Street)
NeurIPS 2022 Machine learning emulation of a local-scale UK climate model (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change is causing the intensification of rainfall extremes. Precipitation projections with high spatial resolution are important for society to prepare for these changes, e.g. to model flooding impacts. Physics-based simulations for creating such projections are very computationally expensive. This work demonstrates the effectiveness of diffusion models, a form of deep generative models, for generating much more cheaply realistic high resolution rainfall samples for the UK conditioned on data from a low resolution simulation. We show for the first time a machine learning model that is able to produce realistic high-resolution rainfall predictions based on a physical model that resolves atmospheric convection, a key process behind extreme rainfall. By adding self-learnt, location-specific information to low resolution relative vorticity, quantiles and time-mean of the samples match well their counterparts from the high-resolution simulation.

Authors: Henry Addison (University of Bristol); Elizabeth Kendon (Met Office Hadley Centre); Suman Ravuri (DeepMind); Laurence Aitchison (University of Bristol); Peter Watson (Bristol)
NeurIPS 2022 Bridging the Microwave Data Gap; Using Bayesian Deep Learning to “See” the Unseen (Papers Track)
Abstract and authors: (click to expand)

Abstract: Having microwave data with the spatial and temporal resolution of infrared data would provide a large positive impact on many climate and weather applications. We demonstrate that Bayesian deep learning is a promising technique for both creating and improving synthetic microwave data from infrared data. We report 0.7% mean absolute percentage error for 183+/-3 GHz microwave brightness temperature and uncertainty metrics and find that more training data is needed to achieve improved performance at 166 GHz, 37 GHz, and 23 GHz. Analysis of the spatial distribution of uncertainty reveals that additional cloud data will provide the greatest increase in skill, which will potentially allow for generation of many secondary products derived from microwave data in the future.

Authors: Pedro Ortiz (Naval Postgraduate School); Eleanor Casas (Naval Postgraduate School); Marko Orescanin (Naval Postgraduate School); Scott Powell (Naval Postgraduate School)
NeurIPS 2022 Learning evapotranspiration dataset corrections from water cycle closure supervision (Papers Track)
Abstract and authors: (click to expand)

Abstract: Evapotranspiration (ET) is one of the most uncertain components of the global water cycle. Improving global ET estimates is needed to better our understanding of the global water cycle so as to forecast the consequences of climate change on the future of global water resource distribution. This work presents a methodology to derive monthly corrections of global ET datasets at 0.25 degree resolution. We use ML to generalize sparse catchment-level water cycle closure residual information to global and dense pixel-level residuals. Our model takes a probabilistic view on ET datasets and their correction that we use to regress catchment-level residuals using a sum-aggregated supervision. Using four global ET datasets, we show that our learned model has learned ET corrections that accurately generalize its water cycle-closure results to unseen catchments.

Authors: Tristan E.M Hascoet (Kobe University); Victor Pellet (LERMA); Filipe Aires (LERMA)
NeurIPS 2022 Optimizing Japanese dam reservoir inflow forecast for efficient operation (Papers Track)
Abstract and authors: (click to expand)

Abstract: Despite a climate and topology favorable to hydropower (HP) generation, HP only accounts for 4% of today’s Japanese primary energy consumption mix. In recent years, calls for improving the efficiency of Japanese HP towards achieving a more sustainable energy mix have emerged from prominent voices in the Ministry of Land, Infrastructure, Transport and Tourism (MILT). Among potential optimizations, data-driven dam operation policies using accurate river discharge forecasts have been advocated for. In the meantime, Machine Learning (ML) has recently made important strides in hydrological modeling, with forecast accuracy improvements demonstrated on both precipitation nowcasting and river discharge prediction. We are motivated by the convergence of these societal and technological contexts: our final goal is to provide scientific evidence and actionable insights for dam infrastructure managers and policy makers to implement more energy-efficient and flood-resistant dam operation policies on a national scale. Towards this goal this work presents a preliminary study of ML-based dam inflow forecasts on a dataset of 127 Japanese public dams we assembled. We discuss our preliminary results and lay out a path for future studies.

Authors: Keisuke Yoshimi (Kobe University); Tristan E.M Hascoet (Kobe University); Rousslan F. Julien Dossa (Kobe University); Ryoichi Takashima (Kobe University); Tetsuya Takiguchi (Kobe University); Satoru Oishi (Kobe University)
NeurIPS 2022 Don't Waste Data: Transfer Learning to Leverage All Data for Machine-Learnt Climate Model Emulation (Papers Track)
Abstract and authors: (click to expand)

Abstract: How can we learn from all available data when training machine-learnt climate models, without incurring any extra cost at simulation time? Typically, the training data comprises coarse-grained high-resolution data. But only keeping this coarse-grained data means the rest of the high-resolution data is thrown out. We use a transfer learning approach, which can be applied to a range of machine learning models, to leverage all the high-resolution data. We use three chaotic systems to show it stabilises training, gives improved generalisation performance and results in better forecasting skill. Our code is at https://github.com/raghul-parthipan/dont_waste_data

Authors: Raghul Parthipan (University of Cambridge); Damon Wischik (Univeristy of Cambridge)
NeurIPS 2022 Towards a spatially transferable super resolution model for downscaling Antarctic surface melt (Papers Track)
Abstract and authors: (click to expand)

Abstract: Surface melt on the Antarctic Ice Sheet is an important climate indicator, yet the spatial scale of modeling and observing surface melt is insufficient to capture crucial details and understand local processes. High-resolution climate models could provide a solution, but they are computationally expensive and require finetuning for some model parameters. An alternative method, pioneering in geophysics, is single-image super resolution (SR) applied on lower-resolution model output. However, often input and output of such SR models are available on the same, fixed spatial domain. High-resolution model simulations over Antarctica are available only in some regions. To be able to apply an SR model elsewhere, we propose to make the single-image SR model physics-aware, using surface albedo and elevation as additional input. Our results show a great improvement in the spatial transferability of the conventional SR model. Although issues with the input satellite-derived albedo remain, adding physics awareness paves a way toward a spatially transferable SR model for downscaling Antarctic surface melt.

Authors: Zhongyang Hu (IMAU); Yao Sun (TUM); Peter Kuipers Munneke (IMAU); Stef Lhermitte (TU Delft); Xiaoxiang Zhu (Technical University of Munich,Germany)
NeurIPS 2022 Forecasting European Ozone Air Pollution With Transformers (Papers Track)
Abstract and authors: (click to expand)

Abstract: Surface ozone is an air pollutant that contributes to hundreds of thousands of premature deaths annually. Accurate short-term ozone forecasts may allow improved policy to reduce the risk to health, such as air quality warnings. However, forecasting ozone is a difficult problem, as surface ozone concentrations are controlled by a number of physical and chemical processes which act on varying timescales. Accounting for these temporal dependencies appropriately is likely to provide more accurate ozone forecasts. We therefore deploy a state-of-the-art transformer-based model, the Temporal Fusion Transformer, trained on observational station data from three European countries. In four-day test forecasts of daily maximum 8-hour ozone, the novel approach is highly skilful (MAE = 4.6 ppb, R2 = 0.82), and generalises well to two European countries unseen during training (MAE = 4.9 ppb, R2 = 0.79). The model outperforms standard machine learning models on our data, and compares favourably to the published performance of other deep learning architectures tested on different data. We illustrate that the model pays attention to physical variables known to control ozone concentrations, and that the attention mechanism allows the model to use relevant days of past ozone concentrations to make accurate forecasts.

Authors: Seb Hickman (University of Cambridge); Paul Griffiths (University of Cambridge); Alex Archibald (University of Cambridge); Peer Nowack (Imperial College London); Elie Alhajjar (USMA)
NeurIPS 2022 Exploring Randomly Wired Neural Networks for Climate Model Emulation (Papers Track)
Abstract and authors: (click to expand)

Abstract: Exploring the climate impacts of various anthropogenic emissions scenarios is key to making informed decisions for climate change mitigation and adaptation. State-of-the-art Earth system models can provide detailed insight into these impacts, but have a large associated computational cost on a per-scenario basis. This large computational burden has driven recent interest in developing cheap machine learning models for the task of climate model emulation. In this manuscript, we explore the efficacy of randomly wired neural networks for this task. We describe how they can be constructed and compare them to their standard feedforward counterparts using the ClimateBench dataset. Specifically, we replace the dense layers in multilayer perceptrons, convolutional neural networks, and convolutional long short-term memory networks with randomly wired ones and assess the impact on model performance for models with 1 million and 10 million parameters. We find average performance improvements of 4.2% across model complexities and prediction tasks, with substantial performance improvements of up to 16.4% in some cases. Furthermore, we find no significant difference in prediction speed between networks with standard feedforward dense layers and those with randomly wired layers. These findings indicate that randomly wired neural networks may be suitable direct replacements for traditional dense layers in many standard models.

Authors: William J Yik (Harvey Mudd College); Sam J Silva (The University of Southern California); Andrew Geiss (Pacific Northwest National Laboratory); Duncan Watson-Parris (University of Oxford)
NeurIPS 2022 AutoML for Climate Change: A Call to Action (Papers Track)
Abstract and authors: (click to expand)

Abstract: The challenge that climate change poses to humanity has spurred a rapidly developing field of artificial intelligence research focused on climate change applications. The climate change ML (CCML) community works on a diverse, challenging set of problems which often involve physics-constrained ML or heterogeneous spatiotemporal data. It would be desirable to use automated machine learning (AutoML) techniques to automatically find high-performing architectures and hyperparameters for a given dataset. In this work, we benchmark popular Auto ML libraries on three high-leverage CCML applications: climate modeling, wind power forecasting, and catalyst discovery. We find that out-of-the-box AutoML libraries currently fail to meaningfully surpass the performance of human-designed CCML models. However, we also identify a few key weaknesses, which stem from the fact that most AutoML techniques are tailored to computer vision and NLP applications. For example, while dozens of search spaces have been designed for image and language data, none have been designed for spatiotemporal data. Addressing these key weaknesses can lead to the discovery of novel architectures that yield substantial performance gains across numerous CCML applications. Therefore, we present a call to action to the AutoML community, since there are a number of concrete, promising directions for future work in the space of AutoML for CCML. We release our code and a list of resources at https://github.com/climate-change-automl/climate-change-automl.

Authors: Renbo Tu (University of Toronto); Nicholas Roberts (University of Wisconsin-Madison); Vishak Prasad C (Indian Institute Of Technology, Bombay); Sibasis Nayak (Indian Institute of Technology, Bombay); Paarth Jain (Indian Institute of Technology Bombay); Frederic Sala (University of Wisconsin-Madison); Ganesh Ramakrishnan (IIT Bombay); Ameet Talwalkar (CMU); Willie Neiswanger (Stanford University); Colin White (Abacus.AI)
NeurIPS 2022 Identifying Compound Climate Drivers of Forest Mortality with β-VAE (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change is expected to lead to higher rates of forest mortality. Forest mortality is a complex phenomenon driven by the interaction of multiple climatic variables at multiple temporal scales, further modulated by the current state of the forest (e.g. age, stem diameter, and leaf area index). Identifying the compound climate drivers of forest mortality would greatly improve understanding and projections of future forest mortality risk. Observation data are, however, limited in accuracy and sample size, particularly in regard to forest state variables and mortality events. In contrast, simulations with state-of-the-art forest models enable the exploration of novel machine learning techniques for associating forest mortality with driving climate conditions. Here we simulate 160,000 years of beech, pine and spruce forest dynamics with the forest model FORMIND. We then apply β-VAE to learn disentangled latent representations of weather conditions and identify those that are most likely to cause high forest mortality. The learned model successfully identifies three characteristic climate representations that can be interpreted as different compound drivers of forest mortality.

Authors: Mohit Anand (Helmholtz Centre for Environmental Research - UFZ); Lily-belle Sweet (Helmholtz Centre for Environmental Research - UFZ); Gustau Camps-Valls (Universitat de València); Jakob Zscheischler (Helmholtz Centre for Environmental Research - UFZ)
NeurIPS 2022 Hybrid Recurrent Neural Network for Drought Monitoring (Papers Track)
Abstract and authors: (click to expand)

Abstract: Droughts are pervasive hydrometeorological phenomena and global hazards, whose frequency and intensity are expected to increase in the context of climate change. Drought monitoring is of paramount relevance. Here we propose a hybrid model for drought detection that integrates both climatic indices and data-driven models in a hybrid deep learning approach. We exploit time-series of multi-scale Standardized Precipitation Evapotranspiration Index together with precipitation and temperature as inputs. We introduce a dual-branch recurrent neural network with convolutional lateral connections for blending the data. Experimental and ablative results show that the proposed system outperforms both the considered drought index and purely data-driven deep learning models. Our results suggest the potential of hybrid models for drought monitoring and open the door to synergistic systems that learn from data and domain knowledge altogether.

Authors: Mengxue Zhang (Universitat de València); Miguel-Ángel Fernández-Torres (Universitat de València); Gustau Camps-Valls (Universitat de València)
NeurIPS 2022 Deep Learning for Global Wildfire Forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change is expected to aggravate wildfire activity through the exacerbation of fire weather. Improving our capabilities to anticipate wildfires on a global scale is of uttermost importance for mitigating their negative effects. In this work, we create a global fire dataset and demonstrate a prototype for predicting the presence of global burned areas on a sub-seasonal scale with the use of segmentation deep learning models. Particularly, we present an open-access global analysis-ready datacube, which contains a variety of variables related to the seasonal and sub-seasonal fire drivers (climate, vegetation, oceanic indices, human-related variables), as well as the historical burned areas and wildfire emissions for 2001-2021. We train a deep learning model, which treats global wildfire forecasting as an image segmentation task and skillfully predicts the presence of burned areas 8, 16, 32 and 64 days ahead of time. Our work motivates the use of deep learning for global burned area forecasting and paves the way towards improved anticipation of global wildfire patterns.

Authors: Ioannis Prapas (National Observatory of Athens); Akanksha Ahuja (NOA); Spyros Kondylatos (National Observatory of Athens); Ilektra Karasante (National Observatory of Athens); Lazaro Alonso (Max Planck Institute for Biogeochemistry); Eleanna Panagiotou (Harokopio University of Athens); Charalampos Davalas (Harokopio University of Athens); Dimitrios Michail (Harokopio University of Athens); Nuno Carvalhais (Max Planck Institute for Biogeochemistry); Ioannis Papoutsis (National Observatory of Athens)
NeurIPS 2022 Comparing the carbon costs and benefits of low-resource solar nowcasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: Mitigating emissions in line with climate goals requires the rapid integration of low carbon energy sources, such as solar photovoltaics (PV) into the electricity grid. However, the energy produced from solar PV fluctuates due to clouds obscuring the sun's energy. Solar PV yield nowcasting is used to help anticipate peaks and troughs in demand to support grid integration. This paper compares multiple low-resource approaches to nowcasting solar PV yield. To do so, we use a dataset of UK satellite imagery and solar PV energy readings over a 1 to 4-hour time range. Our work investigates the performance of multiple nowcasting models. The paper also estimates the carbon emissions generated and averted by deploying models such as these, and finds that short-term PV forecasting may have a benefit several orders of magnitude greater than its carbon cost and that this benefit holds for small models that could be deployable in low-resource settings around the globe.

Authors: Ben Dixon (UCL); Jacob Bieker (Open Climate Fix); Maria Perez-Ortiz (University College London)
NeurIPS 2022 Controllable Generation for Climate Modeling (Papers Track)
Abstract and authors: (click to expand)

Abstract: Recent years have seen increased interest in modeling future climate trends, especially from the point of view of accurately predicting, understanding and mitigating downstream impacts. For instance, current state-of-the-art process-based agriculture models rely on high-resolution climate data during the growing season for accurate estimation of crop yields. However, high-resolution climate data for future climates is unavailable and needs to be simulated, and that too for multiple possible climate scenarios, which becomes prohibitively expensive via traditional methods. Meanwhile, deep generative models leveraging the expressivity of neural networks have shown immense promise in modeling distributions in high dimensions. Here, we cast the problem of simulation of climate scenarios in a generative modeling framework. Specifically, we leverage the GAN (Generative Adversarial Network) framework for simulating synthetic climate scenarios. We condition the model by quantifying the degree of ``extremeness" of the observed sample, which allows us to sample from different parts of the distribution. We demonstrate the efficacy of the proposed method on the CHIRPS precipitation dataset.

Authors: Moulik Choraria (University of Illinois at Urbana-Champaign); Daniela Szwarcman (IBM Research); Bianca Zadrozny (IBM Research); Campbell D Watson (IBM Reserch); Lav Varshney (UIUC: ECE)
NeurIPS 2022 Short-range forecasts of global precipitation using deep learning-augmented numerical weather prediction (Papers Track)
Abstract and authors: (click to expand)

Abstract: Precipitation drives the hydroclimate of Earth and its spatiotemporal changes on a day to day basis have one of the most notable socioeconomic impacts. The success of numerical weather prediction (NWP) is measured by the improvement of forecasts for various physical fields such as temperature and pressure. Large biases however exist in the precipitation predictions. Pure deep learning based approaches lack the advancements acheived by NWP in the past two to three decades. Hybrid methodology using NWP outputs as inputs to the deep learning based refinement tool offer an attractive means taking advantage of both NWP and state of the art deep learning algorithms. Augmenting the output from a well-known NWP model: Coupled Forecast System ver.2 (CFSv2) with deep learning for the first time, we demonstrate a hybrid model capability (DeepNWP) which shows substantial skill improvements for short-range global precipitation at 1-, 2- and 3-days lead time. To achieve this hybridization, we address the sphericity of the global data by using modified DLWP-CS architecture which transforms all the fields to cubed-sphere projection. The dynamical model outputs corresponding to precipitation and surface temperature are ingested to a UNET for predicting the target ground truth precipitation. While the dynamical model CFSv2 shows a bias in the range of +5 to +7 mm/day over land, the multivariate deep learning model reduces it to -1 to +1 mm/day over global land areas. We validate the results by taking examples from Hurricane Katrina in 2005, Hurricane Ivan in 2004, Central European floods in 2010, China floods in 2010, India floods in 2005 and the Myanmar cyclone Nargis in 2008.

Authors: Manmeet Singh (The University of Texas at Austin); Vaisakh SB (Indian Institute of Tropical Meteorology); Nachiketa Acharya (Department of Meteorology and Atmospheric Science,Pennsylvania State University); Aditya Grover (UCLA); Suryachandra A. Rao (Indian Institute of Tropical Meteorology); Bipin Kumar (Indian Institute of Tropical Meteorology); Zong-Liang Yang (The University of Texas at Austin); Dev Niyogi (The University of Texas at Austin)
NeurIPS 2022 EnhancedSD: Downscaling Solar Irradiance from Climate Model Projections (Papers Track)
Abstract and authors: (click to expand)

Abstract: Renewable energy-based electricity systems are seen as a keystone of future decarbonization efforts. However, power system planning does not currently consider the impacts of climate change on renewable energy resources such as solar energy, chiefly due to a paucity of climate-impacted high-resolution solar power data. Existing statistical downscaling (SD) methods that learn to map coarse-resolution versions of historical reanalysis data to generate finer resolution outputs are of limited use when applied to future climate model projections due to the domain gap between climate models and reanalysis data. In contrast, we present EnhancedSD, a deep learning-based framework for downscaling coarse-scale climate model outputs to high-resolution observational (reanalysis) data. Our proposed ML based downscaling allows for future reanalysis projections, which can be pivotal for mitigating climate change’s impacts on power systems planning.

Authors: Nidhin Harilal (University of Colorado, Boulder); Bri-Mathias S Hodge (University of Colorado Boulder); Claire Monteleoni (University of Colorado Boulder); Aneesh Subramanian (University of California, San Diego)
NeurIPS 2022 Positional Encoder Graph Neural Networks for Geographic Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: Modeling spatial dependencies in geographic data is of crucial importance for the modeling of our planet. Graph neural networks (GNNs) provide a powerful and scalable solution for modeling continuous spatial data. However, in the absence of further context on the geometric structure of the data, they often rely on Euclidean distances to construct the input graphs. This assumption can be improbable in many real-world settings, where the spatial structure is more complex and explicitly non-Euclidean (e.g., road networks). In this paper, we propose PE-GNN, a new framework that incorporates spatial context and correlation explicitly into the models. Building on recent advances in geospatial auxiliary task learning and semantic spatial embeddings, our proposed method (1) learns a context-aware vector encoding of the geographic coordinates and (2) predicts spatial autocorrelation in the data in parallel with the main task. We show the effectiveness of our approach on two climate-relevant regression tasks: 3d spatial interpolation and air temperature prediction. The code for this study can be accessed via: https://bit.ly/3xDpfyV.

Authors: Konstantin Klemmer (Microsoft Research); Nathan S Safir (University of Georgia); Daniel B Neill (New York University)
NeurIPS 2022 Generating physically-consistent high-resolution climate data with hard-constrained neural networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: The availability of reliable, high-resolution climate and weather data is important to inform long-term decisions on climate adaptation and mitigation and to guide rapid responses to extreme events. Forecasting models are limited by computational costs and therefore often can only make coarse resolution predictions. Statistical downscaling can provide an efficient method of upsampling low-resolution data. In this field, deep learning has been applied successfully, often using image super-resolution methods from computer vision. Despite achieving visually compelling results in some cases, such models often violate conservation laws when predicting physical variables. In order to conserve important physical quantities, we developed a deep downscaling method that guarantees physical constraints are satisfied, by adding a renormalization layer at the end of the neural network. Furthermore, the constrained model also improves the performance according to standard metrics. We show the applicability of our methods across different popular architectures and upsampling factors using ERA5 reanalysis data.

Authors: Paula Harder (Mila); Qidong Yang (New York University); Venkatesh Ramesh (Mila); Prasanna Sattigeri (IBM Research); Alex Hernandez-Garcia (Mila - Quebec AI Institute); Campbell D Watson (IBM Reserch); Daniela Szwarcman (IBM Research); David Rolnick (McGill University, Mila)
NeurIPS 2022 Transformers for Fast Emulation of Atmospheric Chemistry Box Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: When modeling atmospheric chemistry, concentrations are determined by numerically solving large systems of ordinary differential equations that represent a set of chemical reactions. These solvers can be very computationally intensive, particularly those with the thousands or tens of thousands of chemical species and reactions that make up the most accurate models. We demonstrate the application of a deep learning transformer architecture to emulate an atmospheric chemistry box model, and show that this attention-based model outperforms LSTM and autoencoder baselines while providing interpretable predictions that are more than 2 orders of magnitude faster than a numerical solver. This work is part of a larger study to replace the numerical solver in a 3D global chemical model with a machine learned emulator and achieve significant speedups for global climate simulations.

Authors: Herbie Bradley (University of Cambridge); Nathan Luke Abraham (National Centre for Atmospheric Science, UK); Peer Nowack (Imperial College London); Doug McNeall (Met Office Hadley Centre, UK)
NeurIPS 2022 Neural Representation of the Stratospheric Ozone Layer (Papers Track)
Abstract and authors: (click to expand)

Abstract: In climate modeling, the stratospheric ozone layer is typically only considered in a highly simplified form due to computational constraints. For climate projections, it would be of advantage to include the mutual interactions between stratospheric ozone, temperature, and atmospheric dynamics to accurately represent radiative forcing. The overarching goal of our research is to replace the ozone chemistry in climate models with a machine-learned neural representation of the stratospheric ozone chemistry that allows for a particularly fast, but accurate and stable simulation. We created a benchmark data set from pairs of input and output variables that we stored from simulations of a chemistry and transport model. We analyzed several variants of multilayer perceptrons suitable for physical problems to learn a neural representation of a function that predicts 24-hour ozone tendencies based on input variables. We performed a comprehensive hyperparameter optimization of the multilayer perceptron using Bayesian search and Hyperband early stopping. We validated our model by implementing it in a chemistry and transport model and comparing computation time, accuracy, and stability with the full chemistry module. We found that our model had a computation time that was a factor of 700 faster than the full chemistry module. The accuracy of our model compares favorably to the full chemistry module within a two-year simulation run, also outperforms a previous polynomial approach for fast ozone chemistry, and reproduces seasonality well in both hemispheres. In conclusion, the neural representation of stratospheric ozone chemistry in simulation resulted in an ozone layer that showed a high accuracy, significant speed-up, and stability in a long-term simulation.

Authors: Helge Mohn (Alfred Wegener Institute for Polar and Marine Research); Daniel Kreyling (Alfred Wegener Institute for Polar and Marine Research); Ingo Wohltmann (Alfred Wegener Institute for Polar and Marine Research); Ralph Lehmann (Alfred Wegener Institute for Polar and Marine Research); Peter Maaß (University of Bremen); Markus Rex (Alfred Wegener Institute for Polar and Marine Research)
NeurIPS 2022 DL-Corrector-Remapper: A grid-free bias-correction deep learning methodology for data-driven high-resolution global weather forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: Data-driven models, such as FourCastNet (FCN), have shown exemplary performance in high-resolution global weather forecasting. This performance, however, is based on supervision on mesh-gridded weather data without the utilization of raw climate observational data, the gold standard ground truth. In this work we develop a methodology to correct, remap, and fine-tune gridded uniform forecasts of FCN so it can be directly compared against observational ground truth, which is sparse and non-uniform in space and time. This is akin to bias-correction and post-processing of numerical weather prediction (NWP), a routine operation at meteorological and weather forecasting centers across the globe. The Adaptive Fourier Neural Operator (AFNO) architecture is used as the backbone to learn continuous representations of the atmosphere. The spatially and temporally non-uniform output is evaluated by the non-uniform discrete inverse Fourier transform (NUIDFT) given the output query locations. We call this network the Deep-Learning-Corrector-Remapper (DLCR). The improvement in DLCR’s performance against the gold standard ground truth over the baseline’s performance shows its potential to correct, remap, and fine-tune the mesh-gridded forecasts under the supervision of observations.

Authors: Tao Ge (Washington University in St. Louis); Jaideep Pathak (NVIDIA Corporation); Akshay Subramaniam (NVIDIA); Karthik Kashinath (NVIDIA)
NeurIPS 2022 Adaptive Bias Correction for Improved Subseasonal Forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: Subseasonal forecasting — predicting temperature and precipitation 2 to 6 weeks ahead — is critical for effective water allocation, wildfire management, and drought and flood mitigation. Recent international research efforts have advanced the subseasonal capabilities of operational dynamical models, yet temperature and precipitation prediction skills remains poor, partly due to stubborn errors in representing atmospheric dynamics and physics inside dynamical models. To counter these errors, we introduce an adaptive bias correction (ABC) method that combines state-of-the-art dynamical forecasts with observations using machine learning. When applied to the leading subseasonal model from the European Centre for Medium-Range Weather Forecasts (ECMWF), ABC improves temperature forecasting skill by 60-90% and precipitation forecasting skill by 40-69% in the contiguous U.S. We couple these performance improvements with a practical workflow, based on Cohort Shapley, for explaining ABC skill gains and identifying higher-skill windows of opportunity based on specific climate conditions.

Authors: Soukayna Mouatadid (University of Toronto); Paulo Orenstein (IMPA); Genevieve E Flaspohler (MIT); Judah Cohen (AER); Miruna Oprescu (Cornell University); Ernest Fraenkel (MIT); Lester Mackey (Microsoft Research New England)
NeurIPS 2022 An Unsupervised Learning Perspective on the Dynamic Contribution to Extreme Precipitation Changes (Papers Track)
Abstract and authors: (click to expand)

Abstract: Despite the importance of quantifying how the spatial patterns of extreme precipitation will change with warming, we lack tools to objectively analyze the storm-scale outputs of modern climate models. To address this gap, we develop an unsupervised machine learning framework to quantify how storm dynamics affect precipitation extremes and their changes without sacrificing spatial information. Over a wide range of precipitation quantiles, we find that the spatial patterns of extreme precipitation changes are dominated by spatial shifts in storm regimes rather than intrinsic changes in how these storm regimes produce precipitation.

Authors: Griffin S Mooers (UC Irvine); Tom Beucler (University of Lausanne); Michael Pritchard (UCI); Stephan Mandt (University of California, Irivine)
NeurIPS 2022 An Interpretable Model of Climate Change Using Correlative Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Determining changes in global temperature and precipitation that may indicate climate change is complicated by annual variations. One approach for finding potential climate change indicators is to train a model that predicts the year from annual means of global temperatures and precipitations. Such data is available from the CMIP6 ensemble of simulations. Here a two-hidden-layer neural network trained on this data successfully predicts the year. Differences among temperature and precipitation patterns for which the model predicts specific years reveal changes through time. To find these optimal patterns, a new way of interpreting what the neural network has learned is explored. Alopex, a stochastic correlative learning algorithm, is used to find optimal temperature and precipitation maps that best predict a given year. These maps are compared over multiple years to show how temperature and precipitations patterns indicative of each year change over time.

Authors: Charles Anderson (Colorado State University); Jason Stock (Colorado State University)
NeurIPS 2022 Using uncertainty-aware machine learning models to study aerosol-cloud interactions (Papers Track)
Abstract and authors: (click to expand)

Abstract: Aerosol-cloud interactions (ACI) include various effects that result from aerosols entering a cloud, and affecting cloud properties. In general, an increase in aerosol concentration results in smaller droplet sizes which leads to larger, brighter, longer-lasting clouds that reflect more sunlight and cool the Earth. The strength of the effect is however heterogeneous, meaning it depends on the surrounding environment, making ACI one of the most uncertain effects in our current climate models. In our work, we use causal machine learning to estimate ACI from satellite observations by reframing the problem as a treatment (aerosol) and outcome (change in droplet radius). We predict the causal effect of aerosol on clouds with uncertainty bounds depending on the unknown factors that may be influencing the impact of aerosol. Of the three climate models evaluated, we find that only one plausibly recreates the trend, lending more credence to its estimate cooling due to ACI.

Authors: Maëlys Solal (University of Oxford); Andrew Jesson (University of Oxford); Yarin Gal (University of Oxford); Alyson Douglas (University of Oxford)
NeurIPS 2022 Dynamic weights enabled Physics-Informed Neural Network for simulating the mobility of Engineered Nano Particles in a contaminated aquifer (Papers Track)
Abstract and authors: (click to expand)

Abstract: Numerous polluted groundwater sites across the globe require an active remediation strategy for the restoration of natural environmental conditions and local ecosystem. The Engineered Nanoparticles (ENPs) has emerged as an efficient reactive agent for the in-situ degradation of groundwater contaminants. While the performance of these ENPs has been highly promising on the laboratory scale, their application in a real field case conditions is still limited. The optimized injection of the ENPs in the contaminated aquifer and its subsequent monitoring are hindered by the complex transport and retention mechanisms of ENPs. Therefore, a predictive tool for understanding the transport and retention behavior of ENPs becomes highly important. The existing tools in the literature are dominated with numerical simulators, which have limited flexibility and accuracy in the presence of sparse dataset. In this work, a dynamic weights enabled Physics-Informed Neural network (dw-PINN) framework is applied to model the nano-particle´s behavior within an aquifer. The result from the forward model demonstrates the effective capability of dw-PINN in accurately predicting the ENPs mobility. The model verification step shows that the mean squared error of the predicted ENPs concentration using dw-PINN converges to a minimum value of 1.3e-5. In the subsequent step, the result from the inverse model estimates the governing parameters of ENPs mobility with reasonable accuracy. The research work demonstrates the tool´s capability in providing predictive insights for the development of an efficient groundwater remediation strategy.

Authors: Shikhar Nilabh (Amphos21)
NeurIPS 2022 Calibration of Large Neural Weather Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: Uncertainty quantification of weather forecasts is a necessity for reliably planning for and responding to extreme weather events in a warming world. This motivates the need for well-calibrated ensembles in probabilistic weather forecasting. We present initial results for the calibration of large-scale deep neural weather models for data-driven probabilistic weather forecasting. By explicitly accounting for uncertainties about the forecast's initial condition and model parameters, we generate ensemble forecasts that show promising results on standard diagnostics for probabilistic forecasts. Specifically, we are approaching the Integrated Forecasting System (IFS), the gold standard on probabilistic weather forecasting, on: (i) the spread-error agreement; and (ii) the Continuous Ranked Probability Score (CRPS). Our approach scales to state-of-the-art data-driven weather models, enabling cheap post-hoc calibration of pretrained models with tens of millions of parameters and paving the way towards the next generation of well-calibrated data-driven weather models.

Authors: Andre Graubner (Nvidia); Kamyar Kamyar Azizzadenesheli (Nvidia); Jaideep Pathak (NVIDIA Corporation); Morteza Mardani (Nvidia); Mike Pritchard (Nvidia); Karthik Kashinath (Nvidia); Anima Anandkumar (NVIDIA/Caltech)
NeurIPS 2022 Generative Modeling of High-resolution Global Precipitation Forecasts (Papers Track)
Abstract and authors: (click to expand)

Abstract: Forecasting global precipitation patterns and, in particular, extreme precipitation events is of critical importance to preparing for and adapting to climate change. Making accurate high-resolution precipitation forecasts using traditional physical models remains a major challenge in operational weather forecasting as they incur substantial computational costs and struggle to achieve sufficient forecast skill. Recently, deep-learning-based models have shown great promise in closing the gap with numerical weather prediction (NWP) models in terms of precipitation forecast skill, opening up exciting new avenues for precipitation modeling. However, it is challenging for these deep learning models to fully resolve the fine-scale structures of precipitation phenomena and adequately characterize the extremes of the long-tailed precipitation distribution. In this work, we present several improvements to the architecture and training process of a current state-of-the art deep learning precipitation model (FourCastNet) using a novel generative adversarial network (GAN) to better capture fine scales and extremes. Our improvements achieve superior performance in capturing the extreme percentiles of global precipitation, while comparable to state-of-the-art NWP models in terms of forecast skill at 1--2 day lead times. Together, these improvements set a new state-of-the-art in global precipitation forecasting.

Authors: James Duncan (University of California, Berkeley); Peter Harrington (Lawrence Berkeley National Laboratory (Berkeley Lab)); Shashank Subramanian (Lawrence Berkeley National Laboratory)
NeurIPS 2022 Deep-S2SWind: A data-driven approach for improving Sub-seasonal wind predictions (Proposals Track)
Abstract and authors: (click to expand)

Abstract: A major transformation to mitigate climate change implies a rapid decarbonisation of the energy system and thus, increasing the use of renewable energy sources, such as wind power. However, renewable resources are strongly dependent on local and large-scale weather conditions, which might be influenced by climate change. Weather-related risk assessments are essential for the energy sector, in particular, for power system management decisions, for which forecasts of climatic conditions from several weeks to months (i.e. sub-seasonal scales) are of key importance. Here, we propose a data-driven approach to predict wind speed at longer lead-times that can benefit the energy sector. The main goal of this study is to assess the potential of machine learning algorithms to predict periods of low wind speed conditions that have a strong impact on the energy sector.

Authors: Noelia Otero Felipe (University of Bern); Pascal Horton (University of Bern)
NeurIPS 2022 Towards Low Cost Automated Monitoring of Life Below Water to De-risk Ocean-Based Carbon Dioxide Removal and Clean Power (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Oceans will play a crucial role in our efforts to combat the growing climate emergency. Researchers have proposed several strategies to harness greener energy through oceans and use oceans as carbon sinks. However, the risks these strategies might pose to the ocean and marine ecosystem are not well understood. It is imperative that we quickly develop a range of tools to monitor ocean processes and marine ecosystems alongside the technology to deploy these solutions on a large scale into the oceans. Large arrays of inexpensive cameras placed deep underwater coupled with machine learning pipelines to automatically detect, classify, count and estimate fish populations have the potential to continuously monitor marine ecosystems and help study the impacts of these solutions on the ocean. In this proposal, we discuss the challenges presented by a dark artificially lit underwater video dataset captured 500m below the surface, propose potential solutions to address these challenges, and present preliminary results from detecting and classifying 6 species of fish in deep underwater camera data.

Authors: Kameswari Devi Ayyagari (Dalhousie University); Christopher Whidden (Dalhousie University); Corey Morris (Department of Fisheries and Oceans); Joshua Barnes (National Research Council Canada)
NeurIPS 2022 Towards the Automatic Analysis of Ceilometer Backscattering Profiles using Unsupervised Learning (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Ceilometers use a laser beam to capture certain phenomena in the atmosphere like clouds, precipitation, or aerosol layers. These measurements can be visualized in so-called quick looks that at the moment are mostly analyzed manually by meteorology experts. In this work, we illustrate the path towards the automatic analysis of quick looks by using a hybrid approach combining an image segmentation algorithm with unsupervised representation learning and clustering. We present a first proof of concept and give an outlook on possible future work.

Authors: Michael Dammann (HAW Hamburg); Ina Mattis (Deutscher Wetterdienst); Michael Neitzke (HAW Hamburg); Ralf Möller (University of Lübeck)
NeurIPS 2022 An Inversion Algorithm of Ice Thickness and InSAR Data for the State of Friction at the Base of the Greenland Ice Sheet (Proposals Track)
Abstract and authors: (click to expand)

Abstract: With the advent of climate change and global warming, the Greenland Ice Sheet (GrIS) has been melting at an alarming rate, losing over 215 Gt per yr, and accounting for 10% of mean global sea level rise since the 1990s. It is imperative to understand what dynamics are causing ice loss and influencing ice flow in order to successfully project mass changes of ice sheets and associated sea level rise. This work applies machine learning, ice thickness data, and horizontal ice velocity measurements from satellite radar data to quantify the magnitudes and distributions of the basal traction forces that are holding the GrIS back from flowing into the ocean. Our approach uses a hybrid model: InSAR velocity data trains a linear regression model, and these model coefficients are fed into a geophysical algorithm to estimate basal tractions that capture relationships between the ice motion and physical variables. Results indicate promising model performance and reveal significant presence of large basal traction forces around the coastline of the GrIS.

Authors: Aryan Jain (Amador Valley High School); Jeonghyeop Kim (Stony Brook University); William Holt (Stony Brook University)
NeurIPS 2022 Deep learning-based bias adjustment of decadal climate predictions (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Decadal climate predictions are key to inform adaptation strategies in a warming climate. Coupled climate models used for decadal predictions are, however, imperfect representations of the climate system leading to forecast biases. Biases can also result from a poor model initialization that, when combined with forecast drift, can produce errors depending non-linearly on lead time. We propose a deep learning-based bias correction approach for the post-processing of gridded forecasts to enhance the accuracy of decadal predictions.

Authors: Reinel Sospedra-Alfonso (Environment and Climate Change Canada); Johannes Exenberger (Graz University of Technology); Marie C McGraw (Cooperative Institute for Research in the Atmosphere | CIRA); Trung Kien Dang (National University of Singapore)
NeurIPS 2022 Surrogate Modeling for Methane Dispersion Simulations Using Fourier Neural Operator (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Methane leak detection and remediation are critical for tackling climate change, where methane dispersion simulations play an important role in emission source attribution. As 3D modeling of methane dispersion is often costly and time-consuming, we train a deep-learning-based surrogate model using the Fourier Neural Operator to learn the PDE solver in our study. Our preliminary result shows that our surrogate modeling provides a fast, accurate and cost-effective solution to methane dispersion simulations, thus reducing the cycle time of methane leak detection.

Authors: Qie Zhang (Microsoft); Mirco Milletari (Microsoft); Yagna Deepika Oruganti (Microsoft); Philipp A Witte (Microsoft)
NeurIPS 2022 Urban Heat Island Detection and Causal Inference Using Convolutional Neural Networks (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Compared to rural areas, urban areas experience higher temperatures for longer periods of time because of the urban heat island (UHI) effect. This increased heat stress leads to greater mortality, increased energy demand, regional changes to precipitation patterns, and increased air pollution. Urban developers can minimize the UHI effect by incorporating features that promote air flow and heat dispersion (e.g., increasing green space). However, understanding which urban features to implement is complex, as local meteorology strongly dictates how the environment responds to changes in urban form. In this proposal we describe a methodology for estimating the causal relationship between changes in urban form and changes in the UHI effect. Changes in urban form and temperature changes are measured using convolutional neural networks, and a causal inference matching approach is proposed to estimate causal relationships. The success of this methodology will enable urban developers to implement city-specific interventions to mitigate the warming planet's impact on cities.

Authors: Zachary D Calhoun (Duke University); Ziyang Jiang (Duke University); Mike Bergin (Duke University); David Carlson (Duke University)
NeurIPS 2022 Forecasting Global Drought Severity and Duration Using Deep Learning (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Drought detection and prediction is challenging due to the slow onset of the event and varying degrees of dependence on numerous physical and socio-economic factors that differentiate droughts from other natural disasters. In this work, we propose DeepXD (Deep learning for Droughts), a deep learning model with 26 physics-informed input features for SPI (Standardised Precipitation Index) forecasting to identify and classify droughts using monthly oceanic indices, global meteorological and vegetation data, location (latitude, longitude) and land cover for the years 1982 to 2018. In our work, we propose extracting features by considering the atmosphere and land moisture and energy budgets and forecasting global droughts on a seasonal and an annual scale at 1, 3, 6, 9, 12 and 24 months lead times. SPI helps us to identify the severity and the duration of the drought to classify them as meteorological, agricultural and hydrological.

Authors: Akanksha Ahuja (NOA); Xin Rong Chua (Centre for Climate Research Singapore)
NeurIPS 2022 Interpretable Spatiotemporal Forecasting of Arctic Sea Ice Concentration at Seasonal Lead Times (Proposals Track)
Abstract and authors: (click to expand)

Abstract: There are many benefits from the accurate forecasting of Arctic sea ice, however existing models struggle to reliably predict sea ice concentration at long lead times. Many numerical models exist but can be sensitive to initial conditions, and while recent deep learning-based methods improve overall robustness, they either do not utilize temporal trends or rely on architectures that are not performant at learning long-term sequential dependencies. We propose a method of forecasting sea ice concentration using neural circuit policies, a form of continuous time recurrent neural architecture, which improve the learning of long-term sequential dependencies compared to existing techniques and offer the added benefits of adaptability to irregular sequence intervals and high interpretability.

Authors: Matthew Beveridge (Independent Researcher); Lucas Pereira (ITI, LARSyS, Técnico Lisboa)
NeurIPS 2022 Disaster Risk Monitoring Using Satellite Imagery (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: Natural disasters such as flood, wildfire, drought, and severe storms wreak havoc throughout the world, causing billions of dollars in damages, and uprooting communities, ecosystems, and economies. Unfortunately, flooding events are on the rise due to climate change and sea level rise. The ability to detect and quantify them can help us minimize their adverse impacts on the economy and human lives. Using satellites to study flood is advantageous since physical access to flooded areas is limited and deploying instruments in potential flood zones can be dangerous. We are proposing a hands-on tutorial to highlight the use of satellite imagery and computer vision to study natural disasters. Specifically, we aim to demonstrate the development and deployment of a flood detection model using Sentinel-1 satellite data. The tutorial will cover relevant fundamental concepts as well as the full development workflow of a deep learning-based application. We will include important considerations such as common pitfalls, data scarcity, augmentation, transfer learning, fine-tuning, and details of each step in the workflow. Importantly, the tutorial will also include a case study on how the application was used by authorities in response to a flood event. We believe this tutorial will enable machine learning practitioners of all levels to develop new technologies that tackle the risks posed by climate change. We expect to deliver the below learning outcomes: • Develop various deep learning-based computer vision solutions using hardware-accelerated open-source tools that are optimized for real-time deployment • Create an optimized pipeline for the machine learning development workflow • Understand different performance metrics for model evaluation that are relevant for real world datasets and data imbalances • Understand the public sector’s efforts to support climate action initiatives and point out where the audience can contribute

Authors: Kevin Lee (NVIDIA); Siddha Ganju (NVIDIA); Edoardo Nemni (UNOSAT)
NeurIPS 2022 Machine Learning for Predicting Climate Extremes (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: Climate change has led to a rapid increase in the occurrence of extreme weather events globally, including floods, droughts, and wildfires. In the longer term, some regions will experience aridification while others will risk sinking due to rising sea levels. Typically, such predictions are done via weather and climate models that simulate the physical interactions between the atmospheric, oceanic, and land surface processes that operate at different scales. Due to the inherent complexity, these climate models can be inaccurate or computationally expensive to run, especially for detecting climate extremes at high spatiotemporal resolutions. In this tutorial, we aim to introduce the participants to machine learning approaches for addressing two fundamental challenges. We will walk the participants through a hands-on tutorial for predicting climate extremes relating to temperature and precipitation in 2 setups: (1) temporal forecasting: the goal is to predict climate variables into the future (both direct single step approaches and iterative approaches that roll out the model for several timesteps), and (2) spatial downscaling: the goal is to learn a mapping that transforms low-resolution outputs of climate models into high-resolution regional forecasts. Through introductory presentations and colab notebooks, we aim to expose the participants to (a) APIs for accessing and navigating popular repositories that host global climate data, such as the Copernicus data store, (b) identifying relevant datasets, including auxiliary data (e.g., other climate variables such as geopotential), (c) scripts for downloading and preprocessing relevant datasets, (d) algorithms for training machine learning models, (d) metrics for evaluating model performance, and (e) visualization tools for both the dataset and predicted outputs. The coding notebooks will be in Python. No prior knowledge of climate science is required.

Authors: Hritik Bansal (UCLA); Shashank Goel (University of California Los Angeles); Tung Nguyen (University of California, Los Angeles); Aditya Grover (UCLA)
NeurIPS 2022 FourCastNet: A practical introduction to a state-of-the-art deep learning global weather emulator (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: Accurate, reliable, and efficient means of forecasting global weather patterns are of paramount importance to our ability to mitigate and adapt to climate change. Currently, real-time weather forecasting requires repeated numerical simulation and data assimilation cycles on dedicated supercomputers, which restricts the ability to make reliable, high-resolution forecasts to a handful of organizations. However, recent advances in deep learning, specifically the FourCastNet model, have shown that data-driven approaches can forecast important atmospheric variables with excellent skill and comparable accuracy to standard numerical methods, but at orders-of-magnitude lower computational and energy cost during inference, enabling larger ensembles for better probabilistic forecasts. In this tutorial, we demonstrate various applications of FourCastNet for high-resolution global weather forecasting, with examples including real-time forecasts, uncertainty quantification for extreme events, and adaptation to specific variables or localized regions of interest. The tutorial will provide examples that will demonstrate the general workflow for formatting and working with global atmospheric data, running autoregressive inference to obtain daily global forecasts, saving/visualizing model predictions of atmospheric events such as hurricanes and atmospheric rivers, and computing quantitative evaluation metrics for weather models. The exercises will primarily use PyTorch and do not require detailed understanding of the climate and weather system. With this tutorial, we hope to equip attendees with basic knowledge about building deep learning-based weather model surrogates and obtaining forecasts of crucial atmospheric variables using these models.

Authors: Jaideep Pathak (NVIDIA Corporation); Shashank Subramanian (Lawrence Berkeley National Laboratory); Peter Harrington (Lawrence Berkeley National Laboratory (Berkeley Lab)); Thorsten Kurth (Nvidia); Andre Graubner (Nvidia); Morteza Mardani (NVIDIA Corporation); David M. Hall (NVIDIA); Karthik Kashinath (Lawrence Berkeley National Laboratory); Anima Anandkumar (NVIDIA/Caltech)
NeurIPS 2022 Automating the creation of LULC datasets for semantic segmentation (Tutorials Track)
Abstract and authors: (click to expand)

Abstract: High resolution and accurate Land Use and Land Cover mapping (LULC) datasets are increasingly important and can be widely used in monitoring climate change impacts in agriculture, deforestation, and the carbon cycle. These datasets represent physical classifications of land types and spatial information over the surface of the Earth. These LULC datasets can be leveraged in a plethora of research topics and industries to mitigate and adapt to environmental changes. High resolution urban mappings can be used to better monitor and estimate building albedo and urban heat island impacts, and accurate representation of forests and vegetation can even be leveraged to better monitor the carbon cycle and climate change through improved land surface modelling. The advent of machine learning (ML) based CV techniques over the past decade provides a viable option to automate LULC mapping. One impediment to this has been the lack of large ML datasets. Large vector datasets for LULC are available, but can’t be used directly by ML practitioners due to a knowledge gap in transforming the input into a dataset of paired satellite images and segmentation masks. We demonstrate a novel end-to-end pipeline for LULC dataset creation that takes vector land cover data and provides a training-ready dataset. We will use Sentinel-2 satellite imagery and the European Urban Atlas LULC data. The pipeline manages everything from downloading satellite data, to creating and storing encoded segmentation masks and automating data checks. We then use the resulting dataset to train a semantic segmentation model. The aim of the pipeline is to provide a way for users to create their own custom datasets using various combinations of multispectral satellite and vector data. In addition to presenting the pipeline, we aim to provide an introduction to multispectral imagery, geospatial data and some of the challenges in using it for ML.

Authors: Sambhav S Rohatgi (Spacesense.ai); Anthony Mucia (Spacesense.ai)
AAAI FSS 2022 Data-Driven Reduced-Order Model for Atmospheric CO2 Dispersion
Abstract and authors: (click to expand)

Abstract: Machine learning frameworks have emerged as powerful tools for the enhancement of computational fluid dynamics simulations and the construction of reduced-order models (ROMs). The latter are particularly desired when their full-order counterparts portray multiple spatiotemporal features and demand high processing power and storage capacity, such as climate models. In this work, a ROM for CO2 dispersion across Earth‘s atmosphere was built from NASA’s gridded daily OCO-2 carbon dioxide assimilated dataset. For that, a proper orthogonal decomposition was performed, followed by a non-intrusive operator inference (OpInf). This scientific machine learning technique was capable of accurately representing and predicting the detailed CO2 concentration field for about one year ahead, with a normalized root-mean-square error below 5%. It suggests OpInf-based ROMs may be a reliable alternative for fast response climate-related predictions.

Authors: Pedro Roberto Barbosa Rocha (IBM Research), Marcos Sebastião de Paula Gomes (Pontifical Catholic University of Rio de Janeiro), João Lucas de Sousa Almeida (IBM Research), Allan Moreira Carvalho (IBM Research) and Alberto Costa Nogueira Junior (IBM Research)
AAAI FSS 2022 Generating physically-consistent high-resolution climate data with hard-constrained neural networks
Abstract and authors: (click to expand)

Abstract: The availability of reliable, high-resolution climate and weather data is important to inform long-term decisions on climate adaptation and mitigation and to guide rapid responses to extreme events. Forecasting models are limited by computational costs and therefore often predict quantities at a coarse spatial resolution. Statistical downscaling can provide an efficient method of upsampling low-resolution data. In this field, deep learning has been applied successfully, often using methods from the super-resolution domain in computer vision. Despite often achieving visually compelling results, such models often violate conservation laws when predicting physical variables. In order to conserve important physical quantities, we develop methods that guarantee physical constraints are satisfied by a deep downscaling model while also increasing their performance according to traditional metrics. We introduce two ways of constraining the network: a renormalization layer added to the end of the neural network and a successive approach that scales with increasing upsampling factors. We show the applicability of our methods across different popular architectures and upsampling factors using ERA5 reanalysis data.

Authors: Paula Harder (Fraunhofer Institute ITWM, Mila Quebec AI Institute), Qidong Yang (Mila Quebec AI Institute, New York University), Venkatesh Ramesh (Mila Quebec AI Institute, University of Montreal), Alex Hernandez-Garcia (Mila Quebec AI Institute, University of Montreal), Prasanna Sattigeri (IBM Research), Campbell D. Watson (IBM Research), Daniela Szwarcman (IBM Research) and David Rolnick (Mila Quebec AI Institute, McGill University).
AAAI FSS 2022 Contrastive Learning for Climate Model Bias Correction and Super-Resolution
Abstract and authors: (click to expand)

Abstract: Climate models often require post-processing in order to make accurate estimates of local climate risk. The most common post-processing applied is bias-correction and spatial resolution enhancement. However, the statistical methods typically used for this not only are incapable of capturing multivariate spatial correlation information but are also reliant on rich observational data often not available outside of developed countries, limiting their potential. Here we propose an alternative approach to this challenge based on a combination of image super resolution (SR) and contrastive learning generative adversarial networks (GANs). We benchmark performance against NASA’s flagship post-processed CMIP6 climate model product, NEX-GDDP. We find that our model successfully reaches a spatial resolution double that of NASA’s product while also achieving comparable or improved levels of bias correction in both daily precipitation and temperature. The resulting higher fidelity simulations of present and forward-looking climate can enable more local, accurate models of hazards like flooding, drought, and heatwaves.

Authors: Tristan Ballard (Sust Global) and Gopal Erinjippurath (Sust Global)
AAAI FSS 2022 Probabilistic Machine Learning in Polar Earth and Climate Science: A Review of Applications and Opportunities
Abstract and authors: (click to expand)

Abstract: Our world’s climate future is on thin ice. The study of longterm weather patterns in the polar regions is an important building block in tackling Climate Change. Our understanding of the past, the present and the future of the earth system, and the inherent uncertainty, informs planning, mitigation, and adaptation strategies. In this work we review previous applications of machine learning and statistical computing to polar climate research, and we highlight promising probabilistic machine learning methods that address the modelling needs of climate-related research in the Arctic and the Antarctic. We discuss common challenges in this interdisciplinary field and provide an overview of opportunities for future work in this novel area of research.

Authors: Kim Bente (The University of Sydney), Judy Kay (The University of Sydney) and Roman Marchant (Commonwealth Scientific and Industrial Research Organisation (CSIRO))
NeurIPS 2021 Towards Representation Learning for Atmospheric Dynamics (Papers Track)
Abstract and authors: (click to expand)

Abstract: The prediction of future climate scenarios under anthropogenic forcing is critical to understand climate change and to assess the impact of potentially counter-acting technologies. Machine learning and hybrid techniques for this prediction rely on informative metrics that are sensitive to pertinent but often subtle influences. For atmospheric dynamics, a critical part of the climate system, no well established metric exists and visual inspection is currently still often used in practice. However, this ``eyeball metric'' cannot be used for machine learning where an algorithmic description is required. Motivated by the success of intermediate neural network activations as basis for learned metrics, e.g. in computer vision, we present a novel, self-supervised representation learning approach specifically designed for atmospheric dynamics. Our approach, called AtmoDist, trains a neural network on a simple, auxiliary task: predicting the temporal distance between elements of a randomly shuffled sequence of atmospheric fields (e.g. the components of the wind field from reanalysis or simulation). The task forces the network to learn important intrinsic aspects of the data as activations in its layers and from these hence a discriminative metric can be obtained. We demonstrate this by using AtmoDist to define a metric for GAN-based super resolution of vorticity and divergence. Our upscaled data matches both visually and in terms of its statistics a high resolution reference closely and it significantly outperform the state-of-the-art based on mean squared error. Since AtmoDist is unsupervised, only requires a temporal sequence of fields, and uses a simple auxiliary task, it has the potential to be of utility in a wide range of applications.

Authors: Sebastian Hoffmann (University of Magdeburg); Christian Lessig (Otto-von-Guericke-Universitat Magdeburg)
NeurIPS 2021 Addressing Deep Learning Model Uncertainty in Long-Range Climate Forecasting with Late Fusion (Papers Track)
Abstract and authors: (click to expand)

Abstract: Global warming leads to the increase in frequency and intensity of climate extremes that cause tremendous loss of lives and property. Accurate long-range climate prediction allows more time for preparation and disaster risk management for such extreme events. Although machine learning approaches have shown promising results in long-range climate forecasting, the associated model uncertainties may reduce their reliability. To address this issue, we propose a late fusion approach that systematically combines the predictions from multiple models to reduce the expected errors of the fused results. We also propose a network architecture with the novel denormalization layer to gain the benefits of data normalization without actually normalizing the data. The experimental results on long-range 2m temperature forecasting show that the framework outperforms the 30-year climate normals, and the accuracy can be improved by increasing the number of models.

Authors: Ken C. L. Wong (IBM Research – Almaden Research Center); Hongzhi Wang (IBM Almaden Research Center); Etienne E Vos (IBM); Bianca Zadrozny (IBM Research); Campbell D Watson (IBM Reserch); Tanveer Syeda-Mahmood (IBM Research)
NeurIPS 2021 Towards debiasing climate simulations using unsuperviserd image-to-image translation networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate models form the basis of a vast portion of earth system research, and inform our climate policy. Due to the complex nature of our climate system, and the approximations which must necessarily be made in simulating it, these climate models may not perfectly match observations. For further research, these outputs must be bias corrected against observations, but current methods of debiasing do not take into account spatial correlations. We evaluate unsupervised image-to-image translation networks, specifically the UNIT model architecture, for their ability to produce more spatially realistic debiasing than the standard techniques used in the climate community.

Authors: James Fulton (University of Edinburgh); Ben Clarke (Oxford University)
NeurIPS 2021 Using Non-Linear Causal Models to Study Aerosol-Cloud Interactions in the Southeast Pacific (Papers Track)
Abstract and authors: (click to expand)

Abstract: Aerosol-cloud interactions include a myriad of effects that all begin when aerosol enters a cloud and acts as cloud condensation nuclei (CCN). An increase in CCN results in a decrease in the mean cloud droplet size (r$_{e}$). The smaller droplet size leads to brighter, more expansive, and longer lasting clouds that reflect more incoming sunlight, thus cooling the earth. Globally, aerosol-cloud interactions cool the Earth, however the strength of the effect is heterogeneous over different meteorological regimes. Understanding how aerosol-cloud interactions evolve as a function of the local environment can help us better understand sources of error in our Earth system models, which currently fail to reproduce the observed relationships. In this work we use recent non-linear, causal machine learning methods to study the heterogeneous effects of aerosols on cloud droplet radius.

Authors: Andrew Jesson (University of Oxford); Peter Manshausen (University of Oxford); Alyson Douglas (University of Oxford); Duncan Watson-Parris (University of Oxford); Yarin Gal (University of Oxford); Philip Stier (University of Oxford)
NeurIPS 2021 Hurricane Forecasting: A Novel Multimodal Machine Learning Framework (Papers Track)
Abstract and authors: (click to expand)

Abstract: This paper describes a machine learning (ML) framework for tropical cyclone intensity and track forecasting, combining multiple distinct ML techniques and utilizing diverse data sources. Our framework, which we refer to as Hurricast (HURR), is built upon the combination of distinct data processing techniques using gradient-boosted trees and novel encoder-decoder architectures, including CNN, GRU and Transformers components. We propose a deep-learning feature extractor methodology to mix spatial-temporal data with statistical data efficiently. Our multimodal framework unleashes the potential of making forecasts based on a wide range of data sources, including historical storm data, and visual data such as reanalysis atmospheric images. We evaluate our models with current operational forecasts in North Atlantic (NA) and Eastern Pacific (EP) basins on 2016-2019 for 24-hour lead time, and show our models consistently outperform statistical-dynamical models and compete with the best dynamical models. Furthermore, the inclusion of Hurricast into an operational forecast consensus model leads to a significant improvement of 5% - 15% over NHC's official forecast, thus highlighting the complementary properties with existing approaches.

Authors: Léonard Boussioux (MIT, CentraleSupélec); Cynthia Zeng (MIT); Dimitris Bertsimas (MIT); Théo J Guenais (Harvard University)
NeurIPS 2021 Improved Drought Forecasting Using Surrogate Quantile And Shape (SQUASH) Loss (Papers Track)
Abstract and authors: (click to expand)

Abstract: Droughts are amongst the most damaging natural hazard with cascading impacts across multiple sectors of the economy and society. Improved forecasting of drought conditions ahead of time can significantly improve strategic planning to mitigate the impacts and enhance resilience. Though significant progress in forecasting approaches has been made, the current approaches focus on the overall improvement of the forecast, with less attention on the extremeness of drought events. In this paper, we focus on improving the accuracy of forecasting extreme and severe drought events by introducing a novel loss function Surrogate Quantile and Shape loss (SQUASH) that combines weighted quantile loss and dynamic time-warping-based shape loss. We show the effectiveness of the proposed loss functions for imbalanced time-series drought forecasting tasks on two regions in India and the USA.

Authors: Devyani Lambhate Lambhate (Indian Institute of Science); Smit Marvaniya (IBM Research India); Jitendra Singh (IBM Research - India); David Gold (IBM)
NeurIPS 2021 On the Generalization of Agricultural Drought Classification from Climate Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change is expected to increase the likelihood of drought events, with severe implications for food security. Unlike other natural disasters, droughts have a slow onset and depend on various external factors, making drought detection in climate data difficult. In contrast to existing works that rely on simple relative drought indices as ground-truth data, we build upon soil moisture index (SMI) obtained from a hydrological model. This index is directly related to insufficiently available water to vegetation. Given ERA5-Land climate input data of six months with land use information from MODIS satellite observation, we compare different models with and without sequential inductive bias in classifying droughts based on SMI. We use PR-AUC as the evaluation measure to account for the class imbalance and obtain promising results despite a challenging time-based split. We further show in an ablation study that the models retain their predictive capabilities given input data of coarser resolutions, as frequently encountered in climate models.

Authors: Julia Gottfriedsen (1Deutsches Zentrum für Luft- und Raumfahrt (DLR), LMU); Max Berrendorf (Ludwig-Maximilians-Universität München); Pierre Gentine (Columbia University); Markus Reichstein (Max Planck Institute for Biogeochemistry, Jena; Michael Stifel Center Jena for Data-Driven and Simulation Science, Jena); Katja Weigel (niversity of Bremen, Institute of Environmental Physics (IUP), Bremen, Germany); Birgit Hassler (Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Institut für Physik der Atmosphäre, Oberpfaffenhofen, Germany); Veronika Eyring (Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Institut für Physik der Atmosphäre, Oberpfaffenhofen, Germany; University of Bremen, Institute of Environmental Physics (IUP), Bremen, Germany)
NeurIPS 2021 WiSoSuper: Benchmarking Super-Resolution Methods on Wind and Solar Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: The transition to green energy grids depends on detailed wind and solar forecasts to optimize the siting and scheduling of renewable energy generation. Operational forecasts from numerical weather prediction models, however, only have a spatial resolution of 10 to 20-km, which leads to sub-optimal usage and development of renewable energy farms. Weather scientists have been developing super-resolution methods to increase the resolution, but often rely on simple interpolation techniques or computationally expensive differential equation-based models. Recently, machine learning-based models, specifically the physics-informed resolution-enhancing generative adversarial network (PhIREGAN), have outperformed traditional downscaling methods. We provide a thorough and extensible benchmark of leading deep learning-based super-resolution techniques, including the enhanced super-resolution generative adversarial network (ESRGAN) and an enhanced deep super-resolution (EDSR) network, on wind and solar data. We accompany the benchmark with a novel public, processed, and machine learning-ready dataset for benchmarking super-resolution methods on wind and solar data.

Authors: Rupa Kurinchi-Vendhan (Caltech); Björn Lütjens (MIT); Ritwik Gupta (University of California, Berkeley); Lucien D Werner (California Institute of Technology); Dava Newman (MIT); Steven Low (California Institute of Technology)
NeurIPS 2021 MS-nowcasting: Operational Precipitation Nowcasting with Convolutional LSTMs at Microsoft Weather (Papers Track)
Abstract and authors: (click to expand)

Abstract: We present the encoder-forecaster convolutional long short-term memory (LSTM) deep-learning model that powers Microsoft Weather's operational precipitation nowcasting product. This model takes as input a sequence of weather radar mosaics and deterministically predicts future radar reflectivity at lead times up to 6 hours. By stacking a large input receptive field along the feature dimension and conditioning the model's forecaster with predictions from the physics-based High Resolution Rapid Refresh (HRRR) model, we are able to outperform optical flow and HRRR baselines by 20-25% on multiple metrics averaged over all lead times.

Authors: Sylwester Klocek (Microsoft Corporation); Haiyu Dong (Microsoft); Matthew Dixon (Microsoft Corporation); Panashe Kanengoni (Microsoft Corporation); Najeeb Kazmi (Microsoft); Pete Luferenko (Microsoft Corporation); Zhongjian Lv (Microsoft Corporation); Shikhar Sharma (); Jonathan Weyn (Microsoft); Siqi Xiang (Microsoft Corporation)
NeurIPS 2021 ClimART: A Benchmark Dataset for Emulating Atmospheric Radiative Transfer in Weather and Climate Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: Numerical simulations of Earth's weather and climate require substantial amounts of computation. This has led to a growing interest in replacing subroutines that explicitly compute physical processes with approximate machine learning (ML) methods that are fast at inference time. Within weather and climate models, atmospheric radiative transfer (RT) calculations are especially expensive. This has made them a popular target for neural network-based emulators. However, prior work is hard to compare due to the lack of a comprehensive dataset and standardized best practices for ML benchmarking. To fill this gap, we introduce the \climart dataset, with more than \emph{10 million samples from present, pre-industrial, and future climate conditions}. ClimART poses several methodological challenges for the ML community, such as multiple out-of-distribution test sets, underlying domain physics, and a trade-off between accuracy and inference speed. We also present several novel baselines that indicate shortcomings of the datasets and network architectures used in prior work.

Authors: Salva Rühling Cachay (Technical University of Darmstadt); Venkatesh Ramesh (MILA); Jason N. S. Cole (Environment and Climate Change Canada); Howard Barker (Environment and Climate Change Canada); David Rolnick (McGill University, Mila)
NeurIPS 2021 Learned Benchmarks for Subseasonal Forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: We develop a subseasonal forecasting toolkit of simple learned benchmark models that outperform both operational practice and state-of-the-art machine learning and deep learning methods. Our new models include (a) Climatology++, an adaptive alternative to climatology that, for precipitation, is 9% more accurate and 250\% more skillful than the United States operational Climate Forecasting System (CFSv2); (b) CFSv2++, a learned CFSv2 correction that improves temperature and precipitation accuracy by 7-8% and skill by 50-275%; and (c) Persistence++, an augmented persistence model that combines CFSv2 forecasts with lagged measurements to improve temperature and precipitation accuracy by 6-9% and skill by 40-130%. Across the contiguous U.S., these models consistently outperform standard meteorological baselines, state-of-the-art learning methods, and the European Centre for Medium-Range Weather Forecasts ensemble. Overall, we find that augmenting traditional forecasting approaches with learned enhancements yields an effective and computationally inexpensive strategy for building the next generation of subseasonal forecasting benchmarks.

Authors: Soukayna Mouatadid (University of Toronto); Paulo Orenstein (IMPA); Genevieve E Flaspohler (MIT); Miruna Oprescu (Microsoft Research); Judah Cohen (AER); Franklyn Wang (Harvard University); Sean Knight (MIT); Maria Geogdzhayeva (MIT); Sam Levang (Salient Predictions Inc.); Ernest Fraenkel (MIT); Lester Mackey (Microsoft Research)
NeurIPS 2021 Predicting Atlantic Multidecadal Variability (Papers Track) Best Paper: Pathway to Impact
Abstract and authors: (click to expand)

Abstract: Atlantic Multidecadal Variability (AMV) describes variations of North Atlantic sea surface temperature with a typical cycle of between 60 and 70 years. AMV strongly impacts local climate over North America and Europe, therefore prediction of AMV, especially the extreme values, is of great societal utility for understanding and responding to regional climate change. This work tests multiple machine learning models to improve the state of AMV prediction from maps of sea surface temperature, salinity, and sea level pressure in the North Atlantic region. We use data from the Community Earth System Model 1 Large Ensemble Project, a state-of-the-art climate model with 3,440 years of data. Our results demonstrate that all of the models we use outperform the traditional persistence forecast baseline. Predicting the AMV is important for identifying future extreme temperatures and precipitation as well as hurricane activity, in Europe and North America up to 25 years in advance.

Authors: Glenn Liu (Massachusetts Institute of Technology); Peidong Wang (MIT); Matthew Beveridge (Massachusetts Institute of Technology); Young-Oh Kwon (Woods Hole Oceanographic Institution); Iddo Drori (MIT)
NeurIPS 2021 Identifying the atmospheric drivers of drought and heat using a smoothed deep learning approach (Papers Track)
Abstract and authors: (click to expand)

Abstract: Europe was hit by several, disastrous heat and drought events in recent summers. Besides thermodynamic influences, such hot and dry extremes are driven by certain atmospheric situations including anticyclonic conditions. Effects of climate change on atmospheric circulations are complex and many open research questions remain in this context, e.g., on future trends of anticyclonic conditions. Based on the combination of a catalog of labeled circulation patterns and spatial atmospheric variables, we propose a smoothed convolutional neural network classifier for six types of anticyclonic circulations that are associated with drought and heat. Our work can help to identify important drivers of hot and dry extremes in climate simulations, which allows to unveil the impact of climate change on these drivers. We address various challenges inherent to circulation pattern classification that are also present in other climate patterns, e.g., subjective labels and unambiguous transition periods.

Authors: Magdalena Mittermeier (Ludwig-Maximilians-Universität München); Maximilian Weigert (Ludwig-Maximilians-Universität München); David Ruegamer (LMU Munich)
NeurIPS 2021 High-resolution rainfall-runoff modeling using graph neural network (Papers Track)
Abstract and authors: (click to expand)

Abstract: Time-series modeling has shown great promise in recent studies using the latest deep learning algorithms such as LSTM (Long Short-Term Memory). These studies primarily focused on watershed-scale rainfall-runoff modeling or streamflow forecasting, but the majority of them only considered a single watershed as a unit. Although this simplification is very effective, it does not take into account spatial information, which could result in significant errors in large watersheds. Several studies investigated the use of GNN (Graph Neural Networks) for data integration by decomposing a large watershed into multiple sub-watersheds, but each sub-watershed is still treated as a whole, and the geoinformation contained within the watershed is not fully utilized. In this paper, we propose the GNRRM (Graph Neural Rainfall-Runoff Model), a novel deep learning model that makes full use of spatial information from high-resolution precipitation data, including flow direction and geographic information. When compared to baseline models, GNRRM has less over-fitting and significantly improves model performance. Our findings support the importance of hydrological data in deep learning-based rainfall-runoff modeling, and we encourage researchers to include more domain knowledge in their models.

Authors: Zhongrun Xiang (University of Iowa); Ibrahim Demir (The University of Iowa)
NeurIPS 2021 Leveraging Machine Learning to Predict the Autoconversion Rates from Satellite Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: One way of reducing the uncertainty involved in determining the radiative forcing of climate change is by understanding the interaction between aerosols, clouds, and precipitation processes. This can be studied using high-resolution simulations such as the ICOsahedral Non-hydrostatic Large-Eddy Model (ICON-LEM). However, due to the extremely high computational cost required, this simulation-based approach can only be run for a limited amount of time within a limited area. To address this, we developed new models using emerging machine learning approaches that leverage a plethora of satellite observations providing long-term global spatial coverage. In particular, our machine learning models are capable of capturing the key process of precipitation formation which greatly control cloud lifetime, namely autoconversion rates -- the term used to describe the collision and coalescence of cloud droplets responsible for raindrop formation. We validate the performance of our models against simulation data, showing that our models are capable of predicting the autoconversion rates fairly well.

Authors: Maria C Novitasari (University College London); Johannes Quaas (University of Leipzig); Miguel Rodrigues (University College London)
NeurIPS 2021 Towards Automatic Transformer-based Cloud Classification and Segmentation (Papers Track)
Abstract and authors: (click to expand)

Abstract: Clouds have been demonstrated to have a huge impact on the energy balance, temperature, and weather of the Earth. Classification and segmentation of clouds and coverage factors is crucial for climate modelling, meteorological studies, solar energy industry, and satellite communication. For example, clouds have a tremendous impact on short-term predictions or 'nowcasts' of solar irradiance and can be used to optimize solar power plants and effectively exploit solar energy. However even today, cloud observation requires the intervention of highly-trained professionals to document their findings, which introduces bias. To overcome these issues and contribute to climate change technology, we propose, to the best of our knowledge, the first two transformer-based models applied to cloud data tasks. We use the CCSD Cloud classification dataset and achieve 90.06% accuracy, outperforming all other methods. To demonstrate the robustness of transformers in this domain, we perform Cloud segmentation on SWIMSWG dataset and achieve 83.2% IoU, also outperforming other methods. With this, we signal a potential shift away from pure CNN networks.

Authors: Roshan Roy (Birla Institute of Technology and Science, Pilani); Ahan M R (BITS Pilani); Vaibhav Soni (MANIT Bhopal); Ashish Chittora (BITS Pilani)
NeurIPS 2021 Scalable coastal inundation mapping using machine learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Coastal flooding is a significant climate hazard with impacts across economic sectors and society. This study provides a proof of concept for data-driven models for coastal flood inundation at the country scale, incorporating storm dynamics and geospatial characteristics to improve upon simpler geomorphological models. The best fit machine learning model scores an AUC of 0.92 in predicting flooded locations. For a case study storm event in December 2013 we find that all models over-predict flood extents, but that the machine learning model extents were closest to those observed.

Authors: Ophelie Meuriot (IBM Research Europe); Anne Jones (IBM Research)
NeurIPS 2021 On the use of Deep Generative Models for "Perfect" Prognosis Climate Downscaling (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Deep Learning has recently emerged as a "perfect" prognosis downscaling technique to compute high-resolution fields from large-scale coarse atmospheric data. Despite their promising results to reproduce the observed local variability, they are based on the estimation of independent distributions at each location, which leads to deficient spatial structures, especially when downscaling precipitation. This study proposes the use of generative models to improve the spatial consistency of the high-resolution fields, very demanded by some sectoral applications (e.g., hydrology) to tackle climate change.

Authors: Jose González-Abad (Institute of Physics of Cantabria); Jorge Baño-Medina (Institute of Physics of Cantabria); Ignacio Heredia (Institute of Physics of Cantabria)
NeurIPS 2021 Toward Foundation Models for Earth Monitoring: Proposal for a Climate Change Benchmark (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Recent progress in self-supervision shows that pre-training large neural networks on vast amounts of unsupervised data can lead to impressive increases in generalisation for downstream tasks. Such models, recently coined as foundation models, have been transformational to the field of natural language processing. While similar models have also been trained on large corpuses of images, they are not well suited for remote sensing data. To stimulate the development of foundation models for Earth monitoring, we propose to develop a new benchmark comprised of a variety of downstream tasks related to climate change. We believe that this can lead to substantial improvements in many existing applications and facilitate the development of new applications. This proposal is also a call for collaboration with the aim of developing a better evaluation process to mitigate potential downsides of foundation models for Earth monitoring.

Authors: Alexandre Lacoste (ServiceNow); Evan D Sherwin (Stanford University, Energy and Resources Engineering); Hannah R Kerner (University of Maryland); Hamed Alemohammad (Radiant Earth Foundation); Björn Lütjens (MIT); Jeremy A Irvin (Stanford); David Dao (ETH Zurich); Alex Chang (Service Now); Mehmet Gunturkun (Element Ai); Alexandre Drouin (ServiceNow); Pau Rodriguez (Element AI); David Vazquez (ServiceNow)
NeurIPS 2021 DeepQuake: Artificial Intelligence for Earthquake Forecasting Using Fine-Grained Climate Data (Proposals Track) Best Paper: Proposals
Abstract and authors: (click to expand)

Abstract: Earthquakes are one of the most catastrophic natural disasters, making accurate, fine-grained, and real-time earthquake forecasting extremely important for the safety and security of human lives. In this work, we propose DeepQuake, a hybrid physics and deep learning model for fine-grained earthquake forecasting using time-series data of the horizontal displacement of earth’s surface measured from continuously operating Global Positioning System (cGPS) data. Recent studies using cGPS data have established a link between transient deformation within earth's crust to climate variables. DeepQuake’s physics-based pre-processing algorithm extracts relevant features including the x, y, and xy components of strain in earth’s crust, capturing earth’s elastic response to these climate variables, and feeds it into a deep learning neural network to predict key earthquake variables such as the time, location, magnitude, and depth of a future earthquake. Results across California show promising correlations between cGPS derived strain patterns and the earthquake catalog ground truth for a given location and time.

Authors: Yash Narayan (The Nueva School)
ICML 2021 A human-labeled Landsat-8 contrails dataset (Papers Track)
Abstract and authors: (click to expand)

Abstract: Contrails (condensation trails) are the ice clouds that trail behind aircraft as they fly through cold and moist regions of the atmosphere. Avoiding these regions could potentially be an inexpensive way to reduce over half of aviation's impact on global warming. Development and evaluation of these avoidance strategies greatly benefits from the ability to detect contrails on satellite imagery. Since little to no public data is available to develop such contrail detectors, we construct and release a dataset of several thousand Landsat-8 scenes with pixel-level annotations of contrails. The dataset will continue to grow, but currently contains 4289 scenes (of which 47% have at least one contrail) representing 950+ person-hours of labeling time.

Authors: Kevin McCloskey (Google); Scott Geraedts (Google); Brendan Jackman (Google); Vincent R. Meijer (Laboratory for Aviation and the Environment, Massachusetts Institute of Technology); Erica Brand (Google); Dave Fork (Google); John C. Platt (Google); Carl Elkin (Google); Christopher Van Arsdale (Google)
ICML 2021 Urban Tree Species Classification Using Aerial Imagery (Papers Track)
Abstract and authors: (click to expand)

Abstract: Urban trees help regulate temperature, reduce energy consumption, improve urban air quality, reduce wind speeds, and mitigating the urban heat island effect. Urban trees also play a key role in climate change mitigation and global warming by capturing and storing atmospheric carbon-dioxide which is the largest contributor to greenhouse gases. Automated tree detection and species classification using aerial imagery can be a powerful tool for sustainable forest and urban tree management. Hence, This study first offers a pipeline for generating labelled dataset of urban trees using Google Map's aerial images and then investigates how state of the art deep Convolutional Neural Network models such as VGG and ResNet handle the classification problem of urban tree aerial images under different parameters. Experimental results show our best model achieves an average accuracy of 60% over 6 tree species.

Authors: Emily Waters (Anglia Ruskin University); Mahdi Maktabdar Oghaz (Anglia Ruskin University); Lakshmi Babu Saheer (Anglia Ruskin University)
ICML 2021 Seasonal Sea Ice Presence Forecasting of Hudson Bay using Seq2Seq Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Accurate and timely forecasts of sea ice conditions are crucial for safe shipping operations in the Canadian Arctic and other ice-infested waters. Given the advancement of machine-learning methods and the recent observations on the declining trend of Arctic sea ice extent over the past decades due to global warming, new machine learning approaches are deployed to provide additional sea ice forecasting products. This study is novel in comparison with previous machine learning (ML) approaches in the sea-ice forecasting domain as it provides a daily spatial map of probability of ice presence in the domain up to 90 days. The predictions are further used to predict freeze-up/breakup dates and show their capability to capture both the variability and the increasing trend of open water season in the domain over the past decades.

Authors: Nazanin Asadi (University of Waterloo); K Andrea Scott (University of Waterloo); Philippe Lamontagne (National Research Council Canada)
ICML 2021 Prediction of Boreal Peatland Fires in Canada using Spatio-Temporal Methods (Papers Track)
Abstract and authors: (click to expand)

Abstract: Peat fires are the largest fires on earth in terms of fuel consumption and are responsible for a significant portion of global carbon emissions. Predicting fires in the peatlands can help decision-makers and researchers monitor and prevent peat fires. Despite this, research on predicting peatland fires remains largely understudied as compared to the prediction of other forms of fires. However, peatland fires are unique among fires and therefore require datasets and architectures attuned to their particular characteristics. In this paper, we present a new dataset, PeatSet, designed specifically for the problem of peatland fire prediction. In addition, we propose several models to tackle the problem of fire prediction for the peatlands. We develop novel neural architectures for peatland fire prediction, PeatNet, and PT-Net, with a graph-based and a transformer-based architecture, respectively. Our results indicate that these new deep-learning architectures outperform a regression baseline from existing peatland research. Among all the tested models, PT-Net achieves the highest F1 score of 0.1006 and an overall accuracy of 99.84%.

Authors: Shreya Bali (Carnegie Mellon University); Sydney Zheng (Carnegie Mellon University); Akshina Gupta (Carnegie Mellon University); Yue Wu (None); Blair Chen (Carnegie Mellon University); Anirban Chowdhury (Carnegie Mellon University); Justin Khim (Carnegie Mellon University)
ICML 2021 Revealing the impact of global warming on climate modes using transparent machine learning and a suite of climate models (Papers Track)
Abstract and authors: (click to expand)

Abstract: The ocean is key to climate through its ability to store and transport heat and carbon. From studies of past climates, it is clear that the ocean can exhibit a range of dramatic variability that could have catastrophic impacts on society, such as changes in rainfall, severe weather, sea level rise and large scale climate patterns. The mechanisms of change remain obscure, but are explored using a transparent machine learning method, Tracking global Heating with Ocean Regimes (THOR) presented here. We investigate two future scenarios, one where CO2 is increased by 1% per year, and one where CO2 is abruptly quadrupled. THOR is engineered combining interpretable and explainable methods to reveal its source of predictive skill. At the core of THOR, is the identification of dynamically coherent regimes governing the circulation, a fundamental question within oceanography. Three key regions are investigated here. First, the North Atlantic circulation that delivers heat to the higher latitudes is seen to weaken and we identify associated dynamical changes. Second, the Southern Ocean circulation, the strongest circulation on earth, is seen to intensify where we reveal the implications for interactions with the ice on Antarctica. Third, shifts in ocean circulation regimes are identified in the tropical Pacific region, with potential impacts on the El Nino Southern Oscillation, Earth’s dominant source of year-to-year climate variations affecting weather extremes, ecosystems, agriculture, and fisheries. Together with revealing these climatically relevant ocean dynamics, THOR also constitutes a step towards trustworthy machine learning called for within oceanography and beyond because its predictions are physically tractable. We conclude with by highlighting open questions and potentially fruitful avenues of further machine learning applications to climate research.

Authors: Maike Sonnewald (Princeton University); Redouane Lguensat (LSCE-IPSL); Aparna Radhakrishnan (Geophysical Fluid Dynamics Laboratory); Zoubero Sayibou (Bronx Community College); Venkatramani Balaji (Princeton University); Andrew Wittenberg (NOAA)
ICML 2021 Reconstructing Aerosol Vertical Profiles with Aggregate Output Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Aerosol-cloud interactions constitute the largest source of uncertainty in assessments of the anthropogenic climate change. This uncertainty arises in part from the inability to observe aerosol amounts at the cloud formation levels, and, more broadly, the vertical distribution of aerosols. Hence, we often have to settle for less informative two-dimensional proxies, i.e. vertically aggregated data. In this work, we formulate the problem of disaggregation of vertical profiles of aerosols. We propose some initial solutions for such aggregate output regression problem and demonstrate their potential on climate model data.

Authors: Sofija Stefanovic (University of Oxford); Shahine Bouabid (University of Oxford); Philip Stier (University of Oxford); Athanasios Nenes (EPFL); Dino Sejdinovic (University of Oxford)
ICML 2021 Self-Attentive Ensemble Transformer: Representing Ensemble Interactions in Neural Networks for Earth System Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: Ensemble data from Earth system models has to be calibrated and post-processed. I propose a novel member-by-member post-processing approach with neural networks. I bridge ideas from ensemble data assimilation with self-attention, resulting into the self-attentive ensemble transformer. Here, interactions between ensemble members are represented as additive and dynamic self-attentive part. As proof-of-concept, I regress global ECMWF ensemble forecasts to 2-metre-temperature fields from the ERA5 reanalysis. I demonstrate that the ensemble transformer can calibrate the ensemble spread and extract additional information from the ensemble. As it is a member-by-member approach, the ensemble transformer directly outputs multivariate and spatially-coherent ensemble members. Therefore, self-attention and the transformer technique can be a missing piece for a non-parametric post-processing of ensemble data with neural networks.

Authors: Tobias S Finn (Universität Hamburg)
ICML 2021 DroughtED: A dataset and methodology for drought forecasting spanning multiple climate zones (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change exacerbates the frequency, duration and extent of extreme weather events such as drought. Previous attempts to forecast drought conditions using machine learning have focused on regional models which have two major limitations for national drought management: (i) they are trained on localised climate data and (ii) their architectures prevent them from being applied to new heterogeneous regions. In this work, we present a new large-scale dataset for training machine learning models to forecast national drought conditions, named DroughtED. The dataset consists of globally available meteorological features widely used for drought prediction, paired with location meta-data which has not previously been utilised for drought forecasting. Here we also establish a baseline on DroughtED and present the first research to apply deep learning models - Long Short-Term Memory (LSTMs) and Transformers - to predict county-level drought conditions across the full extent of the United States. Our results indicate that DroughtED enables deep learning models to learn cross-region patterns in climate data that contribute to drought conditions and models trained on DroughtED compare favourably to state-of-the-art drought prediction models trained on individual regions.

Authors: Christoph D Minixhofer (The University of Edinburgh); Mark Swan (The University of Edinburgh); Calum McMeekin (The University of Edinburgh); Pavlos Andreadis (The University of Edinburgh)
ICML 2021 Estimation of Air Pollution with Remote Sensing Data: Revealing Greenhouse Gas Emissions from Space (Papers Track)
Abstract and authors: (click to expand)

Abstract: Air pollution is a major driver of climate change. Anthropogenic emissions from the burning of fossil fuels for transportation and power generation emit large amounts of problematic air pollutants, including Greenhouse Gases (GHGs). Despite the importance of limiting GHG emissions to mitigate climate change, detailed information about the spatial and temporal distribution of GHG and other air pollutants is difficult to obtain. Existing models for surface-level air pollution rely on extensive land-use datasets which are often locally restricted and temporally static. This work proposes a deep learning approach for the prediction of ambient air pollution that only relies on remote sensing data that is globally available and frequently updated. Combining optical satellite imagery with satellite-based atmospheric column density air pollution measurements enables the scaling of air pollution estimates (in this case NO2) to high spatial resolution (up to ~10m) at arbitrary locations and adds a temporal component to these estimates. The proposed model performs with high accuracy when evaluated against air quality measurements from ground stations (mean absolute error <6 microgram/m^3). Our results enable the identification and temporal monitoring of major sources of air pollution and GHGs.

Authors: Linus M. Scheibenreif (University of St. Gallen); Michael Mommert (University of St. Gallen); Damian Borth (University of St. Gallen)
ICML 2021 Emulating Aerosol Microphysics with a Machine Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Aerosol particles play an important role in the climate system by absorbing and scattering radiation and influencing cloud properties. They are also one of the biggest sources of uncertainty for climate modeling. Many climate models do not include aerosols in sufficient detail. In order to achieve higher accuracy, aerosol microphysical properties and processes have to be accounted for. This is done in the ECHAM-HAM global climate aerosol model using the M7 microphysics model, but increased computational costs make it very expensive to run at higher resolutions or for a longer time. We aim to use machine learning to approximate the microphysics model at sufficient accuracy and reduce the computational cost by being fast at inference time. The original M7 model is used to generate data of input-output pairs to train a neural network on it. By using a special logarithmic transform we are able to learn the variables tendencies achieving an average $R^2$ score of $89\%$. On a GPU we achieve a speed-up of 120 compared to the original model.

Authors: Paula Harder (Fraunhofer ITWM); Duncan Watson-Parris (University of Oxford); Dominik Strassel (Fraunhofer ITWM); Nicolas Gauger (TU Kaiserslautern); Philip Stier (University of Oxford); Janis Keuper (hs-offenburg)
ICML 2021 Decadal Forecasts with ResDMD: a residual DMD neural network (Papers Track)
Abstract and authors: (click to expand)

Abstract: Significant investment is being made by operational forecasting centers to produce decadal (1-10 year) forecasts that can support long-term decision making for a more climate-resilient society. One method that has been employed for this task is the Dynamic Mode Decomposition (DMD) algorithm – also known as the Linear Inverse Model– which is used to fit linear dynamical models to data. While the DMD usually approximates non-linear terms in the true dynamics as a linear system with random noise, we investigate an extension to the DMD to explicitly represent the non-linear terms as a neural network. Our weight initialization allows the network to produce sensible results before training and then improve the prediction after training as data becomes available. In this short paper, we evaluate the proposed architecture for simulating global sea surface temperatures and compare the results with the standard DMD and seasonal forecasts produced by the state-of-the-art dynamical model, CFSv2.

Authors: EDUARDO ROCHA RODRIGUES (IBM Research); Campbell Watson (IBM Reserch); Bianca Zadrozny (IBM Research); David Gold (IBM Global Business Services)
ICML 2021 Learning Granger Causal Feature Representations (Papers Track)
Abstract and authors: (click to expand)

Abstract: Tackling climate change needs to understand the complex phenomena occurring on the Planet. Discovering teleconnection patterns is an essential part of the endeavor. Events like El Niño Southern Oscillation (ENSO) impact essential climate variables at large distances, and influence the underlying Earth system dynamics. However, their automatic identification from the wealth of observational data is still unresolved. Nonlinearities, nonstationarities and the (ab)use of correlation analyses hamper the discovery of true causal patterns. We here introduce a deep learning methodology that extracts nonlinear latent functions from spatio-temporal Earth data and that are Granger causal with the index altogether. We illustrate its use to study the impact of ENSO on vegetation, which allows for a more rigorous study of impacts on ecosystems globally.

Authors: Gherardo Varando (Universitat de València); Miguel-Ángel Fernández-Torres (Universitat de València); Gustau Camps-Valls (Universitat de València)
ICML 2021 Fast-Slow Streamflow Model Using Mass-Conserving LSTM (Papers Track)
Abstract and authors: (click to expand)

Abstract: Streamflow forecasting is key to effectively managing water resources and preparing for the occurrence of natural calamities being exacerbated by climate change. Here we use the concept of fast and slow flow components to create a new mass-conserving Long Short-Term Memory (LSTM) neural network model. It uses hydrometeorological time series and catchment attributes to predict daily river discharges. Preliminary results evidence improvement in skills for different scores compared to the recent literature.

Authors: Miguel Paredes Quinones (IBM Research); Maciel Zortea (IBM Research); Leonardo Martins (IBM Research)
ICML 2021 Controlling Weather Field Synthesis Using Variational Autoencoders (Papers Track)
Abstract and authors: (click to expand)

Abstract: One of the consequences of climate change is an observed increase in the frequency of extreme climate events. That poses a challenge for weather forecast and generation algorithms, which learn from historical data but should embed an often uncertain bias to create correct scenarios. This paper investigates how mapping climate data to a known distribution using variational autoencoders might help explore such biases and control the synthesis of weather fields towards more extreme climate scenarios. We experimented using a monsoon-affected precipitation dataset from southwest India, which should give a roughly stable pattern of rainy days and ease our investigation. We report compelling results showing that mapping complex weather data to a known distribution implements an efficient control for weather field synthesis towards more (or less) extreme scenarios.

Authors: Dario Augusto Borges Oliveira (IBM Research); Jorge Luis Guevara Diaz (IBM Research); Bianca Zadrozny (IBM Research); Campbell Watson (IBM Reserch)
ICML 2021 Self-supervised Contrastive Learning for Irrigation Detection in Satellite Imagery (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change has caused reductions in river runoffs and aquifer recharge resulting in an increasingly unsustainable crop water demand from reduced freshwater availability. Achieving food security while deploying water in a sustainable manner will continue to be a major challenge necessitating careful monitoring and tracking of agricultural water usage. Historically, monitoring water usage has been a slow and expensive manual process with many imperfections and abuses. Ma-chine learning and remote sensing developments have increased the ability to automatically monitor irrigation patterns, but existing techniques often require curated and labelled irrigation data, which are expensive and time consuming to obtain and may not exist for impactful areas such as developing countries. In this paper, we explore an end-to-end real world application of irrigation detection with uncurated and unlabeled satellite imagery. We apply state-of-the-art self-supervised deep learning techniques to optical remote sensing data, and find that we are able to detect irrigation with up to nine times better precision, 90% better recall and 40% more generalization ability than the traditional supervised learning methods.

Authors: Chitra Agastya (UC Berkeley, IBM); Sirak Ghebremusse (UC Berkeley); Ian Anderson (UC Berkeley); Colorado Reed (UC Berkeley); Hossein Vahabi (University California Berkeley); Alberto Todeschini (UC Berkeley)
ICML 2021 Extreme Precipitation Seasonal Forecast Using a Transformer Neural Network (Papers Track)
Abstract and authors: (click to expand)

Abstract: An impact of climate change is the increase in frequency and intensity of extreme precipitation events. However, confidently predicting the likelihood of extreme precipitation at seasonal scales remains an outstanding challenge. Here, we present an approach to forecasting the quantiles of the maximum daily precipitation in each week up to six months ahead using the temporal fusion transformer (TFT) model. Through experiments in two regions, we compare TFT predictions with those of two baselines: climatology and a calibrated ECMWF SEAS5 ensemble forecast (S5). Our results show that, in terms of quantile risk at six month lead time, the TFT predictions significantly outperform those from S5 and show an overall small improvement compared to climatology. The TFT also responds positively to departures from normal that climatology cannot.

Authors: Daniel Salles Civitarese (IBM Research, Brazil); Daniela Szwarcman (IBM Research); Bianca Zadrozny (IBM Research); Campbell Watson (IBM Reserch)
ICML 2021 Quantification of Carbon Sequestration in Urban Forests (Papers Track)
Abstract and authors: (click to expand)

Abstract: Vegetation, trees in particular, sequester carbon by absorbing carbon dioxide from the atmosphere, however, the lack of efficient quantification methods of carbon stored in trees renders it difficult to track the process. Here we present an approach to estimate the carbon storage in trees based on fusing multispectral aerial imagery and LiDAR data to identify tree coverage, geometric shape, and tree species, which are crucial attributes in carbon storage quantification. We demonstrate that tree species information and their three-dimensional geometric shapes can be estimated from remote imagery in order to calculate the tree's biomass. Specifically, for Manhattan, New York City, we estimate a total of 52,000 tons of carbon sequestered in trees.

Authors: Levente Klein (IBM Research); Wang Zhou (IBM Research); Conrad M Albrecht (IBM Research)
ICML 2021 Refining Ice Layer Tracking through Wavelet combined Neural Networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: Rise in global temperatures is resulting in polar ice caps to melt away, which can lead to drastic sea level rise and coastal floods. Accurate calculation of the ice cap reduction is necessary in order to project its climatic impact. Ice sheets are monitored through Snow Radar sensors which give noisy profiles of subsurface ice layers. The sensors take snapshots of the entire ice sheet regularly, and thus result in large datasets. In this work, we use convolutional neural networks (CNNs) for their property of feature extraction and generalizability on large datasets. We also use wavelet transforms and embed them as a layer in the architecture to help in denoising the radar images and refine ice layer detection. Our results show that incorporating wavelets in CNNs helps in detecting the position of deep subsurface ice layers, which can be used to analyse their change overtime.

Authors: Debvrat Varshney (University of Maryland Baltimore County); Masoud Yari (College of Engineering and Information Technology, University of Maryland Balitimore County); Tashnim Chowdhury (University of Maryland Baltimore County); Maryam Rahnemoonfar (University of Maryland Baltimore County)
ICML 2021 Forecasting Sea Ice Concentrations using Attention-based Ensemble LSTM (Papers Track)
Abstract and authors: (click to expand)

Abstract: Accurately forecasting Arctic sea ice from sub-seasonal to seasonal scales has been a major scientific effort with fundamental challenges at play. In addition to physics-based earth system models, researchers have been applying multiple statistical and machine learning models for sea ice forecasting. Looking at the potential of data-driven sea ice forecasting, we propose an attention-based Long Short Term Memory (LSTM) ensemble method to predict monthly sea ice extent up to 1 month ahead. Using daily and monthly satellite retrieved sea ice data from NSIDC and atmospheric and oceanic variables from ERA5 reanalysis product for 39 years, we show that our multi-temporal ensemble method outperforms several baseline and recently proposed deep learning models. This will substantially improve our ability in predicting future Arctic sea ice changes, which is fundamental for forecasting transporting routes, resource development, coastal erosion, threats to Arctic coastal communities and wildlife.

Authors: Sahara Ali (University of Maryland, Baltimore County); Yiyi Huang (University of Maryland, Baltimore County); Xin Huang (University of Maryland, Baltimore County); Jianwu Wang (University of Maryland, Baltimore County)
ICML 2021 Toward efficient calibration of higher-resolution Earth System Models (Papers Track) Best Paper: Pathway to Impact
Abstract and authors: (click to expand)

Abstract: Projections of future climate change to support decision-making require high spatial resolution, but this is computationally prohibitive with modern Earth system models (ESMs). A major challenge is the calibration (parameter tuning) process, which requires running large numbers of simulations to identify the optimal parameter values. Here we train a convolutional neural network (CNN) on simulations from two lower-resolution (and thus much less expensive) versions of the same ESM, and a smaller number of higher-resolution simulations. Cross-validated results show that the CNN's skill exceeds that of a climatological baseline for most variables with as few as 5-10 examples of the higher-resolution ESM, and for all variables (including precipitation) with at least 20 examples. This proof-of-concept study offers the prospect of significantly more efficient calibration of ESMs, by reducing the required CPU time for calibration by 20-40 %.

Authors: Christopher Fletcher (University of Waterloo); William McNally (University of Waterloo); John Virgin (University of Waterloo)
ICML 2021 Sky Image Prediction Using Generative Adversarial Networks for Solar Forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: Large-scale integration of solar photovoltaics (PV) is challenged by high variability in its power output, mainly due to local and short-term cloud events. To achieve accurate solar forecasting, it is paramount to accurately predict the movement of clouds. Here, we use generative adversarial networks (GANs) to predict future sky images based on past sky image sequences and show that our trained model can generate realistic future sky images and capture the dynamics of clouds in the context frames. The generated images are then evaluated for a downstream solar forecasting task; results show promising performance.

Authors: Yuhao Nie (Stanford University); Andea Scott (Stanford University); Eric Zelikman (Stanford University); Adam Brandt (Stanford University)
ICML 2021 Multivariate climate downscaling with latent neural processes (Papers Track)
Abstract and authors: (click to expand)

Abstract: Statistical downscaling is a vital tool in generating high resolution projections for climate impact studies. This study applies convolutional latent neural processes to multivariate downscaling of maximum temperature and precipitation. In contrast to existing downscaling methods, this model is shown to produce spatially coherent predictions at arbitrary locations specified at test time, regardless of whether training data are available at these points.

Authors: Anna Vaughan (Univeristy of Cambridge); Nic Lane (University of Cambridge); Michael Herzog (University of Cambridge)
ICML 2021 Short-term Hourly Streamflow Prediction with Graph Convolutional GRU Networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: The frequency and impact of floods are expected to increase due to climate change. It is crucial to predict streamflow, consequently flooding, in order to prepare and mitigate its consequences in terms of property damage and fatalities. This paper presents a Graph Convolutional GRUs based model to predict the next 36 hours of streamflow for a sensor location using the upstream river network. As shown in experiment results, the model presented in this study provides better performance than the persistence baseline and a Convolutional Bidirectional GRU network for the selected study area in short-term streamflow prediction.

Authors: Muhammed A Sit (The University of Iowa); Bekir Demiray (The University of Iowa); Ibrahim Demir (The University of Iowa)
ICML 2021 Wildfire Smoke Plume Segmentation Using Geostationary Satellite Imagery (Papers Track)
Abstract and authors: (click to expand)

Abstract: Wildfires have increased in frequency and severity over the past two decades, especially in the Western United States. Beyond physical infrastructure damage caused by these wildfire events, researchers have increasingly identified harmful impacts of particulate matter generated by wildfire smoke on respiratory, cardiovascular, and cognitive health. This inference is difficult due to the spatial and temporal uncertainty regarding how much particulate matter is specifically attributable to wildfire smoke. One factor contributing to this challenge is the reliance on manually drawn smoke plume annotations, which are often noisy representations limited to the United States. This work uses deep convolutional neural networks to segment smoke plumes from geostationary satellite imagery. We compare the performance of predicted plume segmentations versus the noisy annotations using causal inference methods to estimate the amount of variation each explains in Environmental Protection Agency (EPA) measured surface level particulate matter <2.5μm in diameter (PM2.5).

Authors: Jeffrey L Wen (Stanford University); Marshall Burke (Stanford University)
ICML 2021 Deep learning applied to sea surface semantic segmentation: Filtering sunglint from aerial imagery (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Water waves are an ubiquitous feature of the oceans, which serve as a pathway for interactions with the atmosphere. Wave breaking in particular is crucial in developing better understanding of the exchange of momentum, heat, and gas fluxes between the ocean and the atmosphere. Characterizing the properties of wave breaking using orbital or suborbital imagery of the surface of the ocean can be challenging, due to contamination from sunglint, a persistent feature in certain lighting conditions. Here we propose a supervised learning approach to accurately detect whitecaps from airborne imagery obtained under a broad range of lighting conditions. Finally, we discuss potential applications for improving ocean and climate models.

Authors: Teodor Vrecica (UCSD); Quentin Paletta (University of Cambridge); Luc Lenain (UCSD)
ICML 2021 MethaNet - an AI-driven approach to quantifying methane point-source emission from high-resolution 2-D plume imagery (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Methane (CH4) is one of the most powerful anthropogenic greenhouse gases with a significant impact on global warming trajectory and tropospheric air quality. Quantifying an emission rate of observed CH4 plumes from aerial or satellite images is a critical step for understanding the local distributions and subsequently prioritizing mitigation target sites. However, there exists no method that can reliably predict emission rates from detected plumes in real-time without ancillary data. Here, we trained a convolutional neural network model, called MethaNet, to predict methane point-source emission directly from high-resolution 2-D plume images without relying on other local measurements such as background wind speeds. Our results support the basis for the applicability of using deep learning techniques to quantify CH4 point sources in an automated manner over large geographical areas. MethaNet opens the way for real-time monitoring systems, not only for present and future airborne field campaigns but also for upcoming space-based observations in this decade.

Authors: Siraput Jongaramrungruang (Caltech)
ICML 2021 Learning Why: Data-Driven Causal Evaluations of Climate Models (Proposals Track)
Abstract and authors: (click to expand)

Abstract: We plan to use nascent data-driven causal discovery methods to find and compare causal relationships in observed data and climate model output. We will look at ten different features in the Arctic climate collected from public databases and from the Energy Exascale Earth System Model (E3SM). In identifying and comparing the resulting causal networks, we hope to find important differences between observed causal relationships and those in climate models. With these, climate modeling experts will be able to improve the coupling and parameterization of E3SM and other climate models.

Authors: Jeffrey J Nichol (University of New Mexico); Matthew Peterson (Sandia National Laboratories); George M Fricke (UNM); Kara Peterson (Sandia National Laboratories)
ICML 2021 Enhancing Laboratory-scale Flow Imaging of Fractured Geological Media with Deep Learning Super Resolution (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Injection into deep geological formations is a promising approach for the utilization, sequestration, and removal from the atmosphere of CO2 emissions. Laboratory experiments are essential to characterize how CO2 flows and reacts in various types of geological media. We reproduce such dynamic injection processes while imaging using Computed Tomography (CT) at sufficient temporal resolution to visualize changes in the flow field. The resolution of CT, however, is on the order of 100's of micrometers and insufficient to characterize fine-scale reaction-induced alterations to micro-fractures. Super resolution deep learning is, therefore, an essential tool to improve spatial resolution of dynamic CT images. We acquired and processed pairs of multi-scale low- and high-resolution CT rock images. We also show the performance of our baseline model on fractured rock images using peak signal to noise ratio and structural similarity index. Coupling dynamic CT imaging with deep learning results in visualization with enhanced spatial resolution of about a factor of 4 thereby enabling improved interpretation.

Authors: Manju Pharkavi Murugesu (Stanford University); Timothy Anderson (Stanford University); Niccolo Dal Santo (MathWorks, Inc.); Vignesh Krishnan (The MathWorks Ltd); Anthony Kovscek (Stanford University)
ICML 2021 Deep learning network to project future Arctic ocean waves (Proposals Track)
Abstract and authors: (click to expand)

Abstract: The Arctic Ocean is warming at an alarming rate and will likely become ice-free in summer by mid-century. This will be accompanied by higher ocean surface waves, which pose a risk to coastal communities and marine operations. In order to develop climate change adaptation strategies, it is imperative to robustly assess the future changes in the Arctic ocean wave climate. This requires a large ensemble of regional ocean wave projections to properly capture the range of climate modeling uncertainty in the Arctic region. This has been proven challenging, as ocean wave information is typically not provided by climate models, ocean wave numerical modeling is computationally expensive, and most global wave climate ensembles exclude the Arctic region. Here we present a framework to develop a deep learning network based on CNN and LSTM which could be potentially used to obtain such a large ensemble of Arctic wave projections with an affordable cost.

Authors: Merce Casas Prat (Environment and Climate Change Canada); Lluis Castrejon (Mila, Université de Montréal, Facebook AI Research); Shady Moahmmed (University of Ottawa)
ICML 2021 Deep Learning for Spatiotemporal Anomaly Forecasting: A Case Study of Marine Heatwaves (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Spatiotemporal data have unique properties and require specific considerations. Forecasting spatiotemporal processes is a difficult task because the data are high-dimensional, often are limited in extent, and temporally correlated. Hence, we propose to evaluate several deep learning-based approaches that are relevant to spatiotemporal anomaly forecasting. We will use marine heatwaves as a case study. Those are observed around the world and have strong impacts on marine ecosystems. The evaluated deep learning methods will be integrated for the task of marine heatwave prediction in order to overcome the limitations of spatiotemporal data and improve data-driven seasonal marine heatwave forecasts.

Authors: Ding Ning (University of Canterbury); Varvara Vetrova (University of Canterbury); Karin Bryan (University of Waikato); Sébastien Delaux (Meteorological Service of New Zealand)
NeurIPS 2020 Short-Term Solar Irradiance Forecasting Using Calibrated Probabilistic Models (Papers Track)
Abstract and authors: (click to expand)

Abstract: Advancing probabilistic solar forecasting methods is essential to supporting the integration of solar energy into the electricity grid. In this work, we develop a variety of state-of-the-art probabilistic models for forecasting solar irradiance. We investigate the use of post-hoc calibration techniques for ensuring well-calibrated probabilistic predictions. We train and evaluate the models using public data from seven stations in the SURFRAD network, and demonstrate that the best model, NGBoost, achieves higher performance at an intra-hourly resolution than the best benchmark solar irradiance forecasting model across all stations. Further, we show that NGBoost with CRUDE post-hoc calibration achieves comparable performance to a numerical weather prediction model on hourly-resolution forecasting.

Authors: Eric Zelikman (Stanford University); Sharon Zhou (Stanford University); Jeremy A Irvin (Stanford); Cooper Raterink (Stanford University); Hao Sheng (Stanford University); Avati Anand (Stanford University); Jack Kelly (Open Climate Fix); Ram Rajagopal (Stanford University); Andrew Ng (Stanford University); David J Gagne (National Center for Atmospheric Research)
NeurIPS 2020 Deep Learning for Climate Model Output Statistics (Papers Track) Best ML Innovation
Abstract and authors: (click to expand)

Abstract: Climate models are an important tool for the assessment of prospective climate change effects but they suffer from systematic and representation errors, especially for precipitation. Model output statistics (MOS) reduce these errors by fitting the model output to observational data with machine learning. In this work, we explore the feasibility and potential of deep learning with convolutional neural networks (CNNs) for MOS. We propose the CNN architecture ConvMOS specifically designed for reducing errors in climate model outputs and apply it to the climate model REMO. Our results show a considerable reduction of errors and mostly improved performance compared to three commonly used MOS approaches.

Authors: Michael Steininger (University of Würzburg); Daniel Abel (University of Würzburg); Katrin Ziegler (University of Würzburg); Anna Krause (Universität Würzburg, Department of Computer Science, CHair X Data Science); Heiko Paeth (University of Würzburg); Andreas Hotho (Universitat Wurzburg)
NeurIPS 2020 Learning the distribution of extreme precipitation from atmospheric general circulation model variables (Papers Track)
Abstract and authors: (click to expand)

Abstract: Precipitation extremes are projected to become more frequent and severe in a warming atmosphere over the coming decades. However, the accurate prediction of precipitation, in particular of extremes, remains a challenge for numerical weather prediction models. A large source of error are subgrid-scale parameterizations of processes that play a crucial role in the complex, multi-scale dynamics of precipitation, but are not explicitly resolved in the model formulation. Here we follow a hybrid, data-driven approach, in which atmospheric variables such as wind fields are forecast in time by a general circulation model (GCM) ensemble and then mapped to precipitation using a deep convolutional autoencoder. A frequency-based weighting of the loss function is introduced to improve the learning with regard to extreme values. Our results show an improved representation of extreme precipitation frequencies, as well as better error and correlation statistics compared to a state-of-the-art GCM ensemble.

Authors: Philipp Hess (Free University Berlin); Niklas Boers (Free University Berlin)
NeurIPS 2020 Meta-modeling strategy for data-driven forecasting (Papers Track)
Abstract and authors: (click to expand)

Abstract: Accurately forecasting the weather is a key requirement for climate change mitigation. Data-driven methods offer the ability to make more accurate forecasts, but lack interpretability and can be expensive to train and deploy if models are not carefully developed. Here, we make use of two historical climate data sets and tools from machine learning, to accurately predict temperature fields. Furthermore, we are able to use low fidelity models that are cheap to train and evaluate, to selectively avoid expensive high fidelity function evaluations, as well as uncover seasonal variations in predictive power. This allows for an adaptive training strategy for computationally efficient geophysical emulation.

Authors: Dominic J Skinner (MIT); Romit Maulik (Argonne National Laboratory)
NeurIPS 2020 NightVision: Generating Nighttime Satellite Imagery from Infra-Red Observations (Papers Track)
Abstract and authors: (click to expand)

Abstract: The recent explosion in applications of machine learning to satellite imagery often rely on visible images and therefore suffer from a lack of data during the night. The gap can be filled by employing available infra-red observations to generate visible images. This work presents how Deep Learning can be applied successfully to create those images by using U-Net based architectures. The proposed methods show promising results, achieving a structural similarity index (SSIM) up to 86\% on an independent test set and providing visually convincing output images, generated from infra-red observations.

Authors: Paula Harder (Fraunhofer ITWM); William Jones (University of Oxford); Redouane Lguensat (LSCE-IPSL); Shahine Bouabid (University of Oxford); James Fulton (University of Edinburgh); Dánnell Quesada-Chacón (Technische Universität Dresden); Aris Marcolongo (University of Bern); Sofija Stefanovic (University of Oxford); Yuhan Rao (North Carolina Institute for Climate Studies); Peter Manshausen (University of Oxford); Duncan Watson-Parris (University of Oxford)
NeurIPS 2020 Automated Identification of Oil Field Features using CNNs (Papers Track)
Abstract and authors: (click to expand)

Abstract: Oil and gas production sites have been identified as a major source of anthropogenic methane emissions. Emissions studies utilize counts of equipment to estimate emissions from production facilities. However these counts are poorly documented, including both information about well pad locations and major equipment on each well pad. While these data can be found by manually reviewing satellite imagery, it is prohibitively difficult and time consuming. This work, part of a larger study of methane emission studies in Colorado, US, adapted a machine learning (ML) algorithm to detect well pads and associated equipment. Our initial model showed an average well pad detection accuracy of 95% on the Denver-Julesburg (DJ) basin in northeastern Colorado. Our example demonstrates the potential for this type of automated detection from satellite imagery, leading to more accurate and complete models of production emissions.

Authors: SONU DILEEP (Colorado State University); Daniel Zimmerle (Colorado State University); Ross Beveridge (CSU); Timothy Vaughn (Colorado State University)
NeurIPS 2020 Counting Cows: Tracking Illegal Cattle Ranching From High-Resolution Satellite Imagery (Papers Track)
Abstract and authors: (click to expand)

Abstract: Cattle farming is responsible for 8.8\% of greenhouse gas emissions worldwide. In addition to the methane emitted due to their digestive process, the growing need for grazing areas is an important driver of deforestation. While some regulations are in place for preserving the Amazon against deforestation, these are being flouted in various ways. Hence the need to scale and automate the monitoring of cattle ranching activities. Through a partnership with \textit{Anonymous under review}, we explore the feasibility of tracking and counting cattle at the continental scale from satellite imagery. With a license from Maxar Technologies, we obtained satellite imagery of the Amazon at 40cm resolution, and compiled a dataset of 903 images containing a total of 28498 cattle. Our experiments show promising results and highlight important directions for the next steps on both counting algorithms and the data collection processes for solving such challenges.

Authors: Issam Hadj Laradji (Element AI); Pau Rodriguez (Element AI); Alfredo Kalaitzis (University of Oxford); David Vazquez (Element AI); Ross Young (Element AI); Ed Davey (Global Witness); Alexandre Lacoste (Element AI)
NeurIPS 2020 RainBench: Enabling Data-Driven Precipitation Forecasting on a Global Scale (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change is expected to aggravate extreme precipitation events, directly impacting the livelihood of millions. Without a global precipitation forecasting system in place, many regions -- especially those constrained in resources to collect expensive groundstation data -- are left behind. To mitigate such unequal reach of climate change, a solution is to alleviate the reliance on numerical models (and by extension groundstation data) by enabling machine-learning-based global forecasts from satellite imagery. Though prior works exist in regional precipitation nowcasting, there lacks work in global, medium-term precipitation forecasting. Importantly, a common, accessible baseline for meaningful comparison is absent. In this work, we present RainBench, a multi-modal benchmark dataset dedicated to advancing global precipitation forecasting. We establish baseline tasks and release PyRain, a data-handling pipeline to enable efficient processing of decades-worth of data by any modeling framework. Whilst our work serves as a basis for a new chapter on global precipitation forecast from satellite imagery, the greater promise lies in the community joining forces to use our released datasets and tools in developing machine learning approaches to tackle this important challenge.

Authors: Catherine Tong (University of Oxford); Christian A Schroeder de Witt (University of Oxford); Valentina Zantedeschi (GE Global Research); Daniele De Martini (University of Oxford); Alfredo Kalaitzis (University of Oxford); Matthew Chantry (University of Oxford); Duncan Watson-Parris (University of Oxford); Piotr Bilinski (University of Warsaw / University of Oxford)
NeurIPS 2020 FlowDB: A new large scale river flow, flash flood, and precipitation dataset (Papers Track)
Abstract and authors: (click to expand)

Abstract: Flooding results in 8 billion dollars of damage annually in the US and causes the most deaths of any weather related event. Due to climate change scientists expect more heavy precipitation events in the future. However, no current datasets exist that contain both hourly precipitation and river flow data. We introduce a novel hourly river flow and precipitation dataset and a second subset of flash flood events with damage estimates and injury counts. Using these datasets we create two challenges (1) general stream flow forecasting and (2) flash flood damage estimation. We also create a public benchmark and an Python package to enable easy adding of new models . Additionally, in the future we aim to augment our dataset with snow pack data and soil index moisture data to improve predictions

Authors: Isaac Godfried (CoronaWhy)
NeurIPS 2020 FireSRnet: Geoscience-driven super-resolution of future fire risk from climate change (Papers Track)
Abstract and authors: (click to expand)

Abstract: With fires becoming increasingly frequent and severe across the globe in recent years, understanding climate change’s role in fire behavior is critical for quantifying current and future fire risk. However, global climate models typically simulate fire behavior at spatial scales too coarse for local risk assessments. Therefore, we propose a novel approach towards super-resolution (SR) enhancement of fire risk exposure maps that incorporates not only 2000 to 2020 monthly satellite observations of active fires but also local information on land cover and temperature. Inspired by SR architectures, we propose an efficient deep learning model trained for SR on fire risk exposure maps. We evaluate this model on resolution enhancement and find it outperforms standard image interpolation techniques at both 4x and 8x enhancement while having comparable performance at 2x enhancement. We then demonstrate the generalizability of this SR model over northern California and New South Wales, Australia. We conclude with a discussion and application of our proposed model to climate model simulations of fire risk in 2040 and 2100, illustrating the potential for SR enhancement of fire risk maps from the latest state-of-the-art climate models.

Authors: Tristan C Ballard (Sust Global, Stanford University); Gopal Erinjippurath (Sust Global)
NeurIPS 2020 EarthNet2021: A novel large-scale dataset and challenge for forecasting localized climate impacts (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change is global, yet its concrete impacts can strongly vary between different locations in the same region. Seasonal weather forecasts currently operate at the mesoscale (> 1 km). For more targeted mitigation and adaptation, modelling impacts to < 100 m is needed. Yet, the relationship between driving variables and Earth’s surface at such local scales remains unresolved by current physical models. Large Earth observation datasets now enable us to create machine learning models capable of translating coarse weather information into high-resolution Earth surface forecasts encompassing localized climate impacts. Here, we define high-resolution Earth surface forecasting as video prediction of satellite imagery conditional on mesoscale weather forecasts. Video prediction has been tackled with deep learning models. Developing such models requires analysis-ready datasets. We introduce EarthNet2021, a new, curated dataset containing target spatio-temporal Sentinel 2 satellite imagery at 20 m resolution, matched with high-resolution topography and mesoscale (1.28 km) weather variables. With over 32000 samples it is suitable for training deep neural networks. Comparing multiple Earth surface forecasts is not trivial. Hence, we define the EarthNetScore, a novel ranking criterion for models forecasting Earth surface reflectance. For model intercomparison we frame EarthNet2021 as a challenge with four tracks based on different test sets. These allow evaluation of model validity and robustness as well as model applicability to extreme events and the complete annual vegetation cycle. In addition to forecasting directly observable weather impacts through satellite-derived vegetation indices, capable Earth surface models will enable downstream applications such as crop yield prediction, forest health assessments, coastline management, or biodiversity monitoring. Find data, code, and how to participate at www.earthnet.tech .

Authors: Christian Requena-Mesa (Computer Vision Group, Friedrich Schiller University Jena; DLR Institute of Data Science, Jena; Max Planck Institute for Biogeochemistry, Jena); Vitus Benson (Max-Planck-Institute for Biogeochemistry); Jakob Runge (Institute of Data Science, German Aerospace Center (DLR)); Joachim Denzler (Computer Vision Group, Friedrich Schiller University Jena, Germany); Markus Reichstein (Max Planck Institute for Biogeochemistry, Jena; Michael Stifel Center Jena for Data-Driven and Simulation Science, Jena)
NeurIPS 2020 Machine Learning Climate Model Dynamics: Offline versus Online Performance (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate models are complicated software systems that approximate atmospheric and oceanic fluid mechanics at a coarse spatial resolution. Typical climate forecasts only explicitly resolve processes larger than 100 km and approximate any process occurring below this scale (e.g. thunderstorms) using so-called parametrizations. Machine learning could improve upon the accuracy of some traditional physical parametrizations by learning from so-called global cloud-resolving models. We compare the performance of two machine learning models, random forests (RF) and neural networks (NNs), at parametrizing the aggregate effect of moist physics in a 3 km resolution global simulation with an atmospheric model. The NN outperforms the RF when evaluated offline on a testing dataset. However, when the ML models are coupled to an atmospheric model run at 200 km resolution, the NN-assisted simulation crashes with 7 days, while the RF-assisted simulations remain stable. Both runs produce more accurate weather forecasts than a baseline configuration, but globally averaged climate variables drift over longer timescales.

Authors: Noah D Brenowitz (Vulcan Inc.); Brian Henn (Vulcan, Inc.); Spencer Clark (Vulcan, Inc.); Anna Kwa (Vulcan, Inc.); Jeremy McGibbon (Vulcan, Inc.); W. Andre Perkins (Vulcan, Inc.); Oliver Watt-Meyer (Vulcan, Inc.); Christopher S. Bretherton (Vulcan, Inc.)
NeurIPS 2020 VConstruct: Filling Gaps in Chl-a Data Using a Variational Autoencoder (Papers Track)
Abstract and authors: (click to expand)

Abstract: Remote sensing of Chlorophyll-a is vital in monitoring climate change. Chlorphylla measurements give us an idea of the algae concentrations in the ocean, which lets us monitor ocean health. However, a common problem is that the satellites used to gather the data are commonly obstructed by clouds and other artifacts. This means that time series data from satellites can suffer from spatial data loss. There are a number of algorithms that are able to reconstruct the missing parts of these images to varying degrees of accuracy, with Data INterpolating Empirical Orthogonal Functions (DINEOF) being the current standard. However, DINEOF is slow, suffers from accuracy loss in temporally homogenous waters, reliant on temporal data, and only able to generate a single potential reconstruction. We propose a machine learning approach to reconstruction of Chlorophyll-a data using a Variational Autoencoder (VAE). Our accuracy results to date are competitive with but slightly less accurate than DINEOF. We show the benefits of our method including vastly decreased computation time and ability to generate multiple potential reconstructions. Lastly, we outline our planned improvements and future work.

Authors: Matthew Ehrler (University of Victoria); Neil Ernst (University of Victoria)
NeurIPS 2020 A Comparison of Data-Driven Models for Predicting Stream Water Temperature (Papers Track)
Abstract and authors: (click to expand)

Abstract: Changes to the Earth's climate are expected to negatively impact water resources in the future. It is important to have accurate modelling of river flow and water quality to make optimal decisions for water management. Machine learning and deep learning models have become promising methods for making such hydrological predictions. Using these models, however, requires careful consideration both of data constraints and of model complexity for a given problem. Here, we use machine learning (ML) models to predict monthly stream water temperature records at three monitoring locations in the Northwestern United States with long-term datasets, using meteorological data as predictors. We fit three ML models: a Multiple Linear Regression, a Random Forest Regression, and a Support Vector Regression, and compare them against two baseline models: a persistence model and historical model. We show that all three ML models are reasonably able to predict mean monthly stream temperatures with root mean-squared errors (RMSE) ranging from 0.63-0.91 degrees Celsius. Of the three ML models, Support Vector Regression performs the best with an error of 0.63-0.75 degrees Celsius. However, all models perform poorly on extreme values of water temperature. We identify the need for machine learning approaches for predicting extreme values for variables such as water temperature, since it has significant implications for stream ecosystems and biota.

Authors: Helen Weierbach (Lawrence Berkeley); Aranildo Lima (Aquatic Informatics); Danielle Christianson (Lawrence Berkeley National Lab); Boris Faybishenko (Lawrence Berkeley National Lab); Val Hendrix (Lawrence Berkeley National Lab); Charuleka Varadharajan (Lawrence Berkeley National Lab)
NeurIPS 2020 Automated Salmonid Counting in Sonar Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: The prosperity of salmonids is crucial for several ecological and economic functions. Accurately counting spawning salmonids during their seasonal migration is essential in monitoring threatened populations, assessing the efficacy of recovery strategies, guiding fishing season regulations, and supporting the management of commercial and recreational fisheries. While several different methods exist for counting river fish, they all rely heavily on human involvement, introducing a hefty financial and time burden. In this paper we present an automated fish counting method that utilizes data captured from ARIS sonar cameras to detect and track salmonids migrating in rivers. Our results show that our fully automated system has a 19.3% per-clip error when compared to human counting performance. There is room to improve, but our system can already decrease the amount of time field biologists and fishery managers need to spend manually watching ARIS clips.

Authors: Peter Kulits (Caltech); Angelina Pan (Caltech); Sara M Beery (Caltech); Erik Young (Trout Unlimited); Pietro Perona (California Institute of Technology); Grant Van Horn (Cornell University)
NeurIPS 2020 Short-term PV output prediction using convolutional neural network: learning from an imbalanced sky images dataset via sampling and data augmentation (Papers Track)
Abstract and authors: (click to expand)

Abstract: Integrating photovoltaics (PV) into electricity grids is one of the major pathways towards a low-carbon energy system. However, the biggest challenge is the strong fluctuation in PV power generation. In recent years, sky-image-based PV output prediction using deep neural networks has emerged as a promising approach to alleviate the uncertainty. Despite the research surge in exploring different model architectures, there is currently no study addressing the issue of learning from an imbalanced sky images dataset, the outcome of which would be highly beneficial for improving the reliability of existing and new solar forecasting models. In this study, we train convolutional neural network (CNN) models from an imbalanced sky images dataset for two disparate PV output prediction tasks, i.e., nowcast and forecast. We empirically examine the efficacy of using different sampling and data augmentation approaches to create synthesized dataset for model development. We further apply a three-stage selection approach to determine the optimal sampling approach, data augmentation technique and oversampling rate.

Authors: Yuhao Nie (Stanford University); Ahmed S Zamzam (The National Renewable Energy Laboratory); Adam Brandt (Stanford University)
NeurIPS 2020 Loosely Conditioned Emulation of Global Climate Models With Generative Adversarial Networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate models encapsulate our best understanding of the Earth system, allowing research to be conducted on its future under alternative assumptions of how human-driven climate forces are going to evolve. An important application of climate models is to provide metrics of mean and extreme climate changes, particularly under these alternative future scenarios, as these quantities drive the impacts of climate on society and natural systems. Because of the need to explore a wide range of alternative scenarios and other sources of uncertainties in a computationally efficient manner, climate models can only take us so far, as they require significant computational resources, especially when attempting to characterize extreme events, which are rare and thus demand long and numerous simulations in order to accurately represent their changing statistics. Here we use deep learning in a proof of concept that lays the foundation for emulating global climate model output for different scenarios. We train two "loosely conditioned" Generative Adversarial Networks (GANs) that emulate daily precipitation output from a fully coupled Earth system model: one GAN modeling Fall-Winter behavior and the other Spring-Summer. Our GANs are trained to produce spatiotemporal samples: 32 days of precipitation over a 64x128 regular grid discretizing the globe. We evaluate the generator with a set of related performance metrics based upon KL divergence, and find the generated samples to be nearly as well matched to the test data as the validation data is to test. We also find the generated samples to accurately estimate the mean number of dry days and mean longest dry spell in the 32 day samples. Our trained GANs can rapidly generate numerous realizations at a vastly reduced computational expense, compared to large ensembles of climate models, which greatly aids in estimating the statistics of extreme events.

Authors: Alex Ayala (Western Washington University); Chris Drazic (Western Washington University); Brian Hutchinson (Western Washington University); Ben Kravitz (Indiana University); Claudia Tebaldi (Joint Global Change Research Institute)
NeurIPS 2020 Understanding global fire regimes using Artificial Intelligence (Papers Track)
Abstract and authors: (click to expand)

Abstract: Improved understanding of fire activity and its influencing factors will impact the way we interact and coexist with not only the fire itself but also with the ecosystem as a whole. We consolidate more than 20 million wildfire records between 2000 and 2018 across the six continents. This data is processed with artificial intelligence methods to discover global fire regimes, areas with characteristic fire behavior over long periods. We discover 15 groups with clear differences in fire-related historical behavior. Despite sharing historical fire behavior, regions belonging to the same group present significant differences in location and influencing factors. Groups are further divided into 62 regimes based on spatial aggregation patterns, providing a comprehensive characterization. This allows an interpretation of how a combination of vegetation, climate, and demographic features results in a specific fire regime. The current work expands on existing classification efforts and is a step forward in addressing the complex challenge of characterizing global fire regimes.

Authors: Cristobal Pais (University of California Berkeley); Jose-Ramon Gonzalez (CTFC); Pelagy Moudio (University of California Berkeley); Jordi Garcia-Gonzalo (CTFC); Marta C. González (Berkeley); Zuo-Jun Shen (University of California, Berkeley)
NeurIPS 2020 Towards Data-Driven Physics-Informed Global Precipitation Forecasting from Satellite Imagery (Papers Track)
Abstract and authors: (click to expand)

Abstract: Under the effects of global warming, extreme events such as floods and droughts are increasing in frequency and intensity. This trend directly affects communities and make all the more urgent widening the access to accurate precipitation forecasting systems for disaster preparedness. Nowadays, weather forecasting relies on numerical models necessitating massive computing resources that most developing countries cannot afford. Machine learning approaches are still in their infancy but already show the promise for democratizing weather predictions, by leveraging any data source and requiring less compute. In this work, we propose a methodology for data-driven and physics-aware global precipitation forecasting from satellite imagery. To fully take advantage of the available data, we design the system as three elements: 1. The atmospheric state is estimated from recent satellite data. 2. The atmospheric state is propagated forward in time. 3. The atmospheric state is used to derive the precipitation intensity within a nearby time interval. In particular, our use of stochastic methods for forecasting the atmospheric state represents a novel application in this domain.

Authors: Valentina Zantedeschi (GE Global Research); Daniele De Martini (University of Oxford); Catherine Tong (University of Oxford); Christian A Schroeder de Witt (University of Oxford); Piotr Bilinski (University of Warsaw / University of Oxford); Alfredo Kalaitzis (University of Oxford); Matthew Chantry (University of Oxford); Duncan Watson-Parris (University of Oxford)
NeurIPS 2020 Long-Range Seasonal Forecasting of 2m-Temperature with Machine Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: A significant challenge in seasonal climate prediction is whether a prediction can beat climatology. We hereby present results from two data-driven models - a convolutional (CNN) and a recurrent (RNN) neural network - that predict 2m temperature out to 52 weeks for six geographically-diverse locations. The motivation for testing the two classes of ML models is to allow the CNN to leverage information related to teleconnections and the RNN to leverage long-term historical temporal signals. The ML models boast improved accuracy of long-range temperature forecasts up to a lead time of 30 weeks for PCC and up 52 weeks for RMSESS, however only for select locations. Further iteration is required to ensure the ML models have value beyond regions where the climatology has a noticeably reduced correlation skill, namely the tropics.

Authors: Etienne E Vos (IBM); Ashley Gritzman (IBM); Sibusisiwe Makhanya (IBM Research); Thabang Mashinini (IBM); Campbell Watson (IBM)
NeurIPS 2020 Hyperspectral Remote Sensing of Aquatic Microbes to Support Water Resource Management (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Harmful algal blooms in drinking water supply and at recreational sites endanger human health. Excessive algal growth can result in low oxygen environments, making them uninhabitable for fish and other aquatic life. Harmful algae and algal blooms are predicted to increase in frequency and extent due to the warming climate, but microbial dynamics remain difficult to predict. Existing satellite remote sensing monitoring technologies are ill-equipped to discriminate harmful algae, while models do not adequately capture the complex controls on algal populations. This proposal explores the potential for Bayesian neural networks to detect phytoplankton pigments from hyperspectral remote sensing reflectance retrievals. Once developed, such a model could enable hyperspectral remote sensing retrievals to support decision making in water resource management as more advanced ocean color satellites are launched in the coming decade. While uncertainty quantification motivates the proposed use of Bayesian models, the interpretation of these uncertainties in an operational context must be carefully considered.

Authors: Grace E Kim (Booz Allen Hamilton); Evan Poworoznek (NASA GSFC); Susanne Craig (NASA GSFC)
NeurIPS 2020 HECT: High-Dimensional Ensemble Consistency Testing for Climate Models (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Climate models play a crucial role in understanding the effect of environmental and man-made changes on climate to help mitigate climate risks and inform governmental decisions. Large global climate models such as the Community Earth System Model (CESM), developed by the National Center for Atmospheric Research, are very complex with millions of lines of code describing interactions of the atmosphere, land, oceans, and ice, among other components. As development of the CESM is constantly ongoing, simulation outputs need to be continuously controlled for quality. To be able to distinguish a ``climate-changing'' modification of the code base from a true climate-changing physical process or intervention, there needs to be a principled way of assessing statistical reproducibility that can handle both spatial and temporal high-dimensional simulation outputs. Our proposed work uses probabilistic classifiers like tree-based algorithms and deep neural networks to perform a statistically rigorous goodness-of-fit test of high-dimensional spatio-temporal data.

Authors: Niccolo Dalmasso (Carnegie Mellon University); Galen Vincent (Carnegie Mellon University); Dorit Hammerling (Colorado School of Mines); Ann Lee (Carnegie Mellon University)
NeurIPS 2020 Graph Neural Networks for Improved El Niño Forecasting (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Deep learning-based models have recently outperformed state-of-the-art seasonal forecasting models, such as for predicting El Ni\~no-Southern Oscillation (ENSO). However, current deep learning models are based on convolutional neural networks which are difficult to interpret and can fail to model large-scale atmospheric patterns called teleconnections. Hence, we propose the application of spatiotemporal Graph Neural Networks (GNN) to forecast ENSO at long lead times, finer granularity and improved predictive skill than current state-of-the-art methods. The explicit modeling of information flow via edges may also allow for more interpretable forecasts. Preliminary results are promising and outperform state-of-the art systems for projections 1 and 3 months ahead.

Authors: Salva Rühling Cachay (Technical University of Darmstadt); Emma Erickson (University of Illinois at Urbana-Champaign); Arthur F C Bucker (University of São Paulo); Ernest J Pokropek (Warsaw University of Techology); Willa Potosnak (Duquesne University); Salomey Osei (African Master's of Machine Intelligence(AMMI-GH)); Björn Lütjens (MIT)
NeurIPS 2020 Machine Learning towards a Global Parameterisation of Atmospheric New Particle Formation and Growth (Proposals Track)
Abstract and authors: (click to expand)

Abstract: New particle formation (NPF) and growth in the atmosphere affects climate, weather, air quality, and human health. It is the first step of the complex process leading to cloud condensation nuclei (CCN) formation. Even though there is a wealth of observations from field measurements (in forests, high-altitude, polar regions, coastal and urban sites, aircraft campaigns), as well as laboratory studies of multi-component nucleation (including the CLOUD chamber at CERN), and improved nucleation theories, the NPF parameterisations in regional and global models are lacking. These deficiencies make the impacts of aerosols one of the highest sources of uncertainty in global climate change modelling, and associated impacts on weather and human health. We propose to use Machine Learning methods to overcome the challenges in modelling aerosol nucleation and growth, by ingesting the data from the multitude of available sources to create a single parameterisation applicable throughout the modelled atmosphere (troposphere and stratosphere at all latitudes) that efficiently encompasses all input ambient conditions and concentrations of relevant species.

Authors: Theodoros Christoudias (Cyprus Institute); Mihalis A Nicolaou (Cyprus Institute)
ICLR 2020 Embedding Hard Physical Constraints in Convolutional Neural Networks for 3D Turbulence (Papers Track)
Abstract and authors: (click to expand)

Abstract: Deep learning approaches have shown much promise for climate sciences, especially in dimensionality reduction and compression of large datasets. A major issue in deep learning of climate phenomena, like geophysical turbulence, is the lack of physical guarantees. In this work, we propose a general framework to directly embed the notion of incompressible fluids into Convolutional Neural Networks, for coarse-graining of turbulence. These \textbf{physics-embedded neural networks} leverage interpretable strategies from numerical methods and computational fluid dynamics to enforce physical laws and boundary conditions by taking advantage the mathematical properties of the underlying equations. We demonstrate results on 3D fully-developed turbulence, showing that the \textit{physics-aware inductive bias} drastically improves local conservation of mass, without sacrificing performance according to several other metrics characterizing the fluid flow.

Authors: Arvind T Mohan (Los Alamos National Laboratory); NIcholas Lubbers (Los Alamos National Laboratory); Daniel Livescu (Los Alamos National Laboratory); Misha Chertkov (University of Arizona)
ICLR 2020 DETECTION OF HOUSING AND AGRICULTURE AREAS ON DRY-RIVERBEDS FOR THE EVALUATION OF RISK BY LANDSLIDES USING LOW-RESOLUTION SATELLITE IMAGERY BASED ON DEEP LEARNING. STUDY ZONE: LIMA, PERU (Papers Track)
Abstract and authors: (click to expand)

Abstract: The expansion of human settlements in Peru has caused risk exposure to landslides. However, this risk could increase because the intensity of the El niño phenomenon will be greater in the coming years, increasing rainfall on the Peruvian coast. In this paper, we present a novel methodology for detecting housing areas and agricultural lands in low-resolution satellite imagery in order to analyze potential risk in case of unexpected landslides. It was developed by creating two datasets from Lima Metropolitana in Peru, one of which is for detecting dry riverbeds and agriculture lands, and the other for classifying housing areas. We applied data augmentation based on geometrical methods and trained architectures based on U-net methods separately and then, overlap the results for risk assessment. We found that there are areas with significant potential risk that have been classified by the Peruvian government as medium or low risk areas. On this basis, it is recommended obtain a dataset with better resolution that can identify how many housing areas will be affected and take the appropriate prevention measures. Further research in post-processing is needed for suppress noise in our results.

Authors: Brian Cerrón (National University of Engineering); Cristopher Bazan (National University of Engineering); Alberto Coronado (National University of Engineering)
ICLR 2020 WeatherBench: A benchmark dataset for data-driven weather forecasting (Papers Track) Best Paper Award
Abstract and authors: (click to expand)

Abstract: Accurate weather forecasts are a crucial prerequisite for climate change adaptation. Can these be provided by deep learning? First studies show promise, but the lack of a common dataset and evaluation metrics make inter-comparison between the proposed models difficult. In fact, despite the recent research surge in data-driven weather forecasting, there is currently no standard approach for evaluating the proposed models. Here we introduce WeatherBench, a benchmark dataset for data-driven medium-range weather forecasting. We provide data derived from an archive of assimilated earth observations for the last 40 years that has been processed to facilitate the use in machine learning models. We propose a simple and clear evaluation metric which will enable a direct comparison between different proposed methods. Further, we provide baseline scores from simple linear regression techniques, purely physical forecasting models as well as existing deep learning weather forecasting models. All data and code are made publicly available along with tutorials for getting started. We believe WeatherBench will provide a useful and reproducible way of evaluating data-driven weather forecasting models and we hope that it will accelerate research in this direction.

Authors: Stephan Rasp (Technical University of Munich); Soukayna Mouatadid (University of Toronto); Peter Dueben (European Centre for Medium-Range Weather Forecasts (ECMWF)); Sebastian Scher (Stockholm University); Jonathan Weyn (University of Washington); Nils Thuerey (nils.thuerey@tum.de)
ICLR 2020 Modeling Cloud Reflectance Fields using Conditional Generative Adversarial Networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: We introduce a conditional Generative Adversarial Network (cGAN) approach to generate cloud reflectance fields (CRFs) conditioned on large scale meteorological variables such as sea surface temperature and relative humidity. We show that our trained model can generate realistic CRFs from the corresponding meteorological observations, which represents a step towards a data-driven framework for stochastic cloud parameterization.

Authors: Victor Schmidt (Mila); Mustafa Alghali Muhammed (University of Khartoum); Kris Sankaran (Montreal Institute for Learning Algorithms); Tianle Yuan (NASA); Yoshua Bengio (Mila)
ICLR 2020 Prediction of Bayesian Intervals for Tropical Storms (Papers Track)
Abstract and authors: (click to expand)

Abstract: Building on recent research for prediction of hurricane trajectories using recurrent neural networks (RNNs), we have developed improved methods and generalized the approach to predict a confidence interval region of the trajectory utilizing Bayesian methods. Tropical storms are capable of causing severe damage, so accurately predicting their trajectories can bring significant benefits to cities and lives, especially as they grow more intense due to climate change effects. By implementing the Bayesian confidence interval using dropout in an RNN, we improve the actionability of the predictions, for example by estimating the areas to evacuate in the landfall region. We used an RNN to predict the trajectory of the storms at 6-hour intervals. We used latitude, longitude, windspeed, and pressure features from a Statistical Hurricane Intensity Prediction Scheme (SHIPS) dataset of about 500 tropical storms in the Atlantic Ocean. Our results show how neural network dropout values affect our predictions and Bayesian intervals.

Authors: Max Chiswick (Independent); Sam Ganzfried (Ganzfried Research)
ICLR 2020 Wavelet-Powered Neural Networks for Turbulence (Papers Track)
Abstract and authors: (click to expand)

Abstract: One of the fundamental driving phenomena for climate effects is fluid turbulence in geophysical flows. Modeling these flows and explaining its associated spatio-temporal phenomena are notoriously difficult tasks. Navier-Stokes (NS) equations describe all the details of the fluid motions, but require accounting for unfeasibly many degrees of freedom in the regime of developed turbulence. Model reduction and surrogate modeling of turbulence is a general methodology aiming to circumvent this curse of dimensionality. Originally driven by phenomenological considerations, multiple attempts to model-reduce NS equations got a new boost recently with Deep Learning (DL), trained on the ground truth data, e.g. extracted from high-fidelity Direct Numerical Simulations (DNS). However, early attempts of building NNs to model turbulence has also revealed its lack of interpretability as the most significant shortcoming. In this paper we address the key challenge of devising reduced but, at least partially, interpretable model. We take advantage of the balance between strong mathematical foundations and the physical interpretability of wavelet theory to build a spatio-temporally reduced dynamical map which fuses wavelet based spatial decomposition with spatio-temporal modeling based on Convolutional Long Short Term Memory (C-LSTM) architecture. It is shown that the wavelet-based NN makes progress in scaling to large flows, by reducing computational costs and GPU memory requirements.

Authors: Arvind T Mohan (Los Alamos National Laboratory); Daniel Livescu (Los Alamos National Laboratory); Misha Chertkov (University of Arizona)
ICLR 2020 Machine Learning Approaches to Safeguarding Continuous Water Supply in the Arid and Semi-arid Lands of Northern Kenya (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Arid and semi-arid regions (ASALs) in developing countries are heavily affected by the effects of global warming and climate change, leading to adverse climatic conditions such as drought and flooding. This paper explores the problem of fresh-water access in northern Kenya and measures being taken to safeguard water access despite these harsh climatic changes. We present an integrated water management and decision-support platform, eMaji Manager, that we developed and deployed in five ASAL counties in northern Kenya to manage waterpoint access for people and livestock. We then propose innovative machine learning methods for understanding waterpoint usage and repair patterns for sensor-instrumented waterpoints (e.g., boreholes). We explore sub-sequence discriminatory models and recurrent neural networks to predict water-point failures, improve repair response times and ultimately support continuous access to water.

Authors: Fred Otieno (IBM); Timothy Nyota (IBM); Isaac Waweru (IBM); Celia Cintas (IBM Research); Samuel C Maina (IBM Research); William Ogallo (IBM Research); Aisha Walcott-Bryant (IBM Research - Africa)
ICLR 2020 Accelerated Data Discovery for Scalable Climate Action (Proposals Track)
Abstract and authors: (click to expand)

Abstract: According to the Intergovernmental Panel on Climate Change (IPCC), the planet must decarbonize by 50% by 2030 in order to keep global warming below 1.5C. This goal calls for a prompt and massive deployment of solutions in all societal sectors - research, governance, finance, commerce, health care, consumption. One challenge for experts and non-experts is access to the rapidly growing body of relevant information, which is currently scattered across many weakly linked domains of expertise. We propose a large-scale, semi-automatic, AI-based discovery system to collect, tag, and semantically index this information. The ultimate goal is a near real-time, partially curated data catalog of global climate information for rapidly scalable climate action.

Authors: Henning Schwabe (Private); Sumeet Sandhu (Elementary IP LLC); Sergy Grebenschikov (Private)
ICLR 2020 Using ML to close the vocabulary gap in the context of environment and climate change in Chichewa (Proposals Track)
Abstract and authors: (click to expand)

Abstract: In the west, alienation from nature and deteriorating opportunities to experience it, have led educators to incorporate educational programs in schools, to bring pupils in contact with nature and to enhance their understanding of issues related to the environment and its protection. In Africa, and in Malawi, where most people engage in agriculture, and spend most of their time in the 'outdoors', alienation from nature is happening too, although in different ways. Large portion of the indigenous vocabulary and knowledge remains unknown or is slowly disappearing, and there is a need to build a glossary of terms regarding environment and climate change in the vernacular to improve the dialog regarding climate change and environmental protection.. We believe that ML has a role to play in closing the ‘vocabulary gap’ of terms and concepts regarding the environment and climate change that exists in Chichewa and other Malawian languages by helping to creating a visual dictionary of key terms used to describe the environment and explain the issues involved in climate change and their meaning. Chichewa is a descriptive language, one English term may be translated using several words. Thus, the task is not to detect just literal translations, but also translations by means of ‘descriptions’ and illustrations and thus extract correspondence between terms and definitions and to measure how appropriate a term is to convey the meaning intended. As part of this project, ML can be used to identify ‘loanword patterns’, which may be useful in understanding the transmission of cultural items.

Authors: Amelia Taylor (University of Malawi, The Polytechnic)
NeurIPS 2019 Using LSTMs for climate change assessment studies on droughts and floods (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change affects occurrences of floods and droughts worldwide. However, predicting climate impacts over individual watersheds is difficult, primarily because accurate hydrological forecasts require models that are calibrated to past data. In this work we present a large-scale LSTM-based modeling approach that - by training on large data sets - learns a diversity of hydrological behaviors. Previous work shows that this model is more accurate than current state-of-the-art models, even when the LSTM-based approach operates out-of-sample and the latter in-sample. In this work, we show how this model can assess the sensitivity of the underlying systems with regard to extreme (high and low) flows in individual watersheds over the continental US.

Authors: Frederik Kratzert (LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Austria); Daniel Klotz (LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Austria); Johannes Brandstetter (LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Austria); Pieter-Jan Hoedt (Johannes Kepler University Linz); Grey Nearing (Department of Geological Sciences, University of Alabama, Tuscaloosa, AL United States); Sepp Hochreiter (LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Austria)
NeurIPS 2019 Learning to Focus and Track Hurricanes (Papers Track)
Abstract and authors: (click to expand)

Abstract: This paper tackles the task of extreme climate event tracking. We propose a simple but robust end-to-end model based on multi-layered ConvLSTMs, suitable for climate event tracking. It first learns to imprint the location and the appearance of the target at the first frame in an auto-encoding fashion. Next, the learned feature is fed to the tracking module to track the target in subsequent time frames. To tackle the data shortage problem, we propose data augmentation based on conditional generative adversarial networks. Extensive experiments show that the proposed framework significantly improves tracking performance of a hurricane tracking task over several state-of-the-art methods.

Authors: Sookyung Kim (Lawrence Livermore National Laboratory); Sunghyun Park (Korea University); Sunghyo Chung (Kakao Corp.); Joonseok Lee (Google Research); Jaegul Choo (Korea University); Mr Prabhat (Lawrence Berkeley National Laboratory); Yunsung Lee (Korea University)
NeurIPS 2019 Streamflow Prediction with Limited Spatially-Distributed Input Data (Papers Track)
Abstract and authors: (click to expand)

Abstract: Climate change causes more frequent and extreme weather phenomena across the globe. Accurate streamflow prediction allows for proactive and mitigative action in some of these events. As a first step towards models that predict streamflow in watersheds for which we lack ground truth measurements, we explore models that work on spatially-distributed input data. In such a scenario, input variables are more difficult to acquire, and thus models have access to limited training data. We present a case study focusing on Lake Erie, where we find that tree-based models can yield more accurate predictions than both neural and physically-based models.

Authors: Martin Gauch (University of Waterloo); Juliane Mai (University of Waterloo); Shervan Gharari (University of Saskatchewan); Jimmy Lin (University of Waterloo)
NeurIPS 2019 Make Thunderbolts Less Frightening — Predicting Extreme Weather Using Deep Learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Forecasting severe weather conditions is still a very challenging and computationally expensive task due to the enormous amount of data and the complexity of the underlying physics. Machine learning approaches and especially deep learning have however shown huge improvements in many research areas dealing with large datasets in recent years. In this work, we tackle one specific sub-problem of weather forecasting, namely the prediction of thunderstorms and lightning. We propose the use of a convolutional neural network architecture inspired by UNet++ and ResNet to predict thunderstorms as a binary classification problem based on satellite images and lightnings recorded in the past. We achieve a probability of detection of more than 94% for lightnings within the next 15 minutes while at the same time minimizing the false alarm ratio compared to previous approaches.

Authors: Christian Schön (Saarland Informatics Campus); Jens Dittrich (Saarland University)
NeurIPS 2019 Cumulo: A Dataset for Learning Cloud Classes (Papers Track) Best Paper Award
Abstract and authors: (click to expand)

Abstract: One of the greatest sources of uncertainty in future climate projections comes from limitations in modelling clouds and in understanding how different cloud types interact with the climate system. A key first step in reducing this uncertainty is to accurately classify cloud types at high spatial and temporal resolution. In this paper, we introduce Cumulo, a benchmark dataset for training and evaluating global cloud classification models. It consists of one year of 1km resolution MODIS hyperspectral imagery merged with pixel-width `tracks' of CloudSat cloud labels. Bringing these complementary datasets together is a crucial first step, enabling the Machine-Learning community to develop innovative new techniques which could greatly benefit the climate community. To showcase Cumulo, we provide baseline performance analysis using an invertible flow generative model (IResNet), which further allows us to discover new sub-classes for a given cloud class by exploring the latent space.

Authors: Valentina Zantedeschi (Jean Monnet University); Fabrizio Falasca (Georgia Institute of Technology); Alyson Douglas (University of Wisconsin Madison); Richard Strange (University of Oxford); Matt Kusner (University College London); Duncan Watson-Parris (University of Oxford)
NeurIPS 2019 Coupling Oceanic Observation Systems to Study Mesoscale Ocean Dynamics (Papers Track)
Abstract and authors: (click to expand)

Abstract: Understanding local currents in the North Atlantic region of the ocean is a key part of modelling heat transfer and global climate patterns. Satellites provide a surface signature of the temperature of the ocean with a high horizontal resolution while in situ autonomous probes supply high vertical resolution, but horizontally sparse, knowledge of the ocean interior thermal structure. The objective of this paper is to develop a methodology to combine these complementary ocean observing systems measurements to obtain a three-dimensional time series of ocean temperatures with high horizontal and vertical resolution. Within an observation-driven framework, we investigate the extent to which mesoscale ocean dynamics in the North Atlantic region may be decomposed into a mixture of dynamical modes, characterized by different local regressions between Sea Surface Temperature (SST), Sea Level Anomalies (SLA) and Vertical Temperature fields. Ultimately we propose a Latent-class regression method to improve prediction of vertical ocean temperature.

Authors: Gautier Cosne (Mila); Pierre Tandeo (IMT-Atlantique); Guillaume Maze (Ifremer)
NeurIPS 2019 Background noise trends and the detection of calving events in a glacial fjord (Papers Track)
Abstract and authors: (click to expand)

Abstract: Predicting future sea levels depends on accurately estimating the rate at which ice sheets deliver fresh water and ice to the oceans, and projecting rates of iceberg calving will be improved with more observations of calving events. The background noise environment in a glacial fjord was measured and the data were analyzed. This paper includes an analysis of methods useful for evaluating background noise. It explores the utility of spectral probability density in evaluating background noise characteristics in the frequency domain, models probability density functions of spectral levels and introduces a parameter \(\sigma_T\) that quantifies the character of noise in frequency bands of interest. It also explores the utility of k-medoids clustering as a pre-sorting method to inform the selection of features on which to base the training of more complex algorithms.

Authors: Dara Farrell (Graduate of University of Washington)
NeurIPS 2019 Predicting ice flow using machine learning (Papers Track)
Abstract and authors: (click to expand)

Abstract: Though machine learning has achieved notable success in modeling sequential and spatial data for speech recognition and in computer vision, applications to remote sensing and climate science problems are seldom considered. In this paper, we demonstrate techniques from unsupervised learning of future video frame prediction, to increase the accuracy of ice flow tracking in multi-spectral satellite images. As the volume of cryosphere data increases in coming years, this is an interesting and important opportunity for machine learning to address a global challenge for climate change, risk management from floods, and conserving freshwater resources. Future frame prediction of ice melt and tracking the optical flow of ice dynamics presents modeling difficulties, due to uncertainties in global temperature increase, changing precipitation patterns, occlusion from cloud cover, rapid melting and glacier retreat due to black carbon aerosol deposition, from wildfires or human fossil emissions. We show the adversarial learning method helps improve the accuracy of tracking the optical flow of ice dynamics compared to existing methods in climate science. We present a dataset, IceNet, to encourage machine learning research and to help facilitate further applications in the areas of cryospheric science and climate change.

Authors: Yimeng Min (Mila); Surya Karthik Mukkavilli (Mila); Yoshua Bengio (Mila)
NeurIPS 2019 DeepClimGAN: A High-Resolution Climate Data Generator (Papers Track)
Abstract and authors: (click to expand)

Abstract: Earth system models (ESMs), which simulate the physics and chemistry of the global atmosphere, land, and ocean, are often used to generate future projections of climate change scenarios. These models are far too computationally intensive to run repeatedly, but limited sets of runs are insufficient for some important applications, like adequately sampling distribution tails to characterize extreme events. As a compromise, emulators are substantially less expensive but may not have all of the complexity of an ESM. Here we demonstrate the use of a conditional generative adversarial network (GAN) to act as an ESM emulator. In doing so, we gain the ability to produce daily weather data that is consistent with what ESM might output over any chosen scenario. In particular, the GAN is aimed at representing a joint probability distribution over space, time, and climate variables, enabling the study of correlated extreme events, such as floods, droughts, or heatwaves.

Authors: Alexandra Puchko (Western Washington University); Brian Hutchinson (Western Washington University); Robert Link (Joint Global Change Research Institute)
NeurIPS 2019 Measuring Impact of Climate Change on Tree Species: analysis of JSDM on FIA data (Papers Track) Honorable Mention
Abstract and authors: (click to expand)

Abstract: One of the first beings affected by changes in the climate are trees, one of our most vital resources. In this study tree species interaction and the response to climate in different ecological environments is observed by applying a joint species distribution model to different ecological domains in the United States. Joint species distribution models are useful to learn inter-species relationships and species response to the environment. The climates’ impact on the tree species is measured through species abundance in an area. We compare the model’s performance across all ecological domains and study the sensitivity of the climate variables. With the prediction of abundances, tree species populations can be predicted in the future and measure the impact of climate change on tree populations.

Authors: Hyun Choi (University of Florida); Sergio Marconi (University of Florida); Ali Sadeghian (University of Florida); Ethan White (University of Florida); Daisy Zhe Wang (Univeresity of Florida)
NeurIPS 2019 Machine Learning for Precipitation Nowcasting from Radar Images (Papers Track)
Abstract and authors: (click to expand)

Abstract: High-resolution nowcasting is an essential tool needed for effective adaptation to climate change, particularly for extreme weather. As Deep Learning (DL) techniques have shown dramatic promise in many domains, including the geosciences, we present an application of DL to the problem of precipitation nowcasting, i.e., high-resolution (1 km x 1 km) short-term (1 hour) predictions of precipitation. We treat forecasting as an image-to-image translation problem and leverage the power of the ubiquitous UNET convolutional neural network. We find this performs favorably when compared to three commonly used models: optical flow, persistence and NOAA's numerical one-hour HRRR nowcasting prediction.

Authors: Shreya Agrawal (Google); Luke Barrington (Google); Carla Bromberg (Google); John Burge (Google); Cenk Gazen (Google); Jason Hickey (Google)
NeurIPS 2019 Enhancing Stratospheric Weather Analyses and Forecasts by Deploying Sensors from a Weather Balloon (Papers Track)
Abstract and authors: (click to expand)

Abstract: The ability to analyze and forecast stratospheric weather conditions is fundamental to addressing climate change. However, our capacity to collect data in the stratosphere is limited by sparsely deployed weather balloons. We propose a framework to collect stratospheric data by releasing a contrail of tiny sensor devices as a weather balloon ascends. The key machine learning challenges are determining when and how to deploy a finite collection of sensors to produce a useful data set. We decide when to release sensors by modeling the deviation of a forecast from actual stratospheric conditions as a Gaussian process. We then implement a novel hardware system that is capable of optimally releasing sensors from a rising weather balloon. We show that this data engineering framework is effective through real weather balloon flights, as well as simulations.

Authors: Kiwan Maeng (Carnegie Mellon University); Iskender Kushan (Microsoft); Brandon Lucia (Carnegie Mellon University); Ashish Kapoor (Microsoft)
NeurIPS 2019 VideoGasNet: Deep Learning for Natural Gas Methane Leak Classification Using An Infrared Camera (Papers Track)
Abstract and authors: (click to expand)

Abstract: Mitigating methane leakage from the natural gas system have become an increasing concern for climate change. Efficacious methane leak detection and classification can make the mitigation process more efficient and cost effective. Optical gas imaging is widely used for the purpose of leak detection, but it cannot directly provide detection results and leak sizes. Few studies have examined the possibility of leak classification using videos taken by the infrared camera (IR), an optical gas imaging device. In this study, we consider the leak classification problem as a video classification problem and investigated the application of deep learning techniques in methane leak detection. Firstly we collected the first methane leak video dataset - GasVid, which has ~1 M frames of labeled videos of methane leaks from different leaking equipment, covering a wide range of leak sizes (5.3-2051.6 g\ce{CH4}/h) and imaging distances (4.6-15.6 m). Secondly, we studied three deep learning algorithms, including 2D Convolutional Neural Networks (CNN) model, 3D CNN and the Convolutional Long Short Term Memory (ConvLSTM). We find that 3D CNN is the most outstanding and robust architecture, which was named VideoGasNet. The leak-non-leak detection accuracy can reach 100%, and the highest small-medium-large classification accuracy is 78.2% with our 3D CNN network. In summary, VideoGasNet greatly extends the capabilities of IR camera-based leak monitoring system from leak detection only to automated leak classification with high accuracy and fast processing speed, significant mitigation efficiency.

Authors: Jingfan Wang (Stanford University)
NeurIPS 2019 Detecting Avalanche Deposits using Variational Autoencoder on Sentinel-1 Satellite Imagery (Papers Track)
Abstract and authors: (click to expand)

Abstract: Avalanche monitoring is a crucial safety challenge, especially in a changing climate. Remote sensing of avalanche deposits can be very useful to identify avalanche risk zones and time periods, which can in turn provide insights about the effects of climate change. In this work, we use Sentinel-1 SAR (synthetic aperture radar) data on the French Alps for the exceptional winter of 2017-18, with the goal of automatically detecting avalanche deposits. We address our problem with an unsupervised learning technique. We treat an avalanche as a rare event, or an anomaly, and we learn a variational autoencoder, in order to isolate the anomaly. We then evaluate our method on labeled test data, using an independent in-situ avalanche inventory as ground truth. Our empirical results show that our unsupervised method obtains comparable performance to a recent supervised learning approach that trained a convolutional neural network on an artificially balanced version of the same SAR data set along with the corresponding ground-truth labels. Our unsupervised approach outperforms the standard CNN in terms of balanced accuracy (63% as compared to 55%). This is a significant improvement, as it allows our method to be used in-situ by climate scientists, where the data is always very unbalanced (< 2% positives). This is the first application of unsupervised deep learning to detect avalanche deposits.

Authors: Saumya Sinha (University of Colorado, Boulder); Sophie Giffard-Roisin (University of Colorado Boulder); Fatima Karbou (Meteo France); Michael Deschatres (Irstea); Nicolas Eckert (Irstea); Anna Karas (Meteo France); Cécile Coléou (Meteo France); Claire Monteleoni (University of Colorado Boulder)
NeurIPS 2019 A Deep Learning-based Framework for the Detection of Schools of Herring in Echograms (Papers Track)
Abstract and authors: (click to expand)

Abstract: Tracking the abundance of underwater species is crucial for understanding the effects of climate change on marine ecosystems. Biologists typically monitor underwater sites with echosounders and visualize data as 2D images (echograms); they interpret these data manually or semi-automatically, which is time-consuming and prone to inconsistencies. This paper proposes a deep learning framework for the automatic detection of schools of herring from echograms. Experiments demonstrated that our approach outperforms a traditional machine learning algorithm that uses hand-crafted features. Our framework could easily be expanded to detect more species of interest to sustainable fisheries.

Authors: Alireza Rezvanifar (University of Victoria); Tunai Porto Marques (University of Victoria ); Melissa Cote (University of Victoria); Alexandra Branzan Albu (University of Victoria); Alex Slonimer (ASL Environmental Sciences); Thomas Tolhurst (ASL Environmental Sciences ); Kaan Ersahin (ASL Environmental Sciences ); Todd Mudge (ASL Environmental Sciences ); Stephane Gauthier (Fisheries and Oceans Canada)
NeurIPS 2019 Emulating Numeric Hydroclimate Models with Physics-Informed cGANs (Papers Track) Honorable Mention
Abstract and authors: (click to expand)

Abstract: Process-based numerical simulations, including those for climate modeling applications, are compute and resource intensive, requiring extensive customization and hand-engineering for encoding governing equations and other domain knowledge. On the other hand, modern deep learning employs a significantly simpler and more efficient computational workflow, and has been shown impressive results across a myriad of applications in the computational sciences. In this work, we investigate the potential of deep generative learning models, specifically conditional Generative Adversarial Networks (cGANs), to simulate the output of a physics-based model of the spatial distribution of the water content of mountain snowpack - the snow water equivalent (SWE). We show preliminary results indicating that the cGAN model is able to learn diverse mappings between meteorological forcings and SWE output. Thus physics based cGANs provide a means for fast and accurate SWE modeling that can have significant impact in a variety of applications (e.g., hydropower forecasting, agriculture, and water supply management). In climate science, the Snowpack and SWE are seen as some of the best indicative variables for investigating climate change and its impact. The massive speedups, diverse sampling, and sensitivity/saliency modelling that cGANs can bring to SWE estimation will be extremely important to investigating variables linked to climate change as well as predicting and forecasting the potential effects of climate change to come.

Authors: Ashray Manepalli (terrafuse); Adrian Albert (terrafuse, inc.); Alan Rhoades (Lawrence Berkeley National Lab); Daniel Feldman (Lawrence Berkeley National Lab)
NeurIPS 2019 Forecasting El Niño with Convolutional and Recurrent Neural Networks (Papers Track)
Abstract and authors: (click to expand)

Abstract: The El Niño Southern Oscillation (ENSO) is the dominant mode of variability in the climate system on seasonal to decadal timescales. With foreknowledge of the state of ENSO, stakeholders can anticipate and mitigate impacts in climate-sensitive sectors such as agriculture and energy. Traditionally, ENSO forecasts have been produced using either computationally intensive physics-based dynamical models or statistical models that make limiting assumptions, such as linearity between predictors and predictands. Here we present a deep-learning-based methodology for forecasting monthly ENSO temperatures at various lead times. While traditional statistical methods both train and validate on observational data, our method trains exclusively on physical simulations. With the entire observational record as an out-of-sample validation set, the method’s skill is comparable to that of operational dynamical models. The method is also used to identify disagreements among climate models about the predictability of ENSO in a world with climate change.

Authors: Ankur Mahesh (ClimateAi); Maximilian Evans (ClimateAi); Garima Jain (ClimateAi); Mattias Castillo (ClimateAi); Aranildo Lima (ClimateAi); Brent Lunghino (ClimateAi); Himanshu Gupta (ClimateAi); Carlos Gaitan (ClimateAi); Jarrett Hunt (ClimateAi); Omeed Tavasoli (ClimateAi); Patrick Brown (ClimateAi, San Jose State University); V. Balaji (Geophysical Fluid Dynamics Laboratory)
NeurIPS 2019 Deep learning predictions of sand dune migration (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Climate change is making many desert regions warmer, drier, and sandier. These conditions kill vegetation, and release once-stable sand into the wind, allowing it to form dunes that threaten roads, farmland, and solar panel installations. With enough warning, people can mitigate dune damages by moving infrastructure or restoring vegetation. Current dune simulations, however, do not scale well enough to provide useful forecasts for the ~5% of Earth's land surface that is covered by mobile sands. We propose to train a deep learning simulation to emulate the output of a community-standard physics-based dune simulation. We will base the new model on a GAN-based video prediction model with an excellent track record for predicting spatio-temporal patterns to model, and use it to simulate dune topographies over time. Our preliminary work indicates that the new model will run up to ten million times faster than existing dune simulations, which would turn dune modelling from an exercise that covers a handful of dunes to a practical forecast for large desert regions.

Authors: Kelly Kochanski (University of Colorado Boulder); Divya Mohan (University of California Berkeley); Jenna Horrall (James Madison University); Ghaleb Abdulla (Lawrence Livermore National Laboratory)
NeurIPS 2019 FutureArctic - beyond Computational Ecology (Proposals Track)
Abstract and authors: (click to expand)

Abstract: This paper presents the Future Arctic initiative, a multi-disciplinary training network where machine learning researchers and ecologists cooperatively study both long- and short-term responses to future climate in Iceland.

Authors: Steven Latre (UAntwerpen); Dimitri Papadimitriou (UAntwerpen); Ivan Janssens (UAntwerpen); Eric Struyf (UAntwerpen); Erik Verbruggen (UAntwerpen); Ivika Ostonen (UT); Josep Penuelas (UAB); Boris Rewald (RootEcology); Andreas Richter (University of Vienna); Michael Bahn (University of Innsbruck)
NeurIPS 2019 DeepRI: End-to-end Prediction of Tropical Cyclone Rapid Intensification from Climate Data (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Predicting rapid intensification (RI) is extremely critical in tropical cyclone forecasting. Existing deep learning models achieve promising results, however still rely on hand-craft feature. We propose to design an end-to-end deep learning architecture that directly predict RI from raw climate data without intermediate heuristic feature, which allows joint optimization of the whole system for higher performance.

Authors: Renzhi Jing (Princeton University); Ning Lin (Princeton University); Yinda Zhang (Google LLC)
NeurIPS 2019 Autonomous Sensing and Scientific Machine Learning for Monitoring Greenhouse Gas Emissions (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Greenhouse gas emissions are a key driver of climate change. In order to develop and tune climate models, measurements of natural and anthropogenic phenomenon are necessary. Traditional methods (i.e., physical sample collection and ex situ analysis) tend to be sample sparse and low resolution, whereas global remote sensing methods tend to miss small- and mid-scale dynamic phenomenon. In situ instrumentation carried by a robotic platform is suited to study greenhouse gas emissions at unprecedented spatial and temporal resolution. However, collecting scientifically rich datasets of dynamic or transient emission events requires accurate and flexible models of gas emission dynamics. Motivated by applications in seasonal Arctic thawing and volcanic outgassing, we propose the use of scientific machine learning, in which traditional scientific models (in the form of ODEs/PDEs) are combined with machine learning techniques (generally neural networks) to better incorporate data into a structured, interpretable model. Our technical contributions will primarily involve developing these hybrid models and leveraging model uncertainty estimates during sensor planning to collect data that efficiently improves gas emission models in small-data domains.

Authors: Genevieve Flaspohler (MIT); Victoria Preston (MIT); Nicholas Roy (MIT); John Fisher (MIT); Adam Soule (Woods Hole Oceanographic Institution); Anna Michel (Woods Hole Oceanographic Institution)
NeurIPS 2019 Toward Resilient Cities: Using Deep Learning to Downscale Climate Model Projections (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Climate projections from Earth System Models (ESM) are widely used to assess climate change impacts. These projections, however, are too coarse in spatial and temporal resolution (e.g. 25-50 kms, monthly) to be used in local scale resilience studies. High-resolution (<4 km) climate projections at dense temporal resolution (hourly) from multiple Earth System models under various scenarios are necessary to assess potential future changes in climate variables and perform meaningful and robust climate resilience studies. Running ESMs in high-resolution is computationally too expensive, therefore downscaling methods are applied to ESM projections to produce high-resolution projections. Using a regional climate model to downscale climate projections is preferred but dynamically downscaling several ESM projections to < 4km resolution under different scenarios is currently not feasible. In this study, we propose to use a 60 year dynamically downscaled climate dataset with hourly output for the Northeastern United States to train Deep Learning models and achieve a computationally efficient method of downscaling climate projections. This method will allow for more ESM projections to be downscaled to local scales under more scenarios in an efficient manner and significantly improve robustness of regional resilience studies.

Authors: Muge Komurcu (MIT); Zikri Bayraktar (IEEE)
NeurIPS 2019 Predicting Arctic Methane Seeps via Satellite Imagery (Proposals Track)
Abstract and authors: (click to expand)

Abstract: The arctic has seen significant warming and releases of methane, a potent greenhouse gas have been reported. We aim to apply computer vision to satellite imagery in order to quantify geological methane emissions from the permafrost as well as track and predict their change due to increasing temperatures.

Authors: Olya (Olga) Irzak (Frost Methane Labs); Amber Leigh Thomas (Stanford); Stephanie Schneider (Stanford); Catalin Voss (Stanford University)
NeurIPS 2019 A deep learning approach for classifying black carbon aerosol morphology (Proposals Track)
Abstract and authors: (click to expand)

Abstract: Black carbon (BC) is a sub-micron aerosol sourced from incomplete combustion which strongly absorbs solar radiation, leading to both direct and indirect climate impacts. The state-of-the-art technique for characterizing BC is the single particle soot photometer (SP2) instrument, which detects these aerosols in real time via laser-induced incandescence (L-II). This measurement technique allows for quantification of BC mass on a single particle basis, but time-resolved signals may also provide constraints on BC morphology, which impacts both its optical properties and atmospheric lifetime. No methods currently exist to use this information. I propose applying a deep learning based approach to classify the fractal dimension of single BC particles from time-resolved L-II signals. This method would provide the first on-line measurement technique for quantifying BC morphology. These observations could be used to improve representations of BC optical properties and atmospheric processing in climate models.

Authors: Kara Lamb (Cooperative Institute for Research in the Environmental Sciences)
ICML 2019 Focus and track: pixel-wise spatio-temporal hurricane tracking (Research Track)
Abstract and authors: (click to expand)

Abstract: We tackle extreme climate event tracking problem. It has unique challenges to other visual object tracking problems, including wider range of spatio-temporal dynamics, blur boundary of the target, and shortage of labeled dataset. In this paper, we propose a simple but robust end-to-end model based on multi-layered ConvLSTM, suitable for the climate event tracking problem. It first learns to imprint location and appearance of the target at the first frame with one-shot auto-encoding fashion, and then, the learned feature is consumed by the tracking module to track the target in subsequent time frames. To tackle the data shortage problem, we propose data augmentation based on Social GAN. Extensive experiments show that the proposed framework significantly improves tracking performance on hurricane tracking task over several state-of-the-art methods.

Authors: Sookyung Kim (Lawrence Livermore National Laboratory); Sunghyun Park (Korea University); Sunghyo Chung (Korea University); Yunsung Lee (Korea University); Hyojin Kim (LLNL); Joonseok Lee (Google Research); Jaegul Choo (Korea University); Mr Prabhat (Lawrence Berkeley National Laboratory)
ICML 2019 Recovering the parameters underlying the Lorenz-96 chaotic dynamics (Research Track)
Abstract and authors: (click to expand)

Abstract: Climate projections suffer from uncertain equilibrium climate sensitivity. The reason behind this uncertainty is the resolution of global climate models, which is too coarse to resolve key processes such as clouds and convection. These processes are approximated using heuristics in a process called parameterization. The selection of these parameters can be subjective, leading to significant uncertainties in the way clouds are represented in global climate models. Here, we explore three deep network algorithms to infer these parameters in an objective and data-driven way. We compare the performance of a fully-connected network, a one-dimensional and, a two-dimensional convolutional networks to recover the underlying parameters of the Lorenz-96 model, a non-linear dynamical system that has similar behavior to the climate system.

Authors: Soukayna Mouatadid (University of Toronto); Pierre Gentine (Columbia University); Wei Yu (University of Toronto); Steve Easterbrook (University of Toronto)
ICML 2019 A quantum mechanical approach for data assimilation in climate dynamics (Research Track)
Abstract and authors: (click to expand)

Abstract: A framework for data assimilation in climate dynamics is presented, combining aspects of quantum mechanics, Koopman operator theory, and kernel methods for machine learning. This approach adapts the Dirac-von Neumann formalism of quantum dynamics and measurement to perform data assimilation (filtering) of climate dynamics, using the Koopman operator governing the evolution of observables as an analog of the Heisenberg operator in quantum mechanics, and a quantum mechanical density operator to represent the data assimilation state. The framework is implemented in a fully empirical, data-driven manner, using kernel methods for machine learning to represent the evolution and measurement operators via matrices in a basis learned from time-ordered observations. Applications to data assimilation of the Nino 3.4 index for the El Nino Southern Oscillation (ENSO) in a comprehensive climate model show promising results.

Authors: Dimitrios Giannakis (Courant Institute of Mathematical Sciences, New York University); Joanna Slawinska (University of Wisconsin-Milwaukee); Abbas Ourmazd (University of Wisconsin-Milwaukee)
ICML 2019 Targeted Meta-Learning for Critical Incident Detection in Weather Data (Research Track)
Abstract and authors: (click to expand)

Abstract: Due to imbalanced or heavy-tailed nature of weather- and climate-related datasets, the performance of standard deep learning models significantly deviates from their expected behavior on test data. Classical methods to address these issues are mostly data or application dependent, hence burdensome to tune. Meta-learning approaches, on the other hand, aim to learn hyperparameters in the learning process using different objective functions on training and validation data. However, these methods suffer from high computational complexity and are not scalable to large datasets. In this paper, we aim to apply a novel framework named as targeted meta-learning to rectify this issue, and show its efficacy in dealing with the aforementioned biases in datasets. This framework employs a small, well-crafted target dataset that resembles the desired nature of test data in order to guide the learning process in a coupled manner. We empirically show that this framework can overcome the bias issue, common to weather-related datasets, in a bow echo detection case study.

Authors: Mohammad Mahdi Kamani (The Pennsylvania State University); Sadegh Farhang (Pennsylvania State University); Mehrdad Mahdavi (Penn State); James Z Wang (The Pennsylvania State University)
ICML 2019 Improving Subseasonal Forecasting in the Western U.S. with Machine Learning (Research Track)
Abstract and authors: (click to expand)

Abstract: Water managers in the western United States (U.S.) rely on longterm forecasts of temperature and precipitation to prepare for droughts and other wet weather extremes. To improve the accuracy of these long-term forecasts, the Bureau of Reclamation and the National Oceanic and Atmospheric Administration (NOAA) launched the Subseasonal Climate Forecast Rodeo, a year-long real-time forecasting challenge, in which participants aimed to skillfully predict temperature and precipitation in the western U.S. two to four weeks and four to six weeks in advance. We present and evaluate our machine learning approach to the Rodeo and release our SubseasonalRodeo dataset, collected to train and evaluate our forecasting system. Our predictive system is an ensemble of two regression models, and exceeds that of the top Rodeo competitor as well as the government baselines for each target variable and forecast horizon.

Authors: Paulo Orenstein (Stanford); Jessica Hwang (Stanford); Judah Cohen (AER); Karl Pfeiffer (AER); Lester Mackey (Microsoft Research New England)
ICML 2019 A Flexible Pipeline for Prediction of Tropical Cyclone Paths (Research Track)
Abstract and authors: (click to expand)

Abstract: Hurricanes and, more generally, tropical cyclones (TCs) are rare, complex natural phenomena of both scientific and public interest. The importance of understanding TCs in a changing climate has increased as recent TCs have had devastating impacts on human lives and communities. Moreover, good prediction and understanding about the complex nature of TCs can mitigate some of these human and property losses. Though TCs have been studied from many different angles, more work is needed from a statistical approach of providing prediction regions. The current state-of-the-art in TC prediction bands comes from the National Hurricane Center at NOAA, whose proprietary model provides "cones of uncertainty" for TCs through an analysis of historical forecast errors. The contribution of this paper is twofold. We introduce a new pipeline that encourages transparent and adaptable prediction band development by streamlining cyclone track simulation and prediction band generation. We also provide updates to existing models and novel statistical methodologies in both areas of the pipeline respectively.

Authors: Niccolo Dalmasso (Carnegie Mellon University); Robin Dunn (Carnegie Mellon University); Benjamin LeRoy (Carnegie Mellon University); Chad Schafer (Carnegie Mellon University)
ICML 2019 Achieving Conservation of Energy in Neural Network Emulators for Climate Modeling (Research Track)
Abstract and authors: (click to expand)

Abstract: Artificial neural-networks have the potential to emulate cloud processes with higher accuracy than the semi-empirical emulators currently used in climate models. However, neural-network models do not intrinsically conserve energy and mass, which is an obstacle to using them for long-term climate predictions. Here, we propose two methods to enforce linear conservation laws in neural-network emulators of physical models: Constraining (1) the loss function or (2) the architecture of the network itself. Applied to the emulation of explicitly-resolved cloud processes in a prototype multi-scale climate model, we show that architecture constraints can enforce conservation laws to satisfactory numerical precision, while all constraints help the neural-network better generalize to conditions outside of its training set, such as global warming.

Authors: Tom G Beucler (Columbia University & UCI); Stephan Rasp (Ludwig-Maximilian University of Munich); Michael Pritchard (UCI); Pierre Gentine (Columbia University)
ICML 2019 Evaluating aleatoric and epistemic uncertainties of time series deep learning models for soil moisture predictions (Research Track)
Abstract and authors: (click to expand)

Abstract: Soil moisture is an important variable that determines floods, vegetation health, agriculture productivity, and land surface feedbacks to the atmosphere, etc.. The recently available satellite-based observations give us a unique opportunity to directly build data-driven models to predict soil moisture instead of using land surface models, but previously there was no uncertainty estimate. We tested Monte Carlo dropout with an aleatoric term (MCD+A) for our long short-term memory models for this problem, and ask if the uncertainty terms behave as they were argued to. We show that MCD+A indeed gave a good estimate of our predictive error, provided we tune a hyperparameter and use a representative training dataset. The aleatoric term responded strongly to observational noise and the epistemic term clearly acted as a detector for physiographic dissimilarity from the training data. However, when the training and test data are characteristically different, the aleatoric term could be misled, undermining its reliability. We will also discuss some of the major challenges for which we anticipate the geoscientific communities will need help from computer scientists in applying AI to climate or hydrologic modeling.

Authors: Chaopeng Shen (Pennsylvania State University)
ICML 2019 Detecting anthropogenic cloud perturbations with deep learning (Research Track) Best Paper Award
Abstract and authors: (click to expand)

Abstract: One of the most pressing questions in climate science is that of the effect of anthropogenic aerosol on the Earth's energy balance. Aerosols provide the `seeds' on which cloud droplets form, and changes in the amount of aerosol available to a cloud can change its brightness and other physical properties such as optical thickness and spatial extent. Clouds play a critical role in moderating global temperatures and small perturbations can lead to significant amounts of cooling or warming. Uncertainty in this effect is so large it is not currently known if it is negligible, or provides a large enough cooling to largely negate present-day warming by CO2. This work uses deep convolutional neural networks to look for two particular perturbations in clouds due to anthropogenic aerosol and assess their properties and prevalence, providing valuable insights into their climatic effects.

Authors: Duncan Watson-Parris (University of Oxford); Sam Sutherland (University of Oxford); Matthew Christensen (University of Oxford); Anthony Caterini (University of Oxford); Dino Sejdinovic (University of Oxford); Philip Stier (University of Oxford)
ICML 2019 Data-driven surrogate models for climate modeling: application of echo state networks, RNN-LSTM and ANN to the multi-scale Lorenz system as a test case (Research Track)
Abstract and authors: (click to expand)

Abstract: Understanding the effects of climate change relies on physics driven computationally expensive climate models which are still imperfect owing to ineffective subgrid scale parametrization. An effective way to treat these ineffective parametrization of largely uncertain subgrid scale processes are data-driven surrogate models with machine learning techniques. These surrogate models train on observational data capturing either the embed- dings of their (subgrid scale processes’) underlying dynamics on the large scale processes or to simulate the subgrid processes accurately to be fed into the large scale processes. In this paper an extended version of the Lorenz 96 system is studied, which consists of three equations for a set of slow, intermediate, and fast variables, providing a fitting prototype for multi-scale, spatio-temporal chaos, and in particular, the complex dynamics of the climate system. In this work, we have built a data-driven model based on echo state net- works (ESN) aimed, specifically at climate modeling. This model can predict the spatio-temporal chaotic evolution of the Lorenz system for several Lyapunov timescales. We show that the ESN model outperforms, in terms of the prediction horizon, a deep learning technique based on recurrent neural network (RNN) with long short-term memory (LSTM) and an artificial neural network by factors between 3 and 10. The results suggest that ESN has the potential for being a powerful method for surrogate modeling and data-driven prediction for problems of interest to the climate community.

Authors: Ashesh K Chattopadhyay (Rice University); Pedram Hassanzadeh (Rice University); Devika Subramanian (Rice University); Krishna Palem (Rice University); Charles Jiang (Rice University); Adam Subel (Rice University)
ICML 2019 Learning Radiative Transfer Models for Climate Change Applications in Imaging Spectroscopy (Research Track)
Abstract and authors: (click to expand)

Abstract: According to a recent investigation, an estimated 33-50% of the world's coral reefs have undergone degradation, believed to be as a result of climate change. A strong driver of climate change and the subsequent environmental impact are greenhouse gases such as methane. However, the exact relation climate change has to the environmental condition cannot be easily established. Remote sensing methods are increasingly being used to quantify and draw connections between rapidly changing climatic conditions and environmental impact. A crucial part of this analysis is processing spectroscopy data using radiative transfer models (RTMs) which is a computationally expensive process and limits their use with high volume imaging spectrometers. This work presents an algorithm that can efficiently emulate RTMs using neural networks leading to a multifold speedup in processing time, and yielding multiple downstream benefits.

Authors: Shubhankar V Deshpande (Carnegie Mellon University), Brian D Bue (NASA JPL/Caltech), David R Thompson (NASA JPL/Caltech), Vijay Natraj (NASA JPL/Caltech), Mario Parente (UMass Amherst)
ICML 2019 Finding Ship-tracks Using Satellite Data to Enable Studies of Climate and Trade Related Issues (Deployed Track)
Abstract and authors: (click to expand)

Abstract: Ship-tracks appear as long winding linear features in satellite images and are produced by aerosols from ship exhausts changing low cloud properties. They are one of the best examples of aerosol-cloud interaction experiments, which is currently the largest source of uncertainty in our understanding of climate forcing. Manually finding ship-tracks from satellite data on a large-scale is prohibitively costly while a large number of samples are required to better understand aerosol-cloud interactions. Here we train a deep neural network to automate finding ship-tracks. The neural network model generalizes well as it not only finds ship-tracks labeled by human experts, but also detects those that are occasionally missed by humans. It increases our sampling capability of ship-tracks by orders of magnitude and produces a first global map of ship-track distributions using satellite data. Major shipping routes that are mapped by the algorithm correspond well with available commercial data. There are also situations where commercial data are missing shipping routes that are detected by our algorithm. Our technique will enable studying aerosol effects on low clouds using ship-tracks on a large-scale, which will potentially narrow the uncertainty of the aerosol-cloud interactions. The product is also useful for applications such as coastal air pollution and trade.

Authors: Tianle Yuan (NASA)
ICML 2019 Stratospheric Aerosol Injection as a Deep Reinforcement Learning Problem (Ideas Track) Honorable Mention
Abstract and authors: (click to expand)

Abstract: As global greenhouse gas emissions continue to rise, the use of geoengineering in order to artificially mitigate climate change effects is increasingly considered. Stratospheric aerosol injection (SAI), which reduces solar radiative forcing and thus can be used to offset excess radiative forcing due to the greenhouse effect, is both technically and economically feasible. However, naive deployment of SAI has been shown in simulation to produce highly adversarial regional climatic effects in regions such as India and West Africa. Wealthy countries would most likely be able to trigger SAI unilaterally, i.e. China, Russia or US could decide to fix their own climates and, by collateral damage, drying India out by disrupting the monsoon or inducing termination effects with rapid warming. Understanding both how SAI can be optimised and how to best react to rogue injections is therefore of crucial geostrategic interest. In this paper, we argue that optimal SAI control can be characterised as a high-dimensional Markov Decision Process. This motivates the use of deep reinforcement learning in order to automatically discover non-trivial, and potentially time-varying, optimal injection policies or identify catastrophic ones. To overcome the inherent sample inefficiency of deep reinforcement learning, we propose to emulate a Global Circulation Model using deep learning techniques. To our knowledge, this is the first proposed application of deep reinforcement learning to the climate sciences.

Authors: Christian A Schroeder (University of Oxford); Thomas Hornigold (University of Oxford)
ICML 2019 Predicting Marine Heatwaves using Global Climate Models with Cluster Based Long Short-Term Memory (Ideas Track)
Abstract and authors: (click to expand)

Abstract: Marine heatwaves make human and natural systems vulnerable to disaster risk through the disruption of ecological services and biological function. These extreme warming events in sea surface temperature are expected to become more frequent and longer lasting as a result of climate change. Large ensembles of global climate models now provide petabytes of climate-relevant data and an opportunity to probe machine learning to glean new insights about the climate conditions that cause marine heatwaves. Here we propose a k-means cluster based learning objective to map the geography of marine heatwave drivers globally to build a forecast for extreme sea surface temperatures using Long Short-Term Memory. We describe our machine learning approach to predict when and where future marine heatwaves will occur while leveraging the massive output of data from global climate models where traditional forecasting approaches fall short. The impacts of this work could warn coastal communities by providing a forecast for marine heatwaves, which would mitigate the negative effects on fishery productivity, ecosystem health, and tourism.

Authors: Hillary S Scannell (University of Washington); Chris Fraley (Tableau Software); Nathan Mannheimer (Tableau Software); Sarah Battersby (Tableau Software); LuAnne Thompson (University of Washington)
ICML 2019 Harness the Power of Artificial intelligence and -Omics to Identify Soil Microbial Functions in Climate Change Projection (Ideas Track)
Abstract and authors: (click to expand)

Abstract: Contemporary Earth system models (ESMs) omit one of the significant drivers of the terrestrial carbon cycle, soil microbial communities. Soil microbial community not only directly emit greenhouse gasses into the atmosphere through the respiration process, but also release diverse enzymes to catalyze the decomposition of soil organic matter and determine nutrient availability for aboveground vegetation. Therefore, soil microbial community control over terrestrial carbon dynamics and their feedbacks to climate. Currently, inadequate representation of soil microbial communities in ESMs has introduced significant uncertainty in current terrestrial carbon-climate feedbacks. Mitigation of this uncertainty requires to identify functions, diversity, and environmental adaptation of soil microbial communities under global climate change. The revolution of -omics technology allows high throughput quantification of diverse soil enzymes, enabling large-scale studies of microbial functions in climate change. Such studies may lead to revolutionary solutions to predicting microbial-mediated climate-carbon feedbacks at the global scale based on gene-level environmental adaptation strategies of the microbial community. A key initial step in this direction is to identify the biogeography and environmental adaptation of soil enzyme functions based on the massive amount of data generated by -omics technologies. Here we propose to make this step. Artificial intelligence is a powerful, ideal tool for this leap forward. Our project is to integrate Artificial intelligence technologies and global -omics data to represent climate controls on microbial enzyme functions and mapping biogeography of soil enzyme functional groups at global scale. This outcome of this study will allow us to improve the representation of microbial function in earth system modeling and mitigate uncertainty in current climate projection.

Authors: Yang Song (Oak Ridge National Lab); Dali Wang (Oak Ridge National Lab); Melanie Mayes (Oak Ridge National Lab)