Technical Thrust Area
Uncertainty Quantification and Probabilistic Modeling
Committee
Chair: Serge Prudhomme, Polytechnique Montreal
ViceChair: Johann Guilleminot, Duke University
MembersatLarge: JianXun Wang, Notre Dame University Michael Shields, Johns Hopkins University
Members Not a member? Click here to learn how to become one.
Webinar Series
October 19, 2023; 12pm CDT
Join via Zoom: https://us06web.zoom.us/j/92756548524?pwd=cTFoRXIvNVN4dVFoaHEzK0pQQjhldz09 (Meeting ID: 927 5654 8524/Passcode: 934745)
Speaker: Pinar Acar, Virginia Tech
Title: Computational and datadriven design of Materials under uncertainty: Applications to metallic microstructures and beyond
The area of computational and datadriven design of materials has been garnering significant interest due to the increasing need for highperformance materials in electronics, energy and structural applications, and extreme environments. The research on the computational design of materials and its integration into advanced manufacturing techniques will potentially be leveraged in the future to develop newgeneration composites, alloys, ceramics, and other materials for extreme environments such as hypersonics applications, adaptive thermal response materials, energetic composites in fuel cells, thermal energy harvesting in satellites, and materials for green energy applications.
In this talk, Dr. Acar will present an overview of the computational and datadriven methods developed by her research group to design metallic alloys under the effects of uncertainty. The mechanical performance of these materials is enhanced by modeling them in terms of microscale (~106 m) features. The talk will also discuss the impact of manufacturingrelated uncertainty arising from the imperfections and defects during processing on the reliability and performance of these materials. Additional topics will cover the integration of Artificial Intelligence (AI)/Machine Learning (ML) techniques into physicsbased and physicsinformed material models to accelerate the design of different material systems processed with conventional and additive manufacturing techniques.
November 16; 12pm CST
Join via Zoom: https://us06web.zoom.us/j/92756548524?pwd=cTFoRXIvNVN4dVFoaHEzK0pQQjhldz09 (Meeting ID: 927 5654 8524/Passcode: 934745)
Speaker: Sanjay Govindjee, University of California, Berkeley
Title: TBA
December 14; 12pm CST
Join via Zoom: https://us06web.zoom.us/j/92756548524?pwd=cTFoRXIvNVN4dVFoaHEzK0pQQjhldz09 (Meeting ID: 927 5654 8524/Passcode: 934745)
Speaker: Ramin Bostanabad, University of California, Irvine
Title: TBA
Past Webinars
September 21, 2023; 12pm CDT
Speaker: Elisabeth Ullmann, Technical University of Munich
Title: Particle dynamics for rare event estimation with PDEbased models (Not recorded at the request of the presenter)
The estimation of the probability of rare events is an important task in reliability and risk assessment of critical societal systems, for example, groundwater flow and transport, and engineering structures. In this talk we consider rare events that are expressed in terms of a limit state function which depends on the solution of a partial differential equation (PDE). We present two novel estimators for the rare event probability based on (1) the Ensemble Kalman filter for inverse problems, and (2) a consensusbuilding mechanism. Both approaches use particles which follow a suitable stochastic dynamics to reach the failure states. The particle methods have historically been used for Bayesian inverse problems. We connect them to rare event estimation.
This is joint work with Konstantin Althaus, Fabian Wagner and Iason Papaioannou (TUM).
May 25, 2023
Speaker: Laura Swiler, Sandia National Laboratories
Title: Uncertainty Quantification and Sensitivity Analysis Supporting Performance Assessment of Nuclear Waste Repositories Slides
Performance assessment (PA) of geologic disposal of nuclear waste involves the modeling and analysis of underground repositories in the “postclosure” period. The postclosure period can be on the order of hundreds of thousands to millions of years due to the long halflives of the radionuclides present in nuclear waste. The models used for PA are typically coupled multiphysics models such as groundwater flow and transport, heat conduction and convection, geomechanics, and radionuclide decay. The characterization, quantification, and analysis of uncertainty are all integral components of performance assessment.
This talk will present uncertainty and sensitivity analysis methods that have been used or are being considered for use in PA of nuclear waste repositories. These include Sobol’ sensitivity indices calculated using a variety of surrogate models, as well as momentindependent methods. The talk will also present a history of epistemic uncertainty and provide some context for the separation of epistemic and aleatory uncertainty in nuclear power safety assessments and waste repository assessments. Treatment of epistemic uncertainty with DempsterShafer belief structures, interval analysis, and secondorder probability will be discussed. Funding acknowledgment: SNL is managed and operated by NTESS under DOE NNSA contract DENA0003525.
April 20, 2023
Speaker: John D. Jakeman, Sandia National Laboratories
Title: Recent Advances in MultiFidelity Uncertainty Quantification
Recent advances in computational power and numerical algorithms have enabled revolutionary prediction of complex multiscale multiphysics. However, because of their significant computational cost, it remains challenging to use these new highfidelity (highaccuracy) for uncertainty quantification (UQ), which requires repeated evaluation of a model. Addressing this core challenge requires utilizing multiple simulation models and experiments of varying cost and accuracy.
This talk will provide an overview of the multifidelity (MF) strategies for combining limited highfidelity data with a greater amount of lowerfidelity data to substantially increase the accuracy of uncertainty estimates for a limited computational budget. Focus will be given to multifidelity quadrature methods that leverage the correlation between different models, arising from varying numerical discretizations and/or idealized physics, to reduce the cost of computing statistical estimators of uncertainty. Initial discussion will contrast MF methods that assume a hierarchy of models ordered by accuracy per unit costs, e.g. multilevel Monte Carlo (MLMC) [1], with methods that can be applied to unordered model ensembles, e.g approximate control variates (ACV) [2] and multilevel best linear unbiased estimators (MLBLUE) [3]. The talk will then present recent developments in the latter class of nonhierarchical methods. Specifically, we will show that ACV and MLBLUE are equivalent and present a new method for estimating uncertainty that uses multiarm bandits to balance the cost of computing the correlation between models (exploration), needed for ACV and MLBLUE, with the cost of computing the MF estimate of uncertainty (exploitation); the exploration cost is typically ignored by existing methods. The talk will conclude with some vignettes demonstrating the efficacy of MF quadrature on applications in plasma physics and icesheet modeling. References [1] Michael B. Giles. Multilevel monte carlo methods. Acta Numerica, 24:259–328, 2015. [2] A.A. Gorodetsky, G. Geraci, M.S. Eldred, and J.D. Jakeman. A generalized approximate control variate framework for multifidelity uncertainty quantification. Journal of Computational Physics, 408:109257, 2020. [3] Daniel Schaden and Elisabeth Ullmann. On multilevel best linear unbiased estimators. SIAM/ASA Journal on Uncertainty Quantification, 8(2):601–635, 2020.
Join via Zoom: https://us06web.zoom.us/j/92756548524?pwd=cTFoRXIvNVN4dVFoaHEzK0pQQjhldz09 (Meeting ID: 927 5654 8524/Passcode: 934745)
March 23, 2023
Speaker: Somdatta Goswami, Brown University
Title: Integration of Numerical Modeling and Machine Learning in Mechanics
A new paradigm in scientific research has been established with the integration of datadriven and physicsinformed methodologies in the domain of deep learning, and it is certain to have an impact on all areas of science and engineering. This field, popularly termed "Scientific Machine Learning," relies on a known model, some (or no) highfidelity data, and partially known constitutive relationships or closures, to be able to close the gap between the physical models and the observational data. Despite the fact that these strategies have been effective in many fields, they still face significant obstacles, such as the need for accurate and precise knowledge transmission in a datarestricted environment, and the investigation of datadriven methodologies in the centuryold field of mechanics is still in its infancy. The application of deep learning techniques within the context of functional and operator regression to resolve PDEs in mechanics will be the major focus of this presentation. The approaches' extrapolation ability, accuracy, and computing efficiency in big and small data regimes, including transfer learning, would serve as indicators of their effectiveness.
Bio: Somdatta Goswami is an Assistant Professor of Research in the Division of Applied Mathematics at Brown University. Her research is focused on the development of efficient scientific machinelearning algorithms for highdimensional physicsbased systems in the fields of computational mechanics and biomechanics. After completing her Ph.D. at Bauhaus University in Germany under the supervision of Prof. Timon Rabczuk, she joined Brown University as a postdoctoral research associate under the supervision of Prof. George Karniadakis in January 2021.
March 8, 2023
Speaker: Bruno Sudret, ETH Zürich
Computational models, a.k.a. simulators, are used in all fields of engineering and applied sciences to help design and assess complex systems in silico. Advanced analyses such as optimization or uncertainty quantification, which require repeated runs by varying input parameters, cannot be carried out with brute force methods such as Monte Carlo simulation due to computational costs. Thus the recent development of surrogate models such as polynomial chaos expansions and Gaussian processes, among others.
For socalled stochastic simulators used e.g. in epidemiology, mathematical finance or wind turbine design, there exists an intrinsic source of stochasticity on top of wellidentified system parameters. Hence, for a given vector of inputs, repeated runs of the simulator (called replications) will provide different results, as opposed to the case of deterministic simulators. Consequently, for each input realization, the response is a random variable to be characterized.
In this talk we present recent developments in surrogate models for such simulators, which we call stochastic emulators. We first focus on generalized lambda models (GLaM), which combine socalled parametric lambda distributions and polynomial chaos expansions [1]. Stochastic polynomial chaos expansions allows us to address more complex, e.g. multimodal, output distributions [2]. In both cases, no replicated runs of the simulator are needed. Finally, a spectral approach based on a random field representation can be used when trajectories of the stochastic simulator are available [3]. The various methods will be illustrated with engineering application examples.
Acknowledgments: This work was supported by the Swiss National Science Foundation under Grant Number #175524 “SurrogAte Modelling for stOchastic Simulators (SAMOS)”.
References [1] X. Zhu and B. Sudret. Emulation of stochastic simulators using generalized lambda models. SIAM/ASA Journal on Uncertainty Quantification, 9(4):1345–1380, 2021. [2] X. Zhu and B. Sudret. Stochastic polynomial chaos expansions to emulate stochastic simulators. International Journal for Uncertainty Quantification, 13(2):31–52, 2023. [3] N. N. Lüthen, S. Marelli, and B. Sudret. A spectral surrogate model for stochastic simulators computed from trajectory samples. Computer Methods in Applied Mechanics and Engineering, 406(115875):1– 29, 2023.
January 19, 2023
Speaker: Sankaran Mahadevan, Vanderbilt University
Title: Uncertainty Aggregation and Extrapolation from Testing to Prediction
Computational model prediction involves many approximations and assumptions; therefore, quantifying the prediction uncertainty is an important need when decisions are made using the model prediction. Sources of uncertainty are both aleatory and epistemic, arising from natural variability, information uncertainty, and modeling approximations; the aggregation of these uncertainty sources is not straightforward. Activities such as calibration, verification and validation are needed as part of the model development process, and the results of these activities also need to be systematically incorporated within the overall uncertainty quantification (UQ) process. This presentation will discuss two approaches, one based on Bayesian parameter estimation and the other based on Bayesian state estimation, to quantify the uncertainty in prediction configurations that are often different from experimental configurations. Bayesian parameter estimation approaches for estimating the discrepancy in single models have been studied in the past, but the discrepancy cannot be extrapolated to a prediction quantity, location or configuration that is different from that observed in the experimental condition. Therefore, we approach the problem of discrepancy prediction in coupled multiple models (especially multidisciplinary models) using Bayesian state estimation of the model form errors in the governing system equations. Model form error can be used in prediction, as opposed to discrepancy; this facilitates prediction uncertainty quantification. The proposed approach is further extended to systems with blackbox computational models. The approach is first illustrated using simple mechanics and heat transfer problems, followed by a coupled fourdisciplinary model prediction of aerothermoelastic behavior in a hypersonic aircraft panel.
May 12, 2022
Speaker: Jianxun Wang, University of Notre Dame; Discussant: Michael Brenner, Harvard University
Title: Leveraging PhysicsInduced Bias in Scientific Machine Learning for Computational Mechanics – PhysicsInformed, StructurePreserved Learning for Problems with Complex Geometries
Firstprinciple modeling and simulation of complex systems based on partial differential equations (PDEs) and numerical discretization have been developed for decades and achieved great success. Nonetheless, traditional numerical solvers face significant challenges in many practical scenarios, e.g., inverse problems, uncertainty quantification, design, and optimizations. Moreover, for complex systems, the governing equations might not be fully known due to a lack of complete understanding of the underlying physics, for which a firstprincipled numerical solver cannot be built. Recent advances in data science. and machine learning, combined with the everincreasing availability of highfidelity simulation and measurement data, open up new opportunities for developing dataenabled computational mechanics models. Although the stateoftheart machine/deep learning techniques hold great promise, there are still many challenges  e.g., requirement of “big data”, the challenge in generalizability/extrapolibity, lack of interpretability/explainability, etc. On the other hand, there is often a richness of prior knowledge of the systems, including physical laws and phenomenological principles, which can be leveraged in this regard. Thus, there is an urgent need for fundamentally new and transformative machine learning techniques, closely grounded in physics, to address the aforementioned challenges in computational mechanics problems.
This talk will briefly discuss our recent developments of scientific machine learning for computational mechanics, focusing on several different aspects of how to bake physicsinduced bias into machine/deep learning models for dataenabled predictive modeling. Specifically, the following topics will be covered: (1) PDEstructure preserved deep learning, where the neural network architectures are built by preserving mathematical structures of the (partially) known governing physics for predicting spatiotemporal dynamics, (2) physicsinformed geometric deep learning for predictive modeling involving complex geometries and irregular domains.
Bio: Dr. Jianxun Wang is an assistant professor of Aerospace and Mechanical Engineering at the University of Notre Dame. He received a Ph.D. in Aerospace Engineering from Virginia Tech in 2017 and was a postdoctoral scholar at UC Berkeley before joining Notre Dame in 2018. He is a recipient of the 2021 NSF CAREER Award. His research focuses on scientific machine learning, dataenabled computational modeling, Bayesian data assimilation, and uncertainty quantification.
May 5, 2022
Speaker: Ioannis Kougioumtzoglou, Columbia University; Discussant: George Deodatis, Columbia University
Title: Path Integrals in Stochastic Engineering Dynamics
Everincreasing computational capabilities, development of potent signal processing tools, as well as advanced experimental setups have contributed to a highly sophisticated modeling of engineering systems and related excitations. As a result, the form of the governing equations has become highly complex from a mathematics perspective. Examples include high dimensionality, complex nonlinearities, joint timefrequency representations, as well as generalized/fractional calculus modeling. In many cases even the deterministic solution of such equations is an open issue and an active research topic. Clearly, solving the stochastic counterparts of these equations becomes orders of magnitude more challenging. To address this issue, the speaker and coworkers have developed recently a solution framework, based on the concept of Wiener path integral, for stochastic response analysis and reliability assessment of diverse dynamical systems of engineering interest. Significant novelties and advantages that will be highlighted in this talk include: i) The methodology can readily account for complex nonlinear/hysteretic behaviors, for fractional calculus modeling, as well as for nonwhite and nonGaussian stochastic process representations. ii) Highdimensional systems can be readily addressed by relying on a variational formulation with mixed fixed/free boundary conditions, which renders the computational cost independent of the total number of degreesoffreedom (DOFs) or stochastic dimensions; and thus, the ‘curse of dimensionality’ in stochastic dynamics is circumvented. iii) The computational cost can be further drastically reduced by employing sparse representations for the system response probability density function (PDF) in conjunction with compressive sampling schemes and group sparsity concepts. Moreover, the methodology is capable of uncertainty quantification associated with the system response PDF estimate by relying on a Bayesian formulation. Various examples are presented and discussed pertaining to a wide range of engineering systems including, indicatively, a class of nonlinear electromechanical energy harvesters and a 100DOF stochastically excited nonlinear dynamical system modeling the behavior of large arrays of coupled nanomechanical oscillators.
Bio: Prof. Ioannis A. Kougioumtzoglou received his fiveyear Diploma in Civil Engineering from the National Technical University of Athens (NTUA) in Greece (2007), and his M.Sc. (2009) and Ph.D. (2011) degrees in Civil Engineering from Rice University, TX, USA. He joined Columbia University in 2014, where he is currently an Associate Professor in the Department of Civil Engineering & Engineering Mechanics. He is the author of approximately 150 publications, including more than 80 technical papers in archival International Journals. Prof. Kougioumtzoglou was chosen in 2018 by the National Science Foundation (NSF) to receive the CAREER Award, which recognizes earlystage scholars with high levels of promise and excellence. He is also the 2014 European Association of Structural Dynamics (EASD) Junior Research Prize recipient “for his innovative influence on the field of nonlinear stochastic dynamics”. Prof. Kougioumtzoglou is an Associate Editor for the ASCEASME Journal of Risk and Uncertainty in Engineering Systems and an Editorial Board Member of the following Journals: Mechanical Systems and Signal Processing, Probabilistic Engineering Mechanics, and International Journal of NonLinear Mechanics. He is also a coEditor of the Encyclopedia of Earthquake Engineering (Springer) and has served as a Guest Editor for several Special Issues in International Journals. Prof. Kougioumtzoglou has cochaired the ASCE Engineering Mechanics Institute Conference 2021 and Probabilistic Mechanics & Reliability Conference 2021 (EMI 2021 / PMC 2021) and has served in the scientific and/or organizing committees of many international technical conferences. Prof. Kougioumtzoglou is a member both of the American Society of Civil Engineers (M.ASCE) and the American Society of Mechanical Engineers (M.ASME), while he currently serves as a member of the ASCE EMI committees on Dynamics and on Probabilistic Methods. He is a Licensed Professional Civil Engineer in Greece, and a Fellow of the Higher Education Academy (FHEA) in the UK.
March 16, 2022
Speaker: Danial Faghihi, University at Buffalo; Discussant: J. Tinsley Oden, The University of Texas at Austin
Title: Toward Selecting Optimal Predictive Computational Models
An overriding importance in the scientific prediction of complex physical systems is validating mechanistic models in the presence of uncertainties. In addition to data uncertainty and numerical error, uncertainties in selecting the optimal model formulation pose a significant challenge to predictive computational modeling. In a Bayesian setting, the choice of models for computational prediction relies on the available observational data and prior belief on the model and its parameters. This talk discusses a systematic framework in selecting an “optimal” predictive model, among the numerous possible models with different fidelities and complexities, that delivers sufficiently accurate computational prediction. In particular, we extend an adaptive computational framework, known as OccamPlausibility ALgorithm (OPAL), that leverages Bayesian inference and the notion of model plausibility to select the simplest valid model. The key feature of our modification is the design of modelspecific validation experiments to provide observational data reflecting, in some sense, the structure of the target prediction. An application of this framework in selecting an optimal discretetocontinuum multiscale model for predicting the performance of microscale materials systems will be presented. We will also provide an example of leveraging validated and selected models for predicting heterogeneous tumor morphology in specific subjects and via a scalable solution algorithm to the highdimensional Bayesian inference. Finally, we will discuss challenges and possible future directions to develop strategies for selecting optimal neural networks in the context of hybrid physical  machine learning multiscale models of mesoporous thermal insulation materials.
February 23, 2022
Speaker: Johann Guilleminot, Duke University; Discussant: Roger Ghanem, University of Southern California
Title: Stochastic Modeling for PhysicsConsistent Uncertainty Quantification on Constrained Spaces, with Various Applications in Computational Mechanics
In this talk, we discuss the construction of admissible, physicsconsistent and identifiable stochastic models for uncertainty quantification. We consider the case of variables taking values in constrained spaces (with boundaries defined by manifolds) and indexed by complex geometries described by nonconvex sets. This setting is relevant to a broad variety of applications in computational mechanics, ranging from mechanical simulations on parts produced by additive manufacturing to multiscale analyses with stochastic connected phases. We first present theoretical and computational procedures to ensure wellposedness and to generate random field representations defined by arbitrary marginal transport maps. The sampling scheme relies, in particular, on the construction of a family of stochastic differential equations driven by an ad hoc spacetime process and involves an adaptive step sequence that ensures stability near the boundaries of the state space. Finally, we provide results pertaining to modeling, sampling, and statistical inverse identification for various applications including additive manufacturing, phasefield fracture modeling, multiscale analyses on nonlinear microstructures, and patientspecific computations on soft biological tissues.
December 8, 2021
Speaker: Audrey Olivier, University of Southern California; Discussant: Lori GrahamBrady, Johns Hopkins University
Title: Bayesian Learning of Neural Networks for Small or Imbalanced Data Sets
Databased predictive models such as neural networks are showing great potential to be used in various scientific and engineering fields. They can be used in conjunction with physicsbased models to account for missing or hardtomodel physics, or as surrogates to replace highfidelity, overly costly physicsbased simulations. However, in many engineering fields data is expensive to obtain and data scarcity and / or data imbalance is a challenge. Many physical processes are also random in nature and exhibit large aleatory uncertainties. Bayesian methods allow for a comprehensive account of both aleatory and epistemic uncertainties; however, they are challenging to use for overly parameterized problems such as neural networks. This talk will present methods based on variational inference and model averaging for probabilistic training of neural networks. An application in surrogate materials modeling will be presented, where data is scarce as it is obtained from expensive highfidelity materials simulations. Finally, we will show how this probabilistic approach allows to integrate scientific intuitions by defining a meaningful prior and likelihood for training. The example presented pertains to the prediction of ambulance travel time, using real data provided by the New York City Fire Department.
November 4, 2021
Speaker: Catherine Gorlé; Discussant: Gianluca Iaccarino
Title: Uncertainty Quantification and Data Assimilation for Predictive Computational Wind Engineering
Computational fluid dynamics (CFD) can inform sustainable design of buildings and cities in terms of optimizing pedestrian wind comfort, air quality, thermal comfort, energy efficiency, and resiliency to extreme wind events. An important limitation is that the accuracy of CFD results can be compromised by the large natural variability and complex physics that are characteristic of urban flow problems. In this talk I will show how uncertainty quantification and data assimilation can be leveraged to evaluate and improve the predictive capabilities of Reynoldsaveraged NavierStokes simulations for urban flow and dispersion. I will focus on quantifying inflow and turbulence model form uncertainties for two different urban environments: Oklahoma City and Stanford’s campus. For both test cases, the predictive capabilities of the models will be evaluated by comparing the model results to field measurements.
October 7, 2021
Speaker: Yeonjong Shin; Discussant: Dongbin Xiu
Title: Mathematical approaches for robustness and reliability in scientific machine learning
Machine learning (ML) has achieved unprecedented empirical success in diverse applications. It now has been applied to solve scientific problems, which has become a new subfield under the name of Scientific Machine Learning (SciML). Many ML techniques, however, are very sophisticated, requiring trialanderror and numerous tricks. These result in a lack of robustness and reliability, which are critical factors for scientific applications. This talk centers around mathematical approaches for SciML to provide robustness and reliability. The first part will focus on the datadriven discovery of dynamical systems. I will present a general framework of designing neural networks (NNs) for the GENERIC formalism, resulting in the GENERIC formalism informed NNs (GFINNs). The framework provides flexible ways of leveraging available physics information into NNs. Also, the universal approximation theorem for GFINNs is established. The second part will be on the Active Neuron Least Squares (ANLS), an efficient training algorithm for NNs. ANLS is designed from the insight gained from the analysis of gradient descent training of NNs, particularly, the analysis of Plateau Phenomenon. The performance of ANLS will be demonstrated and compared with existing popular methods in various learning tasks ranging from function approximation to solving PDEs.
June 25, 2021
Speaker: Jiaxin Zhang; Discussant: Richard Archibald
Title: Uncertaintyaware inverse learning using generative flows
Solving inverse problems is a longstanding challenge in mathematics and the natural sciences, where the goal is to determine the hidden parameters given a set of specific observations. Typically, the forward problem going from parameter space to observation space is wellestablished, but the inverse process is often illposed and ambiguous, with multiple parameter sets resulting in the same measurement. Recently, deep invertible architectures have been proposed to solve the reverse problem, but these currently struggle in precisely localizing the exact solutions and in fully exploring the parameter spaces without missing solutions. In this talk, we will present a novel approach leveraging recent advances in normalizing flows and deep invertible neural network architectures for efficiently and accurately solving inverse problems. Given a specific observation and latent space sampling, the learned invertible model provides a posterior over the parameter space; we identify these posterior samples as an implicit prior initialization which enables us to narrow down the search space. We then use gradient descent with backpropagation to calibrate the inverse solutions within a local region. Meanwhile, an exploratory sampling strategy is imposed on the latent space to better explore and capture all possible solutions. We evaluate our approach on analytical benchmark tasks, crystal design in quantum chemistry, image reconstruction in medicine and astrophysics, and find it achieves superior performance compared to several stateoftheart baseline methods.
May 17, 2021
Speaker: Tan BuiThanh; Discussant: Omar Ghattas
Title: Modelaware deep learning approaches for forward and PDEconstrained inverse problems
The fast growth in practical applications of machine learning in a range of contexts has fueled a renewed interest in machine learning methods over recent years. Subsequently, scientific machine learning is an emerging discipline which merges scientific computing and machine learning. Whilst scientific computing focuses on largescale models that are derived from scientific laws describing physical phenomena, machine learning focuses on developing datadriven models which require minimal knowledge and prior assumptions. With the contrast between these two approaches follows different advantages: scientific models are effective at extrapolation and can be fit with small data and few parameters whereas machine learning models require "big data" and a large number of parameters but are not biased by the validity of prior assumptions. Scientific machine learning endeavours to combine the two disciplines in order to develop models that retain the advantages from their respective disciplines. Specifically, it works to develop explainable models that are datadriven but require less data than traditional machine learning methods through the utilization of centuries of scientific literature. The resulting model therefore possesses knowledge that prevents overfitting, reduces the number of parameters, and promotes extrapolatability of the model while still utilizing machine learning techniques to learn the terms that are unexplainable by prior assumptions. We call these hybrid datadriven models as "modelaware machine learning” (MAML) methods. In this talk, we present a few efforts in this MAML direction: 1) ROMML approach, and 2) Autoencoderbased Inversion (AI) approach. Theoretical results for linear PDEconstrained inverse problems and numerical results various nonlinear PDEconstrained inverse problems will be presented to demonstrate the validity of the proposed approaches.
April 6, 2021
Speaker: Xun Huan; Discussant: Habib Najm
Title: Modelbased Sequential Experimental Design
Experiments are indispensable for learning and developing models in engineering and science. When experiments are expensive, a careful design of these limited dataacquisition opportunities can be immensely beneficial. Optimal experimental design (OED), while leveraging the predictive capabilities of a simulation model, provides a statistical framework to systematically quantify and maximize the value of an experiment. We will describe the main ingredients in setting up an OED problem in a general manner, that also captures the synergy among multiple experiments conducted in sequence. We cast this sequential learning problem in a Bayesian setting with informationbased utilities and solve it numerically via policy gradient methods from reinforcement learning.
March 18, 2021
Speaker: Nathaniel Trask; Discussant: Jim Stewart
Title: Deep learning architectures for structure preservation and hpconvergence
Deep learning has attracted attention as a powerful means of developing datadriven models due to its exceptional approximation properties, particularly in highdimensions. Application to scientific machine learning (SciML) settings however mandate guarantees regarding: convergence, stability of extracted models, and physical realizability. In this talk, we present development of deep learning architectures incorporating ideas from traditional numerical discretization to obtain SciML tools as trustworthy as e.g. finite element discretization of forward problems. In the first half, we demonstrate how ideas from the approximation theory literature can be used to develop partition of unity network (pouNet) architectures which are able to realize hpconvergence for smooth data and < 1% error for piecewise constant data, and may be applied to highdimensional data with latent lowdimensional structure. In the second half, we establish how ideas from mimetic discretization of PDE may be used to design structure preserving neural networks. The de Rham complex underpinning compatible PDE discretization may be extended to graphs, allowing design of architectures which respect exact sequence requirements, allowing construction of invertible Hodge Laplacians. The resulting "datadriven exterior calculus" provides building blocks for designing robust structure preserving surrogates for elliptic problems with solvability guarantees.
February 10, 2021
Speaker: Rebecca Morrison; Discussant: Youssef Marzouk
Title: Learning Sparse NonGaussian Graphical Models
Identification and exploitation of a sparse undirected graphical model (UGM) can simplify inference and prediction processes, illuminate previously unknown variable relationships, and even decouple multidomain computational models. In the continuous realm, the UGM corresponding to a Gaussian data set is equivalent to the nonzero entries of the inverse covariance matrix. However, this correspondence no longer holds when the data is nonGaussian. In this talk, we explore a recently developed algorithm called SING (Sparsity Identification of NonGaussian distributions), which identifies edges using Hessian information of the log density. Various data sets are examined, with sometimes surprising results about the nature of nonGaussianity.
Members Only (password required)
Discussion Forum
