Bayesian inverse problems use observed data to update a prior probability distribution for an unknown state or parameter of a scientific system to a posterior distribution conditioned on the data. In many applications, the unknown parameter is high-dimensional, making computation of the posterior expensive due to the need to sample in a high-dimensional space and the need to evaluate an expensive high-dimensional forward model relating the unknown parameter to the data. However, inverse problems often exhibit low-dimensional structure due to the fact that the available data are only informative in a low-dimensional subspace of the parameter space. Dimension reduction approaches exploit this structure by restricting inference to the low-dimensional subspace informed by the data, which can be sampled more efficiently. Further computational cost reductions can be achieved by replacing expensive high-dimensional forward models with cheaper lower-dimensional reduced models. In this work, we propose new dimension and model reduction approaches for linear Bayesian inverse problems with rank-deficient prior covariances, which arise in many practical inference settings. The dimension reduction approach is applicable to general linear Bayesian inverse problems whereas the model reduction approaches are specific to the problem of inferring the initial condition of a linear dynamical system. We provide theoretical approximation guarantees as well as numerical experiments demonstrating the accuracy and efficiency of the proposed approaches.

Variational data assimilation is a technique for combining measured data with dynamical models. It is a key component of Earth system state estimation and is commonly used in weather and ocean forecasting. The approach involves a large-scale generalized nonlinear least-squares problem. Solving the resulting sequence of sparse linear subproblems requires the use of sophisticated numerical linear algebra methods. In practical applications, the computational demands severely limit the number of iterations of a Krylov subspace solver that can be performed and so high-quality preconditioners are vital. In this paper, we introduce variational data assimilation from a numerical linear algebra perspective and review current solution techniques, with a focus on the challenges that arise in large-scale geophysical systems.

Dynamic inverse problems are challenging to solve due to the need to identify and incorporate appropriate regularization in both space and time. Moreover, the very large scale nature of such problems in practice presents an enormous computational challenge.
In this work, in addition to the use of edge-enhancing regularization of spatial features, we propose a new regularization method that incorporates a temporal model that estimates the motion of objects in time. In particular, we consider the optical flow model that simultaneously estimates the motion and provides an approximation for the desired image, and we incorporate this information into the cost functional as an additional form of temporal regularization. We propose a computationally efficient algorithm to solve the jointly regularized problem that leverages a generalized Krylov subspace method. We illustrate the effectiveness of the prescribed approach on a wide range of numerical experiments, including limited angle and single-shot computerized tomography.

We discuss two approaches to solving the parametric (or stochastic) eigenvalue problem A(μ)λ(μ)=A(μ)v(μ). One of them uses a Taylor expansion and the other a Chebyshev expansion. The parametric eigenvalue problem assumes that the matrix A depends on a parameter μ, where μ might be a random variable. Consequently, the eigenvalues and eigenvectors are also functions of μ. We compute a Taylor approximation of these functions about μ₀ by iteratively computing the Taylor coefficients. The complexity of this approach is O(n³) for all eigenpairs, if the derivatives of A(μ) at μ₀ are given.  The Chebyshev expansion works similarly. We first find an initial approximation iteratively which we then refine with Newton's method.  This second method is more expensive but provides a good approximation over the whole interval of the expansion instead around a single point.
 
We present numerical experiments confirming the complexity and demonstrating that the approaches are capable of tracking eigenvalues at intersection points.  Further experiments shed light on the limitations of the Taylor expansion approach with respect to the distance from the expansion point μ₀.

The posterior distribution for nonlinear Bayesian inverse problems often has to be estimated via sampling and requires many simulations of the forward model, which can be computationally expensive when the forward model requires simulating a high-dimensional dynamical system. This can be remedied by using a reduced forward model that captures the important dynamics of the high-dimensional dynamical system. In systems theory, balanced truncation methods obtain efficient reduced models by projecting the high-dimensional model operators onto the space spanned by dominant eigenvectors of the system Gramians. In this paper, we consider Bayesian smoothing problems for quadratic dynamical systems and introduce inference-oriented Gramians that define a procedure for model reduction by balanced truncation. We provide a stability analysis of the resulting quadratic nonlinear model and support the analysis with a numerical example.

A non-intrusive model order reduction method for bilinear stochastic differential equations with additive noise is proposed. A reduced order model (ROM) is designed in order to approximate the statistical properties of high-dimensional systems. The drift and diffusion coefficients of the ROM are inferred from state observations by solving appropriate least-squares problems. The closeness of the ROM obtained by the presented approach to the intrusive ROM obtained by the proper orthogonal decomposition (POD) method is investigated. Two generalisations of the snapshot-based dominant subspace construction to the stochastic case are presented. Numerical experiments are provided to compare the developed approach to POD.

In this paper, we consider low-rank approximations for the solutions to the stochastic Helmholtz equation with random coefficients. A Stochastic Galerkin finite element method is used for the discretization of the Helmholtz problem. Existence theory for the low-rank approximation is established when the system matrix is indefinite. The low-rank algorithm does not require the construction of a large system matrix which results in an advantage in terms of CPU time and storage. Numerical results show that, when the operations in a low-rank method are performed efficiently, it is possible to obtain an advantage in terms of storage and CPU time compared to computations in full rank. We also propose a general approach to implement a preconditioner using the low-rank format efficiently.

The application of X-ray spectro-microscopy to image changes in the chemical state in application areas such as catalysis, environmental science, or biological samples can be limited by factors such as the speed of measurement, the presence of dilute concentrations, radiation damage, and thermal drift during the measurement. We have adapted a reduced-order model approach, known as the discrete empirical interpolation method, which identifies how to optimally subsample the spectroscopic information, accounting for background variations in the signal, to provide an accurate approximation of an equivalent full spectroscopic measurement from the sampled material. This approach uses readily available prior information to guide and significantly reduce the sampling requirements impacting both the total X-ray dose and the acquisition time. The reduced-order model approach can be adapted more broadly to any spectral or spectro-microscopy measurement where a low-rank approximation can be made from prior information on the possible states of a system, and examples of the approach are presented.

Balanced truncation is a well-established model order reduction method in system theory that has been applied to a variety of problems. Recently, a connection between linear Gaussian Bayesian inference problems and the system theoretic concept of balanced truncation was drawn for the first time. Although this connection is new, the application of balanced truncation to data assimilation is not a novel concept: It has already been used in four-dimensional variational data assimilation (4D-Var) in its discrete formulation. In this paper, the link between system theory  and data assimilation is further strengthened by discussing the application of balanced truncation to standard linear Gaussian Bayesian inference, and, in particular, the 4D-Var method.  similarities between both data assimilation problems allow a discussion of established methods as well as a generalisation of the state-of-the-art approach to arbitrary prior covariances as  reachability Gramians. Furthermore, we propose an enhanced approach using time-limited balanced truncation that allows to balance Bayesian inference for unstable systems and in addition mproves the numerical results for short observation periods.

Neural networks have emerged as powerful and versatile tools in the field of deep learning. As the complexity of the task increases, so do size and architectural complexity of the causing compression techniques to become a focus of current research. Parameter truncation can provide a significant reduction in memory and computational complexity. Originating from a model order reduction framework, the Discrete Empirical Interpolation Method is applied to the gradient descent training of neural networks and analyze for important parameters. The approach for various state-of-the-art neural networks is compared to established truncation methods. Further metrics like L₂ and Cross-Entropy Loss, as well as accuracy and compression rate are reported.

Four-dimensional variational data assimilation (4D-Var) is a data assimilation method often used in weather forecasting. Based on a numerical model and observations of a system, it predicts the system state beyond the last time of measurement. This requires the minimisation of a functional. At each step of the optimisation algorithm, a full nonlinear model evaluation and its adjoint is required. This quickly becomes very costly, especially in high dimensions. For this reason, a surrogate model is needed that approximates the full model well, but requires significantly less computational effort. In this paper, we propose time-limited balanced truncation to build such a reduced-order model. Our approach is able to deal with unstable system matrices. We demonstrate its performance in experiments and compare it with α-bounded balanced truncation, which is an another reduction approach for unstable systems.

We discuss discrete-time dynamical systems depending on a parameter μ. Assuming that the system matrix A(μ) is given, but the parameter μ is unknown, we infer the most-likely parameter μ_m≈μ from an observed trajectory x of the dynamical system. We use parametric eigenpairs (v_i(μ),lambda_i(μ) of the system matrix A(μ) computed with Newton's method based on a Chebyshev expansion. We then represent x in the eigenvector basis defined by the v_i(μ) and compare the decay of the components with predictions based on the lambda_i(μ). The resulting estimates for μ are combined using a kernel density estimator to find the most likely value for μ_m and a corresponding uncertainty quantification.

We present a Reduced Order Model (ROM) which exploits recent developments in Physics Informed Neural Networks (PINNs) for solving inverse problems for the Navier--Stokes equations (NSE). In the proposed approach, the presence of simulated data for the fluid dynamics fields is assumed. A POD-Galerkin ROM is then constructed by applying POD on the snapshots matrices of the fluid fields and performing a Galerkin projection of the NSE (or the modified equations in case of turbulence modeling) onto the POD reduced basis. A POD-Galerkin PINN ROM is then derived by introducing deep neural networks which approximate the reduced outputs with the input being time and/or parameters of the model. The neural networks incorporate the physical equations (the POD-Galerkin reduced equations) into their structure as part of the loss function. Using this approach, the reduced model is able to approximate unknown parameters such as physical constants or the boundary conditions. A demonstration of the applicability of the proposed ROM is illustrated by two cases which are the steady flow around a backward step and the unsteady turbulent flow around a surface mounted cubic obstacle.

In this paper, we examine conditioning of the discretization of the Helmholtz problem. Although the discrete Helmholtz problem has been studied from different perspectives, to the best of our knowledge, there is no conditioning analysis for it. We aim to fill this gap in the literature. We propose a novel method in 1D to observe the near-zero eigenvalues of a symmetric indefinite matrix. Standard classification of ill-conditioning based on the matrix condition number is not true for the discrete Helmholtz problem. We relate the ill-conditioning of the discretization of the Helmholtz problem with the condition number of the matrix. We carry out analytical conditioning analysis in 1D and extend our observations to 2D with numerical observations. We examine several discretizations. We find different regions in which the condition number of the problem shows different characteristics. We also explain the general behavior of the solutions in these regions.

Der Sonderforschungsbereich SFB1294 „Datenassimilation: Die nahtlose Verschmelzung von Daten und Modellen“ wird seit 2017 von der Deutschen Forschungsgemeinschaft an der Universität of Potsdam gefördert und befindet sich gegenwärtig am Beginn der zweiten Förderphase. Gemeinsam mit den Projektpartnern der Humboldt Universität zu Berlin, der Technischen Universität Berlin, dem Weierstraß-Institut für Angewandte Analysis und Stochastik sowie dem Helmholtz-Zentrum Potsdam: Deutsches GeoForschungsZentrum GFZ, werden innerhalb des Verbundes sowohl die statistischen und mathematischen Grundlagen also auch mannigfaltige Anwendungen zeitabhängiger inverser Problemeuntersucht. In diesem Beitrag werden in kurzen Zügen sowohl die grundlegende mathematische Fragestellung also auch die aktuellen wissenschaftlichen Projekte des SFBs vorgestellt

MOFs and COFs are porous materials with a large variety of applications including gas storage and separation. Synthesised in a modular fashion from distinct building blocks, a near infinite number of structures can be constructed and the properties of the material can be tailored for a specific application. While this modularity is a very attractive feature it also poses a challenge. Attempting to identify the best performing material(s) for a given application is experimentally intractable. Current research efforts combine molecular simulations and machine learning techniques to evaluate the simulated performance of hundreds of thousands of materials to identify top performing MOFs and COFs for a given application. These approaches typically rely on moderated brute-force screening which is still resource-intensive as typically between 70‒100 % of the hundreds of thousands of materials must be simulated to create a training set for the machine learning models used, restricting screening to relatively simple molecules. In this work we demonstrate our novel Bayesian mining approach to materials screening which allows 62‒92 % of the top 100 porous materials for a range of applications to be readily identified from large materials databases after only assessing less than one percent of all materials. This is a stark contrast to the 0‒1 % achieved by conventional brute-force screening where porous materials are just chosen at random during a high throughput screening. Through this accelerated virtual screening process, the identification of high performing materials can be used to more rapidly inform experimental efforts and hence lead to an acceleration of the entire research and development pipeline of porous materials.