The workshop focuses on algorithms that allow computers to learn based on data, in particular to automatically learn to recognize complex patterns and make intelligent decisions based on data. Thus, the topic of the workshop is closely related to fields such as statistics, probability theory, data mining, pattern recognition, artificial intelligence, adaptive control, and theoretical computer science.
Organizers
University of Vienna
LMU Munich
University of California
Speakers
Semi-plenary speakers
INRIA
Stanford University
Invited speakers
Universität Bern
National University of Singapore
New York University
TU Darmstadt
Université de Poitiers
University Pompeu Fabra
ETH Zurich
University of Vienna
LMU Munich
University of California, Davis
John Hopkins University
ETH Zurich
JKU Linz
John Hopkins University
KU Eichstätt
Thursday, 9 July
![[HS 04 – Ground Floor]](https://focm2026.univie.ac.at/wp-content/uploads/2026/06/HS04.png){kind=link}
14:00-14:30
Soledad Villar (John Hopkins University)
Many modern learning tasks require models that can take inputs of varying sizes. Consequently, dimension-independent architectures have been proposed for domains where the inputs are graphs, sets, and point clouds. These are known as any-dimensional models and under some assumptions they can be trained on small inputs and generalize to larger ones.
In a different line of research, several influential works have identified conditions under which the optimal hyperparameters such as learning rate can be transferred from small models to larger ones. In this talk we identified connections between these two literatures that allow for using small pretrained models as a warm start for larger ones.
14:30-15:00
Thomas Strohmer (University of California, Davis)
Synthetic data are an attractive concept to enable privacy in data sharing. A fundamental question is how similar the privacy-preserving synthetic data are compared to the true data. Using metric privacy, an effective generalization of differential privacy beyond the discrete setting, we raise the problem of characterizing the optimal privacy-accuracy tradeoff by the metric geometry of the underlying space. We provide a partial solution to this problem in terms of the “entropic scale”, a quantity that captures the multiscale geometry of a metric space via the behavior of its packing numbers. We illustrate the applicability of our privacy-accuracy tradeoff framework via a diverse set of examples of metric spaces.
15:00-16:00
Joan Bruna (New York University)
16:00-16:30 Coffee Break
16:30-17:30 semi-plenary talk
Andrea Montanari (Stanford University)
Classical statistical learning theory decouples three aspects of the learning problem:
(1) Approximation (how well the model class approximates the data distribution);
(2) Generalization (the statistical error that results from learning from a finite data sample);
(3) Optimization (the algorithmic problem of minimizing the error on the training data).
Modern deep learning models are often overparametrized and trained by minimizing
a non-convex empirical risk. As a consequence, the model learnt from data is not
uniquely determined my the data and risk function. It is instead entirely
determined by the learning dynamics and sensitive to the details of the learning algorithm.
I will provide a general introduction to these phenomena, and explain how different
dynamical regime impact the properties of the learnt model.
17:30-18:00
Felix Krahmer (TU Darmstadt)
Machine learning methods are commonly used to solve inverse problems, wherein an unknown signal must be estimated from few indirect measurements generated via a known acquisition procedure. In particular, neural networks perform well empirically but have limited theoretical guarantees. In this work, we study an underdetermined linear inverse problem that admits several possible solution operators that map measurements to estimates of the target signal. A standard remedy (e.g., in compressed sensing) for establishing the uniqueness of the solution mapping is to assume the existence of a latent low-dimensional structure in the source signal. We ask the following question: do deep linear neural networks adapt to unknown low-dimensional structure when trained by gradient descent with weight decay regularization? We prove that mildly overparameterized deep linear networks trained in this manner converge to an approximate solution mapping that accurately solves the inverse problem while implicitly encoding latent subspace structure. We show rigorously that deep linear networks trained with weight decay automatically adapt to latent subspace structure in the data under practical stepsize and weight initialization schemes. Our work highlights that regularization and overparameterization improve generalization, while overparameterization also accelerates convergence during training. This is joint work with Hannah Laus (Technical University of Munich and Munich Center for Machine Learning), Suzanna Parkinson (University of Chicago), Vasileios Charisopoulos (University of Washington), and Rebecca Willett (University of Chicago)
18:00-18:30
Mitchell Taylor (ETH Zurich)
Phase retrieval using a frame for a finite-dimensional Hilbert space is known to always be Lipschitz stable. However, phase retrieval using a frame or a continuous frame for an infinite-dimensional Hilbert space is always unstable. In order to bridge the gap between the finite and infinite dimensional phenomena, Cahill-Casazza-Daubechies gave a construction of a family of nonlinear subsets of an infinite-dimensional Hilbert space where phase retrieval could be performed with a Hölder stability estimate. They then posed the question of whether these subsets satisfied Lipschitz stable phase retrieval. We solve this problem both by giving examples which fail Lipschitz stability and by giving examples which satisfy Lipschitz stability.
Friday, 10 July
14:00-14:30
Philipp Petersen (University of Vienna)
Reliable deep learning increasingly depends on computations carried out in low precision, where efficiency gains can come at the cost of numerical instability. In this talk, I will discuss how finite-precision effects arise in neural networks and transformer models, how errors can be amplified by depth, attention, and normalization, and why the placement of normalization layers matters. I will then introduce look-ahead mixed precision (LAMP), a new strategy that uses the next nonlinear operation to identify which intermediate quantities should be recomputed in higher precision. We will demonstrate how this approach leads to substantial accuracy gains for LLMs computed in low precision at almost no additional cost. We will also discuss certified computation of neural-network residuals and function-space norms, highlighting complementary ways to make deep learning computations more trustworthy.
14:30-15:00
Felix Voigtlaender (KU Eichstätt)
Despite their decisive success in many applications, trained deep neural networks
are known to be vulnerable to so-called adversarial examples,
meaning small (sometimes imperceptible) perturbations to the input that cause
large adversarial perturbations to the network outputs.
It is thus of great interest to study the robustness of neural networks,
as measured for instance by their Lipschitz constant.
In this talk, we will present several results in this direction,
which together yield an almost sharp characterization of the Lipschitz constant
of randomly initialized neural networks with the ReLU activation function,
both in expectation and with high probability.
Such randomly initialized networks are important objects of study,
since they serve as the initialization for training, and
since it has been observed that they can be used to quickly gauge
the relative performance of neural networks of different sizes.
Unlike existing results, our bounds are fully sharp with respect
to the network width; the upper and lower bounds only differ
by a factor that is logarithmic in the network depth. This is joint work with Sjoerd Dirksen (Utrecht University), Paul Geuchen (KU),
Dominik Stoeger (KU), and Thomas Telaar (ex KU).
15:00-15:30
Holger Rauhut (LMU Munich)
Deep neural networks are usually trained by minimizing a non-convex loss functional via (stochastic) gradient descent methods. In the overparameterized scenario where there are more parameters than training data it is observed empirically that the training loss is commonly driven to zero by gradient descent methods, i.e., the neural network interpolates the data exactly. It is puzzling that at the same the learned network often generalize well to unseen data. This in in stark contrast to intuition from classical statistics which would predict overfitting. The chosen optimization algorithm and algorithm parameters such as initialization have a significant influence on the selection of the learned network. In fact, in the overparameterized context there are many global minimizers so that the optimization method induces an implicit bias on the computed solution. It seems that gradient descent methods and their stochastic variants favor networks of low complexity (in a suitable sense to be understood), and, hence, appear to be very well suited for large classes of real data. Initial attempts in understanding the implicit bias phenomen considers the simplified setting of linear networks, i.e., (deep) factorizations of matrices. This has revealed a surprising relation to the field of sparse and low rank recovery (compressive sensing) in the sense that gradient descent favors sparse diagonal or low rank matrices in certain situations. Moreover, initial results on learning two-layer ReLU networks show that sparse ReLU-expansions may be favored by gradient flow.
Despite such initial theoretical results on simplified scenarios, the understanding of the implicit bias phenomenon in deep learning is widely open.
Based on joint works with El Mehdi Achour, Wiebe Bartolomaeus, Hung-Hsu Chou, Johannes Maly, Maria Matveev, Rachel Ward
15:30-16:00
Mario Ullrich (JKU Linz)
I consider the approximation of functions based on function evaluations and discuss the optimality of least-squares approximation in a worst-case setting.
In addition, I present a novel semi-constructive algorithm to generate near-optimal sampling points, which is based on a generalization of the famous sparsification technique of Batson-Spielman-Srivastava to an infinite-dimensional setting. I conclude with some pictures of point sets.
16:00-16:30 Coffee Break
16:30-17:30
Gabor Lugosi (University Pompeu Fabra)
The dependence structure of high-dimensional distributions is often modeled by graphical models. The problem of learning the graph underlying such distributions has received a lot of attention in statistics and machine learning. In problems of very high dimension, it is often too costly even to store the sample covariance matrix. We propose a model in which one can query single entries of the covariance matrix. We construct efficient algorithms for structure recovery in Gaussian graphical models with query complexity that is quasi-linear in the dimension. We present algorithms that work for trees and, more generally, for graphs of small treewidth. We also discuss hypothesis testing of properties of the underlying graph. The talk is based on joint work with Sofiya Burova, Francisco Calvillo, Luc Devroye, Jakub Truszkowski, Vasiliki Velona, and Piotr Zwiernik.
17:30-18:30
Daniel Bartl (National University of Singapore)
We study the problem of learning with respect to the squared loss over a convex class of functions. It has long been believed that the sample complexity in this setting is governed by localized Rademacher complexities. We show that, assuming access to coarse information on the covariance structure of the model class, the sample complexity is instead controlled by a localized complexity associated with the limiting Gaussian process. In heavy-tailed regimes, this quantity can be significantly smaller than the Rademacher complexity. Joint work with S. Mendelson.
Saturday, 11 July
14:00-15:00
Alexander Rakhlin (MIT)
We study high-accuracy sampling, where the goal is to draw a delta-accurate sample in polylog(1/delta) queries. Building on a new meta-algorithm, first-order rejection sampling (FORS), which simulates rejection sampling using only gradient queries, we obtain exponential improvements for diffusion-model sampling under minimal data assumptions and the first polylog-complexity sampler for general log-concave distributions from gradient evaluations alone. We further show that these high-accuracy guarantees persist under stochastic gradients with subexponential tails — a striking separation from convex optimization, where even Gaussian gradient noise forces poly(1/delta) complexity — and prove that light tails are necessary, with a matching lower bound in the bounded-variance case.
15:00-16:00 semi-plenary talk
Francis Bach (INRIA)
Estimating relative densities and information-theoretic divergences from samples is a central problem in statistics and machine learning, but standard variational approaches to Kullback-Leibler (KL) divergence estimation often require nonlinear optimization and may suffer from numerical instability because of exponential terms. This talk presents a closed-form spectral framework for relative log-density estimation in linearly parameterized probabilistic models, including unnormalized and conditional models. The key idea is to express the KL divergence as an integral of weighted chi-squared divergences. This converts divergence estimation into a family of least-squares problems and yields explicit spectral formulas depending only on first- and second-order feature moments. The framework extends naturally to kernel methods and learned representations, including neural networks, with convergence guarantees and efficient learning algorithms in both settings.
16:00-16:30 Coffee Break
17:00-17:30
Jeremias Sulam (John Hopkins University)
Diffusion models have quickly become some of the most popular and powerful generative models for high-dimensional data. The key insight that enabled their development was the realization that access to the score (the gradient of the log-density at different noise levels) allows for sampling from data distributions by solving a reverse-time stochastic differential equation (SDE) via forward discretization, and that popular denoisers allow for unbiased estimators of this score. In this talk I will demonstrate that an alternative, backward discretization of these SDEs, using proximal maps in place of the score, leads to theoretical and practical benefits. We will leverage recent results in proximal matching to learn proximal operators of the log-density and, with them, develop Proximal Diffusion Models (ProxDM). Theoretically, and assuming oracle access to distributional quantities, these models provide faster convergence to target distributions. Empirically, I will show that ProxDM achieves significantly faster convergence within just a few sampling steps compared to conventional score-matching methods for unconditional, conditional, and latent diffusion models.
17:30-18:00
Joseph Lehec (Université de Poitiers)
The Langevin algorithm is a popular method for sampling high-dimensional measures. Theoretical convergence guarantees seem to have started with a seminal work of Dalalyan around 2012. In this talk, we discuss recent developments around a variation on this algorithm called the kinetic Langevin algorithm. The bottom line is that this kinetic version has better theoretical performances than the regular one, in particular in terms of dimension dependence.
18:00-18:30
Petar Nizic-Nikolac (ETH Zurich)
Matrix concentration inequalities and their recently discovered sharp counterparts provide powerful tools to bound the spectrum of random matrices whose entries are linear functions of independent random variables. However, in many applications in theoretical computer science and in other areas one encounters more general random matrix models, called matrix chaoses, whose entries are polynomials of independent random variables. In this talk we discuss how to develop general matrix concentration inequalities for matrix chaoses, allowing these models to be treated in a systematic and unified way. Joint work with Afonso S. Bandeira, Kevin Lucca, and Ramon van Handel.
Posters
![[PZ01 & PZ04 – Ground Floor]](https://focm2026.univie.ac.at/wp-content/uploads/2026/06/poster_zones_erdgeschoss.png){kind=link}
Valentin Abadie
(ETH Zurich)
Classical neural network approximation results take the form: for every function f and every error tolerance ϵ > 0, one constructs a neural network whose architecture and weights depend on ϵ. This paper introduces a fundamentally different approximation paradigm that reverses this quantifier order. For each target function f , we construct a single recurrent neural network (RNN) with fixed topology and fixed weights that approximates f to within any prescribed tolerance ϵ > 0 when run for sufficiently many time steps. The key mechanism enabling this quantifier reversal is temporal computation combined with weight sharing: rather than increasing network depth, the approximation error is reduced solely by running the RNN longer. This yields exponentially decaying approximation error as a function of runtime while requiring storage of only a small, fixed set of weights. Such architectures are appealing for hardware implementations where memory is scarce and runtime is comparatively inexpensive.
We show that this approximation paradigm can be achieved for very general functions, and that we recover classical approximation rates for usual function classes. Interestingly, the proof involves techniques that are quite different from standard approximation-theoretic proofs. The core of the argument relies on the introduction of a hybrid computation model that captures both the essential algorithmic and approximation-theoretic properties of RNNs, and is used as pivotal ingredient of the proof.
Ernesto Araya Valdivia
(LMU Munich)
We study the problem of aligning multivariate time series, where an observed sequence is paired with a perturbed and permuted version, and the goal is to recover the underlying correspondence. We introduce a probabilistic model in which both series jointly follow a correlated vector autoregressive (VAR) process, extending classical matching problems for independent point clouds to a structured temporal setting with applications in privacy and sensor fusion.
We derive the maximum likelihood estimator (MLE), which leads to a quadratic optimization problem over permutations, and analyze a computationally tractable alternative based on linear assignment. For this estimator, we establish recovery guarantees, identifying correlation regimes that enable exact or partial recovery. We further investigate convex relaxations of the MLE, including formulations over the Birkhoff polytope, allowing joint estimation of the permutation and VAR parameters, and propose an alternating optimization scheme.
Empirical results show that the linear assignment approach often matches or outperforms MLE-based relaxations, even when the latter have oracle access to model parameters. Our findings highlight the effectiveness of tractable methods for structured time series alignment and provide theoretical insight into the role of temporal dependence in matching problems.
Johnson Daddy Audu
(Prince Mohammed Bin Fahd University)
We propose a PINN-assisted two-level mixed finite element method for the Darcy–Forchheimer equation. The coarse nonlinear solve of the
classical two-level scheme is replaced by a physics-informed neural network (PINN) surrogate; the network output is projected onto the
coarse finite element space and used as the linearization point for the fine-grid correction. We introduce a computable residual indicator
$\eta_{H}^{NN}$ that measures surrogate quality ahead of the fine solve. Numerical results demonstrate that the proposed approach
achieves accuracy comparable to the classical method, with nearly first-order convergence for both velocity and pressure. In multi-query
settings where the trained network is reused across repeated solves with fixed physical parameters, the one-time training cost is offset by savings on repeated coarse solves.
Javier Castro
(TU Berlin)
Physics-Informed Neural Networks (PINNs) are a class of deep learning models aiming to approximate solutions of PDEs by training neural networks to minimize the residual of the equation. Focusing on non-equilibrium fluctuating systems, we propose a physically informed choice of penalization that is consistent with the underlying fluctuation structure, as characterized by a large deviations principle. This approach yields a novel formulation of PINNs in which the penalty term is chosen to penalize improbable deviations, rather than being selected heuristically. The resulting thermodynamically consistent extension of PINNs, termed THINNs, is subsequently analyzed by establishing analytical aposteriori estimates, and providing empirical comparisons to established penalization strategies. This poster is based on a preprint with Prof. Benjamin Gess.
Yanfan Chen
(EPFL)
The mean-field limit has emerged as a powerful mathematical framework for analyzing the training dynamics of infinite-width neural networks. While extensive convergence guarantees have been established for networks with bounded and smooth activation functions (e.g., Song Mei, Theodor Misiakiewicz, Andrea Montanari, 2019), extending these limits to non-smooth activations, particularly the ReLU function, remains a significant theoretical bottleneck.
In this work-in-progress, we aim to rigorously characterize the population stochatic gradient descent dynamics via a PDE (which is the Wasserstein Gradient Flow on the space of probability measure). Specifically, by initializating the weight within a “good parameter cone” and levage data regularity assumption (building upon Wojtowytsch, 2020), we extend similar results to ReLU activation.
This poster presents our current mathematical formulation of the problem, preliminary analytical results on the dynamics under specific data regularity assumptions, and our ongoing efforts to establish uniform convergence bounds as well as regimes with diverging input dimensions (when input dim D goes to infinity).
We look forward to discussing these theoretical challenges and potential solutions with the computational mathematics community.
Marco Dell’Orto
(University Of Padova)
Under certain conditions, trained neural networks exhibit a well-documented tendency to concentrate learned information in a few dominant singular components of weight matrices, resulting in large singular values known as ”spikes”. This behavior has attracted significant attention because of its connections to implicit regularization and model compression.
While recent works provide insight into this phenomenon for linear networks, the mechanism underlying spike formation and the factors determining their number is not fully understood for nonlinear networks.
In this ongoing work we address this gap by studying spikes in ReLU networks using a teacher–student setup, aiming to characterize their number and the factors that determine it.
Finally, we intend to apply our findings to the study of VarMiONs (Variationally Mimetic Operator Networks) for solving parameterized PDEs. Thanks to their highly interpretable structure, VarMiONs provide a promising setting in which to explore connections between spike formation, continuum physics, and model parsimony.
Thomas Dittrich
(RICAM Linz)
Given a set of distinct regression problems, it is possible to solve each problem individually with a distinct scalar model (single-task learning) or solve all at once by a single model with multi-dimensional output (multi-task learning). Experience has shown that exploiting similarity across the different problems can significantly improve the overall performance. This phenomenon has been investigated from different perspectives such as its generalization aspects and sampling complexity. However, up to now the rigorous investigation of the parametric complexity of two paradigms for joint learning problems has been sparse.
The novelty of this work lies in proving lower and upper bounds on the number of parameters for single-task and multi-task learning approximation classes, respectively. To this end, we rely on general hypothesis classes and the corresponding polynomial and exponential approximation spaces. The regression problem is constructed as follows: a single shared feature with polynomial hardness is concatenated with several individual modes stemming from an exponential approximation space. With that, we can then show that each regression problem individually retains at least the complexity of the shared feature. Consequently we get a lower bound on the necessary number of parameters. Contrary, for the multi-task procedure we develop an upper bound by first approximating the shared feature and subsequently the individual modes. This results in an overall upper bound as a sum of the upper bounds of the individual sub-tasks.
Note that the previous theoretical results hold independent of the norm in which the error is measured. In the second part of the paper we illustrate the theoretical results in the uniform norm on the example of a random wavelet series as the shared feature. For this example, we consider compression codecs as the hypothesis class and derive precise bounds on the approximation rates.
Chenguang Duan
(RWTH Aachen University)
Inference-time alignment for diffusion models aims to adapt a pre-trained reference diffusion model toward a target distribution without retraining the reference score network, thereby preserving the generative capacity of the reference model while enforcing desired properties at the inference time. A central mechanism for achieving such alignment is guidance, which modifies the sampling dynamics through an additional drift term. In this work, we introduce variationally stable Doob’s matching, a novel framework for provable guidance estimation grounded in Doob’s h-transform. Our approach formulates guidance as the gradient of logarithm of an underlying Doob’s h-function and employs gradient-regularized regression to simultaneously estimate both the h-function and its gradient, resulting in a consistent estimator of the guidance. Theoretically, we establish non-asymptotic convergence rates for the estimated guidance. Moreover, we analyze the resulting controllable diffusion processes and prove non-asymptotic convergence guarantees for the generated distributions in the 2-Wasserstein distance. Finally, we show that variationally stable guidance estimators are adaptive to unknown low dimensionality, effectively mitigating the curse of dimensionality under low-dimensional subspace assumptions.
Christian Fiedler
(Technical University of Munich and Munich Center for Machine Learning)
In many applications of supervised learning, distributional inputs appear as inputs, and quite often these are not directly accessible during training, but only through samples thereof.
This setting, called distributional learning with two-stage sampling, is particularly amenable to kernel-based methods, where the distributions or samples are first embedded into a Hilbert space, often using kernel mean embeddings (KMEs), and then a standard kernel method like Support Vector Machines (SVMs) is applied, using a kernel defined on the embedding Hilbert space. While regression problems in this setup have received considerable attention both from a practical and theoretical perspective, much less is known about classification, despite its relevance in applications like AI-assisted medical diagnosis or causal learning. In recent work, we address this gap by developing statistical learning theory for kernel-based distributional learning with general loss functions under two-stage sampling. We establish very general and flexible oracle inequalities and derive consistency and learning rate results under standard assumptions. Furthermore, for SVMs using the hinge loss and Gaussian kernels, we formulate a novel variant of an established noise assumption from the binary classification literature, under which we can establish learning rates. Finally, some of our technical tools like a new feature space for Gaussian kernels on Hilbert spaces are of independent interest. Our analysis provides not only theoretical insights into a challenging and practically relevant learning setup, but provides a theoretical foundation for downstream applications.
References
Fiedler, C., Massiani, P. F., Solowjow, F., & Trimpe, S. (2024). On statistical learning theory for distributional inputs. Forty-first International Conference on Machine Learning (ICML 2024).
Fiedler, C. (2026). “Statistical Learning Theory for Distributional Classification.” The 40th Annual AAAI Conference on Artificial Intelligence (AAAI 2026), to be published
Paz Fink Shustin
(University of Oxford)
Learning data representations under uncertainty is an important task that emerges in numerous scientific computing and data analysis applications. However, uncertainty quantification techniques are computationally intensive and become prohibitively expensive for high-dimensional data. In this study, we introduce a dimensionality reduction surrogate modeling (DRSM) approach for representation learning and uncertainty quantification that aims to deal with data of moderate to high dimensions. The approach involves a two-stage learning process: 1) employing a variational autoencoder to learn a low-dimensional representation of the input data distribution; and 2) harnessing polynomial chaos expansion (PCE) formulation to map the low dimensional distribution to the output target. The model enables us to (a) capture the system dynamics efficiently in the low-dimensional latent space, (b) learn under uncertainty, a representation of the data and a mapping between input and output distributions, (c) estimate this uncertainty in the high-dimensional data system, and (d) match high-order moments of the output distribution; without any prior statistical assumptions on the data. Numerical results are presented to illustrate the performance of the proposed method.
Jonathan García Rebellón
(University of Vienna)
We prove that a classifier with a Barron-regular decision boundary $ \mathbbm{1}{\Omega} $ can be approximated using a ReLU neural network $\Phi$ with three hidden layers and $ O(N) $ neurons, with $ N\in \mathbb{N} $, achieving an error of $ \mu\left(\left{\mathbbm{1}{\Omega}
eq \Phi\right}\right)\lesssim N^{-\gamma/2} $ for all probability measures $ \mu $ that satisfy a certain margin condition with a positive parameter $ \gamma $. Moreover, when the network is trained using the hinge loss, the learning rate is of the order $ O(n^{-\gamma/(2+\gamma)}(1+\log n)) $, where $ N\lesssim n^{\gamma/(2+\gamma)} $ and $ n $ is the size of the training set. This implies that for strong margin condition, one can approximate $ \mathbbm{1}_{\Omega} $ (a high-dimensional discontinuous function) with the network at a rate comparable to that of a low-dimensional smooth function, achieving a fast learning rate of approximately $ n^{-1}(1+\log n) $. In addition, we present numerical simulations in various dimensions. We observed that as the strength of the margin and the number of samples increase, the test error decreases at a rate close to the predicted one, supporting the theoretical results. In addition, we establish lower bounds for the minimax learning rate over the Barron-regular function class and under the margin condition using Assouad’s lemma arguments, concluding that the learning rate is optimal up to logarithmic factors.
Konstantin Häberle
(ETH Zurich)
A common intuition behind the remarkable success of deep learning models in classification and regression is that real-world data, although embedded in high-dimensional spaces, often exhibit intrinsic low-dimensional structure. This work attempts to provide a mathematical framework for binary classification on low-dimensional data, building on Cover’s (1965) function-counting theory. With our framework, we aim to address the question of how the low-dimensional structure of the data affects the classification performance of learning models. We specifically consider datasets of positive and $\sigma$-finite $s$-dimensional Hausdorff measure embedded in Euclidean spaces and provide exact function-counting results under mild technical assumptions. Particularizing to homogeneous linear separation on $s$-sparse signals, we show that the number of realizable dichotomies (i.e., binary label assignments) is independent of the ambient dimension and instead governed by the sparsity parameter $s$. When considering the more general case of $s$-rectifiable sets, we establish a connection between the geometry of the set and the number of realizable dichotomies. Finally, by extending Cover’s notion of separation capacity to $s$-rectifiable sets, we show that the effective dimension which determines the separation capacity of a homogeneous linear classifier is the rectifiability parameter $s$.
Xingchao Jian
(TU Munich)
Detecting the origin of information or infection spread in networks is a fundamental challenge with applications in misinformation tracking, epidemiology, and beyond. We study the multi-source detection problem: given snapshot observations of node infection status on a graph, estimate the set of source nodes that initiated the propagation. Existing methods either lack statistical guarantees or are limited to specific diffusion models and assumptions. We propose a novel conformal prediction framework that provides statistically valid recall guarantees for source set detection, independent of the underlying diffusion process or data distribution. Our approach is assumption‑free—requiring only minimal knowledge of the propagation model—and can be applied to any detection pipeline, including arbitrary neural network architectures and graph structures. This is achieved by extending the notion of non‑conformity scores from point to set estimation. We introduce principled score design principles, establish connections to existing conformal prediction methods, and systematically analyze detection performance under different score designs. Leveraging a calibration set, our method constructs prediction sets with user‑specified recall and coverage levels, applicable to both single‑ and multi‑source scenarios and general network diffusion dynamics.
Jana Jovcheva
(UCLouvain)
Modern deep neural networks achieve strong performance across domains but often rely on highly overparameterised dense layers. This motivates the study of structured, low-rank operators that retain expressive power while controlling computational complexity. Tensor network formats, such as the tensor train (TT) decomposition, provide compressed representations of high-order linear maps, but their use in deep learning has largely been limited to linear layers or post-compression of pre-trained layers. We introduce the double haircomb (DH) operator, a nonlinear tensor network layer that can be composed into multi-layer architectures and trained directly in compressed form. The DH consists of two TTs coupled through a single connecting bond, forming a tree topology that can represent both multilinear and nonlinear maps between high-order tensor spaces through the incorporation of activation functions within the tensor network structure. We study both the computational and representational properties of this architecture. Empirically, small DH networks achieve competitive performance on hyperspectral image classification while using an order of magnitude fewer parameters than comparable dense models. We further characterise the architectural properties underlying this performance through theoretical analysis and synthetic validation experiments.
Clemens Kinn
(University of Vienna)
Since o-minimal structures satisfy the non-independence property, every definable family has finite VC dimension and is therefore PAC learnable. In this work, we study binary classification problems whose decision boundaries are given by definable families in o-minimal structures, connecting tame geometry with neural network theory. Using cell decomposition of definable sets together with results on approximation and statistical learning via ReLU neural networks, we establish approximation and learning rates for such classifiers.
This is joint work with Philipp Petersen.
Vicky Kouni
(PSL University)
Unfolding networks are interpretable networks emerging from iterative algorithms, incorporate prior knowledge of data structure, and are designed to solve inverse problems like compressed sensing, which deals with recovering data from noisy, missing observations. Compressed sensing finds applications in critical domains, from medical imaging to cryptography, where adversarial robustness is crucial to prevent catastrophic failures. However, a solid theoretical understanding of the performance of unfolding networks in the presence of adversarial attacks is still in its infancy. In this paper, we study the adversarial generalization of unfolding networks when perturbed with l2-norm constrained attacks, generated by the fast gradient sign method. Particularly, we choose a family of state-of-the-art overaparameterized unfolding networks and deploy a new framework to estimate their adversarial Rademacher complexity. Given this estimate, we provide adversarial generalization error bounds for the networks under study, which are tight with respect to the attack level. To our knowledge, this is the first theoretical analysis on the adversarial generalization of unfolding networks. We further present a series of experiments on real-world data, with results corroborating our derived theory, consistently for all data. Finally, we observe that the family’s overparameterization can be exploited to promote adversarial robustness, shedding light on how to efficiently robustify neural networks.
Jing Li
We propose Alternating Phase-Field Fourier Neural Networks (APF-FNNs) as a unified and physics-based framework for topology optimization. The approach decouples the design problem by representing the state, adjoint, and topology fields with three separate Fourier neural networks, which are trained via a stable collaborative alternating scheme applicable to both self-adjoint and non-self-adjoint problems. To obtain well-resolved designs, the Ginzburg–Landau energy functional is embedded in the loss of the topology network as an intrinsic regularizer, naturally enforcing smooth and distinct interfaces between the two phases. Phase-field updates are driven by adjoint-based optimality conditions, and design sensitivities are evaluated efficiently using automatic differentiation, ensuring that the gradients correspond to exact total derivatives rather than naive partial derivatives. In contrast to classical FEM-based phase-field methods that rely on pseudo-time gradient-flow evolution and repeated high-fidelity PDE solves, APF-FNNs directly optimize the steady-state phase field using physically consistent design gradients. This avoids inner gradient-flow iterations and alleviates the strong coupling between interface resolution and mesh refinement. By formulating physics-driven losses from variational principles or strong-form PDE residuals, the framework is broadly applicable to 2D and 3D benchmark problems, including compliance minimization, eigenvalue maximization, and Stokes/Navier–Stokes flow optimization. Numerical comparisons with classical finite element phase-field solvers demonstrate that APF-FNNs achieve comparable or improved objective values while offering enhanced scalability in three-dimensional settings, establishing a versatile and efficient foundation for physics-driven computational design.
Morris-Luca Kühmeier
(Universität Innsbruck)
We study the convergence behaviour of the Iterative Thresholding and K-residual Means (ITKrM) algorithm for dictionary learning under a sparse generative signal model with random signs and permutations.
Numerical experiments indicate that dictionary algorithms often converge to stable configurations that do not coincide with the generating dictionary: some atoms are merged, while others appear as duplicates with small deviations. Motivated by these observations and by the fact that in practice the correct number of atoms $K$ is often unknown, we analyse a setting in which the learned dictionary contains one atom fewer than the generating dictionary. Thus we cannot recover all atoms, but analyse the stability of merged configurations we see in experiments. In particular, each stable configuration consists of all generating atoms except that one pair of atoms is covered by a single merged atom of the form $\psi_i \pm \psi_j$ with $i
eq j$. Since the indices $i$ and $j$ are arbitrarily, this gives rise to many distinct stable configurations.
Under standard sparsity and incoherence assumptions, we establish atom-wise local convergence of ITKrM with high probability towards such stable configurations up to an accuracy of order $\mathcal{O}(S^2/K^2)$, where $S$ denotes the number of active atoms per signal. This limitation is caused by the mismatch of the dictionary sizes, which renders the thresholding step unpredictable in certain cases.
These results provide a first theoretical step towards understanding the behaviour of ITKrM under dictionary size mismatch and shed light on the robustness of dictionary learning algorithms in practice.
This is joint work with Univ.-Prof. Dr. Karin Schnass.
Micaela Long
(Universidad De La República)
One of the challenges in population genetics data analysis is high dimensionality. Dimensionality reduction via principal components analysis (PCA) is commonly used for genetic data visualization and to infer population structure. However, PCA primarily captures global linear trends, while local genetic variation often appears in higher components. As a result, standard two-dimensional projections may fail to represent key structure, particularly in admixed populations.
Non-linear approaches such as t-distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP) emphasize local neighborhoods but typically distort global geometry, making distances between clusters difficult to interpret.
We propose a graph-based framework for genomic data visualization and analysis. Individuals are represented as nodes in a weighted graph, where edge weights are computed using Fermat distance, a density-sensitive geodesic metric that captures intrinsic structure in the data. To obtain low dimensional embeddings, we explore several graph-based representations, including node2vec, spectral embeddings, and force-directed (spring) layouts, each providing complementary perspectives on population structure.
Constructing the graph using Fermat distance yields visualizations that preserve both local and global patterns, enhancing the separation of small or isolated populations and, crucially, improving the representation of admixed individuals such as those in the Uruguayan population. Beyond visualization, the graph structure enables extended analyses, including the extraction of ancestry proportions via community detection. This makes the approach a flexible and powerful tool for exploring genetic diversity and admixture.
Moritz Maibaum
(University of Vienna)
Joint work with Philipp Petersen:
Neural network methods for PDEs require reliable error control in function space norms. However, trained neural networks can typically only be probed at a finite number of point values. Without strong assumptions, point evaluations alone do not provide enough information to derive tight deterministic and guaranteed bounds on function space norms. In this work, we move beyond a purely black-box setting and exploit the neural network structure directly. We present a framework for the certified and accurate computation of integral quantities of neural networks, including Lebesgue and Sobolev norms, by combining interval arithmetic enclosures on axis-aligned boxes with adaptive marking/refinement and quadrature-based aggregation. On each box, we compute guaranteed lower and upper bounds for function values and derivatives, and propagate these local certificates to global lower and upper bounds for the target integrals. Our analysis provides a general convergence theorem for such certified adaptive quadrature procedures and instantiates it for function values, Jacobians, and Hessians, yielding certified computation of Lp, W1,p, and W2,p norms. We further show how these ingredients lead to practical certified bounds for PINN interior residuals. Numerical experiments illustrate the accuracy and practical behavior of the proposed methods.
Anh Neuman
(University of Vienna)
We study the statistical behaviour of reasoning probes in a stylized model of looped reasoning, given by Boolean circuits whose computational graph is a perfect $
u$-ary tree ($
u\ge 2$) and whose output is appended to the input and fed back iteratively for subsequent computation rounds.
A reasoning probe has access to a sampled subset of internal computation nodes—possibly without covering the entire graph—and seeks to infer which $
u$-ary Boolean gate is executed at each queried node, representing uncertainty via a probability distribution over a fixed collection of $\mathtt{m}$ admissible $
u$-ary gates. This partial observability induces a generalization problem, which we analyze in a realizable, transductive setting.
We show that, when the reasoning probe is parameterized by a graph convolutional network (GCN)–based hypothesis class and queries $N$ nodes, the worst-case generalization error attains the optimal rate $\mathcal{O}(\sqrt{\log(2/\delta)}/\sqrt{N})$ with probability at least $1-\delta$, for $\delta\in (0,1)$.
Our analysis combines snowflake metric embedding techniques with tools from statistical optimal transport.
A key insight is that this optimal rate is \emph{achievable independently of graph size}, owing to the existence of a low-distortion one-dimensional snowflake embedding of the induced graph metric.
As a consequence, our results provide a sharp characterization of how structural properties of the computational graph govern the statistical efficiency of reasoning under partial access.
Yang Pan
(ETH Zurich)
Suppose we are interested in sampling from a multidimensional density. The most common method is to consider some time discretization of the Langevin diffusion, such as constructing a suitable Markov chain approximation. We consider the possibility of deforming the underlying geometry of the Euclidean space and consider instead sampling using a particular Markov chain approximation of the so-called Mirror Langevin diffusions, first introduced in the context of machine learning in 2020/21. I will give an overview of this family and how it relates to Wasserstein gradient flows, Schrödinger bridges, Sinkhorn algorithms, and many other topics of recent interest.
Shrunal Pothagoni
(George Mason University)
Convolutional neural networks are increasingly being used to analyze and classify material microstructures, motivated by the possibility that they will be able to identify relevant microstructural features more efficiently and impartially than human experts.
While up to now convolutional neural networks have mostly been applied to light optimal microscopy and scanning electron microscope micrographs, application to EBSD micrographs will be increasingly common as rational design generates materials with unknown textures and phase compositions. This raises the question of how crystallographic orientation should be represented in such a convolutional neural network, and whether this choice has a significant effect on the network’s analysis and classification accuracy. Four representations of orientation information are examined and are used with convolutional neural networks to classify five synthetic microstructures with varying textures and grain geometries. Of these, a spectral embedding of crystallographic orientations in a space that respects the crystallographic symmetries performs by far the best, even when the network is trained on small volumes of data such as could be accessible by practical experiments.
Subhendu Pramanick
(LUT University)
The Matérn family of covariance functions is uniquely powerful in machine learning and spatial statistics due to its explicit parametrization of mean-square differentiability via the smoothness parameter, $u$. While crucial for reliable uncertainty quantification and optimal predictive performance, accurately estimating $u$ poses significant theoretical and computational challenges. This study presents a comprehensive comparison of three prominent methodologies for learning Matérn smoothness: Maximum Likelihood Estimation (MLE), Leave-One-Out Cross-Validation (LOO-CV), and the recently developed Kernel Flow (KF) estimator. We analyze these methods across two primary dimensions: theoretical properties and applied machine learning utility. Theoretically, we contrast the asymptotic efficiency of the frequentist MLE approach against the predictive robustness of LOO-CV and the multi-scale, data-driven consistency of Kernel Flows. To demonstrate these differences in practice, we conduct extensive numerical simulations across varying noise regimes, sample sizes, and degrees of model mis-specification. Our results illuminate the fundamental trade-offs between parameter identifiability, predictive accuracy, and sensitivity to model mis-specification, providing a practical framework for selecting the appropriate smoothness estimator for diverse Gaussian Process applications.
Boris Shustin
(Rutherford Appleton Laboratory)
The detection of particles is a fundamental problem in physics with numerous applications in materials science, tomography, and beyond. In particular, muon detection is carried out at major facilities worldwide, such as the ISIS neutron and muon source at RAL. The advance of modern digitized detectors enables the collection of large volumes of data in the form of 1-D signal traces containing multiple peaks over background noise. The core mathematical challenge is to accurately and efficiently determine the peak locations and their amplitudes, while accounting for effects such as pile-up and detector dead time. In this poster, we present several approaches to this problem, ranging from classical signal processing techniques to our proposed multi-scale analysis and model-based function fitting.
Manjot Singh
(LMU Munich)
Spiking neural networks (SNNs), especially when coupled with neuromorphic hardware, enable event-driven computation with the potential for low-latency and energy-efficiency. In SNNs, information propagates through the network asynchronously through spike times, a mechanism fundamentally different from synchronous layer-wise updates in standard ReLU networks. We study the geometric expressivity of time-to-first-spike (TTFS) SNNs with a linear response function, where the input–output map is continuous piecewise linear. In this setting, each neuron’s firing time admits a maxout-like representation, but the affine pieces are governed by causal feasibility constraints that specify which presynaptic spikes occur before a neuron fires. We formalize causal regions as unions of polyhedral cells with identical causal set label, and derive bounds on the number of causal regions for both shallow and multilayer networks, including sharp counts in structured shared-weight regimes. Our results show that SNNs can generate richer partitions of the input space than activation regions in ReLU networks, even under restrictive parameter choices. Finally, the experiments on static, event-based, and synthetic tasks quantify the region statistics and reveal inductive biases linked to asynchronous propagation and delay parameters.
Pranjal Srivastava
(University of Michigan)
We study collision entropy (Rényi entropy of order 2), classically known as Yule’s K in corpus linguistics, as a diagnostic for diversity loss in autoregressive language models. Empirically, natural English text converges to a stable collision entropy regardless of genre or corpus size, establishing a reference value. We prove two results connecting standard LLM sampling parameters to entropy reduction. First, for a softmax output distribution at temperature T, collision entropy is a monotone function of T, so that H₂ computed from output text alone serves as a proxy for the effective generation temperature. Second, for the heavy-tailed token distributions characteristic of natural language, both top-k and top-p (nucleus) sampling provably reduce collision entropy relative to the untruncated distribution — renormalization over the truncated nucleus inflates the collision probability of retained tokens, guaranteeing a strict decrease. These results provide a precise mechanism for model collapse under iterative self-training: each generation’s output has lower collision entropy than its input, and sampling truncation accelerates the contraction. The standard mitigation — mixing real data into each generation — succeeds because natural text maintains higher collision entropy than model-generated text, pulling the mixture back toward the empirical reference value. This reframes model collapse as a measurable, contraction in a single information-theoretic quantity with known behavior.
Tom Szwagier
(IRIT Toulouse)
Gaussian mixture models (GMMs) are ubiquitous in statistical learning, particularly for unsupervised problems. While full GMMs suffer from the overparameterization of their covariance matrices in high-dimensional spaces, spherical GMMs (with isotropic covariance matrices) certainly lack flexibility to fit anisotropic distributions. Connecting these two extremes, we introduce a new family of parsimonious GMMs with piecewise-constant covariance eigenvalue profiles. These extend several low-rank models like the celebrated mixtures of probabilistic principal component analyzers (MPPCA), by enabling any possible sequence of eigenvalue multiplicities. If the latter are prespecified, then we can naturally derive an expectation–maximization (EM) algorithm to learn the mixture parameters. Otherwise, to address the notoriously-challenging issue of jointly learning the mixture parameters and hyperparameters, we propose a componentwise penalized EM algorithm and prove its monotonicity. We show the superior likelihood–parsimony tradeoffs achieved by our models on a variety of unsupervised experiments: density fitting, clustering and single-image denoising.
Paul Valsecchi Oliva
(Imperial College London)
This poster presents interacting underdamped Langevin algorithms, termed Kinetic Interacting Particle Langevin Monte Carlo (KIPLMC) methods, for statistical inference in latent variable models. We propose a diffusion process that evolves jointly in the space of parameters and latent variables and exploit the fact that the stationary distribution of this diffusion concentrates around the maximum marginal likelihood estimate of the parameters. We then provide two explicit discretisations of this diffusion as practical algorithms to estimate parameters of statistical models. For each algorithm, we obtain nonasymptotic rates of convergence for the case where the joint log-likelihood is strongly concave with respect to latent variables and parameters. We achieve accelerated convergence rates clearly demonstrating improvement in dimension dependence, similar to underdamped samplers. To demonstrate the utility of the introduced methodology, we provide numerical experiments that demonstrate the effectiveness of the proposed diffusion for statistical inference and the stability of the numerical integrators utilised for discretisation. Our setting covers a broad number of applications, including unsupervised learning, statistical inference, and inverse problems.
Manon Verbockhaven
(ENS Lyon)
In supervised machine learning, models are trained on a finite dataset of independant draws of some random varibales $\left(X, Y\right)$ on which we suppose a statistic model of the form $Y \sim y^(X) + \sigma\epsilon$ where $\epsilon\sim \mathcal{E}$ is a noise term, $\sigma$ is the level of noise and $y^$ is a deterministic function of $X$. In regression ($Y\in \mathbb{R}^q$), the noise is understood as a physical phenomenon, while in classification ($Y\in \Delta^q$), it can either stems from a miscalssification error or, as introduced by the student-teacher networks, stems from the interpretation of the cross entropy criterion as an approximation of the Kullback-Leibler divergence with the caricatural density defined by the one-hot encoding $Y$. In both cases, when the variable $X$ lives in a high-dimensional space, the dataset size is often insufficient to cancel the impact of the noise in the training dynamic, leading to a generalization gap in performance. In this work, we take a functional perspective on the optimization criterion $L$ as an application from a function class $\mathcal{F}$ to $\mathbb{R}^+$ and define its functional derivative $
abla_f L$. For the usual criteria $L$, we show that the application $
abla_f L$ satisfies a linearity property which grasps the impact of the noise on the training dynamic and that allows simple Monte Carlo estimates of it.
From this theoretical analysis and for any prior on $\epsilon$, we propose an estimate of $\sigma$ and a hard thresholding strategy on the gradient descent algorithm that is model and parameter-dependent. We show empirically, on regression and classification tasks, that this strategy mitigates the effect of the noise on the training dynamic while preserving the final performance of the model.
Yuguang Wang
(Monash University)
Hypergraphs effectively model higher-order relationships in natural phenomena, capturing complex interactions beyond pairwise connections. We introduce a novel hypergraph message passing framework inspired by interacting particle systems, where hyperedges act as fields inducing shared node dynamics. By incorporating attraction, repulsion, and Allen-Cahn forcing terms, particles of varying classes and features achieve class-dependent equilibrium, enabling separability through the particle-driven message passing. We investigate both first-order and second-order particle system equations for modeling these dynamics, which mitigate over-smoothing and heterophily thus can capture complete interactions. The more stable second-order system permits deeper message passing. Furthermore, we enhance deterministic message passing with stochastic element to account for interaction uncertainties. We prove theoretically that our approach mitigates over-smoothing by maintaining a positive lower bound on the hypergraph Dirichlet energy during propagation and thus to enable hypergraph message passing to go deep. Empirically, our models demonstrate competitive performance on diverse real-world hypergraph node classification tasks, excelling on both homophilic and heterophilic datasets.
Tizian Wenzel
(LMU Munich)
The search for the optimal shape parameter for Radial Basis Function (RBF) kernel methods has been an outstanding research problem for decades. In this work, we establish a theoretical framework for this problem by leveraging a recently established theory on sharp direct, inverse and saturation statements for kernel based approximation. In particular, we link the search for the optimal shape parameter to superconvergence phenomena.
Hanfei Zhou
(Fudan University Shanghai)
Physics-informed neural networks (PINNs), owing to their mesh-free nature, offer a powerful approach for directly solving high-dimensional partial differential equations (PDEs) in complex geometries, including irregular domains. This capability effectively circumvents the challenges of mesh generation that traditional numerical methods face in high-dimensional or geometrically intricate settings. While recent studies have extended PINNs to manifold domains, the theoretical foundations in this context remain scarce. Existing theoretical analyses of PINNs in Euclidean space often rely on smoothness assumptions for the PDE solutions. However, recent empirical evidence indicates that PINNs may struggle to efficiently approximate solutions with low regularity, such as those arising from nonlinear hyperbolic equations. In this paper, we develop a framework for PINNs tailored to the efficient approximation of weak solutions of PDEs, particularly nonlinear hyperbolic equations defined on Riemannian manifolds $\mathcal{M}^d$. We introduce a novel weak PINN (wPINN) formulation on Riemannian manifolds that leverages the well-posedness theory to approximate entropy solutions of geometry-compatible hyperbolic conservation laws on manifolds. Employing tools from approximation theory, we establish a convergence analysis of the proposed algorithm, including an analysis of approximation errors for time-dependent entropy solutions on manifolds. This analysis provides insight into the accumulation of approximation errors over long time horizons. Notably, the network complexity depends only on the intrinsic dimension $d$, independent of the ambient space dimension. Our results match the minimax rate in the $d$-dimensional Euclidean space, effectively demonstrating that PINNs can alleviate the curse of dimensionality in the context of low-dimensional Riemannian manifolds. Finally, we validate the performance of the proposed wPINN framework through numerical experiments, confirming its ability to efficiently approximate entropy solutions on manifolds.