I’m a mathematics Ph.D. student at the University of Wisconsin–Madison working at the intersection of mathematics and machine learning. My current research develops the mathematical foundations of singular learning theory.

A central question of out-of-distribution generalization is as follows. Suppose the set of parameters that achieve minimal training loss is not a singleton, and different optimal parameters implement different behavior outside the training distribution. Which solutions are favored by the training algorithm?

Singular learning theory answers, at least when the training algorithm is Bayesian inference and certain regularity conditions hold, that solutions with minimal real log canonical threshold (RLCT) are asymptotically favored. You can think of the RLCT as a measure of local geometric degeneracy of the loss function. Researchers in many fields have proposed notions of “effective dimensionality” to capture the intuition that not all parameters matter equally, so learning good points in parameter space may be easier than the literal dimensionality of parameter space suggests. It may help to think of RLCT as the notion of effective dimensionality that appears in the asymptotic prediction error of Bayesian inference, λ/n, where λ is the minimal RLCT and n is the number of samples.

Now suppose two population-loss minimizers θ1,θ2 have RLCTs λ1<λ2. For small neighborhoods U1,U2 containing θ1,θ2, the Bayesian posterior assigns nλ2−λ1 times more mass in favor of U1, up to logarithmic factors. This turns the central question above into an algebraic geometry question: for a given behavior out of distribution, is the minimal RLCT among solutions which exhibit that behavior smaller or greater than the minimal RLCT among solutions which do not exhibit that behavior?

Suppose you train your AI not to cheat its tasks. The AI could learn the policy “do not cheat” or learn the policy “do not cheat if and only if I am being trained.” Both policies have the same behavior on the training distribution, so loss minimization alone cannot predict which policy the AI learns. In the Bayesian setting, singular learning theory predicts an asymptotic preference for either policy if it has smaller RLCT than the other policy. If we could understand which policies minimize RLCT, we could better predict which behaviors Bayesian inference favors. And if we could design architectures and loss functions to affect those RLCTs, we could shape that preference.

My current agenda seeks to understand how designing additional loss functions can affect the geometry of the loss landscape. Concretely, index a sequence of data sets by t. For each loss j, define the empirical loss

Kj,t(w)=1nj,t∑i=1nj,tfj(Xj,i,w).

Here nj,t is the sample size for loss j, and the observations Xj,1,Xj,2,… are independent with distribution ν within each stream. Consider the family of Gibbs distributions

Πt(dw)=1Ztexp[−∑j=1ℓnj,tβj,tKj,t(w)]φ(w)dw,

where the weights nj,tβj,t can grow at different rates, and βj,t is the inverse temperature for loss j. The normalizing constant Zt and its derivatives contain important information about the system. With one loss, its leading decay is governed by the RLCT. With several losses, the corresponding exponent depends on their relative weighting.

My framework describes this dependence using polyhedral geometry. For weights expressed in the form nj,tβj,t=taj, the leading exponent is piecewise linear in the vector a=(a1,…,aℓ). This gives a precise sense in which the system has a phase diagram: the limiting normalized free energy can become nondifferentiable as the relative growth rates change.

The derivatives of −logZt with respect to the weights give the posterior expectations of the individual losses. The framework therefore describes how strengthening one objective affects both its own expected loss and those of the other objectives.

So far, my main application is to prove that the RLCT estimator proposed by Lau et al. in The Local Learning Coefficient: A Singularity-Aware Complexity Measure is consistent. Watanabe’s work in A Widely Applicable Bayesian Information Criterion establishes the consistency of a closely related estimator assuming the prior is compact. For computational reasons, Lau et al. instead use a Gaussian localization factor

e−γt‖w−w*‖2

to softly concentrate the prior near a point w*.

There are two competing requirements. The quadratic penalty must become strong enough to concentrate the posterior near w*. But if its weight grows too quickly, it can dominate the posterior, causing the estimator to average the original loss under a distribution shaped primarily by the localization penalty rather than by the original loss.

My result identifies a broad range of growth rates satisfying both requirements. Write At=n1,tβ1,t for the primary loss’s weight, where β1,t is the inverse temperature. Under the theorem’s other regularity conditions, it is sufficient that

γtlogAt⟶∞,logγtlogAt⟶0,β1,tn1,t⟶0.

The third condition allows the result to not assume that the model satisfies the relatively finite variance assumption. The first two conditions permit the localization weight to grow faster than logarithmically with respect to At while remaining slower than every positive power of At.

More generally, my result gives conditions under which finitely many auxiliary losses make the posterior distribution of the scaled primary loss asymptotically Gamma:

LΠt(AtK1,t(w))→ℙGamma(λZ,1),

where the Gamma distribution has shape λZ and rate 1, and λZ is the minimum RLCT of the primary population loss along the common zero set Z of all the population losses. Here convergence means weak convergence of the conditional posterior law, in probability over the training data.

This application illustrates one way auxiliary losses can shape Bayesian selection: they can restrict where the posterior concentrates while preserving the primary loss’s asymptotics locally. Further work in this agenda may use the phase diagram to understand regimes where multiple loss functions are asymptotically influential. For instance, splitting one loss function into a mean term and an error term, we may better understand Bayesian asymptotics when the mean term and error term have very different geometry. We may also better understand when training on one dataset aids fine tuning on another dataset.

Multivariate zeta functions, Laplace integrals, and singular learning theory

Master's thesis, 2026

This thesis develops a framework for understanding Laplace integrals involving several non-negative analytic loss functions whose weights grow at different rates. It shows how the associated multivariate zeta function determines an asymptotic expansion into exponential terms multiplied by polynomials, with errors smaller than any prescribed exponential scale. The proof combines resolution of singularities to prove meromorphic continuation of the zeta function, followed by multivariate residues and a carefully constructed graph traversal to organize contour shifts with controlled remainders, so that the residues from the poles of the zeta function contribute the dominant terms. The work is motivated by the application to singular learning theory that I have developed since submitting the thesis.

Quantifying cell-state densities in single-cell phenotypic landscapes using Mellon

Nature Methods, 2024 · Co-author

This paper introduces Mellon, a non-parametric density estimator using Gaussian processes and a nearest-neighbors-distance distribution that scales to millions of cells. We cluster the density landscape in systems of cell differentiation systems, revealing a consistent pattern of high-density regions corresponding to major cell types intertwined with low-density, rare transitory states.

Neural networks learn statistics of increasing complexity

Extended version of an ICML 2024 paper · Co-author

This paper provides evidence supporting the distributional simplicity bias hypothesis by showing that neural networks learn low order moments, means and covariances, before learning higher order correlations. Using synthetic data through maximum entropy sampling and optimal transport edits to image statistics, we show that early models rely heavily on these simple patterns, while later models become sensitive to high order structure.

I want to share developing results early so that others can engage with them, offer feedback, and build on them. This feels especially valuable given the pace needed for mathematical work in AI safety. The papers below are works in progress that develop subproblems from my master’s thesis in greater depth and especially build the connection to singular learning theory.

Affine residues and chamber jumps for hyperplane arrangements

My master’s thesis generalizes singular learning theory to handle finitely many energy (loss) functions. The increased difficulty is the difference between single-variable and multivariable complex analysis. To handle multivariate residues, I develop a residue theory of the cohomology of hyperplane arrangements.

Acyclic traversal of affine hyperplane arrangements

The residue paper provides a jump formula for contour integrals across faces of affine hyperplane arrangements, and I need to apply the jump formula many times to make remainder terms small. Naive approaches to doing so may jump cyclicly, making the resulting iterated jump formula useless. Here I pose and solve the purely affine-geometric problem of constructing a strategy to traverse an affine hyperplane arrangement that avoids such cycles.

Mellin asymptotics under anisotropic dilations

One of the main steps of singular learning theory is to use Mellin inversion on the zeta function to obtain a limiting distribution of the posterior supported on the minimizing set of the energy. For several energy functions, I analogously use multivariate Mellin inversion to obtain a limiting distribution on the common minimizing set. By shifting Mellin inversion's vertical contour integral across poles, I find that the affine residues developed above contribute the limiting distribution.

A localization framework for singular learning theory

Using the above results, I develop a framework for singular learning theory with several analytic losses on a possibly non-compact parameter space. Additionally, I support loss functions beyond KL divergence and do not assume relatively finite variance. I then prove a broad generalization of the conjecture in Appendix D of The local learning coefficient: A singularity-aware complexity measure that the proposed estimator of the RLCT is consistent.