research interests

My research interests are in probability and its applications to machine learning and high-dimensional statistics. Recent work has focused on:

See also my pages at arXiv and Google Scholar Citations.

mathematics and statistics research

Likelihood landscape and maximum likelihood estimation for the discrete orbit recovery model (with Z. Fan, T. Wang, and Y. Wu), Communications on Pure and Applied Mathematics, to appear. (pdf) (arXiv) (video)
We study the non-convex optimization landscape for maximum likelihood estimation in the discrete orbit recovery model with Gaussian noise. This model is motivated by applications in molecular microscopy and image processing, where each measurement of an unknown object is subject to an independent random rotation from a rotational group. Equivalently, it is a Gaussian mixture model where the mixture centers belong to a group orbit.

We show that fundamental properties of the likelihood landscape depend on the signal-to-noise ratio and the group structure. At low noise, this landscape is "benign" for any discrete group, possessing no spurious local optima and only strict saddle points. At high noise, this landscape may develop spurious local optima, depending on the specific group. We discuss several positive and negative examples, and provide a general condition that ensures a globally benign landscape. For cyclic permutations of coordinates on \(\mathbb{R}^d\) (multi-reference alignment), there may be spurious local optima when \(d\geq 6\), and we establish a correspondence between these local optima and those of a surrogate function of the phase variables in the Fourier domain.

We show that the Fisher information matrix transitions from resembling that of a single Gaussian in low noise to having a graded eigenvalue structure in high noise, which is determined by the graded algebra of invariant polynomials under the group action. In a local neighborhood of the true object, the likelihood landscape is strongly convex in a reparametrized system of variables given by a transcendence basis of this polynomial algebra. We discuss implications for optimization algorithms, including slow convergence of expectation-maximization, and possible advantages of momentum-based acceleration and variable reparametrization for first- and second-order descent methods.

Probabilistic conformal blocks for Liouville CFT on the torus (with P. Ghosal, G. Remy, and X. Sun), submitted (2020). (pdf) (arXiv)
Virasoro conformal blocks are a family of important functions defined as power series via the Virasoro algebra. They are a fundamental input to the conformal bootstrap program for 2D conformal field theory (CFT) and are closely related to four dimensional supersymmetric gauge theory through the Alday-Gaiotto-Tachikawa correspondence. The present work provides a probabilistic construction of the 1-point toric Virasoro conformal block for central change greater than 25. More precisely, we construct an analytic function using a probabilistic tool called Gaussian multiplicative chaos (GMC) and prove that its power series expansion coincides with the 1-point toric Virasoro conformal block. The range \((25,\infty)\) of central charges corresponds to Liouville CFT, an important CFT originating from 2D quantum gravity and bosonic string theory. Our work reveals a new integrable structure underlying GMC and opens the door to the study of non-perturbative properties of Virasoro conformal blocks such as their analytic continuation and modular symmetry. Our proof combines an analysis of GMC with tools from CFT such as Belavin-Polyakov-Zamolodchikov differential equations, operator product expansions, and Dotsenko-Fateev type integrals.

Principal components in linear mixed models with general bulk (with Z. Fan and Z. Wang), Annals of Statistics, to appear. (pdf) (arXiv)
We study the outlier eigenvalues and eigenvectors in variance components estimates for high-dimensional mixed effects linear models using a free probability approach. We quantify the almost-sure limits of these eigenvalues and their eigenvector alignments, under a general bulk-plus-spikes assumption for the population covariances of the random effects, extending previous results in the identity-plus-spikes setting. Our analysis develops two tools in free probability and random matrix theory which are of independent interest—strong asymptotic freeness of GOE and deterministic matrices, and a method of proving deterministic anisotropic approximations to resolvents using free deterministic equivalents. Statistically, our results quantify bias and aliasing effects for the leading principal components of variance components estimates in modern high-dimensional applications.

Gaussian fluctuations for products of random matrices (with V. Gorin), American Journal of Mathematics, to appear. (pdf) (arXiv)
We study global fluctuations for singular values of \(M\)-fold products of several right-unitarily invariant \(N \times N\) random matrix ensembles. As \(N \to \infty\), we show the fluctuations of their height functions converge to an explicit Gaussian field, which is log-correlated for \(M\) fixed and has a white noise component for \(M \to \infty\) jointly with \(N\). Our technique centers on the study of the multivariate Bessel generating functions of these spectral measures, for which we prove a central limit theorem for global fluctuations via certain conditions on the generating functions. We apply our approach to a number of ensembles, including rectangular Ginibre matrices, truncated Haar-random unitary matrices, and right-unitarily invariant matrices with fixed singular values, using a detailed asymptotic analysis of multivariate Bessel functions to verify the necessary conditions.

Spiked covariances and principal components analysis in high-dimensional random effects models (with Z. Fan and I. Johnstone), preprint (2018). (arXiv)
We study principal components analyses in multivariate random and mixed effects linear models, assuming a spherical-plus-spikes structure for the covariance matrix of each random effect. We characterize the behavior of outlier sample eigenvalues and eigenvectors of MANOVA variance components estimators in such models under a high-dimensional asymptotic regime. Our results show that an aliasing phenomenon may occur in high dimensions, in which eigenvalues and eigenvectors of the MANOVA estimate for one variance component may be influenced by the other components. We propose an alternative procedure for estimating the true principal eigenvalues and eigenvectors that asymptotically corrects for this aliasing problem.

Affine Macdonald conjectures and special values of Felder-Varchenko functions (with E. Rains and A. Varchenko), Selecta Mathematica N. S. 24 (2018), 1549-1591. (pdf) (arXiv) (journal) (video)
We refine the statement of the denominator and evaluation conjectures for affine Macdonald polynomials proposed by Etingof-Kirillov Jr. and prove the first non-trivial cases of these conjectures. Our results provide a q-deformation of the computation of genus 1 conformal blocks via elliptic Selberg integrals by Felder-Stevens-Varchenko. They allow us to give precise formulations for the affine Macdonald conjectures in the general case which are consistent with computer computations.
Our method applies recent work of the second named author to relate these conjectures in the case of \(U_q(\widehat{\mathfrak{sl}}_2)\) to evaluations of certain theta hypergeometric integrals defined by Felder-Varchenko. We then evaluate the resulting integrals, which may be of independent interest, by well-chosen applications of the elliptic beta integral.

Laguerre and Jacobi analogues of the Warren process (with an appendix by A. Sarantsev), submitted (2016). (pdf) (arXiv) (video 1, video 2)
We define Laguerre and Jacobi analogues of the Warren process. That is, we construct local dynamics on a triangular array of particles so that the projections to each level recover the Laguerre and Jacobi eigenvalue processes of König-O'Connell and Doumerc and the fixed time distributions recover the joint distribution of eigenvalues in multilevel Laguerre and Jacobi random matrix ensembles. Our techniques extend and generalize the framework of intertwining diffusions developed by Pal-Shkolnikov. One consequence is the construction of particle systems with local interactions whose fixed time distribution recovers the hard edge of random matrix theory. An appendix by Andrey Sarantsev establishes strong existence and uniqueness for solutions to SDER's satisfied by these processes.

Traces of intertwiners for quantum affine algebras and difference equations (after Etingof-Schiffmann-Varchenko), Transformation Groups 23 (2018), 1167-1215. (pdf) (arXiv) (journal)
We modify and give complete proofs for the results of Etingof-Schiffmann-Varchenko on traces of intertwiners of untwisted quantum affine algebras in the opposite coproduct and the standard grading. More precisely, we show that certain normalized generalized traces \(F^{V_1, \ldots, V_n}(z_1, \ldots, z_n; \lambda, \omega, \mu, k)\) for \(U_q(\widehat{\mathfrak{g}})\) solve four commuting systems of q-difference equations: the Macdonald-Ruijsenaars, dual Macdonald-Ruijsenaars, q-KZB, and dual q-KZB equations. In addition, we show a symmetry property for these renormalized trace functions. Our modifications are motivated by their appearance in the recent work of the author.

Matrix models for multilevel Heckman-Opdam and multivariate Bessel measures, submitted (2016). (pdf) (arXiv) (video 1, video 2)
We study multilevel matrix ensembles at general beta by identifying them with a class of processes defined via the branching rules for multivariate Bessel and Heckman-Opdam hypergeometric functions. For beta = 1, 2, we express the joint multilevel density of the eigenvalues of a generalized beta-Wishart matrix as a multivariate Bessel ensemble, generalizing a result of Dieker-Warren. In the null case, we prove the conjecture of Borodin-Gorin that the joint multilevel density of the beta-Jacobi ensemble is given by a principally specialized Heckman-Opdam measure.

Traces of intertwiners for quantum affine \(\mathfrak{sl}_2\) and Felder-Varchenko functions, Communications in Mathematical Physics 347 (2016), 573-653. (pdf) (arXiv) (journal)
We show that the traces of \(U_q(\widehat{\mathfrak{sl}}_2)\)-intertwiners of Etingof-Schiffmann-Varchenko valued in the three-dimensional evaluation representation converge in a certain region of parameters and give a representation-theoretic construction of Felder-Varchenko's hypergeometric solutions to the \(q\)-KZB heat equation. This gives the first proof that such a trace function converges and resolves the first case of the Etingof-Varchenko conjecture.

As applications, we prove a symmetry property for traces of intertwiners and prove Felder-Varchenko's conjecture that their elliptic Macdonald polynomials are related to the affine Macdonald polynomials defined as traces over irreducible integrable \(U_q(\widehat{\mathfrak{sl}}_2)\)-modules by Etingof-Kirillov Jr. In the trigonometric and classical limits, we recover results of Etingof-Kirillov Jr. and Etingof-Varchenko. Our method relies on an interplay between the method of coherent states applied to the free field realization of the \(q\)-Wakimoto module of Matsuo, convergence properties given by the theta hypergeometric integrals of Felder-Varchenko, and rationality properties originating from the representation-theoretic definition of the trace function.

The polynomial representation of the type \(A_{n−1}\) rational Cherednik algebra in characteristic \(p\mid n\) (with S. Devadas), Communications in Algebra 45 (2017), 1926-1934. (pdf) (arXiv) (journal)
We study the polynomial representation of the rational Cherednik algebra of type \(A_{n−1}\) with generic parameter in characteristic \(p\) for \(p \mid n\). We give explicit formulas for generators for the maximal proper graded submodule, show that they cut out a complete intersection, and thus compute the Hilbert series of the irreducible quotient. Our methods are motivated by taking characteristic \(p\) analogues of existing characteristic \(0\) results.

A representation-theoretic proof of the branching rule for Macdonald polynomials, Mathematical Research Letters 23 (2016), 887-927. Extended abstract in FPSAC 2015. (pdf) (arXiv) (journal) (proceedings)
We give a new representation-theoretic proof of the branching rule for Macdonald polynomials using the Etingof-Kirillov Jr. expression for Macdonald polynomials as traces of intertwiners of \(U_q(\mathfrak{gl}_n)\). In the Gelfand-Tsetlin basis, we show that diagonal matrix elements of such intertwiners are given by application of Macdonald's operators to a simple kernel. An essential ingredient in the proof is a map between spherical parts of double affine Hecke algebras of different ranks based upon the Dunkl-Kasatani conjecture.

A new integral formula for Heckman-Opdam hypergeometric functions, Advances in Mathematics 289 (2016), 1157-1204. (pdf) (arXiv) (journal)
We provide Harish-Chandra type formulas for the multivariate Bessel functions and Heckman-Opdam hypergeometric functions as representation-valued integrals over dressing orbits. Our expression is the quasi-classical limit of the realization of Macdonald polynomials as traces of intertwiners of quantum groups given by Etingof and Kirillov Jr. Integration over the Liouville tori of the Gelfand-Tsetlin integrable system and adjunction for higher Calogero-Moser Hamiltonians relates our expression to the integral realization over Gelfand-Tsetlin polytopes which appeared in the recent work of Borodin and Gorin on the beta-Jacobi corners ensemble.

Finite dimensional representations of the rational Cherednik algebra for \(G_4\), Journal of Algebra 323 (2010), 2864-2887. (pdf) (arXiv) (journal)
In this paper, we study representations of the rational Cherednik algebra associated to the complex reflection group \(G_4\). In particular, we classify the irreducible finite dimensional representations and compute their characters.

computer science research

B. Hanin\(^*\) and Y. Sun\(^*\), Data augmentation as stochastic optimization. (arXiv)
We present a theoretical framework recasting data augmentation as stochastic optimization for a sequence of time-varying proxy losses. This provides a unified approach to understanding techniques commonly thought of as data augmentation, including synthetic noise and label-preserving transformations, as well as more traditional ideas in stochastic optimization such as learning rate and batch size scheduling. We prove a time-varying Monro-Robbins theorem with rates of convergence which gives conditions on the learning rate and augmentation schedule under which augmented gradient descent converges. Special cases give provably good joint schedules for augmentation with additive noise, minibatch SGD, and minibatch SGD with noise.

D. Kang\(^*\), Y. Sun\(^*\), D. Hendrycks, T. Brown, and J. Steinhardt, Testing robustness against unforeseen adversaries. (arXiv) (blog) (code) (workshop)
Considerable work on adversarial defense has studied robustness to a fixed, known family of adversarial distortions, most frequently L_p-bounded distortions. In reality, the specific form of attack will rarely be known and adversaries are free to employ distortions outside of any fixed set. The present work advocates measuring robustness against this much broader range of unforeseen attacks---attacks whose precise form is not known when designing a defense.

We propose a methodology for evaluating a defense against a diverse range of distortion types together with a summary metric UAR that measures the Unforeseen Attack Robustness against a distortion. We construct novel JPEG, Fog, Gabor, and Snow adversarial attacks to simulate unforeseen adversaries and perform a careful study of adversarial robustness against these and existing distortion types. We find that evaluation against existing L_p attacks yields highly correlated information that may not generalize to other attacks and identify a set of 4 attacks that yields more diverse information. We further find that adversarial training against either one or multiple distortions, including our novel ones, does not confer robustness to unforeseen distortions. These results underscore the need to study robustness against unforeseen distortions and provide a starting point for doing so.

T. Hashimoto, Y. Sun, and T. Jaakkola, From random walks to distances on unweighted graphs, NIPS 2015. (pdf) (arXiv) (supplement and code) (poster) (proceedings)
Large unweighted directed graphs are commonly used to capture relations between entities. A fundamental problem in the analysis of such networks is to properly define the similarity or dissimilarity between any two vertices. Despite the significance of this problem, statistical characterization of the proposed metrics has been limited.

We introduce and develop a class of techniques for analyzing random walks on graphs using stochastic calculus. Using these techniques we generalize results on the degeneracy of hitting times and analyze a metric based on the Laplace transformed hitting time (LTHT). The metric serves as a natural, provably well-behaved alternative to the expected hitting time. We establish a general correspondence between hitting times of the Brownian motion and analogous hitting times on the graph. We show that the LTHT is consistent with respect to the underlying metric of a geometric graph, preserves clustering tendency, and remains robust against random addition of non-geometric edges. Tests on simulated and real-world data show that the LTHT matches theoretical predictions and outperforms alternatives.

T. Hashimoto, Y. Sun, and T. Jaakkola, Metric recovery from directed unweighted graphs, AISTATS 2015. NIPS Networks Workshop 2014. (pdf) (arXiv) (supplement and code) (workshop) (poster) (proceedings)
We analyze directed, unweighted graphs obtained from \(x_i \in \mathbb{R}^d\) by connecting vertex \(i\) to \(j\) iff \(|x_i−x_j| < \varepsilon(x_i)\). Examples of such graphs include \(k\)-nearest neighbor graphs, where \(\varepsilon(x_i)\) varies from point to point, and, arguably, many real world graphs such as co-purchasing graphs. We ask whether we can recover the underlying Euclidean metric \(\varepsilon(x_i)\) and the associated density \(p(x_i)\) given only the directed graph and \(d\). We show that consistent recovery is possible up to isometric scaling when the vertex degree is at least \(\omega(n^{2/(2+d)}\log(n)^{d/(d+2)})\). Our estimator is based on a careful characterization of a random walk over the directed graph and the associated continuum limit. As an algorithm, it resembles the PageRank centrality metric. We demonstrate empirically that the estimator performs well on simulated examples as well as on real-world co-purchasing graphs even with a small number of points and degree scaling as low as \(\log(n)\).

Y. Sun and M. Sundararajan, Axiomatic attribution for multilinear functions, Electronic Commerce 2011, 177-178. (pdf) (arXiv) (extended abstract) (proceedings)
We study the attribution problem, that is, the problem of attributing a change in the value of a characteristic function to its independent variables. We make three contributions. First, we propose a formalization of the problem based on a standard cost sharing model. Second, we show that there is a unique attribution method that satisfies Dummy, Additivity, Conditional Nonnegativity, Affine Scale Invariance, and Anonymity for all characteristic functions that are the sum of a multilinear function and an additive function. We term this the Aumann-Shapley-Shubik method. Conversely, we show that such a uniqueness result does not hold for characteristic functions outside this class. Third, we study multilinear characteristic functions in detail; we describe a computationally efficient implementation of the Aumann-Shapley-Shubik method and discuss practical applications to pay-per-click advertising and portfolio analysis.

presentations

2021: UChicago (x2)

2020: UChicago, UW Madison, Google X, Bernoulli-IMS One World Symposium, DeepMath 2020, NeurIPS OPT 2020 Workshop

2019: UCSD, Virginia, OpenAI, ICML Workshop on Uncertainty and Robustness in Deep Learning, AMS Western Sectional Meeting

2018: Simons Society of Fellows Retreat, Yale

2017: ESI, Columbia-Princeton Probability Day, Rutgers, Perimeter Institute (video), Rochester, PCMI (video 1, video 2)

2016: HCM, MIT (x2), IESC, Columbia (x2)

2015: ICERM, AISTATS, CMI, FPSAC, Yale, Columbia, Northeastern, ETH Zurich (x2), UC Berkeley, NIPS

2014: MIT, IHP, UC Berkeley, NIPS Workshop on Networks