Data Science Colloquium

of the ENS

Welcome to the Data Science Colloquium of the ENS.

This colloquium is organized around data sciences in a broad sense with the goal of bringing together researchers with diverse backgrounds (including for instance mathematics, computer science, physics, chemistry and neuroscience) but a common interest in dealing with large scale or high dimensional data.

The colloquium is followed by an open buffet around which participants can meet and discuss collaborations.

These seminars are made possible by the support of the CFM-ENS Chair “Modèles et Sciences des Données”.

You can check the list of the next seminars below and the list of past seminars.

Videos of some of the past seminars are available online.

Organizers

The colloquium is organized by:

Next seminars

2 June 2026, 12h00-13h00 (Paris time), room Amphi Jaurès (29 Rue d'Ulm).
Rebecca Willett (University of Chicago)
Title: How do simple rotations affect the implicit bias of Adam?
Abstract: Adaptive gradient methods such as Adam and Adagrad are widely used in machine learning, yet their effect on the generalization of learned models – relative to methods like gradient descent – remains poorly understood. Prior work on binary classification suggests that Adam exhibits a “richness bias,” which can help it learn nonlinear decision boundaries closer to the Bayes-optimal decision boundary relative to gradient descent. However, the coordinate-wise preconditioning scheme employed by Adam renders the overall method sensitive to orthogonal transformations of feature space. We show that this sensitivity can manifest as a reversal of Adam’s competitive advantage: even small rotations of the underlying data distribution can make Adam forfeit its richness bias and converge to a linear decision boundary that is farther from the Bayes-optimal decision boundary than the one learned by gradient descent. To alleviate this issue, we show that a recently proposed reparameterization method – which applies an orthogonal transformation to the optimization objective – endows any first-order method with equivariance to data rotations, and we empirically demonstrate its ability to restore Adam’s bias towards rich decision boundaries. This is joint work with Adela DePavia and Vasileios Charisopoulos.

Supported by