Differentially private M-estimation via noisy optimization

Add to your list(s) Download to your calendar using vCal

• Po-Ling Loh
• Thursday 25 April 2024, 15:00-16:00
• Venue to be confirmed.

If you have a question about this talk, please contact Catarina.

We present a noisy composite gradient descent algorithm for differentially private statistical estimation in high dimensions. We begin by providing general rates of convergence for the parameter error of successive iterates under assumptions of local restricted strong convexity and local restricted smoothness. Our analysis is local, in that it ensures a linear rate of convergence when the initial iterate lies within a constant-radius region of the true parameter. At each iterate, multivariate Gaussian noise is added to the gradient in order to guarantee that the output satisfies Gaussian differential privacy. We then derive consequences of our theory for linear regression and mean estimation. Motivated by M-estimators used in robust statistics, we study loss functions which downweight the contribution of individual data points in such a way that the sensitivity of function gradients is guaranteed to be bounded, even without the usual assumption that our data lie in a bounded domain. We prove that the objective functions thus obtained indeed satisfy the restricted convexity and restricted smoothness conditions required for our general theory. We then show how the private estimators obtained by noisy composite gradient descent may be used to obtain differentially private confidence intervals for regression coefficients, by leveraging work in Lasso debiasing proposed in high-dimensional statistics. We complement our theoretical results with simulations that illustrate the favorable finite-sample performance of our methods.

This talk is part of the Cambridge Ellis Seminar series – Prof Po-Ling Loh – 25 Apr 2024 @ 15:00 series.

This talk is included in these lists:

• Cambridge Ellis Seminar series – Prof Po-Ling Loh – 25 Apr 2024 @ 15:00
• Cambridge Ellis Unit
• Hanchen DaDaDash
• Interested Talks
• Venue to be confirmed

Note that ex-directory lists are not shown.