Load Balancing under Data Locality: Extending Mean-Field Framework to Constrained Large-Scale Systems

Add to your list(s) Download to your calendar using vCal

• Dr Debankur Mukherjee, Georgia Tech
• Wednesday 04 October 2023, 14:00-15:00
• MR5, CMS Pavilion A.

If you have a question about this talk, please contact Dr Varun Jog.

Large-scale parallel-processing infrastructures such as data centers and cloud networks form the cornerstone of the modern digital environment. Central to their efficiency are resource management policies, especially load balancing algorithms (LBAs), which are crucial for meeting stringent delay requirements of tasks. A contemporary challenge in designing LBAs for today’s data centers is navigating data locality constraints that dictate which tasks are assigned to which servers. These constraints can be naturally modeled as a bipartite graph between servers and various task types. Most LBA heuristics lean on the mean-field approximation’s accuracy. However, the non-exchangeability among servers induced by the data locality invalidates this mean-field framework, causing real-world system behaviors to significantly diverge from theoretical predictions. From a foundational standpoint, advancing our understanding in this domain demands the study of stochastic processes on large graphs, thus needing fundamental advancements in classical analytical tools.

In this presentation, we will delve into recent advancements made in extending the accuracy of mean-field approximation for a broad class of graphs. In particular, we will talk about how to design resource-efficient, asymptotically optimal data locality constraints and how the system behavior changes fundamentally, depending on whether the above bipartite graph is an expander, a spatial graph, or is inhomogeneous in nature.

Bio:

Debankur Mukherjee is the Leo and Louise Benatar Early Career Professor and Assistant Professor in the H. Milton Stewart School of Industrial and Systems Engineering at the Georgia Institute of Technology. Before joining Georgia Tech in 2019, he was a Prager assistant professor for a year in the Division of Applied Mathematics at Brown University. Debankur got his Ph.D. in Stochastic Operations Research from the Eindhoven University of Technology in the Netherlands. Debankur’s research spans the area of applied probability, at the interface of stochastic processes and computer science, with applications to performance analysis, online algorithms, and machine learning. His primary focus is to develop a foundational understanding of the challenges that arise in large-scale systems, such as data centers and cloud networks. His work was a finalist in the INFORMS JFIG paper competition in 2022 and INFORMS George Nicholson Student Paper Competition 2023 and received the Best Paper Award at ACM SIGMETRICS 2023 and the Best Student Paper Award at ACM SIGMETRICS 2018 . His research has been funded by the NSF and he is currently serving on the editorial boards of Stochastic Systems, QUESTA , and Stochastic Models.

This talk is part of the Information Theory Seminar series.

This talk is included in these lists:

• All CMS events
• All Talks (aka the CURE list)
• CMS Events
• DPMMS Lists
• DPMMS info aggregator
• DPMMS lists
• Hanchen DaDaDash
• Information Theory Seminar
• Interested Talks
• MR5, CMS Pavilion A
• School of Physical Sciences
• Statistical Laboratory info aggregator
• bld31

Note that ex-directory lists are not shown.