Design and Analysis of ML Algorithms for Efficient and Fair Operations in Large-Scale Systems

Abstract
The pervasive integration of Machine Learning (ML) and Artificial Intelligence (AI) into large-scale systems is reshaping our interactions within physical infrastructure and other online services. Unfortunately, the conventional paradigm of ML, relying on iid data from a stationary environment, proves inadequate in such contexts. Specifically, there are two key challenges when using the conventional ML paradigm for design and analysis in such large-scale systems. First, algorithms simultaneously learn and adapt in dynamic, uncertain, and resource-constrained environments in these systems. Typically, the algorithms cannot communicate or coordinate with each other, resulting in non-stationarity in the learning data received by any algorithm. Second, the data generated in such large-scale systems is typically used to make consequential design decisions, such as designing tolls on transportation networks, or developing cost-effective AI-driven financial and medical services. As the data in such systems is generated from the strategic responses of reactive users, these datasets are biased and correlated. This underscores the necessity for a nuanced design of ML algorithms for efficient and fair operations in large-scale systems.

Drawing on examples from my research, I will highlight how to incorporate tools from diverse disciplines, such as learning theory, algorithmic game theory, market design, optimization, and dynamical systems, to effectively overcome the challenges associated with the standard ML paradigm. Specifically, I will discuss design and analysis techniques for novel decentralized learning algorithms, which do not communicate or coordinate with each other, in two prevalent frameworks of multi-agent interactions: two-sided matching markets and multi-agent reinforcement learning. Additionally, by leveraging high-fidelity datasets capturing real-world, large-scale interactions in the San Francisco Bay Area freeway network, I will illustrate how game-theoretic modeling can be fused within ML pipeline to design tolling schemes that not only alleviate congestion but also minimize the disproportionate impact based on the income level of users.

The talk will conclude by highlighting open directions in the design and analysis of ML algorithms to support efficient and fair decision-making in large-scale systems.

Bio
Chinmay Maheshwari is PhD candidate in the Electrical Engineering and Computer Sciences department at University of California Berkeley. He obtained M. Tech and B. Tech degrees from Indian Institute of Technology (IIT) Bombay, both in 2019. His research focuses on developing theoretical, algorithmic, and methodological foundations for the design and analysis of machine learning algorithms operating in large-scale physical infrastructure or online services. On the technical side, his research builds on and extends tools and techniques from machine learning, algorithmic game theory, market design, dynamical systems, control theory and optimization.