RegionFed: Federated Learning for Personalized Query Understanding in Heterogeneous Retail Environments

Imagine you're building a retail search system that needs to cater to different regions, each with its own unique preferences and query styles. The challenge is that while federated learning (FL) can help protect user privacy by training models across devices, traditional FL methods often create a single global model that doesn't perform well for specific regions. This is particularly problematic when using modern transformer models, which can struggle significantly under these conditions, leading to poor accuracy. This issue is known as model collapse, where the model fails to adapt to the diverse data it encounters.

To tackle this, the authors propose RegionFed, a federated learning framework that operates at the gradient level rather than the parameter level. This means it can effectively diagnose the differences between regional and global data and adaptively choose the best personalization strategy for each region without compromising performance. By treating models as black boxes, RegionFed can be applied to various architectures like T5 and RoBERTa without any code changes. The results are promising, showing significant accuracy improvements across multiple datasets while maintaining strong privacy guarantees. For builders, this means a more effective way to deploy personalized models in diverse environments without sacrificing user privacy.

RegionFed introduces a novel gradient-level approach to federated learning that addresses regional performance issues.

The evaluation across multiple datasets and architectures shows solid improvements, though some metrics are close to the centralized upper bound.