Learned Distributed Query Optimizer: Architecture and Challenges

Abstract: The query processing in distributed database management systems (DBMS) faces more challenges, such as more operators, more factors in cost models and meta-data, than those in the single-node DMBS, in which query optimization is already an NP-hard problem. Learned query optimizer (mainly in a single-node DBMS) receives attention due to its capability to capture data distributions and flexible way to avoid hard-craft rules in refinement and adaptation to new hardware. In this paper, we focus on extensions of learned query optimizers to distributed DBMSs. Specifically, we propose one possible but general architecture of the learned query optimizer in the distributed context and highlight differences from the learned optimizer in the single-node ones. In addition, we discuss the challenges and possible solutions.

Keywords: distributed query processing; query optimization; learned query optimizer