OLAP Practice Based on Apache Doris at Xiaomi

Jin Jiayi

Chinese Session 2023-08-20 15:45 GMT+8  #olap

Introduction: Apache Doris has been widely used in Xiaomi Group, mainly to meet the internal needs of report analysis and BI dashboards, supporting nearly 100 core businesses within Xiaomi and forming a data ecosystem with Apache Doris as the core. This presentation will focus on sharing the usage and application of Apache Doris in the OLAP scenario at Xiaomi.

Outline:

  • Historical selection and current application status Introduce the development history and selection situation of Xiaomi’s internal OLAP system and the current application status of Apache Doris in Xiaomi Group.
  • Support for group data dashboards Introduce the position and usage of Apache Doris in the group’s BI platform data flow, and Xiaomi’s permission management, metadata management mechanism, and data governance for Apache Doris.
  • Practice of Xiaomi’s user behavior analysis scenario Introduce the application practice of Apache Doris in Xiaomi’s user behavior analysis platform, including: (1) the implementation mechanism of user behavior analysis functions (funnel analysis, retention analysis, path analysis, event analysis, etc.) based on Apache Doris; (2) the application of bitmap precise deduplication, materialized view accelerated query, etc.; (3) the launch practice of Apache Doris vectorized version.
  • Pain points and future planning Introduce the pain points encountered by Xiaomi users in the process of using Apache Doris, and the future development plan (storage and computation separation, resource isolation, cloud-native solutions, etc.).

Speakers:


Wei Zuo is a database kernel development engineer at Xiaomi and a PMC member of Apache Doris. He is responsible for the research and development, optimization, and operation and maintenance of the OLAP system at Xiaomi Group.