Unlocking the Power of CI/CD for Data Pipelines in Distributed Data Warehouses
Summary: Production-configuration-driven CI tests distributed warehouse pipelines in isolated production environments, avoiding costly replicas while preserving fidelity. Lineage-aware impact analysis propagates quality checks across owners; deployed at YouTube, it detects 94.5% of pre-production issues. (summarized by gpt-5.6-luna on Jul 24 2026)
Incoming Non-self Citations Over Time
No non-self incoming citations found for this paper in this database.
Authors
- 1. Hongtao Yang (Google)
- 2. Zhichen Xu (Google)
- 3. Sergey Yudin (Google)
- 4. Andrew Davidson (Google)
BibTeX Citation
@article{yang_vldb25,
title = {{Unlocking the Power of CI/CD for Data Pipelines in Distributed Data Warehouses}},
author = {Yang, Hongtao and Xu, Zhichen and Yudin, Sergey and Davidson, Andrew},
journal = {PVLDB},
series = {{VLDB} '25},
volume = {18},
number = {12},
pages = {4887--4895},
doi = {10.14778/3750601.3750613},
url = {https://doi.org/10.14778/3750601.3750613},
year = {2025}
}Incoming Citations (Sorted by Pagerank)
Showing 0 of 0 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 3 of 3 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank | Cited Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 1,494 | Procella: Unifying serving and analytical data at YouTube | 2019 | VLDB | 0.00010577585 |
| 3,630 | TensorFlow Data Validation: Data Analysis and Validation in Continuous ML Pipelines | 2020 | SIGMOD | 7.2387749e-05 |
| 10,063 | Keep Your Distributed Data Warehouse Consistent at a Minimal Cost | 2023 | SIGMOD | 5.1650158e-05 |
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 4,756 | Improving Reproducibility of Data Science Pipelines through Transparent Provenance Capture | 2020 | VLDB |
| 2 | 10,761 | Fast and Scalable Data Transfer Across Data Systems | 2025 | SIGMOD |
| 3 | 1,147 | Data Management Challenges in Production Machine Learning | 2017 | SIGMOD |
| 4 | 585 | Lineage Tracing for General Data Warehouse Transformations | 2001 | VLDB |
| 5 | 11,784 | A Framework for Emulating Database Operations in Cloud Data Warehouses | 2020 | SIGMOD |
| 6 | 6,309 | Materialization and Reuse Optimizations for Production Data Science Pipelines | 2022 | SIGMOD |
| 7 | 2,657 | Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities | 2021 | SIGMOD |
| 8 | 1,350 | Automating Large-Scale Data Quality Verification | 2018 | VLDB |
| 9 | 10,063 | Keep Your Distributed Data Warehouse Consistent at a Minimal Cost | 2023 | SIGMOD |
| 10 | 9,245 | Towards Observability for Production Machine Learning Pipelines | 2022 | VLDB |