DBScholar

Back to papers

Data Management Challenges in Production Machine Learning

Summary: Survey of data-management challenges in production ML pipelines, focusing on understanding, validating, cleaning, and enriching training data. Connects to database literature and outlines open questions on data quality, provenance, validation, and enrichment not yet addressed by prior art. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h3f0f203f6d885cbb
Venue
SIGMOD
Year
2017
Pagerank
0.00011798912
Overall Rank
1,153 | 92.25%
DOI
10.1145/3035918.3054782

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{polyzotis_sigmod17,
        title = {{Data Management Challenges in Production Machine Learning}},
        author = {Polyzotis, Neoklis and Roy, Sudip and Whang, Steven Euijong and Zinkevich, Martin},
        series = {{SIGMOD} '17},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3035918.3054782},
        url = {https://dl.acm.org/doi/10.1145/3035918.3054782},
        year = {2017}
}

Incoming Citations (Sorted by Pagerank)

Showing 28 of 28 citing papers.

Rank Citing Paper Year Venue Pagerank
1,308 Automating Large-Scale Data Quality Verification 2018 VLDB 0.0001107886
1,362 Towards Model-based Pricing for Machine Learning in a Data Marketplace 2019 SIGMOD 0.0001091784
2,187 Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities 2021 SIGMOD 8.8896655e-05
2,326 SliceLine: Fast, Linear-Algebra-based Slice Finding for ML Model Debugging 2021 SIGMOD 8.6309237e-05
2,598 Complaint-driven Training Data Debugging for Query 2.0 2020 SIGMOD 8.2385793e-05
2,975 In-Database Learning with Sparse Tensors 2018 PODS 7.7907759e-05
3,112 Incremental View Maintenance with Triple Lock Factorization Benefits 2018 SIGMOD 7.6357579e-05
3,226 Opportunities for Quantum Acceleration of Databases: Optimization of Queries and Transaction Schedules 2023 VLDB 7.5095541e-05
3,670 Are Key-Foreign Key Joins Safe to Avoid when Learning High-Capacity Classifiers? 2018 VLDB 7.1108704e-05
4,161 The Relational Data Borg is Learning 2020 VLDB 6.7700593e-05
4,171 Data Integration and Machine Learning: A Natural Synergy 2018 SIGMOD 6.7608137e-05
4,195 PrIU: A Provenance-Based Approach for Incrementally Updating Regression Models 2020 SIGMOD 6.7422305e-05
4,719 OmniFair: A Declarative System for Model-Agnostic Group Fairness in Machine Learning 2021 SIGMOD 6.4548076e-05
4,925 TPCx-AI - An Industry Standard Benchmark for Artificial Intelligence and Machine Learning Systems 2023 VLDB 6.3511742e-05
5,071 Data Collection and Quality Challenges for Deep Learning 2020 VLDB 6.2879908e-05
5,247 Enabling SQL-based Training Data Debugging for Federated Learning 2022 VLDB 6.2121767e-05
5,572 Saga: A Scalable Framework for Optimizing Data Cleaning Pipelines for Machine Learning Applications 2023 SIGMOD 6.0802555e-05
5,766 Survivability of Cloud Databases - Factors and Prediction 2018 SIGMOD 6.0025753e-05
6,552 Finding Label and Model Errors in Perception Data With Learned Observation Assertions 2022 SIGMOD 5.750057e-05
6,662 UPLIFT: Parallelization Strategies for Feature Transformations in Machine Learning Workloads 2022 VLDB 5.7171651e-05
6,898 Unit Testing Data with Deequ 2019 SIGMOD 5.6530554e-05
7,415 Data Integration and Machine Learning: A Natural Synergy 2018 VLDB 5.5332662e-05
7,738 Auto-Validate: Unsupervised Data Validation Using Data-Domain Patterns Inferred from Data Lakes 2021 SIGMOD 5.4623434e-05
8,012 ItemSuggest: A Data Management Platform for Machine Learned Ranking Services 2019 CIDR 5.4072178e-05
9,417 Towards Observability for Production Machine Learning Pipelines 2022 VLDB 5.1803615e-05
11,825 Data Management Opportunities for Foundation Models 2022 CIDR 4.9793485e-05
11,992 Toto - Benchmarking the Efficiency of a Cloud Service 2021 SIGMOD 4.9793485e-05
13,815 DEEM 2019: Workshop on Data Management for End-to-End Machine Learning 2019 SIGMOD -
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 9 of 9 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
322 Tuning Database Configuration Parameters with iTuned 2009 VLDB 0.00021041865
372 Approximate Query Processing: Taming the TeraBytes! A Tutorial 2001 VLDB 0.00019720059
395 SeeDB: Efficient Data-Driven Visualization Recommendations to Support Visual Analytics 2015 VLDB 0.00019165452
483 ActiveClean: Interactive Data Cleaning For Statistical Modeling 2016 VLDB 0.00017590977
509 Goods: Organizing Google's Datasets 2016 SIGMOD 0.00017071087
777 To Join or Not to Join? Thinking Twice about Joins before Feature Selection 2016 SIGMOD 0.00014054709
915 Querying and Mining of Time Series Data: Experimental Comparison of Representations and Distance Measures 2008 VLDB 0.00013099703
925 Intelligent Rollups in Multidimensional OLAP Data 2001 VLDB 0.00013051939
984 User-adaptive exploration of multidimensional data 2000 VLDB 0.00012686375
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 11,825 Data Management Opportunities for Foundation Models 2022 CIDR
2 5,506 Managing ML Pipelines: Feature Stores and the Coming Wave of Embedding Ecosystems 2021 VLDB
3 11,426 ML-Asset Management: Curation, Discovery, and Utilization 2025 VLDB
4 4,938 Machine Learning for Big Data 2013 SIGMOD
5 624 Data Lake Management: Challenges and Opportunities 2019 VLDB
6 9,417 Towards Observability for Production Machine Learning Pipelines 2022 VLDB
7 5,071 Data Collection and Quality Challenges for Deep Learning 2020 VLDB
8 7,806 Machine Learning for Cloud Data Systems: the Progress so far and the Path Forward 2021 VLDB
9 2,187 Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities 2021 SIGMOD
10 1,255 Data Management in Machine Learning: Challenges, Techniques, and Systems 2017 SIGMOD