DBScholar

Back to papers

Data Management Challenges in Production Machine Learning

Summary: Survey of data-management challenges in production ML pipelines, focusing on understanding, validating, cleaning, and enriching training data. Connects to database literature and outlines open questions on data quality, provenance, validation, and enrichment not yet addressed by prior art. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h3f0f203f6d885cbb
Venue
SIGMOD
Year
2017
Pagerank
0.00011793347
Overall Rank
1,153 | 92.26%
DOI
10.1145/3035918.3054782
PDF
Download (CC BY 4.0)

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{polyzotis_sigmod17,
        title = {{Data Management Challenges in Production Machine Learning}},
        author = {Polyzotis, Neoklis and Roy, Sudip and Whang, Steven Euijong and Zinkevich, Martin},
        series = {{SIGMOD} '17},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3035918.3054782},
        url = {https://dl.acm.org/doi/10.1145/3035918.3054782},
        year = {2017}
}

Incoming Citations (Sorted by Pagerank)

Showing 28 of 28 citing papers.

Rank Citing Paper Year Venue Pagerank
1,308 Automating Large-Scale Data Quality Verification 2018 VLDB 0.00011073863
1,363 Towards Model-based Pricing for Machine Learning in a Data Marketplace 2019 SIGMOD 0.00010912672
2,189 Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities 2021 SIGMOD 8.8854572e-05
2,329 SliceLine: Fast, Linear-Algebra-based Slice Finding for ML Model Debugging 2021 SIGMOD 8.6268411e-05
2,600 Complaint-driven Training Data Debugging for Query 2.0 2020 SIGMOD 8.2346824e-05
2,978 In-Database Learning with Sparse Tensors 2018 PODS 7.7872011e-05
3,114 Incremental View Maintenance with Triple Lock Factorization Benefits 2018 SIGMOD 7.6321464e-05
3,228 Opportunities for Quantum Acceleration of Databases: Optimization of Queries and Transaction Schedules 2023 VLDB 7.5059992e-05
3,673 Are Key-Foreign Key Joins Safe to Avoid when Learning High-Capacity Classifiers? 2018 VLDB 7.1075403e-05
4,161 The Relational Data Borg is Learning 2020 VLDB 6.7669004e-05
4,171 Data Integration and Machine Learning: A Natural Synergy 2018 SIGMOD 6.7576159e-05
4,197 PrIU: A Provenance-Based Approach for Incrementally Updating Regression Models 2020 SIGMOD 6.7390419e-05
4,721 OmniFair: A Declarative System for Model-Agnostic Group Fairness in Machine Learning 2021 SIGMOD 6.451752e-05
4,926 TPCx-AI - An Industry Standard Benchmark for Artificial Intelligence and Machine Learning Systems 2023 VLDB 6.3481677e-05
5,074 Data Collection and Quality Challenges for Deep Learning 2020 VLDB 6.2850141e-05
5,251 Enabling SQL-based Training Data Debugging for Federated Learning 2022 VLDB 6.2092359e-05
5,574 Saga: A Scalable Framework for Optimizing Data Cleaning Pipelines for Machine Learning Applications 2023 SIGMOD 6.0773771e-05
5,767 Survivability of Cloud Databases - Factors and Prediction 2018 SIGMOD 5.9997338e-05
6,554 Finding Label and Model Errors in Perception Data With Learned Observation Assertions 2022 SIGMOD 5.7473435e-05
6,666 UPLIFT: Parallelization Strategies for Feature Transformations in Machine Learning Workloads 2022 VLDB 5.7144587e-05
6,900 Unit Testing Data with Deequ 2019 SIGMOD 5.6503837e-05
7,418 Data Integration and Machine Learning: A Natural Synergy 2018 VLDB 5.5306468e-05
7,742 Auto-Validate: Unsupervised Data Validation Using Data-Domain Patterns Inferred from Data Lakes 2021 SIGMOD 5.4599258e-05
8,017 ItemSuggest: A Data Management Platform for Machine Learned Ranking Services 2019 CIDR 5.4046581e-05
9,426 Towards Observability for Production Machine Learning Pipelines 2022 VLDB 5.1779092e-05
11,831 Data Management Opportunities for Foundation Models 2022 CIDR 4.9769913e-05
11,998 Toto - Benchmarking the Efficiency of a Cloud Service 2021 SIGMOD 4.9769913e-05
13,820 DEEM 2019: Workshop on Data Management for End-to-End Machine Learning 2019 SIGMOD -
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 9 of 9 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
322 Tuning Database Configuration Parameters with iTuned 2009 VLDB 0.00021034201
372 Approximate Query Processing: Taming the TeraBytes! A Tutorial 2001 VLDB 0.0001971778
395 SeeDB: Efficient Data-Driven Visualization Recommendations to Support Visual Analytics 2015 VLDB 0.00019156481
483 ActiveClean: Interactive Data Cleaning For Statistical Modeling 2016 VLDB 0.00017584249
509 Goods: Organizing Google's Datasets 2016 SIGMOD 0.00017063491
779 To Join or Not to Join? Thinking Twice about Joins before Feature Selection 2016 SIGMOD 0.00014048128
916 Querying and Mining of Time Series Data: Experimental Comparison of Representations and Distance Measures 2008 VLDB 0.00013093656
925 Intelligent Rollups in Multidimensional OLAP Data 2001 VLDB 0.00013045856
984 User-adaptive exploration of multidimensional data 2000 VLDB 0.00012680398
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 11,831 Data Management Opportunities for Foundation Models 2022 CIDR
2 5,509 Managing ML Pipelines: Feature Stores and the Coming Wave of Embedding Ecosystems 2021 VLDB
3 11,432 ML-Asset Management: Curation, Discovery, and Utilization 2025 VLDB
4 4,935 Machine Learning for Big Data 2013 SIGMOD
5 624 Data Lake Management: Challenges and Opportunities 2019 VLDB
6 9,426 Towards Observability for Production Machine Learning Pipelines 2022 VLDB
7 5,074 Data Collection and Quality Challenges for Deep Learning 2020 VLDB
8 7,813 Machine Learning for Cloud Data Systems: the Progress so far and the Path Forward 2021 VLDB
9 2,189 Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities 2021 SIGMOD
10 1,223 Data Management in Machine Learning: Challenges, Techniques, and Systems 2017 SIGMOD