DBScholar

Back to papers

Data Management Challenges in Production Machine Learning

Summary: Survey of data-management challenges in production ML pipelines, focusing on understanding, validating, cleaning, and enriching training data. Connects to database literature and outlines open questions on data quality, provenance, validation, and enrichment not yet addressed by prior art. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
5402
Venue
SIGMOD
Year
2017
Pagerank
0.00011974846
Overall Rank
1,147 | 92.14%
DOI
10.1145/3035918.3054782

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{polyzotis_sigmod17,
        title = {{Data Management Challenges in Production Machine Learning}},
        author = {Polyzotis, Neoklis and Roy, Sudip and Whang, Steven Euijong and Zinkevich, Martin},
        series = {{SIGMOD} '17},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3035918.3054782},
        url = {https://dl.acm.org/doi/10.1145/3035918.3054782},
        year = {2017}
}

Incoming Citations (Sorted by Pagerank)

Showing 28 of 28 citing papers.

Rank Citing Paper Year Venue Pagerank
1,350 Automating Large-Scale Data Quality Verification 2018 VLDB 0.00011065626
1,488 Towards Model-based Pricing for Machine Learning in a Data Marketplace 2019 SIGMOD 0.00010612416
2,273 SliceLine: Fast, Linear-Algebra-based Slice Finding for ML Model Debugging 2021 SIGMOD 8.8230899e-05
2,584 Complaint-driven Training Data Debugging for Query 2.0 2020 SIGMOD 8.3783546e-05
2,657 Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities 2021 SIGMOD 8.2887895e-05
2,927 In-Database Learning with Sparse Tensors 2018 PODS 7.9531195e-05
3,206 Incremental View Maintenance with Triple Lock Factorization Benefits 2018 SIGMOD 7.6367549e-05
3,681 Are Key-Foreign Key Joins Safe to Avoid when Learning High-Capacity Classifiers? 2018 VLDB 7.2037388e-05
4,012 Opportunities for Quantum Acceleration of Databases: Optimization of Queries and Transaction Schedules 2023 VLDB 6.9578554e-05
4,100 Data Integration and Machine Learning: A Natural Synergy 2018 SIGMOD 6.9016092e-05
4,128 The Relational Data Borg is Learning 2020 VLDB 6.8850804e-05
4,130 PrIU: A Provenance-Based Approach for Incrementally Updating Regression Models 2020 SIGMOD 6.8839621e-05
4,658 OmniFair: A Declarative System for Model-Agnostic Group Fairness in Machine Learning 2021 SIGMOD 6.5817368e-05
5,040 TPCx-AI - An Industry Standard Benchmark for Artificial Intelligence and Machine Learning Systems 2023 VLDB 6.390395e-05
5,131 Enabling SQL-based Training Data Debugging for Federated Learning 2022 VLDB 6.3537809e-05
5,681 Survivability of Cloud Databases - Factors and Prediction 2018 SIGMOD 6.1237253e-05
6,315 Data Collection and Quality Challenges for Deep Learning 2020 VLDB 5.915447e-05
6,431 Finding Label and Model Errors in Perception Data With Learned Observation Assertions 2022 SIGMOD 5.8802622e-05
6,538 UPLIFT: Parallelization Strategies for Feature Transformations in Machine Learning Workloads 2022 VLDB 5.8477764e-05
6,801 Unit Testing Data with Deequ 2019 SIGMOD 5.7690726e-05
7,232 Saga: A Scalable Framework for Optimizing Data Cleaning Pipelines for Machine Learning Applications 2023 SIGMOD 5.6659017e-05
7,424 Data Integration and Machine Learning: A Natural Synergy 2018 VLDB 5.6214566e-05
7,690 Auto-Validate: Unsupervised Data Validation Using Data-Domain Patterns Inferred from Data Lakes 2021 SIGMOD 5.5669307e-05
7,855 ItemSuggest: A Data Management Platform for Machine Learned Ranking Services 2019 CIDR 5.530673e-05
9,245 Towards Observability for Production Machine Learning Pipelines 2022 VLDB 5.2992628e-05
11,516 Data Management Opportunities for Foundation Models 2022 CIDR 5.093636e-05
11,685 Toto - Benchmarking the Efficiency of a Cloud Service 2021 SIGMOD 5.093636e-05
13,501 DEEM 2019: Workshop on Data Management for End-to-End Machine Learning 2019 SIGMOD -
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 9 of 9 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
347 Tuning Database Configuration Parameters with iTuned 2009 VLDB 0.00020651582
363 Approximate Query Processing: Taming the TeraBytes! A Tutorial 2001 VLDB 0.0002005475
410 SeeDB: Efficient Data-Driven Visualization Recommendations to Support Visual Analytics 2015 VLDB 0.0001890421
514 Goods: Organizing Google's Datasets 2016 SIGMOD 0.00017178673
582 ActiveClean: Interactive Data Cleaning For Statistical Modeling 2016 VLDB 0.00016148948
764 To Join or Not to Join? Thinking Twice about Joins before Feature Selection 2016 SIGMOD 0.00014226652
898 Querying and Mining of Time Series Data: Experimental Comparison of Representations and Distance Measures 2008 VLDB 0.00013339042
915 Intelligent Rollups in Multidimensional OLAP Data 2001 VLDB 0.00013243386
965 User-adaptive exploration of multidimensional data 2000 VLDB 0.00012933878
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 11,516 Data Management Opportunities for Foundation Models 2022 CIDR
2 5,925 Managing ML Pipelines: Feature Stores and the Coming Wave of Embedding Ecosystems 2021 VLDB
3 11,066 ML-Asset Management: Curation, Discovery, and Utilization 2025 VLDB
4 4,859 Machine Learning for Big Data 2013 SIGMOD
5 749 Data Lake Management: Challenges and Opportunities 2019 VLDB
6 9,245 Towards Observability for Production Machine Learning Pipelines 2022 VLDB
7 6,315 Data Collection and Quality Challenges for Deep Learning 2020 VLDB
8 7,661 Machine Learning for Cloud Data Systems: the Progress so far and the Path Forward 2021 VLDB
9 2,657 Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities 2021 SIGMOD
10 1,250 Data Management in Machine Learning: Challenges, Techniques, and Systems 2017 SIGMOD