DBScholar

Back to papers

In-Database Machine Learning with CorgiPile: Stochastic Gradient Descent without Full Data Shuffle

Summary: Proposes CorgiPile, a hierarchical data shuffling method for in-database SGD that avoids full shuffles yet preserves convergence. Systematic study of existing shuffles, convergence theory, and PostgreSQL integration via three new operators; achieves 1.6–12.8× speedups over MADlib/Bismarck on HDD/SSD. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h797de3fca3c68a69
Venue
SIGMOD
Year
2022
Pagerank
5.7773842e-05
Overall Rank
6,458 | 56.59%
DOI
10.1145/3514221.3526150

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@inproceedings{xu_sigmod22,
        title = {{In-Database Machine Learning with CorgiPile: Stochastic Gradient Descent without Full Data Shuffle}},
        author = {Xu, Lijie and Qiu, Shuang and Yuan, Binhang and Jiang, Jiawei and Renggli, Cedric and Gan, Shaoduo and Kara, Kaan and Li, Guoliang and Liu, Ji and Wu, Wentao and Ye, Jieping and Zhang, Ce},
        series = {{SIGMOD} '22},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3514221.3526150},
        url = {https://dl.acm.org/doi/10.1145/3514221.3526150},
        year = {2022}
}

Incoming Citations (Sorted by Pagerank)

Showing 5 of 5 citing papers.

Rank Citing Paper Year Venue Pagerank
9,212 Powering In-Database Dynamic Model Slicing for Structured Data Analytics 2024 VLDB 5.2060505e-05
10,342 Towards Communication-efficient Vertical Federated Learning Training via Cache-enabled Local Updates 2022 VLDB 5.0167904e-05
10,582 NeurStore: Efficient In-database Deep Learning Model Management System 2026 SIGMOD 4.9793485e-05
11,250 GPEmu: A GPU Emulator for Faster and Cheaper Prototyping and Evaluation of Deep Learning System Research 2025 VLDB 4.9793485e-05
11,549 Database Native Model Selection: Harnessing Deep Neural Networks in Database Systems 2024 VLDB 4.9793485e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 20 of 20 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
105 The MADlib Analytics Library or MAD Skills, the SQL 2012 VLDB 0.00033638251
503 Towards a Unified Architecture for in-RDBMS Analytics 2012 SIGMOD 0.00017202276
521 Learning Linear Regression Models over Factorized Joins 2016 SIGMOD 0.00016929744
730 Learning Generalized Linear Models Over Normalized Data 2015 SIGMOD 0.00014406936
851 Scaling Factorization Machines to Relational Data 2013 VLDB 0.00013457975
1,009 Simulation of Database-Valued Markov Chains Using SimSQL 2013 SIGMOD 0.00012539827
1,152 Cerebro: A Data System for Optimized Deep Learning Model Selection 2020 VLDB 0.00011801961
1,167 DimmWitted: A Study of Main-Memory Statistical Analytics 2014 VLDB 0.00011729888
1,256 Towards Linear Algebra over Normalized Data 2017 VLDB 0.00011314687
2,184 Heterogeneity-aware Distributed Parameter Servers 2017 SIGMOD 8.8958335e-05
2,386 Vertica-ML: Distributed Machine Learning in Vertica Database 2020 SIGMOD 8.538263e-05
2,400 Declarative Recursive Computation on an RDBMS or, Why You Should Use a Database For Distributed Machine Learning 2019 VLDB 8.5213814e-05
2,830 DB4ML – An In-Memory Database Kernel with Machine Learning Support 2020 SIGMOD 7.9619979e-05
2,975 In-Database Learning with Sparse Tensors 2018 PODS 7.7907759e-05
3,373 F: Regression Models over Factorized Views 2016 VLDB 7.3661025e-05
4,095 Distributed Deep Learning on Data Systems: A Comparative Analysis of Approaches 2021 VLDB 6.8095767e-05
5,289 Tensor Relational Algebra for Distributed Machine Learning System Design 2021 VLDB 6.1940481e-05
6,169 Automatic Optimization of Matrix Implementations for Distributed Machine Learning and Linear Algebra 2021 SIGMOD 5.8624857e-05
6,246 ColumnML: Column-Store Machine Learning with On-The-Fly Data Transformation 2019 VLDB 5.8370793e-05
10,021 Distributed Numerical and Machine Learning Computations via Two-Phase Execution of Aggregated Join Trees 2021 VLDB 5.0943545e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 1,475 Analyzing and Mitigating Data Stalls in DNN Training 2021 VLDB
2 6,308 Is Your Learned Query Optimizer Behaving As You Expect? A Machine Learning Perspective 2024 VLDB
3 6,844 Serving Deep Learning Models with Deduplication from Relational Databases 2022 VLDB
4 7,363 PerfGuard: Deploying ML-for-Systems without Performance Regressions, Almost! 2021 VLDB
5 6,592 Tuple-oriented Compression for Large-scale Mini-batch Stochastic Gradient Descent 2019 SIGMOD
6 9,556 Towards an Optimized GROUP BY Abstraction for Large-Scale Machine Learning 2021 VLDB
7 10,154 Structure-Aware Machine Learning over Multi-Relational Databases 2021 SIGMOD
8 7,201 Coresets over Multiple Tables for Feature-rich and Data-efficient Machine Learning 2023 VLDB
9 6,246 ColumnML: Column-Store Machine Learning with On-The-Fly Data Transformation 2019 VLDB
10 4,901 Scalable Asynchronous Gradient Descent Optimization for Out-of-Core Models 2017 VLDB