DBScholar

Back to papers

Towards Scalable Dataframe Systems

Summary: Scalable dataframe systems via MODIN; scaling pandas-like APIs with a simple dataframe data model and algebra. Signature features: flexible schemas, ordering, row/column equivalence, data/metadata fluidity; a trial-and-error interaction model spurs open data-management research. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID
h03a9681978c91d52
Venue
VLDB
Year
2020
Pagerank
0.00010899832
Overall Rank
1,369 | 90.80%
DOI
10.14778/3407790.3407807

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{petersohn_vldb20,
        title = {{Towards Scalable Dataframe Systems}},
        author = {Petersohn, Devin and Macke, Stephen and Xin, Doris and Ma, William and Lee, Doris and Mo, Xiangxi and Gonzalez, Joseph E. and Hellerstein, Joseph M. and Joseph, Anthony D. and Parameswaran, Aditya},
        journal = {PVLDB},
        series = {{VLDB} '20},
        volume = {13},
        number = {11},
        pages = {2033--2046},
        doi = {10.14778/3407790.3407807},
        url = {https://doi.org/10.14778/3407790.3407807},
        year = {2020}
}

Incoming Citations (Sorted by Pagerank)

Showing 22 of 22 citing papers.

Rank Citing Paper Year Venue Pagerank
1,199 Balsa: Learning a Query Optimizer Without Expert Demonstrations 2022 SIGMOD 0.00011563985
2,207 Magpie: Python at Speed and Scale using Cloud Backends 2021 CIDR 8.8487039e-05
2,460 Query Processing on Tensor Computation Runtimes 2022 VLDB 8.4348335e-05
3,098 Lux: Always-on Visualization Recommendations for Exploratory Dataframe Workflows 2022 VLDB 7.6517623e-05
3,180 A Critique of Modern SQL And A Proposal Towards A Simple and Expressive Query Language 2024 CIDR 7.5639428e-05
3,337 The Composable Data Management System Manifesto 2023 VLDB 7.4104865e-05
3,930 Flexible Rule-Based Decomposition and Metadata Independence in Modin: A Parallel Dataframe System 2022 VLDB 6.9185978e-05
4,815 PolyFrame: A Retargetable Query-based Approach to Scaling Dataframes 2021 VLDB 6.3992929e-05
5,640 DataPrep.EDA: Task-Centric Exploratory Data Analysis for Statistical Modeling in Python 2021 SIGMOD 6.0548401e-05
6,116 ConnectorX: Accelerating Data Loading From Databases to Dataframes 2022 VLDB 5.8815309e-05
6,662 UPLIFT: Parallelization Strategies for Feature Transformations in Machine Learning Workloads 2022 VLDB 5.7171651e-05
7,212 Capturing and Querying Fine-grained Provenance of Preprocessing Pipelines in Data Science 2021 VLDB 5.5840773e-05
7,269 Decentralized Actor Scheduling and Reference-based Storage in Xorbits: a Native Scalable Data Science Engine 2025 VLDB 5.5694935e-05
7,538 Automating and Optimizing Data-Centric What-If Analyses on Native Machine Learning Pipelines 2023 SIGMOD 5.4995874e-05
9,246 DQDF: Data-Quality-Aware Dataframes 2022 VLDB 5.2056825e-05
10,279 ElasticNotebook: Enabling Live Migration for Computational Notebooks 2024 VLDB 5.0455234e-05
10,305 Fast and Scalable Data Transfer Across Data Systems 2025 SIGMOD 5.0400722e-05
10,898 stratum: A System Infrastructure for Massive Agent-Centric ML Workloads 2026 VLDB 4.9793485e-05
11,258 Accio: Bolt-on Query Federation 2025 VLDB 4.9793485e-05
11,570 SplitDF: Splitting Dataframes for Memory-Efficient Data Analysis 2024 VLDB 4.9793485e-05
11,902 DPDS: Assisting Data Science with Data Provenance 2022 VLDB 4.9793485e-05
11,934 Leam: An Interactive System for In-situ Visual Text Analysis 2021 CIDR 4.9793485e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 24 of 24 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
9 Online Aggregation 1997 SIGMOD 0.00076195956
23 Spark SQL: Relational Data Processing in Spark 2015 SIGMOD 0.00055406774
71 DuckDB: an Embeddable Analytical Database 2019 SIGMOD 0.00037720227
95 Potter's Wheel: An Interactive Data Cleaning System 2001 VLDB 0.00034382643
129 Efficient and Extensible Algorithms for Multi Query Optimization 2000 SIGMOD 0.0003040756
389 QPipe: A Simultaneously Pipelined Relational Query Engine 2005 SIGMOD 0.00019269777
667 SharedDB: Killing One Thousand Queries With One Stone 2012 VLDB 0.00014978213
784 VerdictDB: Universalizing Approximate Query Processing 2018 SIGMOD 0.00014012614
1,030 Rate-Based Query Optimization for Streaming Information Sources 2002 SIGMOD 0.00012407749
1,108 Maximizing the Output Rate of Multi-Way Join Queries over Streaming Information Sources 2003 VLDB 0.00011991195
1,267 Querying XML Views of Relational Data 2001 VLDB 0.00011269152
1,568 HELIX: Holistic Optimization for Accelerating Iterative Machine Learning 2019 VLDB 0.0001021302
1,578 SchemaSQL - A Language for Interoperability in Relational Multi-database Systems 1996 VLDB 0.00010187094
1,661 Rapid Sampling for Visualizations with Ordering Guarantees 2015 VLDB 9.9535453e-05
1,696 PIVOT and UNPIVOT: Optimization and Execution Strategies in an RDBMS 2004 VLDB 9.8545415e-05
1,802 Hash joins and hash teams in Microsoft SQL Server 1998 VLDB 9.6081518e-05
1,871 Predictive Interaction for Data Transformation 2015 CIDR 9.464979e-05
1,916 The Analytical Bootstrap: a New Method for Fast Error Estimation in Approximate Query Processing 2014 SIGMOD 9.3837729e-05
2,000 Sample + Seek: Approximating Aggregates with Distribution Precision Guarantee 2016 SIGMOD 9.2112617e-05
4,921 CQL: A Query Language for Manipulating Object-oriented Databases 1989 VLDB 6.3524669e-05
5,021 Adaptive Sampling for Rapidly Matching Histograms 2018 VLDB 6.3106761e-05
6,048 Query Unnesting in Object-Oriented Databases 1998 SIGMOD 5.9048465e-05
6,256 DataSpread: Unifying Databases and Spreadsheets 2015 VLDB 5.8335602e-05
6,992 Skimmer: Rapid Scrolling of Relational Query Results 2012 SIGMOD 5.6267971e-05
Previous Page 1 / 1 Next

Semantically Similar Papers