When sweet and cute isn't enough anymore: Solving scalability issues in Python Pandas with Grizzly
Summary: Grizzly addresses Pandas' scalability limits by compiling Pandas DataFrame pipelines into SQL/SparkSQL and executing them in a DBMS to leverage optimized storage and query engines. Retains a Pandas-friendly API while dramatically reducing memory and CPU overhead. (summarized by gpt-5-mini on Feb 09 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Stefan Hagedorn (Technical University of Ilmenau)
BibTeX Citation
@inproceedings{hagedorn_cidr20,
address = {Amsterdam, Netherlands},
series = {{CIDR} '20},
title = {{When sweet and cute isn't enough anymore: Solving scalability issues in Python Pandas with Grizzly}},
booktitle = {Proceedings of the {Conference} on {Innovative} {Data} {Systems} {Research}},
author = {Hagedorn, Stefan},
year = {2020}
}Incoming Citations (Sorted by Pagerank)
Showing 2 of 2 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 4,292 | Putting Pandas in a Box | 2021 | CIDR | 6.7784325e-05 |
| 11,235 | SplitDF: Splitting Dataframes for Memory-Efficient Data Analysis | 2024 | VLDB | 5.093636e-05 |
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 1 of 1 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank | Cited Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 1,765 | Selecting Subexpressions to Materialize at Datacenter Scale | 2018 | VLDB | 9.8079546e-05 |
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 11,987 | The Best of Both Worlds: Big Data Programming with Both Productivity and Performance | 2017 | SIGMOD |
| 2 | 3,757 | Panda: Performance Debugging for Databases using LLM Agents | 2024 | CIDR |
| 3 | 11,329 | PD-Explain: A Unified Python-native Framework for Query Explanations Over DataFrames | 2024 | VLDB |
| 4 | 4,079 | Flexible Rule-Based Decomposition and Metadata Independence in Modin: A Parallel Dataframe System | 2022 | VLDB |
| 5 | 5,791 | Accelerating Python UDFs in Vectorized Query Execution | 2022 | CIDR |
| 6 | 6,349 | Grizzly: Efficient Stream Processing Through Adaptive Query Compilation | 2020 | SIGMOD |
| 7 | 4,728 | PolyFrame: A Retargetable Query-based Approach to Scaling Dataframes | 2021 | VLDB |
| 8 | 2,651 | Magpie: Python at Speed and Scale using Cloud Backends | 2021 | CIDR |
| 9 | 1,431 | Towards Scalable Dataframe Systems | 2020 | VLDB |
| 10 | 4,292 | Putting Pandas in a Box | 2021 | CIDR |