DBScholar

Back to papers

Auto-Pipeline: Synthesizing Complex Data Pipelines By-Target Using Reinforcement Learning and Search

Summary: Auto-Pipeline synthesizes multi-step data-cleaning pipelines from input tables and a target schema, rather than examples. It exploits implicit constraints (FDs, keys) to guide reinforcement learning and search, recovering ~70% of real pipelines up to 10 steps. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID
ha59dbf405ea88f97
Venue
VLDB
Year
2021
Pagerank
6.8448428e-05
Overall Rank
4,005 | 73.16%
DOI
10.14778/3476249.3476303
PDF
Download (CC BY-NC-ND 4.0)

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{yang_vldb21,
        title = {{Auto-Pipeline: Synthesizing Complex Data Pipelines By-Target Using Reinforcement Learning and Search}},
        author = {Yang, Junwen and He, Yeye and Chaudhuri, Surajit},
        journal = {PVLDB},
        series = {{VLDB} '21},
        volume = {14},
        number = {11},
        pages = {2563--2575},
        doi = {10.14778/3476249.3476303},
        url = {https://doi.org/10.14778/3476249.3476303},
        year = {2021}
}

Incoming Citations (Sorted by Pagerank)

Showing 11 of 11 citing papers.

Rank Citing Paper Year Venue Pagerank
4,740 Auto-Tables: Synthesizing Multi-Step Transformations to Relationalize Tables without Using Examples 2023 VLDB 6.429338e-05
5,120 Explaining Dataset Changes for Semantic Data Versioning with Explain-Da-V 2023 VLDB 6.254003e-05
5,466 HAIPipe: Combining Human-generated and Machine-generated Pipelines for Data Preparation 2023 SIGMOD 6.1038307e-05
7,548 Auto-Prep: Holistic Prediction of Data Preparation Steps for Self-Service Business Intelligence 2025 VLDB 5.481283e-05
7,802 Auto-Formula: Recommend Formulas in Spreadsheets using Contrastive Learning for Table Representations 2024 SIGMOD 5.4336348e-05
10,397 FlowPipe: LLM-Enhanced Conditional Generative Flow Networks for Data Preparation Pipeline Construction 2027 SIGMOD 4.9605343e-05
10,496 BAT: Target-Instance-Free Data Preparation Synthesis via LLM-Driven Tree Search 2026 SIGMOD 4.9605343e-05
10,716 FlowPilot: A Suggestion System for Designing Scientific Workflows 2026 SIGMOD 4.9605343e-05
10,908 EcoTable: Cost-effective Table Integration in Data Lakes for Natural Language Queries 2026 VLDB 4.9605343e-05
10,939 DeepPrep: An LLM-Powered Agentic System for Autonomous Data Preparation 2026 VLDB 4.9605343e-05
11,681 LucidScript: Bottom-up Standardization for Data Preparation 2024 VLDB 4.9605343e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 12 of 12 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
520 Query by Output 2009 SIGMOD 0.00016910379
602 A Hybrid Approach to Functional Dependency Discovery 2016 SIGMOD 0.00015670535
727 Functional Dependency Discovery: An Experimental Evaluation of Seven Algorithms 2015 VLDB 0.00014426954
972 The Data Civilizer System 2017 CIDR 0.00012742827
1,094 BlinkFill: Semi-supervised Programming By Example for Syntactic String Transformations 2016 VLDB 0.00012039085
1,111 Foofah: Transforming Data By Example 2017 SIGMOD 0.00011946262
1,875 Predictive Interaction for Data Transformation 2015 CIDR 9.4433754e-05
2,608 Auto-Join: Joining Tables by Leveraging Transformations 2017 VLDB 8.209962e-05
2,645 Auto-Suggest: Learning-to-Recommend Data Preparation Steps Using Data Science Notebooks 2020 SIGMOD 8.1641056e-05
2,746 Transform-Data-by-Example (TDE): An Extensible Search Engine for Data Transformations 2018 VLDB 8.0458815e-05
4,444 Auto-Transform: Learning-to-Transform by Patterns 2020 VLDB 6.5812416e-05
7,170 Transform-Data-by-Example (TDE): Extensible Data Transformation in Excel 2018 SIGMOD 5.5784388e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 10,863 Replacing Multi-Step Assembly of Data Preparation Pipelines with One-Step LLM Pipeline Generation for Table QA 2026 VLDB
2 6,044 A Scalable AutoML Approach Based on Graph Neural Networks 2022 VLDB
3 4,444 Auto-Transform: Learning-to-Transform by Patterns 2020 VLDB
4 2,645 Auto-Suggest: Learning-to-Recommend Data Preparation Steps Using Data Science Notebooks 2020 SIGMOD
5 9,008 Auto-BI: Automatically Build BI-Models Leveraging Local Join Prediction and Global Schema Graph 2023 VLDB
6 6,173 Automatic Data Acquisition for Deep Learning 2021 VLDB
7 6,438 CtxPipe: Context-aware Data Preparation Pipeline Construction for Machine Learning 2024 SIGMOD
8 5,466 HAIPipe: Combining Human-generated and Machine-generated Pipelines for Data Preparation 2023 SIGMOD
9 7,548 Auto-Prep: Holistic Prediction of Data Preparation Steps for Self-Service Business Intelligence 2025 VLDB
10 4,740 Auto-Tables: Synthesizing Multi-Step Transformations to Relationalize Tables without Using Examples 2023 VLDB