DBScholar

Back to papers

Auto-Pipeline: Synthesizing Complex Data Pipelines By-Target Using Reinforcement Learning and Search

Summary: Auto-Pipeline synthesizes multi-step data-cleaning pipelines from input tables and a target schema, rather than examples. It exploits implicit constraints (FDs, keys) to guide reinforcement learning and search, recovering ~70% of real pipelines up to 10 steps. (summarized by gpt-5.6-luna on Jul 24 2026)

Paper ID
ha59dbf405ea88f97
Venue
VLDB
Year
2021
Pagerank
6.713221e-05
Overall Rank
4,235 | 71.53%
DOI
10.14778/3476249.3476303

Incoming Non-self Citations Over Time

Authors

BibTeX Citation

@article{yang_vldb21,
        title = {{Auto-Pipeline: Synthesizing Complex Data Pipelines By-Target Using Reinforcement Learning and Search}},
        author = {Yang, Junwen and He, Yeye and Chaudhuri, Surajit},
        journal = {PVLDB},
        series = {{VLDB} '21},
        volume = {14},
        number = {11},
        pages = {2563--2575},
        doi = {10.14778/3476249.3476303},
        url = {https://doi.org/10.14778/3476249.3476303},
        year = {2021}
}

Incoming Citations (Sorted by Pagerank)

Showing 10 of 10 citing papers.

Rank Citing Paper Year Venue Pagerank
5,097 Explaining Dataset Changes for Semantic Data Versioning with Explain-Da-V 2023 VLDB 6.2742361e-05
5,167 Auto-Tables: Synthesizing Multi-Step Transformations to Relationalize Tables without Using Examples 2023 VLDB 6.2462501e-05
5,848 HAIPipe: Combining Human-generated and Machine-generated Pipelines for Data Preparation 2023 SIGMOD 5.9720806e-05
7,526 Auto-Prep: Holistic Prediction of Data Preparation Steps for Self-Service Business Intelligence 2025 VLDB 5.5020723e-05
7,812 Auto-Formula: Recommend Formulas in Spreadsheets using Contrastive Learning for Table Representations 2024 SIGMOD 5.4486106e-05
10,418 BAT: Target-Instance-Free Data Preparation Synthesis via LLM-Driven Tree Search 2026 SIGMOD 4.9793485e-05
10,644 FlowPilot: A Suggestion System for Designing Scientific Workflows 2026 SIGMOD 4.9793485e-05
10,837 EcoTable: Cost-effective Table Integration in Data Lakes for Natural Language Queries 2026 VLDB 4.9793485e-05
10,869 DeepPrep: An LLM-Powered Agentic System for Autonomous Data Preparation 2026 VLDB 4.9793485e-05
11,628 LucidScript: Bottom-up Standardization for Data Preparation 2024 VLDB 4.9793485e-05
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 12 of 12 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank Cited Paper Year Venue Pagerank
518 Query by Output 2009 SIGMOD 0.00016944862
603 A Hybrid Approach to Functional Dependency Discovery 2016 SIGMOD 0.00015699254
727 Functional Dependency Discovery: An Experimental Evaluation of Seven Algorithms 2015 VLDB 0.00014443394
972 The Data Civilizer System 2017 CIDR 0.00012763234
1,094 BlinkFill: Semi-supervised Programming By Example for Syntactic String Transformations 2016 VLDB 0.0001205355
1,115 Foofah: Transforming Data By Example 2017 SIGMOD 0.00011964111
1,871 Predictive Interaction for Data Transformation 2015 CIDR 9.464979e-05
2,612 Auto-Join: Joining Tables by Leveraging Transformations 2017 VLDB 8.2265197e-05
2,641 Auto-Suggest: Learning-to-Recommend Data Preparation Steps Using Data Science Notebooks 2020 SIGMOD 8.1787073e-05
2,745 Transform-Data-by-Example (TDE): An Extensible Search Engine for Data Transformations 2018 VLDB 8.0668395e-05
4,424 Auto-Transform: Learning-to-Transform by Patterns 2020 VLDB 6.6048065e-05
7,148 Transform-Data-by-Example (TDE): Extensible Data Transformation in Excel 2018 SIGMOD 5.5989952e-05
Previous Page 1 / 1 Next

Semantically Similar Papers

# Overall Rank Paper Year Venue
1 10,792 Replacing Multi-Step Assembly of Data Preparation Pipelines with One-Step LLM Pipeline Generation for Table QA 2026 VLDB
2 6,028 A Scalable AutoML Approach Based on Graph Neural Networks 2022 VLDB
3 4,424 Auto-Transform: Learning-to-Transform by Patterns 2020 VLDB
4 2,641 Auto-Suggest: Learning-to-Recommend Data Preparation Steps Using Data Science Notebooks 2020 SIGMOD
5 8,979 Auto-BI: Automatically Build BI-Models Leveraging Local Join Prediction and Global Schema Graph 2023 VLDB
6 6,156 Automatic Data Acquisition for Deep Learning 2021 VLDB
7 6,921 CtxPipe: Context-aware Data Preparation Pipeline Construction for Machine Learning 2024 SIGMOD
8 5,848 HAIPipe: Combining Human-generated and Machine-generated Pipelines for Data Preparation 2023 SIGMOD
9 7,526 Auto-Prep: Holistic Prediction of Data Preparation Steps for Self-Service Business Intelligence 2025 VLDB
10 5,167 Auto-Tables: Synthesizing Multi-Step Transformations to Relationalize Tables without Using Examples 2023 VLDB