HAIPipe: Combining Human-generated and Machine-generated Pipelines for Data Preparation

Summary: HAIPipe fuses HI-pipelines with AI-pipelines to form HAI-pipelines that outperform either. It uses an enumeration-sampling framework and RL-guided AI-pipeline search, with experiments on 1400+ real-world HI-pipelines showing gains. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: hf0c181f7f293ceb7
Venue: SIGMOD
Year: 2023
Pagerank: 5.9720806e-05
Overall Rank: 5,848 | 60.69%
DOI: 10.1145/3588945

Incoming Non-self Citations Over Time

Authors

1. Sibei Chen (Renmin University of China)
2. Nan Tang (Hong Kong University of Science and Technology; Qatar Computing Research Institute)
3. Ju Fan (Renmin University of China)
4. Xuemi Yan (Renmin University of China)
5. Chengliang Chai (Beijing Institute of Technology)
6. Guoliang Li (Tsinghua University)
7. Xiaoyong Du (Renmin University of China)

BibTeX Citation

@inproceedings{chen_sigmod23,
        title = {{HAIPipe: Combining Human-generated and Machine-generated Pipelines for Data Preparation}},
        author = {Chen, Sibei and Tang, Nan and Fan, Ju and Yan, Xuemi and Chai, Chengliang and Li, Guoliang and Du, Xiaoyong},
        series = {{SIGMOD} '23},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3588945},
        url = {https://dl.acm.org/doi/10.1145/3588945},
        year = {2023}
}

Incoming Citations (Sorted by Pagerank)

Showing 10 of 10 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
3,466	In-depth Analysis of Graph-based RAG in a Unified Framework	2025	VLDB	7.2783993e-05
4,114	Automatic Database Configuration Debugging using Retrieval-Augmented Language Models	2025	SIGMOD	6.7971182e-05
6,921	CtxPipe: Context-aware Data Preparation Pipeline Construction for Machine Learning	2024	SIGMOD	5.6432616e-05
7,270	AutoPrep: Natural Language Question-Aware Data Preparation with a Multi-Agent Framework	2025	VLDB	5.5694935e-05
7,812	Auto-Formula: Recommend Formulas in Spreadsheets using Contrastive Learning for Table Representations	2024	SIGMOD	5.4486106e-05
10,466	HAMMER: An Automatic RAG Tuning System via Hierarchical Memory-Guided Monte Carlo Tree Search	2026	SIGMOD	4.9793485e-05
10,791	BookRAG: A Hierarchical Structure-aware Index-based Approach for Retrieval-Augmented Generation on Complex Documents	2026	VLDB	4.9793485e-05
10,804	Document-to-Database: Extraction Meets Relational Semantics	2026	VLDB	4.9793485e-05
10,831	PrepBench: How Far Are We from Natural-Language-Driven Data Preparation?	2026	VLDB	4.9793485e-05
10,990	MAPPipe: A System for Bridging Efficiency and Quality in Data Preprocessing via Knowledge-Augmented Structural Pruning	2026	VLDB	4.9793485e-05

Outgoing Citations (Sorted by Pagerank)

Showing 7 of 7 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
975	Democratizing Data Science through Interactive Curation of ML Pipelines	2019	SIGMOD	0.00012750518
2,036	Ease.ml: Towards Multi-tenant Resource Sharing for Machine Learning Workloads	2018	VLDB	9.1520279e-05
2,048	Automatically Generating Data Exploration Sessions Using Deep Reinforcement Learning	2020	SIGMOD	9.1229916e-05
2,641	Auto-Suggest: Learning-to-Recommend Data Preparation Steps Using Data Science Notebooks	2020	SIGMOD	8.1787073e-05
4,235	Auto-Pipeline: Synthesizing Complex Data Pipelines By-Target Using Reinforcement Learning and Search	2021	VLDB	6.713221e-05
5,814	Domain Adaptation for Deep Entity Resolution	2022	SIGMOD	5.9852808e-05
8,476	DADER: Hands-Off Entity Resolution with Domain Adaptation	2022	VLDB	5.333639e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	3,601	Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines	2022	SIGMOD
2	5,495	HAIChart: Human and AI Paired Visualization System	2024	VLDB
3	10,869	DeepPrep: An LLM-Powered Agentic System for Autonomous Data Preparation	2026	VLDB
4	7,538	Automating and Optimizing Data-Centric What-If Analyses on Native Machine Learning Pipelines	2023	SIGMOD
5	4,524	DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data	2023	SIGMOD
6	10,990	MAPPipe: A System for Bridging Efficiency and Quality in Data Preprocessing via Knowledge-Augmented Structural Pruning	2026	VLDB
7	6,028	A Scalable AutoML Approach Based on Graph Neural Networks	2022	VLDB
8	12,049	Active Reinforcement Learning for Data Preparation: Learn2Clean with Human-In-The-Loop	2020	CIDR
9	6,921	CtxPipe: Context-aware Data Preparation Pipeline Construction for Machine Learning	2024	SIGMOD
10	4,235	Auto-Pipeline: Synthesizing Complex Data Pipelines By-Target Using Reinforcement Learning and Search	2021	VLDB