CtxPipe: Context-aware Data Preparation Pipeline Construction for Machine Learning

Summary: CtxPipe automates context-aware data-prep pipeline construction for ML using pretrained embeddings to capture semantics and guide component choice. A deep RL framework searches the pipeline, delivering higher feature quality and faster models. (summarized by gpt-5-nano on Feb 09 2026)

Paper ID: 7062
Venue: SIGMOD
Year: 2024
Pagerank: 5.3483178e-05
Overall Rank: 8,914 | 38.85%
DOI: 10.1145/3698831

Incoming Non-self Citations Over Time

Authors

1. Haotian Gao (National University of Singapore)
2. Shaofeng Cai (National University of Singapore)
3. Tien Tuan Anh Dinh (Deakin University)
4. Zhiyong Huang (National University of Singapore)
5. Beng Chin Ooi (National University of Singapore)

BibTeX Citation

@inproceedings{gao_sigmod24,
        title = {{CtxPipe: Context-aware Data Preparation Pipeline Construction for Machine Learning}},
        author = {Gao, Haotian and Cai, Shaofeng and Dinh, Tien Tuan Anh and Huang, Zhiyong and Ooi, Beng Chin},
        series = {{SIGMOD} '24},
        booktitle = {Proceedings of the {ACM} {SIGMOD} International Conference on Management of Data},
        publisher = {Association for Computing Machinery},
        doi = {10.1145/3698831},
        url = {https://dl.acm.org/doi/10.1145/3698831},
        year = {2024}
}

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
10,457	FlowPilot: A Suggestion System for Designing Scientific Workflows	2026	SIGMOD	5.093636e-05

Outgoing Citations (Sorted by Pagerank)

Showing 24 of 24 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
397	TURL: Table Understanding through Representation Learning	2021	VLDB	0.00019278189
582	ActiveClean: Interactive Data Cleaning For Statistical Modeling	2016	VLDB	0.00016148948
729	Functional Dependency Discovery: An Experimental Evaluation of Seven Algorithms	2015	VLDB	0.00014539421
1,004	Democratizing Data Science through Interactive Curation of ML Pipelines	2019	SIGMOD	0.00012701932
1,323	Data Cleaning: Overview and Emerging Challenges	2016	SIGMOD	0.00011152602
1,351	Detecting Data Errors: Where are we and what needs to be done?	2016	VLDB	0.00011064851
1,741	Efficient Denial Constraint Discovery with Hydra	2018	VLDB	9.8755846e-05
1,756	SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle	2020	CIDR	9.8172465e-05
2,019	RPT: Relational Pre-trained Transformer Is Almost All You Need towards Democratizing Data Preparation	2021	VLDB	9.2983994e-05
2,147	Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions	2021	VLDB	9.0831495e-05
2,657	Production Machine Learning Pipelines: Empirical Analysis and Optimization Opportunities	2021	SIGMOD	8.2887895e-05
2,685	Approximate Denial Constraints	2020	VLDB	8.2594662e-05
2,759	Data X-Ray: A Diagnostic Tool for Data Errors	2015	SIGMOD	8.1577506e-05
3,387	Data Profiling – A Tutorial	2017	SIGMOD	7.4534663e-05
4,183	Scalable Discovery of Unique Column Combinations	2014	VLDB	6.8467773e-05
4,757	Pattern Functional Dependencies for Data Cleaning	2020	VLDB	6.5220005e-05
5,291	DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data	2023	SIGMOD	6.2801343e-05
6,114	Fundamentals of Order Dependencies	2012	VLDB	5.9713126e-05
7,068	DataPrism: Exposing Disconnect between Data and Systems	2022	SIGMOD	5.7119157e-05
7,232	Saga: A Scalable Framework for Optimizing Data Cleaning Pipelines for Machine Learning Applications	2023	SIGMOD	5.6659017e-05
7,397	Conformance Constraint Discovery: Measuring Trust in Data-Driven Systems	2021	SIGMOD	5.6256632e-05
7,746	WindTunnel: Towards Differentiable ML Pipelines Beyond a Single Model	2022	VLDB	5.5531058e-05
8,177	HAIPipe: Combining Human-generated and Machine-generated Pipelines for Data Preparation	2023	SIGMOD	5.4730821e-05
8,185	BugDoc: Algorithms to Debug Computational Processes	2020	SIGMOD	5.4707797e-05

Semantically Similar Papers

#	Overall Rank	Paper	Year	Venue
1	3,541	Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines	2022	SIGMOD
2	7,395	Automating and Optimizing Data-Centric What-If Analyses on Native Machine Learning Pipelines	2023	SIGMOD
3	6,250	Lightweight Inspection of Data Preprocessing in Native Machine Learning Pipelines	2021	CIDR
4	5,291	DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data	2023	SIGMOD
5	5,901	A Scalable AutoML Approach Based on Graph Neural Networks	2022	VLDB
6	11,746	Active Reinforcement Learning for Data Preparation: Learn2Clean with Human-In-The-Loop	2020	CIDR
7	13,303	Demonstrating CatDB: LLM-based Generation of Data-centric ML Pipelines	2025	SIGMOD
8	10,882	CatDB: Data-catalog-guided, LLM-based Generation of Data-centric ML Pipelines	2025	VLDB
9	4,875	Auto-Pipeline: Synthesizing Complex Data Pipelines By-Target Using Reinforcement Learning and Search	2021	VLDB
10	8,177	HAIPipe: Combining Human-generated and Machine-generated Pipelines for Data Preparation	2023	SIGMOD