MLBench: Benchmarking Machine Learning Services Against Human Experts
Summary: MLBench repurposes Kaggle competitions into a benchmark containing raw and winning-team features, establishing a best-human-effort baseline. It systematically quantifies ML services’ functionality trade-offs and weaknesses via relative ranking and accuracy against that baseline. (summarized by gpt-5.6-luna on Jul 24 2026)
Incoming Non-self Citations Over Time
Authors
- 1. Yu Liu (ETH Zurich)
- 2. Hantian Zhang (ETH Zurich)
- 3. Luyuan Zeng (ETH Zurich)
- 4. Wentao Wu (Microsoft)
- 5. Ce Zhang (ETH Zurich)
BibTeX Citation
@article{liu_vldb18,
title = {{MLBench: Benchmarking Machine Learning Services Against Human Experts}},
author = {Liu, Yu and Zhang, Hantian and Zeng, Luyuan and Wu, Wentao and Zhang, Ce},
journal = {PVLDB},
series = {{VLDB} '18},
volume = {11},
number = {10},
pages = {1220--1232},
doi = {10.14778/3231751.3231770},
url = {https://doi.org/10.14778/3231751.3231770},
year = {2018}
}Incoming Citations (Sorted by Pagerank)
Showing 3 of 3 citing papers.
| Rank | Citing Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 4,798 | Accelerating Generalized Linear Models with MLWeaving: A One-Size-Fits-All System for Any-Precision Learning | 2019 | VLDB | 6.5024772e-05 |
| 5,040 | TPCx-AI - An Industry Standard Benchmark for Artificial Intelligence and Machine Learning Systems | 2023 | VLDB | 6.390395e-05 |
| 11,871 | doppioDB 2.0: Hardware Techniques for Improved Integration of Machine Learning into Databases | 2019 | VLDB | 5.093636e-05 |
Previous
Page 1 / 1
Next
Outgoing Citations (Sorted by Pagerank)
Showing 1 of 1 cited papers.
Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.
| Rank | Cited Paper | Year | Venue | Pagerank |
|---|---|---|---|---|
| 2,029 | Ease.ml: Towards Multi-tenant Resource Sharing for Machine Learning Workloads | 2018 | VLDB | 9.2843642e-05 |
Previous
Page 1 / 1
Next
Semantically Similar Papers
| # | Overall Rank | Paper | Year | Venue |
|---|---|---|---|---|
| 1 | 532 | MLbase: A Distributed Machine-learning System | 2013 | CIDR |
| 2 | 4,436 | Serving and Optimizing Machine Learning Workflows on Heterogeneous Infrastructures | 2023 | VLDB |
| 3 | 9,777 | PBench: Workload Synthesizer with Real Statistics for Cloud Analytics Benchmarking | 2025 | VLDB |
| 4 | 11,694 | CBench: Towards Better Evaluation of Question Answering Over Knowledge Graphs | 2021 | VLDB |
| 5 | 2,029 | Ease.ml: Towards Multi-tenant Resource Sharing for Machine Learning Workloads | 2018 | VLDB |
| 6 | 7,114 | Serverless Data Science - Are We There Yet? A Case Study of Model Serving | 2022 | SIGMOD |
| 7 | 9,433 | FEBench: A Benchmark for Real-Time Relational Data Feature Extraction | 2023 | VLDB |
| 8 | 5,040 | TPCx-AI - An Industry Standard Benchmark for Artificial Intelligence and Machine Learning Systems | 2023 | VLDB |
| 9 | 5,051 | Towards Benchmarking Feature Type Inference for AutoML Platforms | 2021 | SIGMOD |
| 10 | 3,543 | A Comparison of Platforms for Implementing and Running Very Large Scale Machine Learning Algorithms | 2014 | SIGMOD |