DBScholar

Back to papers

Magnus: A Holistic Approach to Data Management for Large-Scale Machine Learning Workloads

Summary: Magnus is a holistic data-management layer on Apache Iceberg tailored to large-scale ML workloads (wide tables, multimodal) combining resource-efficient storage formats with built-in vector/inverted indexes to speed retrieval. It adds scalable Git-like metadata branching, lightweight merge-on-read upsert, and native LRM/LMM training support; deployed at ByteDance with substantial real-world gains. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID
14287
Venue
VLDB
Year
2025
Pagerank
5.093636e-05
Overall Rank
11,006 | 24.49%
DOI
10.14778/3750601.3750620

Incoming Non-self Citations Over Time

No non-self incoming citations found for this paper in this database.

Authors

BibTeX Citation

@article{song_vldb25,
        title = {{Magnus: A Holistic Approach to Data Management for Large-Scale Machine Learning Workloads}},
        author = {Song, Jun and Ding, Jingyi and Kandy, Irshad and Lin, Yanghao and Wei, Zhongjia and Zhou, Zilong and Peng, Zhiwei and Shan, Jixi and Mao, Hongyue and Huang, Xiuqi and Song, Xun and Chen, Cheng and Li, Yanjia and Yang, Tianhao and Jia, Wei and Dong, Xiaohong and Lei, Kang and Shi, Rui and Zhao, Pengwei and Chen, Wei},
        journal = {PVLDB},
        series = {{VLDB} '25},
        volume = {18},
        number = {12},
        pages = {4964--4977},
        doi = {10.14778/3750601.3750620},
        url = {https://doi.org/10.14778/3750601.3750620},
        year = {2025}
}

Incoming Citations (Sorted by Pagerank)

Showing 0 of 0 citing papers.

Rank Citing Paper Year Venue Pagerank
Previous Page 1 / 1 Next

Outgoing Citations (Sorted by Pagerank)

Showing 14 of 14 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Previous Page 1 / 1 Next

Semantically Similar Papers