Making Data Engineering Declarative

Summary: Introduces Delta Live Tables: declarative pipelines that model ELT as a DAG of incrementally maintained, full‑SQL materialized views (vs CTAS), enabling automated incrementalization and holistic, pipeline‑aware optimization. Adds CI/CD‑aware MV evolution, autoscaling/orchestration to reduce cost/latency, plus declarative correctness monitoring and policies for handling dirty source data. (summarized by gpt-5-mini on Feb 09 2026)

Paper ID: 503
Venue: CIDR
Year: 2023
Pagerank: 4.4094312e-05
Overall Rank: 9,016 | 37.28%
DOI: 10.1145/1234567890

Incoming Non-self Citations Over Time

Authors

Incoming Citations (Sorted by Pagerank)

Showing 1 of 1 citing papers.

Rank	Citing Paper	Year	Venue	Pagerank
9,689	LST-Bench: Benchmarking Log-Structured Tables in the Cloud	2024	SIGMOD	4.3043822e-05

Outgoing Citations (Sorted by Pagerank)

Showing 3 of 3 cited papers.

Citations counted here include only citations to other VLDB/SIGMOD/CIDR/PODS papers in this database.

Rank	Cited Paper	Year	Venue	Pagerank
66	Spark SQL: Relational Data Processing in Spark	2015	SIGMOD	0.00061639801
746	Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores	2020	VLDB	0.00017326979
1,377	Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics	2021	CIDR	0.00012296941

Semantically Similar Papers

Overall Rank	Paper	Year	Venue	Pagerank
1,482	Automating Large-Scale Data Quality Verification	2018	VLDB	0.00011725533
9,666	Breaking the Chains: On Declarative Data Analysis and Data Independence in the Big Data Era	2014	VLDB	4.3082114e-05
6,123	Data Ingestion for the Connected World	2017	CIDR	5.1991194e-05
199	Declarative Data Cleaning: Language, Model, and Algorithms	2001	VLDB	0.00035041015
10,788	Streaming View: An Efficient Data Processing Engine for Modern Real-time Data Warehouse of Alibaba Cloud	2025	VLDB	4.1945683e-05
10,320	ELT-Bench: An End-to-End Benchmark for Evaluating AI Agents on ELT Pipelines	2026	VLDB	4.1945683e-05
1,548	Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark	2018	SIGMOD	0.00011431383
746	Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores	2020	VLDB	0.00017326979
10,417	Streaming Democratized: Ease Across the Latency Spectrum with Delayed View Semantics and Snowflake Dynamic Tables	2025	SIGMOD	4.1945683e-05
9,973	End-to-End Declarative Data Analytics: Co-designing Engines, Interfaces, and Cloud Infrastructure	2026	CIDR	4.1945683e-05