SQL and Python are the same kind of node
Not a Python escape hatch bolted onto a SQL tool. A .py model and a .sql model are interchangeable in one DAG — either can depend on the other, and the
planner does not care which you wrote.
published as interlaced; the import name and CLI are interlace
One graph
A Python model in the hot path
Ten models over 25 million rows, from the benchmark project. Click any node to see its
source — user_ltv.py sits between two SQL models, and nothing around it acknowledges
the difference.
elsewhere in the same graph
from interlace import model
@model(depends_on=["by_user"], strategy="merge", key=["user_id"])
def user_ltv(by_user):
for batch in by_user.reader():
score = pc.add(pc.multiply(batch.column("spend"), 0.1), batch.column("events"))
yield pa.RecordBatch.from_arrays(
[batch.column("user_id"), batch.column("spend"), pc.round(score, 2)],
names=["user_id", "spend", "ltv"],
)Python, in the hot path. Its upstream is SQL. It is a generator, so 100k rows stream through Arrow with bounded memory — and merge is the same keyed upsert a SQL model gets.
Materialisation
Two axes, not one
materialise decides where a result lands and who owns it. strategy decides how it is written. They compose — and every combination above comes
from one real project.
virtual · view · ephemeral
Interlace builds an immutable fingerprinted snapshot and serves it through an environment
view. That ownership is what makes rebuild-skip, sandboxes, atomic promotion, rollback and gc possible.
table · file
A destination Interlace does not own. It delivers into an external table or overwrites a file, evolves the target additively, and never drops it — so grants, indexes and RLS survive. Environment-gated to production by default.
A breaking change cannot apply to a terminal target: there is no old version to serve during the build and no atomic cutover. That constraint is why the two planes exist.
Strategies
Seven ways to land a result
Every strategy is the same contract — given a query and a target, emit the SQL statements
that reconcile them, atomically. Nine panels for seven strategies: scd and incremental each change behaviour enough with one extra key to be worth showing twice.
The same scenario throughout, so only the outcome differs.
Every diagram uses the same scenario: row 1 changed in source, row 2 is unchanged, row 3 exists only in the target, row 4 is new in source.
Rewrite the whole table. The target ends up an exact copy of the source.
Every existing row goes. Row 3 has no source row, so it does not come back.
Add the query's rows. Nothing is deleted and nothing is matched, so the target only grows.
No key, so ids 1 and 2 now appear twice — right for a log, wrong for anything you expect to be unique.
Upsert by key. Keys already in the target but absent from this run are left alone.
Row 3 survives, because merge never deletes. Row 2 is rewritten even though nothing changed.
Treat the query as the complete desired state, and apply only the difference.
Same end state as replace, reached incrementally — and a key that vanished upstream is a delete.
A keyed upsert that stores an _hash of the non-key columns and writes only what actually changed.
Row 2's hash matches, so nothing is written for it. Unlike full_merge, a vanished key is kept.
Never overwrite. A changed row has its version closed and a new one opened, so the old value stays queryable.
Nothing is destroyed. Row 2 is in neither difference, so re-running writes nothing.
The same shape, but the validity windows follow the data — they abut on when the change happened, not on when interlace saw it.
Row 1 closes at 09:15 and reopens at 09:15 — no gap, no overlap. Row 3 has no succeeding event, so it still closes at processing time.
Read only the rows inside the window, then rewrite it — or, with a key, upsert within it instead.
Row 9 is outside the window and never read. Row 3 is inside it and gone from source, so the rewrite drops it — add a key and it would survive instead.
Same window, same rows read — but upserted by key instead of the period being rewritten.
Identical inputs to the panel beside it, opposite outcome for row 3: only keys the window supplies are touched, so a row that stopped being produced survives.
Build
One run, every plane
A single command builds owned snapshots, delivers into external tables and writes files — each with its own strategy, in dependency order. This is the benchmark: 25M synthetic events fanned out through every strategy, start to finish in seconds.
$ interlace run
Build results
Model Output Strategy Engine Depends on Rows Time
───────────────────────────────────────────────────────────────────────────────────────
events virtual replace default — +25,000,000 3.85s
daily_revenue virtual incremental default events +29 0.28s
daily_feed table append default daily_revenue +29 0.30s
revenue_report file replace default daily_revenue +29 0.07s
by_day virtual replace default enriched +30 0.08s
by_device virtual replace default enriched +4 0.11s
by_product virtual replace default enriched +15,000 0.21s
by_user virtual replace default enriched +100,000 0.45s
product_catalog virtual full_merge default by_product +15,000 0.34s
top_products view replace default by_product — 0.36s
user_history virtual scd default by_user +100,000 0.09s
user_ltv virtual merge default by_user +100,000 0.15s
Checks: 2/2 passed
Ran 12 model(s) (12 task(s)); promoted 13 to 'prod'.Preview every change
interlace plan classifies each model as breaking, non-breaking or forward-only before anything is built.
Reuse instead of rebuild
Column-level impact analysis proves when a downstream output is unchanged, so its existing table is reused.
Checks gate promotion
An error-severity check failure blocks the apply before the environment view ever moves.
The Daemon
One process, not a stack
interlace serve runs the web UI, HTTP API, scheduler and stream ingestion together.
There is no separate orchestrator to deploy and no broker to operate.
Durable streams
POST an event and it is fsynced before the 200, deduplicated by idempotency key, and materialised with exactly-once landing — the watermark commits in the same warehouse transaction as the data.
Free sandboxes
An environment is a set of views over fingerprinted tables, so a dev environment reuses production’s for free. Promotion is an atomic view swap; rollback is the same move backwards.
Built-in scheduling
Cron and interval triggers over a durable run queue with leases, retries and cooperative cancellation. No Airflow, no broker, no second deployment.
Pin models to engines
DuckDB/DuckLake by default; Postgres and quack are also stable. Spark is beta; Snowflake, BigQuery, Redshift and MotherDuck are alpha. Cross-engine dependencies move as Arrow, or over a federated ATTACH.
Ready to simplify your data pipelines?
Get started with Interlace in minutes. Install, define your first model, and run.