Skip to content
PipelineZ
Get Started

Blog

The words Retry, not rerun, above a row of pipeline nodes where a run broke at the last step and a yellow path picks up from the break without going back to the start.The words Retry, not rerun, above a row of pipeline nodes where a run broke at the last step and a yellow path picks up from the break without going back to the start.

October 11, 2026

Retry without starting over

A run that fails on its last write shouldn't cost another hour of extraction. pz retry re-runs only what didn't finish, copies the reads that already landed, and carries forward the writes that already committed.

The words Backfill in slices, above a backlog cut into equal slices, the current one glowing yellow and rising into a pipeline, with a flag marking where the backlog ends.The words Backfill in slices, above a backlog cut into equal slices, the current one glowing yellow and rising into a pipeline, with a flag marking where the backlog ends.

October 10, 2026

A year of history, one slice at a time

A backfill doesn't have to be one huge extract. max_window bounds every pz run to a fixed slice past the watermark, caughtUp says when to stop, and until: now keeps the same windows running on a schedule.

The words Change data capture, above a yellow change-log line dotted with events, one window of it bracketed and rising into a pipeline terminal.The words Change data capture, above a yellow change-log line dotted with events, one window of it bracketed and rising into a pipeline terminal.

October 9, 2026

Change data capture without a daemon

A cursor column can't see a deleted row. mode: cdc reads the source's own change log instead, one bounded drain per pz run, with no replication service left running in between.

The words Every connector, next to a glowing yellow pipeline network diagram.The words Every connector, next to a glowing yellow pipeline network diagram.

September 13, 2026

A tour of pz connectors

Fifteen builtin connectors, a tier of first-party ones outside the binary, and the native-versus-universal split that decides how fast a read or write actually runs.

The words Checks, not gates, next to a large translucent PipelineZ Z-mark watermark.The words Checks, not gates, next to a large translucent PipelineZ Z-mark watermark.

September 12, 2026

Checks, not gates

A check is a data-quality assertion that runs inside the same dependency graph as your pipeline, close enough to free to run. It observes, it doesn't block, unless you tell it to.

The word Inside above the accent-colored words 'a run', next to a yellow circuit path over a dot grid.The word Inside above the accent-colored words 'a run', next to a yellow circuit path over a dot grid.

September 11, 2026

What happens when you run pz

The seven phases between typing pz run and seeing a result: load, compile, validate, plan, dispatch, finalize, report, and what each one leaves on disk.

A yellow step-chart line climbing across a grid background, next to the words Incremental loads.A yellow step-chart line climbing across a grid background, next to the words Incremental loads.

September 10, 2026

Incremental loads without the ceremony

Full refresh reads everything, every time. Watermarks, sync modes, and merge writes let pz read only what changed, safely.

The PipelineZ wordmark over a grid-lined background with a glowing yellow pipeline diagram.The PipelineZ wordmark over a grid-lined background with a glowing yellow pipeline diagram.

September 9, 2026

Introducing PipelineZ

Why we built pz: a lightweight, SQL-based batch pipeline engine powered by DuckDB, for the pipelines that don't need a whole data platform.