How-to guides

Each guide solves one task and shows how to verify the result. If you are new to pz, read the Quickstart first; the guides assume a project already exists.

Ingest

Extract from an HTTP API: paginated reads, cursors, and writing back to an endpoint.
Capture changes with CDC: read change feeds from SQL Server and PostgreSQL.
Backfill in slices: load history in bounded windows without losing your place.

Reliability

Run checks and retry: pz test, pz retry, and what a retry reuses.
Tune retries: attempts, backoff, and jitter per connection or per call.
Throttle a source: rate limits, concurrency caps, and the circuit breaker.
Guard against schema changes: validate-time contracts and sink schema policy.
Detect source drift at run time: on_source_drift and pz schema accept.
Debug a failed run: read the event stream, run results, and staged tables.

Production

Secure connection config: secrets through environment variables and managed identity.
Move state off the local disk: SQL Server or HTTP state backends for containers.
Run in CI: a GitHub Actions job that validates and runs a project.
Run on a schedule on Windows: Task Scheduler and the wrapper script.
Observe runs with Azure Monitor: OpenTelemetry metrics and traces.
Inspect and validate: pz ls, pz plan, and pz validate --connect.

AI agents

Use pz with an AI agent: the MCP server and what an agent can do with it.

Extend

Author a connector: build, test, and package a connector.

Connectors: per-connector options when a guide points at one.
CLI reference: the flags the guides use.