Push to Databricks Unity Catalog

kiri catalog push ships Kirimana’s governed metadata out to Databricks Unity Catalog. The point is not to replace Unity — it is to keep the contract corpus as the source of truth and let everyone working in the Databricks catalog UI see exactly what Kirimana sees: classifications, PII flags, owners, lineage edges, review state. Your governance signal lives where your engineers already work, without anyone needing a separate Kirimana URL.

The portable part — contracts, catalog state, AI policy — stays platform-agnostic. The push layer is the only place Unity specifics live. Everything flows one way: git is the authority, and the catalog is a materialisation of it. Changes made inside Unity are drift; Kirimana surfaces them but does not treat them as authoritative.

What flows out

A push projects the governance a Kirimana asset carries into Unity’s model. For each asset and its columns, the push carries:

• Owner — the domain owner, onto the Unity object’s owner/ description fields.
• Classification — the data class (public, internal, confidential, restricted), as a column and table tag.
• PII flags — is_pii and the PII category, as tags.
• Attribute review state — whether the column is human-approved, so a consumer can tell reviewed metadata from freshly-discovered.
• Contract version — the version of the contract governing the asset, so the catalog annotation is pinned to a known contract.
• Lineage edges — the medallion graph (bronze → silver → gold), so Unity renders the dependency view.
• Bilingual descriptions — where a contract carries descriptions in more than one language, both project onto the column comment so the catalog reads correctly for every audience.

Column tags route through Unity’s tags API; description, owner, and properties go through the Tables API PATCH. The push translates a Kirimana asset into Unity’s catalog.schema.table namespace and emits one write per annotation.

Dry-run by default

The push is dry-run by default, and that is a safety property, not an inconvenience. A dry run emits NDJSON — one JSON object per line — describing exactly the API calls the live executor will issue, without touching the catalog. You wire the config, review the diff against the live catalog, and only then flip to live mode.

# 1. Import the project corpus into the local catalog DB first.
kiri catalog import --project examples/acme-studios

# 2. Dry-run — inspect the write plan (default).
kiri catalog push --target unity \
  --project examples/acme-studios \
  --output /tmp/unity-push.ndjson

# 3. Review the proposed writes.
jq . /tmp/unity-push.ndjson | less

# 4. Execute against the live catalog once you're satisfied.
kiri catalog push --target unity --no-dry-run

Each NDJSON record carries a kind (asset.upsert, column.upsert, lineage.upsert, tag.assign), the target_urn (Unity’s three-part name), the exact payload the API call will receive, and the source_urn — the Kirimana URN it came from — so any annotation in Unity traces back to the contract that produced it. Pipe to jq to spot anomalies before anything is written.

--no-dry-run executes the annotation writes against the live catalog via the Unity Catalog executor. Always inspect the dry-run NDJSON first.

Per-column annotation projection

Push works at column granularity, not just table granularity. Each column’s classification, PII category, review state, and description project onto that column’s Unity tags and comment individually. This is what makes the catalog view genuinely useful downstream: a consumer browsing a silver table in Unity sees, per column, its data class, its PII category, and whether a human has approved it — the same signal a Kirimana user sees, in the Databricks UI.

Scoping a push

You rarely need to push the whole estate at once. Filter the scope:

• --type (repeatable) — restrict to bronze, silver, gold, source, view. Default: every type.
• --urn (repeatable) — restrict to specific assets, useful while testing.
• --no-lineage — skip lineage edges when you only want the classification/PII annotations.

kiri catalog push --target unity \
  --project examples/acme-studios \
  --type silver --type gold \
  --no-lineage

One target per run — pushing to multiple catalogs from one command is not supported. Split the call; it keeps the dry-run output analysable per target. Additional catalog sink targets are available with the same dry-run discipline.

Sync cadence

The push reads from the local catalog DB (.kiri/catalog.sqlite), not directly from the contracts, so the local catalog must be current before you push. The rule of thumb: kiri catalog import after every kiri apply, so the local catalog tracks the contracts, then push on the cadence that suits your estate — per release, nightly, or on demand. A bad sink profile fails fast as a config error before the first asset is read, so a mis-configured push never half-writes.

To confirm the catalog still matches after a push — and to detect drift introduced inside Unity — use kiri catalog verify, which diffs live catalog annotations against the local catalog and reports missing, stale, or extra findings. Pair a scheduled push with a scheduled verify and the two together keep Unity honest against the contract corpus.

To go the other direction — reading schema drift, observed lineage, and usage back out of Unity — see Pull from Databricks Unity Catalog.