Kirimana for Databricks

For data platforms running on Databricks Lakehouse with Unity Catalog and Workflows. Kirimana sits above them as the contract layer your data platform’s been missing — open source, ecosystem-friendly, anchored in capabilities implemented today. The most mature product path in the project, currently in Private Preview with active design partners on Azure Databricks. An AWS path is on the roadmap from the same codebase.

Files an AI agent can read. Every contract is YAML on disk in git; every Kiri-driven Databricks action emits a single audit line with a correlation id. Drop Claude Code or Cursor into your repo and the agent reads the files and understands your Databricks platform without an API integration.

Built for enterprise scale. Hub-and-spoke governance across dozens of domains, OIDC RBAC pinned to your IdP, multi-environment CI/CD, federation across three transports with fail-closed health. Light enough for any team. A small data team gets the same architecture; the wizard takes you from zero to a contracted bronze layer in an afternoon.

What Kirimana adds on Databricks

These are the value-adds that make Databricks customers more Databricks-engaged: more Unity Catalog usage, more Workflows runs, all governed, audit-clean, and contract-driven — anchored in what’s implemented today.

1. Contract-driven Bronze / Silver / Gold

Every Delta table has an ODCS v3 contract upstream. UC stays the catalog of record; Kirimana feeds it owner, classification, lineage, and attribute review-state. Contracts move with the table when domains migrate or merge.

2. Mandatory one-line Databricks audit

Every Kiri → Databricks action — CREATE, INSERT, SHOW, job submit, classification override — emits a single audit line. CLI calls and MCP calls are audited identically with the same correlation id. tail -f databricks-audit.log is the entire demo. The regulator can read it without us being in the room.

3. Workflows compilation from ODCS

The contract graph compiles to native Databricks Workflows JSON. No third runtime, no parallel orchestrator. Kirimana drives the Workflows your platform team already operates.

4. Fail-closed authoring guardrails

Inferred PII (names, emails, free-text identifiers, classified attributes) cannot be authored as public without an audited override and a written reason. Mandatory ownership: every contract names a real human; the CLI rejects what doesn’t. Default is refuse, not warn.

5. PR-time governance gates

kiri contract lint, kiri contract validate, and kiri contract diff run in CI before merge. Classification present, owner valid, lineage resolves, no drift between docs and code. UC stays clean; governance moves left, into the developer’s flow.

6. Governed AI access — gateway + query gate

Every AI call runs through one gateway: classification-gated before it leaves (a call that would send restricted data to the model is refused, and the refusal is audited with its reason) and logged — model, caller, classification, tokens, cost, contract, trace id, the prompt hashed rather than stored. Before a natural-language question runs, the query gate resolves each table to its contract and each column to its classification and returns allowed / gated / forbidden — so Databricks Genie (via the evaluate_query MCP tool) knows whether a question is safe before it touches your gold. And AI is barred from the generation pipeline itself by a CI import check: AI proposes, deterministic code disposes.

kiri compliance report generates DORA, EU AI Act, and GDPR evidence over a defined control set (JSON + Markdown), and kiri audit redact does redact-not-delete for GDPR Art. 17 with a two-approver gate. The outputs are structured evidence artefacts, not legal certification — gaps report as inconclusive, and human attestation and review by qualified personnel remain part of every regulatory framework we cover.

7. Audit as an explicit tier, not a silent dial

How much evidence Kirimana records is a choice you make in one line of configuration, and it is a conformance contract — every tier names exactly which stores and fields it produces. Pick native to lean on the Databricks audit log alone, thin for a durable apply-log correlated to that audit log on a shared trace id, or full for the compliance tier that adds warehouse-side column lineage and access-binding evidence. A run configured below the tier a compliance obligation requires fails loudly rather than degrading in silence.

8. Zero-footprint mode — an accelerator that emits native artefacts

Some teams want Kirimana as an accelerator that leaves nothing behind in the Lakehouse. The databricks-native profile does exactly that: Kirimana writes no control-plane metadata into your workspace on any path, so there is not a single Kirimana-owned table in Unity Catalog — the durable audit trail relocates to Kirimana’s own store, so you keep full auditability regardless. Point an apply at an output directory and it also writes native artefacts straight into your repository: a Databricks Workflows definition, and an Auto Loader (cloudFiles) configuration per source. Files you own outright. Because Kirimana holds no state in the warehouse under this profile, you can stop using it at any time and keep running on pure Databricks.

What’s included

• Databricks platform adapter: Delta Lake bronze + silver + gold generation, OAuth M2M auth, typed parameter binding, retry on transient failures
• Unity Catalog pass-through: owner, classification, lineage, attribute review-state pushed bidirectionally so UC stays the metadata surface for users while Kirimana is the source of contract truth
• Mandatory one-line Databricks audit on every Kiri action, CLI and MCP audited identically with a shared correlation id
• Native Databricks Workflows orchestration: DAGs compiled from contracts to native Workflows JSON; no third runtime
• Guided source-schema introspection: sample a live Databricks source, classify columns, propose populated bronze contracts — editable scaffolding, not auto-applied
• Databricks Vault adapter: ${vault:...} resolves to Databricks Secret Scopes
• MCP server: Databricks AI Assistants, Claude.ai, Cursor, Continue, Cline read your contracts, classifications, lineage, and release status through the same protocol
• Federation across three transports (in-process, HTTP+ETag, fs-static) with health() returning OK / STALE / UNAVAILABLE; federated reads fail closed when a source isn’t reachable
• Migration as a verb set: AI-assisted business-vault → ODCS translation, parity harness with row-hash check, shadow → diff → promote → rollback cutover, KEEP / DROP / MERGE / REDESIGN review actions, quarantine reprocess as a typed state machine
• Helm chart for the AKS-host: runs the Kirimana control plane in your own AKS cluster, dispatches to the Databricks workspace
• kiri databricks setup wizard: interactive provision of service principal, secret scopes, workspace permissions

Files an AI agent can read

This is the difference between Kirimana and other governance tools.

Tool	Where its metadata lives	What an AI agent has to do
Atlan	Proprietary catalog	Authenticate to the API, paginate, parse JSON, guess relationships
Collibra	Proprietary catalog	Same
Unity Catalog	Databricks workspace	API + token, query SQL warehouse, workspace-scoped
Microsoft Purview	Azure Purview tenant	API + token, tenant-scoped
Kirimana	YAML in git, JSONL audit log, graph lineage	Read the files

Drop Claude Code into your kirimana repo. It immediately knows: who owns what, what’s PII, what changed last week, what the SLA is, what the lineage is. No special integration. Just files an AI can read.

This becomes a force multiplier when your AI agents are doing real work — drafting contracts, debugging applies, suggesting silver models — because the agents have full context without you wiring it.

Pass-through to Unity Catalog (we feed UC, we don’t replace it)

Kirimana is not a catalog replacement. The Databricks product path treats Unity Catalog as the metadata surface for users; Kirimana is the source of contract truth feeding it.

Direction	What flows
Push to UC	Owner, classification, attribute review-state, contract version, lineage edges
Pull from UC	Schema drift detection, observed lineage, downstream usage signals
Sync cadence	Every apply + nightly reconciliation; manual kiri catalog sync always available

Unity Catalog stays the place your analytics engineers and BI team browse. Kirimana stays the place your contracts live and your audit log records.

Integrations available out of the box

• AI assistants: Databricks AI Assistants, Claude.ai, Claude Code, Cursor, Continue.dev, Cline (via MCP)
• Catalogs: Unity Catalog (primary), Snowflake Horizon push, Microsoft Purview push (cross-cloud tenants)
• Ingest: Databricks-native ingestion patterns, Databricks DLT, REST, database direct, and landing-zone ingestion
• Vault: Databricks Secret Scopes, Azure Key Vault, AWS Secrets Manager
• Auth: generic OIDC, Entra ID, GitHub (Okta / Auth0 / Google connect via the generic OIDC provider)
• BI: dbt Semantic Layer / MetricFlow / Cube exports; Power BI / Tableau / Qlik connection guides

How to deploy

Pattern	Cloud	When
AKS-host + Databricks workspace	Azure	Recommended day-1. Helm chart deploys control plane to AKS; dispatches Workflows to Databricks.
EKS-host + Databricks workspace	AWS	On the roadmap. Same product, AWS-native Terraform.
Self-host on existing Kubernetes	Any	If you already run K8s elsewhere; chart works on any compliant K8s 1.28+.

Pricing posture

• Source-available under FSL-1.1-ALv2 (each release converts to Apache-2.0 two years after it ships). Full adapter, full Helm chart, full CLI. There is no “community edition” feature-gating in the codebase today; if that ever changes, we’d say so before doing it. See /pricing.

A note on the Databricks ecosystem

Kirimana exists to make Databricks customers more Databricks- engaged. Every value-add above increases your customers’ use of Unity Catalog and Workflows — operationalised through contracts, mandatory ownership, fail-closed authoring, and mandatory audit that make them safe for regulated industries.

Source-available (FSL-1.1-ALv2, converts to Apache-2.0 after two years) and ecosystem-friendly. ODCS aligns with the open standards Databricks already champions (Delta Lake, Unity Catalog OSS, MLflow, Apache Spark). Roadmap target: Databricks Marketplace listing + Databricks App deployment.

Linked resources

• Azure-native production deployment guide
• Public-sector adoption scenario on Databricks
• kiri databricks setup reference
• Standards + integrations
• Compare to Atlan / Collibra / dbt Cloud / UC