Kirimana for Databricks
Governance for Databricks built on contracts. Bronze → Silver → Gold, AI-policy enforcement, Unity Catalog pass-through, native Workflows orchestration, MCP for Databricks AI Assistants. Currently in Private Preview — invite only.
For data platforms running on Databricks Lakehouse with Unity Catalog. The most mature edition in the project, currently in Private Preview with active design partners.
Built for enterprise scale. Hub-and-spoke governance across dozens of domains, OIDC RBAC pinned to your IdP, multi-environment CI/CD, federated contract library, DORA + EU AI Act + GDPR generators. Light enough for any team. A small data team gets the same architecture; the wizard takes you from zero to a contracted bronze layer in an afternoon.
What’s included
- Databricks platform adapter — Delta Lake bronze + silver + gold generation, OAuth M2M auth, typed parameter binding, retry on transient failures
- Unity Catalog pass-through — owner, classification, lineage, attribute review-state pushed bidirectionally so UC stays the metadata surface for users while Kirimana is the source of contract truth
- Native Databricks Workflows orchestration — DAGs compiled from contracts to native Workflows JSON; no third runtime
- Databricks Vault adapter —
${vault:...}resolves to Databricks Secret Scopes - MCP server — Databricks AI Assistants read your contracts, classifications, lineage, AI policy, release status through the same MCP server Claude.ai / Cursor / Continue / Cline use
- AI policy gate — every AI call (drafting, lineage proposals, Kiri-assist, Databricks AI Assistants) classification-checked before reaching Anthropic / Azure OpenAI / Bedrock; restricted data never leaves the workspace
- Helm chart for the AKS-host — runs the Kirimana control plane in your own AKS cluster, dispatches to the Databricks workspace
dca databricks setupwizard — interactive provision of service principal, secret scopes, workspace permissions- Incident dispatch — apply failures, SLA breaches, schema drift dispatched to Jira / ServiceNow / Zendesk with trace links
What Kirimana adds that Databricks alone doesn’t
Databricks ships strong primitives — Unity Catalog, Workflows, AI Assistants. Kirimana sits above them and adds the contract layer that operationalises governance.
Per-contract AI policy enforcement
Databricks AI Assistants honor workspace permissions. They don’t gate by per-contract data classification. Kirimana refuses any LLM call against a contract whose classification disallows it — including calls from Databricks AI Assistants themselves, via the MCP server. The gate is in the metadata layer, not the workspace ACL.
Contract state machine + PR-time approval workflow
Unity Catalog has tags + lineage. It does not have a draft → reviewed → approved → deprecated state machine, nor PR-time linting that fails the build when a contract violates governance. Kirimana does both — six star-schema lint rules at gold, classification-presence rules everywhere, two-approver gate for redaction events.
Goal-to-data lineage
Unity Catalog tracks table-level lineage. Kirimana tracks ReportingGoal → Contract → Table — when the CFO asks “where does Q3 revenue come from?”, the answer is one query, with classifications attached at every hop.
Cross-tenant federated contract library
Unity Catalog is per-account. The Kirimana Library is federated — contracts you publish are usable by other organisations, and packs they publish are installable in your account. Patterns travel.
Compliance generators that ship in the box
DORA, EU AI Act, GDPR Art. 17 redaction reports generate from contract metadata + audit log. Databricks Compliance Manager (and its Microsoft cousins) require you to compose this story yourself; Kirimana ships the report templates.
Multi-platform contract portability
If a domain later moves to Fabric or Trino, the same contract runs. Unity Catalog cannot follow it; Kirimana does. The cost of changing your mind drops by an order of magnitude.
Pass-through to Unity Catalog
Kirimana is not a catalog replacement. The Databricks edition treats Unity Catalog as the metadata surface for users; Kirimana is the source of contract truth feeding it.
| Direction | What flows |
|---|---|
| Push to UC | Owner, classification, attribute review-state, contract version, lineage edges, AI-policy summary |
| Pull from UC | Schema drift detection, observed lineage, downstream usage signals |
| Sync cadence | Every apply + nightly reconciliation; manual dca catalog sync always available |
Unity Catalog stays the place your analytics engineers and BI team browse. Kirimana stays the place your contracts live, your AI policy gates, and your audit log records.
Integrations available out of the box
- AI providers: Anthropic Claude, Azure OpenAI, AWS Bedrock, Ollama (air-gapped)
- AI assistants: Databricks AI Assistants (via MCP), Claude.ai, Cursor, Continue.dev, Cline
- Catalogs: Unity Catalog (primary), Snowflake Horizon push, Microsoft Purview push (cross-cloud tenants)
- Ingest: Airbyte (default), Kafka, Debezium CDC, dlt, REST, database direct, landing zone
- Vault: Databricks Secret Scopes, Azure Key Vault, AWS Secrets Manager
- ITSM: Jira (REST v3), ServiceNow (Table API), Zendesk (REST v2)
- Comms: Slack governance bot, Microsoft Teams
- Auth: OIDC — Entra ID, GitHub, Okta, Auth0
- BI: dbt Semantic Layer / MetricFlow / Cube exports; Power BI / Tableau / Qlik connection guides
How to deploy
| Pattern | When |
|---|---|
| AKS-host + Databricks workspace | Recommended. Helm chart deploys control plane to AKS; dispatches Workflows to Databricks. |
| Self-host on existing Kubernetes | If you already run K8s elsewhere; chart works on any compliant K8s 1.28+. |
Pricing posture
- OSS (free) — full adapter, full Helm chart, full CLI. Apache-2.0.
- Professional Services — installation + first-domain bring-up, contract migration from existing pipelines, training. Day rates.
- Enterprise Support — SLA-backed support, named on-call rotation, audit assistance for DORA / GDPR. From $20k/yr.