Schema Registry (Planned)
Planned — Q4 2026
Schema Registry is on the committed roadmap for Q4 2026 as a Kafka add-on. Requires the Streaming service (Managed Kafka) to be operational.
Purpose: For platform engineers and data engineers, explains the planned Schema Registry service — scope, compatibility modes, integration with Kafka, and deployment model.
Overview
Schema Registry provides schema versioning and compatibility enforcement for Kafka topics. It prevents breaking schema changes from propagating through streaming pipelines by validating schemas at produce time against configured compatibility rules.
Deployed as an add-on to the existing Kafka infrastructure in the data-kafka namespace.
Planned Scope
| Capability | Description |
|---|---|
| Schema registration | Register and version schemas for Kafka topics |
| Compatibility enforcement | Validate new schema versions against compatibility rules |
| Format support | Avro, Protobuf, JSON Schema |
| Subject strategies | TopicName, RecordName, TopicRecordName |
| REST API | Standard Schema Registry API for client integration |
| GitOps managed | Registry deployment and configuration via FluxCD |
| Monitoring | Prometheus metrics for schema registrations, compatibility checks |
| TLS | cert-manager issued certificates |
| Authentication | Integrated with Keycloak (OAuth2/OIDC) |
Compatibility Modes
| Mode | Rule | Use Case |
|---|---|---|
| BACKWARD | New schema can read data written by previous schema | Default — safe for consumer upgrades |
| FORWARD | Previous schema can read data written by new schema | Safe for producer upgrades |
| FULL | Both backward and forward compatible | Strictest — safe for independent upgrades |
| NONE | No compatibility checking | Development only |
Kafka Integration
Schema Registry integrates with the Kafka ecosystem at multiple points:
- Producers — serialize messages using registered schemas; registry validates compatibility before registration
- Consumers — deserialize messages using schema ID embedded in message header
- KafkaConnect — converters use registry for source/sink schema resolution
- CDC (Debezium) — registers database table schemas automatically on capture
Deployment Model
- Deployed in
data-kafkanamespace alongside Kafka operator - Backed by Kafka topic for schema storage (internal topic
_schemas) - HA mode with multiple replicas behind a Service
- TLS for client and inter-instance communication
- Access control integrated with Keycloak
Dependencies
| Dependency | Required | Notes |
|---|---|---|
| Managed Kafka | Yes | Backend storage (internal Kafka topic) + integration target |
| cert-manager | Yes | TLS certificates |
| Keycloak | Recommended | Authentication for registry API |
| kube-prometheus-stack | Yes | Metrics and alerting |
| FluxCD | Yes | GitOps deployment lifecycle |
What Is Not Included
- Data catalog / metadata management (separate concern; OpenMetadata on long-term roadmap)
- Schema generation from database DDL (CDC handles this via Debezium)
- Schema migration tooling (application responsibility)
- Multi-cluster schema federation (single registry per Kafka cluster)
Further Reading
- Data Services Overview — family overview
- Streaming Blueprint — Kafka details
- CDC (Planned) — uses Schema Registry for captured schemas
- Portfolio Strategy — sequencing rationale