Managed Kafka: Architecture Reference
Purpose: For platform engineers and operators, provides a structured lookup of Kafka CRDs, topology options, platform dependencies, and lifecycle operations.
CRDs
| CRD | Kind | Scope | Purpose |
|---|
kafka.strimzi.io/v1beta2 | Kafka | Namespaced | Cluster definition (brokers, storage, listeners, KRaft) |
kafka.strimzi.io/v1beta2 | KafkaTopic | Namespaced | Topic creation and configuration |
kafka.strimzi.io/v1beta2 | KafkaUser | Namespaced | User creation, authentication, ACLs |
kafka.strimzi.io/v1beta2 | KafkaConnect | Namespaced | Connect cluster with plugin management |
kafka.strimzi.io/v1beta2 | KafkaConnector | Namespaced | Individual connector lifecycle |
kafka.strimzi.io/v1beta2 | KafkaMirrorMaker2 | Namespaced | Cross-cluster replication |
kafka.strimzi.io/v1beta2 | KafkaBridge | Namespaced | HTTP REST bridge |
kafka.strimzi.io/v1beta2 | KafkaNodePool | Namespaced | Node pool definitions (broker/controller roles) |
Topology Options
| Profile | Brokers | Replication | Use Case |
|---|
| Development | 1 | None | Local dev and testing |
| Production | 3 | min.insync.replicas: 2 | Standard workloads |
| Stretch (multi-AZ) | 3+ | Rack-aware partition assignment | High availability across failure domains |
Production and stretch topologies enable Cruise Control for partition rebalancing after scaling events.
| Dependency | Required | Purpose |
|---|
| cert-manager | Yes | TLS certificate issuance and rotation |
| Keycloak | Optional | OAuth2/OIDC authentication (alternative: SCRAM/mTLS) |
| kube-prometheus-stack | Yes | Metrics collection and alerting |
| Grafana | Yes | Pre-built Kafka dashboards |
| Loki | Yes | Operator and broker log aggregation |
| Harbor | Recommended | Image scanning and provenance verification |
| Kyverno | Yes | Policy enforcement (image sources, resource limits) |
| FluxCD | Yes | GitOps reconciliation of Kafka CRDs |
| Velero | Recommended | Backup of operator state and PVCs |
All dependencies are provided by the Platform Foundation.
Lifecycle Operations
| Operation | Mechanism | Trigger |
|---|
| Provision | Strimzi Cluster Operator creates StatefulSets, Services, Secrets | Kafka CR applied |
| Scale out | Add brokers to Kafka CR spec | Git commit → FluxCD reconcile |
| Scale in | Remove brokers + Cruise Control partition reassignment | Git commit + manual rebalance trigger |
| Upgrade (Kafka) | Version bump in Kafka CR | Git commit → rolling restart |
| Upgrade (operator) | Helm chart version bump in gitops-base | Git tag → FluxCD reconcile |
| Backup | Velero PVC snapshots + topic config export | Scheduled or manual |
| Restore | Velero restore + Strimzi re-reconciliation | Manual |
| Credential rotation | Update KafkaUser CR or SCRAM secret | Git commit → User Operator reconcile |
Further Reading