CDC with Postgres Debezium to Kafka Strimzi

Postgres Database — Kafka Connect — Kafka

A little intro to Strimzi:

A little intro to Debezium:

Starting up Minikube

minikube config set memory 4096
minikube config set cpus 4
minikube start

Create a Postgres database

Kubectl create namespace postgres
Username: data_engineerPassword: password
kubectl get pods -n postgres

Strimzi for Kafka

Kubectl create namespace kafka
curl -L | sed ‘s/namespace: .*/namespace: kafka/’ | kubectl apply -f — -n kafka
kubectl -n kafka apply -f && kubectl wait kafka/my-cluster — for=condition=Ready — timeout=300s -n kafka

Create the KafkaConnect

Create Postgres Connector

kubectl -n kafka port-forward svc/postgres-connect-cluster-connect-api 8083:8083

How To Test:

kubectl -n postgres port-forward svc/postgres 5432:5432
kubectl -n kafka exec my-cluster-kafka-0 -c kafka -i -t --   bin/ --bootstrap-server localhost:9092 --topic server1.inventory.customers




Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store