Table of Contents

1. Challenge

   Set Up the Cluster

   Use Docker Compose to build the Kafka Cluster:

   cd content-kafka-deep-dive
   
   docker-compose up -d --build
   

   Now, let's make sure Java is installed (if you receive a dpkg frontend lock message, wait a few minutes and then try again):

   sudo apt install default-jdk
   

   Unzip and change into the Kafka binaries directory:

   tar -xvf kafka_2.12-2.2.0.tgz && mv kafka_2.12-2.2.0/ kafka
   

2. Challenge

   Create a Topic with Three Partitions and a Replication Factor of `3`

   Create a topic named transaction:

   bin/kafka-topics.sh --zookeeper localhost:2181 
   --create 
   --topic transaction 
   --replication-factor 3 
   --partitions 3
   

3. Challenge

   Add a Custom Configuration to the Topic

   Add the custom configuration min.insync.replicas=3 to the topic transaction:

   bin/kafka-configs.sh --zookeeper localhost:2181 
   --alter 
   --entity-type topics 
   --entity-name transaction 
   --add-config min.insync.replicas=3
   

   Verify the topic configuration applied:

   bin/kafka-configs.sh --zookeeper localhost:2181 
   --describe 
   --entity-type topics 
   --entity-name transaction
   

4. Challenge

   Change the Replica Count for the Topic

   First off, let's create a JSON file named replicacount.json with these contents:

   {"partitions":
    [{"topic": "transaction", "partition": 0,
    "replicas": [
    2
    ]
    }
    ],
    "version":1
   }
   

   Now, we can execute the replica count change but using that JSON file:

   bin/kafka-reassign-partitions.sh --zookeeper localhost:2181 
   --execute 
   --reassignment-json-file replicacount.json
   

   Once we get a "Successfully started..." message, let's describe the topic to see the replica change:

   bin/kafka-topics.sh --zookeeper localhost:2181 
   --topic transaction 
   --describe
   

5. Challenge

   Run a Producer to Get an Error Message

   Open a producer and send some messages to your topic:

   bin/kafka-console-producer.sh --broker-list localhost:9092 
   --topic transaction 
   --producer-property acks=all