Skip to content

Contact sales

By filling out this form and clicking submit, you acknowledge our privacy policy.
  • Labs icon Lab
  • A Cloud Guru
Google Cloud Platform icon

Creating a Topic with Custom Configurations in Kafka

There are many custom configurations that we can apply to topics in Kafka. In this hands-on lab, we'll go through creating a topic, applying a custom configuration to that topic, and then testing the custom configuration by alerting the state of the partitions tied to that topic.

Google Cloud Platform icon

Path Info

Clock icon Intermediate
Clock icon 45m
Clock icon Jul 17, 2019

Contact sales

By filling out this form and clicking submit, you acknowledge our privacy policy.

Table of Contents

  1. Challenge

    Set Up the Cluster

    Use Docker Compose to build the Kafka Cluster:

    cd content-kafka-deep-dive
    docker-compose up -d --build

    Now, let's make sure Java is installed (if you receive a dpkg frontend lock message, wait a few minutes and then try again):

    sudo apt install default-jdk

    Unzip and change into the Kafka binaries directory:

    tar -xvf kafka_2.12-2.2.0.tgz && mv kafka_2.12-2.2.0/ kafka
  2. Challenge

    Create a Topic with Three Partitions and a Replication Factor of `3`

    Create a topic named transaction:

    bin/ --zookeeper localhost:2181 
    --topic transaction 
    --replication-factor 3 
    --partitions 3
  3. Challenge

    Add a Custom Configuration to the Topic

    Add the custom configuration min.insync.replicas=3 to the topic transaction:

    bin/ --zookeeper localhost:2181 
    --entity-type topics 
    --entity-name transaction 
    --add-config min.insync.replicas=3

    Verify the topic configuration applied:

    bin/ --zookeeper localhost:2181 
    --entity-type topics 
    --entity-name transaction
  4. Challenge

    Change the Replica Count for the Topic

    First off, let's create a JSON file named replicacount.json with these contents:

     [{"topic": "transaction", "partition": 0,
     "replicas": [

    Now, we can execute the replica count change but using that JSON file:

    bin/ --zookeeper localhost:2181 
    --reassignment-json-file replicacount.json

    Once we get a "Successfully started..." message, let's describe the topic to see the replica change:

    bin/ --zookeeper localhost:2181 
    --topic transaction 
  5. Challenge

    Run a Producer to Get an Error Message

    Open a producer and send some messages to your topic:

    bin/ --broker-list localhost:9092 
    --topic transaction 
    --producer-property acks=all

The Cloud Content team comprises subject matter experts hyper focused on services offered by the leading cloud vendors (AWS, GCP, and Azure), as well as cloud-related technologies such as Linux and DevOps. The team is thrilled to share their knowledge to help you build modern tech solutions from the ground up, secure and optimize your environments, and so much more!

What's a lab?

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Provided environment for hands-on practice

We will provide the credentials and environment necessary for you to practice right within your browser.

Guided walkthrough

Follow along with the author’s guided walkthrough and build something new in your provided environment!

Did you know?

On average, you retain 75% more of your learning if you get time for practice.

Start learning by doing today

View Plans