Event-Driven Microservices with Apache Kafka and Spring Boot

When an order is placed, many things should happen: send a receipt, start payment, update analytics, notify the instructor. If order-service called each of them directly, it would be slow and would break whenever one of them was down. Instead it publishes an event — "order placed" — and every interested service reacts on its own.

1. Kafka in one picture

Unlike a classic queue, Kafka keeps events for a set time (days), so a consumer that was down catches up, and new consumers can replay history.

2. The event and the producer

// File: order-service/src/main/java/com/skills101/orders/OrderPlacedEvent.java
package com.skills101.orders;

// The message other services receive through Kafka
public record OrderPlacedEvent(long orderId, Long courseId, String email, int fee) {
}

In OrderService (previous lesson) one line publishes it, keyed by the order id:

// File: order-service/src/main/java/com/skills101/orders/OrderService.java (excerpt)
// asynchronous: publish an event; payments, email and analytics react in their own time
kafka.send(TOPIC, String.valueOf(orderId), new OrderPlacedEvent(orderId, course.id(), request.email(), course.fee()));

KafkaTemplate is auto-configured by Spring Boot. Events are sent as JSON:

# File: order-service/src/main/resources/application.properties (excerpt)
spring.kafka.bootstrap-servers=localhost:9092
spring.kafka.producer.value-serializer=org.springframework.kafka.support.serializer.JacksonJsonSerializer
spring.kafka.consumer.value-deserializer=org.springframework.kafka.support.serializer.JacksonJsonDeserializer
spring.kafka.consumer.properties.spring.json.trusted.packages=com.skills101.orders
spring.kafka.consumer.auto-offset-reset=earliest

(Spring Kafka 4 uses Jackson 3, so the classes are JacksonJsonSerializer / JacksonJsonDeserializer. The deserializer only creates classes from trusted packages — a security rule.)

3. The consumer

// File: order-service/src/main/java/com/skills101/orders/ReceiptListener.java
package com.skills101.orders;

import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;

// In a real system this would live in a separate notification-service
@Component
public class ReceiptListener {

    private final List<String> sent = new CopyOnWriteArrayList<>();

    @KafkaListener(topics = OrderService.TOPIC, groupId = "notifications")
    public void onOrderPlaced(OrderPlacedEvent event) {
        String receipt = "Receipt for order " + event.orderId() + " sent to " + event.email() + " (Rs " + event.fee() + ")";
        sent.add(receipt);
    }

    public List<String> sent() {
        return sent;
    }
}

@KafkaListener runs the method for every event on the topic, on Kafka's own threads. In a real system this would be a separate notification-service; another group (analytics) could read the same events independently.

4. Tested with an embedded Kafka broker

Our test starts a real Kafka broker inside the test (@EmbeddedKafka), places an order, and waits for the listener:

// File: order-service/src/test/java/com/skills101/orders/OrderServiceApplicationTests.java (excerpt)
// the Kafka consumer runs on its own thread — wait up to 20 s for the receipt
long end = System.currentTimeMillis() + 20_000;
while (receipts.sent().isEmpty() && System.currentTimeMillis() < end) {
    Thread.sleep(200);
}
assertThat(receipts.sent()).containsExactly("Receipt for order 1 sent to asha@skills101.in (Rs 4999)");

It passes: the receipt arrives as "Receipt for order 1 sent to asha@skills101.in (Rs 4999)".

5. Delivery guarantees (interview favourites)

• At-most-once: may lose events. At-least-once (the usual default): may deliver twice. Exactly-once: Kafka transactions, more complex.
• So make consumers idempotent: processing the same event twice must be harmless (e.g. ignore an order id you've already handled).
• Writing to your database and publishing an event can't be one transaction — the outbox pattern stores the event in your database first and publishes it afterwards.

Common mistakes

• Using events for things that need an immediate answer (use a request/response call).
• Consumers that aren't idempotent.
• Random keys when order matters (all events for one order should share a key).
• Trusting all packages in the JSON deserializer (*).

Quick recap

• Publish events so services stay decoupled and keep working when others are down.
• Topic → partitions → offsets; each consumer group gets every event.
• KafkaTemplate.send(topic, key, value) to produce; @KafkaListener to consume.
• At-least-once delivery → idempotent consumers; outbox pattern for reliable publishing.