Kafka

It’s 3 AM again. A customer placed an order, payment went through, and inventory reservation failed — the warehouse is out of stock. The customer has been charged for something that will never ship. The order row exists and the customer was charged, but nothing will ship — and no single transaction can unwind it, because there is no @Transactional that spans your order service, your payment service, and your inventory service. Each one already committed its own local transaction. ...

It’s 3 AM. A customer paid, the orders row is in your database, and the OrderCreated event never reached Kafka. The warehouse never shipped. The email never sent. Somewhere in your service there is code that looks like this: @Transactional public Order placeOrder(PlaceOrder cmd) { Order order = orderRepository.save(new Order(cmd)); // write #1 kafkaTemplate.send("orders", order.getId(), toJson(order)); // write #2 — outside the safety net return order; } Two writes. No shared atomicity. If the broker is down, if the pod gets killed, if the send times out after the commit — the database and the downstream world silently disagree, and you get to find out from an angry customer. This is the dual-write problem, and the fix is the transactional outbox pattern. ...

Kafka - Distributed Stream Processing System Kafka is an open-source distributed streaming platform. In simpler terms, it’s a system that excels at handling large amounts of data that is constantly being generated. This data, often called streaming data, comes from various sources and needs to be processed quickly and efficiently. Push Poll Model Kafka Components Kafka Broker Default port 9092 Producer: Producer produces content Consumer: Consumer consumes the content The producer/Consumer creates a TCP connection to the broker, which is bi-directional. That means both producer and broker can send and receive data from each other. ...