Apache Kafka on AWS – Amazon Managed Streaming for Apache Kafka / AWS MSK

This tutorial will show you how to create a Kafka cluster using AWS Managed Streaming For Kafka (MSK) and stream data to that cluster from an EC2. We use the AWS ‘Getting Started’ guide for MSK as our reference point as we create a Kafka streaming solution utilizing the Managed Streaming Kafka service. The end result produces a solution on which we can send a typed message on an AWS EC2 onto to the stream, which in turn is polled by a consumer using a topic so the message is rendered back to the user in real time.

Apache Kafka is one the most popular open-source projects for building messaging and streaming applications. Kafka takes data from various sources, makes it available to different applications, and therefore helps to eliminate daily batch jobs.

AWS Reference Guide
https://www.youtube.com/redirect?q=https%3A%2F%2Fdocs.aws.amazon.com%2Fmsk%2Flatest%2Fdeveloperguide%2Fgetting-started.html&event=video_description&v=y4wowEQd4Os&redir_token=QUFFLUhqbWdfcS01MXV3WjVHTl80MlhndVg2RWtneDh6Z3xBQ3Jtc0trUGlZR0VkZ3VORUVvRl9PZ3o5d0RRZE4zTFNrZElMbkJCS291WUt5eTEyMWNCeV9CSm14VHFuMlcyTjkyYWhLWjliU3M1ci1Qc3d5bFZ5SDFXT1NkMVM0MzUzZnJpSGtUT3VDLUFMOXpUNlJ0b2tFUQ%3D%3D

Kafka plays an important role for Change Data Capture (CDC) and in the world of microservices. This presentation gives an overview of the new Amazon Managed Streaming for Kafka (Amazon MSK).

Based on knowledge gained from several Kafka implementation projects I will explain some of the technical underpinnings first. You will learn about brokers, topics, and Zookeeper. Then I will explain what makes Kafka special, analyze major pain points in on-prem Kafka projects, critically analyze how Kafka differs from Kinesis, and why the cloud is the best way to use Kafka

#amazonmsk #awsmsk #aws

Written by admin