Apache Kafka

Enable participants to design, develop, and operate robust real‑time data pipelines using Apache Kafka, Kafka Streams, and Kafka Connect.

Get Course Info

Audience: Developers, Architects

Duration: 3 days

Format: Lectures and hands‑on labs (50% lecture, 50% lab)

Overview

This course will teach Apache Kafka – a popular distributed messaging system – covering design, architecture, programming APIs, operations, best practices, and real‑world use cases.

Objective

Enable participants to design, develop, and operate robust real‑time data pipelines using Apache Kafka, Kafka Streams, and Kafka Connect.

What You Will Learn

Streaming technologies and architecture
Kafka concepts and architecture
Programming using Kafka Producer & Consumer APIs
Kafka Streams API
Kafka Connect
KSQL / KSQLdb
Monitoring & tuning Kafka
Best practices & case studies

Course Details

Audience: Developers, Architects

Duration: 3 days

Format: Lectures and hands‑on labs (50% lecture, 50% lab)

Prerequisites:

Comfortable with Java programming language (exercises are in Java) • Basic Linux command‑line skills

Setup: Zero‑install cloud lab • Modern laptop with unrestricted Internet • Chrome browser • SSH client

Detailed Outline

Understanding fast data & streaming terminologies
Exactly‑once / at‑least‑once / at‑most‑once patterns
Popular streaming architectures & Lambda architecture

Compare Kafka with JMS / MQ
Kafka architecture: topics, partitions, brokers, commit logs
Kafka & Zookeeper
Producing and consuming messages
Consumer groups & message retention
Scaling Kafka
Labs: start a Kafka cluster & use utilities

Producer & Consumer APIs
Offsets and commits (auto / manual)
Producer send modes & compression
Labs: coding producer / consumer & offset management

Streams library design
KStream / KTable concepts
Transformations, joins, aggregations
Labs: Kafka Streams applications

Connect ecosystem & popular connectors
KSQL / KSQLdb
Schema Registry

Kafka metrics & Grafana dashboards
Instrumenting applications
Labs: monitor cluster & instrument apps

Avoiding common mistakes
Sizing clusters, partitions, hardware
Compression & batching
Real‑world Kafka use cases