Introduction in Apache Kafka
Începător
Introduction in Apache Kafka
Durată: 3 zile
Certificare: Diploma de participare
Cui îi este dedicat cursul?
Being an introductory level this course can be attended by architects, devs, devops, product managers.
Cunoștințe și abilități inițiale
- This course requires completion of the Introduction to Big Data Hadoop course. We will position Apache Kafka in Big Data architectures in that course as well as present basic concepts that will be required for understanding Kafka position in an end to end big data solution;
- We will mostly reference with the Console Producer/Consumer but we will address as well the Java API’s for producers and consumers and although familiarity with Java is a plus, this course it is not addressed (only) to devs thus anybody with IT background can participate. Knowledge of distributed systems/architectures is a plus.
Prezentarea cursului
The course provides an overview of the main concepts and architectural components as well going in detail in the 3 main use cases of Apache Kafka: messaging bus, data storage and streaming platform.
Ce subiecte abordează cursul
Why Apache Kafka & use cases
- where in the Big Data architecture is Kafka positioned
- ingestion layer overview
Apache Kafka Fundamentals
- Producers: writing messages to Kafka
- Overview, sending a message, configuring producers
- Serializers
- Partitions
- - Consumers: reading from Kafka
- Overview, subscribing to topics, offsets, configuring consumers
- Deserializers
- Topics
- Keys
Kafka Architecture
- Brokers
- Replication
- Storage of data: options
- Role of Zookeeper in Kafka clusters (depending on Kafka version)
- What means installing a Kafka cluster
Kafka as a messaging bus
- Producers - how to submit messages to Kafka (for examples we’ll use kafka-console-producer)
- Producer Java API demo
- Consumers - how to consume messages from Kafka (kafka-console-consumer)
- Consumer Java API demo
- Hands on using producers and consumers shell
Kafka Connect for Data Movement
- What is Kafka Connect
- Simple Message Transform
- Demo: Using Kafka connector for Twitter to create a Kafka stream
Stream Processing
- What is
- Kafka streams architecture overview
- Streams API
- KSQL overview
- Hands on using Kafka SQL
Ce abilități se dobândesc în urmă cursului
- Understand the main concepts and architectural components and the main use cases of Apache Kafka.
Course Requirements:
- We will need open Internet connection throughout the course, we will run the exercises on cloud - thus is mandatory for the Internet connection to be open and reliable;
- Each participant need to have it’s own computer in order to run the hands on exercises, also the computer settings has to allow access to Google docs and Github for getting access to presenters slides, documents, data and exercises;
- Google Chrome browser;
- For the local computers we will need an SSH client to connect to the cloud environment.