Real-time Architectures with Spark and Kafka

Training course on the architectural principles involved in operating big data systems

Request now
Logo von Spark kafka

At a glance

General information

2 days practical training, in-house

Target group

Software Architects, Software Developers

Application examples

Operating big data systems capable of processing large amounts of real-time data

Making real-time data highly available for queries

Description

The training sessions are usually held in German. Please contact us if you are interested in training sessions in English.

Training course on the architectural principles involved in operating big data systems

With the number of use cases increasing, big data systems are now facing the challenge of processing data streams instead of files. As this circumstance necessitates transitioning from a batch processing to an event streaming model, it often means reorganizing (or even redeveloping) the entire architecture of a big data system.

This training course examines the architectural principles needed to operate big data systems capable of processing large amounts of real-time data and making it highly available for queries. In order to explore these, course participants will use Spark and Kafka to set up a sample big data system capable of processing the Wikipedia edit stream, a real-time data stream containing every single edit made to every single Wikipedia article.

Agenda

  • Event streams (brokers, topics, partitions in Kafka)
  • Stream processing (transformations, processing patterns, error handling (at-least-once vs exactly once)
  • Offloading / archiving large volumes of data (Lambda Architecture, Flume, Kafka Connect, Camus/Gobblin)
  • Storing analysis results (caches (HBase, Cassandra, Riak, Redis), dashboards (ES, Kibana), handling historical data)

Typical questions we answer:

  • What are the key components of event streams in Kafka and how are brokers, topics and partitions defined in this context?
  • What concepts and practices are important in the stream processing environment - including transformations, processing patterns and dealing with errors (e.g., at-least-once vs. exactly once)?
  • How does the offloading and archiving of large amounts of data work within the Lambda architecture and what role do tools such as Flume, Kafka Connect, Camus and Gobblin play in this?
  • What options are available to store and retrieve the results of stream processing operations? How can caches (such as HBase, Cassandra, Riak and Redis) and dashboards (such as Elasticsearch and Kibana) be used for this purpose?
  • How do you deal with historical data in an event stream processing architecture and what challenges can arise when storing and accessing this data?
Real-time Architectures with Spark and Kafka
This training is currently on demand only - contact us now.
Request now

Training forms

Training forms according to your needs: Open trainings take place on fixed dates in mixed groups at an inovex location, inhouse trainings you book individually – configurable as desired.

Inhouse training

  • Training agenda customizable to the group and the project
  • Confidential atmosphere (trainers are under NDA)
  • Configurable according to your needs: place, time, language, tooling
Request now

Open training

  • optimal for individuals
  • new impulses from other participants
  • getting to know other people interested in tech

Frequently Asked Questions

Will I receive a certification as a result of the training?
All participants will receive a certificate of participation from the inovex Academy after the training.
When does the training start?
Our trainings start at 09:00 Central European Time.
Do I get an invitation? When do I get it?
The trainer sends out the invitations about 1 week before the start of the training. In addition to the agenda and the schedule, any preparations (installation of software, etc.) will be pointed out again.
Portraitbild von Collin Rogowski
Collin Rogowski
Head of inovex Academy
inovex Logo
Go back
Portraitbild von Collin Rogowski

I look forward to your inquiry.

Collin Rogowski

We are your partner for successful training

We would be happy to talk to you personally about your concerns. Get in touch now!

Portraitbild von Collin Rogowski
Collin Rogowski
Head of inovex Academy
  • Customized training courses for your company
  • Over 25 years of experience

Real-time Architectures with Spark and Kafka

Expand your skills and develop your expertise! Our experienced trainers will help you achieve your goals. Sign up and take your know-how to a new level! Request now