Mehrwerte von Cloud Services in hybriden DWH-Architekturen

Die Gründe, IT-Infrastruktur aus dem eigenen Rechenzentrum in eine (Public) Cloud zu verschieben, sind vielfältig und einleuchtend: Niedrigere Kosten, schnellere Time-to-market und Effizienz des Kapitaleinsatzes, um nur die Wichtigsten zu nennen. Die komplette Verlagerung eines Corporate Data Warehouse in die Cloud ist allerdings bislang eher die Ausnahme. Abgesehen von der gerade in Deutschland sehr ernsthaft geführten Datenschutz-Diskussion würde es viel Aufwand und Kosten bedeuten, die Daten aller relevanten Quellsysteme mit der Cloud zu synchronisieren oder alle Systeme des Unternehmens dorthin umzuziehen. Ein Zwischenweg stellen hybride DWH-Architekturen dar, welche selektiv Cloud-Dienste nutzen und dabei die On-Premises und Cloud-Datenbestände intelligent integrieren. Die Vorteile dieser Services für Data-Management- und Analytics-Aufgaben werden in den folgenden Abschnitten dargestellt. Im Anschluss werden einige typische Szenarien aus der Praxis vorgestellt. Weiterlesen

Kubernetes Logging with Fluentd and the Elastic Stack

Kubernetes and Docker are great tools to manage your microservices, but operators and developers need tools to debug those microservices if things go south. Log messages and application metrics are the usual tools in this cases. To centralize the access to log events, the Elastic Stack with Elasticsearch and Kibana is a well-known toolset. In this blog post I want to show you how to integrate the logging of Kubernetes with the Elastic Stack. To start off, I will give an introduction to the log mechanism of Kubernetes, then I’ll show you how to collect the resulting log events and ship them into the Elastic Stack. I also provide a GitHub repository with a working demo. Finally, I highlight some considerations for the production deployment. Weiterlesen

Modern CI/CD with Jenkins 2 and GitLab CI [Comparison]

Continuous integration (CI) and continuous delivery (CD) are a great help, providing the flexibility needed for agile software development methods like Scrum and Kanban. With CI/CD, you don’t have to constantly struggle with the build and deployment processes of your software project. Once correctly configured, you can be assured that the whole build and delivery process is just a matter of pushing the code into the source code management system or even more simply pressing a button. Weiterlesen

Drastic Elastic [Part 3]: Cluster Setup

ElasticSearch does not offer support for clusters spanning data centres. However, on our project we had access to a network latency of 400 *micro*seconds (0.4 ms) between three separate locations in the same city, and decided to test a cluster spanning all three data centres. Network latency did not prove to be a problem, but a more tricky issue was deciding how to set up the cluster to best guard against network partitioning. Weiterlesen