Kubernetes and Docker are great tools to manage your microservices, but operators and developers need tools to debug those microservices if things go south. Log messages and application metrics are the usual tools in this cases. To centralize the access to log events, the Elastic Stack with Elasticsearch and Kibana is a well-known toolset. In this blog post I want to show you how to integrate the logging of Kubernetes with the Elastic Stack. To start off, I will give an introduction to the log mechanism of Kubernetes, then I’ll show you how to collect the resulting log events and ship them into the Elastic Stack. I also provide a GitHub repository with a working demo. Finally, I highlight some considerations for the production deployment. Weiterlesen
At a large client in the German food retailing industry, we have been running Spark Streaming on Apache Hadoop™ YARN in production for close to a year now. Overall, Spark Streaming has proved to be a flexible, robust and scalable streaming engine. However, one can tell that streaming itself has been retrofitted into Apache Spark™. Many of the default configurations are not suited for a 24/7 streaming application. The same applies to YARN, which was not primarily designed with long-running applications in mind. Weiterlesen
Continuous integration (CI) and continuous delivery (CD) are a great help, providing the flexibility needed for agile software development methods like Scrum and Kanban. With CI/CD, you don’t have to constantly struggle with the build and deployment processes of your software project. Once correctly configured, you can be assured that the whole build and delivery process is just a matter of pushing the code into the source code management system or even more simply pressing a button. Weiterlesen
Beside the ongoing competition between React and Angular, some people try to benefit from both technologies. The ngReact library is an example of such an approach. The Angular module allows using React as a view component inside Angular applications. Usual Angular templates can be replaced with React components which try to give you more flexibility and promise to improve the overall performance.
This blog article examines the use of the ngReact module to speed up Angular’s list rendering performance. Weiterlesen
When web developers talk about the web today they often discuss topics around web performance optimization (WPO). Nowadays it’s an even more important topic, since we use the browser for almost any type of application with many different devices and different connection types from all over the world. It’s a complex environment where dozens of lines of code get written and executed. Companies like Amazon and eBay have huge decreases in revenue when their site loading times increase. In 2008, Amazon reported that they approximately lose 1% of revenue for every 100 ms increase in loading time, which when we think about the revenues of Amazon in 2015 (107 billion dollar a year) would imply a loss of 1.07 billion dollar a year.
And that’s by far not everything. In a recent talk given by one of the head Opera developers, Bruce Lawson mentioned that more users and devices will hit the WWW: Emerging markets like Africa and India are growing fast. He predicts that we will have roughly 3 billion more users using the WWW in the next 50 years and that they will come from emerging markets. They will not connect to the internet with a high end desktop computer, they will start using the internet with a low budget smartphone, with slow internet connections and low processing capabilities. These markets will eventually be our next customers, and therefore we need to emphasise the need of performance, so that everybody is able to use our web applications no matter their device or internet connection quality!
In this blog series I want to discuss what web performance is, how to look at it from a user-centric perspective and show which kind of optimization techniques we can use to make our web applications faster.
First, let’s start this series by showing the limits of web performance optimization to increase awareness for measurement parameters like seconds (sec) and milliseconds (ms). Weiterlesen
Sometimes the analytical power of built-in Hive functions is just not enough. In this case it is possible to write hand-tailored User-Defined Functions (UDFs) for transformations and even aggregations which are therefore called User-Defined Aggregation Functions (UDAFs). In this post we focus on how to write sophisticated UDFs and UDAFs in Python. Weiterlesen
The ELK/Elastic stack is a common open source solution for collecting and analyzing log data from distributed systems. This article will show you how to run an ELK on Docker using Docker Compose. This will enable you to run ELK distributed on your docker infrastructure or test it on your local system. Weiterlesen
No, that’s not a typo and I’m not talking about the alien species, either. Ha Ha… I’m sorry – it had to be done. So yeah, object-oriented design. One example of it. Here we go 🙂 Weiterlesen
This article will guide you through updating the ELK stack from version 1.x to 2.x, taking into account the correct order of its components Elasticsearch, Logstash and Kibana. Weiterlesen
In previous blog articles we talked about the basic Redis features and learned how to persist, backup and restore your dataset in cause of a disaster scenario. Today we want to introduce you to a more complex setup. In fact, you can teach your Redis instances to be highly available for your clients. Weiterlesen