With the sustained success of the Spark data processing platform even data scientists with a strong focus on the Python ecosystem can no longer ignore it. Fortunately [...]
Differential Privacy is a topic of growing interest in the world of Big Data. It is currently being deployed by tech giants like Google and Apple to gain knowledge ab [...]
In today’s blog I am going to take a look at a fairly mundane and unspectacular use of a Hive UDF (user-defined function), that of performing lookups against re [...]
A common pattern in most data science projects I participated in is that it’s all fun and games until someone wants to put it into production. From that point in time [...]
In this part of our network anomaly detection series we want to compare two basically different styles of learning. The very first post introduced the simple k-means [...]
In diesem Mittschnitt unseres Meetups in Karlsruhe zeigt Wolfgang, ein begeisterter Triathlet und Tracking Fan, wie er die Daten seiner Garmin Watch selbst mit Elasti [...]
Nowadays, Spark surely is one of the most prevalent technologies in the fields of data science and big data. Luckily, even though it is developed in Scala and runs in [...]
The previous two posts gave a short introduction of network anomaly detection in general. We also introduced the k-means algorithm as a simple clustering technique an [...]
Before we actually dive into this topic, imagine the following: You just moved to a new place and the time is ripe for a little house-warming dinner with your best fr [...]
In the previous post we talked about network anomaly detection in general and introduced a clustering approach using the very popular k-means algorithm. In this blog [...]
In order to build data services or advanced machine learning models, organizations must integrate large amounts of information from diverse sources. As a central plac [...]
In the field of machine learning and particularly in supervised learning, correlation is crucial to predict the target variable with the help of the feature variables [...]
At a large client in the German food retailing industry, we have been running Spark Streaming on Apache Hadoop™ YARN in production for close to a year now. Overall, S [...]
Sometimes the analytical power of built-in Hive functions is just not enough. In this case it is possible to write hand-tailored User-Defined Functions (UDFs) for tra [...]
The ELK/Elastic stack is a common open source solution for collecting and analyzing log data from distributed systems. This article will show you how to run an ELK on [...]