In this article we present a simple solution for managing Isolated Environments with PySpark that we have been using in production for more than a year.

With the sustained success of the Spark data processing platform even data scientists with a strong focus on the Python ecosystem can no longer ignore it. Fortunately

Here's what changes when your data science project grows from a proof of concept. How do you deploy your model, how can updates be rolled out, ...?

A common pattern in most data science projects I participated in is that it’s all fun and games until someone wants to put it into production. From that point in time

Nowadays, Spark surely is one of the most prevalent technologies in the fields of data science and big data. Luckily, even though it is developed in Scala and runs in

The previous two posts gave a short introduction of network anomaly detection in general. We also introduced the k-means algorithm as a simple clustering technique an

Before we actually dive into this topic, imagine the following: You just moved to a new place and the time is ripe for a little house-warming dinner with your best fr

In the previous post we talked about network anomaly detection in general and introduced a clustering approach using the very popular k-means algorithm. In this blog

In supervised learning, correlation is crucial to predict the target variable with the help of the feature variables. But what good is causation?

In the field of machine learning and particularly in supervised learning, correlation is crucial to predict the target variable with the help of the feature variables

In this post we focus on how to write sophisticated User Defined (Aggregated) Functions (UD(A)Fs) for Apache Hive in Python.

Sometimes the analytical power of built-in Hive functions is just not enough. In this case it is possible to write hand-tailored User-Defined Functions (UDFs) for tra