Big Data: Optimised Fraud Detection in Microsoft Azure

arvato Financial Solutions is collaborating with Microsoft, cloud and big data specialists inovex GmbH, and three pilot e-commerce customers on an innovation project to create a big-data architecture based on Microsoft Azure. The team will use the project to evaluate how the combination of cloud computing, big data and advanced analytics can improve fraud prevention and facilitate the development of new financial BPO services.

Starting Point

This innovative project will combine cloud computing, big data and advanced analytics to explore new opportunities for considerably improving fraud detection. It will also allow the development of new financial BPO services.


The solution will comprise a big-data architecture, which will be realised on Microsoft Azure. The architecture will combine Azure services for data management and advanced analytics with open-source components to create an integrated, cloud-based, real-time solution. The frontend for the interactive will be implemented using Power BI.


The combination of cloud-based big-data technologies with advanced analytics has proven itself as a model for continuous improvement in e-business fraud detection. In addition to being technically innovative, the solution is compelling in its flexibility and scalability and in the options it provides for simple, international deployments.

„Our big-data project has enabled us to improve fraud detection rates and to lay the foundations for innovative financial BPO services.“

Kai Kalchthaler, Executive Vice President Risk Management, arvato Financial Solutions

Opportunity makes a thief ? an old adage that applies more than ever to online business. In Germany, 70 percent of online retailers have already fallen victim to attempted fraud, whereas just 14 percent of retailers use fraud detection methods (according to a study carried out by the HÑndlerbund entitled ?Betrugserkennung im Online-Shop [Fraud Detection in Online Stores]?). For online retailers and e-commerce providers it is, therefore, important that fraud attempts are detected as early as possible.

„Many retailers do not possess the resources for efficient risk and receivables management,“ explains Andreas Czermak, Director Big Data Projects & Strategy at arvato Financial Solutions. The integrated financial services provider handles these important tasks for around 2,000 customers: arvato specialises in outsourced payment flow services (financial BPO) from risk management to invoicing and debitor management, right through to collections.

„In many cases, traditional risk management solutions offered by credit agencies do not go far enough, and fraud techniques, including Fake Identity, Identity Theft, and Account Takeover are becoming more sophisticated all the time,“ Czermak explains. It is, therefore, essential to find intelligent, rapidly scalable solutions which can be used to analyse transaction data in real time in order to keep pace with these developments. In the autumn of 2015, arvato joined forces with Microsoft, cloud and big-data specialists inovex GmbH, and three pilot e-commerce customers to launch an ambitious project: „We wanted to explore the extent to which the combination of cloud computing, big data and advanced analytics can be used to improve fraud detection rates and implement new financial BPO services.“

Big Data at its Best: Hadoop & Storm on Microsoft Azure

„Combining Microsoft Azure services with the Storm and Hadoop open-source frameworks to create an integrated, cloud-based solution is big data at its best,“ says Patrick Thoma, Head of Data Management & Analytics at inovex. Microsoft Azure is an ideal platform for the project. First and foremost, it fulfils all the basic requirements, like performance, scalability, international deployment and data protection. In addition, the use of standardised Microsoft services like Azure HDInsight enables rapid development and deployment in the cloud: with no new hardware or high levels of investment required.

Real-time Analysis and Machine Learning Combined

„Using Azure makes for quick launches,“ explains Dominik Benz, the technical project leader for inovex. Azure HDInsight offers Apache Hadoop and Storm (as well as other frameworks) as a managed cloud service (platform as a service, PaaS). As part of the project, a modern Lambda architecture was implemented: the batch-string uses Hadoop to transform existing data in order to use machine learning algorithms from previous fraud cases to develop self-learning analysis models for early detection. The real-time string records the incoming transaction data in Azure Event Hubs and analyses it using Storm and Azure Machine Learning in order to detect potential fraud attempts in real time. „The real challenge was performance tuning the cloud architecture,“ reports Benz. This is where inovex's years of experience with big data and advanced analytics projects came to the fore.

Visualisation with Power BI

„One of the project?s major aims was the visualisation and monitoring of the frontend models,“ Czermak emphasised. This function is performed by Power BI: at a central monitoring location, multiple big screens visualise the data sets directly from the cloud sources (Azure HDInsight and the SQL database).

International Growth with the New Financial BPO Services

„The design of the architecture is decisive in using cloud services to reliably fulfil SLAs,“ says Czermak. The investment in good design is worth it. The architecture selected provides a high level of flexibility and enables rapid deployment, particularly when it comes to combating e-business fraud on an international level. In addition, the combination of machine learning processes with the real-time analysis of incoming transaction data (real-time analytics) has proven itself in the continuous improvement of analysis models. Kai Kalchthaler, Executive Vice President Risk Management at arvato Financial Solutions, sums up the experience positively: „Overall, this pilot project has enabled us to obtain some extremely valuable insights, which will now be used in our strategic project development of new cloud-based risk management services. We've got the right platform and the appropriate partners.“

Would you like a consultation on this subject?

Call us on +49 (0)721 619 021-0 or send us an E-Mail. We look forward to advising you.

Patrick Thoma

I look forward to hearing from you!

Patrick Thoma

Head of Data Management & Analytics

inovex Services

Analytics: Business Intelligence, Big Data Platforms, Data Science, Search

Read more

About us

Our Technology Partners

inovex cooperates with a range of selected technology partners to offer our customers genuine added value: Cloudera, Elastic, Hortonworks, Liferay, MapR, Microsoft, Red Hat, Quobyte and UID.

Read more