{"id":15502,"date":"2019-05-06T11:44:13","date_gmt":"2019-05-06T09:44:13","guid":{"rendered":"https:\/\/www.inovex.de\/blog\/?p=15502"},"modified":"2025-03-25T07:51:35","modified_gmt":"2025-03-25T06:51:35","slug":"the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning","status":"publish","type":"post","link":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/","title":{"rendered":"The Mystery of Entropy: How to Measure Unpredictability in Machine Learning"},"content":{"rendered":"<p>If you are dealing with Statistics, Data Science, Machine Learning, Artificial Intelligence or even general Computer Science, Mathematics, Engineering or Physics, you&#8217;ve probably come across the term <em>Entropy<\/em> more than once. Entropy is a significant, widely used and above all successful measure for quantifying <strong>inhomogeneity<\/strong>, <strong>impurity<\/strong>, <strong>complexity<\/strong> and <strong>uncertainty<\/strong> or <strong>unpredictability<\/strong>. The concept of entropy is more than 200 years old, and still, or perhaps because of that, it is an integral part of the latest machine learning models that are successfully deployed on real-world data sets. In this article, I want to highlight the simplicity, beauty and meaning of entropy.<!--more--><\/p>\n<p>Entropy and its special forms are able to achieve a <strong>conceptual understanding of the data<\/strong> in an intuitive way\u2014something we should exploit more often.<\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_83 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\"><p class=\"ez-toc-title\" style=\"cursor:inherit\"><\/p>\n<\/div><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#Entropy-What-was-that-again\" >Entropy: What was that again?<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#Shannon-Entropy\" >Shannon-Entropy<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#Cross-entropy\" >Cross entropy<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#Kullback-Leibler-Divergence\" >Kullback-Leibler-Divergence<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#Conditional-Entropy\" >Conditional Entropy<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#More-Entropies\" >More Entropies<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#Power-of-Entropy-in-Machine-Learning\" >Power of Entropy in Machine Learning<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#Classification-with-Logistic-Regression-or-Neural-Networks\" >Classification with Logistic Regression or Neural Networks<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#Decision-Trees\" >Decision Trees<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#Achieving-a-Conceptual-Understanding-of-the-Data\" >Achieving a Conceptual Understanding of the Data<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#Read-on\" >Read on<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"Entropy-What-was-that-again\"><\/span>Entropy: What was that again?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>I remember countless times that I read something about \u2018entropy\u2019, and I asked myself equally countless times \u00a0&#8218;What was that again?&#8216;. So I asked Google for the hundred and first time. Google\u2014once again\u2014<a href=\"https:\/\/www.google.com\/search?q=entropy&amp;rlz=1C5CHFA_enDE830DE830&amp;oq=entropy&amp;aqs=chrome..69i57j69i61j69i60j35i39j0l2.988j0j7&amp;sourceid=chrome&amp;ie=UTF-8\">threw out<\/a> a definition of entropy in thermodynamics with which I (as a mathematician) could not do anything and maybe I did not want to. If you want to understand the concept of entropy, information theory is probably one of the simplest approaches. Later we will see that this approach is absolutely compatible with the statistical-mathematical as well as physical concept of entropy.<\/p>\n<p>Let\u2019s start with an information-theoretical view. Everyone knows that a computer is counting on bits. We can ask ourselves \u2018<i>how many bits does the computer require to display or transmit information, e.g. a string of characters?<\/i>\u2019 If the computer needs few bits, the message seems to be less complex. If the computer needs many bits, the message has a high complexity.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Shannon-Entropy\"><\/span>Shannon-Entropy<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Let&#8217;s suppose we\u2019d like to encode the Latin alphabet including some separators and punctuation marks \\(X=\\{a,b,c,&#8230;,z,\u00a0 `&#8216;\u00a0 .\u00a0 ,\u00a0 ;\u00a0 !\u00a0 ?\\}\\) in bits for displaying them on the screen. In the following figure we consider a tree in which every single character is coded by a sequence of bits. To encode all \\(n=32\\) characters we need sequences of \\(\\log 32=5\\) bits each: \\(a = 00000\\), \\(b = 00001\\), \\(c = 00010\\), etc.<\/p>\n<p><a href=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/encoding.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-16214\" src=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/encoding.png\" alt=\"tree encoding the latin alphabet\" width=\"610\" height=\"346\" srcset=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/encoding.png 1540w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/encoding-300x170.png 300w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/encoding-1024x581.png 1024w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/encoding-768x436.png 768w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/encoding-1536x872.png 1536w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/encoding-400x227.png 400w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/encoding-360x204.png 360w\" sizes=\"auto, (max-width: 610px) 100vw, 610px\" \/><\/a><\/p>\n<p>Well, we can interpret the display of a character as a <i>statistic event<\/i>. Each event occurs with the same probability. The minimum number of bits needed to display a character can therefore also be formulated as its statistical significance:<\/p>\n<p style=\"text-align: center\">\\(\\log(n) = \\log \\frac{1}{p_i} = \\log(1) &#8211; \\log(p_i) = -\\log(p_i)\\) .<\/p>\n<p>At this point, I refer to the <strong>information-theoretical<\/strong> (left side of the equation) and <strong>statistical<\/strong> (right side of the equation) coherence and it becomes clear what the words \u2018inhomogeneity\u2019 or \u2018impurity\u2019 have in common with \u2018uncertainty\u2019 or \u2018unpredictability\u2019.<\/p>\n<p>Characters or events do not always have to be uniformly distributed. In every language there are characters, like the \u2018e\u2019, \u2018a\u2019 or \u2018space\u2019, which appear more frequently than other. For non-uniformly distributed characters we are interested in the <strong>expected number of bits<\/strong> to display a message, which can be determined by<\/p>\n<p style=\"text-align: center\">\\(H((p_1,p_2,&#8230;,p_n))=E(-\\log p_i)= -\\sum_{i=1}^{n} p_i \\log p_i .\\)<\/p>\n<p>This is what we finally call <i>entropy<\/i>. It\u2019s nothing new and was defined in 1948 by Claude Elwood Shannon, an US mathematician, electrical engineer and \u2018father of information theory\u2019. It is therefore often called <i>Shannon entropy<\/i> and in literature referred to as<\/p>\n<p style=\"text-align: center\">\\(H((p_1,p_2,&#8230;,p_n))= H((P(X=x_1),P(X=x_2),&#8230;,P(X=x_n )))=: H_X\\)<\/p>\n<p>The entropy is maximised if all the characters are equally distributed. That means that the uncertainty is greatest. If we know that some events are more likely than others, we should exploit this in our coding scheme. Imagine that we want to code the sentence \u2018this is an example for a huffman tree\u2019. We would like to allocate fewer bits to frequent characters than characters that are rare. An optimal coding scheme as in the figure below can be found by the <a href=\"https:\/\/ieeexplore.ieee.org\/document\/4051119\">huffman coding<\/a>, named after David A. Huffman.<\/p>\n<figure id=\"attachment_16218\" aria-describedby=\"caption-attachment-16218\" style=\"width: 552px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Huffman-tree.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-16218 \" src=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Huffman-tree.png\" alt=\"Huffman tree to exemplify the use of entropy\" width=\"552\" height=\"421\" srcset=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Huffman-tree.png 892w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Huffman-tree-300x229.png 300w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Huffman-tree-768x585.png 768w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Huffman-tree-400x305.png 400w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Huffman-tree-360x274.png 360w\" sizes=\"auto, (max-width: 552px) 100vw, 552px\" \/><\/a><figcaption id=\"caption-attachment-16218\" class=\"wp-caption-text\">Figure: Huffman Coding of the sentence \u2018this is an example for an huffman tree\u2019.<\/figcaption><\/figure>\n<p>&nbsp;<\/p>\n<p>In the picture above the distribution of the letters in the sentence was used for the sake of simplicity. Usually one would use the distribution of the characters over many documents or a general estimated distribution of the characters in German texts. For optimal coding, the probability of \\(1\\) and \\(0\\) is exactly \\(0.5\\).<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Cross-entropy\"><\/span>Cross entropy<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>We talked about the distribution of the character frequencies in a language. Each language has its own distribution of characters and a corresponding optimal code. However, the code for the English frequency distribution will not match the German frequency distribution. The <i>cross entropy<\/i> is the expected number of bits per message needed to encode events drawn from a true distribution \\(q\\), if using a code for distribution\u00a0\\(p\\)\u00a0rather than \\(q\\):<\/p>\n<p style=\"text-align: center\">\\(H_X (p,q) \u2254E_q (-\\log p(x)) = -\\sum_{x \\in X} q(x) \\log p(x).\\)<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Kullback-Leibler-Divergence\"><\/span>Kullback-Leibler-Divergence<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>The <i>relative entropy<\/i> or <i>Kullback-Leibler-Divergence<\/i> is the expected number of <b>extra bits<\/b> per message needed to encode events drawn from true distribution \\(q\\), if using a code for distribution \\(p\\) rather than \\(q\\):<\/p>\n<p style=\"text-align: center\">\\(KL(p,q) \u2254-\\sum_{x \\in X} q(x) \\log \\frac{p(x)}{q(x)}.\\)<\/p>\n<p>Note that \\(KL(p,q)\\) asks for <em>extra bits<\/em>, while \\(H_X(p,q)\\) asks for <em>total bits<\/em>. It is \\(H_X(p,q)=H_X(q)+KL(p,q).\\)<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Conditional-Entropy\"><\/span>Conditional Entropy<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Finally the question arises, what can we say about the entropy of an event \\(X\\), if we already know the previous event \\(Y\\). In other words, conditional entropy measures the uncertainty of a message generated by source \\(X\\) after we know that event \\(Y\\) has generated another message. If \\(Y\\) can functionally determine the value of\u00a0\\(X\\), it is worthwhile that the conditional entropy \\(H_{X|Y}\\) is\u00a0\\(0\\). On the other hand\u00a0\\(H_{X|Y}\\) is \\(H_X\\), if the events\u00a0\\(X\\) and\u00a0\\(Y\\) are stochastically independent. Let&#8217;s suppose we know the conditional probabilities \\(p(x|y)\\),<\/p>\n<p><a href=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Bedingte-Wahrscheinlichkeiten.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-16225\" src=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Bedingte-Wahrscheinlichkeiten.png\" alt=\"Tables showing conditional probability values for y = 1 and y = 0\" width=\"382\" height=\"126\" srcset=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Bedingte-Wahrscheinlichkeiten.png 806w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Bedingte-Wahrscheinlichkeiten-300x99.png 300w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Bedingte-Wahrscheinlichkeiten-768x253.png 768w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Bedingte-Wahrscheinlichkeiten-400x132.png 400w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/Bedingte-Wahrscheinlichkeiten-360x119.png 360w\" sizes=\"auto, (max-width: 382px) 100vw, 382px\" \/><\/a><\/p>\n<p>then the conditional entropy is given by the entropy of the conditional probabilities, weighted by the probabilities of the previous event\u00a0\\(Y\\):<\/p>\n<p><a href=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/cEntropie.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-16226\" src=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/cEntropie.png\" alt=\"The conditional entropy equation explained\" width=\"469\" height=\"109\" \/><\/a><\/p>\n<p>In formula it is \\(\u0397_{X|Y} \u2254-\\sum_{y \\in Y} p(y) \\sum_{x \\in X} p(x|y) \\log p(x|y)\\). One can show that \\(H_{X|Y}=H_{X,Y}-H_Y\\). Assuming that the combined system determined by two messages \\(X\\) and\u00a0\\(Y\\) has a joint entropy\u00a0\\(H_{X,Y}\\), the total uncertainty is reduced if we know the event before.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"More-Entropies\"><\/span>More Entropies<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Of course, there are numerous extensions and modifications of entropies that can\u2019t all be dealt with in this article. In addition to information theory and statistics, entropy is also used in thermodynamics and fluid dynamics, where it does not refer to data, but to the distribution of temperature, gas or liquid mixtures. Look at a glass of KiBa (German abbreviation for cherry-banana juice). We all know the moment when the waiter\/waitress brings a glass to the table\u2014the lower half of yellow banana juice, exactly separated from the top with red cherry juice. At home, when we mix a glass of KiBa ourselves, it resembles a brown mixture. The waiter&#8217;s\/waitress\u2019 glass has a low entropy, a high degree of <em>homogeneity<\/em> or <em>purity<\/em>. We can say with great certainty that there is a banana particle next to another banana particle. Our glass at home, however, has a high degree of <em>inhomogeneity<\/em> or <em>impurity<\/em>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Power-of-Entropy-in-Machine-Learning\"><\/span>Power of Entropy in Machine Learning<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Having discussed the theoretical concept of entropy in information theory, statistics, and physics, entropy is used successfully in real-world applications in machine learning.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Classification-with-Logistic-Regression-or-Neural-Networks\"><\/span>Classification with Logistic Regression or Neural Networks<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Cross entropy is the standard loss function for logistic regression and neural networks. Let\u00a0\\(p\\) be the true distribution (i.e. the distribution of the training set), and let\u00a0\\(p\u0302\\) be the distribution described by a model. For a classification model modeled by a binary logistic regression model, we have class \\(0\\) and class \\(1\\). The model assigns to each input\u00a0\\(x\\) the probability that the output\u00a0\\(y\\) is in class \\(1\\), written as: \\(p\u0302(x)=P(y=1|x)= \u0177\\). It is \\(P(y=0 | x)= 1-\u0177\\). Logistic regression learns by minimizing a loss function. Intuitively, we want to assign more penalty if \\(1\\) is predicted, while the true value is \\(0\\) and if \\(0\\) is predicted, while the true value is \\(1\\). We achieve this by the loss function:<\/p>\n<p>$$ Loss(y,\u0177)=\\begin{cases} -\\log(\u0177) &amp; \\textrm{ if } y=1 \\\\ -\\log(1-\u0177) &amp; \\textrm{ if } y=0 \\end{cases} $$<\/p>\n<p>Summation over all n training data samples yields to<\/p>\n<p>$$ -\\frac{1}{n} \\sum_{j=1}^{n} \u0177_j \\log(\u0177_j)+(1-y_j) \\log(1-\u0177_j).$$<\/p>\n<p>We get by definition the cross entropy between empirical and estimated distribution<\/p>\n<p>$$ -\\frac{1}{n} \\sum_{j=1}^{n} \\sum_{i=1}^{n} p_i \\log(p\u0302_i) + (1-p_i) \\log (1-p\u0302_i)=H(p,p\u0302).$$<\/p>\n<p>Artificial neural networks are closely related to logistic regression. Basically, logistic regression is a single-layer neural network with the sigmoid function as activation function. One of the greatest features of logistic regression is that the logistic loss function is convex, so we are guaranteed to find the global minimum. Therefore, the cross entropy is preferable to other loss functions.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Decision-Trees\"><\/span>Decision Trees<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Decision trees are classification or regression models in the form of a tree structure. They decompose a dataset into smaller and smaller subsets, while simultaneously developing an associated decision tree incrementally. For example, the popular <a href=\"http:\/\/hunch.net\/~coms-4771\/quinlan.pdf\">ID3 algorithm<\/a> from 1985 minimizes the information gain \\(H_{X}-H_{X|Y}\\) to calculate the homogeneity of a dataset. In order to divide the training dataset into homogeneous groups, the minimum is achieved in stepwise splits, if all observations belong exclusively in one of the two classes. Decision trees in machine learning are often dismissed as \u2018old AI\u2019, yet they still make a significant contribution to the success of innovative algorithms. On the platform <a href=\"https:\/\/www.kaggle.com\/\">Kaggle<\/a>, owned by Google LLC, Data Scientists, Machine Learning Engineers and other interested scientists meet for competitions on real-world data sets to face the challenges of artificial intelligence and to build best algorithms. Many winners\u2019 solutions comprise <a href=\"https:\/\/www.kaggle.com\/dansbecker\/xgboost\">XGBoost<\/a>, an implementation of the <b>Gradient Boosted Decision Trees<\/b> algorithm. Gradient boosting generates a predictive model in the form of an ensemble of weak predictive models, typically decision trees. Like other boosting methods, it builds up the model step by step and generalizes it by enabling the optimization of a differentiable loss function. For the reasons mentioned above, cross entropy is a good choice as loss function.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Achieving-a-Conceptual-Understanding-of-the-Data\"><\/span>Achieving a Conceptual Understanding of the Data<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>As we have seen, many machine learning and deep learning models successfully use different kinds of entropies as part of their <i>loss functions<\/i> to train their parameters. We do not need to talk about deep neural networks being able to do amazing things. They do! But the word \u2018deep\u2019 in deep learning refers to a technical, architectural property rather than a conceptual one. Deep neural networks are usually quite complex and contain a variety of parameters that can be optimized across multiple levels of interaction. The problem with such black boxes is the <a href=\"https:\/\/www.inovex.de\/blog\/machine-learning-interpretability\/\">lack of trust and interpretability<\/a> my colleague Marcel Spitzer deals with in another blog post. One can not explain the decision of the model by merely consider its internal elements.<\/p>\n<p>In particular deep neural networks learn a <i>representation<\/i> of their input data in the hidden layers, e.g. via <i>activation functions<\/i>. Each (hidden) layer have an (own) activation function as the identity, sigmoid function, tanh, <a href=\"https:\/\/dl.acm.org\/citation.cfm?id=3104322.3104425\">ReLU<\/a>, Softmax-function or many others. They can be linear or non-linear. Non-linear activation functions make the network especially powerful. This representation is then subsequently used for classification or regression in the output layer. If you want to learn <a href=\"https:\/\/www.inovex.de\/blog\/artificial-neural-networks-concepts-methods\/\">more about deep neural networks<\/a>, read the blog post of my colleague Patrick Schulte.<\/p>\n<p>Activation functions transform the data, but often don\u2019t take into account the intrinsic structures or complexity of the data. Here&#8217;s the chance for entropy in <i>representation learning<\/i>, representing the structure, complexity and inner nature of data. In particular, for sequential data, such as time series or natural language, entropy can be a valuable feature.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Read-on\"><\/span>Read on<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Take a look at our <a href=\"https:\/\/www.inovex.de\/en\/our-services\/data-science-deep-learning\/\">data science portfolio<\/a> and many exciting <a href=\"https:\/\/www.inovex.de\/de\/referenzen\/\">case studies<\/a> from our <a href=\"https:\/\/www.inovex.de\/de\/leistungen\/machine-perception-artificial-intelligence\/\">AI consulting projects<\/a>, where we implement machine learning models scalable and maintainable to daily business.<\/p>\n<p>If you are looking for new challenges, you might also want to consider our <a href=\"https:\/\/www.inovex.de\/de\/karriere\/stellenangebote\/\">job offerings<\/a> for Data Scientists, ML Engineers or BI Developers. For students, we also offer internships, supervision of bachelor\u2019s and master&#8217;s theses or working student positions on exciting and challenging topics.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>If you are dealing with Statistics, Data Science, Machine Learning, Artificial Intelligence or even general Computer Science, Mathematics, Engineering or Physics, you&#8217;ve probably come across the term Entropy more than once. Entropy is a significant, widely used and above all successful measure for quantifying inhomogeneity, impurity, complexity and uncertainty or unpredictability. The concept of entropy [&hellip;]<\/p>\n","protected":false},"author":105,"featured_media":16238,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"ep_exclude_from_search":false,"footnotes":""},"tags":[509,211],"service":[76],"coauthors":[{"id":105,"display_name":"Marisa Mohr","user_nicename":"mmohr"}],"class_list":["post-15502","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","tag-ai-2","tag-inference-uncertainty-quantification","service-artificial-intelligence"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.6 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>The Mystery of Entropy: Measuring Unpredictability in Machine Learning<\/title>\n<meta name=\"description\" content=\"Entropy is a significant, widely used and above all successful measure for quantifying eg. inhomogeneity, uncertainty or unpredictability. It is an integral part of the latest machine learning models deployed on real-world data sets. In this article, I want to highlight the simplicity, beauty and meaning of entropy.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"The Mystery of Entropy: Measuring Unpredictability in Machine Learning\" \/>\n<meta property=\"og:description\" content=\"Entropy is a significant, widely used and above all successful measure for quantifying eg. inhomogeneity, uncertainty or unpredictability. It is an integral part of the latest machine learning models deployed on real-world data sets. In this article, I want to highlight the simplicity, beauty and meaning of entropy.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/\" \/>\n<meta property=\"og:site_name\" content=\"inovex GmbH\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/inovexde\" \/>\n<meta property=\"article:published_time\" content=\"2019-05-06T09:44:13+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-03-25T06:51:35+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/entropy-hero.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"1080\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Marisa Mohr\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/entropy-hero-1024x576.png\" \/>\n<meta name=\"twitter:creator\" content=\"@inovexgmbh\" \/>\n<meta name=\"twitter:site\" content=\"@inovexgmbh\" \/>\n<meta name=\"twitter:label1\" content=\"Verfasst von\" \/>\n\t<meta name=\"twitter:data1\" content=\"Marisa Mohr\" \/>\n\t<meta name=\"twitter:label2\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data2\" content=\"13\u00a0Minuten\" \/>\n\t<meta name=\"twitter:label3\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data3\" content=\"Marisa Mohr\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\\\/\"},\"author\":{\"name\":\"Marisa Mohr\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/person\\\/c2cd3dabbac5b96a8d1c1eff2c591e56\"},\"headline\":\"The Mystery of Entropy: How to Measure Unpredictability in Machine Learning\",\"datePublished\":\"2019-05-06T09:44:13+00:00\",\"dateModified\":\"2025-03-25T06:51:35+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\\\/\"},\"wordCount\":2187,\"commentCount\":1,\"publisher\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2019\\\/05\\\/entropy-hero.png\",\"keywords\":[\"Ai\",\"Inference &amp; Uncertainty Quantification\"],\"articleSection\":[\"Analytics\",\"English Content\",\"General\"],\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\\\/\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\\\/\",\"name\":\"The Mystery of Entropy: Measuring Unpredictability in Machine Learning\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2019\\\/05\\\/entropy-hero.png\",\"datePublished\":\"2019-05-06T09:44:13+00:00\",\"dateModified\":\"2025-03-25T06:51:35+00:00\",\"description\":\"Entropy is a significant, widely used and above all successful measure for quantifying eg. inhomogeneity, uncertainty or unpredictability. It is an integral part of the latest machine learning models deployed on real-world data sets. In this article, I want to highlight the simplicity, beauty and meaning of entropy.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\\\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2019\\\/05\\\/entropy-hero.png\",\"contentUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2019\\\/05\\\/entropy-hero.png\",\"width\":1920,\"height\":1080,\"caption\":\"Stylised glasses of cherry banana juice representing entropy\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"The Mystery of Entropy: How to Measure Unpredictability in Machine Learning\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#website\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\",\"name\":\"inovex GmbH\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\",\"name\":\"inovex GmbH\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/inovex-logo-16-9-1.png\",\"contentUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/inovex-logo-16-9-1.png\",\"width\":1921,\"height\":1081,\"caption\":\"inovex GmbH\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/inovexde\",\"https:\\\/\\\/x.com\\\/inovexgmbh\",\"https:\\\/\\\/www.instagram.com\\\/inovexlife\\\/\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/inovex\",\"https:\\\/\\\/www.youtube.com\\\/channel\\\/UC7r66GT14hROB_RQsQBAQUQ\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/person\\\/c2cd3dabbac5b96a8d1c1eff2c591e56\",\"name\":\"Marisa Mohr\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/cropped-056_DSC4559_INOVEX_HH_5000px_300dpi-scaled-1-96x96.jpg8da2e080f248893419516f226226bfaa\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/cropped-056_DSC4559_INOVEX_HH_5000px_300dpi-scaled-1-96x96.jpg\",\"contentUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/cropped-056_DSC4559_INOVEX_HH_5000px_300dpi-scaled-1-96x96.jpg\",\"caption\":\"Marisa Mohr\"},\"description\":\"Math in AI or everywhere else | Demystification | Girls for STEM\",\"sameAs\":[\"https:\\\/\\\/marisa-mohr.de\\\/\",\"https:\\\/\\\/www.linkedin.com\\\/in\\\/marisa-mohr-17342b108\\\/\"],\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/author\\\/mmohr\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"The Mystery of Entropy: Measuring Unpredictability in Machine Learning","description":"Entropy is a significant, widely used and above all successful measure for quantifying eg. inhomogeneity, uncertainty or unpredictability. It is an integral part of the latest machine learning models deployed on real-world data sets. In this article, I want to highlight the simplicity, beauty and meaning of entropy.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/","og_locale":"de_DE","og_type":"article","og_title":"The Mystery of Entropy: Measuring Unpredictability in Machine Learning","og_description":"Entropy is a significant, widely used and above all successful measure for quantifying eg. inhomogeneity, uncertainty or unpredictability. It is an integral part of the latest machine learning models deployed on real-world data sets. In this article, I want to highlight the simplicity, beauty and meaning of entropy.","og_url":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/","og_site_name":"inovex GmbH","article_publisher":"https:\/\/www.facebook.com\/inovexde","article_published_time":"2019-05-06T09:44:13+00:00","article_modified_time":"2025-03-25T06:51:35+00:00","og_image":[{"width":1920,"height":1080,"url":"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/entropy-hero.png","type":"image\/png"}],"author":"Marisa Mohr","twitter_card":"summary_large_image","twitter_image":"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/entropy-hero-1024x576.png","twitter_creator":"@inovexgmbh","twitter_site":"@inovexgmbh","twitter_misc":{"Verfasst von":"Marisa Mohr","Gesch\u00e4tzte Lesezeit":"13\u00a0Minuten","Written by":"Marisa Mohr"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#article","isPartOf":{"@id":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/"},"author":{"name":"Marisa Mohr","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/person\/c2cd3dabbac5b96a8d1c1eff2c591e56"},"headline":"The Mystery of Entropy: How to Measure Unpredictability in Machine Learning","datePublished":"2019-05-06T09:44:13+00:00","dateModified":"2025-03-25T06:51:35+00:00","mainEntityOfPage":{"@id":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/"},"wordCount":2187,"commentCount":1,"publisher":{"@id":"https:\/\/www.inovex.de\/de\/#organization"},"image":{"@id":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#primaryimage"},"thumbnailUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/entropy-hero.png","keywords":["Ai","Inference &amp; Uncertainty Quantification"],"articleSection":["Analytics","English Content","General"],"inLanguage":"de","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/","url":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/","name":"The Mystery of Entropy: Measuring Unpredictability in Machine Learning","isPartOf":{"@id":"https:\/\/www.inovex.de\/de\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#primaryimage"},"image":{"@id":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#primaryimage"},"thumbnailUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/entropy-hero.png","datePublished":"2019-05-06T09:44:13+00:00","dateModified":"2025-03-25T06:51:35+00:00","description":"Entropy is a significant, widely used and above all successful measure for quantifying eg. inhomogeneity, uncertainty or unpredictability. It is an integral part of the latest machine learning models deployed on real-world data sets. In this article, I want to highlight the simplicity, beauty and meaning of entropy.","breadcrumb":{"@id":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/"]}]},{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#primaryimage","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/entropy-hero.png","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/05\/entropy-hero.png","width":1920,"height":1080,"caption":"Stylised glasses of cherry banana juice representing entropy"},{"@type":"BreadcrumbList","@id":"https:\/\/www.inovex.de\/de\/blog\/the-mystery-of-entropy-how-to-measure-unpredictability-in-machine-learning\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.inovex.de\/de\/"},{"@type":"ListItem","position":2,"name":"The Mystery of Entropy: How to Measure Unpredictability in Machine Learning"}]},{"@type":"WebSite","@id":"https:\/\/www.inovex.de\/de\/#website","url":"https:\/\/www.inovex.de\/de\/","name":"inovex GmbH","description":"","publisher":{"@id":"https:\/\/www.inovex.de\/de\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.inovex.de\/de\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Organization","@id":"https:\/\/www.inovex.de\/de\/#organization","name":"inovex GmbH","url":"https:\/\/www.inovex.de\/de\/","logo":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png","width":1921,"height":1081,"caption":"inovex GmbH"},"image":{"@id":"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/inovexde","https:\/\/x.com\/inovexgmbh","https:\/\/www.instagram.com\/inovexlife\/","https:\/\/www.linkedin.com\/company\/inovex","https:\/\/www.youtube.com\/channel\/UC7r66GT14hROB_RQsQBAQUQ"]},{"@type":"Person","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/person\/c2cd3dabbac5b96a8d1c1eff2c591e56","name":"Marisa Mohr","image":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/wp-content\/uploads\/cropped-056_DSC4559_INOVEX_HH_5000px_300dpi-scaled-1-96x96.jpg8da2e080f248893419516f226226bfaa","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/cropped-056_DSC4559_INOVEX_HH_5000px_300dpi-scaled-1-96x96.jpg","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/cropped-056_DSC4559_INOVEX_HH_5000px_300dpi-scaled-1-96x96.jpg","caption":"Marisa Mohr"},"description":"Math in AI or everywhere else | Demystification | Girls for STEM","sameAs":["https:\/\/marisa-mohr.de\/","https:\/\/www.linkedin.com\/in\/marisa-mohr-17342b108\/"],"url":"https:\/\/www.inovex.de\/de\/blog\/author\/mmohr\/"}]}},"_links":{"self":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/15502","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/users\/105"}],"replies":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/comments?post=15502"}],"version-history":[{"count":5,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/15502\/revisions"}],"predecessor-version":[{"id":61474,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/15502\/revisions\/61474"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/media\/16238"}],"wp:attachment":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/media?parent=15502"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/tags?post=15502"},{"taxonomy":"service","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/service?post=15502"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/coauthors?post=15502"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}