{"id":15672,"date":"2019-05-27T09:19:36","date_gmt":"2019-05-27T07:19:36","guid":{"rendered":"https:\/\/www.inovex.de\/blog\/?p=15672"},"modified":"2022-11-22T10:19:29","modified_gmt":"2022-11-22T09:19:29","slug":"lime-machine-learning-interpretability","status":"publish","type":"post","link":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/","title":{"rendered":"Machine Learning Interpretability: Explaining Blackbox Models with LIME (Part II)"},"content":{"rendered":"<p>This is the second part of our series about Machine Learning interpretability. We want to describe LIME (<strong>L<\/strong>ocal <strong>I<\/strong>nterpretable <strong>M<\/strong>odel-Agnostic <strong>E<\/strong>xplanations), a popular technique to explain blackbox models. It was proposed by Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin in their paper <a href=\"https:\/\/arxiv.org\/abs\/1602.04938\">Why Should I Trust You? Explaining the Predictions of Any Classifier<\/a>, which they first presented at the ACM&#8217;s Conference on Knowledge Discovery and Data Mining in 2016. Please check out our <a href=\"https:\/\/www.inovex.de\/blog\/machine-learning-interpretability\">previous article<\/a>\u00a0if you are not familiar with the concept of interpretability.<\/p>\n<p>We previously made a distinction between model-specific and model-agnostic techniques as well as between global and local techniques. LIME can be classified as a model-agnostic technique with a local scope. In other words, it enables us to explain particular predictions of any model. In contrast to the techniques described in the first article, LIME is even applicable to models for text and image classification.<!--more--><\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_83 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\"><p class=\"ez-toc-title\" style=\"cursor:inherit\"><\/p>\n<\/div><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#What-is-LIME\" >What is LIME?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#How-to-use-LIME\" >How to use LIME?<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#Image-recognition\" >Image recognition<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#Tabular-data\" >Tabular data<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#Conclusion\" >Conclusion<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#Read-on\" >Read on<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"What-is-LIME\"><\/span>What is LIME?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>The idea behind LIME is to approximate a complex model locally by an interpretable model and to use that simple model to explain a prediction of a particular instance of interest. It&#8217;s comparable to the Global Surrogate technique, but it differs in the fact that it is based on sampled instances that are weighted by proximity to the instance to be explained. Hence, instead of trying to capture the overall behavior of the model, LIME just attempts to be\u00a0<strong>locally faithful<\/strong>\u00a0to the classifier. The following image shows the non-linear decision boundary of some complex classifier. LIME fitted a linear model, represented by the dashed decision boundary, to sampled instances. Notice that this simple model mimics the complex model in the vicinity of the instance of interest sufficiently well.<\/p>\n<figure style=\"width: 553px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/raw.githubusercontent.com\/marcotcr\/lime\/master\/doc\/images\/lime.png\"><img loading=\"lazy\" decoding=\"async\" class=\"size-medium\" src=\"https:\/\/raw.githubusercontent.com\/marcotcr\/lime\/master\/doc\/images\/lime.png\" alt=\"Local Faithfulness of LIME\" width=\"553\" height=\"344\" \/><\/a><figcaption class=\"wp-caption-text\">The red and blue areas are separated by the decision boundary of a complex, non-linear model. LIME determined a linear model based on weighted data points to explain the complex models&#8216; prediction for the highlighted instance. (source: https:\/\/arxiv.org\/abs\/1602.04938)<\/figcaption><\/figure>\n<p>Unlike other interpretability techniques, explanations that are generated by LIME are based on so-called\u00a0<strong>interpretable components<\/strong>\u00a0which can completely differ from the input features of the original model. Basically, interpretable components are representations of the underlying data which are understandable to humans. For example, an interpretable component can be a subset of words of a text corpus or a contiguous region of an image. The concept of interpretable components enables LIME to be generalizable to high dimensional domains like text or image classification.<\/p>\n<p>Whatever type of model you try to explain with LIME, you have to think about appropriate ways to determine interpretable components. Common approaches are the bag-of-words model for text classification or pixel segmentation for image classification. Based on that mapping, LIME acts on binary vectors that indicate\u00a0<strong>presence or absence<\/strong>\u00a0of interpretable components. Apart from that, the original model can be based on much more complex features. It estimates strength and direction of influence for each interpretable component by fitting interpretable models to perturbed instances of those binary vectors. LIME evaluates how perturbations (e.g. removing words of a text, hiding parts of an image) affects the prediction of the original model.<\/p>\n<figure style=\"width: 1400px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"size-medium\" src=\"https:\/\/d3ansictanv2wj.cloudfront.net\/figure3-2cea505fe733a4713eeff3b90f696507.jpg\" alt=\"Intrepretable Components of an image\" width=\"1400\" height=\"672\" \/><figcaption class=\"wp-caption-text\">LIME determined contiguous regions of the\u00a0original image by pixel segmentation (source: https:\/\/arxiv.org\/abs\/1602.04938)<\/figcaption><\/figure>\n<p>Technically, any interpretable model can be used within the LIME framework. When fitting interpretable models, LIME attempts to balance simplicity and local fidelity. Therefore, a measure of complexity and some locality-aware loss is needed. In most practical cases, sparse linear models, especially <a href=\"https:\/\/en.wikipedia.org\/wiki\/Lasso_(statistics)\">The LASSO<\/a>, perform sufficiently well. The complexity of LASSO models can be measured by the number of non-zero weights. A squared exponential kernel is usually used as a measure of local accuracy.<\/p>\n<p>Now, let&#8217;s put it all together to describe each step of the LIME procedure:<\/p>\n<ol>\n<li>Determine an interpretable representation of the instance of interest<\/li>\n<li>Draw a sample by disturbing the interpretable representation<\/li>\n<li>Apply the original model to the perturbed instances<\/li>\n<li>Fit an interpretable model to proximity-weighted sampled instances and the predictions of the original model<\/li>\n<li>Use the interpretable model to draw conclusions about the relevance of each interpretable component<\/li>\n<\/ol>\n<figure style=\"width: 1400px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" class=\"size-medium\" src=\"https:\/\/d3ansictanv2wj.cloudfront.net\/figure4-99d9ea184dd35876e0dbae81f6fce038.jpg\" alt=\"Appliation of LIME to image classification task\" width=\"1400\" height=\"707\" \/><figcaption class=\"wp-caption-text\">LIME applies a complex model to perturbed instances of the original image and fits an interpretable model to the outcome in order to estimate the importance of each interpretable component. (source: https:\/\/arxiv.org\/abs\/1602.04938)<\/figcaption><\/figure>\n<h2><span class=\"ez-toc-section\" id=\"How-to-use-LIME\"><\/span>How to use LIME?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Enough has been said about the theory behind LIME. Let&#8217;s finally use it to explain some predictions. We&#8217;ll start with an image classifier:<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Image-recognition\"><\/span>Image recognition<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Let&#8217;s import all relevant libraries and a pre-trained <a href=\"https:\/\/arxiv.org\/abs\/1512.00567\">Inception-V3<\/a>\u00a0for image classification. Inception-V3 is a deep neural network proposed by Szegedy, et al. (Google) which is composed of 42 layers of roughly 7 million parameters. As described in this <a href=\"https:\/\/marcotcr.github.io\/lime\/tutorials\/Tutorial%20-%20images.html\">LIME tutorial<\/a>, we cloned the fork of <a href=\"https:\/\/github.com\/marcotcr\/tf-models\">tf-slim<\/a> and put the\u00a0<a href=\"http:\/\/download.tensorflow.org\/models\/inception_v3_2016_08_28.tar.gz\">pre-trained Inception-V3<\/a>\u00a0into\u00a0<code>tf-models\/slim\/pretrained<\/code>.<\/p>\n<pre class=\"lang:python decode:true\">import tensorflow as tf\r\n\r\nimport sys\r\n\r\nimport matplotlib.pyplot as plt\r\n\r\nimport numpy as np\r\n\r\nimport os\r\n\r\nimport random\r\n\r\nsys.path.append('tf-models\/slim')\r\n\r\nslim = tf.contrib.slim\r\n\r\nsession = tf.Session()\r\n\r\nfrom nets import inception\r\n\r\nfrom preprocessing import inception_preprocessing\r\n\r\nfrom datasets import imagenet\r\n\r\nfrom lime import lime_image\r\n\r\nfrom skimage.segmentation import mark_boundaries\r\n\r\nrandom.seed(4711)\r\n\r\n%load_ext autoreload\r\n\r\n%autoreload 2\r\n\r\n%matplotlib inline<\/pre>\n<p>In addition, let&#8217;s define some functions like pre-process and predict:<\/p>\n<pre class=\"lang:python decode:true\">image_size = inception.inception_v3.default_image_size\r\n\r\ndef load_image(path):\r\n\r\n    image_raw = tf.image.decode_jpeg(open(path, 'rb').read(), channels=3)\r\n\r\n    image = inception_preprocessing.preprocess_image(image_raw, image_size, image_size, is_training=False)\r\n\r\n    return session.run([[image]])[0][0]\r\n\r\nnames = imagenet.create_readable_names_for_imagenet_labels()\r\n\r\nprocessed_images = tf.placeholder(tf.float32, shape=(None, 299, 299, 3))\r\n\r\nwith slim.arg_scope(inception.inception_v3_arg_scope()):\r\n\r\n    logits, _ = inception.inception_v3(processed_images, num_classes=1001, is_training=False)\r\n\r\nprobabilities = tf.nn.softmax(logits)\r\n\r\ncheckpoints_dir = 'tf-models\/slim\/pretrained'\r\n\r\ninit_fn = slim.assign_from_checkpoint_fn(\r\n\r\n    os.path.join(checkpoints_dir, 'inception_v3.ckpt'),\r\n\r\n    slim.get_model_variables('InceptionV3'))\r\n\r\ninit_fn(session)\r\n\r\ndef predict_fn(images):\r\n\r\n return session.run(probabilities, feed_dict={processed_images: images})<\/pre>\n<p>Now to the image recognition task: We want the classifier to recognize the squirrel in the following image. Let&#8217;s see if Inception-V3 is able to spot it:<\/p>\n<pre class=\"lang:python decode:true\">squirrel_image = load_image(\"img\/squirrel.png\")\r\n\r\nplt.imshow(squirrel_image \/ 2 + 0.5)\r\n\r\npreds = predict_fn([squirrel_image])\r\n\r\nfor x in (-preds).argsort()[0][:3]:\r\n\r\n    print(x, names[x], preds[0,x])<\/pre>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-15699\" src=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/squirrel_prediction-300x180.png\" alt=\"squirrel prediction\" width=\"500\" height=\"300\" srcset=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/squirrel_prediction-300x180.png 300w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/squirrel_prediction-1024x614.png 1024w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/squirrel_prediction-768x461.png 768w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/squirrel_prediction-400x240.png 400w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/squirrel_prediction-360x216.png 360w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/squirrel_prediction.png 1114w\" sizes=\"auto, (max-width: 500px) 100vw, 500px\" \/><\/p>\n<p>Actually, the Inception net is quite sure that there is a fox squirrel in the image. Pretty impressive!<\/p>\n<p>Now, that we observed the outcome of our complex classifier, let&#8217;s apply LIME to identify parts of the image having most influence to the squirrel prediction. To do this, we feed an explainer with the squirrel image and provide it the number of patches to be determined (num_features) along with the number of perturbations of the interpretable representation (num_samples):<\/p>\n<pre class=\"lang:python decode:true\">explainer = lime_image.LimeImageExplainer()\r\n\r\nexplanation = explainer.explain_instance(\r\n\r\n    image=squirrel_image,\r\n\r\n    classifier_fn=predict_fn,\r\n\r\n    top_labels=1,\r\n\r\n    hide_color=0,\r\n\r\n    num_features=20,\r\n\r\n    num_samples=500)<\/pre>\n<p>Ok, let&#8217;s see which parts of the image accounted most to the squirrel prediction:<\/p>\n<pre class=\"lang:python decode:true\">masked_image, boundaries = explanation.get_image_and_mask(\r\n\r\n    label=336,\r\n\r\n    positive_only=True,\r\n\r\n    num_features=6,\r\n\r\n    hide_rest=True)\r\n\r\nplt.imshow(mark_boundaries(masked_image \/ 2 + 0.5, boundaries))<\/pre>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-15700\" src=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/squirrel_explanation-300x287.png\" alt=\"squirrel explanation\" width=\"300\" height=\"287\" \/><\/p>\n<p>It was the squirrel&#8217;s face and its bushy tail. Both, quite reasonable causes for the squirrel prediction. Well done, Inception!<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Tabular-data\"><\/span>Tabular data<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>LIME is not only applicable to image data. We can also use it to explain predictions that are based on tabular data.\u00a0As biology teaches us, squirrels are mammals. Let&#8217;s find out whether a neural net classifier is able learn and predict this, too. We therefore train a multi-layer perceptron on the <a href=\"https:\/\/archive.ics.uci.edu\/ml\/datasets\/Zoo\">Zoo Data Set<\/a>, a small tabular dataset in which various species are categorized into seven classes based on 16 different features:<\/p>\n<pre class=\"lang:python decode:true\" title=\"Train neural net classifier\">import pandas as pd\r\n\r\nfrom sklearn.model_selection import train_test_split\r\n\r\nfrom sklearn.neural_network import MLPClassifier\r\n\r\nfrom lime import lime_tabular\r\n\r\nclasses = [\"mammal\", \"bird\", \"reptile\", \"fish\", \"amphibian\", \"bug\", \"invertebrate\"]\r\n\r\ndf = pd.read_csv(\"zoo.data\", header=0, sep=\",\")\r\n\r\nsquirrel = df[df.name == \"squirrel\"]\r\n\r\ndf = df[df.name != \"squirrel\"]\r\n\r\nX_train, X_test, Y_train, Y_test = train_test_split(df.drop([\"type\"], axis=1), df.type)\r\n\r\nX_test = pd.concat([X_test, squirrel.drop([\"type\"], axis=1)])\r\n\r\nY_test = pd.concat([Y_test, squirrel.type])\r\n\r\nX_test_names = X_test.name\r\n\r\nX_train.drop(\"name\", axis=1, inplace=True)\r\n\r\nX_test.drop(\"name\", axis=1, inplace=True)\r\n\r\nclf = MLPClassifier(alpha=0.25, hidden_layer_sizes=(25,)).fit(X_train, Y_train)<\/pre>\n<p>We imported all necessary packages, loaded the dataset and did some preprocessing on the data such as removing the observation for the squirrel from the train dataset. Finally, we trained the neural net on 75% of the original dataset.<\/p>\n<p>Now, let&#8217;s see which class the classifier assigns to the squirrel:<\/p>\n<pre class=\"lang:python decode:true\" title=\"Squirrel prediciton\">print(pd.DataFrame(clf.predict_proba(squirrel_features), columns=classes).transpose())<\/pre>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-16268\" src=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/04\/Bildschirmfoto-von-2019-05-20-14-22-42-300x194.png\" alt=\"\" width=\"175\" height=\"113\" \/><\/p>\n<p>As we can see, the classifier assigns the highest probability of 88.1% to the\u00a0<em>mammal<\/em>\u00a0class, which is actually correct. Nice!<\/p>\n<p>But why does the model decide upon the\u00a0<em>mammal<\/em>\u00a0class? Since the neural network is a complex, non-linear model, it is difficult for us as humans to understand the model&#8217;s decision process. Yet, we can leverage LIME to support us in comprehending its decision. Let&#8217;s have a look at the LIME estimate of relevant features for the squirrel prediction:<\/p>\n<pre class=\"lang:python decode:true\" title=\"LIME explanation for mammal\">explainer = lime_tabular.LimeTabularExplainer(\r\n\r\n    X_train.values, categorical_features=[0,1,2,3,4,5,6,7,8,9,10,11,13,14,15],\r\n\r\n    feature_names=X_train.columns,\r\n\r\n    class_names=classes,\r\n\r\n    discretize_continuous=True\r\n\r\n)\r\n\r\nexp = explainer.explain_instance(\r\n\r\n    squirrel_features.values[0],\r\n\r\n    clf.predict_proba,\r\n\r\n    num_features=16,\r\n\r\n    top_labels=1\r\n\r\n)\r\n\r\nexp.show_in_notebook(show_table=True, show_all=False)<\/pre>\n<p>We took the calculated class probabilities and list the impact of all features (16 in total) for the top class. In that particular case, the class is\u00a0<em>mammal<\/em>.<\/p>\n<p>The graph below shows the estimated impact of each feature. Bars starting in the middle and expanding to the left (colored in turquoise) decrease the probability of the instance in question to belong to the class <em>mammal<\/em>. Bars in blue expanding to the right increase the predicted probability that the squirrel is a mammal. We can see that most features had a positive contribution to the probability of the\u00a0<em>mammal <\/em>class.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\" wp-image-16265 aligncenter\" src=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/04\/lime_relevance-1-e1558359408487-169x300.png\" alt=\"\" width=\"280\" height=\"497\" srcset=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/04\/lime_relevance-1-e1558359408487-169x300.png 169w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/04\/lime_relevance-1-e1558359408487-400x711.png 400w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/04\/lime_relevance-1-e1558359408487-360x640.png 360w, https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/04\/lime_relevance-1-e1558359408487.png 498w\" sizes=\"auto, (max-width: 280px) 100vw, 280px\" \/><\/p>\n<p>For example, the fact that a squirrel gives milk to feed its offspring and that it is not venomous are plausible explanations, and it makes sense that these two facts had a positive contribution to the final prediction. Furthermore, the length of a bar indicates the magnitude of the feature&#8217;s influence. Thus,\u00a0giving milk, having fur and not laying eggs are the most influential or rather most discriminative features for a squirrel being a mammal.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Conclusion\"><\/span>Conclusion<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>While most of the techniques that were described in the first blog post are global techniques, LIME is a technique that has a local scope. Therefore, LIME allows us to explain particular predictions of any classifier. The LIME framework is flexible in the sense that any interpretable model can be used to explain predictions. Furthermore, the concept of interpretable components enables LIME to be applicable to high dimensional domains such as image or text classification. Overall, LIME can be used to support model selection and to generate trust by reviewing expressive examples.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Read-on\"><\/span>Read on<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>You might wanna have a look at our <a href=\"https:\/\/www.inovex.de\/en\/our-services\/data-science-deep-learning\/\">deep learning portfolio<\/a>. If you&#8217;re looking for new challenges, you might also want to consider our <a href=\"https:\/\/www.inovex.de\/de\/karriere\/stellenangebote\/\">job offerings<\/a> for Data Scientists, ML Engineers or BI Developer.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This is the second part of our series about Machine Learning interpretability. We want to describe LIME (Local Interpretable Model-Agnostic Explanations), a popular technique to explain blackbox models. It was proposed by Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin in their paper Why Should I Trust You? Explaining the Predictions of Any Classifier, which [&hellip;]<\/p>\n","protected":false},"author":103,"featured_media":15560,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"ep_exclude_from_search":false,"footnotes":""},"tags":[509,206,264],"service":[76,431],"coauthors":[{"id":103,"display_name":"Marcel Spitzer","user_nicename":"mspitzer"},{"id":108,"display_name":"Alexandra W\u00f6rner","user_nicename":"awoerner"}],"class_list":["post-15672","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","tag-ai-2","tag-data-science","tag-ml-interpretability","service-artificial-intelligence","service-data-science"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.6 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Machine Learning Interpretability: Explaining Blackbox Models with LIME<\/title>\n<meta name=\"description\" content=\"The idea behind the model-agnostic technique LIME is to approximate a complex model locally by an interpretable model and to use that simple model to explain a prediction of a particular instance of interest.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Machine Learning Interpretability: Explaining Blackbox Models with LIME\" \/>\n<meta property=\"og:description\" content=\"The idea behind the model-agnostic technique LIME is to approximate a complex model locally by an interpretable model and to use that simple model to explain a prediction of a particular instance of interest.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/\" \/>\n<meta property=\"og:site_name\" content=\"inovex GmbH\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/inovexde\" \/>\n<meta property=\"article:published_time\" content=\"2019-05-27T07:19:36+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-11-22T09:19:29+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/machine-learning-interpretability-hero.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"1080\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Marcel Spitzer, Alexandra W\u00f6rner\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/machine-learning-interpretability-hero-1024x576.png\" \/>\n<meta name=\"twitter:creator\" content=\"@inovexgmbh\" \/>\n<meta name=\"twitter:site\" content=\"@inovexgmbh\" \/>\n<meta name=\"twitter:label1\" content=\"Verfasst von\" \/>\n\t<meta name=\"twitter:data1\" content=\"Marcel Spitzer\" \/>\n\t<meta name=\"twitter:label2\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data2\" content=\"9\u00a0Minuten\" \/>\n\t<meta name=\"twitter:label3\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data3\" content=\"Marcel Spitzer, Alexandra W\u00f6rner\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/lime-machine-learning-interpretability\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/lime-machine-learning-interpretability\\\/\"},\"author\":{\"name\":\"Marcel Spitzer\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/person\\\/5c0bae71c9067840a9ef27f9bf622b54\"},\"headline\":\"Machine Learning Interpretability: Explaining Blackbox Models with LIME (Part II)\",\"datePublished\":\"2019-05-27T07:19:36+00:00\",\"dateModified\":\"2022-11-22T09:19:29+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/lime-machine-learning-interpretability\\\/\"},\"wordCount\":1489,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/lime-machine-learning-interpretability\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2019\\\/02\\\/machine-learning-interpretability-hero.png\",\"keywords\":[\"Ai\",\"Data Science\",\"ML Interpretability\"],\"articleSection\":[\"Analytics\",\"English Content\",\"General\"],\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/lime-machine-learning-interpretability\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/lime-machine-learning-interpretability\\\/\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/lime-machine-learning-interpretability\\\/\",\"name\":\"Machine Learning Interpretability: Explaining Blackbox Models with LIME\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/lime-machine-learning-interpretability\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/lime-machine-learning-interpretability\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2019\\\/02\\\/machine-learning-interpretability-hero.png\",\"datePublished\":\"2019-05-27T07:19:36+00:00\",\"dateModified\":\"2022-11-22T09:19:29+00:00\",\"description\":\"The idea behind the model-agnostic technique LIME is to approximate a complex model locally by an interpretable model and to use that simple model to explain a prediction of a particular instance of interest.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/lime-machine-learning-interpretability\\\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/lime-machine-learning-interpretability\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/lime-machine-learning-interpretability\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2019\\\/02\\\/machine-learning-interpretability-hero.png\",\"contentUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2019\\\/02\\\/machine-learning-interpretability-hero.png\",\"width\":1920,\"height\":1080,\"caption\":\"A black box with a window cut out\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/lime-machine-learning-interpretability\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Machine Learning Interpretability: Explaining Blackbox Models with LIME (Part II)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#website\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\",\"name\":\"inovex GmbH\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\",\"name\":\"inovex GmbH\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/inovex-logo-16-9-1.png\",\"contentUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/inovex-logo-16-9-1.png\",\"width\":1921,\"height\":1081,\"caption\":\"inovex GmbH\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/inovexde\",\"https:\\\/\\\/x.com\\\/inovexgmbh\",\"https:\\\/\\\/www.instagram.com\\\/inovexlife\\\/\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/inovex\",\"https:\\\/\\\/www.youtube.com\\\/channel\\\/UC7r66GT14hROB_RQsQBAQUQ\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/person\\\/5c0bae71c9067840a9ef27f9bf622b54\",\"name\":\"Marcel Spitzer\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/cropped-22291_2-96x96.jpg029da078b4491202841b116405caf549\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/cropped-22291_2-96x96.jpg\",\"contentUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/cropped-22291_2-96x96.jpg\",\"caption\":\"Marcel Spitzer\"},\"description\":\"As a Data\\\/ML Engineer at inovex, I build streaming and batch pipelines for data processing in distributed systems and use machine learning to make data products smarter.\",\"sameAs\":[\"https:\\\/\\\/www.linkedin.com\\\/in\\\/msp14\\\/\"],\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/author\\\/mspitzer\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Machine Learning Interpretability: Explaining Blackbox Models with LIME","description":"The idea behind the model-agnostic technique LIME is to approximate a complex model locally by an interpretable model and to use that simple model to explain a prediction of a particular instance of interest.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/","og_locale":"de_DE","og_type":"article","og_title":"Machine Learning Interpretability: Explaining Blackbox Models with LIME","og_description":"The idea behind the model-agnostic technique LIME is to approximate a complex model locally by an interpretable model and to use that simple model to explain a prediction of a particular instance of interest.","og_url":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/","og_site_name":"inovex GmbH","article_publisher":"https:\/\/www.facebook.com\/inovexde","article_published_time":"2019-05-27T07:19:36+00:00","article_modified_time":"2022-11-22T09:19:29+00:00","og_image":[{"width":1920,"height":1080,"url":"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/machine-learning-interpretability-hero.png","type":"image\/png"}],"author":"Marcel Spitzer, Alexandra W\u00f6rner","twitter_card":"summary_large_image","twitter_image":"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/machine-learning-interpretability-hero-1024x576.png","twitter_creator":"@inovexgmbh","twitter_site":"@inovexgmbh","twitter_misc":{"Verfasst von":"Marcel Spitzer","Gesch\u00e4tzte Lesezeit":"9\u00a0Minuten","Written by":"Marcel Spitzer, Alexandra W\u00f6rner"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#article","isPartOf":{"@id":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/"},"author":{"name":"Marcel Spitzer","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/person\/5c0bae71c9067840a9ef27f9bf622b54"},"headline":"Machine Learning Interpretability: Explaining Blackbox Models with LIME (Part II)","datePublished":"2019-05-27T07:19:36+00:00","dateModified":"2022-11-22T09:19:29+00:00","mainEntityOfPage":{"@id":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/"},"wordCount":1489,"commentCount":0,"publisher":{"@id":"https:\/\/www.inovex.de\/de\/#organization"},"image":{"@id":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#primaryimage"},"thumbnailUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/machine-learning-interpretability-hero.png","keywords":["Ai","Data Science","ML Interpretability"],"articleSection":["Analytics","English Content","General"],"inLanguage":"de","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/","url":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/","name":"Machine Learning Interpretability: Explaining Blackbox Models with LIME","isPartOf":{"@id":"https:\/\/www.inovex.de\/de\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#primaryimage"},"image":{"@id":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#primaryimage"},"thumbnailUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/machine-learning-interpretability-hero.png","datePublished":"2019-05-27T07:19:36+00:00","dateModified":"2022-11-22T09:19:29+00:00","description":"The idea behind the model-agnostic technique LIME is to approximate a complex model locally by an interpretable model and to use that simple model to explain a prediction of a particular instance of interest.","breadcrumb":{"@id":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/"]}]},{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#primaryimage","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/machine-learning-interpretability-hero.png","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2019\/02\/machine-learning-interpretability-hero.png","width":1920,"height":1080,"caption":"A black box with a window cut out"},{"@type":"BreadcrumbList","@id":"https:\/\/www.inovex.de\/de\/blog\/lime-machine-learning-interpretability\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.inovex.de\/de\/"},{"@type":"ListItem","position":2,"name":"Machine Learning Interpretability: Explaining Blackbox Models with LIME (Part II)"}]},{"@type":"WebSite","@id":"https:\/\/www.inovex.de\/de\/#website","url":"https:\/\/www.inovex.de\/de\/","name":"inovex GmbH","description":"","publisher":{"@id":"https:\/\/www.inovex.de\/de\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.inovex.de\/de\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Organization","@id":"https:\/\/www.inovex.de\/de\/#organization","name":"inovex GmbH","url":"https:\/\/www.inovex.de\/de\/","logo":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png","width":1921,"height":1081,"caption":"inovex GmbH"},"image":{"@id":"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/inovexde","https:\/\/x.com\/inovexgmbh","https:\/\/www.instagram.com\/inovexlife\/","https:\/\/www.linkedin.com\/company\/inovex","https:\/\/www.youtube.com\/channel\/UC7r66GT14hROB_RQsQBAQUQ"]},{"@type":"Person","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/person\/5c0bae71c9067840a9ef27f9bf622b54","name":"Marcel Spitzer","image":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/wp-content\/uploads\/cropped-22291_2-96x96.jpg029da078b4491202841b116405caf549","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/cropped-22291_2-96x96.jpg","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/cropped-22291_2-96x96.jpg","caption":"Marcel Spitzer"},"description":"As a Data\/ML Engineer at inovex, I build streaming and batch pipelines for data processing in distributed systems and use machine learning to make data products smarter.","sameAs":["https:\/\/www.linkedin.com\/in\/msp14\/"],"url":"https:\/\/www.inovex.de\/de\/blog\/author\/mspitzer\/"}]}},"_links":{"self":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/15672","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/users\/103"}],"replies":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/comments?post=15672"}],"version-history":[{"count":6,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/15672\/revisions"}],"predecessor-version":[{"id":38787,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/15672\/revisions\/38787"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/media\/15560"}],"wp:attachment":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/media?parent=15672"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/tags?post=15672"},{"taxonomy":"service","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/service?post=15672"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/coauthors?post=15672"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}