{"id":21242,"date":"2022-01-05T14:37:15","date_gmt":"2022-01-05T13:37:15","guid":{"rendered":"https:\/\/www.inovex.de\/blog\/?p=21242"},"modified":"2022-11-21T11:43:32","modified_gmt":"2022-11-21T10:43:32","slug":"knowledge-graphs-explainability-methods-modern-natural-language-processing","status":"publish","type":"post","link":"https:\/\/www.inovex.de\/de\/blog\/knowledge-graphs-explainability-methods-modern-natural-language-processing\/","title":{"rendered":"Combining Knowledge Graphs and Explainability Methods in modern Natural Language Processing"},"content":{"rendered":"

This blog post elaborates on the interface of knowledge graphs and explainability in natural language processing. We briefly introduce the core concepts and recent approaches in the field. Afterwards, we present our own model and an interactive visualization, which aims to improve the explainability of the performed text classification.<\/p>\n

 <\/p>\n

State of the Art in NLP<\/span><\/p>\n

Modern Natural Language Processing (NLP) methods become more and more powerful. Just recently, OpenAI showed with its giant transformer model GPT-3<\/a> that scaling model and data leads to impressive few-shot capabilities on various NLP tasks.<\/p>\n

\"Abbildung<\/p>\n

Unfortuantely, the generated texts are not really explainable. All of the knowledge in this and other neural models is hiding inside some kind of black-box, it seems to be some kind of black magic. Also, one cannot simply trust these model, often they fail leading potentially dangerous outcomes<\/a>.<\/p>\n

In contrast to that\u00a0 behavior, recent legislation<\/a> demands some kind of explanation for all decisions affecting people, be it decisions from humans or AI algorithms. So somehow, one has to make NLP models more explainable. Ideally, one combines all of the three existing explainability approaches, depicted below: By using other generated or known example cases, explaining the influence of the different inputs in the model (variational approach) and using inherently interpretable models where possible. For a detailed summary of all these methods have a look at the great book<\/a> by Christoph Molnar.<\/p>\n

\"Intrinsically<\/p>\n

In addition to that lack of explainability, it is hard to change or update factual knowledge in neural models, as it is spread across many of their connection weights. Such changes might be necessary, for example, if a new president was elected or a company was renamed. In traditional rule-based systems this is easy, as this information is stored in a knowledge base, often in the form of a knowledge graph<\/a> (KG).<\/p>\n

These KGs have the nice property that knowledge can be easily added, deleted, or updated. The links between the stored entities correspond to real interpretable relations. So their information is also inherently explainable to a human, if annotated correctly with respective tags, so called literals. They are also very flexible in their structure, such that very different kinds of knowledge can be represented in them.<\/p>\n

\"Benefits\"<\/p>\n

These reasons call for an integration of KGs into (neural) NLP models. And there is much recent work in this direction already. Commonly, KGs are integrated into neural network models via one of the following techniques:<\/p>\n

    \n
  1. Knowledge graph embeddings: Here the structural information is vectorized with techniques already known from NLP as in Word2Vec<\/a> and beyond (e.g. TransE<\/a>, ConvE<\/a>, SimplE<\/a>, BoxE<\/a>)<\/li>\n
  2. Paths or subgraphs: The local neighborhood around relevant entities is used directly, as also humans (and rule-based systems) do in their reasoning paths. (e.g. in (Asai et al., 2020)<\/a>, OpenDialKG<\/a>)<\/li>\n
  3. Literal information: As usually a KG’s entities and relations are annotated by text in natural language (literals), this can be used in text processing model like BERT<\/a>. (e.g. in K-BERT<\/a>, KEPLER<\/a>)<\/li>\n<\/ol>\n
    \n

    <\/p>\n<\/div>