{"id":42251,"date":"2023-04-12T17:30:39","date_gmt":"2023-04-12T15:30:39","guid":{"rendered":"https:\/\/www.inovex.de\/?p=42251"},"modified":"2023-04-12T18:19:24","modified_gmt":"2023-04-12T16:19:24","slug":"how-to-build-your-own-kafka-connect-plugin","status":"publish","type":"post","link":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/","title":{"rendered":"How to Build Your Own Kafka Connect Plugin"},"content":{"rendered":"<p>Kafka Connect Plugins are a way to extend Apache Kafka in order to get data into a Kafka Topic or to extract data from it.<\/p>\n<p>There are a lot of <a href=\"https:\/\/www.confluent.io\/hub\/\" target=\"_blank\" rel=\"noopener\">out-of-the-box Sink and Source Connectors<\/a>. Most of them capture some data input from off-the-shelf software like databases and write the captured data back to another storage or database.\u00a0But sometimes the out-of-the-box solution does not fit your specific needs. For such cases, Kafka offers the possibility to write your own Kafka Connect Plugin. In the following article, we will show you how such an individual Connect Plugin can be implemented.<!--more--><\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_83 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\"><p class=\"ez-toc-title\" style=\"cursor:inherit\"><\/p>\n<\/div><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#About-Kafka-Connect\" >About Kafka Connect<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#Introduction-to-the-use-case\" >Introduction to the use case<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#The-Sensor-API\" >The Sensor API<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#Implementation\" >Implementation<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#Kafka-Connect-Source-Connector-Implementation\" >Kafka Connect Source Connector Implementation<\/a><ul class='ez-toc-list-level-4' ><li class='ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#Tasks-and-source-partitions-for-parallelizing\" >Tasks and source partitions for parallelizing<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#Retrieve-sensor-data-seamlessly-with-source-partitions-and-offsets\" >Retrieve sensor data seamlessly with source partitions and offsets<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#Kafka-Connect-Sink-Connector-Implementation\" >Kafka Connect Sink Connector Implementation<\/a><ul class='ez-toc-list-level-4' ><li class='ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#Write-data-to-CSV-files\" >Write data to CSV files<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#Committing-offsets\" >Committing offsets<\/a><\/li><\/ul><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#Conclusion\" >Conclusion<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"About-Kafka-Connect\"><\/span>About Kafka Connect<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Kafka Connect is used for streaming data between Kafka and other external systems. In order to get data into a Kafka Topic so-called Source Connector can be used. It captures the data in a continuous manner and writes to a Kafka Topic while keeping track of the already read data.<\/p>\n<p>One can compare a Source Connector with a Kafka Producer.\u00a0By using a Sink Connector one can read data from a topic and write it into an external data system like a database. Hereby the connector also keeps track of the written data. A Sink Connector acts like Kafka Consumer.\u00a0Both Source Connector and Sink Connector form a Kafka Consumer Group.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Introduction-to-the-use-case\"><\/span>Introduction to the use case<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>In light of the Volkswagen emission scandal in 2015 it was discovered that in many cities in Germany, the air was highly polluted with particulates. Some of these cities were enforced by courts to take action to reduce these emissions. Therefore some cities closed their roads to vehicles with very high emissions. Due to that a very heated debate was held in politics and media.<\/p>\n<p>In one of these cities, namely Stuttgart, the automotive capital of Germany, the public questioned whether banning cars helped to reduce emissions or not. In order to give an answer to this question a local initiative developed easily buildable open-source sensors. The public was encouraged to build and install these sensors outside their homes. The local initiative also provides the infrastructure to collect data from these sensors. The collected data is publicly available via an <a href=\"https:\/\/github.com\/opendata-stuttgart\/meta\/wiki\/APIs#v1\" target=\"_blank\" rel=\"noopener\">API<\/a>.<\/p>\n<p>In this blog post, we will make use of this API in order to demonstrate the functionality of Kafka Connect. Hereby we will show how to implement a Source Connector which retrieves data from the aforementioned API in a periodic manner. That Source Connector will send the data to a Kafka topic which then will be consumed by a Sink Connector. This Sink Connector aggregates the data from that topic and writes it to a CSV file on an hourly basis.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-44721\" src=\"https:\/\/www.inovex.de\/wp-content\/uploads\/Grafik-Artikel-Kafka.png\" alt=\"Kafka Connect Flowchart\" width=\"2157\" height=\"713\" srcset=\"https:\/\/www.inovex.de\/wp-content\/uploads\/Grafik-Artikel-Kafka.png 2157w, https:\/\/www.inovex.de\/wp-content\/uploads\/Grafik-Artikel-Kafka-300x99.png 300w, https:\/\/www.inovex.de\/wp-content\/uploads\/Grafik-Artikel-Kafka-1024x338.png 1024w, https:\/\/www.inovex.de\/wp-content\/uploads\/Grafik-Artikel-Kafka-768x254.png 768w, https:\/\/www.inovex.de\/wp-content\/uploads\/Grafik-Artikel-Kafka-1536x508.png 1536w, https:\/\/www.inovex.de\/wp-content\/uploads\/Grafik-Artikel-Kafka-2048x677.png 2048w, https:\/\/www.inovex.de\/wp-content\/uploads\/Grafik-Artikel-Kafka-1920x635.png 1920w, https:\/\/www.inovex.de\/wp-content\/uploads\/Grafik-Artikel-Kafka-400x132.png 400w, https:\/\/www.inovex.de\/wp-content\/uploads\/Grafik-Artikel-Kafka-360x119.png 360w\" sizes=\"auto, (max-width: 2157px) 100vw, 2157px\" \/><\/p>\n<h3><span class=\"ez-toc-section\" id=\"The-Sensor-API\"><\/span>The Sensor API<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>The sensor API provides an endpoint that returns all sensor data of the last five minutes for a given location. The location is given by a bounding box, i.e., a rectangular area defined by two geo coordinates. In addition, the data can be filtered by sensor type.<\/p>\n<p>The API will return a JSON list with the sensor values and additional information:<\/p>\n<pre class=\"lang:js decode:true \">[\r\n  {\r\n    \"sensordatavalues\": [\r\n      {\r\n        \"id\": 32029110016,\r\n        \"value_type\": \"P1\",\r\n        \"value\": \"15.90\"\r\n      },\r\n      {\r\n        \"id\": 32029110177,\r\n        \"value_type\": \"P2\",\r\n        \"value\": \"10.18\"\r\n      }\r\n    ],\r\n    \"timestamp\": \"2023-02-23 15:11:59\",\r\n    \"location\": {\r\n      \"exact_location\": 0,\r\n      \"longitude\": \"8.394\",\r\n      \"country\": \"DE\",\r\n      \"latitude\": \"48.984\",\r\n      \"id\": 62686,\r\n      \"indoor\": 0,\r\n      \"altitude\": \"115.9\"\r\n    },\r\n    \"sampling_rate\": null,\r\n    \"id\": 14284695600,\r\n    \"sensor\": {\r\n      \"sensor_type\": {\r\n        \"manufacturer\": \"Nova Fitness\",\r\n        \"name\": \"SDS011\",\r\n        \"id\": 14\r\n      },\r\n      \"id\": 38865,\r\n      \"pin\": \"1\"\r\n    }\r\n  },\r\n...\r\n]<\/pre>\n<p>For the city of Karlsruhe, the data can be retrieved by <a href=\"https:\/\/data.sensor.community\/airrohr\/v1\/filter\/box=49.0466,8.4935,48.9720,8.3098\">this URL<\/a>.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Implementation\"><\/span>Implementation<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<h3><span class=\"ez-toc-section\" id=\"Kafka-Connect-Source-Connector-Implementation\"><\/span>Kafka Connect Source Connector Implementation<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>To develop a custom Source Connector, we have to extend the following classes from the Kafka Connect library:<\/p>\n<p>The <a href=\"https:\/\/kafka.apache.org\/34\/javadoc\/org\/apache\/kafka\/connect\/source\/SourceTask.html\" target=\"_blank\" rel=\"noopener\">SourceTask<\/a> class is responsible for writing the desired data into a configured Kafka Topic. In order to do so, the <span class=\"lang:default decode:true crayon-inline \">poll()<\/span> method needs to be provided. Within this method, the data has to be retrieved and provided as a List of SourceRecords to Kafka. <span style=\"font-weight: 400;\">Within our example, the <\/span><b><a href=\"https:\/\/github.com\/inovex\/blog-kafka-connect\/blob\/main\/src\/main\/java\/de\/inovex\/airquality\/connector\/source\/AirQualitySourceTask.java\">AirQualitySourceTask<\/a> <\/b><span style=\"font-weight: 400;\">extends the SourceTask class. The <span class=\"lang:default decode:true crayon-inline\">poll()<\/span> method here calls the Sensor API (see above) and transforms the incoming data into <a href=\"https:\/\/kafka.apache.org\/34\/javadoc\/org\/apache\/kafka\/connect\/source\/SourceRecord.html\" target=\"_blank\" rel=\"noopener\">SourceRecords<\/a>.<\/span><\/p>\n<p>The <span class=\"lang:default decode:true crayon-inline\">poll()<\/span> method is called by Kafka in a constant manner. In order to avoid too much load on the API, we wait between two calls for four minutes (see DataService class for details).<\/p>\n<p><span style=\"font-weight: 400;\">The <a href=\"https:\/\/kafka.apache.org\/34\/javadoc\/org\/apache\/kafka\/connect\/source\/SourceConnector.html\">SourceConnector<\/a> is the main entry point used by Kafka Connect. It receives the connector config from Kafka and is responsible for providing task configurations for the SourceTasks. The<a href=\"https:\/\/github.com\/inovex\/blog-kafka-connect\/blob\/main\/src\/main\/java\/de\/inovex\/airquality\/connector\/source\/AirQualitySourceConnector.java\"><b> AirQualitySourceConnector<\/b> <\/a>class is used to provide a SourceConnector class.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Internally, the AirQualitySourceTask initializes an <\/span><b><a href=\"https:\/\/github.com\/inovex\/blog-kafka-connect\/blob\/main\/src\/main\/java\/de\/inovex\/airquality\/connector\/source\/AirQualitySourcePartition.java\">AirQualitySourcePartition<\/a> <\/b><span style=\"font-weight: 400;\">for every location in its task configuration. The AirQualitySourcePartition keeps track of the source partition and its current offset and ensures that no duplicate sensor readings are created.\u00a0<\/span><span style=\"font-weight: 400;\">By this, the AirQualitySourceTask is able to retrieve the last captured ID from the API in case of a failure.<\/span><\/p>\n<p>In addition to the classes mentioned before, the <a href=\"https:\/\/github.com\/inovex\/blog-kafka-connect\/blob\/main\/src\/main\/java\/de\/inovex\/airquality\/connector\/source\/config\/AirQualitySourceConnectorConfig.java\"><strong>AirQualitySourceConnectorConfig<\/strong> <\/a>(implementing the <a href=\"https:\/\/kafka.apache.org\/34\/javadoc\/org\/apache\/kafka\/common\/config\/AbstractConfig.html\" target=\"_blank\" rel=\"noopener\">AbstractConfig<\/a>) defines the configuration needed by the Source Connector and is used by Kafka Connect to parse the connector JSON config. This allows Kafka Connect to pass the connector config to the connector as a Map.<\/p>\n<h4><span class=\"ez-toc-section\" id=\"Tasks-and-source-partitions-for-parallelizing\"><\/span><strong>Tasks and source partitions for parallelizing<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p><span style=\"font-weight: 400;\">As explained in the section above, a Source Connector can execute multiple tasks for parallel ingestion. To parallelize our requests to the Sensor API, we allow the Source Connector to be configured for multiple locations. This allows us to distribute the locations among multiple tasks, thus reading sensor data in parallel.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In the configuration for the Source Connector, we specify a list of comma-separated locations. A location consists of the lat \/ long value of the first geo coordinate, followed by the lat \/ long values of the second coordinate. The example below shows the configuration of Karlsruhe, Stuttgart, and Cologne:<\/span><\/p>\n<pre class=\"lang:default decode:true\">name=airquality_source_connector\r\nconnector.class=de.inovex.airquality.connector.source.AirQualitySourceConnector\r\ntasks.max=2\r\ntopics=topic1\r\nlocations=48.949;8.265;49.086;8.513,48.755;9.130;48.828;9.249,50.964;6.911;50.899;7.039<\/pre>\n<p>Every configured location for which we retrieve data represents a source partition. The number of locations gives us the maximum level of parallelism.\u00a0We implement this using the taskConfigs function, which receives the maximum number of tasks and creates configs for every task:<\/p>\n<pre class=\"lang:java decode:true\">public List&lt;Map&lt;String, String&gt;&gt; taskConfigs(int maxTasks);<\/pre>\n<p>We create task configs by simply copying the existing connector config. We only modify the locations string such that each location is contained in exactly one task config. The Kafka Connect library already provides the convenient helper function <span class=\"lang:default decode:true crayon-inline\">groupPartitions(List&lt;T&gt; elements, int numGroups)<\/span> <a href=\"https:\/\/kafka.apache.org\/34\/javadoc\/org\/apache\/kafka\/connect\/util\/ConnectorUtils.html#groupPartitions(java.util.List,int)\" target=\"_blank\" rel=\"noopener\">here<\/a>, which distributes a list\u2019s elements evenly into numGroups sublists. This gives us the following code:<\/p>\n<pre class=\"lang:java decode:true\">@Override\r\npublic List&lt;Map&lt;String, String&gt;&gt; taskConfigs(int maxTasks) {\r\n    return ConnectorUtils.groupPartitions(config.getLocations(), maxTasks).stream()\r\n            .map(l -&gt; createTaskConfig(l)).collect(Collectors.toList());\r\n}\r\n\r\nprivate Map&lt;String, String&gt; createTaskConfig(List&lt;String&gt; locations) {\r\n    HashMap&lt;String, String&gt; taskConfig = new HashMap&lt;&gt;(config.originalsStrings());\r\n    taskConfig.put(\"locations\", String.join(\",\", locations));\r\n    return taskConfig;\r\n}<\/pre>\n<p>One element of the list returned by the <span class=\"lang:default decode:true crayon-inline\">taskConfigs(int maxTasks)<\/span> method corresponds to one SourceTask which is spawned by Kafka.<\/p>\n<p>In our example, we have set maxTasks to 3 so that one SourceTask is responsible for one location. This value depends highly on the source your connector shall capture and the resources of the server where your Kafka Connector shall run.<\/p>\n<h4><span class=\"ez-toc-section\" id=\"Retrieve-sensor-data-seamlessly-with-source-partitions-and-offsets\"><\/span>Retrieve sensor data seamlessly with source partitions and offsets<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p><span style=\"font-weight: 400;\">The Sensor API always returns all data of the last five minutes. If we query the API after this time, we will lose records. On the other hand, if we query after two minutes already, we will receive duplicates, as the querying window overlaps.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">To avoid this, we can use the concept of source partition offsets. For every source partition, we define the offset as a tuple of the ID and the timestamp of the sensor reading. When sending a record to Kafka in the Source Connector, we also give this source offset to Kafka Connect which will store the information internally. We also set the offset in our task.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The next time we query the API, we simply ignore all records up until the sensor reading with the ID and timestamp given by the last offset. This way, we avoid both missing records and duplicates.<\/span><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-43071 size-full aligncenter\" src=\"https:\/\/www.inovex.de\/wp-content\/uploads\/test-1.png\" alt=\"diagram of how to avoid reading duplicates in Kafka Connect\" width=\"1600\" height=\"418\" srcset=\"https:\/\/www.inovex.de\/wp-content\/uploads\/test-1.png 1600w, https:\/\/www.inovex.de\/wp-content\/uploads\/test-1-300x78.png 300w, https:\/\/www.inovex.de\/wp-content\/uploads\/test-1-1024x268.png 1024w, https:\/\/www.inovex.de\/wp-content\/uploads\/test-1-768x201.png 768w, https:\/\/www.inovex.de\/wp-content\/uploads\/test-1-1536x401.png 1536w, https:\/\/www.inovex.de\/wp-content\/uploads\/test-1-400x105.png 400w, https:\/\/www.inovex.de\/wp-content\/uploads\/test-1-360x94.png 360w\" sizes=\"auto, (max-width: 1600px) 100vw, 1600px\" \/><\/p>\n<p>Kafka Connect provides the last offset to the connector task, so we can access this information even if the connector is restarted and the tasks are initialized.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Kafka-Connect-Sink-Connector-Implementation\"><\/span>Kafka Connect Sink Connector Implementation<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>For the custom Sink Connector, we implement the following classes:<\/p>\n<p>In the <a href=\"https:\/\/github.com\/inovex\/blog-kafka-connect\/blob\/main\/src\/main\/java\/de\/inovex\/airquality\/connector\/sink\/AirQualitySinkTask.java\"><strong>AirQualitySinkTask<\/strong> <\/a>(implementing the <a href=\"https:\/\/kafka.apache.org\/34\/javadoc\/org\/apache\/kafka\/connect\/sink\/SinkTask.html\" target=\"_blank\" rel=\"noopener\">SinkTask<\/a> class) the <span class=\"lang:default decode:true crayon-inline\">put()<\/span> function defines what shall be done once records are read from a topic. The <span class=\"lang:default decode:true crayon-inline\">preCommit()<\/span> function defines how and when offsets can be committed.<\/p>\n<p>The <a href=\"https:\/\/github.com\/inovex\/blog-kafka-connect\/blob\/main\/src\/main\/java\/de\/inovex\/airquality\/connector\/sink\/AirQualityCsvFileWriter.java\"><strong>AirQualityCsvFileWriter<\/strong> <\/a>(uses an opencsv <a href=\"https:\/\/opencsv.sourceforge.net\/apidocs\/com\/opencsv\/CSVWriter.html\" target=\"_blank\" rel=\"noopener\">CSVWriter<\/a>) is used to open a file and write out the data in the correct CSV format.<\/p>\n<p>The <a href=\"https:\/\/github.com\/inovex\/blog-kafka-connect\/blob\/main\/src\/main\/java\/de\/inovex\/airquality\/connector\/sink\/config\/AirQualitySinkConnectorConfig.java\"><strong>AirQualitySinkConfig<\/strong> <\/a>(implementing the <a href=\"https:\/\/kafka.apache.org\/34\/javadoc\/org\/apache\/kafka\/common\/config\/AbstractConfig.html\" target=\"_blank\" rel=\"noopener\">AbstractConfig<\/a>) creates a very basic config with Kafka\u2019s <span class=\"lang:default decode:true crayon-inline\">configDef()<\/span>.<\/p>\n<p>The <a href=\"https:\/\/github.com\/inovex\/blog-kafka-connect\/blob\/main\/src\/main\/java\/de\/inovex\/airquality\/connector\/sink\/AirQualitySinkConnector.java\"><strong>AirQualitySinkConnector<\/strong> <\/a>(implementing the <a href=\"https:\/\/kafka.apache.org\/34\/javadoc\/org\/apache\/kafka\/connect\/sink\/SinkConnector.html\" target=\"_blank\" rel=\"noopener\">SinkConnector<\/a> class) gets called when deploying the Sink Connector. In this all other classes are collected. The AirQualitySinkConfig gets created in <span class=\"lang:default decode:true crayon-inline\">start()<\/span> , the AirQualitySinkTask gets returned in <span class=\"lang:default decode:true crayon-inline\">taskClass()<\/span> and it can be defined what should happen when the connector stops in <span class=\"lang:default decode:true crayon-inline\">stop()<\/span> .<\/p>\n<h4><span class=\"ez-toc-section\" id=\"Write-data-to-CSV-files\"><\/span>Write data to CSV files<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p>In the task the object topicDateHourWriter is defined. It saves all FileWriter per hour per topic-partition as we want to write one file per hour.<\/p>\n<p>In the <span class=\"lang:default decode:true crayon-inline\">put()<\/span> method for each record a new hour-to-FileWriter-map is created in topicDateHourWriter if it does not exist yet.<\/p>\n<pre class=\"lang:java decode:true\" title=\"put()-method\">public class AirQualitySinkTask extends SinkTask {\r\n...\r\n@Override\r\npublic void put(Collection&lt;SinkRecord&gt; records) {\r\n    logger.info(\"Start put\");\r\n    for (SinkRecord record : records) {\r\n\r\n        \/\/ get topicPartition from record and add if not seen before\r\n        final TopicPartition topicPartition = new TopicPartition(record.topic(), record.kafkaPartition());\r\n        Map&lt;LocalDateTime, CsvFileWriter&gt; dateTimeHourWriter = topicDateHourWriter.computeIfAbsent(\r\n                topicPartition, tp -&gt; new HashMap&lt;&gt;());\r\n   ...\r\n   }\r\n...\r\n}<\/pre>\n<p><span style=\"font-weight: 400;\">After some casts, we write the records to the CSV file. Finally, we save the latest offset per hour. <\/span><\/p>\n<pre class=\"lang:java decode:true \">dateTimeHourWriter.computeIfAbsent(dateTimeHour, k -&gt; {\r\n    try {\r\n        return csvWriterProvider.get().dateTimeHour(dateTimeHour).build();\r\n    } catch (Exception e) {\r\n         throw new IllegalStateException(e);\r\n    }\r\n}).write(sensorData);\r\n\r\n\/\/ put the latest processed offset in latestOffset\r\nlatestOffset.put(dateTimeHour, record.kafkaOffset());<\/pre>\n<h4><span class=\"ez-toc-section\" id=\"Committing-offsets\"><\/span>Committing offsets<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p>For the <span class=\"lang:default decode:true crayon-inline\">preCommit()<\/span> function we need to define which offsets (records) can be committed. Since we want to write one file per hour only offsets of past hours can be committed.<br \/>\nTo achieve this the first thing we need to do is to get all hour-to-FileWriter-maps of past hours.<\/p>\n<pre class=\"lang:java decode:true\">\/\/ write csv when full hour changed, iterate over dateTimeHourWriter\r\n\/\/ get writer from past full hours\r\nMap&lt;LocalDateTime, CsvFileWriter&gt; pastDateHourWriter = dateTimeHourWriter.entrySet().stream()\r\n    .filter(e -&gt; e.getKey().isBefore(currentHour))\r\n    .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));<\/pre>\n<p>Then we need to get the latest offset of the latest past hour.<\/p>\n<pre class=\"lang:java decode:true\">\/\/ get latest hour of past data\r\nOptional&lt;LocalDateTime&gt; lastPastHour = pastDateHourWriter.keySet().stream().max(LocalDateTime::compareTo);\r\n\r\n\/\/ if no pastHour available there is nothing to commit\r\nif (lastPastHour.isEmpty()) {\r\n    logger.info(\"Nothing to commit\");\r\n    continue;\r\n}\r\n\r\n\/\/ get latest offset from latest past hour\r\nLong lastCommitableOffset = latestOffset.get(lastPastHour.get());<\/pre>\n<p>This is the latest offset we can commit, so we return this as commitableOffset. Before returning we can also close all writers of past hours.<\/p>\n<pre class=\"lang:java decode:true\">\/\/ this offset is the last to commit\r\ncommitableOffset.put(topicPartition, new OffsetAndMetadata(lastCommitableOffset));\r\n\r\n\/\/ close all finished writers\r\nfor(CsvFileWriter writer: pastDateHourWriter.values()){\r\n    writer.close();\r\n}<\/pre>\n<h2><span class=\"ez-toc-section\" id=\"Conclusion\"><\/span>Conclusion<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p><span style=\"font-weight: 400;\">In this post, we have shown you how you can build your own Kafka Connect Plugin. Compared to just using Consumers and Producers to read and write to\/from external resources Kafka Connect offers a sophisticated way to handle parallelization (by SourceTask and SinkTask classes) and operations.<\/span><\/p>\n<p><span style=\"font-weight: 400;\"> If you want to try this out yourself check out the Readme in the <a href=\"https:\/\/github.com\/inovex\/blog-kafka-connect\">Repository<\/a> and run the Connectors in a Docker container.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Kafka Connect Plugins are a way to extend Apache Kafka in order to get data into a Kafka Topic or to extract data from it. There are a lot of out-of-the-box Sink and Source Connectors. Most of them capture some data input from off-the-shelf software like databases and write the captured data back to another [&hellip;]<\/p>\n","protected":false},"author":251,"featured_media":44718,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"ep_exclude_from_search":false,"footnotes":""},"tags":[385,723,1033,909,908],"service":[411],"coauthors":[{"id":251,"display_name":"Tilman Berger","user_nicename":"tberger"},{"id":333,"display_name":"Nico Gro\u00dfkreuz","user_nicename":"ngrosskreuz"}],"class_list":["post-42251","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","tag-data-engineering","tag-java","tag-kafka","tag-open-source","tag-stream-processing","service-data-engineering"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.5 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>How to Build Your Own Kafka Connect Plugin - inovex GmbH<\/title>\n<meta name=\"description\" content=\"There are a lot of prebuild Sink and Source Connectors, but not all of them fit your use case. We will show you how to build your own Kafka Connect Plugin!\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How to Build Your Own Kafka Connect Plugin - inovex GmbH\" \/>\n<meta property=\"og:description\" content=\"There are a lot of prebuild Sink and Source Connectors, but not all of them fit your use case. We will show you how to build your own Kafka Connect Plugin!\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/\" \/>\n<meta property=\"og:site_name\" content=\"inovex GmbH\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/inovexde\" \/>\n<meta property=\"article:published_time\" content=\"2023-04-12T15:30:39+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-04-12T16:19:24+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.inovex.de\/wp-content\/uploads\/build-your-own-kafka-connect-plugin.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"1080\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Tilman Berger, Nico Gro\u00dfkreuz\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/www.inovex.de\/wp-content\/uploads\/build-your-own-kafka-connect-plugin-1024x576.png\" \/>\n<meta name=\"twitter:creator\" content=\"@inovexgmbh\" \/>\n<meta name=\"twitter:site\" content=\"@inovexgmbh\" \/>\n<meta name=\"twitter:label1\" content=\"Verfasst von\" \/>\n\t<meta name=\"twitter:data1\" content=\"Tilman Berger\" \/>\n\t<meta name=\"twitter:label2\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data2\" content=\"10\u00a0Minuten\" \/>\n\t<meta name=\"twitter:label3\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data3\" content=\"Tilman Berger, Nico Gro\u00dfkreuz\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/how-to-build-your-own-kafka-connect-plugin\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/how-to-build-your-own-kafka-connect-plugin\\\/\"},\"author\":{\"name\":\"Tilman Berger\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/person\\\/f6eb57fb4c8300d44b3171707723ccdb\"},\"headline\":\"How to Build Your Own Kafka Connect Plugin\",\"datePublished\":\"2023-04-12T15:30:39+00:00\",\"dateModified\":\"2023-04-12T16:19:24+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/how-to-build-your-own-kafka-connect-plugin\\\/\"},\"wordCount\":1607,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/how-to-build-your-own-kafka-connect-plugin\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/build-your-own-kafka-connect-plugin.png\",\"keywords\":[\"Data Engineering\",\"Java\",\"Kafka\",\"Open Source\",\"Stream Processing\"],\"articleSection\":[\"Analytics\",\"English Content\",\"General\"],\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/how-to-build-your-own-kafka-connect-plugin\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/how-to-build-your-own-kafka-connect-plugin\\\/\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/how-to-build-your-own-kafka-connect-plugin\\\/\",\"name\":\"How to Build Your Own Kafka Connect Plugin - inovex GmbH\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/how-to-build-your-own-kafka-connect-plugin\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/how-to-build-your-own-kafka-connect-plugin\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/build-your-own-kafka-connect-plugin.png\",\"datePublished\":\"2023-04-12T15:30:39+00:00\",\"dateModified\":\"2023-04-12T16:19:24+00:00\",\"description\":\"There are a lot of prebuild Sink and Source Connectors, but not all of them fit your use case. We will show you how to build your own Kafka Connect Plugin!\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/how-to-build-your-own-kafka-connect-plugin\\\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/how-to-build-your-own-kafka-connect-plugin\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/how-to-build-your-own-kafka-connect-plugin\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/build-your-own-kafka-connect-plugin.png\",\"contentUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/build-your-own-kafka-connect-plugin.png\",\"width\":1920,\"height\":1080,\"caption\":\"Ein Connector schmiegt sich um einen Ring des Kafka-Logos\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/how-to-build-your-own-kafka-connect-plugin\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How to Build Your Own Kafka Connect Plugin\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#website\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\",\"name\":\"inovex GmbH\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\",\"name\":\"inovex GmbH\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/inovex-logo-16-9-1.png\",\"contentUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/inovex-logo-16-9-1.png\",\"width\":1921,\"height\":1081,\"caption\":\"inovex GmbH\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/inovexde\",\"https:\\\/\\\/x.com\\\/inovexgmbh\",\"https:\\\/\\\/www.instagram.com\\\/inovexlife\\\/\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/inovex\",\"https:\\\/\\\/www.youtube.com\\\/channel\\\/UC7r66GT14hROB_RQsQBAQUQ\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/person\\\/f6eb57fb4c8300d44b3171707723ccdb\",\"name\":\"Tilman Berger\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e82c9e3c6b8ad97aec8cb76f6cd6267de141c5e21cad8e9a74925ad28fb2926b?s=96&d=retro&r=g4d156755b30d90d2be55f98164e31f61\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e82c9e3c6b8ad97aec8cb76f6cd6267de141c5e21cad8e9a74925ad28fb2926b?s=96&d=retro&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/e82c9e3c6b8ad97aec8cb76f6cd6267de141c5e21cad8e9a74925ad28fb2926b?s=96&d=retro&r=g\",\"caption\":\"Tilman Berger\"},\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/blog\\\/author\\\/tberger\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How to Build Your Own Kafka Connect Plugin - inovex GmbH","description":"There are a lot of prebuild Sink and Source Connectors, but not all of them fit your use case. We will show you how to build your own Kafka Connect Plugin!","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/","og_locale":"de_DE","og_type":"article","og_title":"How to Build Your Own Kafka Connect Plugin - inovex GmbH","og_description":"There are a lot of prebuild Sink and Source Connectors, but not all of them fit your use case. We will show you how to build your own Kafka Connect Plugin!","og_url":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/","og_site_name":"inovex GmbH","article_publisher":"https:\/\/www.facebook.com\/inovexde","article_published_time":"2023-04-12T15:30:39+00:00","article_modified_time":"2023-04-12T16:19:24+00:00","og_image":[{"width":1920,"height":1080,"url":"https:\/\/www.inovex.de\/wp-content\/uploads\/build-your-own-kafka-connect-plugin.png","type":"image\/png"}],"author":"Tilman Berger, Nico Gro\u00dfkreuz","twitter_card":"summary_large_image","twitter_image":"https:\/\/www.inovex.de\/wp-content\/uploads\/build-your-own-kafka-connect-plugin-1024x576.png","twitter_creator":"@inovexgmbh","twitter_site":"@inovexgmbh","twitter_misc":{"Verfasst von":"Tilman Berger","Gesch\u00e4tzte Lesezeit":"10\u00a0Minuten","Written by":"Tilman Berger, Nico Gro\u00dfkreuz"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#article","isPartOf":{"@id":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/"},"author":{"name":"Tilman Berger","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/person\/f6eb57fb4c8300d44b3171707723ccdb"},"headline":"How to Build Your Own Kafka Connect Plugin","datePublished":"2023-04-12T15:30:39+00:00","dateModified":"2023-04-12T16:19:24+00:00","mainEntityOfPage":{"@id":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/"},"wordCount":1607,"commentCount":0,"publisher":{"@id":"https:\/\/www.inovex.de\/de\/#organization"},"image":{"@id":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#primaryimage"},"thumbnailUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/build-your-own-kafka-connect-plugin.png","keywords":["Data Engineering","Java","Kafka","Open Source","Stream Processing"],"articleSection":["Analytics","English Content","General"],"inLanguage":"de","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/","url":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/","name":"How to Build Your Own Kafka Connect Plugin - inovex GmbH","isPartOf":{"@id":"https:\/\/www.inovex.de\/de\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#primaryimage"},"image":{"@id":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#primaryimage"},"thumbnailUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/build-your-own-kafka-connect-plugin.png","datePublished":"2023-04-12T15:30:39+00:00","dateModified":"2023-04-12T16:19:24+00:00","description":"There are a lot of prebuild Sink and Source Connectors, but not all of them fit your use case. We will show you how to build your own Kafka Connect Plugin!","breadcrumb":{"@id":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/"]}]},{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#primaryimage","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/build-your-own-kafka-connect-plugin.png","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/build-your-own-kafka-connect-plugin.png","width":1920,"height":1080,"caption":"Ein Connector schmiegt sich um einen Ring des Kafka-Logos"},{"@type":"BreadcrumbList","@id":"https:\/\/www.inovex.de\/de\/blog\/how-to-build-your-own-kafka-connect-plugin\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.inovex.de\/de\/"},{"@type":"ListItem","position":2,"name":"How to Build Your Own Kafka Connect Plugin"}]},{"@type":"WebSite","@id":"https:\/\/www.inovex.de\/de\/#website","url":"https:\/\/www.inovex.de\/de\/","name":"inovex GmbH","description":"","publisher":{"@id":"https:\/\/www.inovex.de\/de\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.inovex.de\/de\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Organization","@id":"https:\/\/www.inovex.de\/de\/#organization","name":"inovex GmbH","url":"https:\/\/www.inovex.de\/de\/","logo":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png","width":1921,"height":1081,"caption":"inovex GmbH"},"image":{"@id":"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/inovexde","https:\/\/x.com\/inovexgmbh","https:\/\/www.instagram.com\/inovexlife\/","https:\/\/www.linkedin.com\/company\/inovex","https:\/\/www.youtube.com\/channel\/UC7r66GT14hROB_RQsQBAQUQ"]},{"@type":"Person","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/person\/f6eb57fb4c8300d44b3171707723ccdb","name":"Tilman Berger","image":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/secure.gravatar.com\/avatar\/e82c9e3c6b8ad97aec8cb76f6cd6267de141c5e21cad8e9a74925ad28fb2926b?s=96&d=retro&r=g4d156755b30d90d2be55f98164e31f61","url":"https:\/\/secure.gravatar.com\/avatar\/e82c9e3c6b8ad97aec8cb76f6cd6267de141c5e21cad8e9a74925ad28fb2926b?s=96&d=retro&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e82c9e3c6b8ad97aec8cb76f6cd6267de141c5e21cad8e9a74925ad28fb2926b?s=96&d=retro&r=g","caption":"Tilman Berger"},"url":"https:\/\/www.inovex.de\/de\/blog\/author\/tberger\/"}]}},"_links":{"self":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/42251","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/users\/251"}],"replies":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/comments?post=42251"}],"version-history":[{"count":7,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/42251\/revisions"}],"predecessor-version":[{"id":44725,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/42251\/revisions\/44725"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/media\/44718"}],"wp:attachment":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/media?parent=42251"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/tags?post=42251"},{"taxonomy":"service","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/service?post=42251"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/coauthors?post=42251"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}