{"id":17485,"date":"2020-03-09T08:50:54","date_gmt":"2020-03-09T07:50:54","guid":{"rendered":"https:\/\/www.inovex.de\/blog\/?p=17485"},"modified":"2022-11-21T16:12:19","modified_gmt":"2022-11-21T15:12:19","slug":"high-throughput-elasticsearch-part-1","status":"publish","type":"post","link":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/","title":{"rendered":"Planning High Throughput Elasticsearch Clusters: Part 1"},"content":{"rendered":"<p>Building an Elasticsearch cluster for log ingestion can be as easy as assembling an Ikea Billy bookshelf and sizing as complex as engineering a 24 hours of Le Mans race car. In this article series we will put a spotlight on what has to be taken into account when planning those clusters. We will take a look at index management strategies and node resource choices, then tune the cluster, indices and shippers in order to achieve optimal data ingestion performance.<\/p>\n<p><!--more--><\/p>\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_79_2 counter-hierarchy ez-toc-counter ez-toc-custom ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\"><p class=\"ez-toc-title\" style=\"cursor:inherit\"><\/p>\n<\/div><nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#Mapping-Out-the-Cluster\" >Mapping Out the Cluster<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#Planning\" >Planning<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#Drafting-the-Elasticsearch-Cluster\" >Drafting the Elasticsearch Cluster<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#Choosing-a-Proper-Heap-Size\" >Choosing a Proper Heap Size<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#Planning-Index-Organization\" >Planning Index Organization<\/a><ul class='ez-toc-list-level-4' ><li class='ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#Shards-Influence-Cluster-Size\" >Shards Influence Cluster Size<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#Consolidate-Data-by-Affiliation\" >Consolidate Data by Affiliation<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-4'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#Index-Patterns\" >Index Patterns<\/a><\/li><\/ul><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#Up-Next\" >Up Next<\/a><\/li><\/ul><\/nav><\/div>\n<h2><span class=\"ez-toc-section\" id=\"Mapping-Out-the-Cluster\"><\/span>Mapping Out the Cluster<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Elasticsearch does not come with a one size fits all recipe in any shape or form\u2014use cases and data shapeu a cluster\u2019s contours. The complexity ranges from a simple yet sufficient single node setup to a distributed multinode cluster with dedicated masters, data and ingest nodes.<\/p>\n<p>The amount of data nodes is determined by data sources, JVM heap size, index lifecycling, data growth and amount of shards. In fact, we cannot properly size and optimize a cluster as long as it does not get hit by production workload. However, by closely examining our use case and data, we can estimate the cluster size, then refine and optimize it further.\u00a0The process can be applied to any Elasticsearch version greater than 6.7 up to release 8.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Planning\"><\/span>Planning<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Building an Elasticsearch cluster follows an iterative procedure:<\/p>\n<ul>\n<li>Clarify the use case<\/li>\n<li>Examine the data and its sources<\/li>\n<li>Define an index management strategy<\/li>\n<li>Draft the cluster based on that information<\/li>\n<li>Benchmark with test data<\/li>\n<li>Refine and tune the cluster with test and real data<\/li>\n<\/ul>\n<p>The draft must respect certain boundaries:<\/p>\n<ol>\n<li>A shard has a capacity of 2.1 billion (2^31) documents.<\/li>\n<li><a href=\"https:\/\/www.elastic.co\/blog\/how-many-shards-should-i-have-in-my-elasticsearch-cluster\">Elastic recommends<\/a>\u00a0having a memory\/shard distribution of 1:20 (1GB heap manages 20 shards) in order to guarantee best performance. Although it is not a hard limit, we want to stay close to that number.<\/li>\n<li>It is advised to not grow a shard larger than 60GB to 80GB for two main reasons:\n<ol>\n<li>In case of a failure, it will take longer to recover and rebalance big shards than it would with small shards.<\/li>\n<li>Merge operations are I\/O intensive jobs. Long CPU and disk consumption cycles can lead to gimped index throughput and worse: dropped messages.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<p>Alongside these boundaries, we define the index management and data retention strategies. Once the initial draft is made, it&#8217;s ready for a benchmark with test and real data.<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Drafting-the-Elasticsearch-Cluster\"><\/span>Drafting the Elasticsearch Cluster<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Our main concern is how data is managed in indices. The way we organize indices and their lifecycles, will affect node resources and therefore influence the cluster size. Thus, we&#8217;re primarily sizing a cluster based on our index management and housekeeping decisions, and secondarily for throughput.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Choosing-a-Proper-Heap-Size\"><\/span>Choosing a Proper Heap Size<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>Although it is tempting to set the JVM heap to the maximum configurable size, it is advised to follow the <a href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/reference\/current\/heap-size.html\">setup guide<\/a>\u00a0and avoid Elasticsearch to start with <a href=\"https:\/\/www.elastic.co\/guide\/en\/elasticsearch\/guide\/current\/heap-sizing.html#compressed_oops\">compressed oops<\/a>.\u00a0Generally speaking, we recommend to start low and increase the heap as needed. <a href=\"https:\/\/www.elastic.co\/blog\/a-heap-of-trouble\">A Heap of Trouble<\/a> discusses the implications of too big and too small JVM heap sizes.<\/p>\n<p><strong>TL;DR:<\/strong><\/p>\n<blockquote><p>[&#8230;] If the heap is too small, applications will be prone to the danger of out of memory errors. [&#8230;] If the heap is too large, the application will be prone to infrequent long latency spikes from full-heap garbage collections. [&#8230;] a long pause is indistinguishable from a node that is unreachable because it is hung [&#8230;].<\/p>\n<p>[&#8230;] it&#8217;s better to set the heap as low as possible while satisfying your requirements for indexing and query throughput, end-user query response times, yet large enough to have adequate heap space for indexing buffers, and large consumers of heap space like aggregations, and suggesters.<\/p><\/blockquote>\n<p>Heap size needs to be monitored and adjusted during production workload. 8GB to 12GB heap size might be a reasonable starting point, unless there is a tremendous amount of data, users and complex queries expected right from the start.<\/p>\n<h3><span class=\"ez-toc-section\" id=\"Planning-Index-Organization\"><\/span>Planning Index Organization<span class=\"ez-toc-section-end\"><\/span><\/h3>\n<p>In order to estimate the resources needed, we have to examine our data and sources. Five things will help with making the first draft:<\/p>\n<ul>\n<li>data retention periods<\/li>\n<li>data structures and affiliation<\/li>\n<li>amount of data received per given time period<\/li>\n<li>number of clients shipping data<\/li>\n<li>number of clients querying data<\/li>\n<\/ul>\n<p>The number of clients becomes valuable, when tuning the cluster for data ingestion and calculating the client shard concurrency. We will take a closer look at concurrency, when benchmarking the cluster. For now, we do not have to take it into account. The expected data per day will give us a hint whether or not basic index management will be sufficient and if it&#8217;s necessary to further enhance index management or the whole cluster.<\/p>\n<h4><span class=\"ez-toc-section\" id=\"Shards-Influence-Cluster-Size\"><\/span>Shards Influence Cluster Size<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p>The total amount of shards is the result of how indices and their lifecycles are managed. If we commit to the 1:20 ratio, a node with 8GB heap is capable of managing 160 shards. A single node can store 80 daily rotated indices with 1 shard and 1 replica configured. If we set <code>index.number_of_shards<\/code> to 2, the node has a capacity of 40 indices.<\/p>\n<h4><span class=\"ez-toc-section\" id=\"Consolidate-Data-by-Affiliation\"><\/span>Consolidate Data by Affiliation<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p>Logs come from many different sources\/applications. They can be either structured or unstructured. In order to reduce the amount of indices, and therefore the amount of shards, we should consider storing logs with similar structure in the same index. Grouping logs by structure and affiliation instead of origin, will help with keeping the cluster small.<\/p>\n<p>Having three applications with similar structure logging into separate indices with 1 shard and 1 replica configured leaves us with 180 shards after 30 days for a single environment. Three environments would generate 540 shards. That equates to at least three data nodes with 8GB heap each. The same applications shipping data into a single index would result in just 180 shards for three environments.<\/p>\n<p>A common example for data affiliation is a highly available web application with a middleware, haproxy and a database. None of these components share a log structure, but the logs are affiliated, due to the fact that they are application components. Whenever we want to track logs across several different components and correlate them, it makes sense to ship the data into the same index, in order to make that data\u00a0 accessible more easily in Kibana.<\/p>\n<h4><span class=\"ez-toc-section\" id=\"Index-Patterns\"><\/span>Index Patterns<span class=\"ez-toc-section-end\"><\/span><\/h4>\n<p>The most basic and common index is daily rotated and environment specific. A pattern like <code><i>logs-%{environment}-%{YYYY.mm.dd}<\/i><\/code> is most likely to encounter. Depending on how much data a single index receives, this might be sufficient, or a waste of shards. Some applications might generate 300GB or more per day, others might only generate 5GB or less. Whenever small indices are encountered, it is worth reconsidering the daily rotation and applying a weekly rotation instead to use shards more efficiently.<\/p>\n<p>When data is consolidated by affiliation and grouped by environment, how do we find a specific application log? Filebeat needs an input for each log file and add extra fields, so that a user can query on that field and find the applications log events.<\/p>\n<pre class=\"lang:default decode:true\">filebeat.inputs:\r\n\r\n- type: log\r\n\r\n\u00a0 enabled: true\r\n\r\n\u00a0 paths:\r\n\r\n\u00a0 - \/var\/log\/application1-*.log\r\n\r\n\u00a0 fields:\r\n\r\n\u00a0 \u00a0 app: application\r\n\r\n<\/pre>\n<p>In order to route the events into the correct index, we need to configure conditions on <code>output.elasticsearch<\/code>.<\/p>\n<pre class=\"lang:default decode:true\">output.elasticsearch:\r\n\r\n  hosts: [\"https:\/\/elastic-1.domain.tld:9200\"]\r\n\r\n  indices:\r\n\r\n  - index: \"application-%{YYYY.MM.dd}\r\n\r\n    when.contains:\r\n\r\n      app: application<\/pre>\n<p>The host on which the application is running will be added to the event by filebeat automatically.<\/p>\n<p>&nbsp;<\/p>\n<h2><span class=\"ez-toc-section\" id=\"Up-Next\"><\/span>Up Next<span class=\"ez-toc-section-end\"><\/span><\/h2>\n<p>Part 2 will focus on ILM pros and cons, shard allocation in a hot-warm-cold architecture, explain client shard concurrency and how it affects cluster size.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Building an Elasticsearch cluster for log ingestion can be as easy as assembling an Ikea Billy bookshelf and sizing as complex as engineering a 24 hours of Le Mans race car. In this article series we will put a spotlight on what has to be taken into account when planning those clusters. We will take [&hellip;]<\/p>\n","protected":false},"author":135,"featured_media":18317,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"inline_featured_image":false,"ep_exclude_from_search":false,"footnotes":""},"tags":[71,82],"service":[414,504],"coauthors":[{"id":135,"display_name":"Daniel Chamot","user_nicename":"dchamot"}],"class_list":["post-17485","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","tag-cloud","tag-search","service-cloud","service-search"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.5 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Planning High Throughput Elasticsearch Clusters: Part 1 - inovex GmbH<\/title>\n<meta name=\"description\" content=\"Building an Elasticsearch cluster can be as easy as assembling an Ikea Billy bookshelf and sizing as complex as engineering a 24 hours of Le Mans race car. In this article series we will put a spotlight on what has to be taken into account when molding clusters.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Planning High Throughput Elasticsearch Clusters: Part 1 - inovex GmbH\" \/>\n<meta property=\"og:description\" content=\"Building an Elasticsearch cluster can be as easy as assembling an Ikea Billy bookshelf and sizing as complex as engineering a 24 hours of Le Mans race car. In this article series we will put a spotlight on what has to be taken into account when molding clusters.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/\" \/>\n<meta property=\"og:site_name\" content=\"inovex GmbH\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/inovexde\" \/>\n<meta property=\"article:published_time\" content=\"2020-03-09T07:50:54+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-11-21T15:12:19+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2020\/03\/high-throughput-elastiocsearch-clusters.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"1080\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Daniel Chamot\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/www.inovex.de\/wp-content\/uploads\/2020\/03\/high-throughput-elastiocsearch-clusters-1024x576.png\" \/>\n<meta name=\"twitter:creator\" content=\"@inovexgmbh\" \/>\n<meta name=\"twitter:site\" content=\"@inovexgmbh\" \/>\n<meta name=\"twitter:label1\" content=\"Verfasst von\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daniel Chamot\" \/>\n\t<meta name=\"twitter:label2\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data2\" content=\"6\u00a0Minuten\" \/>\n\t<meta name=\"twitter:label3\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data3\" content=\"Daniel Chamot\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/\"},\"author\":{\"name\":\"Daniel Chamot\",\"@id\":\"https:\/\/www.inovex.de\/de\/#\/schema\/person\/9c4c40eb2a68e82a5ce9faee6ef607ca\"},\"headline\":\"Planning High Throughput Elasticsearch Clusters: Part 1\",\"datePublished\":\"2020-03-09T07:50:54+00:00\",\"dateModified\":\"2022-11-21T15:12:19+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/\"},\"wordCount\":1216,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/www.inovex.de\/de\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.inovex.de\/wp-content\/uploads\/2020\/03\/high-throughput-elastiocsearch-clusters.png\",\"keywords\":[\"Cloud\",\"Search\"],\"articleSection\":[\"English Content\",\"General\",\"Infrastructure\"],\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/\",\"url\":\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/\",\"name\":\"Planning High Throughput Elasticsearch Clusters: Part 1 - inovex GmbH\",\"isPartOf\":{\"@id\":\"https:\/\/www.inovex.de\/de\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.inovex.de\/wp-content\/uploads\/2020\/03\/high-throughput-elastiocsearch-clusters.png\",\"datePublished\":\"2020-03-09T07:50:54+00:00\",\"dateModified\":\"2022-11-21T15:12:19+00:00\",\"description\":\"Building an Elasticsearch cluster can be as easy as assembling an Ikea Billy bookshelf and sizing as complex as engineering a 24 hours of Le Mans race car. In this article series we will put a spotlight on what has to be taken into account when molding clusters.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#primaryimage\",\"url\":\"https:\/\/www.inovex.de\/wp-content\/uploads\/2020\/03\/high-throughput-elastiocsearch-clusters.png\",\"contentUrl\":\"https:\/\/www.inovex.de\/wp-content\/uploads\/2020\/03\/high-throughput-elastiocsearch-clusters.png\",\"width\":1920,\"height\":1080,\"caption\":\"Shards hiding the elasticsearch logo\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.inovex.de\/de\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Planning High Throughput Elasticsearch Clusters: Part 1\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.inovex.de\/de\/#website\",\"url\":\"https:\/\/www.inovex.de\/de\/\",\"name\":\"inovex GmbH\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.inovex.de\/de\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.inovex.de\/de\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.inovex.de\/de\/#organization\",\"name\":\"inovex GmbH\",\"url\":\"https:\/\/www.inovex.de\/de\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png\",\"contentUrl\":\"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png\",\"width\":1921,\"height\":1081,\"caption\":\"inovex GmbH\"},\"image\":{\"@id\":\"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/inovexde\",\"https:\/\/x.com\/inovexgmbh\",\"https:\/\/www.instagram.com\/inovexlife\/\",\"https:\/\/www.linkedin.com\/company\/inovex\",\"https:\/\/www.youtube.com\/channel\/UC7r66GT14hROB_RQsQBAQUQ\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.inovex.de\/de\/#\/schema\/person\/9c4c40eb2a68e82a5ce9faee6ef607ca\",\"name\":\"Daniel Chamot\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\/\/www.inovex.de\/de\/#\/schema\/person\/image\/ce8881302b8bc4d4ec83ed529d3a6d5e\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e50b526efb9166fcea17e9c466ead3cbca1b18241f3bf213a75f88327d8c8126?s=96&d=retro&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e50b526efb9166fcea17e9c466ead3cbca1b18241f3bf213a75f88327d8c8126?s=96&d=retro&r=g\",\"caption\":\"Daniel Chamot\"},\"url\":\"https:\/\/www.inovex.de\/de\/blog\/author\/dchamot\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Planning High Throughput Elasticsearch Clusters: Part 1 - inovex GmbH","description":"Building an Elasticsearch cluster can be as easy as assembling an Ikea Billy bookshelf and sizing as complex as engineering a 24 hours of Le Mans race car. In this article series we will put a spotlight on what has to be taken into account when molding clusters.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/","og_locale":"de_DE","og_type":"article","og_title":"Planning High Throughput Elasticsearch Clusters: Part 1 - inovex GmbH","og_description":"Building an Elasticsearch cluster can be as easy as assembling an Ikea Billy bookshelf and sizing as complex as engineering a 24 hours of Le Mans race car. In this article series we will put a spotlight on what has to be taken into account when molding clusters.","og_url":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/","og_site_name":"inovex GmbH","article_publisher":"https:\/\/www.facebook.com\/inovexde","article_published_time":"2020-03-09T07:50:54+00:00","article_modified_time":"2022-11-21T15:12:19+00:00","og_image":[{"width":1920,"height":1080,"url":"https:\/\/www.inovex.de\/wp-content\/uploads\/2020\/03\/high-throughput-elastiocsearch-clusters.png","type":"image\/png"}],"author":"Daniel Chamot","twitter_card":"summary_large_image","twitter_image":"https:\/\/www.inovex.de\/wp-content\/uploads\/2020\/03\/high-throughput-elastiocsearch-clusters-1024x576.png","twitter_creator":"@inovexgmbh","twitter_site":"@inovexgmbh","twitter_misc":{"Verfasst von":"Daniel Chamot","Gesch\u00e4tzte Lesezeit":"6\u00a0Minuten","Written by":"Daniel Chamot"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#article","isPartOf":{"@id":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/"},"author":{"name":"Daniel Chamot","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/person\/9c4c40eb2a68e82a5ce9faee6ef607ca"},"headline":"Planning High Throughput Elasticsearch Clusters: Part 1","datePublished":"2020-03-09T07:50:54+00:00","dateModified":"2022-11-21T15:12:19+00:00","mainEntityOfPage":{"@id":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/"},"wordCount":1216,"commentCount":0,"publisher":{"@id":"https:\/\/www.inovex.de\/de\/#organization"},"image":{"@id":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#primaryimage"},"thumbnailUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2020\/03\/high-throughput-elastiocsearch-clusters.png","keywords":["Cloud","Search"],"articleSection":["English Content","General","Infrastructure"],"inLanguage":"de","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/","url":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/","name":"Planning High Throughput Elasticsearch Clusters: Part 1 - inovex GmbH","isPartOf":{"@id":"https:\/\/www.inovex.de\/de\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#primaryimage"},"image":{"@id":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#primaryimage"},"thumbnailUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2020\/03\/high-throughput-elastiocsearch-clusters.png","datePublished":"2020-03-09T07:50:54+00:00","dateModified":"2022-11-21T15:12:19+00:00","description":"Building an Elasticsearch cluster can be as easy as assembling an Ikea Billy bookshelf and sizing as complex as engineering a 24 hours of Le Mans race car. In this article series we will put a spotlight on what has to be taken into account when molding clusters.","breadcrumb":{"@id":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/"]}]},{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#primaryimage","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/2020\/03\/high-throughput-elastiocsearch-clusters.png","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2020\/03\/high-throughput-elastiocsearch-clusters.png","width":1920,"height":1080,"caption":"Shards hiding the elasticsearch logo"},{"@type":"BreadcrumbList","@id":"https:\/\/www.inovex.de\/de\/blog\/high-throughput-elasticsearch-part-1\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.inovex.de\/de\/"},{"@type":"ListItem","position":2,"name":"Planning High Throughput Elasticsearch Clusters: Part 1"}]},{"@type":"WebSite","@id":"https:\/\/www.inovex.de\/de\/#website","url":"https:\/\/www.inovex.de\/de\/","name":"inovex GmbH","description":"","publisher":{"@id":"https:\/\/www.inovex.de\/de\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.inovex.de\/de\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Organization","@id":"https:\/\/www.inovex.de\/de\/#organization","name":"inovex GmbH","url":"https:\/\/www.inovex.de\/de\/","logo":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png","width":1921,"height":1081,"caption":"inovex GmbH"},"image":{"@id":"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/inovexde","https:\/\/x.com\/inovexgmbh","https:\/\/www.instagram.com\/inovexlife\/","https:\/\/www.linkedin.com\/company\/inovex","https:\/\/www.youtube.com\/channel\/UC7r66GT14hROB_RQsQBAQUQ"]},{"@type":"Person","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/person\/9c4c40eb2a68e82a5ce9faee6ef607ca","name":"Daniel Chamot","image":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/person\/image\/ce8881302b8bc4d4ec83ed529d3a6d5e","url":"https:\/\/secure.gravatar.com\/avatar\/e50b526efb9166fcea17e9c466ead3cbca1b18241f3bf213a75f88327d8c8126?s=96&d=retro&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e50b526efb9166fcea17e9c466ead3cbca1b18241f3bf213a75f88327d8c8126?s=96&d=retro&r=g","caption":"Daniel Chamot"},"url":"https:\/\/www.inovex.de\/de\/blog\/author\/dchamot\/"}]}},"_links":{"self":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/17485","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/users\/135"}],"replies":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/comments?post=17485"}],"version-history":[{"count":1,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/17485\/revisions"}],"predecessor-version":[{"id":39482,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/posts\/17485\/revisions\/39482"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/media\/18317"}],"wp:attachment":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/media?parent=17485"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/tags?post=17485"},{"taxonomy":"service","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/service?post=17485"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/coauthors?post=17485"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}