{"id":31864,"date":"2021-09-29T12:18:26","date_gmt":"2021-09-29T11:18:26","guid":{"rendered":"https:\/\/www.inovex.de\/?post_type=events&#038;p=31864"},"modified":"2021-10-19T10:41:00","modified_gmt":"2021-10-19T09:41:00","slug":"online-meetup-databricks-orchestrierung-mit-apache-airflow","status":"publish","type":"events","link":"https:\/\/www.inovex.de\/de\/news-events\/events\/online-meetup-databricks-orchestrierung-mit-apache-airflow\/","title":{"rendered":"[Online Meetup] Databricks-Orchestrierung mit Apache Airflow"},"content":{"rendered":"<p>Hey Tech Heads,<\/p>\n<p>unter Data Scientists und Engineers wird Databricks als Cloud Data Platform immer beliebter. Auf den Databricks Clustern k\u00f6nnen Entwickler:innen mit wenig Aufwand ihre Spark ETL-Strecken ausf\u00fchren oder analytisch und explorativ im bekannten Notebook-Format arbeiten.<\/p>\n<p>Gleicherma\u00dfen ist Apache Airflow aus gro\u00dfen Projekten mit komplexen Workflows nicht mehr wegzudenken:<\/p>\n<p>Die OpenSource Plattform hilft uns, einzelne Tasks zu gerichteten, azyklischen Graphen (DAGs) zu verbinden und diese \u00fcber einen festgelegten Zeitplan oder als Reaktion auf externe Events zu triggern. W\u00e4hrend Databricks zur Zeit ein simples Scheduling einzelner Jobs erm\u00f6glicht, ist eine kollaborative Nutzung der Tools f\u00fcr die meisten Use Cases erstrebenswerter, weshalb bereits ein Databricks Provider Package f\u00fcr Airflow-Nutzer:innen grundlegende Operatoren bereitstellt.<\/p>\n<p>Wir zeigen euch in unserem Meetup, wie wir noch viele weitere Funktionalit\u00e4ten von Databricks in benutzerdefinierten Airflow-Operatoren nutzen k\u00f6nnen, z. B.:<\/p>\n<ul>\n<li>DBFS-File-Sensoren die einen DAG triggern<\/li>\n<li>Abruf von Notebook-Ergebnissen \u00fcber xcom<\/li>\n<li>Ansteuern der Databricks Cluster API<\/li>\n<\/ul>\n<p>Wir freuen uns \u00fcber Interessierte mit oder ohne Vorwissen, die Lust haben, mehr \u00fcber die Orchestrierung Databricks via Airflow zu erfahren.<\/p>\n<p>Infos und Anmeldung:<\/p>\n<p>Das Meetup wird in deutscher Sprache durchgef\u00fchrt.<\/p>\n<p>Anmeldung bitte \u00fcber eine unserer Meetup-Seiten oder unter\u00a0<a title=\"Opens internal link in current window\" href=\"mailto:meetups@inovex.de\">meetups@inovex.de<\/a><\/p>\n<p><a href=\"https:\/\/www.meetup.com\/de-DE\/inovex-karlsruhe\/events\/281088331\/\" target=\"_blank\" rel=\"noopener\">inovex Meetup Karlsruhe<\/a><\/p>\n<p><a href=\"https:\/\/www.meetup.com\/de-DE\/inovex-munich\/events\/281087635\/\" target=\"_blank\" rel=\"noopener\">inovex Meetup M\u00fcnchen<\/a><\/p>\n<p><a href=\"https:\/\/www.meetup.com\/de-DE\/inovex-cologne\/events\/281088309\/\" target=\"_blank\" rel=\"noopener\">inovex Meetup K\u00f6ln<\/a><\/p>\n<p><a href=\"https:\/\/www.meetup.com\/de-DE\/inovex-Meetup-Hamburg\/events\/281088259\/\" target=\"_blank\" rel=\"noopener\">inovex Meetup Hamburg<\/a><\/p>\n<p>Wir freuen uns auf euch!<\/p>\n<p>Zur Durchf\u00fchrung des Online Meetups verwenden wir \u201eZoom\u201c. Hinweise zur Datenverarbeitung findet ihr unter <a href=\"https:\/\/www.inovex.de\/de\/datenschutz\/\">https:\/\/www.inovex.de\/de\/datenschutz\/<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In diesem Online Meetup stellen wir euch vor, wie mittels Apache Airflow die Orchestrierung von Databricks optimal gelingt.<\/p>\n","protected":false},"featured_media":31597,"parent":0,"template":"","events_types":[441],"service":[],"class_list":["post-31864","events","type-events","status-publish","has-post-thumbnail","hentry","events_types-meetup"],"acf":{"start_date":"20211019","finish_date":"20211019","time":"12:00:00","finish_time":"13:00:00","location":"Online","organizer":"inovex","file":"","hubspot_signups":false,"hubspot_signups_text":"","add_speakers":false,"speakers_headline":"Speaker:innen","speakers":null,"contact":null,"contact_heading":"","contact_text":""},"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.6 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>[Online Meetup] Databricks-Orchestrierung mit Apache Airflow - inovex GmbH<\/title>\n<meta name=\"description\" content=\"In diesem Online Meetup stellen wir euch vor, wie mittels Apache Airflow die Orchestrierung von Databricks optimal gelingt.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.inovex.de\/de\/news-events\/events\/online-meetup-databricks-orchestrierung-mit-apache-airflow\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"[Online Meetup] Databricks-Orchestrierung mit Apache Airflow - inovex GmbH\" \/>\n<meta property=\"og:description\" content=\"In diesem Online Meetup stellen wir euch vor, wie mittels Apache Airflow die Orchestrierung von Databricks optimal gelingt.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.inovex.de\/de\/news-events\/events\/online-meetup-databricks-orchestrierung-mit-apache-airflow\/\" \/>\n<meta property=\"og:site_name\" content=\"inovex GmbH\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/inovexde\" \/>\n<meta property=\"article:modified_time\" content=\"2021-10-19T09:41:00+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.inovex.de\/wp-content\/uploads\/managing-databricks-from-airflow.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"1080\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:image\" content=\"https:\/\/www.inovex.de\/wp-content\/uploads\/managing-databricks-from-airflow-1024x576.png\" \/>\n<meta name=\"twitter:site\" content=\"@inovexgmbh\" \/>\n<meta name=\"twitter:label1\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data1\" content=\"1\u00a0Minute\" \/>\n\t<meta name=\"twitter:label2\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data2\" content=\"Daniela Nebeling\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/news-events\\\/events\\\/online-meetup-databricks-orchestrierung-mit-apache-airflow\\\/\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/news-events\\\/events\\\/online-meetup-databricks-orchestrierung-mit-apache-airflow\\\/\",\"name\":\"[Online Meetup] Databricks-Orchestrierung mit Apache Airflow - inovex GmbH\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/news-events\\\/events\\\/online-meetup-databricks-orchestrierung-mit-apache-airflow\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/news-events\\\/events\\\/online-meetup-databricks-orchestrierung-mit-apache-airflow\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/managing-databricks-from-airflow.png\",\"datePublished\":\"2021-09-29T11:18:26+00:00\",\"dateModified\":\"2021-10-19T09:41:00+00:00\",\"description\":\"In diesem Online Meetup stellen wir euch vor, wie mittels Apache Airflow die Orchestrierung von Databricks optimal gelingt.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/news-events\\\/events\\\/online-meetup-databricks-orchestrierung-mit-apache-airflow\\\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.inovex.de\\\/de\\\/news-events\\\/events\\\/online-meetup-databricks-orchestrierung-mit-apache-airflow\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/news-events\\\/events\\\/online-meetup-databricks-orchestrierung-mit-apache-airflow\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/managing-databricks-from-airflow.png\",\"contentUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/managing-databricks-from-airflow.png\",\"width\":1920,\"height\":1080,\"caption\":\"Abstract Illustration of Airflow Managing the Databricks Platform\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/news-events\\\/events\\\/online-meetup-databricks-orchestrierung-mit-apache-airflow\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Events\",\"item\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/news-events\\\/events\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"[Online Meetup] Databricks-Orchestrierung mit Apache Airflow\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#website\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\",\"name\":\"inovex GmbH\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#organization\",\"name\":\"inovex GmbH\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/inovex-logo-16-9-1.png\",\"contentUrl\":\"https:\\\/\\\/www.inovex.de\\\/wp-content\\\/uploads\\\/2021\\\/03\\\/inovex-logo-16-9-1.png\",\"width\":1921,\"height\":1081,\"caption\":\"inovex GmbH\"},\"image\":{\"@id\":\"https:\\\/\\\/www.inovex.de\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/inovexde\",\"https:\\\/\\\/x.com\\\/inovexgmbh\",\"https:\\\/\\\/www.instagram.com\\\/inovexlife\\\/\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/inovex\",\"https:\\\/\\\/www.youtube.com\\\/channel\\\/UC7r66GT14hROB_RQsQBAQUQ\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"[Online Meetup] Databricks-Orchestrierung mit Apache Airflow - inovex GmbH","description":"In diesem Online Meetup stellen wir euch vor, wie mittels Apache Airflow die Orchestrierung von Databricks optimal gelingt.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.inovex.de\/de\/news-events\/events\/online-meetup-databricks-orchestrierung-mit-apache-airflow\/","og_locale":"de_DE","og_type":"article","og_title":"[Online Meetup] Databricks-Orchestrierung mit Apache Airflow - inovex GmbH","og_description":"In diesem Online Meetup stellen wir euch vor, wie mittels Apache Airflow die Orchestrierung von Databricks optimal gelingt.","og_url":"https:\/\/www.inovex.de\/de\/news-events\/events\/online-meetup-databricks-orchestrierung-mit-apache-airflow\/","og_site_name":"inovex GmbH","article_publisher":"https:\/\/www.facebook.com\/inovexde","article_modified_time":"2021-10-19T09:41:00+00:00","og_image":[{"width":1920,"height":1080,"url":"https:\/\/www.inovex.de\/wp-content\/uploads\/managing-databricks-from-airflow.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_image":"https:\/\/www.inovex.de\/wp-content\/uploads\/managing-databricks-from-airflow-1024x576.png","twitter_site":"@inovexgmbh","twitter_misc":{"Gesch\u00e4tzte Lesezeit":"1\u00a0Minute","Written by":"Daniela Nebeling"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.inovex.de\/de\/news-events\/events\/online-meetup-databricks-orchestrierung-mit-apache-airflow\/","url":"https:\/\/www.inovex.de\/de\/news-events\/events\/online-meetup-databricks-orchestrierung-mit-apache-airflow\/","name":"[Online Meetup] Databricks-Orchestrierung mit Apache Airflow - inovex GmbH","isPartOf":{"@id":"https:\/\/www.inovex.de\/de\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.inovex.de\/de\/news-events\/events\/online-meetup-databricks-orchestrierung-mit-apache-airflow\/#primaryimage"},"image":{"@id":"https:\/\/www.inovex.de\/de\/news-events\/events\/online-meetup-databricks-orchestrierung-mit-apache-airflow\/#primaryimage"},"thumbnailUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/managing-databricks-from-airflow.png","datePublished":"2021-09-29T11:18:26+00:00","dateModified":"2021-10-19T09:41:00+00:00","description":"In diesem Online Meetup stellen wir euch vor, wie mittels Apache Airflow die Orchestrierung von Databricks optimal gelingt.","breadcrumb":{"@id":"https:\/\/www.inovex.de\/de\/news-events\/events\/online-meetup-databricks-orchestrierung-mit-apache-airflow\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.inovex.de\/de\/news-events\/events\/online-meetup-databricks-orchestrierung-mit-apache-airflow\/"]}]},{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/de\/news-events\/events\/online-meetup-databricks-orchestrierung-mit-apache-airflow\/#primaryimage","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/managing-databricks-from-airflow.png","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/managing-databricks-from-airflow.png","width":1920,"height":1080,"caption":"Abstract Illustration of Airflow Managing the Databricks Platform"},{"@type":"BreadcrumbList","@id":"https:\/\/www.inovex.de\/de\/news-events\/events\/online-meetup-databricks-orchestrierung-mit-apache-airflow\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.inovex.de\/de\/"},{"@type":"ListItem","position":2,"name":"Events","item":"https:\/\/www.inovex.de\/de\/news-events\/events\/"},{"@type":"ListItem","position":3,"name":"[Online Meetup] Databricks-Orchestrierung mit Apache Airflow"}]},{"@type":"WebSite","@id":"https:\/\/www.inovex.de\/de\/#website","url":"https:\/\/www.inovex.de\/de\/","name":"inovex GmbH","description":"","publisher":{"@id":"https:\/\/www.inovex.de\/de\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.inovex.de\/de\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Organization","@id":"https:\/\/www.inovex.de\/de\/#organization","name":"inovex GmbH","url":"https:\/\/www.inovex.de\/de\/","logo":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/","url":"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png","contentUrl":"https:\/\/www.inovex.de\/wp-content\/uploads\/2021\/03\/inovex-logo-16-9-1.png","width":1921,"height":1081,"caption":"inovex GmbH"},"image":{"@id":"https:\/\/www.inovex.de\/de\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/inovexde","https:\/\/x.com\/inovexgmbh","https:\/\/www.instagram.com\/inovexlife\/","https:\/\/www.linkedin.com\/company\/inovex","https:\/\/www.youtube.com\/channel\/UC7r66GT14hROB_RQsQBAQUQ"]}]}},"_links":{"self":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/events\/31864","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/events"}],"about":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/types\/events"}],"version-history":[{"count":3,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/events\/31864\/revisions"}],"predecessor-version":[{"id":32290,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/events\/31864\/revisions\/32290"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/media\/31597"}],"wp:attachment":[{"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/media?parent=31864"}],"wp:term":[{"taxonomy":"events_types","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/events_types?post=31864"},{"taxonomy":"service","embeddable":true,"href":"https:\/\/www.inovex.de\/de\/wp-json\/wp\/v2\/service?post=31864"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}