In the previous blog post we described the basics and components of Mesos. Now we want to show you how to run tasks/applications on your Mesos cluster with Marathon, an init-system for Mesos built and maintained by Mesosphere. Read on for step-by-step instructions.
Mesosphere is a company that built its product DCOS (Datacenter Operating System) around Mesos to facilitate the use Mesos. They also put much effort into many open source projects around Mesos and Mesos itself. But I divert, back to Marathon: It supports cgroups or Docker applications to run your tasks. A pretty nice thing is that Marathon provides high availability by default, thanks to zookeeper. Also you can secure Marathon with SSL and basic auth. When you want to work with Marathon you need to install it, e.g. from the package sources of Mesosphere. If you want to try out the examples given in this blog post I suggest you use Vagrant.
When your Machine is up you should see the Mesos Web UI at http://192.168.33.10:5050 and on http://192.168.33.10:8080 you should see the Marathon Web-UI. You can start tasks now over the Web UI or (what I would recommend) using the API.
Let’s start with a very simple task that says “Hello Mesos”:
- The id describes the name of the application,
- “cmd” describes the command we want to run. If we want to run anything other than bash commands we have to ensure that the program is installed on the slave (or the slave can fetch the executable),
- the CPU and MEM sections are self-describing; you can define how many resources the task will allocate,
- in the last section we tell marathon how many tasks should be started, in the example we start only 1 instance.
Next we can pass this configuration to Marathon using the API:
When you now go back to the Marathon Web UI you will see that there is one application running which started 1/1 tasks. Maybe you notice that the health bar is grey this is because we didn’t define any health checks. Had we done so we would get a green health bar (if the health checks succeed). On the Mesos UI you can see the task under active tasks. Sadly, when we are running the vagrant setup we can’t dig into the sandbox. Normally you would be able to fetch the stdout and stderr from the web console which is pretty cool if you have to debug stuff and you don’t even need to log into the machine.
Marathon is focused on long running tasks. So if you let the task count from 0 to 25 instead of saying “Hello Mesos” the whole time you will notice that marathon ensures that every time the specified number of instances are running.
Scaling applications with Marathon
When you want to scale out your application you can use marathon to scale the number of instances. There are two ways to do this: The simple one is going to the Marathon web UI, clicking on the task, then clicking on scale and setting the desired number of tasks. After you press enter marathon starts all tasks. Another way is to change the number of instances inside the configuration file and then send it to the API.
Note that instead of POST you have to use PUT and you have to specify the app ID. When you go back to the Marathon web UI you can open the task and then go to configuration to see a version history of configurations. This is useful if you want to see what has been changed since the last deployment. Of course you can also run Docker with Marathon. Marathon supports nearly all functions you know from the command line tool.
When you run a distributed system you always have the challenge that the different tasks/services have to find each other. That’s where Mesos-DNS comes into play. Mesos-DNS generates DNS entries for your Mesos cluster (not only for tasks started by Marathon!). When you run Marathon tasks it is recommended that you also run Bamboo. Bamboo automatically configures HAProxy when you deploy web services with Marathon.
There are is so much more you can do with Marathon. My favorite example is that you can start another framework with Marathon to implement a simple framework deployment. From the perspective of Marathon another framework is just an application it has to start. If you want to fiddle around with Marathon you can find some examples on github and of course on the official Marathon web site.
Some other solutions to start applications on Mesos are:
If you have questions feel free to comment! If you liked this article we’d also appreciate you sharing it. Either way – stay tuned for the final part of our series on Mesos next week.
Get in touch
Looking for a change? We’re hiring BigData Systems Engineers skilled in Hadoop (Hortonworks), Flume, Spark, Flink, Hive and Cloudera. Apply now!