Chat with your data: Unternehmensdaten als Basis für einen eigenen KI-Assistenten nutzen.
Zum Angebot 
The logos of Rust vs. Go vs. C# vs. Kotlin on clean cut background colors.
Backend

Battle Of The Backends: Rust vs. Go vs. C# vs. Kotlin

Lesezeit
12 ​​min

Over the last few years, there has been a lot of movement in the backend scene. A lot of programming languages and frameworks gained people’s interest and mine too. So I decided to write this article about languages and frameworks I have stumbled upon and wanted to compare.

Back in July 2022 when I started at inovex, I only knew two main programming languages: C# and Python. Since I worked for a few years with both languages and corresponding frameworks like .NET, Flask, and FastAPI to build RESTful services I have seen a lot of nice features, but also seen the downsides.

Besides my projects at inovex, I got the opportunity to learn a lot of new stuff. At first glance, I decided to learn Go as a programming language, since I wanted to do it for a long time. In December 2022 started a new Advent of Code season, where I did some tasks in Go but also stumbled upon Kotlin as a programming language. So after a whole day of work, I decided to learn a bit of Kotlin as well via the learning path in the JetBrains Academy. But there was still something more in my mind: What about Rust? Between Christmas and New Year’s Eve, I worked myself through the Rust Book as well as a little of the Comprehensive Rust course developed by the Android Team at Google.

I guess you can see a pattern here: I like to learn new stuff and want to create little RESTful services in order to get a feeling of the language and at least one popular framework. But after doing all this, I came up with a question: How do they perform against each other? I hope this article will give you at least a little insight!

Disclaimer

This will not be a scientific test, this article is more about getting a feeling for a rough performance indicator for the combination of programming language and framework since I am not an expert on all the used languages/frameworks. This article is also about my own opinion and insight during these tests, results may differ if you run the benchmarks on your own or use other toolings.

Prerequisites for comparing the languages/frameworks

Before starting with the benchmark comparing Rust vs. Go vs. C# vs. Kotlin, I needed to set some prerequisites and answer some questions for myself:

  • What are the key indicators I want to know about?
  • How can I guarantee the same environment and behavior for each service?
  • What tool do I use to measure the key indicators?
  • What should I do with the results? (I guess you are reading the answer already)

Key indicators

For the indicators, I wanted to focus on just a few things: requests per second and average response time. Sound not too much at first, but I wanted to know something else: What about CPU and memory consumption? From an ecological and economical point of view this was my main point of interest: If I have a service, which consumes less CPU and memory than the others, that way I can run more services of the same type (and size) on one single instance of hardware and maybe even faster and more efficiently.

Environment and behavior

In order to make my results more comparable, I decided to use Docker for the services, since I know Docker quite well. So I built every single service with a multistep Dockerfile, where the services are built in the first step and then executed on the same base image. But here I have to add a little remark: In order to run the services written in Kotlin and C#, I had to install the corresponding runtimes for Java and .NET on top of the base image. I also did set resource limits on the containers to make sure that every service gets the same amount of resources during the tests and to prevent a potential overload of the docker host.

The key specs for the docker images:

  • debian:11.6-slim as the base image
  • Limited to 4 vCPUs
  • Limited to 1GB of memory usage
  • All containers keep running during the tests

For the hardware, I have created an instance in our own inovex Cloud which I can use to build and run my containers on.

The specs for the instance are:

  • Ubuntu Server 22.04 as the host operating system
  • Docker Engine v.20.10.22
  • 4 vCPUs
  • 12GB RAM

I have also created a second instance in the same subnet as the main instance so that I can also (mostly) eliminate network latency and hops. This machine acts as my client, which performs the load test on the main instance.

The specs for the client instance are:

  • Ubuntu Server 22.04 as the base operating system
  • 4 vCPUs
  • 8GB RAM

The network topology for both instances looks like this:

Network Topology

Both the benchmark and client instance are in the same subnet connected via gigabit ethernet interfaces and a < 1ms latency, whereas the client machine (benching-host) is also connected to the inovex internal network so I can connect via VPN to the machine. The benchmark instance (api-benchmark) has a public IP address and router, so I can build the corresponding docker images remotely from my machine on the host itself to make it more reproducible.

Next up I decided that each service gets 3 REST routes:

  1. The first route is a basic GET, which returns a JSON list of 3 items that are declared inside the application. This is used to eliminate other uncertainties like external databases of file system access. It will be the first route we are running our benchmark against and will also be our comparison to the other services to see that they behave the same.
  2. The next route is a POST, which takes no arguments or request body at all. I will let the service sleep a fixed amount of time (300 ms) to block the request for a little while and test the concurrent handling of requests inside the HTTP handler/framework.
  3. The third route is again a GET, functioning as a health check for a little UI I have written in order to see if all services are up and running.

For monitoring the CPU and memory usage, I am using the built-in command docker stats from the docker daemon itself, since it provides the most accurate and fastest metrics during the load test while keeping the overhead of monitoring quite low.

Last but not least, I had to choose a load testing tool that provides the measurements for the requests per second as well as the average response time. After a bit of research, I decided to use go-wrk as my tool of choice.

Overview of languages and frameworks

Language / VersionFramework / VersionWhy I used it
Rust / 1.66Rocket / 0.5.0-rc.2Was the first result when I searched for a backend framework in Rust which seemed easy to use
Go / 1.19.4Echo / 4.10.0Easy to use, uses the HTTP handler from stdlib
C# / 11.0.NET Web API / 7.0Microsoft's go-to standard to create Web APIs
C# / 11.0.NET Minimal API / 7.0Microsoft's new approach of creating APIs with less boilerplate code and more performance
Kotlin / 1.8.0Ktor / 2.2.1As a JetBrains advocate I wanted to test JetBrains' approach of a web framework based on Kotlin and Coroutines
Kotlin / 1.8.0Spring Boot / 3.0.1The "modern" all-time classic creating APIs with modern Kotlin language features

If you want to try it yourself or want to have a closer look at the source code, you can find all of the demo applications, as well as the UI and docker-compose file inside this repository: https://github.com/timoyo93/common-api-benchmarks

Testing scenario

As mentioned above, I used go-wrk as my benchmarking tool. It is a simple CLI tool that you can pass some arguments for testing. For both of my scenarios, I used the following arguments:

  • 50 concurrent connections
  • 60 seconds test duration
  • Request timeout after 10 seconds

During testing, I am writing the stats of the container where the test is performed on a local file, which I used to calculate the average CPU and memory usage based on the sum of all entries divided by the number of entries.

Also, after every test run I wait for 5 minutes and have a look at the CPU and memory usage again to see if there is a memory leak present or not. In the results, it is called Memory Usage (cooldown).

Enough of talking, let’s show some results!

Results

First run -> GET /employees

Language / FrameworkTotal requestsRequests per secondAverage response timeFastes requestSlowest requestNumber of timeouts
Rust / Rocket106927817927.632.79ms388.81µs29.94ms0
Go / Echo108451518178.042.75ms354.54µs32.90ms0
C# / .NET Web API87643114676.063.41ms458.88µs60.42ms0
C# / .NET Minimal API93692915699.063.18ms402.89µs74.22ms0
Kotlin / Ktor3496725835.448.50ms474.20µs202.46ms0
Kotlin / Spring Boot4576947647.176.54ms613.95µs370.16ms0
Language / FrameworkCPU usage (idle)CPU usage (average)CPU usage (peak)Memory usage (idle)Memory usage (average)Memory usage (peak)Memory usage (cooldown)
Rust / Rocket0.00%62.84%65.73%0.92MB3.04MB3.20MB2.73MB
Go / Echo0.00%60.20%66.07%2.02MB8.62MB9.29MB5.15MB
C# / .NET Web API0.01%64.24%79.38%41.96MB90.60MB98.57MB90.12MB
C# / .NET Minimal API0.01%53.06%58.15%41.09MB86.21MB87.14MB86.59MB
Kotlin / Ktor0.03%56.80%80.40%88.16MB122.15MB126.80MB122.20MB
Kotlin / Spring Boot0.05%77.85%97.42%106.10MB176.78MB193.90MB178.00MB

So the first run was a success. All services were capable of handling the requests while responding with „OK“ for the health check. But as we can see, both services written in Kotlin and the corresponding frameworks had the highest peak in CPU usage and consumed the most memory.

We can also see that the services written in Go and Rust are the most „lightweight“ of all services and were able to handle the most requests per second compared to the other services.

But this was just the warm-up, now let’s get to the real part.

Second run -> POST /employees

Language / FrameworkTotal requestsRequests per secondAverage response timeFastes requestSlowest requestNumber of timeouts
Rust / Rocket9917165.00303.02ms300.76ms317.02ms0
Go / Echo9950165.30302.48ms300.54ms311.17ms0
C# / .NET Web API9950165.45302.20ms295.72ms*370.72ms0
C# / .NET Minimal API9953165.45302.20ms295.97ms*324.31ms0
Kotlin / Ktor9900164.51303.93ms300.89ms455.94ms0
Kotlin / Spring Boot9950165.30302.48ms300.77ms397.33ms0

* The delay/sleep command in C# is not as accurate as in other languages because it uses the OS’s time resolution, therefore the fastest request was below 300ms. Since it is just around 4-5ms, I will let that count

Language / FrameworkCPU usage (idle)CPU usage (average)CPU usage (peak)Memory usage (idle)Memory usage (average)Memory usage (peak)Memory usage (cooldown)
Rust / Rocket0.00%0.87%1.22%2.73MB5.12MB5.70MB5.04MB
Go / Echo0.00%0.87%1.07%4.29MB8.54MB9.13MB5.93MB
C# / .NET Web API0.00%2.34%2.84%90.68MB93.33MB93.62MB93.52MB
C# / .NET Minimal API0.00%2.37%4.52%89.50MB91.67MB91.77MB91.45MB
Kotlin / Ktor0.04%4.30%18.50%122.30MB132.87MB134.20MB133.80MB
Kotlin / Spring Boot0.06%3.45%20.43%178.10MB187.74MB189.20MB184.80MB

This run was also a success but we cannot see much of a difference here, since we specified each service to wait 300ms on each request asynchronously. If you do the math right, there are a maximum of 10.000 requests possible in 60 seconds and around 167 requests per second (with 50 connections).

That means that all services were able to handle around 98 % of the requests in total. Overall my insight is that all frameworks are handling concurrency very well, but Kotlin and the Ktor and Spring Boot frameworks are still the ones using most of the resources (CPU and memory).

Conclusion

So this is it. If I had to nominate a winner, I think I would pick Go and the Echo Framework here, since it was the fastest service during testing, handles concurrency pretty well with the http handler from the stdlib (which the Echo framework is using), and also had quite low CPU and memory usage.

But would I say that all the other frameworks are bad? Definitely NOT.

I had quite a blast coding all of the services in each programming language and framework. And at the end of the day, it depends on personal preference and project or company-wide preferences or standards.

Personally, I made quite a move from C# and .NET into the Go and Rust ecosystem and quite enjoy it! I am looking forward to using Rust and Go more often in the future. But that does not mean that I will quit C# and .NET since it was my first experience in professional software development and I still like and use it.

I also hope that you have enjoyed reading this article. And keep your eyes open on either the repository or this blog, maybe there will be a second part.

17 Kommentare

  1. Nice article, I have a few questions / points, though:

    – Why does the kotlin-spring directory contain Java source code?
    – A (technical) explanation / interpretation of results would be nice. I’m not really familiar with Rust, does Rust reuse the created objects (if so, the comparison would be unfair to a certain degree) or is the garbage collection more efficient (compared to the other languages)? What is the explanation for the low memory footprint?
    – Would be interesting to see, what could be done by the developer to increase performance in each language / framework. For instance, in Kotlin one could use arrayOf() instead of listOf() which should be faster and increase throughput.

    1. First of all thanks for your feedback! I will answer the questions in the same order as asked:

      – I think the only Java source code used is the Thread.Sleep command, since I couldn’t find another sleep function that works in combination with Kotlin and Spring Boot but please correct me if there is a better way doing this
      – Rust does not use garbage collection, therefore the allocated memory get’s freed immediately once the objects get out of scope which also explains the low memory footprint
      – I like this idea, maybe I can do a followup on this in the future

    1. Thanks for the information. As I mentioned in my article, Rocket was the first framework I stumbled upon when I searched for a backend framework for Rust. Since it seemed very easy to use, I gave it a try. In future I will have a look at frameworks which are maintained.

    1. Thanks for your feedback! Could you tell me with which choices you don’t agree or what could have been done better?

  2. Great article and a conclusion I also had in many years of backend-development. All current frameworks and languages are great. It is hard to make a „wrong“ choice. Java/Kotlin might have the largest memory / cpu usage but your team might consist of Java developers. So it still would be a great choice for such a team.
    And it is great to see that the current .NET versions are also up to the task.
    Of course, as a service provider you might wanna have the lowest possible footprint to utilize as much as small VMs as possible – but for many application developers this could almost be considered an „edge case“ IMHO.
    I’m also learning GO at the moment and think it is really refreshing 🙂

  3. Nice article ! I would love to do everything in perl despite the market momentum leaning towards Python and Go.
    The choice of language really is a problem as we want to learn one language that fits all the bills. In reality, this never happens. Strangely, whether anyone likes it or not, perl is the best language ever created. The productivity you get with perl is really awesome. The trade-off is performance, more memory usage and lack of new modules for certain tasks. (Chat-J and online converters can improve this). If you can live this trade-off, Mojolicious is the best web framework out there that can make you happy since you could develop any backend service within the shortest time. Also, your expression matters as you could represent a lot in least possible lines. If your team could follow some good coding practices, perl can be very rewarding. Scale up is needed only if you grow with time and there are many approaches to address scale up.

Hat dir der Beitrag gefallen?

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert