Over the weekend I found myself on a plane from Zurich to Chicago. While in Switzerland, I became aware of an issue related to a microservice I maintained in which it would stop serving responses and need to be restarted. I had an idea of what the problem might have been but didn’t want to risk making a hotfix right before I was to embark on a 16+ hour travel day. So instead I was mulling over the problem while sitting in row 44 seat B.
Soon after takeoff, my seat mate whipped out a portable DVD player and started watching Dragon Ball Z. He had one of those CD binders and it was evident that he had enough discs to last the entire flight. Over the next several hours, as I finished The Da Vinci Code (eh), my senses were frequently roused by the familiar sound effects of DBZ.
After I finished my book, I closed my eyes to reflect on what I had just read. I found myself instead fixating on the Dragon Ball Z battle emanating from my partner’s headphones. It was then that I realized that Dragon Ball Z would help me prove or disprove my hypothesis for what was going wrong with my microservice.
A few months back I stumbled upon an HTTP load testing tool called Vegeta. I suspected the problem with my microservice was that I wasn’t serving the uWSGI app using enough threads and–how could I forget the name–I figured it was time to give Vegeta a shot.
First, I had to install Go since Vegeta is written in Go. After that, I wrote a query similar to this one:
echo "GET http://localhost:8080/api/v0/instances" | vegeta attack -duration=2s
I ran it and I was immediately able to recreate the issue. I confirmed my hypothesis using uwsgitop, which is a top-like interface just for uWSGI servers. When I ran Vegeta, the uWSGI cpu usage meter lit up. I configured uWSGI to use a few additional cores and the issue was mitigated. Okay. Good blog post. See you next time.
After establishing that Vegeta sure as hell beats custom bash scripts with curl requests for load testing, I started to wonder if there was another use for it, aside from DoSing the dev environment of my arch nemesis in the office. I decided to use Vegeta to test autoscaling policies.
Auto-scaling wasn’t prevalent when cloud services were first gaining traction roughly ten years ago. Today, it is considered one of the most important features of any cloud production environment, especially for consumer-facing apps. While auto-scaling offers an amount of resilience that is hard to emulate when self-hosting, it also facilitates significant cost savings by ensuring that users only pay for resources that they need and use.
I planned to showcase one of my personal AWS setups in this article, but it would be more appropriate to do that in a separate post. When it comes time to configure scalable cloud architecture, you need a reliable way of testing the site (and the scaling policies themselves) under load. There are a few tools out there–basically anything transcending a bash script sending parallel cURL requests has merit. In the past I’ve used siege, probably because the guides I followed at the time recommended it (and, yes, I easily could have used it for the problem above but I took the airplane situation as a sign that it was time to try Vegeta).
When testing Vegeta and Siege side-by-side, the core functionality appears to be the same. The tools are similar because they both solve the same problem. However, Siege is written in C and Vegeta is written in Go. Golang attracts criticism because it’s hip and why-did-Google-have-to-make-a-new-language, but Vegeta clearly demonstrates its merits. There is nothing Pythonic about its performance–Vegeta performs as well as Siege. Vegeta outclasses Siege with its reporting features and ability to be extended as a library for custom tests. In addition, distributed testing is easy using pdsh.
Vegeta is a tool that has piqued my interest as a leading open-source HTTP load testing tool. As I continue to use it at work and with my personal projects while improving my rudimentary understanding of Go, I look forward to writing about it in more detail and eventually contributing to its development.