Copy
You're reading the Speedshop newsletter on Ruby on Rails performance by Nate Berkopec.

Bummed out by all the conference closures? Do you and your team need something to do while working from home? For large groups, you can book my performance training by appointment.

What makes an application thread-safe? Can I switch to Puma?

As a maintainer of Puma, I hear a lot of fear about thread-safety. "Is it safe for me to switch to Puma?" "Is my app thread-safe?"

These questions only increased as Puma became the default app server for Rails and the recommended app server at Heroku. Organizations like Gitlab are seeing big memory savings of 30% or more when switching from Unicorn because of Puma's more memory-efficient threaded model.

Thread-safety can be a confusing topic, so I get it. To understand thread-safety in a Ruby application, we need to talk about how Puma works.

In all operating systems, there are processes and threads. Processes are a bit like a shared workspace, or a house for all of our threads to live in. Processes contain all of the data (memory), open files and sockets, and other resources required to run code. However, they don't really run code themselves. That's what our threads do.

Processes have one or many threads. The threads are the things that actually "run" your code. Think of a thread as a context of execution - it contains a stack (where we're at in the program), and not much else. So, threads run our code, and processes have one or many of these. Back in the 90's, processes really one had one thread so threads and processes were the same thing. That's changed since then.

For every request it gets, a Puma process hands that request to a waiting thread in the Puma thread-pool. That one thread will then basically run `YOUR_APP.call(the_rack_environment)`, returning the result back to Puma when its done and calculated the correct response. So, there's a relationship here of one thread to one request. Only one thread will ever work with that request, multiple threads don't work on the same request.

The "multi-threaded" part of Puma comes from processing multiple requests at the same time in separate threads. So if two requests come in at the same time, Puma assigns each request to a separate thread in its thread-pool. The thread-safety problems occur when those two requests are executing concurrently. So, while it's just one thread for one request, many threads are sharing the same global singleton Rack application object (e.g. Rails.application).

So, here's the first point: you can switch any application, even a thread-unsafe one, to Puma today, but just run Puma with one thread in the thread-pool. When running this way, Puma works quite similarly to Unicorn. No thread-safety bugs in your app are possible when running with 1 thread in the threadpool, because requests are never processed concurrently.

So, what kinds of problems can happen when 2 threads are running at the same time and processing requests at the same time? Most thread-safety issues in Rails applications are in one of two categories:

Constants, globals, and class variables. Any piece of data visible to any part of the program, if written to or changed in some way, will usually create thread-safety bugs. Now, this may seem obvious at first glance: if one thread changes the value of SOME_CONSTANT, then yes, other threads may not see that right away and may read the old value or write the wrong one at the same time as the other thread. However, where this trips people up is in state accessed _indirectly_ via a constant. A big one that trips people up is database connections stored in globals or constants. At one time, it was popular to access Redis databases directly through one database connection, stored in a variable like REDIS or $REDIS. If one of your request threads starts using this connection, and then another thread starts using the same connection concurrently, Bad Things Will Happen.

Rack middleware, or "my clients are receiving someone else's responses". We've see this one a few times on the Puma issue tracker. It has to do with the way that Rails and other frameworks build their middleware stack. All of your request threads share the same middleware stack, just like they share the Rack application object itself (Rails.application). When booting your app, Rails instantiates one instance of each middleware in your stack, and those instances are all shared by all of your request threads. This is unlike controllers, for example, which are instantiated fresh for every request, so they're never shared between threads. So, if one thread changes state in a middleware, like in an instance variable, then another thread will probably see that unclean state and cause a bug. Unsafe rack middleware can be detected and fixed with rack-freeze. I'm currently evaluating adding something like that to Puma itself.

So, you can usually just statically analyze your application and look for these two problems. Add rack-freeze, look at any constants you're defining, and you're on your way.

Ideally, you would test for thread safety too. Unfortunately, most people use RSpec, which, unlike Minitest, will only run single-threaded. Minitest can multithread your tests, making it more like that you'll catch thread-safety bugs. System and integration tests also generally run serially rather than concurrently, meaning they won't catch thread bugs either.

In my experience, if you can take care of the two problems outlined above, thread-safety is a very minor issue. Most people never experience a problem - which is why lots of really big application run in multi-threaded Puma setups. Give it a shot!

Until next week,

-Nate
You can share this email with this permalink: https://mailchi.mp/railsspeed/can-i-switch-to-puma-thread-safety-rack-constants-and-more?e=[UNIQID]

Copyright © 2020 Nate Berkopec, All rights reserved.


Want to change how you receive these emails?
You can update your preferences or unsubscribe from this list.

Email Marketing Powered by Mailchimp