Ruby's Global Method Cache
/Hey, folks! Lately I've been exploring Ruby environment settings and how much they can help (or not) your app speed. I feel like I've already hit most of the major tuning knobs on Ruby at one point or another... But let's look at one I haven't yet: Ruby's global method cache. What is it? How do you set it? How much speed does it give?
What's the Global Method Cache?
When you use a particular method, Ruby has to figure out what classes and/or modules and/or refinements define it, and which one to use in that particular location. It's a much more involved process than you'd think, especially with how Ruby handles constants and scope. In a lot of cases you can figure out what defines that method once and keep using the lookup that you did the first time - it's slow to re-run, so we don't.
There are two ways Ruby saves those lookups: the inline method cache, and the global method cache. After I explain what they are, we'll talk about the global method cache.
The inline method cache lives at a specific call site. It is "inline" in the sense that it's cached in your Ruby code where you call the method. That seems simple and sane. When it works, the global method cache doesn't get used - the lookup happens the first time the code is hit, and gets reused afterward.
The global method cache is for cases where that doesn't work - method_missing, respond_to? and refinements are examples. In those cases, it's very unlikely that the same place in your code will always get the same answer for "what is the method here?" Here's how Pat Shaughnessy puts it:
“Depending on the number of superclasses in the chain, method lookup can be time consuming. To alleviate this, Ruby caches the result of a lookup for later use. It records which class or module implemented the method that your code called in two caches: a global method cache and an inline method cache.
Ruby uses the global method cache to save a mapping between the receiver and implementer classes.
The global method cache allows Ruby to skip the method lookup process the next time your code calls a method listed in the first column of the global cache. After your code has called Fixnum#times once, Ruby knows that it can execute the Integer#times method, regardless of from where in your program you call times .”
There are a fixed number of entries in the global method cache - by default, 2048 of them. A Shopify engineer finds that gives a 90%+ hit rate even for a really huge Rails app, so that's not bad.
You can set the number of entries, but only to a power of two, with the environment variable RUBY_GLOBAL_METHOD_CACHE_SIZE. The default is 2048, so you'll normally want to go up from there, not down. Each cache entry is 40 bytes. So the default cache uses about 80kb, and each time you double the number of entries, you double the size. So Shopify's setting of 128k entries at 40 bytes/entry would use about 2.5 megabytes of memory.
How's the Speed?
I write and maintain Rails Ruby Bench, a highly-concurrent Ruby benchmark based on Discourse, a large real-world Rails application. I do a lot of checking Ruby and Rails speed using it. And today I'll do that with Ruby's global method cache.
Discourse isn't as huge as Shopify's Rails app - few Rails apps are. Which means it may not need to increase the cache size as badly. But it certainly has far more possible cache entries than the default 2048. So it's a pretty good indicator of how much a mid-size Rails app benefits from the cache size increase.
Long-time readers will be expecting a pretty graph here, and I have bad news for them: the difference in speed when adjusting the cache size is so small that any reasonable way to graph it makes it look like they're identical - which they nearly are. Here are the results as a table:
| RUBY_GLOBAL_METHOD_CACHE_SIZE | Mean req/sec | Std deviation | Speedup vs Default |
|---|---|---|---|
| 1024 | 155.3 | 1.7 | -1% |
| 2048 | 156.8 | 1.5 | 0% |
| 4096 | 158.3 | 2.3 | 1% |
| 8192 | 159.5 | 3.2 | 1.7% |
| 16384 | 160.3 | 3.4 | 2.2% |
So as you can see, my smaller number of cached entries are... Hm. If I check that Shopify article... They actually only claimed to get about 3% faster results. So my own results are directly in line with theirs. I see a tiny speedup, in return for a very small amount of memory.
So... Is It a Good Idea?
I don't see any harm in using this. But for most users, I don't think a savings of 2%-3% is worth bothering about. And that's assuming your Rails app is fairly large. I would expect a smaller app, or a non-Rails app, to gain very little or even nothing at all.
In most cases, I think Ruby's global method cache does a great job and doesn't require adjustment.
But now you know how to check!