Over the last 5 years, I've worked on dozens of Heroku-deployed applications for clients. Heroku always had this one weird feature that, initially, I found either annoying or just a small oddity, but, over time, I've learned that it's extremely important for how Heroku wants you to approach deployment. It's a feature that I think everyone running a Rails app should probably copy into their own setups.
I'm talking about Heroku's 24-hour restart.
Every 24 hours, Heroku initiates a restart of all of your dynos. Usually this happens during the middle of the night, USA time. It's a super simple system. Here's how Heroku explains it:
> Dynos are also restarted (cycled) at least once per day to help maintain the health of applications running on Heroku. Any changes to the local filesystem will be deleted. The cycling happens once every 24 hours (plus up to 216 random minutes, to prevent every dyno for an application from restarting at the same time)
I love the little 216-minute random counter as well. I wonder how it became 216 minutes, but that's neither here nor there.
Why is this feature (seemingly a limitation) so powerful?
It enforces the idea that the server is ephemeral.
This one simple feature forces you to treat a server as something that can be thrown away, possibly at any moment. Maybe in the next minute, this server could be gone!
Because Heroku spins up an entirely fresh machine every time it restarts a dyno, this also enforces your application to not rely solely on the local filesystem as a source of truth. You can use the filesystem as scratch space, sure, but don't expect anything to be there tomorrow morning.
Here are some bad, fragile deployment designs that are impossible as soon as you restart your servers every 24 hours:
- Running a single server which is a single point of failure for several downstream consumers.
- Having an app with an extremely slow startup time (~5 minutes of downtime every day? Nope!)
Here are some problems you tend not to have as soon as you restart your servers every 24 hours:
- Memory issues finally get a cap on how bad they can get (since things are restarted at least every 24 hours)
- Your entire deployment works correctly with SIGTERM (since it's happening every 24 hours!)
- And, if you actually spin up a new VPS/container/whatever at the same time, it's impossible to have a "snowflake server", because the entire setup must be reproducibly built every 24 hours.
Treating the underlying server as disposable allows you to do all kinds of neat deployment tricks, like using much-cheaper "spot" instances instead of "reserved" capacity (if your servers have fast startup/shutdown, spot instances work great).
All in all, regularly scheduled restarts lead to more robust deployments. If you're not doing it already, I suggest you do.
-Nate
|
|
|
|