Unified Configuration in Rails

A while back, I was exploring some ideas around a simplified and unified way to access configurable values in Rails applications.¹¹The rest of this post will primarily be an updated explanation of the idea and its strengths (convenience) and weaknesses (performance). The post in the Rails forums didn’t receive much interest, so the idea fell by the wayside until I was recently trying to help someone navigate how it all works.

I’ve also encountered a variety of other solutions that attempt to address some of these challenges. Personally, I’ve been hoping for a built-in way to do this in Rails without requiring gems for such a core and critical piece of functionality, but many of these look to be really good.

• An approach using the dry-types and dry-system gems by Pat Allan
• anyway_config
• superconfig
• rubyconfig
• dotenv

With all of that said, I certainly don’t think there’s one perfect answer for this, but there’s a lot of existing evidence that we collectively feel there’s room for improvement. With all that inspiration, surely there’s some potential for a built-in solution.

---

Rails provides great support for initializing configuration, secrets, credentials and environment variables, but I’ve always struggled a little with accessing the values. Not a deep struggle but some frequent friction on the differences between how credentials and configuration are exposed. Using ENV has always felt a little inelegant in some contexts as well.

Onto the idea…

As it stands now, looking at these side-by-side, despite similarity in use cases, there’s some consistency mixed in with just enough variation to make it easy to trip up. (Figure 1) While that’s not inherently a bad thing, it created an itch I wanted to scratch.

Rails.application.credentials.secret_key_baseRails.application.credentials.aws[:secret_key]Rails.configuration.settings[:public_key]Rails.configuration.settings[:domain][:name]ENV['ENV_VAR']ENV.fetch('ENV_VAR', 'Default')ENV.fetch('ENV_VAR') { 'Default' }

Ultimately, each of the approaches for accessing the values are simultaneously incredibly similar and just different enough to get wires crossed. So I thought it might be handy to have source-indifferent lookups that offered a more consistent interface for retrieving the values without having to think as much about nuances of where they were stored or how they’re supposed to be read. (Figure 2)

We’ll look at the code first, and then we’ll dive into the pros and cons that I’ve found based on some initial exploration and trying out this approach on a production Rails app.

Settings.optional(:key) # => Returns the valueSettings.optional(:key_one, :key_two) # => Returns the value of the nested keysSettings.optional(:missing) # => Returns nilSettings.required(:key) # => Returns the valueSettings.required(:key_one, :key_two) # => Returns the value of the nested keysSettings.required(:missing) # => Raises exception when a critical value is missingSettings.default(:key) { 'Default' } # => Returns the value and ignores the default.Settings.default(:missing) { 'Default' } # => Returns 'Default'

That provides the broad strokes of the idea, but I’ve got some working (and tested) code that shows more of the usage and versatility at the end of the post.

The benefits I’ve found from this approach…

1. The interface for retrieving the value is consistent regardless of the source of the value. I spend less time thinking about syntax, return values, and defaults.
2. Accessing the values focuses more on how they’ll be used in context rather than where or how the value is stored. In some cases, for instance, if a value is missing, the application can still run just fine. In other cases, if a value is missing (due to environment differences, typos, or whatever), it’s best to stop right away and draw attention to the fact that the value is missing. Otherwise, a nil configuration setting can be passed around in the code and not fail until it’s much less clear where the problem originated.
3. It’s a little more intention-revealing about the expectation for how the value will be used. Accessing a value via a required or optional method immediately conveys the significance of the value’s presence.
4. With defaults, it provides a consistent approach regardless of the return value for the setting. There’s no need to think about which approach for providing a default return value is appropriate. i.e. || or fetch(:key, 'Default') or fetch(:key) { 'Default' }.
5. In cases where production/staging rely on Environment Variables like Heroku’s Config Vars, the retrieval can work independently so that test/development can explicitly ignore the value or find it via configuration instead.
6. It provides a home for some richer exceptions to help catch configuration issues earlier and reduce debugging second-order effects of configuration value issues.
7. If all else fails the existing ways of accessing values would still be available.

The caveats/room for improvement…

1. Performance. It is currently slower due to the extra level of indirection. In real-world usage for these kinds of values, it seems likely to be negligible relative to the benefits it provides. (Benchmarks at the end.) Or an integrated Rails solution could potentially address that.
2. Abstracting the source from lookups could be problematic. Sometimes, the friction is a good thing and explicitly implementing the lookup logic in the context it’s used can be helpful. While this wouldn’t be prevented, it becomes less obvious that it’s an option to direclty access the values. Maybe it could be extended to help reduce ambiguity between the various simultaneous approaches to secrets/credentials? (i.e. Settings.credentials., Settings.configuration., and Settings.env.)
3. If this existed in parallel to the existing options rather than replacing them, it could create more ambiguity and confusion about which method is the best method. But at the same time, the fact that this does not change any of the existing API’s means it doesn’t directly interfere.
4. As it stands, it’s far from something that could be dropped right into Rails core, but it seemed worth discussing about whether a built-in Rails equivalent could be worth exploring further. The existing approach was more of a way to try it out in practice to see if it felt better. i.e. What is currently the Settings class would likey need to live off of Rails rather than be a standalone class among other things.
5. It does not currently account for the Rails.configuration.x.<method> approach–only the Rails.configuration.<method>. I don’t believe that would be difficult to add support for, though.
6. There are some additional access methods via method_missing and source-specific options as well. They’re documented in the source code. (Below with benchmarks.) I can’t say I have strong feelings yet about whether all of the approaches are necessary or their relative usefulness.
7. It is possible to have ‘conflicts’ where there’s a key or set of keys that exist in more than one source. Right now, when this happens, it raises an exception to draw attention to the problem. Then the conflicting key can be renamed, deleted in which ever file it shouldn’t be in, or accessed via the source-specific options. i.e. Settings.secret(:key), Settings.config(:key), or Settings.env(:key).
8. This implementation behaves as a wrapper, but a better approach for Rails may be to expose more consistent access methods to each of the sources rather than route access through an entirely new layer.
9. It doesn’t currently account for scenarios where the optional/required/default approach could differ by environment. That’s likely not a bad thing, but it could be an opportunity to make it a little more elegant. Or it could complicate matters and make the resulting interface more confusing.
10. Loading/initialization order means the Settings class isn’t currently accessible early in the boot process, but I believe that’s a solvable problem, especially if this was more natively integrated with Rails.

While my specific implementation leaves plenty to be desired, the interface and benefits feel worth pursuing/exploring in the larger context of Rails. And to share the performance aspects, here’s the benchmark I used and the results (Figures 3 & 4) . So far, I haven’t given any thought to improving performance, but I did want to at least consider how much of a problem it could be. So the performance gap could be very easy to close.

n = 1000000Benchmark.benchmark(CAPTION, 30, FORMAT, ">total:", ">avg:") do |x| x.report('Direct One Level') { n.times do; Rails.configuration.settings[:test_one]; end } x.report('Direct Two Levels') { n.times do; Rails.configuration.settings[:test_two][:test_three]; end } x.report('Settings One Level .config') { n.times do; Settings.config(:test_one); end } x.report('Settings Two Levels .config') { n.times do; Settings.config(:test_two, :test_three); end } x.report('Settings One Level .optional') { n.times do; Settings.optional(:test_one); end } x.report('Settings Two Levels .optional') { n.times do; Settings.optional(:test_two, :test_three); end } x.report('Settings One Level .required') { n.times do; Settings.required(:test_one); end } x.report('Settings Two Levels .required') { n.times do; Settings.required(:test_two, :test_three); end } x.report('Settings One Level .default') { n.times do; Settings.default(:test_one) { 'Default' }; end } x.report('Settings Two Levels .default') { n.times do; Settings.default(:test_two, :test_three) { 'Default' }; end } x.report('Settings Miss L1 .default') { n.times do; Settings.default(:missing) { 'Default' }; end } x.report('Settings Miss L2 .default') { n.times do; Settings.default(:test_two, :missing) { 'Default' }; end } x.report('Settings L1 .method_missing') { n.times do; Settings.test_one; end } x.report('Settings l2 .method_missing') { n.times do; Settings.test_two[:test_three]; end }end

The bencharmks illustrate that there is some overhead with this approach, but as infrequently as these kinds of values are loaded and accessed, it may not be a significant issue at all. More likely, I’m fairly confident I could improve the performance if the API is interesting enough to explore further.

Direct One Level 0.826228 0.006609 0.832837 ( 0.832929)Direct Two Levels 0.903825 0.000150 0.903975 ( 0.903999)Settings One Level .config 1.267055 0.000227 1.267282 ( 1.267377)Settings Two Levels .config 1.524995 0.000356 1.525351 ( 1.525690)Settings One Level .optional 4.300620 0.001423 4.302043 ( 4.302907)Settings Two Levels .optional 4.917827 0.001853 4.919680 ( 4.921652)Settings One Level .required 4.830326 0.006278 4.836604 ( 4.841879)Settings Two Levels .required 5.209971 0.008416 5.218387 ( 5.225179)Settings One Level .default 4.832007 0.010729 4.842736 ( 4.854616)Settings Two Levels .default 5.267153 0.009141 5.276294 ( 5.286929)Settings Miss L1 .default 4.666976 0.005361 4.672337 ( 4.675439)Settings Miss L2 .default 5.549376 0.012397 5.561773 ( 5.565845)Settings L1 .method_missing 4.693234 0.006376 4.699610 ( 4.702111)Settings l2 .method_missing 4.868722 0.005042 4.873764 ( 4.876025)

And the actual code and tests in case anybody else wants to take it for a spin. it includes some additional methods and options, but it’s still more of a working prototype than a definitive proposal.

lib/settings.rb # frozen_string_literal: true# Provides a consistent and predictable interface for retrieving configuration values without requiring knowledge of# where or how the values are stored. Methods for specifically looking in settings, credentials, or ENV provide a# consistent interface regardless of how the values use hash storage.## So instead of remembering varying combinations of the following...## Rails.application.credentials.secret_key_base# Rails.application.credentials.aws[:secret_key]# Rails.configuration.settings[:public_key]# Rails.configuration.settings[:domain][:name]# ENV['ENV_VAR']# ENV.fetch('ENV_VAR', 'Default')# ENV.fetch('ENV_VAR') { 'Default' }## Settings provides a set of source-indifferent lookup options that respond differently when a match isn't found.# With all source-indifferent lookups, if multiple matches are found, they will raise an exception to prevent# accidentally using the wrong value from settings.yml when you wanted the value from production.yml.enc.## Settings.optional(:key) # => Returns the value# Settings.optional(:key_one, :key_two) # => Returns the value of the nested keys# Settings.optional(:missing) # => Returns nil# Settings.required(:key) # => Returns the value# Settings.required(:key_one, :key_two) # => Returns the value of the nested keys# Settings.required(:missing) # => Raises exception when a critical value is missing# Settings.default(:key) { 'Default' } # => Returns the value and ignores the default.# Settings.default(:missing) { 'Default' } # => Returns 'Default'## For convenience and syntactic sugar, optional and default lookups can use the key names directly.# This will not work as a substitute for required lookups.## Settings.key # => Same as Settings.optional(:key)# Settings.key_one(:key_two) # => Same as Settings.optional(:key_one, :key_two)# Settings.key { 'Default' } # => Same as Settings.default(:key) { 'Default' }# Settings.key_one(:key_two) { 'Default' } # => Same as Settings.default(:key_one, :key_two) { 'Default' }## Values can also be accessed by their expected source location as well. If there's a case where there are source# conflicts with multiple sources, those can be avoided by specifying the source like so:## Settings.secret(:secret_key_base)# Settings.secret(:aws, :secret_key)# Settings.config(:public_key)# Settings.config(:domain, :name)# Settings.env(:env_var)## One final caveat. As one might expect, there is a performance difference between accessing values directly using the# existing built-in Rails approaches, but for the majority of use cases where these values would be accessed, the# difference should be marginal.class Settings class ConflictError < ::StandardError; end class MissingError < ::StandardError; end class << self # Looks only for secrets matching the symbols # # @example # Settings.secret(:key_one) # => value for :key_one # Settings.secret(:key_one, :key_two) # => value for :key_two nested under :key_one # # @param key [Symbol] first-level key to desired settings value # @param nested_keys [*Symbol] additional keys for nested settings values # # @return [String, Boolean, Integer] the value of the setting # # @raise [Settings::MissingError] if an exact match is not found in the credentials def secret(key, *nested_keys) dig(key, *nested_keys) { Rails.application.credentials.__send__(key) } end # Looks only for config settings matching the symbols # # @example # Settings.config(:key_one) # => value for :key_one # Settings.config(:key_one, :key_two) # => value for :key_two nested under :key_one # # @param key [Symbol] first-level key to desired settings value # @param nested_keys [*Symbol] additional keys for nested settings values # # @return [String, Boolean, Integer] the value of the setting # # @raise [Settings::MissingError] if an exact match is not found in the configuration settings def config(key, *nested_keys) dig(key, *nested_keys) { Rails.configuration.settings[key] } end # Looks only for environment variables matching the symbol # # @example # Settings.env(:key) # => value for :key # # @param key [Symbol] first-level key to desired settings value # @param nested_keys [*Symbol] additional keys for nested settings values # # @return [String, Boolean, Integer] the value of the setting # # @raise [Settings::MissingError] if an exact match is not found in the ENV values def env(key) dig(key) { ENV[key.upcase.to_s] } end # Performs a lookup where returning nil is acceptable if a match is not found # # @note Essentially an alias of `lookup` but with a more intention-revealing name # # @example # Settings.optional(:key) # => value for :key # Settings.optional(:missing_key) # => nil # # @param key [Symbol] first-level key to desired settings value # @param nested_keys [*Symbol] additional keys for nested settings values # # @return [String, Boolean, Integer, nil] the value of the setting # # @raise [Settings::ConflictError] if a set of keys finds values in more than one location def optional(key, *nested_keys) lookup(key, *nested_keys) end # Lookup that raises an exception if all potential sources are nil # # @note Essentially an alias of `lookup` but with a more intention-revealing name # # @example # Settings.required(:key) # => value for :key # Settings.required(:missing_key) # => Raises Settings::MissingError # # @param key [Symbol] first-level key to desired settings value # @param nested_keys [*Symbol] additional keys for nested settings values # # @return [String, Boolean, Integer] the value of the setting # # @raise [Settings::ConflictError] if a set of keys finds values in more than one location # @raise [Settings::MissingError] if an exact match is not found def required(key, *nested_keys) value = lookup(key, *nested_keys) # Critical settings must be present. Best to fail now. raise Settings::MissingError, "no value found for keys: #{[key, *nested_keys].inspect}" if value.nil? value end # Returns the provided default if no match is found # # @example # Settings.default(:key) { 'Default' } # => value for :key # Settings.default(:missing_key) { 'Default' } # => Returns 'Default' # # @param key [Symbol] first-level key to desired settings value # @param nested_keys [*Symbol] additional keys for nested settings values # @param default [Block] a block to evaluate to derive the default if no match is found # # @return [String, Boolean, Integer] the value of the setting # # @raise [ArgumentError] if a block is not provided # @raise [Settings::ConflictError] if a set of keys finds values in more than one location def default(key, *nested_keys, &default) raise ArgumentError, 'block required for default return value' unless block_given? value = lookup(key, *nested_keys) value.nil? ? default.call : value end # Syntactic sugar to expose lookups as attributes rather than hash keys # # @example # Settings.key # => Same as Settings.optional(:key) # Settings.key_one(:key_two) # => Same as Settings.optional(:key_one, :key_two) # Settings.key { 'Default' } # => Same as Settings.default(:key) { 'Default' } # Settings.key_one(:key_two) { 'Default' } # => Same as Settings.default(:key_one, :key_two) { 'Default' } # # @return [String, Boolean, Integer] the value of the setting def method_missing(key, *nested_keys, &default) value = lookup(key, *nested_keys) value = default.call if value.nil? && block_given? value.nil? ? super : value end # Ensures class is respond_to? friendly when using method_missing # # @api private def respond_to_missing?(key, include_private = false) [secret(key), config(key), env(key)].any? || super end private # A standardized way for performing lookups across all possible sources # # @api private # # @param key [Symbol] first-level key to desired settings value # @param nested_keys [*Symbol] additional keys for nested settings values # # @return [String, Boolean, Integer] the value of the setting # # @raise [Settings::ConflictError] if a set of keys finds values in more than one location def lookup(key, *nested_keys) values = possible_values(key, *nested_keys).compact # Multiple matches were found, and conflicts would introduce potential for errors. Best to fail now. if values.size > 1 exception_messsage = "multiple values for keys: #{[key, *nested_keys].inspect} > #{values.inspect}" raise Settings::ConflictError, exception_messsage end values.first end # Builds an array of possible results from the various lookups # # @api private # # @param key [Symbol] first-level key to desired settings value # @param nested_keys [*Symbol] additional keys for nested settings values # # @return [Array<String, Boolean, Integer>] the arroy values for which a match was found def possible_values(key, *nested_keys) [ secret(key, *nested_keys), config(key, *nested_keys), env(key) ] end # A slightly customized version of dig for retrieving settings # # @api private # # @example # dig(key, *nested_keys) { Rails.application.credentials.__send__(key) } # dig(key, *nested_keys) { Rails.configuration.settings[key] } # dig(key) { ENV[key.upcase.to_s] } # # @param key [Symbol] first-level key to desired settings value # @param nested_keys [*Symbol] additional keys for nested settings values # @yield [Value] for providing and handling default vaues for various settings locations # # @return [Boolean] true if all keys are symbols, false otherwise def dig(key, *nested_keys) raise ArgumentError, 'all arguments must be symbols' unless symbols_only?(key, *nested_keys) secret = yield nested_keys.any? ? secret&.dig(*nested_keys) : secret end # Detects whether all parameters are keys # # @api private # # @param keys [*Symbol] additional keys for nested settings values # # @return [Boolean] true if all keys are symbols, false otherwise def symbols_only?(*keys) keys.all? { |key| key.is_a? Symbol } end endend

test/lib/settings_test.rb # frozen_string_literal: truerequire 'test_helper'class SettingsTest < ActiveSupport::TestCase # rubocop:disable Metrics/ClassLength # rubocop:disable Metrics/MethodLength def setup @default_return_value = 'Default Return Value' @secrets = { 'sekrit_one' => %i[test_one_secret], 'sekrit_two_three' => %i[test_two_secret test_three_secret] } @settings = { 'One' => %i[test_one], 'Two Three' => %i[test_two test_three], 'test' => %i[test_erb], 'Adaptable' => %i[org name], 'Adaptable.test' => %i[org domain] } @env = { 'example' => %i[example], 'test' => %i[rails_env] } @all_values = {}.merge(@secrets, @settings, @env) end # rubocop:enable Metrics/MethodLength test "retrieves .secret values with varying key depths" do @secrets.each do |expected_value, keys| assert_equal expected_value, Settings.secret(*keys), "failed to match #{keys.inspect} and return #{expected_value} for Settings.secrets" end end test "retrieves .config values with varying key depths" do @settings.each do |expected_value, keys| assert_equal expected_value, Settings.config(*keys), "failed to match #{keys.inspect} and return #{expected_value} for Settings.config" end end test "retrieves .env values" do @env.each do |expected_value, keys| assert_equal expected_value, Settings.env(*keys), "failed to match #{keys.inspect} and return #{expected_value} for Settings.env" end end test "performs .optional lookups across all sources with varying key depths" do @all_values.each do |expected_value, keys| assert_equal expected_value, Settings.optional(*keys), "failed to match #{keys.inspect} and return #{expected_value}" end end test "performs .required lookups across all sources with varying key depths" do @all_values.each do |expected_value, keys| assert_equal expected_value, Settings.required(*keys), "failed to match #{keys.inspect} and return #{expected_value}" end end test "performs .default lookups across all sources with varying key depths" do @all_values.each do |expected_value, keys| assert_equal expected_value, Settings.default(*keys) { @default_return_value }, "failed to match #{keys.inspect} and return #{expected_value}" end end test "considers `false` a valid return value for a config setting via any method" do # Accidentally relying on || or ||= could mean unintentionally discarding the value when it's explicitly false assert_equal false, Settings.config(:falsey_value) assert_equal false, Settings.optional(:falsey_value) assert_equal false, Settings.required(:falsey_value) assert_equal false, (Settings.default(:falsey_value) { true }) assert_equal false, Settings.falsey_value assert_equal false, (Settings.falsey_value { true }) end test "allows bypassing methods and using keys directly via method_missing" do @all_values.each do |expected_value, keys| method_name, *keys = keys assert Settings.respond_to?(method_name) assert_equal expected_value, Settings.__send__(method_name, *keys), "method_missing failed to match #{keys.inspect} and return #{expected_value}" end end test "raises no method error for a non-existent root setting" do symbol = :method_missing_symbol refute Settings.respond_to?(symbol) assert_raises(NoMethodError) { Settings.__send__(symbol) } end test "allows returning a default value block using keys directly via method_missing" do assert_equal @default_return_value, (Settings.nonexistent { @default_return_value }) assert_equal @default_return_value, Settings.test_two(:nonexistent) { @default_return_value } end test "discards env.yml values when they have already been set" do assert_not_equal 'not_test', Settings.env(:rails_env) assert_equal 'test', Rails.env assert_equal 'test', ENV['RAILS_ENV'] assert_equal 'test', Settings.env(:rails_env) end test "returns the default when a default lookup does not have a match" do assert_equal @default_return_value, Settings.default(:this_key_does_not_exist_anywhere) { @default_return_value } assert_equal @default_return_value, Settings.default(:test_two, :this_key_does_not_exist_anywhere) { @default_return_value } end test "raises exception when retrieving values using non-symbol params" do assert_raises(ArgumentError) { Settings.secret('test_one_secret') } assert_raises(ArgumentError) { Settings.secret(:test_one_secret, 'test_three_secret') } assert_raises(ArgumentError) { Settings.config('test_one') } assert_raises(ArgumentError) { Settings.config(:test_one, 'test_three') } assert_raises(ArgumentError) { Settings.env('example') } assert_raises(ArgumentError) { Settings.optional('example') } assert_raises(ArgumentError) { Settings.required('example') } assert_raises(ArgumentError) { Settings.default('example') { @default_return_value } } end test "raises exception when a broad lookup has multiple matches" do assert_raises(Settings::ConflictError) { Settings.optional(:conflict) } end test "raises exception when a required lookup has no match" do assert_raises(Settings::MissingError) { Settings.required(:this_key_does_not_exist_anywhere) } assert_raises(Settings::MissingError) { Settings.required(:test_two_secret, :this_key_does_not_exist_anywhere) } end test "raises exception when a default lookup is used without providing a default block whether a match exists or not" do assert_raises(ArgumentError) { Settings.default(:test_one) } assert_raises(ArgumentError) { Settings.default(:this_key_does_not_exist_anywhere) } endend