Guide: Quickly Debug a Ruby Gem with Pry and Git Bisect

A quick guide to help you debug a Third-Party Ruby Gem (faker-ruby) using `bundle open`, `pry`, and `git bisect`.

Are you trying to debug a Ruby gem but not sure how? Here’s a quick way to debug any Ruby Project using Pry and Git Bisect.

When triaging some open issues on faker-ruby, I needed to replicate Issue #2482. Turns out the bug was already fixed by this PR, but version v2.20.0 was released before that.

I thought this would be a great opportunity to show you how I debugged this issue so you can learn some new tricks. Let’s assume the issue was not already fixed so I can show you exactly how to debug it yourself. And then you can use these tricks on your projects.

Sounds good?

Note: this guide assumes you have a bug to fix. If you don’t and just want to follow along, clone this repo. You’ll need Pry and the $EDITOR environment variable set to work with your favorite editor.

1. Run the failing code to reproduce the bug

The first step is to run the failing code and find a way to reproduce the bug. In this case, the gem randomly generates a nil game title, which is not good.

To reproduce this bug, install faker v2.20.0 and run the code below. The gem will randomly generate a small number of nil game titles.

Let’s write a short test:

# faker_game_title_test.rb

require 'test/unit/assertions'
include Test::Unit::Assertions

require 'faker'

1000.times do
  assert_not_nil Faker::Game.title
end

Then, run the test and you’ll be able to reproduce the error:

$ ruby faker_game_title_test.rb
<nil> was expected to not be nil. (Test::Unit::AssertionFailedError)

Great! We reproduced the bug and have something to work with.

2. Run bundle open [gem-name] to access the source code

Now we need to understand why the bug is happening.

Let’s take a look at the source code:

$ bundle open faker

With the source code open, we can now take a peek inside 🔎

3. Add binding.pry to invoke the debugger

If you look at how Faker::Game.title is implemented here, you’ll see that it just calls Faker::Base.fetch.

Let’s invoke the debugger where the fetch method is defined:

def fetch(key)
  fetched = sample(translate("faker.#{key}"))

  require "pry"                 # <- require pry
  binding.pry if fetched.nil?   # <- invoke pry debugger

  if fetched&.match(%r{^/}) && fetched&.match(%r{/$})
    regexify(fetched)
  else
    fetched
  end
end

Now whenever this method is called, and fetched is nil, we intercept the execution and inspect the code 🔎

4. Run the failing code again

Now if you run the failing test again, a pry session will start and you’ll be able to debug the code:

$ ruby faker_game_title_test.rb
    97: def fetch(key)
    98:   fetched = sample(translate("faker.#{key}"))
    99:
  100:   require "pry"
=> 101:   binding.pry if fetched.nil?
  102:
  103:   if fetched&.match(%r{^/}) && fetched&.match(%r{/$}) # A regex
  104:     regexify(fetched)
  105:   else
  106:     fetched
  107:   end
  108: end

5. Explore and debug the code using pry

If you need a quick tutorial or a refresher on how to use pry, read Getting Started with Pry Debug in 5 minutes.

We can see that a random key is picked from the list returned by translate("faker.game.title"). Let’s inspect this method with show-source [method-name]:

pry(Faker::Game)> show-source translate
def translate(*args, **opts)
  opts[:locale] ||= Faker::Config.locale
  opts[:raise] = true
  I18n.translate(*args, **opts)  # <------
  # (...)
end

You’ll see that it is just grabbing keys from this yml file.

If we run the code, the last entry is indeed nil:

[17] pry(Faker::Game)> I18n.translate("faker.game.title")[-3..]
[
    [0] "Command & Conquer: Rivals",
    [1] "Command & Conquer: Tiberium Alliances",
    [2] nil
]

Gotcha! 🐞

Run bundle pristine to clear your changes

After changing the gem’s code, we need to discard them by running:

$ bundle pristine

bundle pristine restores the installed gems in the bundle to their pristine condition using the local gem cache from RubyGems.

6. Find out when the bug was introduced with git bisect {#git-bisect-example}

Use git bisect to find the commit that caused the issue. That will give you clues on which files to focus on. You can even use the test script from above to tell you whether a version is good or bad.

Continuing with our issue, we need to clone the Faker repository:

$ git clone git@github.com:faker-ruby/faker.git
$ cd faker

To start the git bisect process, run:

$ git bisect start

We know that version v2.20.0 is broken (bad) and v2.19.0 was working (good), so let’s mark them appropriately:

$ git bisect bad v2.20.0
$ git bisect good v2.19.0
  Bisecting: 38 revisions left to test after this (roughly 5 steps)
  [2a6f0cc] Add general documentation for Faker::Camera (#2421)

Now you can let git bisect do its magic by telling it to run your test. It will jump between revisions, doing a binary search to find where the bug was introduced. It will then mark the revision as either good or bad depending on the output of your test, and stop.

To run a script/test with git bisect, do the following:

$ git bisect run [your-test-script]

Here’s an example:

$ git bisect run rake test TEST=faker_game_title_test.rb
running rake test TEST=faker_game_title_test.rb

  Bisecting: 18 revisions left to test after this (roughly 4 steps)
  [649dd8f] Merge pull request #2442 from sudeeptarlekar/2309-truffleruby-errors
  running rake test TEST=faker_game_title_test.rb

  (...)
  
  Bisecting: 0 revisions left to test after this (roughly 1 step)
  [c1998d0] Merge pull request #2457 from Awilum/patch-2
  running rake test TEST=faker_game_title_test.rb
    <nil> was expected to not be nil. (Test::Unit::AssertionFailedError)

  Bisecting: 0 revisions left to test after this (roughly 0 steps)
  [722172a] add Command & Conquer games
  running rake test TEST=faker_game_title_test.rb
    <nil> was expected to not be nil. (Test::Unit::AssertionFailedError)

  722172a is the first bad commit
  commit 722172a
  Date:   Mon Jan 31 00:52:58 2022 +0300
      add Command & Conquer games
  lib/locales/en/game.yml | 12 ++++++++++++
  1 file changed, 12 insertions(+)

  bisect run success

git-bisect found the commit that introduced the bug. And it consists of changes to only one file: lib/locales/en/game.yml. That narrows the list of possible culprits by a lot!

Really neat, huh?

With this information, you know exactly which files to look at. When you’re done, get back to the original branch by running:

$ git bisect reset

💡 For a great video explanation on how to use git bisect in a large project, check out the talk RailsConf 2015: Breaking Down the Barrier: Demystifying Contributing to Rails @ 50m55s from Eileen Uchitelle.

7. Open a new issue with reproduction steps

With reproduction steps in hand, open a new issue describing the error and how to reproduce it. Give the maintainers a script they can run if you have it.

Since you’re here doing all this work, if you have a solution, submit a PR.

The issue shared here was already solved, but let’s pretend you’re going to fix it. You know that the problem can be fixed by deleting the empty entry on line 233 in this file lib/locales/en/game.yml.

After you removed the empty line, build the gem and install it locally:

$ gem build faker.gemspec
  Successfully built RubyGem
  Name: faker
  Version: 2.20.0
  File: faker-2.20.0.gem

$ gem install faker-2.20.0.gem

Now if you run the test again, the problem is fixed. Yay! The next step would be to submit a PR following the repository’s guidelines.

Checklist: How to Quickly Debug a Ruby Gem

Let’s recap:

1. Run the failing code to reproduce the bug
2. Run bundle open [gem-name] to access the source code
3. Add binding.pry to invoke the debugger
4. Run the failing code again
5. Explore and debug the code using pry
6. Find out when the bug was introduced with git bisect
7. Open a new issue with reproduction steps

Voilà! Now you’re ready to debug any Ruby gem! 🧰

Debug any Ruby gem with your new debugging power!

By now, you have added a couple of new debugging tricks to your toolbox:

• debug gems locally with bundle open.
• find which git reference (commit, tag, etc.) introduced the bug with git bisect.
• using pry effectively.

Now go out there and bash some bugs!