How to search your virtualenv for mystery error messages

How to search

By default, ripgrep obeys .gitignore, and ignores hidden files and folders. In order to include yoru virtualenv for search, you’ll typically need to lift those restrictions with the -uu double option, short for --unrestricted (twice).

To search your project and the virtualenv contained within, you can use:

$ rg -uu 'Error message'

You can restrict your search to only .py files with the -t option, short for --type:

$ rg -uu -tpy 'Error message'

If your virtualenv doesn’t live within your project, you can search both your project and virtualenv by specifying relevant paths:

$ rg -uu 'Error message' . ~/.virtualenvs/example

Neat.

An Ansible eggsample

A few months ago, I was debugging an issue where Ansible would quit with a mysterious message, and no other details:

ERROR! A worker was found in a dead state

🤔🤔🤔

In order to find where in Ansible this message was triggered, I ripgrepped for it:

$ rg -uu -tpy 'ERROR! A worker was found in a dead state'

…but unfortunately this first search turned up zero results.

I figured the ERROR! part may be part of a generic error message display function, so I dropped that for a second search:

$ rg -uu -tpy 'A worker was found in a dead state'
venv/lib/python3.10/site-packages/ansible/plugins/strategy/__init__.py
837:                raise AnsibleError("A worker was found in a dead state")
863:                raise AnsibleError("A worker was found in a dead state")

Aha! From these results, I could dig in to the issue further. In this case I temporarily modified Ansible’s code to display more information.

In general, if your first searsch has no matches, try searching for just a part of the error message. Error logging often adds prefixes or suffixes, and message strings in code can be broken across multiple lines.

Alternatively with grep -r

If you can’t use ripgrep, plain old grep works as well. It doesn’t have any default filtering, so to search all files in the current directory, run it with -r for recursive:

$ grep -r 'Error message'

To filter to just .py files:

$ grep -r --include '*.py' 'Error message'

And if your virtualenv doesn’t live within your project, you can again specify multiple paths:

$ grep -r --include '*.py' 'Error message' . ~/.virtualenvs/example

Allllrighty then.

Beware that grep is much slower than ripgrep. On my machine, ripgrep is nearly ten times faster for an equivalent search.

Fin

For another example, see Anže Pečar’s blog post on searching through his virtualenv for vendored copies of six.

May your debugging adventures be fun and no longer than necessary,

—Adam