Today I was building some retry logic for some code that forwarded data to a third-party service and wanted to visualize the backoff function I had written. I wanted to see how many retries would be made in the 10-15 minute window after the initial retry. Gnuplot was the obvious choice for visualizing this as I had recently used it for some data exploration work and knew it had built-in support for plotting mathematical functions. The final result looks like this:
Backoff Function
I wanted to visualize my linear backoff function defined as:
linear(n) = 250 + (n * 100)With n being the index number of the next retry and the return value being the backoff time in milliseconds. This definition is valid Gnuplot code and can be plotted directly. However, I immediately encountered some limitations of Gnuplot.
Gnuplot Limitations
The backoff function above can be plotted in Gnuplot, but it doesn’t give me any intuition about how frequently retries will be made. For example, just plotting the above function with Gnuplot results in the following plot, which is not what I wanted:
Discrete Retries
Gnuplot is good at plotting continuous functions as curves but I wanted to visualize each retry as a point in time. Typically the x axis is used to represent time so I wanted the whole backoff function represented as a sequence of points plotted along the x axis. Instead of plotting the function for the continuous range, I need to only plot the function for discrete positive numbers.
The easiest way to only plot the function on a such a sequence is to first generate such a sequence by calling out to the seq command on the command line. Gnuplot will read the numbers output by seq and then pass them to the backoff function invoked inside an expression passed to the using modifier. For the x coordinates I use the values generated by the backoff function, and for the y axis I use the literal value 0 so the points are always in a horizontal line. The code to plot the first 150 retries this way looks like this:
plot "<seq 0 150" using (linear($1)):(0) with points
This is better, but there is still an issue with this plot. Can you tell what it is?
Retries as Subsequent Events
The problem with the above plot is that each retry is plotted on the x axis as an independent point in time. What I wanted was a sequence of subsequent retry events plotted along the x axis. To do this each retry must be positioned on the x axis at the time of the previous retry plus the delay computed by the backoff function for the current retry. There are two ways of doing this with Gnuplot. The x coordinate of the previous retry could be stored in a variable and accessed to compute the x coordinate of the next retry. The other solution is to use a function to compute the x coordinate recursively. This seemed like a more straightforward approach to me:
# The function that computes linear backoff
linear(n) = 250 + (n * 100)
# The recursive function that computes the point in time for each retry using the linear backoff function
linear_backoff(n) = (n > 0 ? linear(n) + linear_backoff(n-1) : 0)Putting It All Together
We can do everything above for different backoff strategies and visualize them together on the same plot. Putting all this together with some settings to improve the appearance of the plot I arrived at the code below that plots three retry strategies for over 15 minutes:
# Functions that compute the different backoff values
linear(n) = 250 + (n * 250)
quadratic(n) = 250 * n * n
cubic(n) = 250 * n * n * n
# The recursive functions that compute the point in time for each retry using the backoff functions defined above
linear_backoff(n) = (n > 0 ? linear(n) + linear_backoff(n-1) : 0)
quadratic_backoff(n) = (n > 0 ? quadratic(n) + quadratic_backoff(n-1) : 0)
cubic_backoff(n) = (n > 0 ? cubic(n) + cubic_backoff(n-1) : 0)
# Set some basic styles for the plot
unset grid
unset ytics
set xtics nomirror
set size ratio .2
set border 3
# Configure title and legend
set key inside top center horizontal
set title "Backoff Strategies"
# Configure X axis for time
set xdata time
set format x "%M:%S"
set xlabel "Minutes"
# Finally plot the functions
plot [0:900][0:2] "<seq 0 150" using (cubic_backoff($1)/1000):(1.5) with linespoints pt 7 ps 0.3 lw 0.1 title "Cubic", \
"" using (quadratic_backoff($1)/1000):(1) with linespoints pt 7 ps 0.3 lw 0.1 title "Quadratic", \
"" using (linear_backoff($1)/1000):(0.5) with linespoints lt 7 pt 7 ps 0.3 lw 0.1 title "Linear"