Red Green Repeat Adventures of a Spec Driven Junkie

Bash scripting: echo & zero padding

I want to share how to create a bash script in a safe manner, without tests! I show what I use to help ensure the bash script does what I think it does before executing the script. I share a practical example, renaming a sequence of thousand files.

I hope this helps you in your journey in learning bash and get a quick utility to help manage your files. All of this should take less than four minutes!

Qur'an Manuscript

Introduction

Scripting in bash is a handy way to get work done in certain situations. Bash is a complete programming language, it has variables, loops, arrays, and more! There are whole books on bash and before the devops movement, bash was the tool for system administrators to manage systems.

Bash scripting is essential as you can programmically work on the system. Instead of manually performing repeated operations, the same operations on the same machine, or the same operations on different machines, bash scripts remember how to do stuff and most importantly: keep you from making mistakes.

Where’s the Test??

I am a TDD junkie and since there is code, I expect tests.

In bash, I am usually writing out small scripts, so I’m a bit pragmatic in terms of requiring tests.

Enter echo

In bash scripts, I start coding using the echo command. echo is your friend when scripting in bash. It’s a manual way to test commands before executing the script.

In the script, to use echo to test, wrap any command with echo, in the form of: echo "command" before running the full version.

Pitfalls of Not Testing Bash Scripts

There is nothing worse than mistyping a command and realizing it’s wrong after it’s executed incorrectly. As bash scripts run ON the system directly, there no safeguards! echo is the simplist safeguard I have encountered. This is n times more important with loops!

Rename a Sequence of Files

To demonstrate how I use echo, I’ll solve a problem I encounter: rename 1000 files with scheme: DSC_001.jpg to hawaii_trip_2019_001.jpg.

Doing this by hand would be brutal, but it’s definitely possible. I would not be happy afterwards, but do able.

With a bash script, this would be trivial: this would be the first version:

#!/usr/bin/env bash
for i in {1..9}
do
   echo "mv DSC_00$i.jpg hawaii_trip_2019_00$i.jpg"
done

for i in {10..99}
do
   echo "mv DSC_0$i.jpg hawaii_trip_2019_0$i.jpg"
done

for i in {100..999}
do
   echo "mv DSC_$i.jpg hawaii_trip_2019_$i.jpg"
done

The output looks like:

mv DSC_001.jpg hawaii_trip_2019_001.jpg
mv DSC_002.jpg hawaii_trip_2019_002.jpg
mv DSC_003.jpg hawaii_trip_2019_003.jpg
...
mv DSC_994.jpg hawaii_trip_2019_994.jpg
mv DSC_995.jpg hawaii_trip_2019_995.jpg
mv DSC_996.jpg hawaii_trip_2019_996.jpg
mv DSC_997.jpg hawaii_trip_2019_997.jpg
mv DSC_998.jpg hawaii_trip_2019_998.jpg
mv DSC_999.jpg hawaii_trip_2019_999.jpg

Now that looks good, remove the echo and quotes, and have the script work its magic!

DRYing Things Up

If I use the previous script one time, the above script would suffice. I would perform the renaming and remove the script, forever.

If I want to have a “reference script”, the programmer in me thinks: How can I do better? I have three loops where one can do. I can hear my coworker saying: that’s not DRY enough!

One reason it’s not DRY is the zero padding. It’s natural for humans to think: 1 but to a computer, 1 and 000000001 are equivalent. Secretly, I like zero padded numbers too for regular file lengths and sorting!

DRYing by Zero Padding

How can I solve the zero padding issue in bash without three separate loops?

Given the version of bash I have (3.2), this is the DRYest solution I have found:

#!/usr/bin/env bash
for i in 00{1..9} 0{10..99} {100..999}
do
   echo "mv DSC_$i.jpg hawaii_trip_2019_$i.jpg"
done

reference

With current bash, version 4, this is even better:

#!/usr/bin/env bash
for i in {001..999}
do
   echo "mv DSC_$i.jpg hawaii_trip_2019_$i.jpg"
done

reference

Notice, that I use echo here, I want to make sure code does what I want before trusting it, even if it looks like it’s right. Only when I am sure, I will remove echo and "", and the script becomes:

#!/usr/bin/env bash
for i in 00{1..9} 0{10..99} {100..999} # OR {001..999}
do
   mv DSC_$i.jpg hawaii_trip_2019_$i.jpg
done

As DRY as can be!

Conclusion

Bash scripting is essential to know as it can interact with the system directly. Not having a testing framework shouldn’t be a hinderance, but it just means one has to be more creative in creating verifable code before executing it.

Using echo is essential to ensure the script does what you think it does before it executes. As the script interacts directly with the system, mistakes can be hard to recover from.

I demonstrate how to use echo in solving a common file renaming problem and utilize bash’s zero padding for different versions of bash.

This is a resource I used to help put together this article (and scripts before!)