Using tf.Print() in TensorFlow
I heard you wanted to print some tensors.
I know that you definitely use debuggers the right way, every time, and would never use a print statement to debug your code. Right?
Because if you did, you might find that TensorFlow’s Print statement doesn’t quite work the same way as typical print statements.
Today I’ll show how TensorFlow’s print statements work, and how to make the most of them, hopefully saving you some confusion along the way.
Printing in TensorFlow
There are a couple of ways to get things to print out while writing TensorFlow code. Of course, there’s the classic Python built-in, print (Or the function print(), if we’re being Python 3 about it). And then there’s TensorFlow’s print function, tf.Print (notice the capital P).
When working with TensorFlow, it’s important to remember that everything is ultimately a graph computation. This means that if you print a TensorFlow operation using Python’s print, it will simply show a description of what that operation is, since no values have been passed through it yet. It will also often show the dimensions that are expected to be in that node, if they’re known.
If you want to print the values that are ‘flowing’ through a particular part of the graph as it’s being executed, then we need to turn to using tf.Print.
Structuring your Print node
Often we think of print as something we add on the side of something, after the fact, and just let it run, and then come back to our normal flow of operations. In TensorFlow, only the nodes of the graph that need to be executed to compute the output, will get executed. So if you have a ‘dangling’ Print node in your graph, it won’t be executed.
So what are we to do? We need to splice the Print call into the graph itself, like so:
The way this manifests in code is to pass the Print call, as its first parameter, the node that is its ‘input’, and then assign the return value of tf.Print to a variable that will serve as an input in a later node in the graph, thus linking the Print statement serially into the flow of operations. (It is vitally important that you actually use this returned node, because if you don’t, it will be dangling.)
While the Print statement does no computation and just passes the tensor(s) onward, it does print the desired node as a side effect.
Another behavior that is a bit unlike what we are used to seeing in a print statement is that the Print node that we introduce into the graph is merely setting when the Print statement will happen, namely when the node is reached in the computational graph. However, there aren’t many restrictions on what you print.
That is, it’s only marking where in the graph that you’d like a print statement to occur, but you can print any nodes in the graph that it can access. This is the 2nd argument to the tf.Print call: an array of nodes to print. Often we’ll just use the same node as the first argument, which is the input, but we could include more nodes in the print if we’d like.
There’s also a 3rd argument, a message. This allows you to prepend some string before printing the nodes, so you can easily find a given print statement in your logs.
Putting it in practice
So there you have it, tf.Print in a nutshell. But where might you utilize tf.Print in your code?
I often find myself using tf.Print in my input functions, to debug exactly what is being passed into my training loop. Merely using the Python print is not quite enough here since it will only print once, when the input function graph is constructed.
Instead, we’ll introduce the tf.Print call into the input function’s data pipeline. Notice that we are just printing the same value that we passed in as the input, but you could certainly print other nodes as well.
Some more advice
Notice that the print output is showing up in stderr in the console of my Jupyter notebook, not as an output in the notebook itself. Keep this in mind when searching for your print outputs!
Another word of caution: if you use tf.Print in your input function, be sure to limit the amount of data you pass in, otherwise you might end up scrolling through a very long console window :)
There are many ways in which TensorFlow’s print statement is not your typical print statement, but it can be used to great effect when you need to see the tensors flowing in your graph.
Thanks for reading this episode of Cloud AI Adventures. If you’re enjoying the series, please let me know by clapping for the article. If you want more machine learning action, be sure to follow me on Medium or subscribe to the YouTube channel to catch future episodes as they come out. More episodes coming at you soon!
For now, see how TensorFlow’s print statements can help you get more clarity into how your machine learning is running!
