Visualizing Variable Summaries in TensorBoard

TensorBoard is a great tool which can be used to visualize your computation graph, check out the transformed images, study the variable histograms and distributions and many other things.

The summary of historical values of TensorFlow variables in your program could be of a lot of help as you can traceback how they have changed and what their distribution has been.

In this post, we will quickly check how we can write the historical values of variables in a summary file and then check out its distribution and histogram in TensorBoard.

The first thing to do is to collect the summary histogram of the variables you would like to track.

So if you are working on a simple regression problem suppose, and you would like to track the variables W (weight), b (bias), and your predictions y, then add the following code after your variable declarations.

# Variable devlaration 
w = tf.Variable(tf.zeros([1,1]))
b = tf.Variable(tf.zeros([1]))
x = tf.placeholder(tf.float32, [None, 1])

# Linear regression calculation
wx = tf.matmul(x, w)
y = wx + b

# Adding summary ops to collect data
w_hist = tf.summary.histogram("weights", w)
b_hist = tf.summary.histogram("biases", b)
y_hist = tf.summary.histogram("outputs", y)

Once you have added these summaries, you want to merge all of these summaries into a single object:

# Large merged summary object
merged_summary = tf.summary.merge_all()

Also, you want to initialize a FileWriter object to write these summaries to a file, which will further be the input of your TensorBoard:

writer = tf.summary.FileWriter('./linearRegressionDemo1', sess.graph)

Please don't forget to initialize TensorFlow's session object and initializing global variables before doing so.

Next, you have to run this merged_summary object in your session by giving inputs to all of the placeholders used in the calculation of variables you are writing summary of:

result = sess.run(merged_summary, feed_dict = feed)

Here, feed is a dictionary of containing values of all of the placeholders we want in the calculation of variables for which we are collecting summary.

Lastly, use the writer object to write result to the summary file:

writer.add_summary(result, i)

where i is the current epoch number. This will help you visualize how the variables changed in each epoch.

It is important to close the session and the writer object in the end.

Once this is executed, we have successfully written the summary files and can now proceed to see the visualizations in TensorBoard.

Invoke the TensorBoard pointing the - - logdir to 'LogFilePath' and open TensorBoard in your browser using the local address from the command line.

You can go to the Histogram tab to check out the histograms of all of the relevant variables.

Sample below:

Also, in the Distribution tab, you can check out the distribution of each variable with the epoch number on the x axis.

You can see how my weight has started to be stabilized as the total epochs increase on x axis.