Debugging and Monitoring TensorFlow Programs

This course goes deep into two specific tools in the TensorFlow toolkit - tfdbg and TensorBoard. These tools can be used to examine the internal state of TensorFlow programs and to visualize execution metrics and state.
Course info
Level
Intermediate
Updated
Mar 21, 2018
Duration
2h 17m
Table of contents
Description
Course info
Level
Intermediate
Updated
Mar 21, 2018
Duration
2h 17m
Description

An important facet of building good ML models is the ability to debug TensorFlow code when your models do not converge. Traditional debuggers fall short in this regard which is why tfdbg and TensorBoard are important skills in your toolkit. In this course, Debugging and Monitoring TensorFlow Programs, you will learn how you can adapt TensorFlow commands and library functions to help debug your programs in addition to learning specialized tools like tfdbg and Tensorboard. First, you will go over TensorFlow's special features to debug your code. Partial graph executions, tf.Print() and tf.Assert() statements, traditional Python debuggers and the tf.py_func() to interpose arbitrary Python code into your computation graph all help debug the graph build phase. Next, you will see that the specialized TensorFlow debugger tfdbg works very much like traditional Python debuggers but has the ability to step into session.run() statements and display the state of your computation graph at every step. It also has filters like the has_inf_or_nan which allows you to break at the exact point your model begins to diverge. Finally, you will be shown Tensorboard, which is a browser-based tool that helps you visualize your computation graph and view how control flows through your code. In addition, it can be used to display execution metrics and the current state of your program. After finishing this course, you will be closer to mastering TensorFlow through equipping you with important tools to build and debug robust machine learning models.

About the author
About the author

A problem solver at heart, Janani has a Masters degree from Stanford and worked for 7+ years at Google. She was one of the original engineers on Google Docs and holds 4 patents for its real-time collaborative editing framework.

More from the author
Analyzing Data with Qlik Sense
Intermediate
2h 11m
Jun 17, 2019
More courses by Janani Ravi
Section Introduction Transcripts
Section Introduction Transcripts

Course Overview
Hi, my name is Janani Ravi and welcome to this course on Debugging and Monitoring TensorFlow Programs. I'll introduce myself. I have a master's degree in electrical engineering from Stanford and have worked at companies such as Microsoft, Google, and FlipKart. At Google, I was one of the first engineers working on real-time collaborative editing in Google Docs and I hold four patents for its underlying technologies. I currently work on my own startup, Loonycorn, a studio for high-quality video content. Tensorflow offers a number of special features to debug your code. Partial graph executions, tf. Print and tf. assert statements, traditional Python debuggers, and tf. py_func, which can be used to interpose arbitrary Python code into your computation graph all help debug the graph build phase. In order to debug the execution of your computation graph, you need the specialized TensorFlow debugger tfdbg. It works very much like traditional Python debuggers, but has the ability to step into session. run statements and display the state of computation graph at every step. This debugger also has filters like the has_inf_or_nan filter, which allows you to break at the exact point your model beings to diverge. This course also covers TensorBoard, which is a browser-based tool which allows you to visualize your computation graph and view how control flows through your code. In addition, it can be used to display execution metrics and the current state of your program. This course will help you gain mastery over TensorFlow by equipping you with important tools to build and debug robust machine learning models.

Introducing TensorFlow Debugging Methods
Hi, and welcome to this course on Debugging and Monitoring TensorFlow Programs. You're now comfortable with TensorFlow and you've also built a few machine learning models. But if you run into issues, you need to know how you can track these down, and that's what this course will cover. TensorFlow performs computation in two phases. The first phase is where we build our computation graph, and the second phase where we actually execute it. This is what makes it hard to debug TensorFlow programs. The build phase is typically easier to debug. You can use traditional Python debuggers, print operations, asserts, and so on. The run phase, however, is wrapped in a single session. run object, and it's hard to see what exactly is going on under the hood. TensorFlow provides a few special operations in order to help us with debugging. You can do partial graph evaluations, partial runs of certain nodes. You can use special assert statements and print statements for debugging. In addition, TensorFlow also offers a special debugging tool called tfdbg geared towards debugging computation graph executions. This is the tool that you'd use to see what exactly is going on behind those session. run statements. Where are your tensors being evaluated? What values do they store? Do they store values that make sense to you or not? Computation graphs which represent a machine learning model can get very complex very quickly. TensorBoard is a visualization tool that TensorFlow offers to view the model architecture and also allows you to summarize weights, biases, and other inputs and outputs through your neural networks.

Applying tfdbg to Common Use-cases
Hi and welcome to this module on tfdbg. Tfdbg is the TensorFlow debugger, which you can use to display the values of tensors and debug our session. run invocations. We'll see how we can apply tfdbg common use cases in your machine learning models. Google understood the limitations of traditional Python debuggers while debugging TensorFlow programs, which is why they introduced tfdbg. This is TensorFlow's specialized debugger. Tfdbg has built-in awareness of the computation graph and its execution. We can use it to view the internal structure and state of the various nodes and tensors in this computation graph. This is what makes tfdbg our debugger of choice. It is superior to pdb, ipdb, and other Python debuggers. Tfdbg effectively steps into session. run invocation by creating a wrapper session object, which allows us to monitor what exactly is going on under the hood. A common problem we encounter while building neural network machine learning models is that our model does not converge the weights and biases. They become infinite or not a number. Tfdbg has special built-in filters which help you find and fix nan and inf problems.

Visualizing TensorFlow Using TensorBoard
Hi and welcome to this module on Visualizing TensorFlow Using TensorBoard. All programs in TensorFlow are computation graphs and graphs are best visualized using some kind of user interface, and that is what TensorBoard is. It's a browser-based visualization tool. This comes along with your TensorFlow libraries and once you have TensorFlow installed, you can invoke TensorBoard, once you've written out events from your main TensorFlow program. TensorBoard provides many different kinds of functionality and we'll be covering those in this module, but the most common application is to actually visualize the computation graph to see what tensors flow into which operation, which operation feeds another, and really get a big picture understanding of what's going on. TensorBoard can also be used to plot summaries and display images. You might have weights, biases, and other tensors in your code. You want to quickly summarize them and see how they move over time. That's possible in TensorBoard. You can also view images that you might be using in your training data. TensorBoard also allows you to get a quick idea of how the values of a particular variable is distributed by allowing you to plot histograms.