How to view or track the progress of Notebook while it is running in Microsoft Fabric

I was recently working with a Notebook in Microsoft Fabric that was started via a Data Pipeline.

The challenge I had was that I had no idea how far the notebook had gone (as there were quite a lot of cells in this particular notebook).

In this blog post I am going to show you how I can use Microsoft Fabric to identify exactly which cell my notebook is currently on.

This is what I saw below when my notebook was running, I could see the Status says “In progress” but that was all I could see initially.

I then completed the following steps below which allowed me to identify on what cell is currently being executed in my notebook.

I clicked on the View Detail icon as shown below.

On the left-hand side, I then got the details of my notebook.

I then clicked on View Snapshot as highlighted below.

This then loaded the Snapshot page, I then clicked on Spark UI as shown below to bring up the Spark User Interface.

Once I was in the Spark UI, I could then see that the Description for the Job Id has got the cell number, as well as the first few lines of code in my code cell.

As highlighted below I could see the code cell had the comment of “#Get all Interactive Detail Data from pyspark.sql”

I then navigated to my Notebook and found the cell with the same comment, which was also Cell 30

I could then identify how far my notebook was in its execution and how far it had to go!

Summary

In this blog post I have shown you how to view the process of you to track the progress of your notebook.

I hope you found this interesting and could save you time going forward.
Thanks for reading!