What I have found is that when I created a pipeline in Microsoft Fabric that uses a notebook, when there is an error with the notebook, I do not get an alert that the notebook has failed.

This has happened to me in the past and I have found this pattern below to work consistently to notify me of errors.

In this blog post I will show you how I get notified when a notebook fails in a pipeline.

Using a Power Automate flow for alerting

The first step is I created a Power Automate flow, which will be invoked when a notebook fails. This will include details of the pipeline and

Below is how I have configured my flow.

  • For the first step I put in the “request” step.
  • I then configured it as shown below.
  • 1 – For whom can trigger this flow, I set it to anyone as I do not know where it will be coming from.
  • 2 – I put in the Request BODY JSON Schema
    • This will be expecting to get the following below, all of which will have a schema of string
      • pipelineId – This is the unique pipeline ID from the Pipeline.
      • pipelineName – This is the name of the pipeline (makes it easier to identify)
      • customerName – This for me is a customer name, if I am expecting to get an alert from multiple customers (This could also be departments in a company).
  • 3 – The method I have set to “GET” this is because it means I can even test this in a browser
  • 4 – For the relative Path, this is how I am expected to receive the HTTPS GET, which allows me to later extract this information.

The next step is where I am then confirguring the email alert.

  • Below is how I have configured the step
  • 1 – This is the “Send an email (V2) step.
  • 2 – I put in the email address I want this to be sent to.
  • 3 – Here I am using the “pipelineName” which is passed in the HTTP request.
  • 4 – Here I am using the “customerName” which is passed in the HTTP request.
  • 5 – In the body I am again using the “pipelineName” which is passed in the HTTP request.
  • 6 – I am using the UTC date time and formatting it, making it easier to read.
  • 7 – The final text in the body I am using the “customerName” which is passed in the HTTP request.

I then save the flow to get the HTTP URL which I am then going to use in my pipeline.

The final step is to get the HTTP URL, which needs to be broken up into 2 parts to be used later in the pipeline step.

Configuring the Pipeline to send an alert if the notebook fails

The section below shows you how I configure the pipeline to send an alert.

NOTE: The way the Power Automate flow is created in a way that can be used for any pipeline, notebook or customer. This is so that it can be re-used in any pipeline.

  • Below are the steps on how to send the alert to a flow.
  • I go into my existing pipeline, where I am running a notebook.
  • I then add in a Web Activity as shown below.
  • I then give the Web activity a name “Send Power Automate Email if Fails”
  • I then click on Settings
    • Where it says connection, click on “More”
    • On the Web V2
    • Then configure it as shown below.
    • 1 – Past in the “HTTP URL” from the Power Automate flow
    • 2 – Leave the Token Audience Uri as blank
    • 3 – Create a new connection.
    • 4 – Give the connection a meaningful name “Power Automate – Notebook errors”
    • 5 – Leave the gateway as (none)
    • 6 – The authentication can be left as “Anonymous”
    • Click Connect.
  • Next is to configure the Relative URL
    • A white rectangle with black text AI-generated content may be incorrect.
    • Click on the Add dynamic content, because we want to pass through some parameters to the URL
    • In the Pipeline expression builder window, it is then configured with the following
    • A computer code with text AI-generated content may be incorrect.
    • 1 – This is the pipelineId from the variables in the pipeline.
    • 2 – This is the pipelineName from the variables in the pipeline.
    • 3 – Here I hardcode the value of the customer’s name, but this can use a parameter or variable.
    • NOTE: You will see after the customer’s name, in this example “FourMoo?” is where we add in the last part of the URL which has got the Api version and additional properties.
  • Make sure that the method is set to Get
  • The final step is to then configure the “On Fail” property and drag it to the Web activity for the “Send Power Automate Email if fails” as shown below.
    • A screenshot of a computer AI-generated content may be incorrect.

Below is what happening when the flow is run from a pipeline in Power Automate

I could see it ran successfully.

A screenshot of a computer AI-generated content may be incorrect.

And in the run settings I can see the customer and it was successfully emailed.

A screenshot of a computer code AI-generated content may be incorrect.

Summary

In this blog post I have shown you how to use Power Automate and Microsoft Fabric pipelines to ensure that I get notified when a notebook fails.

Thanks for reading, any comments and suggestions are most welcome.