Looping through data using PySpark notebook in Fabric
Fabric Notebooks – Looping through data using PySpark Continuing with my existing blog series on what I’m learning with notebooks and PySpark. Today, I’m going to explain to you how I found a way to loop through data in a notebook. In this example, I’m going to show you how I loop through a range of dates, which can then…
Using Sempy to Authenticate to Fabric/Power BI APIs using Service Principal and Azure Key Vault
I have been doing a fair amount of work lately with Fabric Notebooks. I am always conscious to ensure that when I am authenticating using a Service Principal, I can make sure it is as secure as possible. To do this I have found that I can use the Azure Key Vault and Azure identity to successfully authenticate. By using…
How to add current DateTime to existing PySpark data frame in a Fabric Notebook
How to add current DateTime to existing PySpark data frame in a Fabric Notebook In the blog post below, I am going to describe how to add the current Date Time to your existing Spark data frame. This is really useful when I am inserting data into a Fabric Lakehouse table, and I want to know when the data got…
All Fabric Workload Items are now available from the Scanner API
All Fabric Workload Items are now available from the Scanner API I was working with the customer and was looking for some information in the Scanner API. For a change I went into Power Query and expanded the workspaces item. And this is where I found now that all the fabric workload items are available. You might notice in the…
How much data is being stored in my Fabric OneLake (Lakehouse files and tables)
How much data is being stored in my Fabric OneLake (Lakehouse files and tables) I recently had a question from someone in the fabric community asking how much storage was being used by the Lakehouse and tables. This got me thinking that currently there is not an easy way to do this. It is essential to know how much storage…
Did you know you can create DAX measures directly in the Power BI Service using Visual Calculations
There have been some updates to Power BI with Visual Calculations, and as of the March 2024 release you can now create and edit visual calculations in the Power BI Service. Here are details from the blog post: Power BI March 2024 Feature Summary | Visual Calculations Update I then saw an email which made me think, could I then…
Creating a Mirrored Azure SQL Database in Fabric
Creating a Mirrored Azure SQL Database in Fabric This week they announced Announcing the Public Preview of Database Mirroring in Microsoft Fabric | Microsoft Power BI Blog | Microsoft Power BI I decided to see how easy it was to create a mirrored database in Fabric and below are my findings (PS it is AMAZING) I did follow the tutorial,…
How to get Sempy (Semantic-link) to run when being triggered from a data pipeline which runs a Notebook in Fabric
Below is where I had an error when trying to run a notebook via a data pipeline and it failed. Below are the steps to get this working. This was the error message I got as shown below. Notebook execution failed at Notebook service with http status code – ‘200’, please check the Run logs on Notebook, additional details –…
Renaming multiple Column Names in a single step using a PySpark Notebook
Following on from my previous blog post this blog post I’m going to demonstrate how to bulk rename column names in a single step instead of having to rename them individually. The reason this came about is because I had a set of data where the column names had the square brackets which I wanted to remove. As shown below…
Fabric – How to rename a column name when using a Python Notebook
I thought it would be good to help others in terms of my learning journey when working with partner notebooks and Microsoft fabric. In today’s blog post, I am going to show you how to rename a column. In my experience this came up because I had a column name which had a forward slash “/” in it which caused…