Review of new Features coming to Power BI by Oct 2018 – From Business Applications Summit
I was fortunate to attend my first ever Business Applications Summit in Seattle. I had the pleasure of sharing an Airbnb with Matt Allington, Phil Seamark and Miguel Escobar, it was great to spend time with these Power BI Legends. I also did meet a lot of people and got to chat with people in the Microsoft Power BI team,…
SSIS – Using Date and Time Cache Lookups for faster inserts into Fact Tables
What happened is that I found that when doing inserts on larger data sets, it would take a long time to complete the date and time joins. If I excluded this the data would be returned a lot quicker. So my solution to the problem was to use the Cache Transform and then the Loopkup Transform in the SSIS…
SQL Server Integration Services (SSIS) – Using Send Mail Task for data validation checks
What I wanted to do was to be notified in my data if there were not enough rows of data, or if the financials were too low, indicating that there was a potential issue with the data coming in. And if it was too low to fire off an email. If it was correct then to not send an email….
SQL Server Integration Services (SSIS) – Re-run a specific Control Flow Item if it has an error
We had an issue where we only wanted to re-run a particular control flow item when it failed or had an error. We did not want to do this for our Entire SSIS Package but just for a specific Control Flow Item. Example: Inserting Data into a Staging Table and if it fails to then re-try the insert 3…
SSIS – Using SQL Server Analysis Services (SSAS) MDX query into Variable in Execute SQL Task
What I wanted to do was to get the output from my MDX query, and then use this in SSIS and take the output and put it into an SSIS Variable which could then be used later in your SSIS Package. Example: · We are going to get the Internet Sales Amounts for 2 days. o 01 June 2008…
SSIS (SQL Server Integration Services) – Using the Lookup Transformation and cache and how to handle NULL Values
I had a situation where I was using the Lookup transformation and then loading this into the SSIS Cache, but I wanted all rows to be inserted using the SSIS Cache, even if there was a NULL Value. Below explains how I overcame this. Example: · In our Source data we have Products, but they might not be in…
SSIS – Running single SSIS packages in Parallel to get faster performance for multiple data loads
What we are going to show you below, is that by using ONE SSIS package, we can re-use this one SSIS package multiple times to get data from multiple sources, possibly from different databases hosted in different locations around the world. So in effect we are loading our data from one SSIS package in parallel. NOTE: By completing the…
SSIS – Getting Partition Names for the past 12 months
I had a situation where I was using a conditional split, so that when data was coming in from my source tables it would dynamically go into the correct partitioned table. Below is the script that I used, as well as for reference the conditional split — ================================================================================================================== — What we are doing here is to create a…
SCOM (Systems Center Operations Manager) Cube and Data warehouse
What I have done is I have found that it is very handy having all our SCOM data in a data warehouse and then having it in a SSAS (SQL Server Analysis Services). Within the zip file which can be located here below it has all the details for the following: · SCOM Data Warehouse and OLAP Documentation.docx o This…
Data warehousing – Bringing down data twice but avoiding deleting on the data warehouse Fact Table
This is better explained with our example below. Example: · We had a situation where we were getting data down, but due to the nature of the data we were getting is we were missing data. · Also another note was the fact that we did not store the granular data, but aggregated the data, so due to this…