Using the Power Function.InvokeAfter to determine how long to wait between API calls
I have been working on a dataset which I will hopefully reveal soon, but part of that was that I was getting rate limited when making an API call.
I found Chris Webb’s insightful blog post (Using Function.InvokeAfter() In Power Query) where he details how to use the Function.InvokeAfter. The one key piece that I personally found missing was how to use this with an existing function that I had created.
I then got another fantastic question from the Power BI Community where they were looking to do an IP Address Lookup. And there are a lot of sites who offer this, but they do limit the rate at which you can query the API (Which I think is perfectly understandable considering they are offering it for free!)
My blog post shows how I ensured that I did not exceed the rate limit for the API using the Function.InvokeAfter
NOTE: I am not going to cover how I converted the IP Address to a location, I have done this previously in (Power BI Query Editor – Getting IP Address Details from IP Address)
Using the Custom Function
I am starting off where I have already created the Custom Function in Power Query Editor. I also have got a table with IP Addresses.
-
I have used a sample file in which I made up the IP Addresses as shown below.
-
I then went into the Add Column in the Ribbon and clicked on Invoke Custom Function
-
This brings up the Invoke Custom Function
window and I put in the following information as shown below.- As you can see from above, I gave my new column a name of Details
- I then clicked the drop down and selected my function I created earlier called fx_GetIPAddressDetails
- And then finally the crucial
part is where I selected
my IP Address Column. - I then clicked Ok.
-
When you do this it returns a table as shown below.
-
Click on the Expand
Table Button on the top right hand side, which will then prompt you which columns you want to select - I then selected Country
-
And below were my results.
Adding in the Function.InvokeAfter to limit the rate at which I query the API
I am now going to modify the step where I Invoke the Custom Function to limit how long it waits between API calls.
-
I created a new Parameter called “Interval (Secs) as shown below.
-
I then went to the following step in my table, and clicked on the Step called “Invoked Custom Function”
- Then in the formula bar I have got the following.
= Table.AddColumn(#”Changed Type”, “fx_GetIPAddressDetails”, each fx_GetIPAddressDetails([IP Address]))
-
Next, I made the following changes to the above code using the Function.InvokeAfter
= Table.AddColumn(#”Changed Type”, “fx_GetIPAddressDetails”, each Function.InvokeAfter(()=>fx_GetIPAddressDetails([IP Address]), #duration(0,0,0,#”Interval (Secs)”)))
- I added the Function.InvokeAfter(()=>
before I called my function fx_GetIPAddressDetails which is highlighted in BLUE above -
I then put in the #duration(0,0,0,#”Interval (Secs)”))) which is highlighted in PURPLE above.
- Within the #duration I also used my Parameter called #”Interval (Secs)”
which is highlighted in ORANGE. - This allowed me the flexibility to change the rate limit timing without having to go into the code.
- Within the #duration I also used my Parameter called #”Interval (Secs)”
- Now when I refreshed the data it will wait 2 seconds between each API Call.
I hope that this has been useful and an easier way to ensure that you can limit how quickly you call an API
As always please leave any comments in the area below.
Here is a copy to the PBIX file that I used in the blog post above: FourMoo – Loading IP Addresses with an Interval.pbix
Hi Gilbert
And how many API calls you have got for 3 rows? I think it still more than 3…
Cheers,
Maxim
It could possibly be more than 3 calls. I would expect it to be one call per row due to it being executed in a single row at a time?
But will you do this in a custom connector where you fire multiple calls in multiple functions?
It could possibly be done in a custom connector, anything that is done in the Power Query Editor could be put into a custom connector.