Hi there, I can't believe that this is so easy!

I came across this post from Tacky Tech Tom and I could not believe how easy it seemed!
I have a lot of experience writing PowerShell scripts extracting the list of workspaces, reports etc via calling the REST API. PowerShell is not a common skill, you can't just ask from help from the next person in the cubicle or in the team, so I relied on Stackoverflow. The Tom's post takes the list of workspaces from the Power BI REST API.
This is not the use-case I am writing about now.

Have you read this post from Marc Lelijveld?
https://data-marc.com/2024/05/28/dynamically-refreshing-historical-partitions-in-power-bi-incremental-refresh-semantic-models-using-fabric-semantic-link/

Marc's notebook is amazing for semantic model refresh automation, the first half of the notebook works for datasets without incremental refresh by the way. Yes, I am using both the semantic model and the dataset terms in the same sentence.

What if you have a Gen 1 dataflow that needs to be refreshed before your dataset, before the notebook runs? If you have a dataflow that refreshes your transaction table, the basic setup is to refresh your dataset at the next 30 minutes increment, for example your dataflow refresh starts at 1PM then you'll set your model to refresh at 13:30. The time difference is not meaningful in most cases but it's also not elegant. I know that you can solve this use-case with a Power Automate sending a HTTP request but I suspect that method has its own difficulties. By the way a data pipeline has better granularity for scheduling refreshes which can be a use-case on its own.

I wanted to see if sending a POST request to trigger a dataflow request is just as simple as getting the list of workspaces. Well, almost. The first error implied that I need the have a body defined in the request which can be the same from the documentation {"notifyOption":"MailOnFailure"} . The second one was about that the relative URL was added to the JSON body as text with quotation marks, to solve this I had to add a header name and value : Content-Type application/json.

Amazing, it works! Now, let's add a semantic model refresh activity and test it. Hmm, the dataflow refreshed at 12:21 and the report refreshed at 12:15. Why? Well in this case the web V2 activity sends a POST request but we don't know how long it takes or whether it was successful. I am sure there is a technical way to overcome this but at this point with my existing Fabric knowledge I don't want to overcomplicate it. I just simply checked how long it usually takes for the dataflow to refresh and multiplied it by 50% and added a Wait activity between the POST request and the semantic refresh. Success! This pipeIine does not use code and only consists of 3 activities.

By the way, I really recommend this study community and course about Fabric, creating pipelines is not something you can learn by just watching YouTube videos. https://www.skool.com/fabricdojo