Do not process all files again, just new files

rolo
Posts: 15
Joined: Tue Sep 27, 2016 8:14 am

Do not process all files again, just new files

Postby rolo » Thu May 18, 2017 9:59 pm

Definition:
I consolidate many files with PowerQuery using "From Folder"
Then my queries do a lot of work to flatten my data
Each week I add lot of new files to my folder

My problem:
When I refresh the queries is taking more and more time to process,
because each week I add 20 new files and PowerQuery process all files again!
By the end of the year I will have to wait a lot!

Question:
Is there any way to let PowerQuery process only new files or just files after some date
to avoid processing all files again and again?

Thanks!

MattAllington
Posts: 884
Joined: Sun May 04, 2014 4:01 pm
Location: Sydney, Australia

Re: Do not process all files again, just new files

Postby MattAllington » Sun May 21, 2017 4:03 pm

In power bi Desktop you can disable refresh. You would process all "history" files and set them to not refresh again. Then append the new ones to this non refreshing table.

This can't currently be done in PQ for Excel.
Matt Allington is Professional Self Service BI Consultant, Trainer and Author of the book "Learn to Write DAX". You can hire me at http://Exceleratorbi.com.au
http://exceleratorbi.com.au/what-is-power-pivot/
http://xbi.com.au/learndax


Return to “Power Query”

Who is online

Users browsing this forum: No registered users and 1 guest