Quantcast
Channel: Table.Buffer for cashing intermediate query results or how workaround Unnecessary Queries Issue
Viewing all articles
Browse latest Browse all 21

Table.Buffer for cashing intermediate query results or how workaround Unnecessary Queries Issue

$
0
0

I still optimizing my task, and meanwhile it seems now that I fully understand what Ehren meant in this phrase: "you could think of Table.Buffer as transferring a table from the fuzzy unpredictable world of folding to the immutable world of pure M values". And this is very important, I would say conceptual thing :-).

What I understand at this moment is that Buffer plus immutability concept is a very cool thing. So as soon as you say to PQ (using Table.Buffer, or Binary.Buffer and so on) that all the sources used for calculating a variable are immutable, the whole value became immutable for PQ, and that means that it won't be recalculated _in the same evaluation process_. I mean that if you have Q0=mutable stuff -> Q1s1 = Table.Buffer(Q0) .. Q1sN -> Q2s1(Q1) .. Q2sN(Q1), Q1 will be calculated only once during the whole evaluation process, despite the fact that it is called twice (usual thing for M, but not at all for variables with mutable sources). And that's really cool. And it seems that it doesn't matter on which step of Q you use Buffer (that is logical).

This is cool because, from my experience, PQ sometime "losts" the state of immutability due to not so obvious reasons. And due to immutability + Buffer you can make (and should) the whole chain immutable (by using Buffer) just at the Q which will be reused elsewhere, and not at the point you open an mutable datasource. And all the calculations before and in the Q won't be recalculated.

But anyway, from my experience, you should re-check whether immutability works or not in your case - through tracing or Process monitor (Thanks a lot to Chris for his very helpful blog!).

And of course this concept won't help while working with UI, because if you launch the same evaluation chain again, the whole chain will be calculated again. So it would be great if, while working in UI, PQ would consider Buffered variables as immutable between explicit refreshes by a user (using Refresh/refresh All of the Ribbon).









Viewing all articles
Browse latest Browse all 21

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>