Hi Andrey and Chris. Apologies for the delay. I've been trying to wrap my head around the details of how evaluation and caching work, since your questions are delving into aspects of the system I've not had to understand before.
Chris, regarding your questions about memory limits: I'm not sure. I'll need more time to answer this. And yes, caching still takes place independent of Table.Buffer. The Excel change was to have each refresh of N queries share a temporary persistent cache, instead of each query getting its own temporary cache.
Andrey, the error you listed is not surprising. It's essentially a cache miss. We encountered an IO error trying to read something from the persistent cache. That said, it's possible it's indicative of a deeper issue (for example, maybe mashup container A is accessing the cache while container B is updating it, resulting in the container A redoing the work B already did). I haven't had time to dig in and find out.
However, what I have had time to do is create a gargantuan diagram showing how refresh and caching and buffering work in a simple-ish sample scenario. I haven't yet vetted it with my teammates, so take it with a grain of salt. (I'll post an updated version once I'm able to gather feedback.) But it does distill what I understand about the innards of PQ, based on debugging through various parts of our code. You may consider it an early Christmas gift or a confusing lump of coal, depending on your view, but I thought I would share it now. It's not pretty, but I hope you find it helpful. Then again, it might just raise more questions than it answers. :)
Here's the sample M:
shared Query1 = let
Source = Sql.Database("SomeSQLServer", "MyDb"),
MyTable = Source{[Item="MyTable"]}[Data],
BufferMyTable = Table.Buffer(dbo_MyTable)
in
BufferMyTable;
shared Query2 = let
Source = Query1
in
Source;
And here's the diagram (you'll likely need to zoom in to read everything):
As you might notice, there are a number of extra evaluations even in this simple scenario that will be eliminated if you disable background analysis (they will still happen on-demand, but won't happen automatically). There can also be extra evaluations performed by the firewall in its attempt to determine how to safely partition a given set of queries (there's one such evaluation in this scenario, but can be more depending on the complexity of your queries). If you disable the firewall, these will also no longer occur. Finally, I hope it's clear that Table.Buffer is limited in scope. It only affects a single evaluation of a top-level query, and in some scenarios (such as the one above) may actually degrade performance due to the extra unnecessary work it performs (such as reading through all the data when only zero or 1k rows are being requested).
EDIT: Updated the diagram on Jan 9th.
Ehren