5

Query results in SEDE are cached:

enter image description here

How can one see when a query result was last cached in SEDE?

3
  • @rene Thanks, sorry I don't see the answer in the duplicate. Where is the answer? The duplicate seems to be about disabling caching. Commented Apr 18 at 19:54
  • 4
    You can't see when it was cached. Surfacing that info is useless as you can't uncache it. If you need the fresh query result, re-run with other parameters and/or add the cache breaker as shown in the duplicate . Query results are reset when the SEDE refresh runs.
    – rene
    Commented Apr 18 at 20:06
  • @rene It's useful to see if SEDE has refreshed. Can you please reopened + move your comment as an answer? thanks Commented Apr 18 at 20:11

1 Answer 1

8

For an answer to your direct question:

How can one see when a query result was last cached in SEDE?

One can't. This information is simply not exposed to SEDE.

  • Cached plans and queries are cleared out during every refresh operation, so its age will always be 7 days or less (unless that specific step in the process fails).
  • If one is trying to defeat the cache (force their query to run again), see How can I avoid the Data Explorer cache?

For the reasoning behind your question:

It's useful to see if SEDE has refreshed.

There are much better and more direct ways to do this. rene used to use this query:

We recently added some views to Data Explorer in part to provide a lot more transparency into this process. With those views, a couple of ways you could check on us to be sure we actually performed the most recent weekend's refresh (along with exactly when it happened, how long each database took, and what data was included):

On any individual database, you can check to be sure that data from the most recent refresh bumps up against midnight UTC on Sunday (which actually has nothing to do with the new views, and could have been done in the past):

You could run that on any site, but there are some caveats:

  • Not every site has enough activity that this information will be conclusive. Some sites go days without Posts, so I use our busiest site as the model.
  • While we've attempted to fix the "jagged edge" problem - where the data included in SEDE depended on when a database was loaded - some straggling information can still get through. For example, if a post created at 23:59 is modified at 00:20, that modification might be captured if the source row in Posts is read later than 00:20. But the corresponding PostHistory row will not be captured.
4
  • 3
    Note that users don't have to run a query as the home page of SEDE shows when the data was refreshed. I have a pending PR on that: meta.stackexchange.com/a/354134 but that rightfully didn't get merged. I think using the new view is preferred in that view (and address the performance / caching due to being run on the homepage).
    – rene
    Commented Apr 19 at 15:28
  • Got it, thanks very much! Fyi not important but just to let you know, I believe that some cached plans and queries were cleared out only yesterday instead of last Sunday, based on 2 queries I've tried and my admittedly imperfect memory. (The 2 queries were returning cache results from the Sunday, April 7 refresh instead of the Sunday, April 14. They only returned the updated results corresponding to the Sunday, April 14 refresh since yesterday.) Commented Apr 19 at 21:33
  • @AaronBertrand thanks, interesting. I'll let you know if that happens again. The two queries were data.stackexchange.com/stackoverflow/query/540572/… and data.stackexchange.com/stackoverflow/query/851061/… I was mostly looking at the first query. Are SEDE results cached on client side, or solely on server side?. Commented Apr 19 at 21:52
  • querying PostHistory is probably better than Posts if you just want to pick up any "sign of life". It'll include community bumps, so even if the site is super dead, I assume there's still a better chance it'll have some movement every SEDE refresh.
    – starball
    Commented Apr 20 at 4:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .