It’s true that the query cache is widely used, though how many times do you end up turning off the query cache? For me it’s about 50% of the time…..so I’m talking specifically about the subset of clients I’ve seen in the past 6 months or so….

Pythian checks that and many other variables on our initial audits of systems; so yes, we see it being used widely, but we ensure that we recommend using it only where appropriate, and we recommend turning it off when appropriate as well.

You said it’s not widely used — actually I’d say about 90% of new clients I see ARE using it. This week even I saw someone with a 1GB query cache (I’ve seen bigger, but not often). I think that probably the most common reason for this is that intermediate-level users turn it on when trying to get more performance from their systems.

That’s all just wild guessing.

My point is that the “noise” from the bad query cache may not have been noticed — at least not yet. I’ve had experience going from 5.0.45 and 5.0.51 to 5.1.31 and up, and experience going to 5.0.84, but I can’t recall going from 5.0.84 to 5.1.3X — there hasn’t been a driving need to upgrade 5.0.84 clients to 5.1 yet…..So maybe I haven’t seen it (and in general it hasn’t been seen) because very few people are migrating from 5.0.84 to 5.1.3X.

On the whole, the worst-case behavior of the query cache is unacceptable even in the “better” stage, and given the many different operations that happen in a “worst-case system” to a “real life system”, it may be that only the worst-case system is affected by whatever was changed. Hard to know, really.

It also might be that the performance is only affected when you’re filling the query cache, ie when it’s mostly empty and you’re adding queries….if that’s the case, slowness wouldn’t be noticed because there’s slowness when mysqld restarts due to filling the innodb_buffer_pool too.

So….it’s hard to say 1) if the problem you found exists in real scenarios where the query cache is used and 2) if it does, the problem might be small and transient, and not really significant.

But to answer your question, we have *not* seen a drastic reduction in queries due to upgrading and the query cache.

(we have seen this: http://bugs.mysql.com/bug.php?id=36259 which is fixed in 5.1.37, and noticed as early as 5.0.22…, but that was after we’d defragmented some tables as part of an upgrade, since we had a long downtime window….)

The point of my post was that worst-case behavior of the query cache changed significantly from 5.0.44 (bad) to 5.0.84 (better) to 5.1.3X (bad again). Whether or not the query cache is useful is a different issue. Changes in code that lead to drastic changes in performance make stable systems much harder to build.

Are you aware of this behavior change in the query cache?

There’s even a memcached storage engine.

http://www.mysqlperformanceblog.com/2006/07/27/mysql-query-cache/
states that there are some good reasons to use the query cache:

“Third party application – You can’t change how it works with MySQL to add caching but you can enable query cache so it works faster.

Low load applications – If you’re building application which is not designed for extreme load, like many personal application query cache might be all you need. Especially if it is mostly read only scenario. ”

And I totally agree, with one addition: If you have the time and knowledge to use the query cache appropriately but not to use other tools (such as memcached) appropriately, then it’s the right solution. Using memcached means rewriting code, which a lot of organizations are loath to do. But definitely, it’s a HUGE win, and I recommend it over the MySQL query cache for most cases.

I just don’t think the query cache should be removed. Perhaps tweaked a bit, so that it won’t check the query cache for every SELECT query, more insight into what’s in the query cache itself, etc.

I have also seen it make things worse when turned on blindly.

I think that if we removed every feature that “most people” might turn on without looking at it because doing that causes bad performance, we’d have to turn off lots of features and remove a lot of custom SQL that is currently allowed.

The query cache is off by default, which is as it should be.

There are warnings on the manual page, which “most users” will actually read.

to at some determined thingy(technical term) move the query cached queries to memcached cluster.. Of course dispose of them per thingy as needed by some mid tier thingy doing all this juggling.. I’ll bet one could get very nice performance..

just a thought..theory.. speculation