Comments on: Solaris Containers and MySQL

By: Frank Mashraqi

Frank Mashraqi — Thu, 20 Mar 2008 00:54:20 +0000

Sheeri,

Aha!

Short answer: If all MySQL settings are similar, then your client’s response *is* the key to the difference in speed you’re seeing.

Long answer: I really can’t go into much detail publicly at this time as the contents of my answer would spoil my upcoming presentation and paper. However, call me and we can discuss it in detail. I am not sure if you have my old 706 number or new 551 number so I will email the current number to you after this.

Thanks!
Frank

By: Ronald Bradford

Ronald Bradford — Tue, 18 Mar 2008 20:12:19 +0000

I agree with the earlier comment via truss, or even dTrace.

Is MySQL doing the same thing every time, are you sure it’s always reading from memory. Reviewing SHOW STATUS difference before and after will just confirm, yes it’s all in memory, and MySQL is acting the same way. The easy part.

Truss with truss -o out.txt -rall -wall -p [pid] to see what internally mysql is doing.

Dtrace under Solaris is obviously another good reference, but I’ve got very little experience here.

Have you been for example to run a direct memory test, outside of the MySQL Database that confirms if the memory access is indeed bad.

Regards

Ronald

By: Sheeri Cabral

Sheeri Cabral — Tue, 18 Mar 2008 17:22:43 +0000

Frank,

The client says:

“Our dedicated X2100 is indeed newer M2 architecture with 667MHz DDR2 RAM vs. 400MHz from X4800 1st generation” but goes on to wonder if that could make the 4-fold difference in even simple queries.

By: John

John — Tue, 18 Mar 2008 13:16:25 +0000

While I’m no expert on the internals of the optimizer, is it possible that the optimizer is taking a slightly different access path for the data? There’s also the issue of I/O to consider. I’ve not yet played with containers in any depth so I can’t comment on I/O configuration.

By: Nitek

Nitek — Tue, 18 Mar 2008 07:51:15 +0000

Did you try to dump the database and reimport it? A few weeks ago i heared about someone having a similar problem, when using databases from older MySQL versions with on a server with a newer version. Dumping fixed that (something about storage format, can’t remember the exact explanation)

By: Frank

Frank — Tue, 18 Mar 2008 05:42:11 +0000

Hi Sheeri,

Which servers are these? V440? T1000? or X4100 series?

Frank

By: K

Tue, 18 Mar 2008 01:14:42 +0000

Interesting. Let’s start with the basiccs here. I assume you looked at EXPLAIN? And truss -c? After that, I think running MySQL in a debugger will give the definitive answer to why this is happening.

By: Justin Swanhart

Justin Swanhart — Mon, 17 Mar 2008 23:42:41 +0000

Even though the boxes may be identical hardware, that hardware might be configured differently.

Check RAID block sizes, filesystem block sizes and make sure all the sysctl variables are the same for both machines.

This could be a filesystem fragmentation issue. Try copying the table to a temporary (memory/shm based) filesystem on both machines and see if the performance characteristics change.

By: Jay Pipes

Jay Pipes — Mon, 17 Mar 2008 22:44:29 +0000

Weird. Though, must be noted that doing a SELECT COUNT(*) on an InnoDB table with no WHERE clause on an indexed column is a death sentence when the table has more than just a few thousand rows…

[[Sheeri’s answer]] True, but the point is to get a baseline. We were seeing it for more complex queries and we just wanted a very simple query on commandline to test. :)

By: Chen Shapira

Chen Shapira — Mon, 17 Mar 2008 21:26:59 +0000

Do you actually have multiple systems on the machine with containers?
If so, it is likely that you are not making full use of CPU cache, which can easily cause noticeable performance issues.

I’d check with Sun support if they have a tool that will allow you to compare how long pages stay in cpu cache between the systems.

I’m not sure which specific sun server you are using, but if it is NUMA architecture, there are different RAM speeds involved.