Pythian Group

Why a New Utility?

A couple of months back, Tim Procter, Sheeri Cabral and I were discussing about how best to diagnose a MySQL server and/or tune its performance, automating the process as much as possible. The Performance Advisors from MySQL Enterprise do this, but most of our customers don’t have a subscription and Pythian’s collective experience is not necessary reflected by its rules.

In our daily work, we have used Major Heyden’s MySQL Tuner, Mark Leith’s Statpack and our own tools to review a MySQL server configuration parameters. However, all of these tools had limitations in regards of what we wanted to achieve. Our major concern was the need to keep historical data to draw conclusions based on trends and cross check with other time based tools. The conclusion of our conversations was that whatever scripts we would implement, it would need to monitor all configuration and status variables, and their evolution over time.

One day, while looking into slave lag problem, the idea of MySAR popped into my head and a few hours later I was using its first incarnation. I was able to relate the server’s I/O activity peaks with these lags and in turn, discovered that it was caused by a great number of INSERT statements coming in in waves. It was an encouraging outcome for what was nothing more than a proof of concept.

Read the rest of this entry . . .

This is an issue that keeps rearing its ugly head over and over again, and since it greatly affects performance, it is most important that DBAs of any DMBS running on Linux come to grips with it. So I decided to do some research and try different settings on my notebook. Here are my findings.

What can you find on the web?

A Wikipedia search for the word swappiness will come up empty (any volunteers out there want to write an article?). A Google search will show some pretty old material—the best article I found is from 2004: Linux: Tuning Swappiness. This article includes a detailed discussion with some interesting remarks by Andrew Morton, a Linux kernel maintainer.

So, what is swappiness?

Read the rest of this entry . . .

A few days ago I had a new idea for a blog post. A post about what it really takes to be a good database administrator. I began by researching what others had done on the topic. At the end of this post you will find links to six of the posts I found that provided some insight into this question. Even after uncovering this information, I thought I could add something to the mix from my own experiences. So here we go!

Read the rest of this entry . . .

I was contacted by the folks at MONyog and asked if I would review MONyog. Since using MONyog is something I have been wanting to do for a while, I jumped at the chance. Of course, “jumped” is relative; Rohit asked me at the MySQL User Conference back in April, and here it is two months later, in June. My apologies to folks for being slow.

This review is an overall review of MONyog as well as specifically reviewing the newest features released in the recent beta (Version 2.5 Beta 2). Feature requests are easily delineated with (feature request). This review is quite long, feel free to bookmark it and read it at your leisure. If you have comments please add them, even if it takes a while for you to read this entire article.

While the webyog website gives some information about what MONyog can do, it is a bit vague about what MONyog is, although there is a link to a PDF whitepaper on What is MONyog? which does answer much of these questions.

The screenshots available from the website are accurate, so I will not reproduce them here. I will note that I have not shared this feedback with the webyog team yet, so I may be upset that a feature is lacking, and the feature may be implemented but I missed it. I will post a follow-up in that case, even though they will likely comment here too.

My reference points — I have used other monitoring and graphing tools such as Nagios, Cacti, and Intermapper as well as MySQL’s Enterprise Monitor.

Overall

As an overall review — MONyog is the best out-of-the-box GUI monitoring tool for MySQL that I have seen. It “just works.” As promised, getting up and running quickly is easy, and having a centralized location for monitoring is very useful. The graphs are beautiful and the statistics that are graphed are useful time-savers.
Read the rest of this entry . . .

In case you are attending Oracle Open World 2008, the biggest Oracle conference in the world, and interested in either (or both) MySQL or Oracle Enterprise Manager Extensibility — I posted a proposal for a new presentation:

Extending Oracle Enterprise Manager by Example — Creating MySQL Management Plug-In

I’ve started looking into Oracle extensibility several years ago and since then I’ve seen lots of improvements in Extensibility Guide and many new plug-ins have seen the light of the day. However, creating a new plug-in is still considered to be something special and not available to mere mortals.
In this presentation we will see how easy it is to create a new plug-in. What are the steps and the method to follow. As an example we will work with MySQL Management Plug-in that I have recently released to public.
This session includes a live demo.

If you are interested, you may as well vote for it. If there are enough interested people, Oracle might select it for the conference.

If neither this nor my previous submission make it, well, I’ll go there anyway to have some fun and meet good old and, hopefully, new friends.

Please note: This plug-in has been created and is now available for download:

MySQL Plugin for Grid Control

It’s been a while since the MySQL Management Plug-in 0.42 was released. Since then, I quietly updated it to version 1.0. The changes were very few; the biggest news was that the plug-in was certified by Oracle and added to OTN Oracle 10g Grid Control Extensions Exchange (see at the bottom).

I think the next version is due, as a few people have come back to me with some issues. The biggest was compatibility with Windows. Since I used the command line MySQL client, *nix and Windows shell incompatibilities were a major headache to solve, and I still couldn’t make it work reliably. I wanted to use DBI and DBD:MySQL, but it required installing and compiling Perl packages, which makes the deployment process very inconvenient.

Finally, I found a solution — Net::MySQL is a native Perl implementation of the MySQL client. I had to fix some bugs and add a few improvements to it, and I hope to get the author to re-introduce them back to the new CPAN distribution. Net::MySQL is dependent on IO::Socket, which is a core module that comes with the standard Perl distributed with the Oracle Management Agent.

Version 1.1 turned out to be a major rewrite for the Perl collection scripts and the net result is that compatibility across platforms is greatly improved. I have successfully tested the new version on Linux and Windows Agent hosts.

So what’s new in version 1.1 compared to 0.42?

• certified by Oracle; see OTN Extensions Exchange
• no MySQL client is required on Agent hosts. The Perl Net::MySQL package is distributed with the plug-in
• fully compatible with Windows
• MySQL client path property removed
• added support for local connection using Unix sockets
• added connection error message when target is down — can be seen in Availability History
• commands statistics skips collection for never-executed commands so less data is collected; thus, I could safely increased default collection frequency; command names are formated better
• changed default collection frequency for network, joins and sort statistics
• % command executions are collected right now — the “Questions” statistic didn’t match the total of all Com_ statistics.
• metric “Processes by Action” now excludes the plug-in’s own connection which was always adding one to “Query”
• a few minor typos fixed

Downloads, requirements, and installation instructions — as well as the datasheet — are available at the MySQL Plug-in for Oracle Grid Control home page.

You can also go directly to the plug-in download page by clicking here:

MySQL Plugin for Grid Control

Update: I’m currently working with Oracle development on certifying this plug-in.

Further update:Oracle has certified this plug-in as the official MySQL plug-in for Oracle Grid Control

To go directly to the download page, click below:

MySQL Plugin for Grid Control

It has finally happened! The first public release of the MySQL plug-in for Oracle Enterprise Manager 10g Grid Control is out.

It’s been a while since I first started to work on this, first as part of the Grid Control Extensibility article that I wrote for IOUG SELECT Magazine in 2006 (thanks to John Kanagaraj for encouraging me to write it), and then later as part of a demo for my presentations. I was already working at Pythian and was considering further developing my example and releasing it.

In the last few months, more and more people showed interest in the potential plug-in, and even few guys from Oracle contacted me independently of each other with their own reasons to have a MySQL plug-in available. This interest accelerated the fermentation of thoughts in my brain, and I got down to work. Pythian generously sponsored my development time.

I’ve called this first release beta but as I didn’t do much testing, it should probably be called alpha. Having said that, it has been tested with MySQL 5.1 and 5.0, and it should also work with MySQL 4.1. I develop and use it with Oracle Management Agents running on the Linux platform, but I changed all the collection scripts to use the Perl that comes with the Oracle agent so it should run on Windows as well. It works quite stably for me, and I have verified it in several real-life environments.

I would probably take some more time before releasing it, but I really wanted to have the plug-in out before I leave for COLLABORATE 08 (should I also tell you the readiness level of my presentation?). I hope to get some feedback and first impressions from DBAs who try the plug-in. Bear in mind that this is the very first release — expect some rough edges. Please do report them here. We will probably set up a more structured set of pages, but for now leaving comments here is the way to go.

Here are the details. The first release version is 0.42 because that’s obviously the perfect number to start something good.

What’s implemented so far

Read the rest of this entry . . .