Archive for the ‘MySQL’ Category

The Architecture Layer

Tuesday, May 13th, 2008

Contemporary software engineering models include many loosely-defined layers. Database developers might help with other layers, but for the most part a database administrator’s domain is the persistence layer.


  • Presentation

  • Application

  • Business Logic

  • Persistence (also called Storage)

The Daily WTF has an article on The Mythical Business Layer makes the case for not separating the business layer and the application layer:

A good system (as in, one that’s maintainable by other people) has no choice but to duplicate, triplicate, or even-more-licate business logic. If Account_Number is a seven-digit required field, it should be declared as CHAR(7) NOT NULL in the database and have some client-side code to validate it was entered as seven digits. If the system allows data entry in other places by other means, that means more duplication of the Account_Number logic is required.

It almost goes without saying that business logic changes frequently and in unpredictable ways. The solution to this problem is not a cleverly coded business layer. Unfortunately, it’s much more boring than that. Accommodating change can only be accomplished through careful analysis and thorough testing.

I will call this merged business/application layer the “functional layer.”

The serious scaling requirements posed by most applications these days call for partitioning, clustering, sharding or some other term for “dividing up the data so it does not become the bottleneck”. Enter the “architecture layer”.

“Wait a minute,” I hear you asking. “Isn’t that just the persistence layer?”

Yes and no. To me, there’s a difference between the storage and the architecture of said storage. The database schema for storing a user profile is a persistence layer issue. Figuring out which database instance to go to is an architecture layer issue.

This is an important distinction for me. Many folks are coding the architecture layer directly into the functional layer. A “save_profile()” API function might call an ORM to deal with the persistence, or it will have MySQL (or other database) connection handling and queries. However, the database will grow, and at some point you will find yourself wanting to split the data [more].

This type of information, like the presentation layer, needs to be separate. Why should the application care whether save_profile(’Sheeri’,'hair color’,'blonde’) accesses database1 or database2? More importantly, why should there be major code changes to the functional layer if the architecture changes? Just like no functionality has changed when you change your website color from blue to red, there is no functionality change when you go from splitting data between 2 database servers to splitting among 3, or 10.

For me, the persistence layer is about how the data is stored. Which, explicitly and for the record, I also believe should be separate from the functional layer — if you store hair color and eye color in one table or 2, the functionality of the application has not changed; all that’s needed is a change in how that data is stored and retrieved.

The architecture layer is all about where the data is stored. Early forms of the architecture layer are configuration files, though most would not call that a “layer”. Database administrators should be able to change the architecture of the database system without requiring mucking about in the application’s functional code.

Thoughts?

Bookmark online using:These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • digg
  • Reddit
  • Spurl
  • Furl
  • blogmarks

Log Buffer #96: a Carnival of the Vanities for DBAs

Friday, May 9th, 2008

This is the 96th edition of the weekly review of database blogs, Log Buffer.

Let’s start this one in SQL Server Land, with a question from Dennis Goboshould SQL Server have the CREATE [OR REPLACE] PROCEDURE syntax? There are, he writes, advantages: “When scripting out a database you don’t have to generate if exists…..drop statements,” and disadvantages: “I can overwrite a proc without even knowing it.” Of course, the commenters have opinions of their own, and the piece becomes a straw poll for the desirability of that syntax as a feature.

Aaron Bertrand has one too: when was my database/table last accessed? Writes Aaron, “SQL Server does not track this information for you. SELECT triggers still do not exist. Third party tools are expensive and can incur unexpected overhead. And people continue to be reluctant or unable to constrain table access via stored procedures, which could otherwise perform simple logging.” He looks at 2008’s built-in auditing, and for those who can’t wait for that, illustrates a workaround for 2005.

Linchi Shea explores something else from 2008, Page Compression, focusing on how the number of processors affects the rebuilding a table with page compression.

Jamie Thomson, the SSIS Junkie writes that he has made a submission to Connect on the matter of absolute and relative paths in SSIS. “. . . I have always agreed that stipulating the use of absolute paths within SSIS was the right thing to do (and indeed I have championed it) however of late I have changed my mind. Support for relative paths would greatly simplify package deployment and package management . . . What do you think? Should SSIS support relative paths?” So far, it looks like a shoo-in.

Brian Knight also explains another little quirk, SSIS Case Sensitivity: “The case sensitivity can in some cases create behavior that is not expected and may give you bad results if you’re not careful.  . . . One such example is with the Lookup Transform, where comparisons against the cache are case sensitive. If you do not expect this, you may have a miss in a match that is actually a hit.”

In the MySQL ’sphere this week, there is plenty of talk about the openness or otherwise of MySQL. (more…)

Bookmark online using:These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • digg
  • Reddit
  • Spurl
  • Furl
  • blogmarks

Log Buffer #95: a Carnival of the Vanities for DBAs

Friday, May 2nd, 2008

The 95th edition of Log Buffer, the weekly review of database blogs, has been published by Mark Schoonover on his Mark’s IT Blog.

We can look forward to LB#98 Jeff Smith’s Jeff’s SQL Server Blog on May 23rd. There’s always plenty of room for more editors, so don’t waste another minute — send an email to me, the Log Buffer coordinator, and get started!

Without further ado, here is Mark Schoonover’s Log Buffer #95.

Bookmark online using:These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • digg
  • Reddit
  • Spurl
  • Furl
  • blogmarks

2008 MySQL Conference Videos, Notes, Slides and Photos!

Thursday, May 1st, 2008

All of the videos from the 2008 MySQL Conference have been processed and uploaded. Links to the videos, slides, notes, photos for each presentation are all on the mega-conference page at:
http://forge.mysql.com/wiki/MySQLConf2008Notes

This represents many hours of my own toil, but it also reflects plenty of people who have blogged, edited the wiki pages and speakers who wrote and gave tutorials and presentations. I am proud of everyone’s efforts to offer so many learning resources for free….

Enjoy! EDIT: I forgot to thank Jay, the folks at O’Reilly and all the speakers for giving me explicit permission to video and freely offer their presentations.

If you know of any video, audio, notes, slides, photos, etc that are not linked, please link them at the wiki page. If you can’t or won’t, please comment here and I will update the wiki for you.

Please note that there’s still some work to be done for a volunteer — Currently there is no one page where you can get all the videos, notes and slides for a presentation. The Forge Wiki page linked above is very close — it is missing many presentations and their corresponding slides.

O’Reilly has all of the slides speakers submitted at http://en.oreilly.com/mysql2008/public/schedule/presentations/. If someone or a few folks work on linking the slides on the O’Reilly site to the presentations on the Forge Wiki page at http://forge.mysql.com/wiki/MySQLConf2008Notes, then the Forge Wiki page will be comprehensive and folks can go to one page to get any and all information about a presentation at the conference.

Bookmark online using:These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • digg
  • Reddit
  • Spurl
  • Furl
  • blogmarks

MySQL Community Member of the Year Award Musings

Monday, April 28th, 2008

(If you want $100, you will have to read the entire blog post. Sorry for the tease, but I did not want folks to miss out on the opportunity to win!)

By now it is no surprise that I won one of the three 2008 MySQL Community Member of the Year awards. And folks may know that I won the same award last year.

One interesting fact you may not know: during the 2006 MySQL Awards Ceremony, where Giuseppe Maxia, Roland Bouman, Markus Popp and Rasmus Lerdorf won community awards, I thought to myself,

Next year I want to be on that stage, collecting that award.

Well, I did that and then some! (note that the image below is both of the awards, side by side, with no photoshopping).

But this post is not about me. This blog post is about you. More specifically, I am going to detail in this blog post the secrets to my success. This year, I thought to myself, “how can I make it so I am not on this stage receiving this award next year?”

So here is my challenge to you. I will open source my methods, and in return I will give $100 to each 2009 MySQL Community Member of the Year (in whatever form they want, whether it’s US cash, a $100 Amazon.com gift certificate, a donation to an organization, whatever).

April 2006 - March 2007


  • 154 blog posts

  • 3 User Group/conference presentations

  • Organized 12 User Group meetings

  • Produced 2 Videos

  • Produced 11 Podcasts (started Nov. 2006)

  • 1 Grant

  • Google Summer of Code mentor– full disclosure, the $500 mentor incentive went directly to MySQL and helped pay for the new MySQL Forge servers, so my only payment was a T-shirt.

April 2007 - March 2008

Besides the above, other ideas for community involvement are:

  • Forum/list involvement

  • Volunteering for the Documentation team

  • Helping to organize user-based conferences
  • (my brain wants to call them UDC’s — “user defined conferences”).

I have only listed non-technical ways to win the award, and only what I could think of. The sky is the limit!

Bookmark online using:These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • digg
  • Reddit
  • Spurl
  • Furl
  • blogmarks

Video: Who is the Dick on My Site Keynote

Monday, April 28th, 2008

I have already blogged about this keynote at http://www.pythian.com/blogs/948/liveblogging-who-is-the-dick-on-my-site.

If you are interested in actually seeing the video, the 286 Mb .wmv file can be downloaded at http://technocation.org/videos/original/mysqlconf2008/2008_04_17_panelDick.wmv and played through your browser by clicking the “play” link at http://tinyurl.com/55c5ps. This is not to be missed!

Bookmark online using:These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • digg
  • Reddit
  • Spurl
  • Furl
  • blogmarks

When SHOW SLAVE STATUS and the error log Disagree

Friday, April 25th, 2008

Or, When MySQL Lies!

When I do a show slave status\G, sometimes mysqld will lie to me and give me a wrong Exec_Master_Log_Pos. Let me explain with a situation from last night.

This is the output of show slave status\G from mysql version 5.0.41-community-log:

mysql> show slave status \G
*************************** 1. row ***************************
                Slave_IO_State: Waiting for master to send event
                   Master_Host: XXX.XXX.XXX.XXX
                   Master_User: replic_username
                   Master_Port: 3306
                 Connect_Retry: 60
               Master_Log_File: mysql-bin.000480
           Read_Master_Log_Pos: 690470773
                Relay_Log_File: db2-relay-bin.000028
                 Relay_Log_Pos: 683977007
         Relay_Master_Log_File: mysql-bin.000480
              Slave_IO_Running: Yes
             Slave_SQL_Running: No
               Replicate_Do_DB:
           Replicate_Ignore_DB:
            Replicate_Do_Table:
        Replicate_Ignore_Table:
       Replicate_Wild_Do_Table:
   Replicate_Wild_Ignore_Table:
                    Last_Errno: 0
                    Last_Error: Could not parse relay log event entry. The possible reasons are: the master’s binary log is corrupted (you can check this by running ‘mysqlbinlog’ on the binary log), the slave’s relay log is corrupted (you can check this by running ‘mysqlbinlog’ on the relay log), a network problem, or a bug in the master’s or slave’s MySQL code. If you want to check the master’s binary log or slave’s relay log, you will be able to know their names by issuing ‘SHOW SLAVE STATUS’ on this slave.
                  Skip_Counter: 0
           Exec_Master_Log_Pos: 126
               Relay_Log_Space: 690471192
               Until_Condition: None
                Until_Log_File:
                 Until_Log_Pos: 0
            Master_SSL_Allowed: No
            Master_SSL_CA_File:
            Master_SSL_CA_Path:
               Master_SSL_Cert:
             Master_SSL_Cipher:
                Master_SSL_Key:
         Seconds_Behind_Master: NULL

So in summary, the slave SQL thread is stuck (in this case because of a problem during the transfer of the binlog data to the slave’s relay log). The show slave status\G command tells me that it is stuck at the master binlog file mysql-bin.000480, position 126.

But, if I look at the error log file entries when the slave got stuck I see:

(more…)

Bookmark online using:These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • digg
  • Reddit
  • Spurl
  • Furl
  • blogmarks

Log Buffer #94: a Carnival of the Vanities for DBAs

Friday, April 25th, 2008

Log Buffer, the weekly review of database blogs, welcomes back for his record-breaking record-tying (Sheeri, are you reading?) third edition Ronald Bradford of Opinions, Expertise, Passion.

Why does Ronald write Log Buffer? Perhaps it’s because he knows that LB is and established and widely read feature, and hence likely to bring his own blog some new readers and improve its ranking. Or maybe he enjoys the fun and challenge of comprehending and presenting the entire DBA blog scene, not just the part that deals with his own favoured technologies. (Or maybe he just likes me? Ronald?)

Since Log Buffer is open to anyone, I encourage you also to join in. If you’d like to edit and publish an edition yourself, take a look at LB’s homepage, read the few guidelines, and then get in touch with me, the Log Buffer coordinator.

You can also contribute by emailing your favourite blog items to the editor.

And now, here’s Ronald Bradford’s Log Buffer #94.

Bookmark online using:These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • digg
  • Reddit
  • Spurl
  • Furl
  • blogmarks

A Challenge to MySQL Employees

Tuesday, April 22nd, 2008

Keith Murphy wrote about the open/closed source debacle and the first comment on that post was:

Monty makes all this money from the Sun acquisition, and pretends to be a free software advocate. How much did he make? How much is he giving back to the MySQL community?

Now, Keith rightfully met this with “grow up”. However, I want to point out that many people in the MySQL employee pool benefited from the sale, not just Monty. I also want to point out that Monty devoted years of his life to developing MySQL long before it was ever profitable.

According to Sun’s press release, “Sun will pay approximately $800 million in cash in exchange for all MySQL stock and assume approximately $200 million in options. The transaction is expected to close in late Q3 or early Q4 of Sun’s fiscal 2008….. The deal is expected to be accretive to FY10 operating income on a GAAP basis.”

Now, there’s financial mumbo-jumbo in there, but basically what that means is in all likelihood, Monty actually has not received any real money yet. And with 20% of the sale being in options (not stock, just options, which means that there is the option to buy stock, so there’s nothing free there), that’s even less cold hard cash floating around.

But I present a challenge to MySQL employees who have derived tangible benefits from the sale to Sun: what percentage have you put back into the MySQL community, and how?

(and thinking outside the box is OK — time is money, so I am OK with you directly translating the number of hours you’ve worked on community projects into $$ given your approximate hourly salary).

For instance, Brian Aker’s list of software is impressive, and of the 28 projects explicitly listed (see “Project list” on the right-hand side, and I’d bet there’s more in the actual repository) I’d guess fewer than 5 were done on time paid for by anyone (much less MySQL/Sun).

I know that Giuseppe Maxia’s mysql sandbox is a project he works on during non-MySQL/Sun time.

But I’d love to see comments on what folks are doing, even without percentages of money and such, because I am willing to wager that most of the folks who work for MySQL give plenty back to the community on non-company time. My theory is based on the fact that most MySQLers that I’ve met do not see working at MySQL as “their job”, they see it as “I get paid to do what I love doing, and would do anyway.”

Bookmark online using:These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • digg
  • Reddit
  • Spurl
  • Furl
  • blogmarks

EXPLAIN Cheatsheet

Tuesday, April 22nd, 2008

At the 2008 MySQL Conference and Expo, The Pythian Group gave away EXPLAIN cheatsheets. They were very nice, printed in full color and laminated to ensure you can spill your coffee* on it and it will survive.

For those not at the conference, or those that want to make more, the file is downloadable as a 136Kb PDF at explain-diagram.pdf

* or tea, for those of us in the civilized world.

Bookmark online using:These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • digg
  • Reddit
  • Spurl
  • Furl
  • blogmarks