This week started out with me posting about International Women’s Day, and has me personally attending Confoo (Montreal) which is an excellent conference I hope to return to next year. I learned a lot from confoo, especially the blending nosql and sql session I attended.

This week was also the Hotsos Symposium. Doug’s Oracle Blog has a series of posts about Hotsos. If all this talk about conferences has gotten you excited, Joshua Drake notes that 14 days and the hotel is almost full for postgresql conference east which is March 25th-28th in Philadelphia. And the Oracle database insider notes that the Oracle OpenWorld call for papers is now open.

According to Susan Visser this week (ending tomorrow) is also read an e-book week. So if you have not already done so, read an e-book! She links a coupon for an e-book in the post.

Craig Mullins notes that the mainframe is a good career choice in Mainframes: The Safe IT Career Choice. He notes that the mainframe is still not dead:

People having been predicting the death of the mainframe since the advent of client/server in the late 1980s. That is more than 20 years! Think of all the things that have died in that timespan while the mainframe keeps on chugging away: IBM’s PC business, Circuit City, Koogle peanut butter, public pay phones, Johnny Cash… the list is endless.

In other career-related news, Antonio Cangiano is looking for [2] top-notch student hackers for a 16-month internship at IBM in Toronto starting in May. All the details, including how to apply, are in Cangiano’s blog post.

Willie Favero wants to know how you “solve the batch dilemma” for issues like “shrinking your batch window, designing your batch to play nicely with … OLTP” in how’s your batch workload doing? Perhaps Favero should read the updated batch best practices posted by Anthony Shorten.

Bryan Smith surveys a more personal question by asking if you go both ways and “manage both DB2 for Linux, UNIX, and Windows and DB2 for z/OS” in don’t ask, don’t tell, bi-platform DBAs. This week’s Log Buffer editor admits to being a tri-platform DBA — she has tried many platforms, and in fact, many databases (MySQL, Oracle, DB2, SQL Server, Sybase, Postgres and Ingres)!

Hari Prasanna Srinivasan promotes a patching survey in Oracle really wants to hear from you! Patching Survey.

Henrik Loeser explains what a deadlock and a hot spot are by using a real life analogy taken from a police report in deadlock and hot spot in real life.

Jamie Thomson asks why do you abbreviate schema names?. Shlomi Noach tries to solve the issue that “there is no consistent convention as for how to write [about table aliases in] an SQL query” in proper sql table alias use conventions. Noach also gives us a tip: faster than truncate.

Leons Petrazickis reminds us that “rulesets are chains” and it is important to have your rulesets in the proper order in iptables firewall pitfall.

Anyone interested in the history of MySQL AB will be informed after reading Dries Buytaert’s article.
Gavin Towey shares his software that helps centrally manage 120 MySQL servers in qsh.pl: distributed query tool For those who want to learn more about column-oriented databases, particularly in MySQL, Robin Schumacher of the InfiniDB blog announces that there is a MySQL University session recording on MySQL column databases now available. MySQL join-fu expert Jay Pipes has moved his blog to www.joinfu.com and starts with An SQL Puzzle and of course a follow up on the sql puzzle.

Ivan Zoratti is happy that finally, slides posted for the MySQL DW breakfast. Venu Anuganti gives you tips on one of the most common MySQL frustrations: optimizing subqueries in how to improve subqueries derived tables performance. Justin Swanhart posts the way in which he Gets Linux performance information from your MySQL database without shell access and emulates a ‘top’ CPU summary using /proc/stat and MySQL using the same method.

The Oracle Apps blog has an introduction to Oracle user productivity kit (UPK). Even though in this editor’s opinion the article is very sales-pitchy, it has valuable information, and does indeed live up to its promise:

UPK is a software tool that can capture all the steps in a system process. It records every keystroke, every click of the mouse, each menu option chosen and each button pressed. All this is done in the UPK Recorder by going through the transaction and pressing “printscreen” after every user action. From this, without any further effort from the developer, UPK builds a number of valuable outputs.

Allen White gives a great tip on how to optimize queries in keep your data clean.

Mike Dietrich reminds you to remove “old” parameters and events from your init.ora when upgrading, “as keeping them will definitely slow down the database performance in the new release.” He shows evidence of slowness when this is not done. Dietrich also shows how you can be gathering workload statistics “to give the optimizer some good knowledge about how powerful your IO-system might be”, especially “a few days after upgrading to the new release…while a real workload is running.”

Brian Aker shows the exciting features coming soon in Drizzle in Drizzle, Cherry, Roadmap for our Next Release.

Maybe you are thinking of migrating, not upgrading…..The O’Reilly Radar shows how to asses an Oracle to MySQL migration in MySQL migration and risk management. Actually, that article interviews Ronald Bradford on the subject — Bradford has been prolific lately, updating free my.cnf advice series and “Don’t Assume”: MySQL for the Oracle DBA series. Nick Quarmby also talks about migrating Oracle, but not to a new database, just to a new platform, in his primer on migrating Oracle Applications to new platforms. And the big news comes from Carlos of dataprix that Twitter will migrate from MySQL to Cassandra DB.

Paul S. Randal explains his way of benchmarking: 1 Tb table population on SQL Server.

Pete Finnigan shares his slides from a webinar on how to secure oracle, and Denis Pilipchuk shares his approaches for discovering security vulnerabilities in software applications.

Jeff Davis shares his thoughts about scalability and the relational model. Robert Treat responds actually, the relational model doesn’t scale and Baron Schwartz counters with NoSQL doesn’t mean non-relational.

Buck Woody explains “whenever you want to know something about SQL Server’s configuration, whether that’s the Instance itself or a database, you have a few options” — and of course what those options are — in system variables, stored procedures or functions for meta data.

This week’s T-SQL Tuesday topic was I/O. There are many links to great blog posts in the comments; three random posts I chose to highlight: Michael Zilberstein talks about IO capacity planning, while Kalen Delaney talks about using STATISTICS IO in I/O, you know, and Merrill Aldrich chimes in with information on real world SSD’s. Aldrich also begs folks not to waste resources and make more work for developers and DBAs in dear ISV, you’re keeping me awake nights with your VARCHAR() dates.

And we end with a bit of fin: Paul Nielsen wants us all to have a bit of fun; he has posted an SQL limerick and asks readers to create there own in there once was in Dublin a query.

Michael Bleigh from Intridea, high-end Ruby and Ruby on Rails consultants, build apps from start to finish, making it scalable. He’s written a lot of stuff, available at http://github.com/intridea. @mbleigh on twitter

NoSQL is a new way to think about persistence. Most NoSQL systems are not ACID compliant (Atomicity, Consistency, Isolation, Durability).

Generally, most NoSQL systems have:

• Denormalization
  
• Eventual Consistency
  
• Schema-Free
  
• Horizontal Scale
  

NoSQL tries to scale (more) simply, it is starting to go mainstream – NY Times, BBC, SourceForge, Digg, Sony, ShopWiki, Meebo, and more. But it’s not *entirely* mainstream, it’s still hard to sell due to compliance and other reasons.

NoSQL has gotten very popular, lots of blog posts about them, but they reach this hype peak and obviously it can’t do everything.

“NoSQL is a (growing) collection of tools, not a new way of life.”

What is NoSQL? Can be several things:

• Key-Value Stores
  
• Document Databases
  
• Column-oriented data stores
  
• Graph Databases
  

Key-Value Stores

Speaker’s favorite is Redis because it’s similar to memcached.

• key-value store + datatypes (list, sets, scored sets, soon hashes will be there)
  
• cache-like functions (like expiration)
  
• (Mostly) in-memory
  

Another interesting key-value store is Riak

• Combination of key-value store and document database
  
• heavy into HTTP REST
  
• You can create links between documents, and do “link walking” that you don’t normally get out of a key-value store
  
• built-in Map Reduce
  

Map Reduce:

• Massively parallel way to process large datasets
  
• First you scour data and “map” a new set of dataM
  
• Then you “reduce” the data down to a salient result — for example, map reduce function to make a tag cloud: map function makes an array with a tag name and a count of 1 for each instance of that tag, and the reduce tag goes through that array and counts them…
  
• http://en.wikipedia.org/wiki/MapReduce
  

Other key-value stores:

• Tokyo Cabinet
  
• Dynomite
  
• memcachedDB
  
• Voldemort
  

Document Databases

• MongoDB – Document store that speaks BSON (Binary JSON, which is compact). This is the speaker’s favorite because it has a rich query syntax that makes it close to SQL. Can’t do joins, but can embed objects in other objects, so it’s a tradeoff


• Also has GridFS that can store large files efficiently, can scale to petabytes of data
  
• does have MapReduce but it’s deliberate and you run it every so often.
  
• CouchDB
  
  • Pure JSON Document Store – can query directly with nearly pure javascript (there are auth issues) but it’s an interesting paradigm to be able to run your app almost entirely through javascript.
    
  • HTTP REST interface
    
  • MapReduce only to see items in CouchDB. Incremental MapReduce, every time you add or modify a document, it dynamically changes the functions you’ve written. You can do really powerful queries as easy as you can do simple queries. However, some things are really complex, ie, pagination is almost impossible to do.
    
  • Intelligent Replication – CouchDB is designed to work with offline integration. Could be used instead of SQLite as the HTML5 data store, but you need CouchDB running locally to be doing offline stuff w/CouchDB
    
  
  

Column-oriented store

• Cassandra – Built by Facebook, also used by Twitter
  
• BigTable
  
• Hypertable
  
• HBase
  

Graph Databases

When should I use this stuff?

Don’t look for a single tool that does every job. Use more than one if it’s appropriate, weigh the tradeoffs (ie, don’t have 7 different data stores either!)

NoSQL solves real scalability and data design issues. But financial transactions HAVE to be atomic, so don’t use NoSQL for those.

A good presentation is http://www.slideshare.net/bscofield/the-state-of-nosql.

Using SQL and NoSQL together

You can blend by hand, but the easy way is DataMapper:
Generic, relational ORM (adapters for many SQL dbs and many NoSQL stores)
Implements Identity Map
Module-based inclusion (instead of extending from a class, you just include into a class).

You can set up multiple data targets (default is MySQL, example sets up MongoDB too).
DataMapper is:

• Ultimate Polyglot ORM
  
• simple r’ships btween persistence engines are easy
  
• jack of all, master none
  
• Sometimes perpetuates false assumptions –
  
• If you’re in Ruby, your legacy stuff is in ActiveRecord, so you’re going to have to rewrite your code anyway.
  

Speaker’s idea to be less generic and better use of features of each data store – Gloo – “Gloo glues together different ORMs by providing relationship proxies.” this software is ALPHA ALPHA ALPHA.

The goal is to be able to define relationships on the terms of any ORM from any class, ORM or not
Right now – partially working activeRecord relationships
Is he doing it wrong? Is it a crazy/stupid idea? Maybe.

Example:

Need	Use
Assume you already have an auth system	it’s already in SQL, so leave it there.
Need users to be able to purchase items from the storefront – Can’t lose transactions, need full ACID compliance	use MySQL.
Social Graph – want to have activity streams and 1-way and 2-way relationships. Need speed, but not consistency	use Redis
Product Listings — selling moves and books, both have different properties, products are pretty much non-relational	use MongoDB

He wrote the example in about 3 hours, so integration of multiple data stores can be done quickly and work.

The session on joind.in, with user comments/feedback, is at http://joind.in/talk/view/1320.

Slides are at http://talks.php.net/show/confoo10

“My name is Rasmus, I’ve been around for a long time. I’ve been doing this web stuff since 1992/1993.”

“Generally performance is not a PHP problem.” Webservers not config’d, no expire headers on images, no favicon.

Tools: Firefox/Firebug extension called YSlow (developed by yahoo) gives you a grade on your site.

Google has developed the Firefox/Firebug pagespeed tool.

Today Rasmus will pick on wordpress. He checks out the code, then uses Siege to do a baseline benchmark — see the slide for the results.

Before you do anything else install an opcode cache like APC. Wordpress really likes this type of caching, see this slide for the results. Set the timezone, to make sure conversions aren’t being done all the time.

Make sure you are cpu-bound, NOT I/O bound. Otherwise, speed up the I/O.

Then strace your webserver processs. There are common config issues that you can spot in your strace code. grep for ENOENT which shows you “No such file or directory” errors.

AllowOverride None to turn off .htaccess for every directory, just read settings once from your config file….(unless you’re an ISP).

Make sure DirectoryIndex is set appropriately, watch your include_path. All this low-hanging fruit has examples on the common config issues slide.

Install pecl/inclued and generate a graph – here is the graph image (I have linked it because you really want to zoom in to the graph…)

In strace output check the open() calls. Conditional includes, function calls that include files, etc. need runtime context before knowing what to open. In the example, every request checks to see if we have the config file, once we have config’d we can get rid of that stuff. Get rid of all the conditionals and hard-code “include wp-config.php”. Examples are on the slide.

His tips to change:
Conditional config include in wp-load.php (as just mentioned)
Conditional did-header check in wp-blog-header.php
Don’t call require_wp_db() from wp-settings.php
Remove conditional require logic from wp_start_object_cache

Then check strace again, now all Rasmus sees is theming and translations, which he decided to keep, because that’s the good benefit of Wordpress – Performance is all about costs vs. flexibility. You don’t want to get rid of all of your flexibility, but you want to be fast.

Set error_reporting(-1) in wp-settings.php to catch all warnings — warnings slow you down, so get rid of all errors. PHP error handling is very slow, so getting rid of errors will make you faster.

The slide of warnings that wordpress throws.

Look at all C-level calls made, using callgrind, which sits under valgrind, a CPU emulator used for debugging. See the image of what callgrind shows.

Now dive into the PHP executor, by installing XDebug.

Check xhprof – Facebook open sourced this about a year ago, it’s a PECL extension. The output is pretty cool, try it on your own site, Rasmus does show you how to use it. It shows you functions sorted by the most expensive to the least expensive.

For example, use $_SERVER[REQUEST_TIME] instead of time(). Use pconnect() if MySQL can handle the amount of webserver connections that will be persistent, etc.

After you have changed a lot of the stuff above, benchmark again with siege to see how much faster you are. In this case there is not much gained so far.

So keep going….the blogroll is very slow — Rasmus gets rid of it by commenting out in the sidebar.php file. I’d like to see something to make it “semi-dynamic” — that is, make it a static file that can be re-generated, since you might want the blogroll but links are not changed every second…..

At this point we’re out of low-hanging fruit.

HipHop is a PHP to C++ converter & compiler, including a threaded, event-driven server that replaces apache. Rasmus’ slide says “Wordpress is well-suited for HipHop because it doesn’t have a lot of dynamic runtime code. This is using the standard Wordpress-svn checkout with a few tweaks.”

Then, of course, benchmark again.

The first time you compile Wordpress with HipHop, you give it a list of files to add to the binary, it will complain about php code that generate file names, so you do have to fix that kind of stuff. There’s a huge mess of errors the first time you run it (”pages and pages”), and Rasmus had to patch HipHop (and Wordpress) but the changes in HipHop have been put back into HipHop, so you should be good for the most part.

Check out the errors, lots of them show logical errors like $foo.”bar” instead of $foo.=”bar” and $foo=”bar” instead of $foo==”bar” in an if statement. Which of course is nice for your own code, to find those logical errors.

(Wordpress takes in a $user_ID argument and immediately initializes a global $user_ID variable, which overwrites the argument passed in, so you can change the name of the argument passed in….)

You can also get rid of some code, things that check for existence of the same thing more than once. So it will take a bit of tweaking, but it’s worth it.

There are limitations to HipHop, for example:

• It doesn’t support any of the new PHP 5.3 language features
  
• Private properties don’t really exist under HipHop. They are treated as if they are protected instead.
  
• You can’t unset variables. unset will clear the variable, but it will still be in the symbol table.
  
• eval and create_function are limited
  
• Variable variables $$var are not supported
  
• Dynamic defines won’t work: define($name,$value)
  
• get_loaded_extensions(), get_extension_funcs(), phpinfo(), debug_backtrace() don’t work
  
• Conditional and dynamically created include filenames don’t work as you might expect
  
• Default unix-domain socket filename isn’t set for MySQL so connecting to localhost doesn’t work

and HipHop does not support all extensions — see the list Rasmus has of extensions HipHop supports.

Then Rasmus showed an example using Twit (which he wrote) including the benchmarks. He shows that you can see what’s going on, like 5 MySQL calls on the home page and what happens when you don’t have a favicon.ico (in yellow).

In summary, “performance is all about architecture”, “know your costs”.

Be careful, because some tools (like valgrind and xdebug) you don’t want to put it on production systems, you could capture production traffic and replay it on a dev/testing box, but “you just have to minimize the differences and do your best”.

Why is social media and social networking essential to you and your business? (because it will drive sales, but there’s very few analytics for ROI on social networking and social media)

Relying on advertising is no longer working for print newspapers and television. So why do we think it will work on internet media?

Blogging — you must post 2-4 quality blog posts every week to maintain readership. This takes a lot of work! Content is king.

No matter how cool the technology/product/service is, people still buy more often and more easily from people they know and trust.

Social media is a way to show people that you are an industry expert, and that is how you should use them (not to spam and only say “buy my product”).

If you do not love your job and try to sell it (say, on social networking), you are going to fail, because you are not passionate about it.

Start small, do not promise a lot, it is better to have more to say than to have dead air time (radio analogy).

Social media is all about building good relationships by having good content that people trust.

Lots of people spend a lot of money on their website, but the website is just a vector to show people your content, and the content is the most important thing.

Cross-pollination – I think he means forward on information you learn (like, say, liveblogging!)

Get expert guest bloggers — he did not explain that you can leverage the relationships you form by asking them to blog. We do this with the Log Buffers….

How to make money:
sponsorships
white paper composition
paid articles
consulting
adjunct tie-ins to other related venues
branded blogs

I am personally disappointed because I wanted to learn more, and I feel as though Pythian already uses the knowledge presented — we have great exposure through our blog, and have started really using Twitter, Facebook and other social media sites, etc.

Hansen’s information was good, and absolutely 100% correct, but I felt that for me it was very basic. I would like to know some more advanced topics, like:
- How do you know when you have reached the tipping point?
- How do you convert anonymous readers/followers to people you know, without turning them away because they feel they’re being watched, spammed or don’t want to give out their info to you?
- When does copy/paste to send out your information start to bother people, how do you know how not to do too much?
- How do you convert readers/followers (anon or not) to paid customers without making them feel like you’re all about $$, what about if you have some free content and some paid content, how do you know how much to have?

And now, on to Mark Pilgrim of Google speaking on HTML5.

Timeline
1991 – HTML 1
1994 – HTML 2
1995 – Netscape discovers web, ruins it
1996 – CSS1 + JavaScript
1996 – Microsoft discovers web, ruins it
1997 – HTML4 + EMCAScript1
1998 – CSS2 + EMCAScript2 + DOM1
2000 – XHTML1 + EMCAScript3 + DOM2
2001 – XHTML 1.1
[long break!]
2009 – HTML 5 + ECMA5 + CSS 2.1

HTML5 is not a spec, it’s a marketing term. It’s really HTML5 + CSS3 + JavaScript.

IsHTML5ReadyYet.com and IsHTML5Ready.com are both real websites that give different answers to the question “is HTML 5 ready?”

Semantics
HTML started as a semantic language (until Netscape came along).

New elements (html tags) that do not do anything – they are for semantic use only:

<header> <footer>
<section>
<article>
<nav>
<aside> (pull quotes and such)
<time> (datetime markup)
<mark> (marking up runs of text)
<figure> <figcaption>

Instead of “div class=_____” use these tags….for example:

<body>
  <header>
    <hgroup>
    <h2>page title</h2>
    <h3>page subtitle</h3>
    </hgroup>
   </header>

<nav>
  <ul> Navigation......
     .....
  </ul>
</nav>

<section>
  <article>
   <header>
    <h2>Title</h2>
   </header>
</section>

Caveat: This doesn’t work in IE but there is a workaround…..

This can help blind people navigate better….and bots too!

“Google is just another blind web user with 7 million friends”

Forms
Web forms 2.0
To make a slider from 0-50:

<input type='range' mix='0' max='50' value='0'></input>

To use autofocus:

<input autofocus>

(works in 3 browsers)

Talking about blind users again: “Focus tracking is VERY important if you can’t see. You really need to know where on the page you are, if you start typing what will happen.”

Placeholder text — in a text box, that light text that goes away when you click:

<input type='text' placeholder='click here and this will disappear'>

(works in 2 browsers)

New input types
These are semantic types, do different things in different browsers

<input type='email'> (on the iphone you get a different keyboard, by default you just get a textfield, so these things degrade gracefully if the browser does not support the feature)
<input type='url'> (a browser like <A HREF="http://www.opera.com">Opera</A> can validate a URL for you instead of you doing it yourself!)
<input type='datetime'> (and more...date pickers are tedious)
<input type='file' multiple> (multiple files without using flash!)

For all the inputs HTML5 supports and which browsers support them (Opera is leading the way) search for “HTML5 input support”

Accessibility
ARIA = “accessible rich internet applications”. Alt-text is technology that’s long behind. ARIA does stuff like making tree views accessible. For example, right now with a tree view you have to tab through each item, which is a pain. With code like this:

<ul id='tree1' role='tree' tabindex='0' aria-labelledby='label_1'>
  <li role='treeitem' tabindex='-1' aria-expanded='true'>Fruits </li>
    <li role='group'>
    <ul>
      <li role='treeitem' tabindex='-1'>Oranges</li>
      <li role='treeitem' tabindex='-1'>Pineapples</li>
    </ul>
   </li>
</ul>

….keyboard users can tab to the treeview itself, then use arrow keys to navigate and spacebar to select. This makes selecting an item at the end of a tree view much easier, and also makes it easy to move beyond the tree view without having to press Tab a million times.

Use your favorite search engine for “ARIA accessibility” to learn more.

CSS
Mark threw this image up on the screen:

(image from http://www.zazzle.com/stevenfrank – on that site you can buy this coffee mug or a T-shirt with the design)

Web fonts finally work in CSS3 – you can use more than Times, Courier, Arial, and occasionally Helvetica. This works EVERYWHERE – Chrome, IE, Firefox, Opera, Safari, etc. Well, it’s true that they all use it, but they all have different fonts they support. Read Bulletproof font face for tips on how to get the font you want no matter what browser is used (yes, even IE).

Opacity is easy [author's note - it's just the "opacity" element, see examples at http://www.css3.info/preview/opacity/].

Rounded corners are EASY – Mark’s slide passed too fast for me, so I grabbed an example from http://24ways.org/2006/rounded-corner-boxes-the-css3-way:

.box {
  border-radius: 1.6em;
}

Gradients are easy [author's note -- looks like you need webkit, there's examples at http://gradients.glrzad.com/]

To test CSS3 stuff, use www.css3please.com – “This element will receive inline changes as you edit the CSS rules on the left.”

[Author's note -- while searching I found http://www.webappers.com/2009/08/10/70-must-have-css3-and-html5-tutorials-and-resources/ which is definitely a "must have".]

Canvas
A canvas is a blank slate where you can draw whatever you want, use the canvas tag and id, width and height attributes, everything else is javascript. Pretty awesome. [Author's note -- Mark had examples but I did not have time to capture them. I did find a nice tutorial at https://developer.mozilla.org/en/Canvas_tutorial.]

Multimedia
Video with no flash! YouTube has HTML5 integration. Here’s sample code of how to do movies in HTML5:

<video src='movie.ogv' controls></video>
<video src='movie.ogv' loop></video>
<video src='movie.ogv' preload='none'></video>  -- don't preload the movie
<video src='movie.ogv' preload='auto'></video>
<video src='movie.ogv' autoplay></video> -- if you don't have this you don't do evil autoplay....

Multimedia is in the DOM and responds to CSS effects, such as reflection:

<video src='movie.ogv' loop style='webkit-box-reflect: below 1px;'></video>

(this code might be wrong, the slide flipped fast)

Of course the problem — codecs. Right now, .ogv and .mp4 (h264).

Audio inline too, same problem — only .oga and .mp3:

<audio src ='podcast.oga' controls></audio>

Geolocation
IsGeolocationPartofHTML5.com is a real site, go to it to get the answer.
Geolocation demos — very much the same, find your location and display it. Simple but cool.

Cache manifest
Get everything you need for offline usage…

<html manifest='another-sky.manifest'>

CACHE MANIFEST
/avatars/zoe.png
/avatars/tamara.png
/scripts/holoband.jpg

search for “google for mobile HTML5 series” – good series of articles on using this stuff.

HTML 5 has much more
Local storage
Web workers
Web sockets (2way connections, like raw tcp/ip cxns over the web)
3D canvas (webgl)
Microdata (enhanced semantics)
Desktop notifications
Drag and Drop

Learn more:
whatwg.org/html5
diveintohtml5.org

I arrived at Omni Mandalay Hotel on Sunday evening with Dan Norris. I was flying through Chicago and it turned out that Dan was on the same flight and only few rows behind me. Small world.

Preparations for the conference were very chaotic on my part and, of course, I didn’t have either of my presentations ready. I was very stressed and getting sick as well — it looked like a complete disaster waiting to happen. I’d like to say that I was feeling like Doug Burns as he often managed to get sick just before a conference. Of course, I worked on my slides for the last few days as well as on the flight and presentation was slowly getting there but boy was I tired!

I quickly said hello to the crowd in the bar on the way to my room and rushed away to do some more damage to my slides. And then I had a brilliant idea — I could still see one of my best mates and do something good about my presentation! I asked Doug if he was interested in the preview (he probably wasn’t interested but he couldn’t say it to me) especially that my session wasn’t on his original agenda. Of course, that would mean that he had to leave a bunch of other good friends and spend some time tete-a-tete. Knowing Doug, this is some of the hardest thing to ask from him but it shows how good of a friend he is! (Plus, everyone thinks that he is anti-social anyway. Shhhh!)

Doug has made my day — while he provided lots of ideas and feedback on few things that I was lucking, he generally approved the idea and confirmed that it wasn’t totally crazy. I guess that was all I needed back then and Doug knew how nervous I was about it. (Thanks mate!)

So I called Sunday a day very early and went to bed before midnight. I really needed some sleep. Woken up by the alarm at 5AM (I woke up few times during the night looking at the clock — making sure I didn’t sleep through) and slides were ready just before lunch. I even managed to do a test run and it took 65 minutes — a wee bit too long for one hour session. But it was good test and I knew I had to be just a bit more concise in few parts.

Mi morning was very productive. Unfortunately, I missed the opening keynote from Tom Kyte. Such a pity! If what Doug wrote is true, Tom was talking about the mistakes we make *because* of our experience and our assumptions. This was exactly one of the points I was making in my Battle Against Any Guess — experience is danger. I wish I could see Tom’s example. Oh well, maybe another time.

I managed to attend half of the Richard Foote’s session on indexes but my mind was far away — with my own slides. Though, I did manage to focus on bitmap indexes part and the myth of bitmap indexes not working well for columns with high cardinality. Very interesting conclusions. I’m still wondering how much overhead updates will do to such bitmap index.

After lunch, it was my turn. I ordered few copies of the latest OakTable book — Expert Oracle Practices: Oracle Database Administration from the Oak Table — that I co-authored with the bunch of other Oakies. I contributed chapter 1 in the book titled just like my presentation — Battle Against Any Guess. The plan was to give a copy away during the presentation and do a draw for another one at the end of the session. I was so nervous that I forgot about it until the end of the session so I just did a draw for two copies. The lucky winners were Lynn-Georgia Tesch and Surendra Anchula. Congratulations! For the rest of you who left the contact details — please stay tuned and we’ll organize few things online.

Now the main topic of this post — my presentation. What’s unusual about this session is that it’s not some technical stuff that I usually do but a more conceptual and motivational talk. Could I pull it off? Well, I think it went fairly well in general even though I did identify few rough places and my lack of English language mastering. Might need to work a little bit more on the flow of the presentation.

We had quite a few good laughs. Later, people in the next hall were asking about it and Dan was making the jokes on the stage so it must have been loud. Anyway, I think nobody fell asleep and I managed to get people thinking about the topic. I received many “thank you” notes yesterday and compliments on a good session so by the end of the day I was more and more pleased. Thanks everyone for attending and especially big thanks to those of you who brought to my attention examples from their own battles. If you have more to discuss — contact me by email (my last name) {at} pythian.com.

Thanks to Marco Gralike for recording some fragments and sharing them. I think he has more to come.

This is the introductory couple minutes. You can definitely notice how nervous I am starting on the stage:

Solving the wrong problem example:

That’s all for now. Stay tuned — more to come.

Start planning your blog posts for Ada Lovelace day now (March 24th, http://findingada.com/ Ada Lovelace Day is an international day of blogging (videologging, podcasting, comic drawing etc.!) to draw attention to the achievements of women in technology and science.)

To that end, I would like to point out all the women currently in science and tech fields that I admire and think are doing great things. I think it would be great if everyone, male or female, made a list like this:

The women that have taught me science/tech along the way:

High School:
Mary Lou Ciavarra (Physics)
Maria Petretti (Pre-Algebra, and Academic Decathlon)
Reneé Fishman (Biology)
Lisa Acquaire (Economics during Academic Decathlon)

College:
Professor Kalpana White (Biology), and in whose fruit fly lab I worked for 2 semesters.
Professor Eve Marder (Introductory Neuroscience)

Though Brandeis does have female faculty in the Computer Science department, I did not manage to have any classes with female Computer Science faculty members.

My current female DBA co-workers at Pythian: Isabel Pinarci (Oracle), Michelle Gutzait (SQL Server), Catherine Chow (Oracle) and Jasmine Wen (Oracle).

And to folks in the greater MySQL/tech community and tech co-workers past and present, especially those I have been inspired and helped by: Tracy Gangwer, Selena Deckelmann (Postgres), Amy Rich, Anne Cross, and more (If I have forgotten you, I apologize!).

Oracle 11g Release 2 Grid Infrastructure has been dramatically redesigned compare to 10g and 11gR1 Clusterware. Coming with impressive set of new features, Grid Infrastructure also uses much more memory. While RAM is rather inexpensive these days, it does pose an inconvenience in some scenarios. Particularly, for sand-box type installations that I use all the time for my own tests and demonstrations. For production upgrades, you need to be aware of and plan for increased memory usage.

I’ve been able to easily run a 2 node 10g RAC cluster on my MacBook with 4 GB of RAM allocating less than 1 GB of RAM to each virtual machine. That was even enough for a mini database instance with a very small memory footprint. Oracle 11g Release 1 was pretty much the same except maybe the database instance itself required a bit more memory but one node could still fit within 1 GB of RAM.

In 11gR2, bare-bone Grid Infrastructure processes alone consume 10+ times more memory (11.2.0.1 on 32 bit Linux to be precise):

[gorby@cheese1 ~]$ ps -eo pid,%mem,rss,user,cmd --sort=rsz --cols 100 | grep -e '^ *PID' -e grid -e ohasd | grep -v grep
  PID %MEM   RSS USER     CMD
 3614  0.0  1080 root     /bin/sh /etc/init.d/init.ohasd run
 4322  0.2  3368 oracle   /nfs/11.2.0/grid/opmn/bin/ons -d
 4323  0.4  5164 oracle   /nfs/11.2.0/grid/opmn/bin/ons -d
 4117  0.6  7860 root     /nfs/11.2.0/grid/bin/oclskd.bin
 3830  0.6  8788 oracle   /nfs/11.2.0/grid/bin/gipcd.bin
 5048  0.7  8992 oracle   /nfs/11.2.0/grid/bin/tnslsnr LISTENER -inherit
 4167  0.7 10052 oracle   /nfs/11.2.0/grid/bin/evmlogger.bin -o /nfs/11.2.0/grid/evm/log/evmlogger.i
 3969  0.9 12412 oracle   /nfs/11.2.0/grid/bin/diskmon.bin -d -f
 3860  0.9 12736 oracle   /nfs/11.2.0/grid/bin/mdnsd.bin
 4067  1.1 14648 root     /nfs/11.2.0/grid/bin/octssd.bin reboot
 5016  1.2 15860 root     /nfs/11.2.0/grid/bin/orarootagent.bin
 3956  1.3 16964 root     /nfs/11.2.0/grid/bin/orarootagent.bin
 4292  1.4 17984 oracle   /nfs/11.2.0/grid/bin/oraagent.bin
 3874  1.5 20112 oracle   /nfs/11.2.0/grid/bin/gpnpd.bin
 3817  1.5 20300 oracle   /nfs/11.2.0/grid/bin/oraagent.bin
 4083  1.8 23700 oracle   /nfs/11.2.0/grid/bin/evmd.bin
 4372  2.4 31548 oracle   /nfs/11.2.0/grid/jdk/jre//bin/java -Doracle.supercluster.cluster.server=eo
 3564  3.2 41532 root     /nfs/11.2.0/grid/bin/ohasd.bin reboot
 4081  3.5 44932 root     /nfs/11.2.0/grid/bin/crsd.bin reboot
 3906 18.6 239428 root    /nfs/11.2.0/grid/bin/cssdagent
 3887 18.6 239444 root    /nfs/11.2.0/grid/bin/cssdmonitor
 3924 20.1 258564 oracle  /nfs/11.2.0/grid/bin/ocssd.bin

The second column above gives you amount of resident memory in KB for processes related to Grid Infrastructure. As you can cleanly see, processes of CSS components consume well above 700MB! In total we can account for 1 GB. (those calculations are flawed — see below)

Compare that with 10g (10.2.0.3 on 32 bit Linux) — bare-bone Clusterware processes consume only 60MB:

[oracle@lh1 ~]$ ps -eo pid,%mem,rss,user,cmd --sort=rsz --cols 100 | grep -e '^ *PID' -e nfs -e crs -e css -e evm | grep -v grep
  PID %MEM  RSS USER     CMD
 6524  0.0  348 oracle   /nfs1/oracle/oracle/product/10.2.0/crs/opmn/bin/ons -d
 4892  0.1  992 oracle   /bin/sh -c cd /nfs1/oracle/oracle/product/10.2.0/crs/log/lh1/cssd/oclsomon;
 3262  0.1 1072 root     /bin/sh /etc/init.d/init.evmd run
 3506  0.1 1100 root     /bin/sh /etc/init.d/init.crsd run
 4575  0.1 1116 root     /bin/su -l oracle -c sh -c 'ulimit -c unlimited; cd /nfs1/oracle/oracle/pro
 4890  0.1 1120 root     /bin/su -l oracle -c /bin/sh -c 'cd /nfs1/oracle/oracle/product/10.2.0/crs/
 4664  0.1 1180 root     /bin/sh /etc/init.d/init.cssd oclsomon
 3263  0.1 1188 root     /bin/sh /etc/init.d/init.cssd fatal
 4677  0.1 1188 root     /bin/sh /etc/init.d/init.cssd daemon
 6525  0.5 4792 oracle   /nfs1/oracle/oracle/product/10.2.0/crs/opmn/bin/ons -d
 4922  0.6 5224 oracle   /nfs1/oracle/oracle/product/10.2.0/crs/bin/oclsomon.bin
 5915  0.7 6280 oracle   /nfs1/oracle/oracle/product/10.2.0/crs/bin/evmlogger.bin -o /nfs1/oracle/or
 4576  1.1 9312 oracle   /nfs1/oracle/oracle/product/10.2.0/crs/bin/evmd.bin
 5018  1.1 9428 oracle   /nfs1/oracle/oracle/product/10.2.0/crs/bin/ocssd.bin
 4606  2.0 16712 root    /nfs1/oracle/oracle/product/10.2.0/crs/bin/crsd.bin reboot

The memory usage above is a bit overstated. There are some shared memory accounted multiple times. I could use Smaps interface to get better per process statistics. For example, you could see that 3 of the “top offenders” (CSS binaries) have about 40MB of shared libraries each:

[root@cheese1 ~]# ./smaps.pl 3924 | head
VMSIZE:     258576 kb
RSS:       258564 kb total
            39164 kb shared
             5180 kb private clean
           214220 kb private dirty
PRIVATE MAPPINGS
     vmsize   rss clean   rss dirty  file
   15052 kb        0 kb    15052 kb
   12016 kb        0 kb    12016 kb
   11184 kb        0 kb    11184 kb
[root@cheese1 ~]# ./smaps.pl 3887 | head
VMSIZE:     239456 kb
RSS:       239444 kb total
            40096 kb shared
             6200 kb private clean
           193148 kb private dirty
PRIVATE MAPPINGS
     vmsize   rss clean   rss dirty  file
   14624 kb        0 kb    14624 kb
   10240 kb        0 kb    10240 kb
   10240 kb        0 kb    10240 kb
[root@cheese1 ~]# ./smaps.pl 3906 | head
VMSIZE:     239440 kb
RSS:       239428 kb total
            40096 kb shared
             6200 kb private clean
           193132 kb private dirty
PRIVATE MAPPINGS
     vmsize   rss clean   rss dirty  file
   14624 kb        0 kb    14624 kb
   10240 kb        0 kb    10240 kb
   10240 kb        0 kb    10240 kb
[root@cheese1 ~]#

One way to get a practical number is to check system memory usage with and without Grid Infrastructure running — the difference is about 750MB (see the “free” column of the second row).

[root@cheese1 ~]# free
             total       used       free     shared    buffers     cached
Mem:       1283040    1131584     151456          0      18504     295668
-/+ buffers/cache:     817412     465628
Swap:       655328         76     655252
[root@cheese1 ~]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'cheese1'
...
...
CRS-4133: Oracle High Availability Services has been stopped.
[root@cheese1 ~]# free
             total       used       free     shared    buffers     cached
Mem:       1283040     397144     885896          0      18640     316632
-/+ buffers/cache:      61872    1221168
Swap:       655328         76     655252
[root@cheese1 ~]# ps -eo pid,%mem,rss,user,cmd --sort=rsz --cols 100 | grep -e '^ *PID' -e grid -e ohasd | grep -v grep
  PID %MEM   RSS USER     CMD
 3614  0.0  1084 root     /bin/sh /etc/init.d/init.ohasd run

I don’t have 11gR1 test cluster handy so I can’t check 100% but Oracle 11g Release 1 Clusterware is not much different from 10g so memory usage must be similar.

The lesson is that if you are upgrading your Oracle RAC Cluster to 11gR2 from 10g or 11gR1, then you have to account for additional 700MB memory for Grid Infrastructure alone on each node. Note that, this doesn’t take into account higher memory usage of the database instances themselves.

Operating Systems

Russia Today-TV announced the existence of “Red Star”, the new OS developed in North Korea and based on Linux. I found this by way of Slashdot of course, citing the source as The Korea herald. According to the article it looks very much like the Windows UI, and features a “My Country” icon that allows connection to Korea’s closed internet-like network and the Woori office application. Slightly more information can be found there in the article N. Korea develops own OS.

The upcoming Ubuntu 10.04 (Lucid Lynx) has had the third alpha version released. The alpha versions are not for the faint of heart, and I have had to put in several bug reports myself already. It’s just part of the process though and will hopefully result in a rock solid release next month. The full release announcement can be found on the mailing list: see Lucid Alpha 3 released. A rundown of some of the things you can expect is on The H Online in Canonical releases Ubuntu 10.04 LTS Alpha 3.

Ars Technica has more on an item mentioned last week about the future of Open Solaris. OpenSolaris not dead; might not get all new Solaris features The exact details of which features may be excluded are not clear from the article but I did not get a chance to dig into some of the source material referenced so you may find more by clicking through them.

Internet

Opera 10.5 is out and Ars Technica has a detailed review of the new features, performance improvements and UI changes. There’s a lot of detail in the article, I suggest you read it for yourself. Hands on: Opera 10.50 makes impressive performance gains is the place to go. I think I’ll give the new Opera a try myself.

Local company makes good! I mentioned last week the takedown of the Waledac botnet, this week the Mariposa botnet suffered a similar fate. Ottawa’s own Defence Intelligence detected this botnet in 2009 and started an investigation that ended with the dismantling of the botnet by Spanish authorities. This is a much more important takedown than Waledac because Mariposa actually put sensitive and financial information at risk. You can read more of this story in Spanish arrests mark the end of dangerous botnet. Ottawa, represent!

Virtualization

Microsoft has released new drivers to improve the performance of Red Hat Enterprise Linux systems running under Hyper-V. The new drivers include new network and storage drivers, but still only support a single virtual CPU. These drivers are not officially included in, or supported by Red Hat, but they are fully support by Microsoft. Read Microsoft releases Hyper-V drivers for RHEL for more, and you can check out Microsoft’s side of the story on the technet blog in Red Hat Enterprise Linux and Hyper-V.

How Server Virtualization Impacts Storage. The title says it all. Penned by Heidi Biggar from Hitachi it has a good perspective on current industry trends and growth. We all know that consolidation methods, like virtualization, add bottlenecks on resources like CPU, memory and storage because they are all shared by many instances. Find out what Heidi has to say on the subject.

Data Center

Data Center Knowledge is reporting that the demand for data centers has been growing continually and no change is in sight for 2010. We’re certainly seeing the trend here with many companies making the push for full, real disaster recovery sites. There seems to be a real shortage of data center space and new ones are popping up slowly, so if you have lots of cash this may be a good business to get into. But I digress. The full story is in Strong Data Center Demand Seen for 2010.

Security

The H Online has a good article about the US government’s new cyber security policy. I haven’t had a chance to review the whole document as yet, but I have it bookmarked for later reading. US government publishes parts of its cyber security directive is where to go, which also links the source document.

Hardware

Tom’s Hardware has an interesting Op-Ed piece on the past, present and future of the 3D graphics industry. See Opinion: AMD, Intel, And Nvidia In The Next Ten Years for 10 pages of goodness.

Allyn Malventano has a very detailed review of Western Digital’s first foray into the SSD market. The new 256GB offering performed well through the tests but is still priced a bit too high for the market. The full review is at PC Perspective, see Western Digital SiliconEdge Blue 256GB SSD Review – WD enters the SSD market. You can find the Western Digital product information page on their site.

Well that about wraps it up for this edition. Comments are always welcome. Until next time, may your days be productive and the levels of administratium be non-toxic.

Having recently moved his blog, Gary approached the Log Buffer coordinator to volunteer for an edition because he knows that, with LB being a popular and established destination in the database blogoshphere, it would help him broadcast his new blog and welcome readers to it. You can do it too–simply send an email to the Log Buffer coordinator.

Here’s Gary’s Log Buffer #181.