Pythian Group

CC BY-NC-SA 2.0

There were two things I have wanted to do for some time now. The first was to come up with a way to quickly and easily set up a DarkPAN mirror so that we would have more control over our dependency chain at work. The second was to make a portable CPAN proxy service, so that I can always have access to my favorite modules, even if the machine I’m working on has no Internet access. Last week, I finally had a few ‘rount tuits’ to spend on this type of background itch, and the result is dpanneur (for dépanneur, French Canadian for convenience store).

Installation

As it stands, dpanneur is a very thin Catalyst application gluing together the goodiness of CPAN::Cache and MyCPAN::App::DPAN, and throwing in Git as the archive manager.

To get it running, first fetch it from Github

$ git clone git://github.com/yanick/dpanneur.git

then check that you have all the dependencies

$ perl Makefile.PL

and run the script that will create the module repository

$ ./script/create_repo

For now, the module repository is hard-coded to be in the subdirectory cpan of dpanneur. A branch called proxy is created and checked out. Eventually, I’ll use GitStore to push newly fetched modules to the repository, but for the time being if dpanneur is to be used as a proxy, that branch must remain as the one being checked out.

All that is left is to fire up the server in whichever mode you prefer (single-thread test server would do nicely for now)

$ ./script/dpanneur_server.pl

and there you are, running your first dpanneur. Congrats! :-)

Using it as a caching proxy

You can use the server as a caching proxy, either for its own sake, or to seed the DarkPAN branches. To do that, you just have to configure your CPAN client to use http://yourmachine:3000/proxy:

$ cpan
cpan[1]> o conf urllist = http://localhost:3000/proxy
cpan[2]> reload index
cpan[3]> install Acme::EyeDrops
Running install for module 'Acme::EyeDrops'
Running make for A/AS/ASAVIGE/Acme-EyeDrops-1.55.tar.gz
Fetching with LWP:
    http://localhost:3000/proxy/authors/id/A/AS/ASAVIGE/Acme-EyeDrops-1.55.tar.gz
etc..

As the modules are downloaded, they are also saved and committed within the repo

[dpanneur]$ cd cpan

[cpan (proxy)]$ git log -n 3
commit d065ad152f2204295334c5475104a3da517b6ae1
Author: Yanick Champoux <yanick@babyl.dyndns.org>
Date:   Wed Mar 10 20:32:52 2010 -0500

    authors/id/A/AS/ASAVIGE/Acme-EyeDrops-1.55.tar.gz

commit e8d2e83d1b16e2e0713d125f9a4bd2742681f859
Author: Yanick Champoux <yanick@babyl.dyndns.org>

Date:   Wed Mar 10 20:31:42 2010 -0500

    authors/id/D/DC/DCONWAY/Acme-Bleach-1.12.tar.gz

commit 7e0b4b600bac8424c519199ee96dc56ffbb177eb
Author: Yanick Champoux <yanick@babyl.dyndns.org>
Date:   Wed Mar 10 20:30:47 2010 -0500

    modules/03modlist.data.gz

Using it as a DarkPAN server

There is not much more involved to enabling DarkPAN repos. All we have to do is to create a branch with the modules we want and have the ‘dpan’ utility bundled with MyCPAN::App::DPAN generate the right files for us.

To continue with the example of the previous section, let’s say that we want a DarkPAN branch containing Acme::EyeDrops, but not Acme::Bleach. Then we’d do

                        # only necessary if you are running
                        # the server while you work on the branch
[dpanneur]$ git clone cpan cpan-work   

[dpanneur]$ cd cpan-work

                        # branch just before we imported Acme::Bleach
[cpan-work (proxy)]$ git branch pictoral 7e0b4b600bac8424c519199ee96dc56ffbb177eb

[cpan-work (proxy)]$ git checkout pictoral
Switched to branch 'pictoral'

                        # cherry-pick the Acme::EyeDrops commit
[cpan-work (pictoral)]$ git cherry-pick d065ad152f2204295334c5475104a3da517b6ae1

                        # rebuild the module list
[cpan-work (pictoral)]$ dpan

                        # commit the new 02packages.details.txt.gz
[cpan-work (pictoral)]$ git add .
[cpan-work (pictoral)]$ git commit -m "dpan processing"

                        # push back to the mothership
[cpan-work (pictoral)]$ git push origin pictoral

And that’s it. Now point the cpan client to http://yourmachine:3000/pictoral, and you’ll get the limited mirror.

cpan[1]> o conf urllist http://localhost:3000/pictoral
cpan[2]> reload index

cpan[3]> i Acme::EyeDrops
Strange distribution name [Acme::EyeDrops]
Module id = Acme::EyeDrops
    CPAN_USERID  ASAVIGE (Andrew J. Savige <asavige@cpan.org>)
    CPAN_VERSION 1.55
    CPAN_FILE    A/AS/ASAVIGE/Acme-EyeDrops-1.55.tar.gz
    UPLOAD_DATE  2008-12-02
    MANPAGE      Acme::EyeDrops - Visual Programming in Perl
    INST_FILE    /usr/local/share/perl/5.10.0/Acme/EyeDrops.pm
    INST_VERSION 1.55

cpan[4]> i Acme::Bleach
Strange distribution name [Acme::Bleach]
No objects found of any type for argument Acme::Bleach

So mothers keep your hackers at home
Don’t let them journey all alone
Tell them this world is full of danger
And to shun the repositories of strangers
– The Tag Set of Strangers,
(with apologies to) Nick Cave and the Bad Seeds

One of the things I love about Git is how I can add branches from remote repositories in mine at will without fearing messing up anything. The remote branches will not clash with mine, even if they share the same names, because they are referenced as repository/branch. However, as for anything else, you can still poke yourself in the eye if you try hard enough:

Read the rest of this entry . . .

The Itch

For many, CPAN is a Canadian Prairies-sized field of modules where it’s darn hard to separate the wheat from the chaff.

While the CPAN Ratings service is the principal and official way CPAN tries to rank its distributions, for me at least, it doesn’t quite scratch the itch because . . . 

1. not all distributions have reviews.
2. even when there are reviews, they generally don’t answer the next question: what should I use, instead?.

Read the rest of this entry . . .

The “Beer” version of DBD::Oracle (1.24) has been released. You can find it at CPAN DBD::Oracle.

DBD::Oracle is a Perl module that works with the DBI module to provide access to Oracle databases. It is maintained by me, John Scoles, under the auspices of Pythian as Open Source/Free Software.

This is largely a maintenance release that fixes a number of bugs. New stuff includes some more improvements to embedded types from Charles Jardine.

Timestamps in Embedded Object now use precision to 6 places.

The big enhancement this round is extending the code to allow more than one row to be returned with a single fetch. Those of you who use Rowcache size will now find that your queries will run much faster.

I have also added the ora_ncs_buff_mtpl parameter, so you can now have very fine grain control over the size of the data buffer using for NCHAR conversions of lobs.

One more little note — thanks to my mistake in compressing a file last week, the version number on CPAN is 1.24a, while the internal Version is still 1.24.

Here is the complete change list: Read the rest of this entry . . .

CPAN Wants You!

Want to contribute to your favorite CPAN module, or maybe create your own, but don’t have the foggiest idea how to do it? Here are a few notes, tips, tricks, and links that might help you get started.

PAUSE id

While bringing awesome street cred, having a PAUSE id is strictly necessary only if you want to maintain or co-maintain a module. If you just want to contribute code, you’ll perfectly be able to do without, as it will usually be done via patches submitted to a bug tracking system, a code repository or using good ol’ email.

Becoming a co-maintainer

Becoming the co-maintainer of a module gives you the power to upload authorized releases of the modules on CPAN. To become one, the maintainer of the module simply has to promote you as such on PAUSE.

Creating a new module

You want to create your own module? Read the rest of this entry . . .

Half the time I want to peek at the doc of a module, I hit perldoc.  The rest of the time I type cpan Some::Module^[1] in Firefox and read the POD straight out of CPAN.  And while it’s pretty and handy, it also feels kinda silly to go on a remote server to read documentation that is also sitting on my computer. Surely, I tell myself, there must be a better way.

Cue in the several Perl modules that act as local POD web servers.  After giving a few of them a quick test-run, I decided to give Pod::POM::Web a try. Being a CLI jockey, I wanted to be able to open the POD of a module from the command line.  Not a problem, I just had to create the script ‘pod’:

#!/bin/bash

POD_PORT=8787

perl -MPod::POM::Web -e"Pod::POM::Web->server($POD_PORT)" 2> /dev/null &

PAGE=`perl -e's(::)(/)g for @ARGV; print @ARGV' $1`

HOSTNAME=`hostname`

kfmclient openURL "http://${HOSTNAME}:$POD_PORT/$PAGE";

There is not even a need to fire up the Pod::POM::Web server beforehand: the script will do it for us (if the server is already running, subsequent calls to pod will harmlessly try to start a new server on the same port and fail).  It should be noted that ‘kfmclient’ is KDE-specific — for any other desktop environment, you might want to change that to a direct call to firefox.

It’s already not too shabby, but wouldn’t it be even better with a little bit of auto-completeness magic?  To do that, we need a short script, pod_complete:

Read the rest of this entry . . .

My old arch-nemesis, the in-line if ($q = $q == $a ? $b : $c;) reared its ugly little head again.

This time, it was in context of an web page that displayed some form values, something like this:

CGI::textfield({name  => 'price_dollars',
                        width => '5',
                        value => ($mode eq 'Edit' ? $line_item[$count]->price_dollars() : '0') });
CGI::textfield({name  => 'price_cents',
                         value => ($mode eq 'Edit' ? $line_item[$count]->price_cents(): '00' });

This carried on for 15 other fields on the form. So, we have 17 if else statements all checking to see if the form is in ‘Edit’ mode. If there were, say, ten line items on the form . . .  well no need to go any further, other than to say that is a whole lot of if statements.

While this does not take up much space, this multiplicity of ifs is not really necessary or even good, since to the compiler, an inline if and a bracketed one are the same. The inline is only a shorthand to make our code more readable.

We could of course just declare 17 scalars at each iteration, and then a use single if statment to set these scalars, like this . . . 

Read the rest of this entry . . .

One thing I find fascinating in Perl is that I am always seeing new ways to perform the same mundane task.

Today I had to output some tabular data, so I thought it would be nice if I alternated colours for each row. Easy enough in Perl—just create a hash with your colours as the value and then the swapping variable as the key, like this:

my %colours=(1=>'red', 0=>'green');
my $swap=1;

foreach $item (@stuff) {
    my $colour=$colours{$swap};
...

Then I though about how to flip the $swap value. I could simply use the tried and true—and like me—Luddite style: Read the rest of this entry . . .

I finally had the opportunity the other day to try and install DBD::Oracle on an IBM AIX 5.1 box, and for once I have some good news to tell.

Anyone who has ever tried this will know of some of the troubles I speak of. When dealing with DBI and any DBD on a AIX box, you either must either be lucky enough to have the same compiler installed that built the version of Perl that comes with the box (I have never seen this happen); or you have to spend a great deal of time downloading and installing your own GCC and the building your own version of Perl.

Fortunately, all the hard work was done for me by other members of my team, and I was only a Johnny-come-lately to the whole process.

In our case, these are the steps that we followed:

1. get a working version of GCC on the box
2. rebuild and install your own version of Perl
3. build and install DBI using your local version of Perl

All of the above worked without a major problem. It was only when we tried to build DBD::Oracle that we ran into a problem:

Read the rest of this entry . . .

This is a little story of a little bug. This gremlin suddenly appeared in a CGI.PM web-based application I work with. To make a long story short, an email was coming out something like this . . . 

389939
Subject:Update to Report #389939 by B. bloggins Description:389939 #389939: TPDD Now Deploying to monitoring for the MySQL servers.

 . . . when it should have been some thing like this:

Subject: TPDD Update to Report #389939 by B. bloggins

TPDD Now Deploying to monitoring for the MySQL servers.

After about an hour tracking things back, my team and I narrowed it down to this line of code:
Read the rest of this entry . . .