It all evens out in the end

Good days, bad days, evened out days.. I seem to get more of the even days then anything else, which I guess is a good thing (it could be worse, right?) take for instance yesterday… I’ve been beating on something at work for a couple days now, this nasty performance bug in an VB application I wrote, basically this part of the applications allows the user to export their entire database of transactions, on a floppy and then import them onto another machine.

It’s actually kinda neat the way it all works, each transaction consists of a single header record, and any number of detail records (could be one, could be a couple hundred, who knows)… Well, even though the database is stored in an access database for the application itself, exporting them to an access db wasn’t a workable solution (as the MDB turned out to be waaaay to big to do anything with on a floppy in practice). So the export dumps two XML files, one containing all the headers, the other has all the details. These two are then compressed down (You ever seen how well xml compresses? Sweet Jeebus!) and copied over to the floppy.

The problem was on the import side of things. It loads the data for all the headers, steps through them one at a time, and looks at the transaction number. If that transaction already exists in the local database, it skips it (don’t want transactions in the system twice, and you can’t trust that a user won’t import the same disk twice, right?), if it doesn’t already exist, it loads the detailrecords from the 2nd XML for that particular transaction and loads both the header and details into the local database… voila, suddenly everything that’s on the disk you don’t already have on the machine is there.

Well, small problem, we’ve never really tested it in a real-life setting. Sure, it worked in the development machine, with a couple dozen transactions at most, but it’s only now, almost two years after the application’s been in use, that we get to find out a problem has existed all along.

Once a fairly large number of transactions (say, almost two years worth, like if your office computer up and died and you need to move my application to another machine using last weekends ‘export disk’ you just happened to make containing all your transactions), the export is great, they’ll all fit on the floppy, no problem (a product of my mad compression algorythm and tight database design no doubt!), but when you move over to the new machine and import two years worth of orders, it *crawled*. On a 3Ghz 1Gig Ram P4 system, it was taking roughly 2 and a half seconds per transaction. Said client had roughly 1900 transactions to import.. on my badass machine that would have been about an hour and 10 minutes. On the machine they were trying to do it on, it ran for 20 hours and was only 20% done, (a p2-200 laptop with I think 64megs?) a five day job to import 2000 transactions?! That’s not good.

Long story short, I spent a couple days beating it around in my head, and yesterday morning found a way to speed the entire process up to the point that on my dev machine (the 3ghz, 1gig system P4), it now imports the 1900 transactions in just under 56 *seconds*. (~ 33 Transactions a second, verus 2 and a half seconds a transaction? I’ll take it).

So I was on a techno-geek high! Nothing could bring me down! Beautiful clouds of warm fuzzy happiness flowed through my mind…

Until sometime in the later in the day when I decided to check something on LtB:

“We’re sorry, the owner of LtB is a moron, and this domain has expired! bahahaha!” is what it said… well, not exactly, but it might as well have.

See, I knew the domain was originally registered in January, but since I really didn’t use it until later in the month, I was thinking I registered it around January 10th, or 12, or something… turns out I registered it on the 3rd… my bad. So the site’s been down since then, and guess who didn’t even notice it until yesterday.. yeah, I know, techno-geek extrodinare at your service.

At any rate, it’s all squared away now, and LtB is back up and running.. happy happy joy joy. Of course I felt foolish and stupid, I let my domain expire! My hip, cool, swanky domain, the place I wanted to do so much with, blah blah blah.. yeah, okay, it hasn’t all happened yet. I’ve got oodles of code written for sections around here that haven’t appeared yet, but they will, someday, I promise… really….. I know some people won’t believe that…. as someone said to me last night as I relayed the story of my forgetfulness “well, no loss, you never use it anyway! An update a month does not a site make!”

Bah.. some people can.. well… Lick the Balls.

Leave a Reply

Your email address will not be published. Required fields are marked *