Note: This post was written a couple of months ago, and never posted, because I always wanted to add more to it. All testing was done while the FeedLounge alpha was on a single server, and FeedLounge has moved on from that. Seeing as I might be switching away from MySQL altogether, I thought I should post this for anyone else running into these types of issues…

I was thinking that the MySQL MyISAM engine’s table level locking was killing the FeedLounge application and user experience in the past week, since the feed update daemon does a large amount of writes to the database almost constantly. I wanted to test the InnoDB storage engine, to see if the row-level locking would offset the fact that I use GUIDs as primary keys, which make primary key inserts not in sort order. So, I started on my journey..

First step, get the data over to the test server. This step was fairly simple and straightforward, as I do a nightly database dump from the alpha server, and rsync it over to the test server, to have a geographically diverse near-online backup. This was just a matter of doing this again to have the latest. Total time: about 25 minutes.

Step two, configure the test server for InnoDB. This was the easiest step, just setting a few variables in the my.cnf file, then restarting mysql. Total time: 10 minutes.

Step three, load the data into the test database. I dropped the existing test database, and then wrote a quick sed script to change the engine type in the massive SQL backup file. I then started the load. A dump takes about 20 minutes now, and a load on the alpha box took about an hour, so I figured it would be in the one to two hour range. Boy was I wrong! The 1.9 GB SQL file ended up taking about 14 hours to load on the test box, but that wasn’t the worst thing.

On the alpha server, using MyISAM, the entire database took about 2.4GB of disk space, including the indices. Once the load was done on the test server, the InnoDB files totaled over 10.6 GB!!! You’ve got that right, the FeedLounge grew to take up almost 5x the space just by changing database storage engines! This obviously throws our disk space calculations out the window. The bin logs on the test box total 1.7GB, so the data seemed to be correct.

So, now I am over an entire day into this testing, but the main question is, can I live with the 5x growth in data size? Doesn’t that seem a little like overkill? Is this stated anywhere? Is this just another one of those ‘known things’ that you are supposed to know as a MySQL DBA?

We are now on InnoDB, and the lockinng/contention issues are gone. The only issue left is the speed of count() queries on InnoDB, which is a known issue, and even documented in Zawodny’s excellent book. I am working on refactoring the code to remove the counting queries, to free up the database to do real work.

Popularity: 71%

After looking at the source for Mochikit, Dojo, Prototype and others, I decided that FeedLounge could use a bit more object-orientation in the client side code.

I created a simple object structure, and wrote a few little test scripts that I ran in the browser. Since I subscribe to 1500+ feeds in FeedLounge, the feed loading/processing on the client side is getting quite slow, so I wanted to make sure that this refactoring was going to help and not hinder client-side performance.

While the test results show what I have built will help performance quite a bit (no more looping over DOM nodes, looking for the right one), the results were a bit surprising…

Given 2 objects, feed and tag, where a tag can contain feeds, I wrote a simple loop to create n feeds, and set an unread count on the feed, then add that feed to a single tag (the worst case in FeedLounge). I wanted to make sure that this solution would scale for the power user, so I started the test at n=5000.

At 5000 feeds to a tag, Firefox was running the code in less than 1 second, which seemed acceptable, so I bumped up n=50000. Firefox choked saying “Out of memory”, so I dropped it in half to n=25000.

The loop at 25000 took 3 seconds of full CPU to run, and the count items return was quite quick. On to test IE, since we support more than just one browser…

In testing IE, I was shocked to find it taking 3 minutes at 100% CPU to run the loop at n=25000. It did eventually finish, but while I can justify 3 seconds on initial home page load to my users, I don’t think they will stand for 3 minutes! I also noticed that memory consumption in IE was over 3 times as much as Firefox. For n=25000, the delta in Firefox had been 50MB. That seems like a lot, but I also didn’t see many people subscribing to 25000 feeds. In IE, the memory usage was 160MB!

Does anyone have any ideas in how to optimize JavaScript for IE? I am doing some extremely simple things here, nothing rocket science at all.

I am not too terribly worried, as the patch for power users of FeedLounge is to switch to Firefox

Popularity: 43%

Having used Skype very little in the past, mostly because I have a lot of cell phone minutes, I had fogotten it was there, until a few folks at Apache set up a Skype conference call to talk about some infrastructure related items.

Since I do not own a headset for the computer, and I lost the charger to the abysmal Bluetooth headset that I have, I was looking for other options. I found that Skype has the ability to forward to your cell phone number when you are not online. After trying to find it in the options, I Googled a bit more and found out it only worked on the Windows client.

Although that sucks, I do still have a Windows machine “just lying around”, and I installed the latest Skype client for Windows. It seems that the call forwarding option works when you make yourself look Offline, but the Windows client is the one doing the work.

I tested it out with Leo, and it seemed to work just fine, but I have to ask myself, if this uses SkypeOut minutes (I am paying for it), shouldn’t it also work when I am not even net-connected? That would seem like the most appropriate time to use such a feature, no?

Popularity: 28%