UPDATE: Jeremy responds:
I guess the low-down is that if you’re a company that provides a service, you either need to be ready to scale or you need to be ready to limit access to your service. Users shouldn’t suffer. But if they do, at least communicate. Thankfully, that’s something FeedLounge does really, really, really well.
Agreed, agreed, agreed, agreed, thanks.
I guess they’re made of different stuff than most of the companies I deal with. Best of luck to you guys, and sorry my rant put you in the crosshairs
Alex and I are very heavily customer focused. We want to keep it that way.
In his post entitled Web 2.0 Companies NEED To Scale, Jeremy Wright makes a few good points, and a few bad ones. I am glad he chose FeedLounge as an example, as it gives me more than enough reason to respond. The points he attempts to make are mostly valid, but not always applicable to the small, bootstrapped player.
I’m not sure when building a scaleable web app became optional. But Feedster, Technorati, Delicious, Google Analytics (and numerous other Google apps of late), BlogPulse and many of the other “big apps” have “suddenly” been hit by scaleability issues.
First, building anything is optional. Building an app, building a web app, building a scalable web app. All optional. You don’t need to do any one of the list. Even when you choose to build a web app, you will pick a target to scale to. FeedLounge chose 2 users as the initial ’scale to’ numbers, seeing if we could build enough functionality and a great user experience. We then released it to a few friends to see if they liked what we built. They did. When we were hit ’suddenly’ by scalability issues, we knew it would happen sometime and dealt with it accordingly.
Yeah. Here’s their process:
1. Start with a handful of users. This is too much for ded box.
2. Move to dedicated server.
3. Add a few more users til they’re at 100. This is too much for one box.
4. Add more hardware. It’s obvious this isn’t enough.
Erm… Hello? Should the recoding have happened after step 1? I mean, if you draw a graph of “okay if we use 10% of a CPU with 10 users, with 100,000 users we’ll need 10K CPU’s” … Something’s wrong.
The FeedLounge development process was more along the lines of:
- Build a webapp, see if the features are compelling to a set of users, keeping a design in mind that is capable of scaling
- Overrun the shared server that you are using, switch to dedicated server, so you can properly measure the effects of the application.
- Add more users, adding requested features from the users, measuring the load in a fixed, known environment, and start work on “Distributed” part of ladder. The is where the build portion of the scalability starts.
- Now that you believe you have something that has value, invest in the hardware and software development necessary to scale. Continue working on priority based tasks towards release of your product.
The design of the application allows scalability/availability to be added as time and money allow. The ‘recoding’ has happened every step along the way. The focus was not and has not been on scalability. It has been on whether we can provide value to our user base. If we were to focus on what you deem important in this article from day one, a lot of people would be able to look at a horrible application, and no one would use it for any significant amount of time. Perhaps you think that FeedLounge has infinite pockets to dip into for hardware infrastructure and development talent? Hint: We don’t.
We are on step 4, and it is going slowly since our team is so small. It was much more important to show what user interaction we could build, and then worry about total performance and scalablility afterwards.
The business model also comes into play. If we were to choose to sell a software product instead of a service, the software we have will work fine for hundreds of users, no problems. Since we have finally chosen to go out as a web service, scalability to many thousands of users is a very important requirement, but only now.
Nothing is wrong. It is all choices that you make trying to start a company. Alex and I chose to focus on the user experience instead of scalability, and now we know that we have to scale. I knew going in to this venture that it would be a huge amount of data to move. I do have a great number of years experience making fast things faster in the software world. You mention that is “astounds you” as to what people define as scalable and available. No one has ever used those two words to define FeedLounge. Nor will they, until we have proven that we can. Ask any of our alpha users. They don’t stick around for the availability, they stick around for the features that they have been given. And yes, scalability is a feature, but not one that should be the major focus in incubation.
Maybe I’m just spoiled, having worked in high performance, high availability apps before, but it constantly astounds me what some folk consider “scaleable” and “available” applications.
Scaling of resources and time is also important in the real world. Expecting Alex and Scott to scale to the level of Google and Yahoo! (or even VC funded companies like feedster and technorati) is just silly. Once you look at what we have done with the resources given, I think your tone will be quite different.
At FeedLounge, we are taking a realistic reactive approach to optimization, versus a predictive, all-encompassing approach. We designed a platform that we knew we could scale in a distributed environment, identified the areas that needed to be refactored to scale, validated those with measurements, and then wrote the code to make it a reality. Remember, premature optimization is the root of all evil.