New Take On Scalability
By Alex Bunardzic under Theory on 08. October 2005Apologies for dragging this corpse into the discussion again, but some latest development prompted me to try and share this with others.
Whenever I give presentations on Ruby and Rails, the number one question invariably pops up from the audience: will it scale?
At first, I was allowing myself to fall prey and to drop down into the detailed and quite meaningless discussion. But then I’ve changed the tactics, and began countering the question with: “Scale to what?”
Amazingly, most people don’t know the answer to that question. They just throw in abstract answers, like to thousands simultaneous requests, etc. But pretty much no one can supply a real life example that is more concrete than the yahoo or amazon or google. In other words, it seems that not too many people are working on the super busy web applications.
If that’s the case, the scalability question is, in most situations, quite moot.
But the real change in the scalability landscape is now dawning with the advent of the rich internet applications. Here is how it goes:
Originally, when we’ve made the transition from fat clients in the client/server world to dumb terminals (i.e. HTML documents rendered in the browser) living in the web 1.0 world, we’ve effectivelly sucked all life out of the clients. The clients got demoted to stupid braindead appliances for rendering HTML.
But someone had to continue doing their work. That someone was the app servers. The central role of the app servers (such as WebLogic, WebSphere, etc.) was impersonating numerous clients out there. The app servers had to do incredible amount of work impersonating the clients who were only known via their requests peppered with some cookies.
This situation resulted in incredible strain placed on the app servers. The whole multi-million (or is it billion?) dollars industry sprang around those beasts, and new careers were forged around WebLogic, WebSphere, JBoss etc. servers.
Seeing how huge the burden of impersonating numerous clients on the app server is, the question of scalability became one of the central issues.
However, now that the clients are finally reclaiming their state, and that the app servers do not have to continue impersonating each and every client, the whole scalability issue becomes meaningless. Let the clients do all the processing that governs their behavior. Suddenly, app servers are left with very little to do. And as the clients are happily humming and taking care of their own state, all that the back-end resources need to worry about is the centralized business logic. And in that arena, there is very little scalability issues.
So, now this whole topic is even more of a moot point. It’s time we gave it a decent burial, and moved on to the greener pastures.
Marko Samastur:
My question would still be does it scale to at least hundreds of simultaneous connections?
I don’t work for the big boys, but I did work for an ISP with 100k users and every morning our webmail just had to survive couple of hundred simultaneous connections. I’m working on an application that will have to meet at least similar load if not bigger.
I started to use what is now called AJAX, because I expected to lower the load on servers. I did, but not by as much as I expected, simply because users click more when cost of a click is much lower (cost measured by loading time).
I agree scalability is not a major issue for most people, but it’s not going away either. At least not for all of us.
comment at 08. October 2005
Scott Moody:
Two quick responses:
1) I think the question most people mean to ask is “How expensive is it to scale?” The last web service I created was getting 15 million page views per day (and it wasn’t a search engine). I would hazard to guess that no one has created a site with Rails that serves-up that many pages in a day — mostly concentrated in a 12 hour period. Can Rails handle that? With enough boxes, of course! What will it cost (and there is more to the cost of hardware-based scaling besides the cost of the hardware) ? Unless someone has crossed that threshold, then the answer is unknown. With other platforms, the cost of serving up that many pages per day can be found out with enough research. And, I would argue that as other language/framework combinations catch-up to Rails from a programmer productivity perspective (the real Rails advantage), it will be an even more important question to be able to answer.
2) I think I disagree with the last few paragraphs. I think that, as will happen more and more, rich DHTML clients will put a greater load on the server. One example is the various implementations of Google-suggest-like functionality. Suddenly, you’re hitting the server with numerous times where, in the days of old, you weren’t hitting it at all. Dynamic-updating views, such as those on Digg Spy are another example. I think you’ll see it more and more. If you buy into that, then the resources required for the back-end will be increasing as clients get more dynamic.
So, although I think that the answer to the question “Can Rails scale?” is “Yes,” it is far from a moot point and it should not be buried. Finally, I am a Rails fan and believe that it can scale effectively.
comment at 08. October 2005
Aristotle Pagaltzis:
This seems like the kind of thinking that Joel Spolsky keeps scoffing about.
Scalability matters now as much as has mattered in the past, and it will continue to matter just as much in the future.
The notion that scalability doesn’t matter anymore is even more of a crock than the Java mindset that scalability is a checklist item to be built into the framework which then automagically becomes a property of any application that is built on top of it.
Asking whether Rails will scale is meaningless—but not because scalability is meaningless. It’s like if you were asking if Rails is secure. These questions are meaningless, because beyond due diligence, the framework can do nothing to ensure the answer as regards your application is “yes.” That responsibility falls squarely with the application programmer.
So let me rephrase above statements.
comment at 08. October 2005
Brook:
“So, now this whole topic is even more of a moot point. It’s time we gave it a decent burial, and moved on to the greener pastures.”
No–in fact, after some of these “Scalability doesn’t matter” posts, I’m even more worried about it since no one is really addressing it head on. It becomes a real issue when all of a sudden your site grows and people are unable to use your application. I’d like to see some tests of Rails scalability. If it doesn’t work well, improvements can be made. If it does work, then you can put the issue to bed.
This “scalability doesn’t matter” stuff comes off as disingenuous.
comment at 08. October 2005
Michael Koziarski:
The thing which annoys me the most is that “Scalability” is a completely meaningless term without a qualifying statement “with respect to #{X}”. It’s like pointing at some function of 15 variables and demanding to see its “derivative”.
Does rails scale to larger teams?
Does rails scale to dumber teams?
Does rails scale to millions of page views?
Does rails scale to thousands of heavy operations?
Does rails scale to ridiculously baroque requirements?
These questions all make sense, and can be answered. But asking whether something ’scales’? It reveals more about the questioner than any answer would about the framework.
comment at 08. October 2005
Alex Bunardzic:
I’ve just double-checked, and yes, we are indeed reading and writing this while living in the 21st century.
For a while there, I was a bit confused; some people were foaming at their mouths that, even though year 2000 had arrived, that portentuous event didn’t really mean that the 21st century had actually arrived with it.
But by now (year 2005) I figure we’re firmly entrenched in the 21st century. No?
So what’s new in the 21st century? Well, to begin with, the business model had somewhat changed, inasmuch as the “innovation in assembly” (think Dell, for instance), seems to be the preferred approach.
Also, the age of ‘applications for a lifetime’ appears to be behind us.
What does it all mean, then? If someone was to grab the unexpected and unanticipated handful of disparate services, and assemble them on the fly, offering a new and irresistible service (the notorious mashup), how can that ‘assembler’ engineer his new-fangled solution for scalability?
Furthermore, if anyone approaches me to build them a Rails app that will have a projected shelf life of 10 to 15 years and would be expected to serve hundreds of millions of users, I’d politely have to bow out. Rails is not meant to be that kind of workhorse, so there.
I’m mostly focused on delivering simple, small scale short-lived solutions that are focused on communal needs. A disposable, throwaway software apps. I’m having troubles imagining under which conditions and circumstances would such solutions ever reach the mythical scalability issues.
comment at 08. October 2005
Anonymous:
“But asking whether something ’scales’? It reveals more about the questioner than any answer would about the framework.”
That’s cute, but answers.com/scalable says this: “A “highly scalable” device or application implies that it can handle a large increase in users, workload or transactions without undue strain.” I’d expect that’s pretty much what most people think of when someone asks if something is scalable.
If someone asks you if you like steak, do you tell them that they are ignorant because there are so many different kinds of “steak” and different ways to cook it?
comment at 08. October 2005
Aristotle Pagaltzis:
Alex: some figures from eToys, in Christmas of 2000:
The site was built with Perl.
Can Ruby do that?
(Answer: d’uh.)
comment at 09. October 2005
Alex Bunardzic:
Aristotle Pagaltzis wrote:
This is like asking a typical restaurateur who’s running a local corner restaurant: McDonalds serves billions and billions; can your business do the same?
Yes, we all know that McDonalds is scalable. But why are we all assuming that every other restaurant or chain must be designed from the ground up to fend the hypothetical onslaught of billions and billions of customers? Who’s to say that the billions of customers will ever materialize?
Why is everyone so obsessed with the scale? Obviously, (and one need not go farther than look at McDonalds), scale usually implies crappy products/services. Yeah, McDonalds can scale like crazy, but at the undeniable expense of quality. Their products, as is well known and documented, are plumbing the ever new depths when it comes to unhealthy eating choices.
And I’m not even going to go into examining in any detail the quality of their service. Suffice it to say here that it’s atrocious.
Us Rails developers are embracing a new philosophy — we want to be the exquisite chefs, not the onion rings and french fries machine operators. Instead of flipping burgers, we want to perfect the ancient art of dry aging those steaks and matching them up with the finest wines.
That’s why we don’t lose our sleep over scalability. We know that there are sufficient resources that will take care of that for us. Why sweat the details?
comment at 09. October 2005
Masklinn:
Aristotle Pagaltzis:
This comment is stupid and meaningless: you don’t give any information whatsoever on the physical part of the website.
Saying that an app can handle “200,000+ sessions/hour, 2.5 million+ page, views/hour, 20,000+ orders/hour” has a net value of 0 because one has no idea of the database used, the DB’s server, the web servers, the distribution of the load, …
This is why asking “can language/framework XXX scale?” is meaningless unless you add WHERE it’s supposed to scale (team size, db requests, page view, specific operation) and which physical limitations are to be taken in account (database, servers, database hosting, network links between the db and application servers) and are essential in answering such a question.
comment at 09. October 2005
Aristotle Pagaltzis:
Okay, you had me there for a while, then you threw this doozy:
That seems to imply that the eToys “service” was in some sense “crappy.” First, your “crappy service” is even more vague than the “scalable” scarecrow! Second, how did you manage to judge the “quality” of the “service” just by looking at the scale figures? Is “it scaled massively, so it must be crappy” (for some vague-at-best definition of crappy) what you’re saying?
I think the point you’re trying to make somewhere in there with your arguments (ie most people don’t need to concern themselves with scalability) is actually a good one, but in attempting to do so you throw in a bunch of strawmen that muddle your message.
comment at 09. October 2005
Aristotle Pagaltzis:
Masklinn: hence the “d’uh” at the end of that comment, and hence my earlier comment that scalability happens at the application level, not the framework level.
Read the whole thing if you want to know more.
comment at 09. October 2005
Paul M. Jones:
Alex asked, “Why is everyone so obsessed with the scale?”
That’s an easy one: everyone wants to believe they are going to be “big.” They want to use tools that let them indulge in that belief, however statistically unlikely their eventual bigness may be. Alternatively, they believe they are in fact already “big,” not understanding how massively unbelievably big* they would have to be for scalability to really matter.
To hear “you’re not big enough for scalability to be important” is a jarring and unwelcome intrusion of reality onto imagination. If they accept reality, the next thought is usually not “Ok, we’ll use less-scalable stuff”; instead, it is “Crap! Now we have to figure how to become big, which means we still need a scalable solution that will handle our eventual bigness.”
In short, it often has more to do with what they think of themselves and their dreams for the future and less so with a realistic appraisal of requirements.
(* With apologies to Douglas Adams.)
comment at 09. October 2005
Scott Yang:
Re: Paul,
No. It is not true. Thinking scalability is not just an intellectual exercise, thinking something unrealistic, dreaming to be big, etc. The need to be scalable is real in the business world, and it will hit you when you are not prepared.
It is quite possible to claim that your existing application handles this number of concurrent users, performing that number of transactions per second. The next morning there might be some other prospect showing up at the door, asking whether the app is scalable if they have 5 times the users. In fact I am the one stuck in this situation, having designed a Python app server at work that has a clear bottleneck that might not be able to easily scale — and a big prospect is now questioning about the performance!
And the idea of scalability is more than just handling thousands of simultaneous users, but will the number served proportional to the amount of hardware you throw at it?
If the answer is a simple yes, then from a business perspective, I can say the system is scalable. Need to serve more users? Just dump in more hardware. Buy a bigger box. Buy more boxes. Buy faster interconnects.
However, the answer is usually not that simple. There usually exists bottleneck in the system, that you just can’t throw more hardware to solve it. You can usually load balance web servers, but you can’t always load balance app servers or cluster databases beyond a certain limit.
Ajax only plays a small part of this game. And I agree with Scott Moody. When you are making more calls from browser to web server to achieve “responsiveness” that a “Web2.0″ application is giving you, it is actually putting more load on the system.
comment at 09. October 2005
Alex Bunardzic:
Aristotle Pagaltzis wrote:
You’re assuming that ‘crappy’ is more vague than ’scalable’ simply because crappy is not quantifiable. The flaw in such way of thinking is twofold:
My thesis is based on the observable fact that quality tends to diminish as the quantity increases. Take, for example, cooking. I can predict, with reasonable accuracy, that I can prepare a decent meal for my family of four. If, however, someone asked me to prepare that exact same meal for a party of 400 people, or 4,000 (or 4 million people), I can guarantee you that the quality of the food I’d prepare would get progressively crappier the more people I try to make it for.
Exact same criteria apply to the art of software development. If I focus on developing a software app for a clearly defined group of friends/colleagues, the app I’ll deliver will be of a much higher quality, simply because I’ll be able to determine what is it that this small group really needs in order to perform their job. If, on the other hand, I build an app that potentially millions of users are going to use, the quality must, by definition, automatically go down the toilet.
In an unlikely case you’re having any doubts about this, just look at Microsoft’s products. If you’re not able to see the level of utmost crappiness built into these products, you’re probably heavily biased towards Microsoft.
The fact is, if we have to build an app to serve millions and millions, we would be forced to pay extremely close attention to the robustness, failover, etc. This means that our energies would be totally spent on such issues, leaving nothing left to focus on real issues, such as the quality of the end-user experience.
comment at 09. October 2005
Anonymous:
“Assuming that ’scalable’ is an objectivelly quantifiable concept”
It is. A dictionary definition was provided above. People can try to take the semantics defense if they want, but everyone else understands what the argument is about.
“My thesis is based on the observable fact that quality tends to diminish as the quantity increases. Take, for example, cooking. I can predict, with reasonable accuracy, that I can prepare a decent meal for my family of four. . If, however, someone asked me to prepare that exact same meal for a party of 400 people…”
Over and over again, you trot out these odd strawmen to try to enforce your arguments. What does cooking a meal for a small family have to do with writing digital web applications to be consumed by potential millions? Terrible analogy. There are several mass-produced products that can be considered of a very high quality, besides.
“If, on the other hand, I build an app that potentially millions of users are going to use, the quality must, by definition, automatically go down the toilet.”
It doesn’t need to be if the tools you use are designed for it. See Joel Reymont’s blog for some inspiration: http://wagerlabs.com/idealab/. He’s been using Erlang to knock out high-performance, scalable applications and arguing about how easier it is than using Java.
So, first it was “Who cares about scalability”, and now it is “We’re creating non-scalable applications on purpose”.
comment at 09. October 2005
Alex Bunardzic:
Someone wrote:
I wholeheartedly agree. If you don’t understand the analogy, then it’s a terrible analogy. However, that’s your problem, not mine. You can lead a horse to the water, but you can’t make it drink. Er, wait, was it a ’straw horse’?
Whatever the case may be, I cannot make you understand something you simply can’t understand. Let’s agree to disagree.
Besides, I never spoke about writing digital web applications to be consumed by potential millions. I will never ever try to do something like that, simply because I know better:-)
If I only had a dime for each time I’ve heard such claims. Still, no one was ever able to show me such a mythical beast. A Sasquatch of the software industry, maybe?
comment at 09. October 2005
Jude:
The problem is is that you are creating a poor analogy and using that as your argument (see Straw Man). Software construction and cooking differ so much that the analogy is bad:
Put away the fallacies and stick to the little boxes with the clicky things that run applications: it’s what we’re here for.
comment at 09. October 2005
Chewing The Cud » Blog Archive » Scalabi…what?:
[…] Alex Bunardzic makes some interesting arguments regarding the scalability issue that seems to blunt the point of attack for LAMP critics: Whenever I give presentations on Ruby and Rails, the number one question invariably pops up from the audience: will it scale? […]
pingback at 10. October 2005
Mitch Barnett:
How about lesscode and more design?
I like Alex B’s restaurant scalability analogy, but more on that in a moment.
I think the issue of scalability is all about the design and has less to do with any particular programming language, application server, framework, or infrastructure. If the design (via the requirements) calls for the application to scale to x, y or z, with meaningful metrics, then it is up to the Architect to “design” an application, system, or whatever, to ensure the design meets the requirements. And btw, we are in the software world, we can make anything scale – nothing that time and money can’t solve in our virtual world.
When I first came to lesscode.org, I thought it was going to be about more design and lesscode. However, it seems that the site is about people that really care about handcrafting excellent code. I can respect that, just different to what I though it was going to be about.
Back to Alex’s restaurant example. In fact, Alex B. has described an analogy that MSFT uses for its software factory vision. Check this out:
“A software factory is a product line that configures extensible development tools like Microsoft Visual Studio Team System (VSTS) with packaged content and guidance, carefully designed for building specific kinds of applications. A software factory contains three key ideas: a software factory schema, a software factory template and an extensible development environment:
•Think of the software factory schema as a recipe. It lists ingredients, like projects, source code directories, SQL files and configuration files, and explains how they should be combined to create the product. It specifies which DSLs should be used and describes how models based on these DSLs can be transformed into code and other artifacts, or into other models. It describes the product line architecture, and key relationships between components and frameworks of which it is comprised.
•The software factory template is like a bag of groceries containing the ingredients listed in the recipe. It provides the patterns, guidance, templates, frameworks, samples, custom tools such as DSL visual editing tools, scripts, XSDs, style sheets, and other ingredients used to build the product.
•An extensible development environment such as VSTS is like the kitchen where the meal is cooked. When configured with the software factory template, VSTS becomes a software factory for the product family.
To press this analogy further, the products are like meals served by a restaurant. Software factory stakeholders are like customers who order meals from the menu. A product specification is like a specific meal order. The product developers are like cooks who prepare the meals described by the orders, and who may modify meal definitions, or prepare meals outside the menu. The product line developers are like chefs who decide what will appear on the menu, and what ingredients, processes, and kitchen equipment will be used to prepare them.”
How about that? In fact, if you look at any “industrialized process” it has been designed to scale. The key word being “designed”. So if the restaurant needs to scale to millions, then design it to scale! While I respect my fellow codecrafters, I am waiting for the industrialization of software to take place so I don’t have to keep writing the same base code over and over again for each job that comes my way. After serving a million (of the same) burgers, you would think the codecrafter’s would get tired of this too?
comment at 10. October 2005
Anonymous:
I was happy when Mitch decided to join our forum, because I knew he would bring a much needed fresh perspective on software development. It’s a nice remix of code crafting/code industrialization.
And yes, some issues do boil down to lesscode –> more design. But that’s not the whole story. I’m planning to write more about the other side very soon.
comment at 11. October 2005
Dan:
From Paul Grahams latest http://www.paulgraham.com/sfp.html:
“That’s why we advise groups to ignore issues like scalability, internationalization, and heavy-duty security at first. [1] I can imagine an advocate of “best practices” saying these ought to be considered from the start. And he’d be right, except that they interfere with the primary function of software in a startup: to be a vehicle for experimenting with its own design. Having to retrofit internationalization or scalability is a pain, certainly. The only bigger pain is not needing to, because your initial version was too big and rigid to evolve into something users wanted.
I suspect this is another reason startups beat big companies. Startups can be irresponsible and release version 1s that are light enough to evolve. In big companies, all the pressure is in the direction of over-engineering.”
comment at 11. October 2005
Harry Fuecks:
Asking “does it scale?” is often used as an angle to shoot down a technology. PHP has been on the receiving end of this for a long time from the Java crowd - it might be worth tracking down some of the online rants on this, at least to get a feel for the common points of ignorance (e.g. performance == scalability). Often the questioner isn’t willing or able to listen to technical detail. That a 10 line Perl script using fork may scale better, depending on the problem, than an app server that cost $$$ is beside the point.
That said, you may get people who genuinely want to know and have the experience to understand. I guess they want to know where using Rails can be problematic for scaling (e.g. ActiveRecord generating significant DB overhead) and how to work round it (e.g. knowing when to insert some hand-crafted queries). And otherwise hearing other peoples experiences, tips on how to profile, monitor Rails applications etc.
You may also get people who lack the understanding but have experienced the impact of a platform that didn’t scale. They’re also asking a valid question but how to answer?
Anyway - best rant on scalability I’ve ever read: (http://www.schlossnagle.org/~george/blog/index.php?/archives/29-Why-PHP-Scales-A-Cranky,-Snarky-Answer.html)
comment at 12. October 2005
Stu Charlton:
My thesis is based on the observable fact that quality tends to diminish as the quantity increases.
This is not a fact. Mass producing items with statistically negligible defects happens every day with many extraordinarily complex products: cars (particularly japanese cars), household appliances, consumer electronics, etc. When was the last time you purchased a hand-crafted refrigerator because would have higher quality?
If, however, someone asked me to prepare that exact same meal for a party of 400 people, or 4,000 (or 4 million people), I can guarantee you that the quality of the food I’d prepare would get progressively crappier the more people I try to make it for.
That would be a case of you not being “scalable”. High-end restaurants regularly serve stunning meals to hundreds of people an evening; they’ve worked out a process for doing so where the food doesn’t get crappier.
If you’re not able to see the level of utmost crappiness built into these products, you’re probably heavily biased towards Microsoft.
And you’re heavily biased against them.
I’ve posted here because I find your attitude does not support your arguments. You make very bald faced claims, use very vague analogies to back things up, arrogantly suggest that if a reader “doesn’t get it” that’s their problem, and then make obnoxious blanket statements like this one. You seriously damage your credibility by taking such stances.
comment at 15. October 2005
Gosling Didn’t Get The Memo [@lesscode.org]:
[…] New Take On Scalability […]
pingback at 12. March 2006
Thought Leadership:
Why Ruby Doesn’t Matter…
You may have noticed that pretty much everyone in the Ruby camp are insultants with many of them being book authors attempting to capitalize on hype. I of course, will remain open minded that Ruby may be better than say Java at some tasks but for the…
trackback at 13. March 2006
G Roper:
An operational definition of scalability was provided by Michael D. Kersey on an old newsgroup thread:
http://groups.google.com/group/microsoft.public.inetserver.asp.components/msg/d9846b908f678f15?hl=en&
To quote from that newsgroup post:
“IMO a reasonable definition of scalability for a given platform P and application A is
S(A,P) = R(A,P) / C(A,P)
where
R = Maximum number of requests processed per second by application A on platform P,
C = Cost of hardware and software to develop and support application A on platform P.
I’ve assumed 100% availability for the purposes of this discussion. Availability could be added as an input to the definition if desired. This term displays the expected behavior shown by common usage of the term “scalability”:
1. As throughput R increases, scalability increases,
2. As cost C increases, scalability decreases,
3. Different platforms and different software may be compared using this definition,
4. You can use this definition to estimate costs of a proposed system, given an anticipated user load.
5. Both R and C can be estimated using known techniques.
So using this definition, scalability’s dimensions would be “requests processed per second per dollar”. Given the following known values for a single application Z:
running on platform X:
R(Z) = 1000 requests/second,
C(Z) = $40,000
S(Z) = 1000 requests/second / $40,000 = 0.025
running on not-so-fast but less expensive platform Y:
R(Z) = 500 requests/second,
C(Z) = $10,000
S(Z) = 500 requests/second / $10,000 = 0.05
While platform Y’s throughput (performance) is much less than that of platform Y, Y is much more scalable than (in fact is twice as scalable as) platform X when running application Z.
This definition can also be used to estimate the utility of using various software methodologies. For example, heavy use of components or object technology may or may not change each factor in the definition: the degree to which each is changed determines whether the resultant system is more or less scalable.”
comment at 19. March 2006
/\ndrew’s Blog » Blog Archive » On JD on Rails:
[…] Come on dude. This argument is really pretty tired now and I’m surprised you would partake in such FUD mongering without actually doing some research. A simple Google yields numerous discussions - Of which New Take on Scalability is a decent one. To paraphrase: […]
pingback at 08. June 2006
Ruby on Rails Scalibility « Open Source Initiative:
[…] http://lesscode.org/2005/10/08/new-take-on-scalability/ […]
pingback at 16. October 2006