[Solar-talk] Framework Benchmarks

[Clay Loveless]

James,

If the performance problem you're raising a a concern about (12% as  
efficient as a hello-world one-liner) is actually significant in some  
scenario where you're paying anything close to $0.0005/request, I  
presume the servers you're running on are hand-carved from blocks of  
solid gold. :)

Look at it this way: server hardware is a commodity resource. Web  
developers and web development productivity are not.

You'll spend more money just trying to *hire* a quality team --  
forget about their salaries and benefits -- than you'll spend in a  
decade of handling web requests at 12% efficiency of "Hello World".

I'm a co-founder of a web services company called Mashery that  
processes millions of very complex requests per day through Solar,  
and I could not be happier. We do it with a team of 1-2 developers,  
and six plain-jane Amazon EC2 instances. The cost of one Amazon EC2  
box is in the neighborhood of $70-100/month.

With that set-up, we have withstood traffic originating from several  
articles about us on TechCrunch over the past year -- while handling  
our normal day-to-day millions of requests -- without breaking a sweat.

We're now in the process of ramping up to handle requests at the rate  
of thousands *per second* with Solar as the underlying core.

Additional costs to scale from millions to hundreds-of-millions per  
day? We'll throw another dozen or so EC2 boxes online. No additional  
staff.

My point: servers are commodity, code quality and developer  
productivity are not. Comparing them is not apples-to-oranges, it's  
more like apples-to-yaks.

-Clay



On Oct 3, 2007, at 5:50 AM, Kilbride, James P. wrote:

> Btw, Paul, my brain pulled a distraction on me while writing that  
> and I forgot to finish the full conceptual thought in there.
>
> Basically it is, has anybody started looking at, from an economics  
> viewpoint, at which point in scaling an application up does  
> hardware costs/maintenance costs become more expensive than the  
> developer costs? Is it possible for that to occur? Is it possible  
> that sometimes we gain productivity on the front end and pay for it  
> on the back end?
>
> Does anybody actually look at that kind of information(this  
> application cost us $100,000 to write but we've spent 2 million on  
> server costs over the last 3 years.. etc..)
>
> I wonder if there is even any concern that could be a problem?  
> Could we quantify the 'cost per request' in some way? given a  
> specific page load or such? "It costs us .0005 cents to show the  
> customer the product"
>
> I liked your supply demand curves btw in the atlanta talk. And I've  
> started bouncing back through some of the rest of the topics/posts.  
> Very good work on that and it's amazing to me how easily people  
> ignore the underlying 'truth' of the statements and ignore that you  
> start with a simple case before arguing a complex case. And yet all  
> of them want to somehow jump straight into the complex case right  
> away.. It makes no sense to me.
>
> Of course, listening to people rant and rave over the last decade  
> about Intel/AMD, NVidia/Whoever the ohter one is.. and so one with  
> benchmark after benchmark and how this benchmark shows the clear  
> winner..
>
> sigh, maybe it comes of having taught college courses in computer  
> architectures and showing how you can specialize things and how  
> benchmarks only tell a small part of the story(important but must  
> be taken in context) that it drives me nuts when people do nothing  
> but ignore the forest for the tree that is falling on top of them  
> while they argue about whether it's falling or simply swaying in  
> the wind.
>
> Now, off to test, 'ease of use' and 'responsiveness' of solar by  
> trying to get it up and running with zend core.
>
> James Kilbride
>
> From: solar-talk-bounces at lists.solarphp.com [mailto:solar-talk- 
> bounces at lists.solarphp.com] On Behalf Of Kilbride, James P.
> Sent: Wednesday, October 03, 2007 8:12 AM
> To: solar-talk at lists.solarphp.com
> Subject: [Solar-talk] Framework Benchmarks
>
> Am I reading the benchmarks correctly Paul on your atlanta talk. At  
> best(solar) a framework comes out to only be about 12% as efficient  
> as writing the PHP code myself and doing some simplistic seperation  
> of business logic and layout?
>
> Ouch.. And some of those are on the 6% order. That's some serious  
> degredation of performance. Not saying I won't use them, since you  
> get some good productivity gains but numbers like that do hurt the  
> cause a bit. If I've got to spend 8-20 times as much on hardware to  
> have the same performance programmer productivity may become  
> cheaper than hardware.
>
> James Kilbride
>
> _______________________________________________
> Solar-talk mailing list
> Solar-talk at lists.solarphp.com
> http://mailman-mail3.webfaction.com/listinfo/solar-talk

--
Killersoft.com