Scaling up vs. Scaling out

July 9, 2009 · Posted in SQL Server 

Jeff Atwood posted about the hidden costs of scaling up vs. scaling out. It is a good a post. In a nutshell, here are his points:

  • Scaling out gets you about 10 times the hardware for the same price.
  • Scaling out costs roughly 8 times more in software licensing.
  • Scaling out my only make financial sense using open source software(or when you can no longer scale up).

Those are good points. However, there are more downsides to scaling out.

  • Administrative overhead of more servers and the replication\load balancing methodology.
  • Scaling out with read only server is a misnomer. The read only server still has to take all the writes one way or another.
  • App dev time. All of the scale out setups I have seen require a good amount of dev work. Some more than others.

It is my opinion that the app be should tuned first, then upgrade the hardware and then tuned some more. After that, look to offload work to the app\web tier with stuff like caching. By then, newer hardware will be available for a tech refresh. Scale out as a last resort. Sound like a pain? Try scaling out.

You see funny problems start popping up when you scale out. Take a read only scale out using transactional replication in SQL Server or Master\Slave in MySQL for example. In this scenario, you can lose read consistency. Lets say you get the product list from the read server, place your order which writes to the write server but in that time that product sold out. It is like NOLOCK but worse because the latency is longer. Sometimes much longer. 

There are diminishing returns and resource splitting with scaling out. The diminishing returns is usually due to the replication overhead while reading from the write server and writing to the read servers. On top of that, now you N times the data with the number of disks\CPU\memory divided by N. You could get around this through sharding but then you are talking major app dev time.

Still want to scale out? Ok, ok, I will talk about good not bad ways to do it in the next post.

This content is published under the Attribution-Share Alike 3.0 Unported license.

Tags: , , ,

Comments

  • The licensing numbers are flawed, at least with regard to Microsoft products that are A) More expensive per core for versions that support more cores (in a scale-up scenario) and B) Are licensed by the core. In other words, 10 servers running SQL Server 2008 Standard Edition on 1 4-core processor each is $50k in licensing fees for just SQL Server. 1 server running SQL Server 2008 Enterprise Edition on 10 4-core processors is $250,000 in licensing fees for SQL Server.

    Agreed on the app dev cost, which is much higher for scale out. Admin, maybe and maybe not. Depends on how well the app devs have designed the scale out methodology. In the best-case scenario it will be almost plug-n-play and admin scripts will just have to learn of the existence of the new server. In the worst-case scenario, managing each new scale-out server will 100% as expensive as managing a single scale-up server. The reality for most companies is probably in the 50% range. The really good shops probably get that down to 15%.
  • I'm curious what your experiences tell you about the power/heat implications of scaling up versus out?
  • statisticsio
    I haven't directly compared power and heat but the 785's mentioned in Jeff's post have 6 power supplies and are 7u. I doubt you would put more than two in a cabinet.
  • It seems to me that until you squeeze out optimal performance from the existing hardware (as you note as the first line of defense), scaling out could have multiples greater power/heat costs than scaling up.

    It's all good and fine to get 10x the hardware for the dollar up front, but that savings can be eroded quite quickly if all you have is a farm of wasteful boxes drin(k)ing power and punishing your A/C. Which is exactly what happens if you aren't first focused on optimizing things.
  • AK
    If I no longer need an application, then all the time spent optimizing it is a loss.
    If I also might possibly want to roll out a system based on Vertica on Linux, then all the money spent on SQL Server licenses is also a loss. So it some cases the best choice is to invest in hardware.
blog comments powered by Disqus