Over the last 10 years I've helped take a web based startup to $1bn market cap in 10 years. Being responsible for the in-house software development, sys-admins and the co-located infrastructure meant that we had to worry about everything. Sometimes that hurts.
Platform as a Service
DotCloud, Heroku and PHPFog are a few of the new breed of services known as PaaS - platform as a service. If I were starting a web-based company today, I would be seriously considering using one of these platforms so that I don't have to worry about IT infrastructure.
Why not rent dedicated servers...
You may be thinking though that by renting a dedicated server you're not worrying about the infrastructure. Perhaps not so much while a single dedicated server is enough. But if your business succeeds, you'll probably need more than one server. Then you have to start thinking about load balancers and then you'll need to start thinking about CDNs like Akamai. You'll need to worry about off-site data backup and disaster recovery. There are tons of things that can keep you up at night.
Keep your headcount focused
PaaS packages up your infrastructure so that all you need at your end are developers that push their code up to the platform. When the web-app starts groaning, you go into the control panel and increase the resources available to your app. No messing around with renting/buying another server, no worrying about load balancing algorithms and wondering why session persistence doesn't work. No need for infrastructure folks. Sorry sys-admins.
Adopting PaaS means that the first developer you hire can be just that - a developer - they don't need to know that much about web infrastructure and how to assemble lamp stacks etc. You can focus on getting a good developer; give them the tools and the PaaS and away you go. Focus your headcount on your product, not support.
Isn't PaaS risky for my new business?
PaaS is new ground. Some of these companies won't make it but I believe that the idea will succeed in the long run and they'll be here to stay. Despite the risk of your chosen PaaS provider going under, the risk to you is minimal. In that scenario all you need is to have another PaaS provider that supports the same development environment as you and if you are unhappy with your primary provider, just push your codebase to the other provider and you're up and running again.