Optimizely (cheap), Visual Website Optimizer (free trial), and Google Website Optimizer (free) will all let you do A/B testing if you want to see how different versions of a particular page element (or even totally different page layouts) affect conversion.
Outside of A/B testing, digging into your analytics can provide many insights as to why people are leaving your site. Three things I'd look at:
- Based on the referring site link or search keywords, what is the visitor looking for? Is the page they're landing on providing them that?
- If visitors are staying at least long enough to view a few pages, which pages are causing them to leave (top exit pages) -- is there something on these pages causing them to leave / lose interest? Are the pages calling for an action too difficult / too confusing? Are they simply reaching a dead end where there's not a clear next action?
- How many visitors are reaching the signup page at all? How long are they spending on it? If they're leaving immediately, your signup is too difficult. If they're leaving after a bit, they're reaching a point of difficulty or something's triggering hesitance, and you need to figure it out. Try Rejoiner to see which forms they're filling out. The ones they're not are likely causing the difficulty.
Also, try alternative analytics tools like CrazyEgg, which can provide click heatmaps and scroll heatmaps -- are people not seeing the signup prompt? Are people clicking on a logo or image that doesn't actually link to anything?
Finally, learn from others. ABtests.com has many conversion case-studies. As always, your industry / audience has unique considerations, so use these to get ideas and improve what you're initially testing, but make sure you are doing your own testing.
2012 Update: Another tool I've started using is Mixpanel (free up to a certain amount of data per month). They're event tracking analytics; the main way to use this for increasing conversions is through their funnels feature. By defining a series of events (more detailed than page views (e.g. form field clicks, progress through a series of lightbox/modal windows)), you could see exactly which step is where most people are falling out of the funnel.
