Performance matters... wow. I know I'm going to be surprised by the variety of responses here. A regular eCommerce site benefits hugely from rapid page loads and response times, with visitors exiting the site much more readily when the site is slow. For an SaaS provider, I would have thought that performance is even more of an issue. When I select SaaS solutions for my business performance is one of the biggest considerations - a brilliant system that does everything you want, but is just a bit slow, is almost good enough reason to not go with that provider. Just trying the sales-end of your website isn't enough to judge performance - the SaaS solution is much more likely to be database-driven and serving more data - so page load times are going to be more of an issue. When I select an SaaS solution for the business which may have, lets say, 100 users, I have to live with the consequences of that site being slow and unwieldy: potentially I'll have 100 people complaining at me. Site speed matters. @Ross also mentioned the potential impact on SEO - as last year Google introduced site speed into their ranking algorithm. So faster website may mean better rankings as well. Marvellous.
Measure the performance from an end-users point of view... For our eCommerce website we use a product from Gomez Inc called Gomez Actual Experience which gives us the page response times and load times for every visit to key pages on my website. These numbers differ from regular backbone monitoring solutions in that you get data directly from the end-users visiting your site. If it took 1.24s for the page to load in front of them, that is the number that is reported. The data is segmentable by region, browser type, etc. and is a great way of monitoring how well you are serving your customers. For SaaS, I would think this type of monitoring is key to understanding whether your customers are getting the page response times you are expecting them to. Edit: Since posting this answer Google has released page load tracking in Google Analytics. Fantastic. Now you will be able to segment your page load times by any of the variables in Google Analytics and really get a good idea of how page load times affect the user experience. Official Google blog post here.
Global location still matters if you can afford CDN... Akamai, Limelight and other CDNs make the physical location of your host matter a little less. However, if your website or SaaS solution is highly dynamic (probably) and database dependent then there is reduced scope for caching which is the CDN's primary mode of delivering content faster. Yes they have solutions that help get around that, but if you are serving a page that needs real-time data from a database from one side of the world to the other side of the world then the CDN is not going to magically eliminate the latency.
I'd stay where my customers are... Unless there are some really compelling reasons to move other than cost, I'd probably keep my servers where the majority of my customers are. Perhaps you are able to segment your offering so that your European customers can use a datacenter in the Netherlands and your US customers can still use the US. The only issue with that is global companies that have offices in the US and Europe, and want to use a single connected system.
If you do move, test thoroughly... Hosting centers will usually let you have access to a machine on their network to do some latency tests - and you can use 'Looking Glass' services from other ISPs located near your customers to test what the latency is like from ISP to datacenter. Furthermore I'd be keen to test the SaaS solution in situ and test the performance from your customer location. Spend the extra money on the testing from the Netherlands before cancelling your contract with your main hosting center; the last thing you want is to have migrated your servers only to find that your customers are now complaining that their SaaS solution is now running slower, and threatening to leave unless it speeds up.
Hope that helps.