Life is full of uncertainty, and you can never know for sure when it is time to call it quits and move on to the next venture...
Some companies survive hit after hit only to end up doing very well on a project that has little to do with what they initially developed, others never recover.
The most important factor is the team. If you have a wining team, you should not hurry too much to quite, but if your team has a fundamental problem, then you need to be on your way out. An A team with a B idea is more likely to make it then a B team with an A idea.
Once you know you have a winning team, then moving on to developing new products is a matter of whether the product is working in the market or not. If you can figure out why not and fix it, go for it, but if the product has no market, leave it behind and move on to the next big idea.
If you know that you have a winning team, and the product makes sense in the market where you are trying to sell it, then it might still make sense to close the company and start over, if the company has lots of debt and is no longer serving its purpose. You can always start another company with the same people, this time with more experience then you had the last round.
One more thing from personal experience - patents - Unless you hold a patent for a chemical or a drug, it is not worth spending more then the bare minimum getting a patent, and then looking for investors to finance filing the patent in more places. If you cannot raise money for that in the 30 months that the PCT2 allows you for global filing, then it is more then likely it will never be worth it.
By the way, in what way were your failures a mistake (other then the obvious lack of success :) )