Talent assessment - knowing what you don't know?

I agree with Angus. Hires are expensive. If it is not your area of expertise and you haven't had good luck in the past, why not get help?

I interviewed Dan Ariely, author of Predictably Irrational, and he said something that really struck home. Business owners tend to assume that they are doing an okay job at things outside their core competencies - without actually checking to see if that assumption is correct. Don't just hire someone to hire someone and hope it works, be analytically and actually measure the results. Tie compensation to results rather than hopes!

To me that is the key diffence between a new business and one moving to the second and third stages of growth: using and measuring key performance metrics to evaluate how the company is doing. Set realistic (or "smart") goals for the new hires and base your compensation on meeting those goals. And definitely check references and portfolios before you make the final offer!

I mention this because if you have a good system to measure the results for a hire outside of your area of expertise (sales, marketing etc.) then you are well positioned to get and keep the right people. Plus, if you can show someone a good system, then it makes the idea of working for your company look more attractive. (When I interviewed an HR expert on best practices for hiring he pointed out that if you make it seem like a challenge to get hired then the hire will appreciate his/her job more and is likely to work harder.)

The responses above are great: Use references. Rely on trusted advisors. Use metrics to evaluate your decision after the fact.

I'd add one thing: no matter what the specialty or expertise of the person you're interviewing, the most important quality to look for is integrity. I've directly hired dozens of people in my career, and, no surprise, some of them weren't a good fit. I've never regretted hiring somebody with integrity, even for the wrong position. But an employee lacking in integrity always hurts me, no matter what their skill set.

Of course, identifying integrity in an interview (even three interviews -- good thought there) is tricky. I like to use scenarios: I'll ask, for example, "one of your employees is showing up for work late, especially on Mondays. Though normally an excellent performer, this employee's work is also suffering, as is their attitude. What steps do you take?" I look for an answer that shows the person really cares for the employee, as well as for the needs of the company.

Bottom line: take all the good advice already offered here, but make sure you're also -- even primarily -- hiring for integrity.

Give yourself three chances (interviews) in which you can meet that person, ask them detailed questions related to your desired business outcomes and how they will achieve them, and take good notes.

Research their answers and really look at their resumes. Personally, I've never even considered doing this, but I've heard of people getting a pretty good job based on a job reference provided by their cousin, whom they never even worked for.

If you don't understand something on their resume, make notes and ask them in one of the interviews. PR, marketing and salespeople can have certifications and portfolios as well, so I'd request that they show me a presentation on Slideshare or PowerPoint, or bring in a portfolio. They have to show and prove, or they're not a very good salesperson, are they?

If you're hiring someone for a job where you are not personally a good judge of those skills, this is a great example of an area where an advisor or board member can help. For example, if you need to hire a sales guy, and you don't know how, get a kick ass sales guy on your advisory board and ask him or her to help you interview the candidates and design some screening questions.

Carefully.

Call references.

Make escalating commitments instead of jumping in head first, e.g. if it's a PR person ask them work as a contractor for a while. If it's a cofounder work together for three months before you decide whether or not it's working out.

Make sure you're communicating clearly about expectations. E.g. if it's a recruiter, how fast will they be able to find great people for you? Be wary of people who seem to be over promising.

It's a great question... not so sure how you'll like this answer but:

You can't.

You know a non-technical person could never evaluate whether someone is a stellar programmer right? Because some people can talk but not walk.

Well that's true in other professions. All you can go on is gut and personality, which is OK but as you say, not great.