I've learned that saying you've been working on another project only brings irritation on their part. Is there a nice way to them we've got other clients to worry about as well?
It is just possible that you are not providing the client with any comfort level on your progress, and all customers reasonably want to know how things are going. I recommend scheduling regular status reports, meetings or phone conferences - schedule similar to appointments with the client, so that the client cannot claim that you are not giving them access. Do this at the frequency that works for you. And do not miss those appointments.
Now, the rest of this is assuming that you have done this but the behavior persists.
You just tell them that you cannot continue to accept calls at this frequency, and you tell them what reporting interval that you intend to keep. You may consider dumping the entire project if the client persists.
There are clients around who demand to treat freelancers and contractors like their personal servants or their full time staff. They expect you to "report in" like an employee and they expect employee like hours kept that give them complete access during all normal business hours. And they expect exclusivity like an employer.
The key problem is that they are simply not paying for this type of relationship.
These expectations are absolutely invalid for a contractor and legally isn't even allowed in the US. A contractor who allows the client to control their work schedule and clientele is theoretically at risk for reclassification as an employee. Search for "IRS twenty questions" elsewhere which explains the topic.
Note especially that many clients are simply unaware of the difference between employees and contractors, and will laugh off your push-back.
Most freelancing and consulting experts recommend to identify such clients early and not do business with them at all, and that they are "C" quality level clients that you take on only if you need the business. You can push back but generally the behavior is uncorrectable.