Tough spot to be in. I think there are 2 things here:
1. estimating the best time and I like Oogie's formula. At least it'll give you a start and if the final figure seems low, then add an hour or two or whatever is relevant depending on the size of the job.
2. Dealing with the manager - I have been in this situation - unfortunately as the boss! But even so, unless the boss is completely unreasonable, I'd offer the estimated time but also ask to speak to them about the working relationship. I'd also have it as a separate discussion too. Often when a staff member makes an appointment with the boss it sends a warning bell to the boss. They will be at least a little more receptive to the feedback, discussion when it occurs.
From there you have to remain calm, state your position and how it makes you feel and then ask them how they see it. You want to stay factual and ask them to stay factual. Once emotion gets involved it's pretty hard to have a conversation. (again, from experience!).
Hope that helps. Not an easy situation if they are like that all the time.