As a teacher at a middle school, I run across this situation too. If you're walking around calmly during a planning period, people start to talk that you don't have enough to do and maybe you don't need a planning period. It's almost like they would rather see you running around like crazy, swearing at the copy machine making 100 copies at the last minute for your next class...

Fortunately, for me, poeple started to see that it was more a matter of organization. I was lucky enough to have people ask me how is it that all my stuff gets done on time, and I'm rarely in a panic about getting things done. I just tell them about how I've organized myself with this great time management system and it really works well for me. (If they want details, then I'll tell them, if not then I just stop there.) I've developed a reputation as a well organized person and someone who can get things done.

But, that first perception can be misleading sometimes.