I agree with Julia. Having the inbox/outbox on my desk is distracting, whether full or empty. I've tried putting it on a side table/cabinet, but since it was out of site it was also out of mind and I never used it.
One thing that's working well for me is a hanging inbox outside my door (similar to what doctors offices use). This allows people to put papers in my inbox without having to come into my office. I also see it every time I come back to the office.
Regarding outgoing material, I just pile it all on the corner of my desk and distribute every time I leave my office.