Take his advice and don't cheap out on a file cabinet.
Other than that... cheap has worked fine for me. Also makes it more likely you'll get enough stuff -- file folders, pens, etc -- as most people underestimate here.
I haven't bit the bullet yet on a labeller. Don't feel like I'm missing it.