You bring up an excellent point.
The whole process has actually been a lot better than I thought it would be and it forced me to make tough decisions. For me, and for a lot of people, I think our libraries are like our intellectual garages. We often have grand dreams for how we'd like them to be, but we never seem to getting around to dealing with them.
At this point in the process, I've thrown out (donated) a ton of books. I just will never read them and they aren't worth saving. Others, I have scanned, but I have also donated. And the last group, I scanned and rebound because I want to keep them.
So now, the vast majority of my library has been emptied and the clutter is gone. It feels great. But I will be the first to say: This is definitely a project!
Another benefit worth mentioning is that I can now scan a lot of annoying things that are worth keeping for reference purposes (like manuals, pamplets, and other extraneous paper items), but that would otherwise have to be filed in some sort of awkward way.