Fiction and poetry are alphabetical by author, with anthologies grouped together at the end. Nonfiction is grouped by broad topic (languages, business, technical reference), then by subtopic or author if necessary. Reference relevant to things I'm currently working on goes in my office, the rest in the library.
If you need a catalog, the fastest way to make one is probably by bar code scanning the books and then looking up the ISBN numbers online. Software assistance is available; whether it's worth the effort depends on how many books you have and how organized you want to be.
If you have truly huge numbers of books, you can use the Dewey or Library of Congress system directly, or use them as models for your own index. (Yes, I know people with collections this large.)