Find out what the registration and licensing requirements are in your state. You'll also need to get Federal and State Tax IDs and sales tax license.
If you'll be taking credit cards for payment you'll need a merchant's account. There may be some local permitting required too.
Your local or state Better Business Bureau should be able to point you in the right direction.

Going to do business online too? See what guidance is available on eBay