Go to the Whitman web site. They are now offering a Starter kit for new collectors for, not exactly sure, about $29.00
It contains the main thing a 2012
Red Book, lots of those 2x2 flips, a box for those and other starter items.
The main thing to do to start is as already noted and just go to a bank and purchase either rolls or a bag of pennies. Also, you can get rolls of Nickels since they are probably the easiest to find coins needed for a collection.
For now stay far, far, far away from
ebay and/or other web sites for coins unless you really know what your doing.
A great thing to do for any beginner is to look up coin shows in your area. Go to Google and type in Coin shows.
Also, try coin clubs since they sometimes have coin shows but are not large enough to make Google.