I collect ancient coins, which is a bit different from more "modern" coin collecting.

The most important thing is staying to your budget. Don't get anything that would set you back financially, because apart from a few specific issues, coins are not for "investing" in. It's always good to keep in mind that it's just a hobby. Try not to get caught up in the rat race of getting a coin just because it's "rare" or spending far more on the more pricey of two nearly indistinguishable options just because that one's a better grade.

If you're interested in a series, or a theme, get the lay of the land first and find out how many coins are in that set and how much it would roughly be. For example, I've heard stories of collectors who had to spent above and beyond just because of the opportunity cost of spending time gathering a collection and paying eye watering amounts to complete the set with that last missing piece they didn't know was going to be so rare and expensive going in.