Don't panic. A few fleas could easily be random chance. You don't really have to go to a vet, but in this case I would just for peace of mind for the humans.

Do buy a flea comb (if you do not already have one) and begin combing every night. Espcially around the neck and head. Don't worry about squishing the fleas, just mix up some mild soapy water in a glass and drop them in that.

Keep count every night and you will be able to tell if the problem is under control or not.

Do vacuum and clean house.

You may want to invest in a flea trap. These combine a light and a sticky pad and can be placed on the floor to attract any loose fleas. Move it around to likely places and it will both help trap fleas and give you an idea of how many might be in the house (if any.)

If you do not go to a vet, do buy some over the counter flea treatment. If regular combing is not enough, it will help, but I would not use it for just a few fleas.

My cat has had fleas a few times, but thankfully I've only needed the ointment once. Mist of the time combing was enough.