Sam aged, Frodo did not. I recall several mentions throughout the books that Frodo was considered unusually young-looking for a 50-year old hobbit, just as Bilbo himself was very young-looking for 111. Owning the ring seems to basically just pause your aging, even if you're not using it.

Sam was also twelve years younger than Frodo, but since Frodo stopped aging for 17 years, he wound up effectively older, physically speaking.