To the confused comments, the mind of a pre-schooler (the target demographic) works differently from that of an adult. There were a ton of episodes, and they tried lots of different things, and I'm sure lots of very little kids were frightened by the show now and then. Actually, I got curious and found this Youtube video of some guy's "top ten scary Sesame Street moments," and while I remember watching most of those skits as a kid and never found them scary myself, he describes decently well how a very young child could be scared.

However while Sesame Street was never scary for me, there were some segments they did in the 80s/early 90s that were, well, in the style of the 80s/early 90s, and as such they were a bit creepy. Kind of in the same way that the Chuck-E-Cheese animatronics were a bit creepy. Some of the shorts I remember were MTV-parody music videos, or there would be a puppet with disembodied legs walking around, or the segment would use liminal space in an ominous way, things like that. I'm sure if I were to watch them today I wouldn't see anything off, but there was something I found slightly unsettling about some Sesame Street clips of that era.