Speech-to-text or speech recognition can be a generative (transformer) model architecture, but there are a lot of models that are not. The transformer ones seem to be better in quality.
Non-transformer examples:
Coqui STT https://stt.readthedocs.io/en/latest/
Kaldi https://kaldi-asr.org/models.html
Creating subtitles / captions requires some more programming to create the text file with the correct timings.