I've definitely struggled with these exact thoughts a lot in the past.
As the longer I've been vegan, and as I've gotten older too, my perspective on veganism (and other decisions I've made for the environment, such as being car-free) has shifted.
I used to be really motivated to try and save the world. Surely, by leading by example, and sharing the facts and science, I could inspire the people around me to make more eco-friendly decisions. While a couple people I know have slightly reduced their meat intake, or now opt for a bicycle instead of a car for the occasional commute; most people only offer excuses and refuse to put their money where their mouth is, values-wise.
My perspective is now somewhat egoist; I'm vegan (and car-free, and other things) for myself. It makes me feel better to not participate in systems which actively harm society, our planet, or any living beings. So I do it. I've accepted the futility of trying to keep our planet livable for our species in the face of capitalism, but it still feels better for me to be vegan.
Do you think you'd feel better if you started eating meat again?
I guess you just have to sit with that shame, shame which I'm assuming from your post you already feel because of these thoughts, and figure out what it means for you. I hope for the animals' sake that you stay vegan, but for your sake I hope you do whatever you need to have your peace.