I very much relate to everything you've said. I too frequently forget that I'm not a child anymore and that I too can leave. Though I'm getting better at it. I also find my tolerance for discomfort waning as I age, but I think it's a good thing. I'm more comfortable being me, even if it means I don't fit in. Personally, I think it's one thing to force yourself into a neurotypical space because you have to earn money or you wish to be supportive of a person you care about, but for an activity that should be fun, like choir, it's really not worth it if it isn't fun anymore.
Two things that help me, maybe you'll find useful, I've gotten good at asking people questions about themselves in small talk situations so I don't say the wrong thing. And for boredom, I love figet spinners. I also find people to be fairly tolerant of activities like crochet or cross stitch in public settings.