I feel you. I’m fine with general “hubbub” and background noise, but if I’m in the middle of a group of people talking (and I can hear their words) then I can feel as if I’m being pulled apart, it’s horrible.
Not long after I was first diagnosed, I attended a psychoeducational workshop for adults with ADHD. There were about 8 of us with 3 psychologists, with our chairs arranged around the outside of the room, facing inwards. I usually sat in a corner as it was easier for me attention wise, but one day after lunch someone else sat in my seat so I took a seat right in the middle along the back. I had to put my hand up after about 15 minutes to ask someone to swap, as I couldn’t cope with the conversation rapidly hopping from one side of the room to the other.