This is not entirely true. Both Gringotts and Forbidden Journey have an alternate "castle tour" (aka Single Rider line). Unfortunately for Gringotts, this passes most of the themed queue so you will miss most of the themed elements.
However, for Forbidden Journey, in the single rider line you can see both the The Headmaster's office and the dark art's classroom. Both provide great photo ops in the single rider line. You will miss the primary portrait room, but there is a separate portrait room just for the single rider line. Both are impressive, although the main queue room has better theming.
You will also be able to enjoy the Ollivanders wand show, a brief interactive show before entering Ollivanders wand shop. The wait for this at Hogsmeade can be long since there is only one theater, but should be easy to enter in Diagon Alley since it has more theaters.
If you can handle the spinning Omnimovers in Haunted Mansion, and the drop in DL's version of Pirate's of the Caribbean, you can probably handle Gringotts. People keep calling it a roller coaster, but it is very tame. It's more like a mine train with cars that pivot to address the show scenes. Think Toy Story Mania only a little faster between scenes.
If you are a Harry Potter fan, I think it's worth it. Buy an interactive wand. Enjoy a butter beer, and explore your dark side in Knockturn Alley.