I love Tower of Terror - I look forward to the drops instead of fearing when they will come. What I think is more scary about it is the story, seeing the ghosts of those people and especially the child...that weird eye thing...and the way the ride isn't what you may expect it to be the first time around. (e.g. I can see how when the elevator doors open to the hallway, people think they're about to drop right then and there...and when the car starts to move forward, it is a little disconcerting!
Though I have not always been the best when it comes to jump scares, so as a...teenager...I used to cling onto my fellow rider with legitimate fear during the graveyard scene of Haunted Mansion. All grown up now though.
Phantom Manor totally creeped me out, in a fun way. That thing is dark.
Haunted Mansion was scary when I was a kid. Seems like those pop-up heads were more frequent and used to scream loudly. Could have been my active imagination, not sure.