The only thing that scares me about the drop(s) in Pirates is potentially getting wet. Usually you don't from such a tiny drop, but I took a trip a few years ago only to find they had apparently switched some jets on at the bottom (apparently they were usually disabled prior to that). Somehow managed to get soaked despite the mildness of the drop...
I'm usually absolutely terrified of falling. But I don't mind the POTC-intensity drops whatsoever. Splash Mountain is more intense, but still perfectly manageable now that i've gotten used to it. Tower of Terror and similar types of drops are where I really have to FORCE myself to ride (and can only rarely build up the courage to do so). I'll only ride ToT to experience the amazing attraction around the terror part.