The most severe side effects they observed in the Pfizer and Moderna trials were anaphylaxis, and a little over half of the subjects who experienced this reaction had a previous history of a similar reaction to something (could have been a medication, food, bee sting, etc.). So initially, one of the precautions for vaccine recipients was to wait in the observation area for a little longer if they had a prior history of anaphylaxis (people with a history of reaction to a known vaccine component, like polyethylene glycol, are advised not to receive the injection at all). However, even with the prior history, provided it isn't a reaction to one of the components, the risk of this complication from the vaccines is still exceptionally low. And although anaphylaxis is scary, it's also relatively easy to treat.
One final thing... most people with a reported reaction to penicillin aren't actually allergic to it. The rash that children frequently get after treatment of strep throat with a penicillin usually results from a non-allergic immune reaction to the cell contents of the dead bacteria, not an allergic reaction to the medication. Now, if there was throat swelling, and shortness of breath... that probably was a real allergy to penicillin.