Hrm, seems it was me. Well, kinda. My Ad Block Plus started blocking the "image.php" call on this site. So I deleted that filter and poof, everyone's avatars came back.
I have no idea why it suddenly started thinking they were ads or how it got marked as such, but problem solved.
Thanks anyway!