I've stayed there a number of times (mostly before becoming a DVC member, only a 1 night stay since then) but I've enjoyed every stay.
Swan and Dolphin are indeed lovely hotels, and the beds supposedly beat every kind of bed in every other Disney resort by a country mile. But as you noticed, there is a resort fee (PLUS a parking fee if you're driving). You don't get access to Magical Express. And the resorts don't feel as "Disney-ish" as others. For some people that lack of theming is more of a plus than a minus; some folks don't want to be drowning in theme the entire trip. Plus, you're only a short walk to three "official Disney resorts" - Yacht Club Beach Club and Boardwalk so you can make reservations in any of their restaurants and get a snootful of disney magic if the mood strikes. You will also be right near the Boardwalk so if you and your mom wanted to enjoy what little nightlife there is - the dueling piano bar, the club that's always been empty any time I checked it out - it's an option.
The pools at AoA are amazing, but the one night I stayed with my wife and kids in a Little Mermaid room was IMO more trouble than it was worth. The food court was cacophonous and the beds are so small. Were we staying in a suite, using the kitchenette a little more, definite maybe. But I wouldn't want to stay in a standard room in any of the Values again, with very few exceptions. As it happens, I'm taking a very short trip with some family, my niece and her husband are letting me stay with them, but on the night of their anniversary I'm clearing out and going to All Star Music so they can have some privacy, but it's literally just that night, I'm just there to shower and sleep and I'm out the door early the next morning. I know other people love the Values, can't imagine why anyone would spend more than they have to on a Disney trip. But they're not my cup of tea. Unless you're thinking about splurging on an AoA suite (and it doesn't sounds like you are), I'd go moderate, not a second's thought.