Ohh that's smart!Yes.
That's why you use Boardwalk or Beach. Disney has never checked magicbands to get on the bus. Not to stop DS workarounds, not to prevent non-guests from riding.
When I stay at the Swolphin, I usually hop on the first bus leaving DAK or MK for Beach, BW, or Swolphin.