Stealing time? I think you're over-exageratting here. And to counteract that, an argument could be made that by not using the fastpass on the day that it was intended for one other guest was able to ride sooner, therefore the "stolen time" from the next day was made up by the guest who was allowed on the ride faster due to the fastpass going unused.
Disney doesn't care if a fastpass is used on another day; hell, in all honesty, from a business perspective, it's better for them..because it ensures that a guest will make a return vist, spending more on admission, food, drinks, souvenirs..etc.
If you truely want to talk about stealing time, lets talk about celebrities or d-list celebrities (which are the majority of those who go to Disney World using a guest relations host as their tour guide) who skip lines their whole day during the visit to the parks. Until that problem is solved, I'd say nobody can shake their finger as anybody who uses a fastpass a day later...they at least earned that fastpass..d-list celebrities like Alfonso Ribiero (using him as an example because I saw him at the parks once..lol) did not earn their right to ditch everybody and "Steal" people's time.