1. Welcome to the WDWMAGIC.COM Forums!
    Please take a look around, and feel free to sign up and join the community.You can use your Twitter or Facebook account to sign up, or register directly.
    Dismiss Notice

(RUMOR) Rivers Of Light to end in 2020 replaced with new show featuring Disney characters

Discussion in 'WDW Parks News, Rumors and Current Events' started by brb1006, Mar 19, 2017.

  1. lentesta

    lentesta Premium Member

    Joined:
    Jul 14, 2001
    Messages:
    938
    Likes Received:
    2,181
    Not sure I understand the "not a random sample" point. UG/TP users aren't a random sample?

    Agreed about the sample size. More is better. I can get the raw data if that would help with #3, once we have enough surveys.
     
    TeriofTerror and Mart123 like this.
  2. MansionButler84

    MansionButler84 Well-Known Member

    Joined:
    Oct 26, 2001
    Messages:
    17,449
    Likes Received:
    28,123
    I was thinking all of these things, too, but trying to be nice lol. Not everyone isn't a statistician. My understanding is that Disney's numbers are not nearly as concerning for RoL. People like it and it will do fine in a park with so many nighttime offerings. Especially come May.

    Len, it is not a random sample because people are choosing to be members and choosing to participate. I assume you aren't picking at random like Disney does. Also, it isn't a representative sample. Folks on these websites view the parks very differently than the average guest.

    Not that you guys have the options that Disney does. And Disney doesn't release its numbers, so we have to work with these.

    Are your final averages weighted? And are you using the rounded-to-the-nearest-0.5 numbers? One shouldn't round sometimes but not always.
     
    Last edited: Mar 21, 2017
    Calm Down Rover likes this.
  3. rle4lunch

    rle4lunch Well-Known Member

    Joined:
    Feb 28, 2012
    Messages:
    608
    Likes Received:
    719
    Well, it had a good run... Time to put the old girl down.. lol
     
    RobbinsDad and MisterPenguin like this.
  4. Seabasealpha1

    Seabasealpha1 Well-Known Member

    Joined:
    Aug 3, 2013
    Messages:
    817
    Likes Received:
    2,008
    Welp, that apparently didn't take long...then again, I take those two with a grain of salt...
     
  5. MCast

    MCast Member

    Joined:
    May 16, 2016
    Messages:
    81
    Likes Received:
    87
    You would think that Disney would run some focus groups for feedback on ROL (perhaps they have and I just don't know). If there's one takeaway from reviews, it's the show needs a more definitive finale with that extra "WOW Factor."
     
  6. lentesta

    lentesta Premium Member

    Joined:
    Jul 14, 2001
    Messages:
    938
    Likes Received:
    2,181
    I'm not sure I'm following here. All surveys include people who are choosing to participate, right?

    Related to membership, do you have the same objection about surveying people who have chosen to spend money visiting a Disney theme park? They seem to have already signaled a predisposition to liking Disney. They've also certainly been exposed to Disney's marketing about the attractions.

    ETA: That response may have sounded a little brusque. I'd be interested in how to design a better survey for this question, taking in to account the two points you made above. Let me know what you think a better process would be, that satisfies those points.
     
    Last edited: Mar 21, 2017
    DallasDis and TeriofTerror like this.
  7. MisterPenguin

    MisterPenguin Premium Member

    Joined:
    Feb 9, 2016
    Messages:
    1,392
    Likes Received:
    4,583
    They do indeed, well at least surveyed... people have reported being interviewed about seeing RoL and their thoughts. As far as using focus groups in that they gathered a group together for a personal, indepth feedback session, I don't know if they do that (though, I'm pretty sure they do that for their movies).

    In a way, yes. They can't force people to take a survey. Which is the beginning of bias in surveys. You hear from people who are willing to give feedback, and don't hear from people who don't want to give feedback. Usually, strong feelings about the thing being surveyed motivate people to respond, which means you're going to get responses that are generally more enthusiastic for or against the thing you're surveying. And then you miss the majority in the middle who are only mildly for or against it. One way around that is to not let people know what it is your surveying when you ask them to participate.

    And then you need to randomize your sample. If your surveyors are standing outside RoL and they get to chose who to survey, their natural human tendency may be to only approach people they regard as approachable whether they're aware they're consciously doing that. And to them, approachable means 'smiling.' And so, they only survey smiling people, which is very likely to skew the survey, because they're only surveying people who liked the show. A true random survey will use a trick like "count people coming out and approach every 17th person whether or not they "look approachable"".

    When you take a survey online like at touringplan.com, that is not a random sample of all people who have seen RoL. Those are a subset of people who use that particular website, IOW, they are likely to be Disney enthusiasts. This means they may be 'pixie dusters' who think Disney can do no wrong. Or, they may be 'Disney snobs' who are aware of every little deficiency in the Disney operation and are angry that Disney took a year to get RoL right and are judging it by impossibly high standards. Either way, they do no necessarily represent the group of people who attended RoL so far.

    Another way to think of online surveys is when a particular cable news channel with an ideological slant surveys online their viewers. Well, their viewers mostly share their same ideological bent, and then those surveys show "people overwhelming think X is bad!", when if you randomly surveyed the general population, the opposite is true.
     
  8. lentesta

    lentesta Premium Member

    Joined:
    Jul 14, 2001
    Messages:
    938
    Likes Received:
    2,181
    I appreciate the detailed response.
     
    DallasDis and MisterPenguin like this.
  9. MansionButler84

    MansionButler84 Well-Known Member

    Joined:
    Oct 26, 2001
    Messages:
    17,449
    Likes Received:
    28,123
    To add to this, while it is impossible to get an ideal sample when working with human opinion, Disney employs folks who use complex statistical models to weight individual groups to be more representative.

    Among us mere mortals, an easy way to gauge a group and ignore bias altogether is to merely report changes. Have 1000 people rate every nighttime show and compare averages. As long as it is the same people, we can ignore bias. All we want to know is what people like more or less, not the actual value of their appreciation which is far more challenging to quantity (and not terribly important for what we are considering. No one is suggesting RoL is UNPOPULAR. Just maybe not as popular as Disney's other projects.
     
    MisterPenguin likes this.
  10. Coaster Lover

    Coaster Lover Well-Known Member

    Joined:
    Mar 1, 2015
    Messages:
    578
    Likes Received:
    1,813
    You can't force people to take a survey, but you can incentivise people to take a survey. I mean, I think something as simple as "take this survey, get a free anytime/anywhere FastPass" would go a long way to getting middle of the road people to take a survey. I mean, I'll be honest to say that Disney survey's (or at least the ones I've taken in the park... where you go to City Hall and they sit you at a computer) are just too long. I mean, unless I had very strong opinions one way or another about something, even if the nicest cast member in the world came up to me again asking me to take a survey, I would likely decline unless there was something in it for me (and even something as simple [and inexpensive to Disney] as a FastPass would be enough).
     
    MisterPenguin likes this.
  11. rushtest4echo

    rushtest4echo Well-Known Member

    Joined:
    Dec 23, 2015
    Messages:
    417
    Likes Received:
    1,465
    We're discussing the intricacies of biased/tainted survey/poll methods for Rivers of Light when the nation's best media couldn't construct and conduct an unbiased/untainted survey/poll if the future of the country depended on it.

    Oh wait...
     
    David2319, DallasDis and DanaG like this.

Share This Page