My Disney Experience system status and outages watch

HoustonHorn

Premium Member
Note? Contacted WDW IT by phone - 11-5-2018, 19:30 ET...
Yup, admission that site is yet again broken. "My Plans" dead on IE, chrome...
works on android cell.

PLEASE - use https://downdetector.com/status/disneyworld to update real statistics on this joke? POST your experience, on an independent Web Server Tracking site.

Now Chrome, Chrome Incognito, MS Edge and Android are all down. Reported on downdetector.

ETA: As of 9:11 Central, Chrome working; Android app still down. Glad I printed everything out....
 
Last edited:

TheGuyThatMakesSwords

Well-Known Member
Now Chrome, Chrome Incognito, MS Edge and Android are all down. Reported on downdetector.

ETA: As of 9:11 Central, Chrome working; Android app still down. Glad I printed everything out....
Confirmed: Chrome operating 11-6-2018, about 17:00 EST...

And THANK YOU for using downdetector :). Only way we can keep WDW honest is to use an INDEPENDENT performance monitoring system.

PS - I have ZERO financial interest in downdetector.com.
 

chelbye

Member
Leaving for Disney Friday morning and cannot access most of my FPs or ADRs on MDE. My husband is also missing some of them.
Made a report on DownDetector.

As an aside, I'm not super concerned about this as I do have a screenshot of all of my reservations saved on my phone already because I've been creeping on this thread for a while. HOWEVER, we make a lot of changes to FP in the park depending on what we rope drop and length of stand by lines. Any thoughts on how to go about this if MDE is down? I assume I'll have to use the FP kiosks?
 

DisneyJoe

Well-Known Member
Confirmed: Chrome operating 11-6-2018, about 17:00 EST...

And THANK YOU for using downdetector :). Only way we can keep WDW honest is to use an INDEPENDENT performance monitoring system.

PS - I have ZERO financial interest in downdetector.com.
Except you're using it as a bug reporting system, and that doesn't get the info to MDE Tech Support, and they aren't going to take the reports from downdetector and create support tickets from them.

Call their tech support and give them the info they need to create support tickets so they can actually be tested and fixed.
 

TheGuyThatMakesSwords

Well-Known Member
Except you're using it as a bug reporting system, and that doesn't get the info to MDE Tech Support, and they aren't going to take the reports from downdetector and create support tickets from them.

Call their tech support and give them the info they need to create support tickets so they can actually be tested and fixed.
Valid.... but I would suggest BOTH :). An independent monitoring system provides historical performance data. THAT is info for the CIO: if you can't measure it, you can't manage it :).
 

marge

Member
I am at the 180 day mark and trying to book and getting error messages. Just booked about 10 reservations and not sure if they went through. Called disney but long wait time. It's after 11:30p here in australia and i'm struggling to stay awake. Hopefully they all will stick and this won't be what its like booking fastpasses!!
 

xdan0920

Think for yourselfer
I am at the 180 day mark and trying to book and getting error messages. Just booked about 10 reservations and not sure if they went through. Called disney but long wait time. It's after 11:30p here in australia and i'm struggling to stay awake. Hopefully they all will stick and this won't be what its like booking fastpasses!!
It will be worse for FP. Good luck though.
 

Radeksgrl

Member
Had planned to go to DHS today. But I wake up to the MDE App not working again. This unfortunately seems to be the new normal. Can't make or modify any FPs. It's been this way since at least 6 am, an hour and a half at this point. WDW website is not working either, can login to my account but it says it can't display any of my reservations or fastpasses.
 
Had planned to go to DHS today. But I wake up to the MDE App not working again. This unfortunately seems to be the new normal. Can't make or modify any FPs. It's been this way since at least 6 am, an hour and a half at this point. WDW website is not working either, can login to my account but it says it can't display any of my reservations or fastpasses.

I had the same problem this morning trying to make my FP for the first time. Got two and then everything crashed, including the app and the site. Took me an hour before it would work again. Incredibly frustrating.
 

Club Cooloholic

Well-Known Member
So I have a bizarre issue, and wonder if this ever happened to any one else. I cannot modify or cancel any of my ADRs. If I try I get system error. This is both online and with the app
 

DisneyJoe

Well-Known Member
So I have a bizarre issue, and wonder if this ever happened to any one else. I cannot modify or cancel any of my ADRs. If I try I get system error. This is both online and with the app
We had some issues with ADRs yesterday, we thought they were cancelled but they really weren't - so perhaps they are working on the ADR part of the system to fix it? One can only hope.
 

Club Cooloholic

Well-Known Member
We had some issues with ADRs yesterday, we thought they were cancelled but they really weren't - so perhaps they are working on the ADR part of the system to fix it? One can only hope.
It's been 3 days since I put in a ticket. On phone I was hung up on or dropped 3 times! Scary thing is tech has no idea what's happening.
 

RustySpork

Oscar Mayer Memer
Generally, this is caused by the WEB SITE, making a data request to the ancient Main Data Base computers - and timing out.

Suggestion: Keep hitting Shift, "REFRESH". BANG on that puppy, up to 20 times. What you are doing is not just "refreshing" the web page - you are forcing it to make new data requests to the ancient Main Data Base computers.

Let others know how this worked for you?

I don't know Disney's architecture, but I'm pretty sure it doesn't work like that. Web servers don't connect to database servers directly in an application of this scale, they send messages to a service bus. Services pick those messages up talk to the database, and send results back to the web application through the bus.

Even back in the old days, refreshing the browser wouldn't force a new connection to a database unless the database was completely down, and then it would only do it once per web server in the application pool. Connection pooling has been a thing for almost 20 years now.
 

TheGuyThatMakesSwords

Well-Known Member
I don't know Disney's architecture, but I'm pretty sure it doesn't work like that. Web servers don't connect to database servers directly in an application of this scale, they send messages to a service bus. Services pick those messages up talk to the database, and send results back to the web application through the bus.

Even back in the old days, refreshing the browser wouldn't force a new connection to a database unless the database was completely down, and then it would only do it once per web server in the application pool. Connection pooling has been a thing for almost 20 years now.
Try it :). Let us know how it turns out for you :).
 

RustySpork

Oscar Mayer Memer
Try it :). Let us know how it turns out for you :).

Why would I do something that I know wouldn't help? Refreshing just tells the application to send another message to the bus, it's possible they have an application cluster that's not responding and the load balancer health check hasn't pulled it from the pool yet. If refreshing works it's the luck of the draw that it's going to another cluster in the pool, but at the same time it's stacking up requests and if enough people start refreshing constantly while they're not running at full capacity it may slow everything down to a crawl.
 

RustySpork

Oscar Mayer Memer
Why would I do something that I know wouldn't help? Refreshing just tells the application to send another message to the bus, it's possible they have an application cluster that's not responding and the load balancer health check hasn't pulled it from the pool yet. If refreshing works it's the luck of the draw that it's going to another cluster in the pool, but at the same time it's stacking up requests and if enough people start refreshing constantly while they're not running at full capacity it may slow everything down to a crawl.

I hope you're happy with yourself @TheGuyThatMakesSwords, I blew the dust off of Visio so I could diagram this for you, look how great this diagram is! Keep in mind that this doesn't talk about APIs, reverse proxies, transparent forwarding, or static content caches all of which Disney is probably doing. If you're talking about the MDE app, erase web servers and write in APIs, the difference is just the protocol and authentication mechanisms.

KIC1oZs.png


Ok, so.. Whenever you go to a really big website like Disney, the page that you see in your browser has to be generated by something that looks like that picture. Yes, my diagram does look better than some server rooms.

For example..

hvxwgqk.jpg


How that works, well that's a topic for another day.

Anyway, so you've gone to the website or opened the app and now you want to go look up some information about your vacation. Great!

Let's say you want to look at your dining reservation, so you tap dining reservations.

We're going to assume you are already logged in, so MDE or your browser has a session cookie or some identifier that defines who you are.

The first thing the web application thread probably does is send a message to the bus asking about what queues are available for information about dining reservations.

Once it has a list, it picks one and sends a message to the queue of something like "Hey, send me the dining reservations for @TheGuyThatMakesSwords!, by the way I am (session id)"

The web application thread now stops work and waits for a return message from the bus.

An application server thread sees the message in the queue, reads it and it may or may not let the web application know that it's working on it.

Now it connects to the database that holds all of the dining reservations, and queries the list of dining reservations for @TheGuyThatMakesSwords.

When it has the data, it needs to create a message and send it back to the queue where the web server can find it, tagging it with with the same session id.

The web application thread wakes up when it sees the returned message, reads it, and sends you back the reservations that you have booked in the system.

Now, let's say that the application server that services that queue for whatever reason is down so it can't pick up the message, or it can't access the database, maybe it has so many requests that it can't handle any more so it's just really really slow causing the the web server thread to decide that it took too long so returns the website without the reservation.

Now let's say that you're refreshing the page 20 times a minute for the last 10 minutes so now there are potentially 200 requests in the queue for your dining reservations depending on how the session data is managed. Since the web server has a list of queues, sometimes it puts messages in a different one which is faster for whatever reason and you get results back.

Meanwhile you * 100,000 other people refreshing are causing that much more work for the backend.

Eventually those queues that were still responding fast stop responding fast.

..so what I'm trying to get at here is stop doing that please. :)
 
Last edited:

Club Cooloholic

Well-Known Member
Are they lying or can nobody make changes to their ADRs?

Thank you for getting back to me with the additional information.

At this time, dining reservations cannot be cancelled or modified online; however, you may cancel or modify by phone by calling (407) 939-1947. We apologize for any inconvenience as we work to improve your future experience.

Thank you again for sharing your observations with us.
 

DisneyJoe

Well-Known Member
Are they lying or can nobody make changes to their ADRs?

Thank you for getting back to me with the additional information.

At this time, dining reservations cannot be cancelled or modified online; however, you may cancel or modify by phone by calling (407) 939-1947. We apologize for any inconvenience as we work to improve your future experience.

Thank you again for sharing your observations with us.
Not lying
 

monothingie

Evil will always triumph, because good is dumb.
Premium Member
I hope you're happy with yourself @TheGuyThatMakesSwords, I blew the dust off of Visio so I could diagram this for you, look how great this diagram is! Keep in mind that this doesn't talk about APIs, reverse proxies, transparent forwarding, or static content caches all of which Disney is probably doing. If you're talking about the MDE app, erase web servers and write in APIs, the difference is just the protocol and authentication mechanisms.

KIC1oZs.png


Ok, so.. Whenever you go to a really big website like Disney, the page that you see in your browser has to be generated by something that looks like that picture. Yes, my diagram does look better than some server rooms.

For example..

hvxwgqk.jpg


How that works, well that's a topic for another day.

Anyway, so you've gone to the website or opened the app and now you want to go look up some information about your vacation. Great!

Let's say you want to look at your dining reservation, so you tap dining reservations.

We're going to assume you are already logged in, so MDE or your browser has a session cookie or some identifier that defines who you are.

The first thing the web application thread probably does is send a message to the bus asking about what queues are available for information about dining reservations.

Once it has a list, it picks one and sends a message to the queue of something like "Hey, send me the dining reservations for @TheGuyThatMakesSwords!, by the way I am (session id)"

The web application thread now stops work and waits for a return message from the bus.

An application server thread sees the message in the queue, reads it and it may or may not let the web application know that it's working on it.

Now it connects to the database that holds all of the dining reservations, and queries the list of dining reservations for @TheGuyThatMakesSwords.

When it has the data, it needs to create a message and send it back to the queue where the web server can find it, tagging it with with the same session id.

The web application thread wakes up when it sees the returned message, reads it, and sends you back the reservations that you have booked in the system.

Now, let's say that the application server that services that queue for whatever reason is down so it can't pick up the message, or it can't access the database, maybe it has so many requests that it can't handle any more so it's just really really slow causing the the web server thread to decide that it took too long so returns the website without the reservation.

Now let's say that you're refreshing the page 20 times a minute for the last 10 minutes so now there are potentially 200 requests in the queue for your dining reservations depending on how the session data is managed. Since the web server has a list of queues, sometimes it puts messages in a different one which is faster for whatever reason and you get results back.

Meanwhile you * 100,000 other people refreshing are causing that much more work for the backend.

Eventually those queues that were still responding fast stop responding fast.

..so what I'm trying to get at here is stop doing that please. :)
giphy.gif
 

Register on WDWMAGIC. This sidebar will go away, and you'll see fewer ads.

Back
Top Bottom