My 39 month old DD has been there 4 times already and on her way back in 6 weeks for the 5th trip. She was there at 10 months. Even though she doesn't remember the "trip" - we have pictures and videos that we watch daily sometimes. She will now bring up things that happened on that first trip like she remembers.
As someone mentioned, take it at his pace. Naps and feeding times will vary from his regular - but it's a great time to rest your own body. My daughters favorite rest area still to this day is the train around the magic kingdom. We call it the "sleepy train" as I can always guarantee a nap before the end of the 2nd go round. It just relaxes her to no end. It's great.
Do it and enjoy it! It's awesome to see it through your own child's eyes. But don't stop at one trip either! Each trip is unique. This year will be my DD's 2nd Christmas Party, and she is so excited that she's got DH and I ready to go now!