It'd probably take a half-hour at a steady clip. Depending on the weather, you might be a giant sweat-ball before you get there.
There are trails from Wilderness Lodge (and the Villas) to Fort Wilderness. One is through a wooded area, one is along the water. There is also an internal bus route twixt the two resorts, and, as you mentioned, the ferry; just make sure the ferry you get on is going to Fort Wilderness next, because they may go to MK first and then all the way to FW, which would make your trip much longer. The bus would be your fastest option to get there (though less scenic). That bus drops you off near Pioneer Hall, and other internal buses there would take you to the campfire. If you wanted to walk some of it, I understand, but I think trying to walk the whole thing might possible be draining to some people. But if you and everyone in your party is healthy and up for the walk, go for it.