I would steer clear of the busses, and the Branch Ave area isn't the best area (in fact, most of SE DC is a place to avoid, in fact, there was a shooting there today, about a block from the Anacostia station). That being said, as rsoxguy mentioned, just keep your head down and you'll be fine, just like most mass transit in any city.
The stations / cars of the Metro, in terms of cleanliness and comfort, are a crap shoot. One time I rode it and the A/C in half the cars were broken. Add to that record setting heat waves that trip and it was unbearable.
Coming from National Harbor, you are already going to have quite a haul to get to Branch Ave. If I were you, I'd hop over the 95 bridge and get off on Richmond Hwy Southbound, then it's an easy right turn onto Huntington Ave. From there you can take the Yellow Line into the city (which is nicer / safer than the Green Line, which can get a bit sketchy in places.). Anyhow, Huntingon Station is much closer to where you are than Branch Ave.