From
@havoc315 's description, the app uses the NY vaccine records systems to verify that you've been vaccinated from the providers data. They're not trusting what anyone just sends them. They're actively taking what someone submits and matching it to specific provider data. No match, no passport. That makes getting one of these robust and very hard to cheat. Assuming the validation app shows something that can be confirmed that it's you and not someone else's pass, validation can be robust too.
Which means this system requires the state to have a vaccine tracking system already built. And that they add an interface to it for the matching and confirmation. It's not going to be consistent from state to state, and very difficult to cross states.
I went to state site to get my shot, and they provided me proof beyond the card via a web page. So, they've clearly got the data, if not an interface to use it. But, if I had gotten the shot at a county site, or a pharmacy, I'm not sure where the data would have gone, if it ends up in that same state system with all the same identifiers or not. Or, if it would be in whatever different system, even if demographic data was shared to the state. Every state is likely different for this.