He didn't seem irritated at all, actually. Also, if he held bad blood towards Ventus, he would have treated Roxas quite a bit worse than he did and certainly would not have made it clear that he cares about Roxas' future. :\
Unfortunately, I don't think we're gonna get to see the scarring...