What you have done is not like spitting on the sidewalk or something. Your emotional comfort is not the issue; simplistic self-forgiveness is not looking at the reality of the whole situation. The sinned-against may "forgive" in some childish sense of the word - "I hereby forgive you" ["but I'll never forget" is what they are really saying to themselves].
The reality is that only God can pardon and from my observations he doesn't do that until he sees you attempting to make balance, i.e., suffer, also. Anybody remember that scene in the Godfather Part III, where Michael Corleone makes his confession to the Cardinal. He was in a state of great anguish and the wise Cardinal said to him, "It is right that you suffer."
Of course, maybe I'm all wrong. This is just my opinion.