I think it is quite possible that you and your former lover are soul mates, but it was not your karma to be together, at least not now.
I do not think he is automatically a looser for having several failed marriages. He might be, but if you don't connect with the right person, it is easy to fail in a relationship. I know I have. So I don't judge people on that verbatim.
What would I do?
I would, first of all, take care of my obligations. That would be your children. Divorce is not always the worst thing that can happen to them, but your marriage does not sound abusive...just boring. I would try to fix my marriage...I don't know how old your children are, so I don't know how long you would have to "stay put" for their sake, but that would be my first choice, unless there is abuse going on. Do what you can to create a happy, safe home for them.
I would not leave my husband for this other man. If nothing else, the baggage and guilt you would bring with you could destroy the love you feel for him once the honeymoon was over. If you don't loose custody of your children, and even if you do, they might not accept him and that would bring its own problems. If I left my husband, I would do so on my own, for reasons having to do with the marriage itself, and once free and clear, let the chips fall where they may.
Remember, life is not necessarily about romance and passion. Some are lucky enough to find it for a lifetime, but most do not. Most marriages fall in the the sleepy, best friend category sooner or later. This is not to say that your yearnings are not real, but satisfying them at any cost might not be in your best interest, unless you are free to move on first.
I would, however, not lock the door on this other man. Send him a Christmas card, or a little friendly note once in awhile. The situation may change some day...you might lose your husband in another way...and if he is not married, the two of you would deserve a chance then. Life is funny that way, sometimes....
I read a true story once about a couple who fell in love and parted, only to think of each other constantly for over 20 years. They got in touch, but each was married with families, and neither wanted to go through the heartbreak of destroying their family. So they lived with honor, but they kept in touch. And then, one day, they found that they both were free, and they married and found the soul mate quality of happiness they had always longed for. It can happen.