Well, I had this situation three times, and though I was a dear patient soul about it for months, the other buddy always won.
Also when people I love smoke marijuana they see everything in life as fine, no problem, not to worry, and they don't hear me when I am worried, and they just drift out of my life.
But that is just me.
NOW, for you I hope for much better things!
Sounds like you really care for this man.
I just wanted to say I really sympathize, and in your place I too would be thinking "Hm...?"
Do you have really nice warm supportive people to keep you company when the guys are out on their excursions? That is a first good thing to have, no matter what you decide later.