I started drinking raw milk about 12 years ago after learning about the primal diet. Worked really well for a long time, then at some point I must have regenerated my enzyme systems and it didn't pack the same punch.
Here is an interesting bit of information concerning pasteurized milk. It looks to me that the problem is too little heat. Raw milk has its obvious benefits, but after studying ayurvedic medicine for some time where milk is a big part of their tradition, I tried boiling pasteurized milk really well for a few minutes. Successful results -- I had no reactions to it at all. The theory is that pasteurization only results in a partial breakdown of the milk proteins which is enough to denature them but not enough to make it digestible, so you get the worst of both worlds.
Also be sure to distinguish between homogenization and pasteurization, whereby the former might be even worse than the latter.