Here's a thought. If you're lactose intolerant ( which I am) you need to be proactive in reading labels and avoiding products that contain casein or milk. Also, lactose intolerance varies from person to person. I also have IBS-C.
With that said, I take charge of my health and eating habits, as I am careful in what I consume. I simply read labels and ask if a certain food items contain any milk products. As for calcium I can tolerate certain brands of yogurt without ill effects.
Bear in mind that there are mammal milk alternatives; almond, soy, rice and coconut milk for examples.
"Most mammals cannot drink milk after infancy, including humans. Since most humans stop producing milk-digesting enzymes in their bodies by 5 years old, undigested milk Sugar ferments in the colon and causes digestive upset. According to a 2009 article in "USA Today," less than 40 percent of the world's population can digest milk after childhood. Descendants from Northern Europe and Eastern Africa have the highest probability of producing enzymes for the digestion of milk into adulthood."