Based on all the evidence,
my understanding is that supplemental Vitamin A is the real culprit. People have been taking variable amounts of (natural food based) vit A in their diets for thousands of years. As far as I know there is no evidence to suggest that naturally occuring levels of any vitamin in foods are dangerous.
Indeed Weston Price says that the native diet was superior to the modern diet because it included an abundance of fat and fat based vitamins/nutrients. Things which the modern diet is lacking.
The research is still at an early stage.
A parallel might be selenium. Mark Konlee in his "keep hope alive" group has discovered that synthetic selenium is toxic in relatively low doses (e.g. 400 micrograms). However naturally occuring food-derived selenium can be taken in high doses with no side effects whatsoever.
I suspect that the same might be true of vitamin A. That what you get in a pill isn't the same as what you get in food.