It's hard to imagine how your body's absorbing a small bit of oil could cause such a big reaction - but its always possible.
I remember suggesting Goji berries to my neighbor who was having issues with an overacting immune system. She immediately developed horrible rashes all over her face ad neck.
The only thing I could imagine more benign than Goji berries might be water. But I know people that even have a problem with drinking water. So you never know. Stop the OP for awhile just to make sure that's the cause of your rashes.
If you decide to start again, I'd try a different milder oil like maybe flaxseed.