NC Man, start supplementing with Udo's 3-6-9...from Udo "the best way to oil our skin is from the inside, using liquid, flowing, unrefined w3 and w6 oils in foods." Avoid meat & dairy & baked goods. Consume that olive oil you love!
I don't doubt that the fact that you have quit coffee is affecting your skin as well. When THAT situation resolves itself(and it will!), I'm sure your skin will improve. Also, I'm reading that Neem oil is good for acne...
BTW, I'm not saying "don't put oil on your skin"...I AM saying...SUPPLEMENT! :)