You need certain vitamins and minerals in order for your body to control the candida growth. Diet is good for controlling the numbers, and for improving digestion - but you want to stop the candida from being pathogenic, otherwise it'll always come back.
1. A good idea may be to /initially/ control the candida numbers with diet, antifungals, & probiotics (separate the antifungals and probiotics appropriately) - then when candida numbers are down, start taking the vitamins. Your digestion will have improved somewhat, and you will be more likely to benefit from the nutrition.
2. When killing off candida, it is important to control the toxic load, i.e. with cheap zeolite powder, activated charcoal, Epsom Salt baths, vitamin C - as the endotoxin release from dead candida cells is what helps keep your immune system impaired.)
3. The necessary nutrients include the B group vitamins and particularly biotin - which good amounts have been shown to prevent normal candida (yeast form) from morphing into the pathological form (elongated, hyphae, etc). Also pantethine, molydbenum, zinc, and iron have uses as well. Do a search each of these items (e.g. "pantethine and candida") and you will learn a lot. I personally also use Concentrace, so assure that I'm getting enough of everything in order to avoid any major imbalances.
Personally I don't think the "yeast foods" are that much of an issue to avoid, considering that many people have gotten over candida/CFS using nutritional yeast.
You can easily buy vitamins without any additives (e.g. Pure Encapsulations are a great brand), but of course B vitamins are usually made from yeast. I've never looked into alternatives because I don't see the need.
(In fact you should buy vitamins without additives, as lubricants such as "magnesium stearate" are only designed to increase the volume of vitamins produced - but significantly reduce the bioavailability of the vitamins.)