I've found the same issues with adrenal fatigue. Having felt I had it months ago before my official "crash", I started trying to take b vitamins, rodeola rosea, large amounts of vitamin C, etc. and found that the only thing I could tolerate was the vitamin C (as common ascorbic acid powder) and that was having almost no effect on my exhaustion. The b vitamins ramped me up, but there was no energy to draw from, so I felt more anxiety-ridden and would get terrible panic attacks when I would take any adaptogen or b vitamin supplement. At the time I was a vegetarian and had blood-work drawn showing that I was low in HDL and cholesterol--so since modifying my diet (about a week and 1/2 ago) I have felt some improvement, but am presently still unable to take anything other than vitamin C and magnesium supplements (which I recently added to help with the fatigue and calm my nerves, panic attacks and heart palpitations). If there is any logic in this, I've found that many b vitamins lower cholesterol--cholesterol is required to make hormones, so if you're low in hormones (e.g. cortisol, dhea, etc.), then many of these vitamins may not reach the intended purpose as your body is already at a deficit.
I feel as you do--that there is something in our bodies that isn't allowing for the proper processing of these supplements. Anything stimulating (like coffee, ginseng, etc.) is too strong for me and has been for some time. It may be due to mineral deficiencies (magnesium, zinc, etc.), the way the liver and the rest of the body processes (or doesn't process) our nutrients (malabsorption), low cholesterol, poor diet (low fat vegetarian diets), etc. As for now, I'm taking the perspective that I can hopefully improve on diet alone. I hear many people who do--others sometimes require prescriptions and supplements in the advanced stages of AF.
Best to you and your healing efforts!