There are still a lot of questions to this supposed link.  For example other reports are trying to link this potential problem with the isoflavones.  So why don't the identical isoflavones form the numerous other plants we eat cause brain atrophy?

And a flaw I see in the study is that it was taken from a small sampling of people living in an isolated. What if is not the soy that caused the atrophy, but rather something the soy picked up from the soil, such as a metal toxic to the brain, or possibly something they use in their processing to make the tofu?

So how much brain damage have I suffered since I have been consuming soy since I was an infant? 

 http://iangoddard.com/soy.htm

In April 2000, Lon White and others reported a dose-dependent positive correlation between tofu consumption and brain atrophy in a large sample of men over several decades. [1] While correlation does not prove causation, study size and duration along with the robust dose-dependent relationship caused me, even as a vegetarian, to avoid tofu and other soy products.

Correlation-based hypotheses should be tested against the availability of possible causal mechanisms. In addition to possible causal mechanisms previously cited by this author, [2] recent findings significantly increase the case for a causal mechanism of soy-induced brain atrophy.

Atrophic Pharmacology Indicated

Brain-derived neurotrophic factor (BDNF) facilitates the survival and genesis of brain cells. [3,4] The neuroprotective effects of caloric restriction are attributed in part to increased BDNF. [5] On the other hand, reduced BDNF is known to cause brain-cell atrophy and is associated with Alzheimer’s disease. [6,7] Now, a study in "Neuroscience Letters" reports that soy significantly reduced BDNF in the hippocampus and cerebral cortex of male rats. [8] Since reduced BDNF can cause neural atrophy, these findings appear to provide compelling evidence for a causal mechanism that might explain the positive correlation between tofu (soy) consumption and brain atrophy reported by White et al. [1]

Bad For Boys, Good For Girls?

While soy appears to reduce BDNF in male rats, it has also been shown to increase BDNF in female rats. [9] In fact, soy appears to affect neurological parameters in a sex-defined fashion wherein females benefit and males suffer. [10-13] There is little doubt among researchers that this is because soy is high in phytoestrogens, which are plant-derived substances that act like the female hormone estrogen.

However, that sex-defined difference fails to explain the findings regarding the wives of male subjects in White et al., who reported: "A similar association of midlife tofu intake with poor late life cognitive test scores was also observed among wives of cohort members, using the husband’s answers to food frequency questions as proxy for the wife’s consumption." [1] White et al. proposed that long-term consumption of weaker soy estrogens may displace the body’s own stronger estrogen along with its benefits.

Evidence Against Soy-Dementia Hypothesis?

A possible signal contrary to a soy-dementia link is the low prevalence of dementia[14] and high consumption of soy in Okinawa, Japan. [15] However, that negative correlation, like any correlation, does not prove causation. For example, perhaps soy does cause dementia but other factors in Okinawa offset the effect.

Also, White et al. explored correlations of a range of foods to neurological parameters, whereas this Okinawa analysis is a sweeping generalization of only tofu to all of Okinawa. In other words, it stands to reason that the study by White et al. finding a positive tofu-dementia correlation has the greater likelihood of providing the more accurate picture. Nevertheless, in my view this Okinawa data warrants further examination as a possible route to falsifying the soy-dementia hypotheses.

In closing, the findings of soy-induced BDNF reduction in male rat brain regions that are central to the onset of dementia, in addition to previous findings, [2]appear to provide compelling evidence of a possible causal mechanism that might explain the soy-dementia correlation reported by White et al. [1] Obviously further research is necessary before a clear picture emerges regarding the effects of long-term soy consumption on the brain. But in the meantime, my inclination is to play it safe and avoid soy.

References

1. White et al.: "In this population, higher midlife tofu consumption was independently associated with indicators of cognitive impairment and brain atrophy in late life."
2. Goddard (scroll to): "Is There Reason to Believe Tofu May Cause Brain Atrophy?"
3. Korte M: "Neurotrophic factors have long been known to promote neuronal survival and differentiation."
4. J Neurochem (Sep 2002): "These findings suggest that BDNF plays an important role in the regulation of the basal level of neurogenesis in dentate gyrus of adult mice [...]."
5. Endocrinology (Jun 2003): "Recent studies have shown that DR [dietary restriction] stimulates the production of brain-derived neurotrophic factor (BDNF) in brain cells, which may mediate neuroprotective and neurogenic actions of DR."
6. Arch Gen Psychiatry (Jul 1997): "stress can decrease the expression of brain-derived neurotrophic factor and lead to atrophy of these same populations of stress-vulnerable hippocampal neurons."
7. Brain Res Mol Brain Res (Oct 3, 1997): "a reduction in BDNF mRNA expression has been observed in human post-mortem Alzheimer’s disease hippocampi. [...] These results support and extend previous findings that BDNF mRNA is reduced in the human Alzheimer’s disease hippocampus and temporal cortex, and suggest that loss of BDNF may contribute to the progressive atrophy of neurons in Alzheimer’s disease."
8. Neurosci Lett (Feb 27, 2003): "significant reductions were found in brain-derived neurotrophic factor (BDNF) mRNA expression in the CA3 and CA4 region of the hippocampus and in the cerebral cortex in the [male] rats fed the diet containing phytoestrogens, compared with those on the soya-free diet."
9. Neurosci Lett (Feb 1999): "soy phytoestrogens significantly increased the mRNA levels of BDNF [...in] female rats."
10. Neurotoxicol Teratol (Jan-Feb 2002): "when learning and memory parameters were examined in a radial arm maze testing visual-spatial memory (VSM), the diet treatments significantly changed the typical sexually dimorphic pattern of VSM. Specifically, adult Phyto-rich fed females outperformed Phyto-free fed females, while in males on the same diets, the opposite pattern of maze performance was observed."
11. BMC Neurosci (2001 2(1):20): "Female rats receiving lifelong exposure to a high-phytoestrogen containing diet (Phyto-600) acquired the maze faster than females fed a phytoestrogen-free diet (Phyto-free); in males the opposite diet effect was identified. [...] These findings suggest that dietary soy derived phytoestrogens can influence learning and memory and alter the expression of proteins involved in neural protection and inflammation in rats."
12. BMC Neurosis (2001 2(1):21): "When a diet change was initiated in adulthood, control phytoestrogen-rich fed females outperformed control females switched to a phytoestrogen-free diet. Whereas, in control males the opposite diet effect was identified."
13. Neurosci Lett (May 15, 2003): "This study is the first to show that lifelong consumption of dietary phytoestrogens alters the HPA stress response in male rats."
14. Dementia Rates in Okinawa vs Japan & US.
15. Soy Phytochemical Intake in Okinawa.