Quoted from Our Stolen Future:

[Anthropologist Patricia] Whitten has found that exposure to plant estrogens early in life can undermine the ability of rat pups to reproduce when they grow up. […]

But the very same foods that disrupt development before birth or early in life might help prevent disease in an adult. Evidence that foods high in pant estrogens, such as soybeans, might protect against breast and prostate cancer has sparked a great deal of scientific interest and new research into plant estrogens. Numerous studies have linked estrogens, even those naturally occurring in the body, to cancer, suggesting that the greater a woman’s lifetime exposure, the greater the risk. Researchers theorize that plant estrogens might be protective because they are weaker than the natural estrogen made in the body. If they occupy estrogen receptors in the breast and displace natural estradiol, they might reduce a women’s lifetime exposure to estrogen.

The thing to keep in mind, [Claude] Hughes says, is that plants and the animals that eat them, including humans, share a long evolutionary history. Over many generations, the most sensitive individuals, those who became sterile from eating estrogenic foods, dropped out of the population. All those who were able to produce at least some offspring passed on a certain degree of resistance. This sort of evolutionary winnowing occurs because of individual differences.

Our Stolen Future, by Theo Colborn et al., pg. 79-80