The condition, called aneuploidy, is an abnormal number of chromosomes.
The researchers found that if a mother's egg cell has a mutation in just one copy of Bub1 gene, then she is less likely to have offspring that survive to birth.
Dr. Sundar Venkatachalam was originally studying the gene for a possible connection to colon cancer, when he found his lab mice showed strange fertility characteristics.
"Where you would normally expect a female to have eight to 10 pups, there were only one or two pups that survived to term in the litters of females that had one copy of Bub1. So this was unusual when we were looking for cancer effects, especially in this group of females," said Venkatachalam.
Ordinarily, both copies of a gene in a chromosome must carry the same mutation in order for an organism to be adversely effected, but the drastic effects of a single mutation were unexpected.
The researchers also found that the harmful effects of this mutation increased with a mother's age.
As the female mice got older, there was eventually a complete loss of their ability to support a full-term pregnancy that lined up with an increase in aneuploidy.
The same is true in humans-the chance of having an aneuploid pregnancy increases with the age of the mother.
Bub1 plays a role in a checkpoint that ensures that chromosomes are properly divided during meiosis, the cell division process that enables a stem cell to become an egg.
This checkpoint halts when Bub1 is mutated, sometimes producing an egg with an extra chromosome or an egg with a missing chromosome.
The team linked the issue to females by mating both a male with one bad copy of the gene with a normal female and a female with a bad copy of the gene with a normal male.
When the female carried the bad copy, there were fewer births.
Further research revealed this is because aneuploidy was generated in the egg and passed on to the single-cell zygote that forms when a sperm fertilizes an egg, which leads to the loss of the embryo.
"This work certainly points to Bub1 having a role in maternal age-induced fertility issues," said Venkatachalam.
The findings appear in the online early edition of the Proceedings of the National Academy of Sciences.