Recent research suggests that children of couples having fertility problems are more likely to have serious conditions like autism and cerebral palsy.
Health problems which are making it difficult for these couples to conceive in the first place is probably what contributes to these conditions, according to scientists.
In addition fertility treatments like IVF, may also play a role in contributing to these conditions as was reported at an American Society for Reproductive Medicine meeting.
However experts stress that the overall risk was still relatively low.
They stated the need for counseling of couples about the risks and encouraging them to improve their health before undergoing fertility treatment.
According to Professor Mary Croughan, lead researcher of the University of California study on 4,000 women and their children aged up to six years, couples with fertility problems were also more likely to have other health problems, like diabetes and heart disease, as well as having a higher risk of pregnancy and labor complications.
She said: "What has caused them to be unable to conceive goes on to cause problems.
"It is as if a brick wall has stopped you becoming pregnant. Treatment allows you to climb over the wall, but it is still there and it goes on to cause problems."
The study revealed that the risk of five serious disorders - autism, cerebral palsy, mental retardation, seizures and cancer - was 2.7 times higher among the children born to 2,000 women who experienced fertility problems than among those born to the 2,000 women who did not have difficult conceiving.
The risk for autism alone was four times higher.
Other developmental problems such as learning disabilities, attention deficit hyperactivity disorder, or serious sight or hearing disorders were also 40% more common among children born to the couples who had difficulties starting a family in the first place.
However Stuart Lavery, a spokesman for the British Fertility Society, questioned the validity of these findings because of the wide range of fertility problems and treatments the women had.
He added, "There is no doubt that people who have difficulties with their fertility have difficulties conceiving and carrying pregnancies, although it has not been shown that it is the infertility that is causing the problems."
Clare Brown of Infertility Network UK proposed that further work was needed to ensure safe treatment for couples and potential children.