The research involved 70 US men of varying ethnicities -- most were Caucasian, five were Asian and 15 were African-American. All were the fathers of children aged one to two.
The larger the volume of their testes, the less the men were involved in daily parenting activities like changing diapers, said the study by researchers at Emory University in Georgia.
In comparison, men with smaller testes showed more nurturing activity in the brain when shown pictures of their children, and also were more involved in their children's upbringing, according to surveys answered separately by both the fathers and their female partners.
All the men in the study were aged 21-55 and lived with the biological mothers of their children. Most were married.
"I wouldn't want to say that men with large testes are always bad fathers but our data show a tendency for them to be less involved in things like changing diapers, bathing children, preparing meals, taking them to the doctor and things like that," said lead author James Rilling, an associate professor of anthropology.
The study sought to test an evolutionary theory that holds that people and animals are either built to breed or to nurture.
The findings support the notion that human beings have a limited amount of energy to invest in reproductive efforts -- so either they put energy into producing offspring or into raising it.
"If you invest more energy in parenting you have less available for mating and vice versa," explained Rilling.
Since the testes are where sperm is made, and their size can be linked to the amount produced, the researchers said their study is unique and the first of its kind.
Previous studies have shown a link between high testosterone levels and lower parental involvement as well as divorce and infidelity. The Emory team also analyzed testosterone levels and found the same inverse relationship to parental involvement in their study.
"Other people have looked at testosterone and parental behavior but as far as we know we are the first to look at testes size and parental behavior and we think we are getting at something different," said Rilling.
"We are suggesting that men with larger testes are more built for a mating effort strategy and as a consequence are less built for investing in children."
Researchers used functional MRI scans to analyze brain activity when the men were shown pictures of their toddlers and also of strangers' children.
To assess the men's daily parenting involvement with their young children, scientists asked the men and their female partners to separately fill out questionnaires.
The volume of the testes was measured in a voluntary MRI scan, to which 55 of the 70 men agreed.
Still, the researchers could not say for sure whether testes size caused the difference in fathering behavior, or if perhaps the act of becoming a father might have caused the testes to shrink in some men.
Urologist Joseph Aluka, who was not involved in the research, said he commonly sees men with smaller testes in a certain context.
"The guy who comes in with smaller testes is more likely to have greater difficulty with getting his wife pregnant," Aluka, an assistant professor at New York University Urology Associates, told AFP.
If such men end up being more involved as parents, "maybe these guys struggled to have kids and appreciate the experience a little bit more," Aluka said.
"I wouldn't be surprised if just a few participants in this study fundamentally affected their data because it is a small study," said Aluka, describing the findings as "a stretch."
The study appeared in the Proceedings of the National Academy of Sciences.