Research leader Gleb Shumyatsky, an assistant professor of genetics in the School of Arts and Sciences at Rutgers, says that the gene called stathmin, or oncoprotein 18, motivates female animals to protect newborn pups and interact cautiously with unknown peers.
The researcher has revealed that the "fear gene" is highly concentrated in the amygdala, a key region of the brain that deals with fear and anxiety.
The new discovery attains significance as it may help improve scientists understanding of human anxiety, including partpartum depression and borderline personality disorders.
For their study, the researchers genetically engineered female mice to have an inactive stathmin gene.
Shumyatsky observed that the mutant mice were slow to retrieve pups placed outside the nest at corners of the cage.
However, mice with normally active stathmin were quick to bring similarly dispersed pups back to the nest.
In another experiment, knockout mice chose to rebuild nests in more vulnerable open spaces instead of in safe corners, where normal mice typically build nests.
Shumyatsky attributed the abnormal behaviour to the mouse's lack of fear for the safety of pups in her care.
Shumyatsky and postdoctoral research collaborators Guillaume Martel and Akinori Nishi conducted several experiments to rule out other reasons why the mutant mice were slow to retrieve pups, and ruled out diminished olfactory perception.
The researchers observed that both normal and mutant mice located missing pups with equal speed using their sense of smell.
Both types of mice also equally passed object perception tests, and the researchers ruled out non-fear motivation, as both types stockpiled food with the expected urgency.
In the social behaviour experiment, the mutant mice showed much less cautious behavior to other peers than did the normal mice.
"The equivalent human behavior would be if a person hugged every stranger she met. In fact, that's something that humans with amygdalar damage might do - they're very trusting," said Shumyatsky.
The researchers say that their findings add further evidence to the amygdala's role in controlling innate fears.
A research article describing the study has been published in the online edition of the journal Proceedings of the National Academy of Sciences (PNAS).