The paper analysed large UK studies, which compared childhood intelligence with political views in adulthood across more than 15,000 people.
The authors claim that people with low intelligence gravitate towards right-wing views because they make them feel safe.
Crucially, people's educational level is not what determines whether they are racist or not, it's innate intelligence, according to the academics.
Social status also appears to play no part in determining an individual's intelligence.
The study claims that right-wing ideology forms a "pathway" for people with low reasoning ability to become prejudiced against groups such as other races and gay people.
"Cognitive abilities are critical in forming impressions of other people and in being open minded," the Daily mail quoted the researchers as saying.
"Individuals with lower cognitive abilities may gravitate towards more socially conservative right-wing ideologies that maintain the status quo.
"It provides a sense of order," they said.
The study used information from two UK studies from 1958 and 1970, where several thousand children were assessed for intelligence at age 10 and 11, and then asked political questions aged 33.
The 1958 National Child Development involved 4,267 men and 4,537 women born in 1958.
The British Cohort Study involved 3,412 men and 3,658 women born in 1970.
In adulthood, the children were asked whether they agreed with statements such as, "I wouldn't mind working with people from other races," and "I wouldn't mind if a family of a different race moved next door."
They were also asked whether they agreed with statements about typically right-wing and socially conservative politics such as, "give law breakers stiffer sentences," and "schools should teach children to obey authority."
The researchers also compared their results against a 1986 American study, which included tests of cognitive ability and questions assessing prejudice against homosexuals.
The authors claim that there is a strong correlation between low intelligence both as a child and an adult, and right-wing politics.
They also claim that conservative politics is part of a complex relationship that leads people to become prejudices.
"Conservative ideology represents a critical pathway through which childhood intelligence predicts racism in adulthood," the paper said.
"In psychological terms, the relation between intelligence and prejudice may stem from the propensity of individuals with lower cognitive ability to endorse more right wing conservative ideologies because such ideologies offer a psychological sense of stability and order.
"Clearly, however, all socially conservative people are not prejudiced, and all prejudiced persons are not conservative," it added.
The study has been published in Psychological Science.