Experts have cautioned against mindlessly prescribing sedatives for dementia.
A three-year study published in The Lancet Neurology reports a doubling of the risk of early death in those on the drugs long-term.
Current guidelines state that anti-psychotics can be given to patients who are severely agitated or violent for short periods of time.
However, figures suggest the drugs are overused and are given for an average of one to two years.
As many as 100,000 people in UK care homes with dementia are routinely prescribed anti-psychotic drugs for aggressiveness or agitation.
Ministers said they were reviewing the use of the drugs in dementia care.
The latest research is not the first time the dangers of anti-psychotics in dementia patients have been reported.
After a 2005 study published in the US, the Food and Drug Administration required anti-psychotic drugs to carry its strongest "black box" warning on its labels regarding dementia patients.
The study, first reported at a conference in 2007, involved 165 patients with Alzheimer's disease living in care homes in Oxfordshire, Tyneside, London and Edinburgh.
The patients who were already taking the drugs were either continued on treatment or given a dummy pill for a year.
There was a significant increase in risk of death for patients who continued taking anti-psychotics during the course of the study.
After two years, 46% of patients treated with anti-psychotics were alive compared with 71% on the placebo.
Three years after the start of the study, fewer than a third of people on anti-psychotics were alive compared to nearly two-thirds taking the placebo.
The researchers said most periods of aggression in dementia were self-limiting and would pass. Better education of staff would take away the need for medication, they said.
Study leader Professor Clive Ballard from King's College London said the drugs were appropriate in some patients with severe aggression for short periods.
"But the serious concerns of the drugs shown by our research emphasise the urgent need to put an end to unnecessary and prolonged prescribing."
Co-author Professor Robin Jacoby, an expert in old age psychiatry at the University of Oxford said: "A large number are on medication for no good reason at all.
"It's a question of education."
He explained it was not entirely clear why the risk of death increased but one explanation could be people on the drugs being more inactive so at risk of things like chest infections and pneumonia.
Rebecca Wood, chief executive of the Alzheimer's Research Trust, said the results were a "wake-up call".
"We must avoid the use of these drugs as a potentially dangerous 'chemical cosh' to patients who would be better off without it."
The government said it was currently carrying out a review of anti-psychotic drugs and the issue would be addressed as part of the long-awaited National Dementia Strategy for England due to be published shortly, BBC reported.
Care services minister Phil Hope said the inappropriate administration of medication is "entirely unacceptable".
"Guidance to health professionals and care staff is very clear, anti-psychotic drugs should only be used when they are appropriate as part of best clinical care practice.
"But there is undoubtedly strong evidence which suggests that these drugs are being over used."
Dr Richard Perry, consultant neurologist at Imperial College Healthcare NHS Trust, said: "This work highlights the pressing need to develop and evaluate alternative pharmacological and non-pharmacological treatments for behavioural symptoms in dementia."
Dr Tim Kendall who drew up the National Institute of Health and Clinical Excellence (NICE) guidelines on dementia said they had advised the drugs were used "sparingly".
"When doctors routinely ignore the evidence in this flagrant way, as recent surveys seem to suggest, for a group of people who are disenfranchised and very dependent, it should be considered a very serious matter indeed."