After nearly 30 years, 25 million deaths and billions of dollars spent with no vaccine to show for it, scientists at this year's international AIDS Vaccine Conference said they were turning to novel approaches to overcome their defeats.
"We are in the middle of quite a profound shift of mindset in the research community," said Alan Bernstein, director of the Global HIV Vaccine Enterprise, an alliance of organisations working on a vaccine.
Speaking to AFP on the sidelines of the meeting that ended Friday in Cape Town, he said that startling setbacks had forced scientists to delve into sophisticated new research to tackle the disease.
Last year scientists were forced to abandon two advanced clinical trials of a vaccine by pharmaceutical company Merck, after it appeared to actually heighten the risk of AIDS infection.
"The Merck result was such a surprise and everyone was kind of knocked off their horses... What happened no one could have predicted," Mitchell Warren of the AIDS Vaccine Advocacy Coalition said.
"They still don't understand exactly what happened. That finding forces people to realign and look at new ways and new approaches to how we are going to find an AIDS vaccine because it was so surprising."
About 30 other clinical trials are underway around the world, but the most watched is a study in Thailand that began in 2003, with results expected next year.
That study is the biggest ever, with 16,000 people enrolled. Scientists say that whatever the outcome, it will provide valuable information on the pandemic, which most agree won't see a vaccine for decades.
Meanwhile, Warren says the failure of the Merck trial has already forced scientists to rethink their basic assumptions about how vaccines work.
"People are really grappling with new ways of doing things," he said.
In the past vaccines have either caused the body to develop antibodies that kill a disease, or to attack infected cells to kill them off before a disease spreads.
But HIV mutates at every turn, making it almost impossible to design a vaccine to attack it.
Bernstein said the most exciting new research involves newly discovered defences in the human body called the innate immune system, which serves as an early warning system for invading diseases.
That system could provide a way to stop HIV, if scientists found a way to trigger it early enough, he said.
"We now know we may have only hours, at most days, before we have a window of opportunity to stop HIV. So that's reason to think this early warning system might be critical to activate if we are going to design a vaccine," he said.
The disappointments in the quest for a vaccine have sparked calls for an end to the research, with critics arguing the money could be better spent on other prevention or treatment efforts.
But Anthony Fauci, one of the world's top AIDS researchers at the US National Institutes of Health, insisted that a vaccine would eventually provide the world's best defence against the disease.
The NIH, the main global funder of HIV vaccine research, spent 1.5 billion on the field in 2008.
"If you look historically, vaccines have been the most cost-effective health interventions in history," by preventing the incredible financial burden of treating diseases, Fauci said.