"Taking selenium or Vitamin E at the doses that we used in the SELECT trial for an average of five years did not prevent prostate cancer," CBS News quoted Dr. Eric Klein, one of the researchers, as saying.
The study of more than 35,000 men in their 50s and older began back in 2001.
During the trials conducted at over 400 sites, doctors gave combinations of Vitamin E, the mineral selenium, and a placebo.
The supplements failed to prevent prostate cancer, and two worrisome trends emerged.
It was observed that patients taking Vitamin E alone had a slightly higher rate of prostate cancer, while men taking selenium alone were more likely to develop diabetes.
The researchers believe that the increases might just be due to chance.
"The safety issues are of more minor concern and bear further follow-up," Klein said.
Previous studies suggested that higher doses of antioxidants like Vitamin E and selenium, both of which exist in relatively small amounts in food, might help prevent prostate cancer.
Dr. Aaron Katz said that it was due to the results of those studies that he had high hoped from the trial.
"I'm disappointed with the study. I'm very concerned about the results of the trial. I would have hoped this would have been the way to prevent cancer in this country," Katz said.
Over the next week, the 35,000 men in the study will be notified of the results and told to stop the supplements.