They put 26 people on three diets - the high-fat Atkins and the low-fat South Beach and Ornish diets.
Eighteen people finished the study, which required each of them to follow one of the diets for a month. All along, doctors checked their cholesterol and the condition of their arteries.
The Atkins Diet raised the study subjects' bad cholesterol by an average of 16 points, and brought on symptoms of hardening of the arteries, a precursor to strokes or heart attacks.
The subjects had better lab tests on the other two diets. The Ornish Diet lowered their bad cholesterol by 25 points, while the South Beach Diet lowered it by 10 points. The conditions of their arteries also improved on both diets.
"It really is the Atkins Diet that is the worst," said Dr. Michael Miller, director of preventive cardiology at the University of Maryland Medical Center in Baltimore, the New York Post said.
"The Atkins diet caused the LDL levels to go up by about 7 percent, whereas in the Ornish and South Beach diets ... they went down 7 to 10 percent."
Low density lipoprotein or LDL is the "bad" cholesterol that clogs blood vessels.
Various researchers have tested the benefits of the popular diets and reached wildly differing conclusions. Miller designed what he said was a unique approach -- to see how people fared once they stopped losing weight on any of the diets.
Studies show that people usually lose weight rapidly on any diet if they follow it properly and the weight loss itself can cause cholesterol to plummet.
"When you lose weight everything looks good but after a while you plateau and you hit a maintenance stage," said Miller, who presented his findings to a meeting of the American Heart Association in Orlando, Florida.
The researchers used ultrasound scans to measure the flexibility and dilation of blood vessels and measured proteins in the blood that can indicate inflammation.
"Some markers of inflammation were increased by as much as 30 to 40 percent during the Atkins phase, whereas during the South Beach and Ornish phases, the markers either were stable or went down, some by as much as 15 to 20 percent," Miller said.
Most studies have shown that diets that stress vegetables, low-fat sources of protein such as beans and legumes, and whole grains provide the best long-term weight loss. Many low-fat diets allow processed carbohydrates such as white flour, which have also been shown to be unhealthy, experts agree.
"We don't recommend the Atkins diet," Miller said. "Why not start out with a diet that will be healthier for you in the long run after weight loss?"