Simon Savage's left elbow was weak, and for repairing it, the doctors had to take tendons from his right wrist and use them.
However, they erroneously took tendons from Savage's left wrist, and operated on his healthy right arm.
The doctors did not even pay a heed to a large arrow drawn on the patient's left thumb, which indicated where the surgery should have been done.
Savage will now have to wait weeks for his right arm to recover, so that he could get a proper operation done.
"I couldn't believe it when I woke from the anaesthetic and saw it. I shouted to the nurse, 'I think he's operated on the wrong arm'. Then the doctor came in and said he was sorry," the Sun quoted him as saying.
"I am absolutely furious. They were supposed to fix it but they made it even worse," he added.
Savage is now planning to file a lawsuit against the hospital.
"I've been left with an arm that's still injured and the other in plaster," he said.
"I've got to have physio. I was already off work for six weeks. Now I've been signed off for another two months," he added.
Roger Jones, Chief of Surgery, Rotherham Hospital Trust, has admitted that there had been a mistake.
"The Trust is investigating a serious incident," he said.