However, 16 percent of people surveyed said that their colleagues should only be paid less if taking more breaks lowered their productivity.
The survey carried out by consumer website Watch My Wallet, was done during Stoptober, the initiative to quit cigarettes over 28 days this month.
"We were stunned. It's amazing to think that a majority of the country is silently seething about the amount of time smokers spend away from their desks," the Daily Express quoted Watch My Wallet spokesman Sean O'Meara as saying.
Amanda Sandford, of anti-smoking charity Ash, said that smokers should have shorter lunchtimes or stay late.
But Simon Clark, of the smokers' lobby group Forest, said taking time out for a cigarette was no different to taking coffee breaks or making personal calls.
Source: ANI