While 222 patients were followed up, 110 of them developed a total of 150 infections, in areas such as the chest, stomach and intestines and the urinary tract, which led to the production of TNF proteins.
Subjects with one more acute systemic inflammation events (SIEs) fared much worse than those who had none.
And those patients who had high baseline levels of TNF and then suffered an SIE over the following six months had a 10 fold increase in the rate of cognitive decline compared to those who were SIE free.
Professor Clive Holmes at the University of Southampton, who led the research, said they had looked at patients with mild, moderate and severe Alzheimer's disease.
"The worse the infection the worse the affect on the memory, but this is only an association at the moment.
'"One might guess that people with a more rapid rate of cognitive decline are more susceptible to infections or injury, but we found no evidence to suggest that people with more severe dementia were more likely to have infections or injuries at the beginning of the study.
"If further work proves that TNF is causing more brain inflammation it may be possible to use drugs that block TNF to help dementia sufferers."
Professor Holmes said although common illnesses like colds and slight wounds could also set up an inflammatory response in the body, the data from his study did not support the idea that even these could cause memory loss.
Dr Susanne Sorensen, Head of Research, Alzheimer's Society said: "This study is an important step towards understanding the processes that occur during the onset of Alzheimer's disease.
"We know there might be a link between inflammatory processes and Alzheimer's but this is not yet fully understood.
"These findings are helping us to understand more about possible reasons for this link.
"In the meantime it's important that older people, people with dementia and carers treat any infection seriously and seek medical help."