This is because most of these substances are not inert and can leach into the foods we eat, they say.
Despite the fact that some of these chemicals are regulated, people who eat packaged or processed foods are likely to be chronically exposed to low levels of these substances throughout their lives, say the authors.
And far too little is known about their long term impact, including at crucial stages of human development, such as in the womb, which is "surely not justified on scientific grounds," the authors claim.
They point out that lifelong exposure to food contact materials or FCMs - substances used in packaging, storage, processing, or preparation equipment - "is a cause for concern for several reasons."
These include the fact that known toxicants, such as formaldehyde, a cancer causing substance, are legally used in these materials. Formaldehyde is widely present, albeit at low levels, in plastic bottles used for fizzy drinks and melamine tableware.
Secondly, other chemicals known to disrupt hormone production also crop up in FCMs, including bisphenol A, tributyltin, triclosan, and phthalates.
"Whereas the science for some of these substances is being debated and policy makers struggle to satisfy the needs of stakeholders, consumers remain exposed to these chemicals daily, mostly unknowingly," the authors point out.
And, thirdly, the total number of known chemical substances used intentionally in FCMs exceeds 4000.
Furthermore, potential cellular changes caused by FCMs, and in particular, those with the capacity to disrupt hormones, are not even being considered in routine toxicology analysis, which prompts the authors to suggest that this "casts serious doubts on the adequacy of chemical regulatory procedures."
They admit that establishing potential cause and effect as a result of lifelong and largely invisible exposure to FCMs will be no easy task, largely because there are no unexposed populations to compare with, and there are likely to be wide differences in exposure levels among individuals and across certain population groups.
But some sort of population-based assessment and biomonitoring are urgently needed to tease out any potential links between food contact chemicals and chronic conditions like cancer, obesity, diabetes, neurological and inflammatory disorders, particularly given the known role of environmental pollutants, they argue.
"Since most foods are packaged, and the entire population is likely to be exposed, it is of utmost importance that gaps in knowledge are reliably and rapidly filled," they urge.