Bisphenol A, or BPA, is a common component of plastic bottles and canned food and drink linings, but some studies have linked it to brain and nervous system problems, reproductive disorders and obesity.
The ANSES agency urged pregnant women to avoid food preserved in cans, or drinking water from polycarbonate water fountains found in many office buildings which it said was a "source of exposure to bisphenol A".
It also advised them to avoid handling till slips.
The agency said bisphenol A could be dangerous if ingested, inhaled or simply by touching products that contain it -- including thermal paper like that used to make cash register slips.
In a report summarising several global scientific studies on the topic up to July 2012, ANSES said its experts were "moderately" confident of the evidence pointing to a risk to babies in the womb.
"The conclusions show that certain exposure of pregnant women to BPA poses a risk for the mammary glands of the unborn child," the agency said in a report that wrapped up a three-year investigation.
The risk "may be characterised... by increased sensitivity of the mammary gland to the formation of tumours. The risk potentially concerns both sexes," said the report.
ANSES stressed there were still many uncertainties in the data.
Advised by agencies like ANSES, the French parliament in December voted to ban BPA's use in baby food packaging from 2013 and in all food containers from 2015.
The European Union, United States and Canada have outlawed its use in baby bottles.
The chemical is still used around the world in plastic products, and the United States said last year it would not impose a general ban of BPA as there was no evidence of harm to adults.
ANSES also cautioned against replacing bisphenol A with a substitute from the same chemical family, saying not enough was known about other bisphenols like M, S, B, AP, AF, F and BADGE.
"These substances share a chemical structure," the agency pointed out.