China's health ministry also ordered all its departments nationwide to immediately report every baby with kidney stones and having problems urinating, symptoms of drinking the tainted milk formula.
One baby has died in northwest China and dozens of others have been hospitalised across the country after developing kidney stones, the ministry said on its website as it announced a national probe.
"A joint inter-ministerial team has been set up to investigtate responsibility and strictly handle those responsible," the ministry said of the team composed of police, health, commercial and drug agencies.
Guidelines on how to treat infants suffering from the tainted formula were also issued by the ministry.
The ministry late on Thursday ordered parents not to feed their babies the Sanlu milk powder, but it did not ban the milk product from being sold.
The company has acknowledged that some of its milk powder was contaminated by the chemical tripolycyanamide, which can cause kidney stones, the ministry said.
According to the ministry, the company has ordered the recall of 700 tonnes of the baby milk powder produced before August 6, after internal company investigations showed the formula had been contaminated.
The Sanlu Group is a leading state-owned dairy company based in northern China's Hebei province that had plans to list on the Shanghai stock exchange this year.
The public relations office at the company did not answer phone calls on Friday, while no notices about a recall or apologies were posted on its website concerning the tainted milk powder.
China has in recent years endured a series of high-profile safety scandals over its food and other products sold overseas as well as domestically.
Chinese exports of fish, dumplings and pet food have raised serious concerns in the United States, Japan and elsewhere, leading to recalls of various products in those countries.