It will be no more "Mr Nice Guy," or "Smiley Dog," "Spice," "Galaxy Gold" or any other brands of the fake pot, which until now had been sold across the counter in convenience stores and tobacco shops.
The ban was issued by New York State Health Commissioner Nirav Shah and his New York City equivalent with immediate effect.
Synthetic marijuana is a plant material coated with chemicals imitating THC, the active ingredient in marijuana.
The state order said the product had been linked to "severe adverse reactions, including death and acute renal failure," and commonly caused increased heart rates, paranoia, nausea, confusion, seizures and loss of consciousness.
"The Commissioner's order calls for sales and distribution of these products to cease immediately," Shah's office said in a statement.
"The drugs that are sold as 'synthetic marijuana' are new and poorly understood, but reports from poison control centers show that they are toxic and can be very dangerous," New York City health chief Thomas Farley said.
"Because they are sold in stores, people may believe they are safe, and use of these drugs is increasing rapidly in New York City. With this order we are getting them off store shelves and telling everyone in New York City to never use them."
The city health department said it expects it will mail the order to about 10,000 retailers. It called on New Yorkers to report non-compliance.
"Synthetic cannabinoids are not marijuana," Robert Hoffman, head of the New York City Poison Control Center, said.
"They are unique drugs made in a lab that have one effect in common with marijuana but potentially many other different effects. They are sold as herbal products, giving users a perception of safety, but in fact they are toxic drugs sprayed on plant leaves."