According to the 2011 census, the number of people living in England and Wales who could not speak any English is 138,000.
After English, the second most reported main language was Polish, with 546,000 speakers, followed by Punjabi and Urdu.
Some 4 million, or eight percent, reported speaking a different main language other than English or Welsh, according to the census report.
English or Welsh was the main language for 92 percent, or 50 million, of residents aged three and over. Of the 8 percent who spoke a different main language, most were proficient in speaking English.
The report showed that of those with a main language other than English, 1.7 million could speak English very well, 1.6 million could speak English well, and 726,000 could speak English, however not well, the BBC reports.
The remaining 138,000 could not speak English at all, the report said.
According to the report, in Wales, 97 percent, 2.9 million of residents, reported English or Welsh as their main language and 19 percent, 562,000, reported that they could speak Welsh.
In London, 22 percent, or 1.7 million of residents, used a main language other than English, the report added.