The boy was used "as a punchbag" for months by the boyfriend. His back and ribs were found broken when he finally died in August last year.
The child's 27-year-old mother pleaded guilty to causing or allowing the death earlier and will be sentenced on December 15. She was cleared of murder earlier on the directions of the judge.
The other two accused didn't enter such pleas, but have been indicted on the same charges.
The family, from Haringey, north London, cannot be named for legal reasons.
The abuse was said to have taken place over eight months, during which time the boy was on the child protection register of Haringey social services.
He was seen 60 times by health or social workers during that period, being seen about twice a week.
By the end, he was unrecognisable, his curly, golden locks shaved off, his cheeks hollow and his eyes dead to the world.
He had more than 50 injuries or bruises and an attempt had been made to cover up the crime.
He should have been protected by social workers, police and health professionals, the Old Bailey heard.
But his mother had been able to manipulate them with lies and even got away with smearing him with chocolate to hide bruises.
In the 48 hours before the boy was found dead in his blood-spattered cot, a doctor failed to spot his broken spine.
And police told the mother she would not be prosecuted after being arrested twice for suspected child cruelty.
The tragedy has echoes of the Victoria Climbié murder in 2000, when eight-year-old Victoria died after care workers and police failed to save her, said Caroline Gammel, reporting for Telegraph.
In an ironic twist, both children's homes were a stone's throw from each other, and they were under the care of the same local services.
Mor Dioum, director of the Victoria Climbie Foundation, set up to improve child protection, said: "This case is worse than Climbié. The signs were there but were not followed."
There were "systematic and operational failures that led to the tragic and sad death of such a beautiful child".
He called for a public inquiry into the failings.
Gillie Christou, in charge of social workers looking after children on the register in Haringey, told the court she had agreed to keep the baby with his mother.
She said: "I made the decision at the time based on the material in front of me and based on the background to the case."
A detective in the case said the boy had more than 50 injuries, 15 of them to the mouth.
He described the boyfriend as "sadistic - fascinated with pain". He had Nazi memorabilia in the house.
The mother was "a slob, completely divorced from reality. She was living in a dream world and put her lover before her child. She closed her eyes to what was going on".
After the case, police said they had complied with a multi-agency long-term care plan for the family.
But procedures have now been toughened up to give police more confidence in challenging decisions.
Detective Superintendent Caroline Bates said police errors were made which caused a delay at the start of the abuse inquiry, but these had not been significant to the outcome.
She said: "With hindsight, having the benefit of a major investigation, we know quite clearly that the mother was lying and trying to subvert agencies involved with the family."
The mother had appeared to be co-operating with agencies but "she constantly conspired to prevent us knowing what was going on".
In June "police officers felt very strongly that he should not be returned" to his mother.
A police inspector asked twice if the threshold had been reached to start care proceedings.
"This was a huge tragedy which should have been avoided. If we had only known the truth about the adults in the house," said Ms Bates.