Ten fire engines and 72 firefighters and officers were called to the fire on Harley Street just before 6pm.
The whole of the fourth floor and half of the fifth floor were alight, the London Fire Brigade said.
A woman was rescued by fire crews via an internal staircase, and a flat on the fourth floor was gutted by the blaze.
There were no reported injuries or fatalities from the incident on the Marylebone street, which is renowned for its cosmetic surgery clinics.
Witnesses took to social media to share pictures and videos of the fire and praise the efforts of the London Fire Brigade.
One cyclist tweeted: “There’s a fire going on in Harley St. Roads blocked off, multiple fire engines and ambulance”.
They added: “Smoke pouring out at an unbelievable rate, even though some of upstairs windows are closed. Looks like a two level flat. Brickwork blackened. Poor old lady rescued.”
Another onlooker filmed firefighters putting out the blaze as black smoke engulfed the top two floors of the building.
The crews had the fire under control by 8pm.
More from UK
PM pays tribute to London Bridge victims and heroes on anniversary of terror attack
Peckham double shooting: Police impose section 60 emergency search powers
Some Sinn Fein voters back DUP on abortion laws, Arlene Foster claims
Teenager stabbed to death ‘on way back from shops’ in Ipswich
Lush ‘intimidated by ex-police officers’ over controversial campaign
Government facing High Court challenge over ‘hostile environment’ immigration policy
Emergency services received more than 30 calls as the fire ripped through the building.
The cause of the blaze is now under investigation.