The 62-year-old duke is no longer allowed to have an office in the building – or use it as a corresponding address, reports The sun.
The skeleton staff who were held there after he resigned three years ago are now facing the axe.
In February, Andrew paid millions of pounds to settle a civil sexual assault case brought by Virginia Giuffre in the US, without admitting any liability.
He is recruiting a new communications team to salvage his battered reputation and is now expected to fund his own staff.
A source said: “Any presence in the palace is officially over.
“The king has made it clear. He’s not a working royal. He’s on his own.”
It comes after Andrew’s cherished role of Colonel of the Grenadier Guards was bestowed on Queen Consort Camilla on Wednesday.
The sun also exclusively revealed that this month the Home Office stripped him of his £3 million a year ($5.3 million), 24-hour armed security – and that the Royal Household will foot the bill.
It is clear that Andrew will keep the 31-bed Royal Lodge on the Windsor estate for now.
He will have Christmas dinner with Charles tomorrow at a bumper bash in Sandringham – which Andrew’s ex-wife Sarah Ferguson will attend for the first time in 30 years.
The king, 73, will also pay tribute to Queen Elizabeth II in his Christmas Day speech.
Buckingham Palace declined to comment.
This article originally appeared in The Sun and is republished here with permission.