The Price of Failure.
Whenever any number of Duchy buildings are removed, the Duchy discards a random card, returns their swayed minister card of highest rank
(lord, then noble, then squire) to their Unswayed Ministers pile, and removes its crown from the game permanently. If they have multiple swayed ministers of highest rank, the Duchy chooses which to return.