I have always thought it was “desserts” because I like picturing desserts. But no.

‘Just Deserts’ or ‘Just Desserts’? - Merriam-Webster