The phrase "with the exception of" is correct and commonly used in written English. It is typically used to introduce the one item or group that is different or excluded from a list of things. For example, "All guests should bring a bathing suit, with the exception of children under four."
With the exception of Spacewar!
With the exception of "Greatest Hits, Etc".
With the exception of this city".
With the exception of McCain".
With the exception of postponing the picture.
With the exception of his favorite show.
(With the exception of the lawyers).
(With the exception of Schalke 04, maybe).
Manufacturer's instructions were followed with the exception of Nebulization.
*With the exception of the former Senator from Alaska.
*with the exception of the Chechen Republic.
Since I tried Ludwig back in 2017, I have been constantly using it in both editing and translation. Ever since, I suggest it to my translators at ProSciEditing.
Justyna Jupowicz-Kozak
CEO of Professional Science Editing for Scientists @ prosciediting.com