This is true for the code part, but executables can also contain data does compress well and maybe not be compressed inside the EXE (e.g. - to avoid the need to decompress it on every run)
I've been on Linux for so long, I already forgot about having to download a zip file with an installer that installs a downloader that downloads and installs the actual application.
A lot of exe files are secretly zip files. zip files can contain arbitrary data at the end of the file. exe files can have arbitrary data at the start of the file. It's a match made at Microsoft.