Hardware designed to play high quality music. (Think using Ubuntu vs Ubuntu Studio for music production)
Dedicated buttons instead of all touch screen.
More options for integration with other devices or systems
No distractions. Phones nowadays demand our attention for every little thing. Every app, no matter what it is, has notifications.
The Bluetooth is better.
You can literally hear the difference in the quality of the music if you use good quality headphones/ear buds. The same song, same file, will not sound the same if it's a good quality FLAC.
While I use my phone for music I can certainly see some advantages:
physical buttons
smaller and lighter
less distraction than a phone
cheaper to replace if stolen or broken