Then you miss out the unmentioned part of the work - idle thinking. Not only the time spent typing something on the keyboard is work. All the time spent thinking how to solve a problem is also work.
Walking to get a coffee is when I've solved some of the most complex problems in my head. Walking to get a coffee was also one of the few times I'd leave my desk. What even was lunch...