Most of us faff about a lot in games too. Grand Theft Auto allegedly has missions you can go on, but pretty much all I've ever seen anyone do is aimlessly cruise around listening to the radio, mow down some pedestrians, get in high speed chases, and struggle with the awkward shooting controls while getting gunned down by police. Lather, rinse, repeat. There's nothing wrong with this sort of gameplay any more than there's a problem with strictly goal-oriented gameplay (though I'm hard-pressed to think of a game that doesn't usually involve at least some faffing about). The act of pursuing goals can also be fun and engaging in and of itself. The problems arise when the goals of the game discourage the most entertaining faff.
Imagine a hypothetical fighter. We'll call it Street Kombat. In SK's practice mode, one can try out all sorts of elaborate combos and specials. If this were the only game mode, it would still be moderately entertaining to try out the countless crazy maneuvers that the developers put in. But there's a problem. The developers put so much time into the various special moves that they neglected to properly balance the game. As it turns out, the safest and most effective way to knock out the opponent in SK is to crouch in a corner and repeatedly trip them. This wouldn't be a problem in and of itself, except that the goal of the game happens to be to knock out the opponent. SK's goals have actually made the game less fun.
This is an edge case, and the best solution would presumably be to tweak the game's balance rather than trying to find goals that would make SK's broken fighting system enjoyable. But this issue crops up all the time in (usually) less noticeable ways. Have you ever wanted to explore a level more but couldn't because of a time limit, or horded the ammo for the guns that are most fun to use because you knew you might need it for a boss fight later? When a game's goals discourage players from doing the things they find most enjoyable, there is potentially a problem. (I say "potentially" because sometimes being hamstrung in certain ways can be extremely enjoyable, such as when you have to sneak past some guard or defeat an enemy without using your weapons. Just make sure that whenever you are preventing or discouraging a player from doing something they want to do, you are doing so intentionally and with purpose.)
So if this problem of imposed goals ruining players' fun can crop up in carefully designed systems like games, what happens when we recklessly impose goals and incentives in real-world environments like jobs or schools? There is research that shows that external incentives can erode and supersede our internal motivation (this is known as the Overjustification Effect), but external incentives can also directly discourage us from having fun.
While cooking a meal, have you ever twirled the spatula, juggled an egg, or perhaps stirred along to the song on the radio? What about when you were in a hurry? Would you still cook 'playfully' if you were a high-paid chef in a high-end/high-pressure restaurant?
Goals, particularly challenging ones, can make an activity more engaging and fun, but all too often they can merely prevent us from having fun. Often we faff because the activity is too easy and monotonous. Maybe rather than trying to stop a student from doodling through class, or disallowing humorous emails in the workplace, we could simply let people have fun, or even encourage it, and trust that motivation and productivity will follow. Google recognized this, and they are currently worth about 200 billion dollars. Just saying.