Dan Moren, writing at Six Colors under the headline “LLMs aren’t writing,” makes a stronger case against John Gruber’s anti-watermarking essay than I did:

I have great respect for John as a writer. He truly cares about details: he’s meticulous about the words he picks, the way he constructs sentences and arguments.

This is why I find his defense so puzzling. Because a large language model could not be further away from those ideals. In the title of his piece, John describes the watermarking system as “a perversion of writing.” That might be the case if AIs were writing. But they are not; they are simply generating text. Those two things are not the same. As per the premise of John’s piece, writing is about carefully picking the words and crafting the meaning of what you’re saying, things that AI do not–and patently cannot–do.

LLMs do not care about the words that they pick because they cannot care about anything.

And:

All of this outrage presupposes the existence of a “correct” answer that the LLM is eschewing because it was told to pick different words, but if there were a correct answer, then shouldn’t it be chosen every time? But an LLM’s non-deterministic nature doesn’t work that way. Because, at the end of the day, the AI does not truly know what any of these words mean. There is no ur-text. You might as well get mad at word choice in a lorem ipsum generator.