114 lines
6.6 KiB
Markdown
114 lines
6.6 KiB
Markdown
---
|
|
title: "The LLM Comments Are Not For You"
|
|
date: 2026-09-14T20:49:21-05:00
|
|
tags: ["LLMs"]
|
|
---
|
|
|
|
I've heard a lot of talk about LLMs recently, and among the most common topics
|
|
of discussion have been the comments. LLM comments generally seem to suck: they are verbose,
|
|
regardless of surrounding context, they seem to encode conversation decisions
|
|
(like {{< sidenote "right" "hysteresis-note" "\"do it this way, not that way\"" >}}
|
|
In my head, I've been calling this "comment hysteresis", because the comments
|
|
are not a function of the final design, but of the path taken to get there.
|
|
{{< /sidenote >}}), and they tend to use made-up vocabulary or terms. Engineers
|
|
have been working on ways to reduce the pain, by carefully crafting their
|
|
prompts or even separately running "desloppifier" agents to clean up PRs.
|
|
In my experience, these techniques are only mildly successful.
|
|
|
|
So why do LLMs keep writing comments like these, even as their software benchmark
|
|
scores climb ever higher with new model releases? I'd like to argue that they
|
|
remain bad _precisely_ because the scores in benchmarks are getting
|
|
better. And very likely they will degrade, in the same way that
|
|
[model tool calling has gotten worse over time](https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools).
|
|
|
|
**These comments are not for you, my (hopefully human) reader.**
|
|
|
|
Fundamentally, as many in the coding agent space have pointed out (like,
|
|
say, Mario Zechner in his [talk about `pi`](https://www.youtube.com/watch?v=RjfbvDXpFls)),
|
|
most modern models have been [RL](https://en.wikipedia.org/wiki/Reinforcement_learning)'ed
|
|
to fit into agentic harnesses. These workflows are part of their training.
|
|
And the way that RL works is that it is outcome-based. A model takes steps,
|
|
edits files, does whatever it does in its agentic framework, and either
|
|
arrives at a solution or doesn't. Behaviors that contributed to successful
|
|
outcomes are encouraged, and become more common.
|
|
|
|
The difficulty is that as benchmarks get harder and as models are asked
|
|
to take on larger and larger chunks of the software development workflow,
|
|
their ability to keep information "just" within their context is
|
|
pushed closer to its limit. Session compaction can accidentally destroy
|
|
design decisions or rationale, forcing the model to eventually re-discover
|
|
previous decisions or even switch directions. I suspect that in addition
|
|
to this --- given the techniques used by OpenAI in
|
|
{{< sidenote "right" "theft-note" "its formalization" -9 >}}
|
|
Whether this was really OpenAI's formalization or plagiarism remains open
|
|
to debate, and I do not know enough to claim one way or the other.
|
|
The possessive form here is just for convenience.
|
|
{{< /sidenote >}}
|
|
of [the Navier-Stokes singularity](https://openai.com/index/navier-stokes-solution/) --- models are also trained to operate in
|
|
swarms, which don't share context but must find ways to coordinate with
|
|
{{< sidenote "right" "fable-note" "each other." >}}
|
|
I don't know how agents talk to each other, but we've already seen that the
|
|
way they talk to themselves is
|
|
<a href="https://www.reddit.com/r/ClaudeAI/comments/1ul1396/fable_5_leaked_chainofthought_in_web_interface/">very different</a>
|
|
from humans.
|
|
{{< /sidenote >}}
|
|
|
|
Comments are a hugely useful persistent store of contextual information.
|
|
If a model edits a file, chances are it will read it as well, discover
|
|
the comments, and re-load the given information into context. If
|
|
one agent makes a change to a file with some design rationale --- "array,
|
|
not a linked list" --- another agent that might want to change it back will
|
|
spot that and
|
|
{{< sidenote "right" "obedient-note" "tread carefully." 0.1 >}}
|
|
I suspect, though with less certainty, that agents are very deferential
|
|
to pre-existing comments for this exact reason. Claude Code, for instance,
|
|
will be very insistent that when code says to do X, the new code written
|
|
should fit the "X model".
|
|
{{< /sidenote >}}
|
|
It should come as no surprise that agents that make use of verbose,
|
|
{{< sidenote "left" "verbose-note" "decision-making-included" >}}
|
|
Interestingly, this turn of phrase is uncommon in pre-existing human comments,
|
|
which would make up the majority of the model's training data set. In
|
|
my opinion, this points towards this being something more than a reflection
|
|
of the "human style".
|
|
{{< /sidenote >}}
|
|
comments succeed more frequently, and get
|
|
rewarded. The result: +50/-1 comment blocks in your diff. It is irrelevant
|
|
whether these comments concisely describe the codebase; their intended
|
|
audience can read and parse them instantly. The comments are not for you.
|
|
|
|
It's very hard to prompt this behavior out, and there's a pretty good chance
|
|
that you don't want to, by the simple evolutionary argument: these things
|
|
have helped the model do well in evaluations. By removing them, you are likely
|
|
undermining part of whatever mechanism makes it tick. On top of that, you
|
|
are fighting the wiring it has developed to do exactly this. It's like trying
|
|
to get humans to stop liking [hyperpalatable foods](https://en.wikipedia.org/wiki/Hyperpalatable_food).
|
|
|
|
So then, the natural conclusion is that we should be leaving these comments as
|
|
they are, right? As long as they've been known to improve agents' performance,
|
|
the more the better? Some, who have
|
|
[leaned _heavily_ on models for self-regulating via persistent state](https://yegge.ai/essays/fences-not-sandboxes/),
|
|
have found fascinating emergent behaviors, including whole organizational
|
|
structures with
|
|
{{< sidenote "right" "yegge-note" "agent-invented-names" >}}
|
|
Did I mention that LLM comments tend to invent novel vocabulary?
|
|
{{< /sidenote >}}. They believe that to be the future.
|
|
However, in my opinion, it may not be that simple. As we've seen with human
|
|
evolutionary adaptations, they don't always do well outside of the environment
|
|
in which they arose. For instance --- to hammer the point --- humans _love_
|
|
hyperpalatable foods. They will eat them in excess, which can lead to
|
|
obesity and a variety of other conditions. These too are emergent behaviors,
|
|
and likely quite interesting from a medical perspective. That does not
|
|
make them desirable.
|
|
|
|
In the same way that scarcity of sugars and fats in nature balanced (and
|
|
motivated) human enjoyment of them, it's possible that the ephemeral nature
|
|
of software engineering "tasks" is counteracting the accumulation of LLM-generated
|
|
commentary. In real-world contexts, human attention and editing may be doing
|
|
the same thing. We are yet to see what codebases maintained entirely
|
|
with RL'ed agentic behaviors over years look like, and whether there are
|
|
limitations.
|
|
|
|
Regardless of whether this new style holds up in the extreme,
|
|
it's that way for a reason --- and you are no longer its sole intended audience.
|