diff --git a/content/blog/comments_not_for_you.md b/content/blog/comments_not_for_you.md new file mode 100644 index 0000000..6ab0d1f --- /dev/null +++ b/content/blog/comments_not_for_you.md @@ -0,0 +1,108 @@ +--- +title: "The LLM Comments Are Not For You" +date: 2026-09-13T20:49:21-05:00 +draft: true +tags: ["LLMs"] +--- + +I've heard a lot of talk about LLMs recently, and among the most common topics +of discussion have been the comments. LLM comments generally seem to suck: they are verbose, +regardless of surrounding context, they seem to encode conversation decisions +(like {{< sidenote "right" "hysteresis-note" "\"do it this way, not that way\"" >}} +In my head, I've been calling this "comment hysteresis", because the comments +are not a function of the final design, but of the path taken to get there. +{{< /sidenote >}}), and they tend to use made-up vocabulary or terms. Engineers +have been working on ways to reduce the pain, by carefully crafting their +prompts or even separately running "desloppifier" agents to clean up PRs. +In my experience, these techniques are only mildly successful. + +So why do LLMs keep writing comments like these, even as their software benchmark +scores climb ever higher with new model releases? I'd like to argue that they +remain bad _precisely_ because the scores in benchmarks are getting +better. And very likely they will degrade, in the same way that +[model tool calling has gotten worse over time](https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools). + +**These comments are not for you, my (hopefully human) reader.** + +Fundamentally, as many in the coding agent space have pointed out (like, +say, Mario Zechner in his [talk about `pi`](https://www.youtube.com/watch?v=RjfbvDXpFls)), +most modern models have been [RL](https://en.wikipedia.org/wiki/Reinforcement_learning)'ed +to fit into agentic harnesses. These workflows are part of their training. +And the way that RL works is that it is outcome-based. A model takes steps, +edits files, does whatever it does in its agentic framework, and either +arrives at a solution or doesn't. Behaviors that contributed to successful +outcomes are encouraged, and become more common. + +The difficulty is that as benchmarks get harder and as models are asked +to take on larger and larger chunks of the software development workflow, +their ability to keep information "just" within their context is +pushed closer to its limit. Session compaction can accidentally destroy +design decisions or rationale, forcing the model to eventually re-discover +previous decisions or even switch directions. I suspect that in addition +to this --- given the techniques used by OpenAI in +{{< sidenote "right" "theft-note" "its formalization" -9 >}} +Whether this was really OpenAI's formalization or plagiarism remains open +to debate, and I do not know enough to claim one way or the other. +The possessive form here is just for convenience. +{{< /sidenote >}} +of [the Navier-Stokes singularity](https://openai.com/index/navier-stokes-solution/) --- models are also trained to operate in +swarms, which don't share context but must find ways to coordinate with +{{< sidenote "right" "fable-note" "each other." >}} +I don't know how agents talk to each other, but we've already seen that the +way they talk to themselves is +very different +from humans. +{{< /sidenote >}} + +Comments are a hugely useful persistent store of contextual information. +If a model edits a file, chances are it will read it as well, discover +the comments, and re-load the given information into context. If +one agent makes a change to a file with some design rationale --- "array, +not a linked list" --- another agent that might want to change it back will +spot that and +{{< sidenote "right" "obedient-note" "tread carefully." 0.1 >}} +I suspect, though with less certainty, that agents are very deferential +to pre-existing comments for this exact reason. Claude Code, for instance, +will be very insistent that when code says to do X, the new code written +should fit the "X model". +{{< /sidenote >}} +It should come as no surprise that agents that make use of verbose, +decision-making-included comments succeed more frequently, and get +rewarded. The result: +50/-1 comment blocks in your diff. It is irrelevant +whether these comments concisely describe the codebase; their intended +audience can read and parse them instantly. The comments are not for you. + +It's very hard to prompt this behavior out, and there's a pretty good chance +that you don't want to, by the simple evolutionary argument: these things +have helped the model do well in evaluations. By removing them, you are likely +undermining part of whatever mechanism makes it tick. On top of that, you +are fighting the wiring it has developed to do exactly this. It's like trying +to get humans to stop liking [hyperpalatable foods](https://en.wikipedia.org/wiki/Hyperpalatable_food). + +So then, the natural conclusion is that we should be leaving these comments as +they are, right? As long as they've been known to improve agents' performance, +the more the better? Some, who have +[leaned _heavily_ on models for self-regulating via persistent state](https://yegge.ai/essays/fences-not-sandboxes/), +have found fascinating emergent behaviors, including whole organizational +structures with +{{< sidenote "right" "yegge-note" "agent-invented-names" >}} +Did I mention that LLM comments tend to invent novel vocabulary? +{{< /sidenote >}}. They believe that to be the future. +However, in my opinion, it may not be that simple. As we've seen with human +evolutionary adaptations, they don't always do well outside of the environment +in which they arose. For instance --- to hammer the point --- humans _love_ +hyperpalatable foods. They will eat them in excess, which can lead to +obesity and a variety of other conditions. These too are emergent behaviors, +and likely quite interesting from a medical perspective. That does not +make them desirable. + +In the same way that scarcity of sugars and fats in nature balanced (and +motivated) human enjoyment of them, it's possible that the ephemeral nature +of software engineering "tasks" is counteracting the accumulation of LLM-generated +commentary. In real-world contexts, human attention and editing may be doing +the same thing. We are yet to see what codebases maintained entirely +with RL'ed agentic behaviors over years look like, and whether there are +limitations. + +Regardless of whether this new style holds up in the extreme, +it's that way for a reason --- and you are no longer its sole intended audience.