3 Commits

Author SHA1 Message Date
4918558893 Tweak styles to make things work nicely with "page-looking things"
Signed-off-by: Danila Fedorin <danila.fedorin@gmail.com>
2026-05-22 23:22:24 -07:00
25f8e31fd3 Finish draft
Signed-off-by: Danila Fedorin <danila.fedorin@gmail.com>
2026-05-22 22:48:12 -07:00
f92b8bcab1 Add initial draft of Pynchon before repurposing
Signed-off-by: Danila Fedorin <danila.fedorin@gmail.com>
2026-05-22 21:01:11 -07:00
4 changed files with 240 additions and 113 deletions

View File

@@ -1,113 +0,0 @@
---
title: "The LLM Comments Are Not For You"
date: 2026-09-14T20:49:21-05:00
tags: ["LLMs"]
---
I've heard a lot of talk about LLMs recently, and among the most common topics
of discussion have been the comments. LLM comments generally seem to suck: they are verbose,
regardless of surrounding context, they seem to encode conversation decisions
(like {{< sidenote "right" "hysteresis-note" "\"do it this way, not that way\"" >}}
In my head, I've been calling this "comment hysteresis", because the comments
are not a function of the final design, but of the path taken to get there.
{{< /sidenote >}}), and they tend to use made-up vocabulary or terms. Engineers
have been working on ways to reduce the pain, by carefully crafting their
prompts or even separately running "desloppifier" agents to clean up PRs.
In my experience, these techniques are only mildly successful.
So why do LLMs keep writing comments like these, even as their software benchmark
scores climb ever higher with new model releases? I'd like to argue that they
remain bad _precisely_ because the scores in benchmarks are getting
better. And very likely they will degrade, in the same way that
[model tool calling has gotten worse over time](https://lucumr.pocoo.org/2026/7/4/better-models-worse-tools).
**These comments are not for you, my (hopefully human) reader.**
Fundamentally, as many in the coding agent space have pointed out (like,
say, Mario Zechner in his [talk about `pi`](https://www.youtube.com/watch?v=RjfbvDXpFls)),
most modern models have been [RL](https://en.wikipedia.org/wiki/Reinforcement_learning)'ed
to fit into agentic harnesses. These workflows are part of their training.
And the way that RL works is that it is outcome-based. A model takes steps,
edits files, does whatever it does in its agentic framework, and either
arrives at a solution or doesn't. Behaviors that contributed to successful
outcomes are encouraged, and become more common.
The difficulty is that as benchmarks get harder and as models are asked
to take on larger and larger chunks of the software development workflow,
their ability to keep information "just" within their context is
pushed closer to its limit. Session compaction can accidentally destroy
design decisions or rationale, forcing the model to eventually re-discover
previous decisions or even switch directions. I suspect that in addition
to this --- given the techniques used by OpenAI in
{{< sidenote "right" "theft-note" "its formalization" -9 >}}
Whether this was really OpenAI's formalization or plagiarism remains open
to debate, and I do not know enough to claim one way or the other.
The possessive form here is just for convenience.
{{< /sidenote >}}
of [the Navier-Stokes singularity](https://openai.com/index/navier-stokes-solution/) --- models are also trained to operate in
swarms, which don't share context but must find ways to coordinate with
{{< sidenote "right" "fable-note" "each other." >}}
I don't know how agents talk to each other, but we've already seen that the
way they talk to themselves is
<a href="https://www.reddit.com/r/ClaudeAI/comments/1ul1396/fable_5_leaked_chainofthought_in_web_interface/">very different</a>
from humans.
{{< /sidenote >}}
Comments are a hugely useful persistent store of contextual information.
If a model edits a file, chances are it will read it as well, discover
the comments, and re-load the given information into context. If
one agent makes a change to a file with some design rationale --- "array,
not a linked list" --- another agent that might want to change it back will
spot that and
{{< sidenote "right" "obedient-note" "tread carefully." 0.1 >}}
I suspect, though with less certainty, that agents are very deferential
to pre-existing comments for this exact reason. Claude Code, for instance,
will be very insistent that when code says to do X, the new code written
should fit the "X model".
{{< /sidenote >}}
It should come as no surprise that agents that make use of verbose,
{{< sidenote "left" "verbose-note" "decision-making-included" >}}
Interestingly, this turn of phrase is uncommon in pre-existing human comments,
which would make up the majority of the model's training data set. In
my opinion, this points towards this being something more than a reflection
of the "human style".
{{< /sidenote >}}
comments succeed more frequently, and get
rewarded. The result: +50/-1 comment blocks in your diff. It is irrelevant
whether these comments concisely describe the codebase; their intended
audience can read and parse them instantly. The comments are not for you.
It's very hard to prompt this behavior out, and there's a pretty good chance
that you don't want to, by the simple evolutionary argument: these things
have helped the model do well in evaluations. By removing them, you are likely
undermining part of whatever mechanism makes it tick. On top of that, you
are fighting the wiring it has developed to do exactly this. It's like trying
to get humans to stop liking [hyperpalatable foods](https://en.wikipedia.org/wiki/Hyperpalatable_food).
So then, the natural conclusion is that we should be leaving these comments as
they are, right? As long as they've been known to improve agents' performance,
the more the better? Some, who have
[leaned _heavily_ on models for self-regulating via persistent state](https://yegge.ai/essays/fences-not-sandboxes/),
have found fascinating emergent behaviors, including whole organizational
structures with
{{< sidenote "right" "yegge-note" "agent-invented-names" >}}
Did I mention that LLM comments tend to invent novel vocabulary?
{{< /sidenote >}}. They believe that to be the future.
However, in my opinion, it may not be that simple. As we've seen with human
evolutionary adaptations, they don't always do well outside of the environment
in which they arose. For instance --- to hammer the point --- humans _love_
hyperpalatable foods. They will eat them in excess, which can lead to
obesity and a variety of other conditions. These too are emergent behaviors,
and likely quite interesting from a medical perspective. That does not
make them desirable.
In the same way that scarcity of sugars and fats in nature balanced (and
motivated) human enjoyment of them, it's possible that the ephemeral nature
of software engineering "tasks" is counteracting the accumulation of LLM-generated
commentary. In real-world contexts, human attention and editing may be doing
the same thing. We are yet to see what codebases maintained entirely
with RL'ed agentic behaviors over years look like, and whether there are
limitations.
Regardless of whether this new style holds up in the extreme,
it's that way for a reason --- and you are no longer its sole intended audience.

View File

@@ -0,0 +1,144 @@
---
title: "Everything's Touch"
date: 2026-05-14T18:01:27-07:00
draft: true
custom_css:
- style.scss
---
{{< halfpage >}}
## Everything's Touch
"Do you guys have any deuterium water?", he said to a baffled lab manager.
"You know, heavy water?"
"No... We don't have that...". She didn't recognize him as a student.
"Do you know where I can get some?", continued his barrage of questions,
"What's a good chemical company? How do I go about ordering heavy water from them?"
What could this guy possibly have to do with heavy water? Why is he so determined?
When he finally turned and left empty-handed, she breathed a sigh of relief.
Until, that is, another man arrived and made the same request: he wanted heavy water.
Again, the lab manager refused him. For the rest of the day, she had
a knot in her stomach. Having failed once, in order to remain under the radar,
had some shadowy cabal switched representatives, and tried again to attain
their goal?
Uneasy still the lab manager had dinner, opening YouTube™ on her phone to pass
the time. On the front page, a video was waiting for her: "the ice cube
is too heavy!". When frozen, you see, heavy water sinks instead of floating.
There was no plot. Two men, having both seen this video, had independently
decided to replicate the trick. On the same day, believing it was their
free will, they visited the same lab and spoke to the same lab manager.
They felt the touch.
---
> Roland too became conscious of the wind, as his mortality had never allowed him.
> Discovered it so. ...so joyful, that the arrow must veer into it.
>
> -- Thomas Pynchon, *Gravity's Rainbow*
Today, we live amidst an invisible ocean, but not in a physical sense;
its tides don't pull us out to sea or push us towards the shore; no
warm undercurrents alternate with cool water as we bob in the waves.
Standing on a hiking trail and looking out at the path ahead, the world
might look exactly as it had forty years ago. However, the ocean is
there, mediated by
{{< /halfpage >}}
{{< halfpage >}}
electromagnetism instead of fluid. Looking up at the sky,
nowadays we are reminded of this by the numerous hurtling dots delivering
the internet to practically every corner of the planet.
Reminded are we of its existence, too, when we hear it speak; when people talk
about Geese and Velvet Underground; when friends repeat nearly verbatim the
top post on /r/bald; when men show up to a teaching lab and ask for deuterium.
Sometimes, these ideas are deliberately planted. Sometimes, they are analogous
to your classic trends™. Sometimes, they just appear. An enormous behemoth
stirs deep beneath the waves, and we sway with the current.
---
> If it is in working order, what is it meant to do? The engineers
> who built it . . . never knew there were any further steps to be taken.
> Their design was "finalized", and they could forget it.
>
> -- Thomas Pynchon, *Gravity's Rainbow*
In _The Age of Surveillance Capitalism_, Shoshana Zuboff
powerfully reframes the actions of tech giants like Google
from the perspective of _behavioral surplus_. Google
and Meta's gluttony for traffic patterns, written sentiment, satellite data,
identified faces seen by smart glasses, all of it is the endless hunger of an
influence-machine. Zuboff decries the "priests of the shadow texts",
Skinnerian manipulators bent on seizing human agency for utopian or,
more likely, capitalistic ends.
I don't think that's the whole picture.
The surveillance-manipulation machine, running at incredible scale and
nudging us every moment we search or share or scroll, is ultimately
unconcerned with truth. Truth is secondary to human behavior. While it
reaches and connects a double-digit percentage of the world's population,
the machine lacks any constraint or fundamental purpose beyond engagement.
It's driven by unfathomably large probabilistic models entangled through
several orders of interactions with other models.
This enormous amalgamation, joined nowadays by (probably) well-intentioned
and only-moderately-grounded-in-reality AI™ agents™, is armed with state-of-the-art
tools and unprecedented influence. With "engagement" its only loose target,
it ceaselessly perturbs our daily thoughts like Maxwell's demon. Whole
cliques of people revisit old shows, "discover" a new band, and try Science™.
{{< /halfpage >}}
{{< halfpage >}}
---
Among _Gravity's Rainbow_'s
numerous subjects and themes is an intricate and densely
connected network of markets, influences, and hidden agendas that overlays
the political conflicts of World War 2. Individuals pursued their own economic
gains; dealers replaced cocaine with powdered milk; Phoebus intentionally limited
the lifetimes of lightbulbs; a man dressed like a Rocket smuggled hash past
an international gathering on behalf of one Sour Bummer. All of this, though, the whole system, took
on the life of its own: it was the Rocket-state. It acted in ways that were
inscrutable, mysterious, and yet behind which, in moments of panic or paranoia,
one could suspect intent.
The Rocket-state, as Pynchon described it, may or may not have existed.
It certainly does not exist today. However, beyond Zuboff's cycle of
dispossession, beyond the powerful surveillance state exposed by Snowden,
today lies the Attention-state. It is the entity stirring occasionally under
the surface of the internet's ocean.
I think this is the missing piece. Yes, Google and Meta are selling your
Gmail inbox's contents to the highest bidder. Yes, the US government is
in on this. Yeah, Meta's Ray-Bans™ are going to be cataloguing every face
you encounter on the street, and yeah, the age verification laws are probably
going to be used to further associate your identity with the rest of
your data point-cloud. But these are just glimpses of the larger system.
Those "priests of the shadow texts" are like GR's Freemasons: they perform
the rituals, but the magic is elsewhere.
This is the closest we've ever been to a truly "unified consciousness".
We've built ourselves a noosphere, but it's not what we thought it would
be. It's not _just_ the "world at our fingertips".
It's not entirely a reflection of human minds. There is an impurity,
an additional active force that decides which whispers are carried across
the ocean and which shouts succumb to the inverse-square law.
There it sits, this impurity, mixed in with most human knowledge, with the
immediate awareness of nearly every event as it occurs in most of the developed
world. It coats our facts like film, and when an acquaintance hands you
his packaged opinion, it remains in his handprints, and your heart sinks
like deuterium ice.
{{< /halfpage >}}

View File

@@ -0,0 +1,93 @@
@import "variables.scss";
@import "margin.scss";
body {
text-align: left;
background-color: white;
color: black;
/* reset to light mode */
@each $varName, $varDefault in $css-vars {
--#{$varName}: #{$varDefault};
}
}
h1, h2, h3, h4, h5, h6 {
font-family: "Times New Roman"
}
h2 {
font-size: 18pt;
margin-bottom: 0.2in;
}
.warning {
max-width: $container-width;
}
.halfpage {
width: 5.5in;
height: 8.5in;
padding: 0.25in;
box-sizing: border-box;
border: $standard-border;
flex-shrink: 0;
margin: 0.25in;
h2 {
display: block;
}
@media screen and (max-width: 5.5in) {
width: auto;
height: auto;
margin: 0;
border-left: none;
border-right: none;
}
}
p {
margin-top: 0;
margin-bottom: 0.1in;
}
html .container {
max-width: 12in;
position: static;
&:not(main) {
@include below-two-margins {
padding: 0 $container-min-padding 0 $container-min-padding;
}
}
}
main.container {
font-family: "Times New Roman";
font-size: 12pt;
line-height: 14pt;
display: flex;
box-sizing: border-box;
flex-wrap: wrap;
justify-content: left;
& > h2 {
display: none; /* copied title is on the page */
}
@media screen and (max-width: 5.5in) {
display: block;
}
@media screen and (max-width: 12in) {
justify-content: center;
}
}
hr {
padding-top: 0.1in;
padding-bottom: 0.1in;
line-height: 0.1in;
}

View File

@@ -0,0 +1,3 @@
<div class="halfpage">
{{ .Inner | markdownify }}
</div>