Compare commits
6 Commits
7a088d6739
...
music-theo
| Author | SHA1 | Date | |
|---|---|---|---|
| 3816599d7a | |||
| 64b2fec8c0 | |||
| e4e513faf0 | |||
| a8f69a71f0 | |||
| 9660c0d665 | |||
| 1e5ed34f0f |
3
.gitignore
vendored
3
.gitignore
vendored
@@ -1 +1,4 @@
|
||||
**/build/*
|
||||
|
||||
/.quarto/
|
||||
**/*.quarto_ipynb
|
||||
|
||||
3
_quarto.yml
Normal file
3
_quarto.yml
Normal file
@@ -0,0 +1,3 @@
|
||||
format:
|
||||
gfm:
|
||||
variant: +yaml_metadata_block
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 81 KiB |
@@ -1,415 +0,0 @@
|
||||
---
|
||||
title: "Reasons to Love the Field of Programming Languages"
|
||||
date: 2025-12-31
|
||||
tags: ["Programming Languages", "Compilers", "Type Systems"]
|
||||
---
|
||||
|
||||
I work at HPE on the
|
||||
[Chapel Programming Language](https://chapel-lang.org). Recently, another HPE
|
||||
person asked me:
|
||||
|
||||
> So, you work on the programming language. What's next for you?
|
||||
|
||||
This caught me off-guard because I hadn't even conceived of moving on.
|
||||
I don't want to move on, because __I love the field of programming languages__.
|
||||
In addition, I have come to think there is something in PL for everyone, from
|
||||
theorists to developers to laypeople.
|
||||
So, in that spirit, I am writing this list as a non-exhaustive survey that holds
|
||||
the dual purpose of explaining my personal infatuation with PL, and providing
|
||||
others with ways to engage with PL that align with their existing interests.
|
||||
I try to provide rationale for each claim, but you can just read the reasons
|
||||
themselves and skip the rest.
|
||||
|
||||
My general thesis goes something like this: programming languages are a unique
|
||||
mix of the __inherently human and social__ and the __deeply mathematical__,
|
||||
a mix that often remains deeply grounded in the practical, __low-level realities of
|
||||
our hardware__.
|
||||
|
||||
Personally, I find all of these properties equally important, but we have to
|
||||
start somewhere. Let's begin with the human aspect of programming languages.
|
||||
|
||||
### Human Aspects of PL
|
||||
|
||||
> Programs must be written for people to read, and only incidentally for machines
|
||||
> to execute.
|
||||
>
|
||||
> --- Abelson & Sussman, _Structure and Interpretation of Computer Programs_.
|
||||
|
||||
As we learn more about the other creatures that inhabit our world, we discover
|
||||
that they are similar to us in ways that we didn't expect. However, our
|
||||
language is unique to us. It gives us the ability to go far beyond
|
||||
the simple sharing of information: we communicate abstract concepts,
|
||||
social dynamics, stories. In my view, storytelling is our birthright more
|
||||
so than anything else.
|
||||
|
||||
I think this has always been reflected in the broader discipline of programming.
|
||||
_Code should always tell a story_, I've heard throughout my education and career.
|
||||
_It should explain itself_. In paradigms such as
|
||||
[literate programming](https://en.wikipedia.org/wiki/Literate_programming),
|
||||
we explicitly mix prose and code. Notebook technologies
|
||||
like [Jupyter](https://jupyter.org/) intersperse computation with explanations
|
||||
thereof.
|
||||
|
||||
* __Reason 1__: programming languages provide the foundation of expressing
|
||||
human thought and stories through code.
|
||||
|
||||
From flowery prose to clinical report, human expression takes a wide variety
|
||||
of forms. The need to vary our descriptions is also well-served by the diversity
|
||||
of PL paradigms. From stateful transformations in languages like Python and C++,
|
||||
through pure and immutable functions in Haskell and Lean, to fully declarative
|
||||
statements-of-fact in Nix, various languages have evolved to
|
||||
support the many ways in which we wish to describe our world and our needs.
|
||||
|
||||
* __Reason 2__: diverse programming languages enable different perspectives
|
||||
and ways of storytelling, allowing us choice in how to express our thoughts
|
||||
and solve our problems.
|
||||
|
||||
Those human thoughts of ours are not fundamentally grounded in logic,
|
||||
mathematics, or anything else. They are a product of millennia of evolution
|
||||
through natural selection, of adaptation to ever-changing conditions.
|
||||
Our cognition is limited, rife with blind spots, and partial to the subject
|
||||
matter at hand. We lean on objects, actors, contracts, and more as helpful,
|
||||
mammal-compatible analogies. I find this to be beautiful; here is something
|
||||
we can really call ours.
|
||||
|
||||
* __Reason 3__: programming languages imbue the universe's fundamental rules of
|
||||
computation with humanity's identity and idiosyncrasies. They carve out
|
||||
a home for us within impersonal reality.
|
||||
|
||||
Storytelling (and, more generally, writing) is not just about communicating
|
||||
with others. Writing helps clarify one's own thoughts, and to think deeper.
|
||||
In his 1979 Turing Award lecture,
|
||||
[Notation as a Tool of Thought](https://www.eecg.utoronto.ca/~jzhu/csc326/readings/iverson.pdf),
|
||||
Kenneth Iverson, the creator of [APL](https://tryapl.org/), highlighted ways
|
||||
in which programming languages, with their notation, can help express patterns
|
||||
and facilitate thinking.
|
||||
|
||||
Throughout computing history, programming languages built abstractions that ---
|
||||
together with advances in hardware --- made it possible to create ever more
|
||||
complex software. Dijkstra's
|
||||
[structured programming](https://en.wikipedia.org/wiki/Structured_programming)
|
||||
crystallized the familiar patterns of `if`/`else` and `while` out of
|
||||
a sea of control flow. Structures and objects partitioned data and state
|
||||
into bundles that could be reasoned about, or put out of mind when irrelevant.
|
||||
Recently, I dare say that notions of ownership and lifetimes popularized
|
||||
by Rust have clarified how we think about memory.
|
||||
|
||||
* __Reason 4__: programming languages combat complexity, and give us tools to
|
||||
think and reason about unwieldy and difficult problems.
|
||||
|
||||
The fight against complexity occurs on more battlegrounds than PL design alone.
|
||||
Besides its syntax and semantics, a programming language is comprised of its
|
||||
surrounding tooling: its interpreter or compiler, perhaps its package manager
|
||||
or even its editor. Language designers and developers take great care to
|
||||
[improve the quality of error messages](https://elm-lang.org/news/compiler-errors-for-humans),
|
||||
to provide [convenient editor tooling](https://chapel-lang.org/blog/posts/chapel-lsp/),
|
||||
and build powerful package managers
|
||||
like [Yarn](https://yarnpkg.com/). Thus, in each language project, there is
|
||||
room for folks who, even if they are not particularly interested in grammars or
|
||||
semantics, care about the user experience.
|
||||
|
||||
* __Reason 5__: programming languages provide numerous opportunities for
|
||||
thoughtful forays into the realms of User Experience and Human-Computer
|
||||
Interaction.
|
||||
|
||||
I hope you agree, by this point, that programming languages are fundamentally
|
||||
tethered to the human. Like any human endeavor, then, they don't exist in
|
||||
isolation. To speak a language, one usually wants a partner who understands
|
||||
and speaks that same language. Likely, one wants a whole community, topics
|
||||
to talk about, or even a set of shared beliefs or mythologies. This desire
|
||||
maps onto the realm of programming languages. When using a particular PL,
|
||||
you want to talk to others about your code, implement established design patterns,
|
||||
use existing libraries.
|
||||
|
||||
I mentioned mythologies earlier. In some ways, language
|
||||
communities do more than share know-how about writing code. In many
|
||||
cases, I think language communities rally around ideals embodied by their
|
||||
language. The most obvious example seems to be Rust. From what I've seen,
|
||||
the Rust community believes in language design that protects its users
|
||||
from the pitfalls of low-level programming. The Go community
|
||||
believes in radical simplicity. Julia actively incorporates contributions from
|
||||
diverse research projects into an interoperable set of scientific packages.
|
||||
|
||||
* __Reason 6__: programming languages are complex collaborative social projects
|
||||
that have the power to champion innovative ideas within the field of
|
||||
computer science.
|
||||
|
||||
So far, I've presented interpretations of the field of PL as tools for expression and thought,
|
||||
human harbor to the universe's ocean, and collaborative social projects.
|
||||
These interpretations coexist and superimpose, but they are only a fraction of
|
||||
the whole. What has kept me enamored with PL is that it blends these human
|
||||
aspects with a mathematical ground truth, through fundamental connections to
|
||||
computation and mathematics.
|
||||
|
||||
### The Mathematics of PL
|
||||
|
||||
> Like buses: you wait two thousand years for a definition of “effectively
|
||||
> calculable”, and then three come along at once.
|
||||
>
|
||||
> --- Philip Wadler, _Propositions as Types_
|
||||
|
||||
There are two foundations,
|
||||
[lambda calculus](https://en.wikipedia.org/wiki/Lambda_calculus) and
|
||||
[Turing machines](https://en.wikipedia.org/wiki/Turing_machine), that underpin
|
||||
most modern PLs. The abstract notion of Turing machines
|
||||
is closely related to, and most similar among the "famous" computational models,
|
||||
to the
|
||||
[von Neumann Architecture](https://en.wikipedia.org/wiki/Von_Neumann_architecture).
|
||||
Through bottom-up organization of "control unit instructions" into
|
||||
"structured programs" into the imperative high-level languages today, we can
|
||||
trace the influence of Turing machines in C++, Python, Java, and many others.
|
||||
At the same time, and running on the same hardware functional programming
|
||||
languages like Haskell represent a chain of succession from the lambda calculus,
|
||||
embellished today with types and numerous other niceties. These two lineages
|
||||
are inseparably linked: they have been mathematically proven to be equivalent.
|
||||
They are two worlds coexisting.
|
||||
|
||||
The two foundations have a crucial property in common: they are descriptions
|
||||
of what can be computed. Both were developed initially as mathematical formalisms.
|
||||
They are rooted not only in pragmatic concerns of "what can I do with
|
||||
these transistors?", but in the deeper questions of "what can be done
|
||||
with a computer?".
|
||||
|
||||
* __Reason 7__: general-purpose programming languages are built on foundations of computation,
|
||||
and wield the power to compute anything we consider "effectively computable at all".
|
||||
|
||||
Because of these mathematical beginnings, we have long had precise and powerful
|
||||
ways to talk about what code written in a particular language _means_.
|
||||
This is the domain of _semantics_. Instead of reference implementations
|
||||
of languages (CPython for Python, `rustc` for Rust), and instead of textual
|
||||
specifications, we can explicitly map constructs in languages either to
|
||||
mathematical objects ([denotational semantics](https://en.wikipedia.org/wiki/Denotational_semantics))
|
||||
or to (abstractly) execute them ([operational semantics](https://en.wikipedia.org/wiki/Operational_semantics)).
|
||||
|
||||
To be honest, the precise and mathematical nature of these tools is, for me,
|
||||
justification enough to love them. However, precise semantics for languages
|
||||
have real advantages. For one, they allow us to compare programs' real
|
||||
behavior with what we _expect_, giving us a "ground truth" when trying to
|
||||
fix bugs or evolve the language. For another, they allow us to confidently
|
||||
make optimizations: if you can _prove_ that a transformation won't affect
|
||||
a program's behavior, but make it faster, you can safely use it. Finally,
|
||||
the discipline of formalizing programming language semantics usually entails
|
||||
boiling them down to their most essential components. Stripping the
|
||||
[syntax sugar](https://en.wikipedia.org/wiki/Syntactic_sugar) helps clarify
|
||||
how complex combinations of features should behave together.
|
||||
|
||||
Some of these techniques bear a noticeable resemblance to the study of
|
||||
semantics in linguistics. Given our preceding discussion on the humanity
|
||||
of programming languages, perhaps that's not too surprising.
|
||||
|
||||
* __Reason 8__: programming languages can be precisely formalized, giving
|
||||
exact, mathematical descriptions of how they should work.
|
||||
|
||||
In talking about how programs behave, we run into an important limitation
|
||||
of reasoning about Turing machines and lambda calculus, stated precisely in
|
||||
[Rice's theorem](https://en.wikipedia.org/wiki/Rice%27s_theorem):
|
||||
all non-trivial semantic properties of programs (termination, throwing errors)
|
||||
are undecidable. There will always be programs that elude not only human analysis,
|
||||
but algorithmic understanding.
|
||||
|
||||
It is in the context of this constraint that I like to think about type systems.
|
||||
The beauty of type systems, to me, is in how they tame the impossible.
|
||||
Depending on the design of a type system, a well-typed program may well be
|
||||
guaranteed not to produce any errors, or produce only the "expected" sort of
|
||||
errors. By constructing reasonable _approximations_ of program
|
||||
behavior, type systems allow us to verify that programs are well-behaved in
|
||||
spite of Rice's theorem. Much of the time, too, we can do so in a way that is
|
||||
straightforward for humans to understand and machines to execute.
|
||||
|
||||
* __Reason 9__: in the face of the fundamentally impossible, type systems
|
||||
pragmatically grant us confidence in our programs for surprisingly little
|
||||
conceptual cost.
|
||||
|
||||
At first, type systems look like engineering formalisms. That
|
||||
may well be the original intention, but in our invention of type systems,
|
||||
we have actually completed a quadrant of a deeper connection: the
|
||||
[Curry-Howard isomorphism](https://en.wikipedia.org/wiki/Curry%E2%80%93Howard_correspondence).
|
||||
[Propositions](https://en.wikipedia.org/wiki/Proposition), in the logical sense,
|
||||
correspond one-to-one with types of programs, and proofs of these propositions
|
||||
correspond to programs that have the matching type.
|
||||
|
||||
This is an incredibly deep connection. In adding parametric polymorphism
|
||||
to a type system (think Java generics, or C++ templates without specialization),
|
||||
we augment the corresponding logic with the "for all x" (\(\forall x\)) quantifier.
|
||||
Restrict the copying of values in a way similar to Rust, and you get an
|
||||
[affine logic](https://en.wikipedia.org/wiki/Affine_logic), capable of reasoning about resources and their use.
|
||||
In languages like Agda with [dependent types](https://en.wikipedia.org/wiki/Dependent_type),
|
||||
you get a system powerful enough [to serve as a foundation for mathematics](https://en.wikipedia.org/wiki/Intuitionistic_type_theory).
|
||||
Suddenly, you can write code and mathematically prove properties about that
|
||||
code in the same language. I've done this in my work with
|
||||
[formally-verified static program analysis]({{< relref "series/static-program-analysis-in-agda" >}}).
|
||||
|
||||
This connection proves appealing even from the perspective of "regular"
|
||||
mathematics. We have developed established engineering practices
|
||||
for writing code: review, deployment, documentation. What if we could use
|
||||
the same techniques for doing mathematics? What if, through the deep
|
||||
connection of programming languages to logic, we could turn mathematics
|
||||
into a computer-verified, collaborative endeavor?
|
||||
I therefore present:
|
||||
|
||||
* __Reason 10__: type systems for programming languages deeply correspond
|
||||
to logic, allowing us to mathematically prove properties about code,
|
||||
using code, and to advance mathematics through the practices of software engineering.
|
||||
|
||||
{{< details summary="Bonus meta-reason to love the mathy side of PL!" >}}
|
||||
In addition to the theoretical depth, I also find great enjoyment in the way that PL is practiced.
|
||||
Here more than elsewhere, creativity and artfulness come into
|
||||
play. In PL, [inference rules](https://en.wikipedia.org/wiki/Rule_of_inference) are a
|
||||
lingua franca through which the formalisms I've mentioned above are expressed
|
||||
and shared. They are such a central tool in the field that I've
|
||||
developed [a system for exploring them interactively]({{< relref "blog/bergamot" >}})
|
||||
on this blog.
|
||||
|
||||
In me personally, inference rules spark joy. They are a concise and elegant
|
||||
way to do much of the formal heavy-lifting I described in this section;
|
||||
we use them for operational semantics, type systems, and sometimes more.
|
||||
When navigating the variety and complexity of the many languages and type
|
||||
systems out there, we can count on inference rules to take us directly to
|
||||
what we need to know. This same variety naturally demands flexibility in
|
||||
how rules are constructed, and what notation is used. Though this can sometimes
|
||||
be troublesome (one [paper](https://labs.oracle.com/pls/apex/f?p=LABS%3A0%3A%3AAPPLICATION_PROCESS%3DGETDOC_INLINE%3A%3A%3ADOC_ID%3A959")
|
||||
I've seen describes __27__ different ways of writing the simple operation of substitution in literature!),
|
||||
it also creates opportunities for novel and elegant ways of formalizing
|
||||
PL.
|
||||
|
||||
* __Bonus Reason__: the field of programming languages has a standard technique
|
||||
for expressing its formalisms, which precisely highlights core concepts
|
||||
and leaves room for creative expression and elegance.
|
||||
{{< /details >}}
|
||||
|
||||
I know that mathematics is a polarizing subject. Often, I find myself
|
||||
torn between wanting precision and eschewing overzealous formalism. The
|
||||
cusp between the two is probably determined by my own tolerance for abstraction.
|
||||
Regardless of how much abstraction you are interested in learning about,
|
||||
PL has another dimension, close to the ground: more often than not, our languages
|
||||
need to execute on real hardware.
|
||||
|
||||
### Pragmatics of PL
|
||||
|
||||
Your perfectly-designed language can be completely useless if there is no
|
||||
way to
|
||||
{{< sidenote "right" "execute-note" "execute it" >}}
|
||||
Technically, there are language that don't care if you execute them at all.
|
||||
Many programs in theorem-proving languages like Agda and Rocq exist only
|
||||
to be type-checked. So, you could nitpick this claim; or, you could take
|
||||
it more generally: your language can be useless if there's no
|
||||
way to make it efficiently do what it's been made to do.
|
||||
{{< /sidenote >}} efficiently. Thus, the field of PL subsumes not only
|
||||
the theoretical foundations of languages and their human-centric design; it
|
||||
includes also their realization as software.
|
||||
|
||||
The overall point of this section is that there is much depth to the techniques
|
||||
involved in bringing a programming language to life. If you are a tinkerer
|
||||
or engineer at heart, you will never run out of avenues of exploration.
|
||||
The reasons are all framed from this perspective.
|
||||
|
||||
One fascinating aspect to programming languages is the "direction" from
|
||||
which they have grown. On one side, you have languages that came
|
||||
together from the need to control and describe hardware. I'd say that
|
||||
this is the case for C and C++, Fortran, and others. More often than not,
|
||||
these languages are compiled to machine code. Still subject to human
|
||||
constraints, these languages often evolve more user-facing features as time
|
||||
goes on. On the other side, you have languages developed to enable
|
||||
people to write software, later faced constraints of actually working
|
||||
efficiently. These are languages like Python, Ruby, and JavaScript. These
|
||||
languages are often interpreted (executed by a dedicated program), with
|
||||
techniques such as [just-in-time compilation](https://en.wikipedia.org/wiki/Just-in-time_compilation).
|
||||
There is no one-size-fits-all way to execute a language, and as a result,
|
||||
|
||||
* __Reason 11__: the techniques of executing programming languages are varied
|
||||
and rich. From compilation, to JIT, to interpretation, the field
|
||||
has many sub-disciplines, each with its own know-hows and tricks.
|
||||
|
||||
At the same time, someone whose goal is to actually develop a compiler
|
||||
likely doesn't want to develop everything from scratch. To do so would
|
||||
be a daunting task, especially if you want the compiler to run beyond
|
||||
the confines of a personal machine. CPU [architectures](https://en.wikipedia.org/wiki/Instruction_set_architecture)
|
||||
and operating system differences are hard for any individual to keep up with.
|
||||
Fortunately, we have a gargantuan ongoing effort in the field:
|
||||
the [LLVM Project](https://llvm.org/). LLVM spans numerous architectures
|
||||
and targets, and has become a common back-end for languages like C++
|
||||
(via [Clang](https://clang.llvm.org/get_started.html)), Swift, and Rust.
|
||||
LLVM helps share and distribute the load of keeping up with the ongoing
|
||||
march of architectures and OSes. It also provides a shared playground upon
|
||||
which to experiment with language implementations, optimizations, and more.
|
||||
|
||||
* __Reason 12__: large projects like LLVM enable language designers to
|
||||
lean on decades of precedent to develop a compiler for their language.
|
||||
|
||||
Though LLVM is powerful, it does not automatically grant languages implemented
|
||||
with it good performance. In fact, no other tool does. To make a language
|
||||
run fast requires a deep understanding of the language itself, the hardware
|
||||
upon which it runs, and the tools used to execute it. That is a big ask!
|
||||
Modern computers are extraordinarily complex. Techniques such as
|
||||
[out-of-order execution](https://en.wikipedia.org/wiki/Out-of-order_execution),
|
||||
[caching](https://en.wikipedia.org/wiki/Cache_(computing)#HARDWARE),
|
||||
and [speculative execution](https://en.wikipedia.org/wiki/Speculative_execution)
|
||||
are constantly at play. This means that any program is subject to hard-to-predict
|
||||
and often unintuitive effects. On top of that, depending on your language's
|
||||
capabilities, performance work can often entail working with additional
|
||||
hardware, such as GPUs and NICs, which have their own distinct performance
|
||||
characteristics. This applies both to compiled and interpreted languages.
|
||||
Therefore, I give you:
|
||||
|
||||
* __Reason 13__: improving the performance of a programming language is rife
|
||||
with opportunities to engage with low-level details of the hardware
|
||||
and operating system.
|
||||
|
||||
In the [mathematics section](#the-mathematics-of-pl), we talked about how constructing correct
|
||||
optimizations requires an understanding of the language's semantics. It
|
||||
was one of the practical uses for having a mathematical definition of a language.
|
||||
Reason 13 is where that comes in, but the synthesis is not automatic. In fact,
|
||||
a discipline sits in-between defining how a language behaves and
|
||||
optimizing programs: program analysis. Algorithms that analyze
|
||||
properties of programs such as [reaching definitions](https://en.wikipedia.org/wiki/Reaching_definition)
|
||||
enable optimizations such as [loop-invariant code motion](https://en.wikipedia.org/wiki/Loop-invariant_code_motion),
|
||||
which can have very significant performance impact. At the same time, for an
|
||||
analysis to be correct, it must be grounded in the program's mathematical
|
||||
semantics. There are many fascinating techniques in this discipline,
|
||||
including [ones that use lattice theory](https://cs.au.dk/~amoeller/spa/spa.pdf).
|
||||
|
||||
* __Reason 14__: the sub-discipline of program analysis serves as a grounded
|
||||
application of PL theory to PL practice, enabling numerous optimizations
|
||||
and transformations.
|
||||
|
||||
The programs your compiler generates are software, and, as we just saw,
|
||||
may need to be tweaked for performance. But the compiler and/or interpreter
|
||||
is itself a piece of software, and its own performance. Today's language
|
||||
implementations are subject to demands that hadn't been there historically.
|
||||
For instance, languages are used to provide [language servers](https://microsoft.github.io/language-server-protocol/)
|
||||
to enable editors to give users deeper insights into their code. Today,
|
||||
a language implementation may be called upon every keystroke to provide
|
||||
a typing user live updates. This has led to the introduction of
|
||||
techniques like the [query architecture](https://ollef.github.io/blog/posts/query-based-compilers.html)
|
||||
(see also [salsa](https://github.com/salsa-rs/salsa)) to avoid
|
||||
redundant work and re-used intermediate results. New language implementations
|
||||
like that of [Carbon](https://github.com/carbon-language/carbon-lang)
|
||||
are exploring alternative representations of programs in memory. In
|
||||
short,
|
||||
|
||||
* __Reason 15__: language implementations are themselves pieces of software,
|
||||
subject to unique constraints and requiring careful and innovative
|
||||
engineering.
|
||||
|
||||
### Conclusion
|
||||
|
||||
I've now given a tour of ways in which I found the PL field compelling,
|
||||
organized across three broad categories. There is just one more reason
|
||||
I'd like to share.
|
||||
|
||||
I was 16 years old when I got involved with the world of programming
|
||||
languages and compilers. Though I made efforts to learn about it through
|
||||
literature (the _Dragon Book_, and _Modern Compiler Design_), I simply
|
||||
didn't have the background to find these resources accessible. However, all
|
||||
was not lost. The PL community online has been, and still is, a vibrant and
|
||||
enthusiastic place. I have found it to be welcoming of folks with backgrounds
|
||||
spanning complete beginners and experts alike. Back then, it gave me
|
||||
accessible introductions to anything I wanted. Now, every week I see new
|
||||
articles go by that challenge my intuitions, teach me new things, or take PL
|
||||
ideas to absurd and humorous extremes. So, my final reason:
|
||||
|
||||
* __Reason 16__: the programming languages community is full of brilliant,
|
||||
kind, welcoming and enthusiastic people, who dedicate much of their
|
||||
time to spreading the joy of the field.
|
||||
|
||||
I ❤️ you.
|
||||
484
content/blog/music_theory/index.qmd
Normal file
484
content/blog/music_theory/index.qmd
Normal file
@@ -0,0 +1,484 @@
|
||||
---
|
||||
title: "Some Music Theory From (Computational) First Principles"
|
||||
date: 2025-09-20T18:36:28-07:00
|
||||
draft: true
|
||||
filters: ["./to-parens.lua"]
|
||||
custom_js: ["playsound.js"]
|
||||
---
|
||||
|
||||
Sound is a perturbation in air pressure that our ear recognizes and interprets.
|
||||
A note, which is a fundamental building block of music, is a perturbation
|
||||
that can be described by a sine wave. All sine waves have a specific and
|
||||
unique frequency. The frequency of a note determines how it sounds (its
|
||||
_pitch_). Pitch is a matter of our perception; however, it happens to
|
||||
correlate with frequency, such that notes with higher frequencies
|
||||
are perceived as higher pitches.
|
||||
|
||||
Let's encode a frequency as a class in Python.
|
||||
|
||||
```{python}
|
||||
#| echo: false
|
||||
import math
|
||||
import colorsys
|
||||
```
|
||||
|
||||
```{python}
|
||||
class Frequency:
|
||||
def __init__(self, hz):
|
||||
self.hz = hz
|
||||
```
|
||||
|
||||
```{python}
|
||||
#| echo: false
|
||||
|
||||
def points_to_polyline(points, color):
|
||||
return """<polyline fill="none" stroke="{color}"
|
||||
stroke-width="4"
|
||||
points="{points_str}" />""".format(
|
||||
color=color,
|
||||
points_str=" ".join(f"{x},{y}" for x, y in points)
|
||||
)
|
||||
|
||||
def wrap_svg(inner_svg, width, height):
|
||||
return f"""<svg xmlns="http://www.w3.org/2000/svg"
|
||||
width="{width}" height="{height}"
|
||||
viewBox="0 0 {width} {height}">
|
||||
{inner_svg}
|
||||
</svg>"""
|
||||
|
||||
OVERLAY_HUE = 0
|
||||
class Superimpose:
|
||||
def __init__(self, *args):
|
||||
global OVERLAY_HUE
|
||||
|
||||
self.args = args
|
||||
self.color = hex_from_hsv(OVERLAY_HUE, 1, 1)
|
||||
OVERLAY_HUE = (OVERLAY_HUE + 1.618033988) % 1
|
||||
|
||||
def points(self, width, height):
|
||||
points = []
|
||||
if not self.args:
|
||||
return [0 for _ in range(width)]
|
||||
|
||||
first_points = [(x, y - height/2) for (x, y) in self.args[0].points(width, height)]
|
||||
for thing in self.args[1:]:
|
||||
other_points = thing.points(width, height)
|
||||
for (i, ((x1, y1), (x2, y2))) in enumerate(zip(first_points, other_points)):
|
||||
assert(x1 == x2)
|
||||
first_points[i] = (x1, y1 + (y2 - height/2))
|
||||
|
||||
# normalize
|
||||
max_y = max(abs(y) for x, y in first_points)
|
||||
if max_y > height / 2:
|
||||
first_points = [(x, y * (0.9 * height / 2) / max_y) for x, y in first_points]
|
||||
|
||||
return [(x, height/2 + y) for x, y in first_points]
|
||||
|
||||
def get_color(self):
|
||||
return self.color
|
||||
|
||||
def _repr_svg_(self):
|
||||
width = 720
|
||||
height = 100
|
||||
|
||||
points = self.points(width, height)
|
||||
return wrap_svg(
|
||||
points_to_polyline(points, self.get_color()),
|
||||
width, height
|
||||
)
|
||||
|
||||
def hugo_shortcode(body):
|
||||
return "{{" + "< " + body + " >" + "}}"
|
||||
|
||||
class PlayNotes:
|
||||
def __init__(self, *hzs):
|
||||
self.hzs = hzs
|
||||
|
||||
def _repr_html_(self):
|
||||
toplay = ",".join([str(hz) for hz in self.hzs])
|
||||
return hugo_shortcode(f"playsound \"{toplay}\"")
|
||||
|
||||
class VerticalStack:
|
||||
def __init__(self, *args):
|
||||
self.args = args
|
||||
|
||||
def _repr_svg_(self):
|
||||
width = 720
|
||||
height = 100
|
||||
buffer = 10
|
||||
|
||||
polylines = []
|
||||
for (i, arg) in enumerate(self.args):
|
||||
offset = i * (height + buffer)
|
||||
points = [(x, y+offset) for (x,y) in arg.points(width, height)]
|
||||
polylines.append(points_to_polyline(points, arg.get_color()))
|
||||
|
||||
return wrap_svg(
|
||||
"".join(polylines),
|
||||
width, len(self.args) * (height + buffer)
|
||||
)
|
||||
|
||||
def hex_from_hsv(h, s, v):
|
||||
r, g, b = colorsys.hsv_to_rgb(h, s, v)
|
||||
return f"#{int(r*255):02x}{int(g*255):02x}{int(b*255):02x}"
|
||||
|
||||
def color_for_frequency(hz):
|
||||
while hz < 261.63:
|
||||
hz *= 2
|
||||
while hz > 523.25:
|
||||
hz /= 2
|
||||
|
||||
hue = (math.log2(hz / 261.63) * 360)
|
||||
return hex_from_hsv(hue / 360, 1, 1)
|
||||
|
||||
def Frequency_points(self, width=720, height=100):
|
||||
# let 261.63 Hz be 5 periods in the width
|
||||
points = []
|
||||
period = width / 5
|
||||
for x in range(width):
|
||||
y = 0.9 * height / 2 * math.sin(x/period * self.hz / 261.63 * 2 * math.pi)
|
||||
points.append((x, height/2 - y))
|
||||
return points
|
||||
|
||||
def Frequency_get_color(self):
|
||||
return color_for_frequency(self.hz)
|
||||
|
||||
def Frequency_repr_svg_(self):
|
||||
# the container on the blog is 720 pixels wide. Use that.
|
||||
width = 720
|
||||
height = 100
|
||||
|
||||
points = self.points(width, height)
|
||||
points_str = " ".join(f"{x},{y}" for x, y in points)
|
||||
return wrap_svg(
|
||||
points_to_polyline(points, self.get_color()),
|
||||
width, height
|
||||
)
|
||||
|
||||
Frequency.points = Frequency_points
|
||||
Frequency.get_color = Frequency_get_color
|
||||
Frequency._repr_svg_ = Frequency_repr_svg_
|
||||
```
|
||||
|
||||
Let's take a look at a particular frequency. For reason that are historical
|
||||
and not particularly interesting to me, this frequency is called "middle C".
|
||||
|
||||
```{python}
|
||||
middleC = Frequency(261.63)
|
||||
middleC
|
||||
```
|
||||
|
||||
```{python}
|
||||
#| echo: false
|
||||
PlayNotes(middleC.hz)
|
||||
```
|
||||
|
||||
Great! Now, if you're a composer, you can play this note and make music out
|
||||
of it. Except, music made with just one note is a bit boring, just like saying
|
||||
the same word over and over again won't make for an interesting story.
|
||||
No big deal -- we can construct a whole variety of notes by picking any
|
||||
other frequency.
|
||||
|
||||
```{python}
|
||||
g4 = Frequency(392.445)
|
||||
g4
|
||||
```
|
||||
|
||||
```{python}
|
||||
#| echo: false
|
||||
PlayNotes(g4.hz)
|
||||
```
|
||||
|
||||
```{python}
|
||||
fSharp4 = Frequency(370.000694) # we write this F#
|
||||
fSharp4
|
||||
```
|
||||
|
||||
```{python}
|
||||
#| echo: false
|
||||
PlayNotes(fSharp4.hz)
|
||||
```
|
||||
|
||||
This is pretty cool. You can start making melodies with these notes, and sing
|
||||
some jingles. However, if your friend sings along with you, and happens to
|
||||
sing F# while you're singing the middle C, it's going to sound pretty awful.
|
||||
So awful does it sound that somewhere around the 18th century, people started
|
||||
calling it _diabolus in musica_ (the devil in music).
|
||||
|
||||
Why does it sound so bad? Let's take a look at the
|
||||
{{< sidenote "right" "superposition-note" "superposition" >}}
|
||||
When waves combine, they follow the principle of superposition. One way
|
||||
to explain this is that their graphs are added to each other. In practice,
|
||||
what this means is that two peaks in the same spot combine to a larger
|
||||
peak, as do two troughs; on the other hand, a peak and a trough "cancel out"
|
||||
and produce a "flatter" line.
|
||||
{{< /sidenote >}} of these two
|
||||
notes, which is what happens when they are played at the same time. For reason I'm
|
||||
going to explain later, I will multiply each frequency by 4. These frequencies
|
||||
still sound bad together, but playing them higher lets me "zoom out" and
|
||||
show you the bigger picture.
|
||||
|
||||
```{python}
|
||||
Superimpose(Frequency(middleC.hz*4), Frequency(fSharp4.hz*4))
|
||||
```
|
||||
```{python}
|
||||
#| echo: false
|
||||
PlayNotes(middleC.hz, fSharp4.hz)
|
||||
```
|
||||
|
||||
Looking at this picture, we can see that it's far more disordered than the
|
||||
pure sine waves we've been looking at so far. There's not much of a pattern
|
||||
to the peaks. This is interpreted by our brain as unpleasant.
|
||||
|
||||
{{< dialog >}}
|
||||
{{< message "question" "reader" >}}
|
||||
So there's no fundamental reason why these notes sound bad together?
|
||||
{{< /message >}}
|
||||
{{< message "answer" "Daniel" >}}
|
||||
That's right. We might objectively characterize the combination of these
|
||||
two notes as having a less clear periodicity, but that doesn't mean
|
||||
that fundamentally it should sound bad. Them sounding good is a purely human
|
||||
judgement.
|
||||
{{< /message >}}
|
||||
{{< /dialog >}}
|
||||
|
||||
If picking two notes whose frequencies don't combine into a nice pattern
|
||||
makes for a bad sound, then to make a good sound we ought to pick two notes
|
||||
whose frequencies *do* combine into a nice pattern.
|
||||
Playing the same frequency twice at the same time certainly will do it,
|
||||
because both waves will have the exact same peaks and troughs.
|
||||
|
||||
```{python}
|
||||
Superimpose(middleC, middleC)
|
||||
```
|
||||
|
||||
In fact, this is just like playing one note, but louder. The fact of the matter
|
||||
is that *playing any other frequency will mean that not all extremes of the graph align*.
|
||||
We'll get graphs that are at least a little bink wonky. Intuitively, let's say
|
||||
that our wonky-ish graph has a nice pattern when they repeat quickly. That way,
|
||||
there's less time for the graph to do other, unpredictable things.
|
||||
|
||||
What's the soonest we can get our combined graph to repeat? It can't
|
||||
repeat any sooner than either one of the individual notes --- how could it?
|
||||
We can work with that, though. If we make one note have exactly twice
|
||||
the frequency of the other, then exactly at the moment the less frequent
|
||||
note completes its first repetition, the more frequent note will complete
|
||||
its second. That puts us right back where we started. Here's what this looks
|
||||
like graphically:
|
||||
|
||||
```{python}
|
||||
twiceMiddleC = Frequency(middleC.hz*2)
|
||||
VerticalStack(
|
||||
middleC,
|
||||
twiceMiddleC,
|
||||
Superimpose(middleC, twiceMiddleC)
|
||||
)
|
||||
```
|
||||
|
||||
```{python}
|
||||
#| echo: false
|
||||
PlayNotes(middleC.hz, twiceMiddleC.hz)
|
||||
```
|
||||
|
||||
You can easily inspect the new graph to verify that it has a repeating pattern,
|
||||
and that this pattern repeats exactly as frequently as the lower-frequency
|
||||
note at the top. Indeed, these two notes sound quite good together. It turns
|
||||
out, our brains consider them the same in some sense. If you have ever tried to
|
||||
sing a song that was outside of your range (like me singing along to Taylor Swift),
|
||||
chances are you sang notes that had half the frequency of the original.
|
||||
We say that these notes are _in the same pitch class_. While only the first
|
||||
of the two notes I showed above was the _middle_ C, we call both notes C.
|
||||
To distinguish different-frequency notes of the same pitch class, we sometimes
|
||||
number them. The ones in this example were C4 and C5.
|
||||
We can keep applying this trick to get C6, C7, and so on.
|
||||
|
||||
```{python}
|
||||
C = {}
|
||||
note = middleC
|
||||
for i in range(4, 10):
|
||||
C[i] = note
|
||||
note = Frequency(note.hz * 2)
|
||||
|
||||
C[4]
|
||||
```
|
||||
|
||||
To get C3 from C4, we do the reverse, and halve the frequency.
|
||||
```{python}
|
||||
note = middleC
|
||||
for i in range(4, 0, -1):
|
||||
C[i] = note
|
||||
note = Frequency(note.hz / 2)
|
||||
|
||||
C[1]
|
||||
```
|
||||
|
||||
You might've already noticed, but I set up this page so that individual
|
||||
sine waves in the same pitch class have the same color.
|
||||
|
||||
All of this puts us almost right back where we started. We might have
|
||||
different pithes, but we've only got one pitch _class_. Let's try again.
|
||||
Previously, we made it so the second repeat of one note lined up with the
|
||||
first repeat of another. What if we pick another note that repeats _three_
|
||||
times as often instead?
|
||||
|
||||
```{python}
|
||||
thriceMiddleC = Frequency(middleC.hz*3)
|
||||
VerticalStack(
|
||||
middleC,
|
||||
thriceMiddleC,
|
||||
Superimpose(middleC, thriceMiddleC)
|
||||
)
|
||||
```
|
||||
```{python}
|
||||
#| echo: false
|
||||
PlayNotes(middleC.hz, thriceMiddleC.hz)
|
||||
```
|
||||
|
||||
That's not bad! These two sound good together as well, but they are not
|
||||
in the same pitch class. There's only one problem: these notes are a bit
|
||||
far apart in terms of pitch. That `triceMiddleC` note is really high!
|
||||
Wait a minute --- weren't we just talking about singing notes that were too
|
||||
high at half their original frequency? We can do that here. The result is a
|
||||
note we've already seen:
|
||||
|
||||
```{python}
|
||||
print(thriceMiddleC.hz/2)
|
||||
print(g4.hz)
|
||||
```
|
||||
|
||||
```{python}
|
||||
#| echo: false
|
||||
PlayNotes(middleC.hz, g4.hz)
|
||||
```
|
||||
|
||||
In the end, we got G4 by multiplying our original frequency by $3/2$. What if
|
||||
we keep applying this process to find more notes? Let's not even worry
|
||||
about the specific frequencies (like `261.63`) for a moment. We'll start
|
||||
with a frequency of $1$. This makes our next frequency $3/2$. Taking this
|
||||
new frequency and again multiplying it by $3/2$, we get $9/4$. But that
|
||||
again puts us a little bit high: $9/4 > 2$. We can apply our earlier trick
|
||||
and divide the result, getting $9/16$.
|
||||
|
||||
```{python}
|
||||
from fractions import Fraction
|
||||
|
||||
note = Fraction(1, 1)
|
||||
seen = {note}
|
||||
while len(seen) < 6:
|
||||
new_note = note * 3 / 2
|
||||
if new_note > 2:
|
||||
new_note = new_note / 2
|
||||
|
||||
seen.add(new_note)
|
||||
note = new_note
|
||||
```
|
||||
|
||||
For an admittedly handwavy reason, let's also throw in one note that we
|
||||
get from going _backwards_: dividing by $2/3$ instead of multiplying.
|
||||
This division puts us below our original frequency, so let's double it.
|
||||
|
||||
```{python}
|
||||
# Throw in one more by going *backwards*. More on that in a bit.
|
||||
seen.add(Fraction(2/3) * 2)
|
||||
fractions = sorted(list(seen))
|
||||
fractions
|
||||
```
|
||||
|
||||
```{python}
|
||||
frequencies = [middleC.hz * float(frac) for frac in fractions]
|
||||
frequencies
|
||||
```
|
||||
|
||||
```{python}
|
||||
steps = [frequencies[i+1]/frequencies[i] for i in range(len(frequencies)-1)]
|
||||
minstep = min(steps)
|
||||
print([math.log(step)/math.log(minstep) for step in steps])
|
||||
```
|
||||
|
||||
Since peaks and troughs
|
||||
in the final result arise when peaks and troughs in the individual waves align,
|
||||
we want to pick two frequencies that align with a nice pattern.
|
||||
|
||||
But that begs the question: what determines
|
||||
how quickly the pattern of two notes played together repeats?
|
||||
|
||||
We can look at things geometrically, by thinking about the distance
|
||||
between two successive peaks in a single note. This is called the
|
||||
*wavelength* of a wave. Take a wave with some wavelength $w$.
|
||||
If we start at a peak, then travel a distance of $w$ from where we started,
|
||||
there ought to be another peak. Arriving at a distance of $2w$ (still counting
|
||||
from where we started), we'll see another peak. Continuing in the same pattern,
|
||||
we'll see peaks at distances of $3w$, $4w$, and so on. For a second wave
|
||||
with wavelength $v$, the same will be true: peaks at $v$, $2v$, $3v$, and so on.
|
||||
|
||||
If we travel a distance that happens to be a multiple of both $w$ and $v$,
|
||||
then we'll have a place where both peaks are present. At that point, the
|
||||
pattern starts again. Mathematically, we can
|
||||
state this as follows:
|
||||
|
||||
$$
|
||||
nw = mv,\ \text{for some integers}\ n, m
|
||||
$$
|
||||
|
||||
As we decided above, we'll try to find a combination of wavelengths/frequencies
|
||||
where the repetition happens early.
|
||||
|
||||
__TODO:__ Follow the logic from Digit Sum Patterns
|
||||
|
||||
The number of iterations of the smaller wave before we reach a cycle is:
|
||||
|
||||
$$
|
||||
k = \frac{w}{\text{gcd}(w, v)}
|
||||
$$
|
||||
|
||||
|
||||
```{python}
|
||||
Superimpose(Frequency(261.63), Frequency(261.63*2))
|
||||
```
|
||||
|
||||
```{python}
|
||||
Superimpose(Frequency(261.63*4), Frequency(392.445*4))
|
||||
```
|
||||
|
||||
```{python}
|
||||
VerticalStack(
|
||||
Frequency(440*8),
|
||||
Frequency(450*8),
|
||||
Superimpose(Frequency(440*8), Frequency(450*8))
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
```{python}
|
||||
from enum import Enum
|
||||
class Note(Enum):
|
||||
C = 0
|
||||
Cs = 1
|
||||
D = 2
|
||||
Ds = 3
|
||||
E = 4
|
||||
F = 5
|
||||
Fs = 6
|
||||
G = 7
|
||||
Gs = 8
|
||||
A = 9
|
||||
As = 10
|
||||
B = 11
|
||||
|
||||
def __add__(self, other):
|
||||
return Note((self.value + other.value) % 12)
|
||||
|
||||
def __sub__(self, other):
|
||||
return Interval((self.value - other.value + 12) % 12)
|
||||
```
|
||||
|
||||
```{python, echo=false}
|
||||
class MyClass:
|
||||
def _repr_svg_(self):
|
||||
return """<svg xmlns="http://www.w3.org/2000/svg"
|
||||
width="120" height="120" viewBox="0 0 120 120">
|
||||
<circle cx="60" cy="60" r="50" fill="red"/>
|
||||
</svg>"""
|
||||
|
||||
MyClass()
|
||||
```
|
||||
15
content/blog/music_theory/playsound.js
Normal file
15
content/blog/music_theory/playsound.js
Normal file
@@ -0,0 +1,15 @@
|
||||
window.addEventListener("load", (event) => {
|
||||
for (const elt of document.getElementsByClassName("mt-sound-play-button")) {
|
||||
elt.addEventListener("click", (event) => {
|
||||
const audioCtx = new (window.AudioContext || window.webkitAudioContext)();
|
||||
for (const freq of event.target.getAttribute("data-sound-info").split(",")) {
|
||||
const oscillator = audioCtx.createOscillator();
|
||||
oscillator.type = "sine"; // waveform: sine, square, sawtooth, triangle
|
||||
oscillator.frequency.value = parseInt(freq); // Hz
|
||||
oscillator.connect(audioCtx.destination);
|
||||
oscillator.start();
|
||||
oscillator.stop(audioCtx.currentTime + 2); // stop after 1 second
|
||||
}
|
||||
});
|
||||
}
|
||||
});
|
||||
13
content/blog/music_theory/svg-inline.lua
Normal file
13
content/blog/music_theory/svg-inline.lua
Normal file
@@ -0,0 +1,13 @@
|
||||
function Image(el)
|
||||
local src = el.src
|
||||
if src:match("%.svg$") then
|
||||
local alt = el.caption and pandoc.utils.stringify(el.caption) or ""
|
||||
local obj = string.format(
|
||||
'{{< inlinesvg "%s" "%s" >}}',
|
||||
src, alt
|
||||
)
|
||||
return pandoc.RawInline("html", obj)
|
||||
else
|
||||
return el
|
||||
end
|
||||
end
|
||||
5
content/blog/music_theory/to-parens.lua
Normal file
5
content/blog/music_theory/to-parens.lua
Normal file
@@ -0,0 +1,5 @@
|
||||
function Math(el)
|
||||
if el.mathtype == "InlineMath" then
|
||||
return pandoc.RawInline("markdown", "\\(" .. el.text .. "\\)")
|
||||
end
|
||||
end
|
||||
1
layouts/shortcodes/inlinesvg.html
Normal file
1
layouts/shortcodes/inlinesvg.html
Normal file
@@ -0,0 +1 @@
|
||||
<object type="image/svg+xml" data="{{ .Get 0 }}" aria-label="{{ .Get 1 }}"></object>
|
||||
1
layouts/shortcodes/playsound.html
Normal file
1
layouts/shortcodes/playsound.html
Normal file
@@ -0,0 +1 @@
|
||||
<button class="mt-sound-play-button" data-sound-info="{{ .Get 0 }}">Play</button>
|
||||
Reference in New Issue
Block a user