blog-static/content/blog/search_polynomials.md

---
title: "Search as a Polynomial"
date: 2023-05-22T21:39:00-07:00
tags: ["Mathematics"]
discussionRooms: ["!qLoehEvJNRndNrdlyU:matrix.org"]
---

I read a really neat paper some time ago, and I've been wanting to write about
it ever since. The paper is called [Algebras for Weighted Search](https://dl.acm.org/doi/pdf/10.1145/3473577),
and it is a tad too deep to dive into in a blog article -- readers of ICFP are
rarely the target audience on this site. However, one particular insight I
gleaned from the paper merits additional discussion and demonstration. I'm
going to do that here.

In particular, the paper pointed out a connection between polynomials and a
general concept of _search_. In the context of the paper, "search" simply
referred to a way of finding various solutions to some problem, perhaps
like "what are the ways of getting from one place to another?". In this
case, a search would be a computation that explores the space of possible
routes.

That all sounds very abstract, so let's start with a concrete example.
Suppose that you're trying to get from city A to city B, and then from city B
to city C. Also suppose that your trips are measured in one-hour intervals
(maybe you round trip lengths, turning 2:45 into 3 hours), and that trips of
equal duration are considered equivalent ("as long as it gets me there!").
Now, I give you a list of possible routes from city A to city B, and
another list of possible routes from city B to city C, grouped by their length.
Given these two lists, what are the possible routes from A to C?

Let's make this even more concrete, and start with some actual lists of routes.
Maybe there are two routes from A to B that take two hours each, and one "quick" trip
that takes only an hour. On top of this, there's one three-hour trip from B
to C, and one two-hour trip. Given these building blocks, the list of
possible trips from A to C is as follows.

1. Two two-hour trips from A to B, followed up by the three-hour trip from B to
C.
2. Two two-hour trips from A to B, followed by the shorter two-hour trip from B
to C.
3. One one-hour trip from A to B, followed by the three-hour trip from B to C.
4. One one-hour trip from A to B, followed by the shorter two-hour trip from B to C.

In the above, to figure out the various ways of getting from A to C, we had to
examine all pairings of A-to-B routes with B-to-C routes. But then, multiple
pairings end up having the same total length: the second and third bullet
points both describe trips that take four hours. Thus, to give
our final report, we need to "combine like terms" - add up the trips from
the two matching bullet points, ending up with total of three four-hour trips.

Does this feel a little bit familiar? To me, this bears a rather striking
resemblance to an operation we've seen in high school algebra class: we're multiplying
two binomials! Here's the corresponding multiplication:

{{< latex >}}
\left(2x^2 + x\right)\left(x^3+x^2\right) = 2x^5 + 2x^4 + x^4 + x^3 = \underline{2x^5+3x^4+x^3}
{{< /latex >}}

It's not just binomials that correspond to our combining paths between cities.
We can represent any combination of trips of various lengths as a polynomial.
Each term \(ax^n\) represents \(a\) trips of length \(n\). As we just
saw, multiplying two polynomials corresponds to "sequencing" the trips they
represent -- matching each trip in one with each of the trips in the other,
and totaling them up.

What about adding polynomials, what does that correspond to? The answer there
is actually quite simple: if two polynomials both represent (distinct) lists of
trips from A to B, then adding them just combines the list. If I know one trip
that takes two hours (\(x^2\)) and someone else knows a shortcut (\(x\)),
then we can combine that knowledge (\(x^2+x\)).

{{< dialog >}}
{{< message "question" "reader" >}}
Wait a moment. Sure, we learned about polynomials in algebra class: they're
functions! You put in a number for \(x\), and get another number out.
But you haven't done that, and in fact you haven't even mentioned
functions at all. What's going on?
{{< /message >}}
{{< message "answer" "Daniel" >}}
In this article (and in the paper it's based on), polynomials are viewed in
a more general way than you might be used to. The point isn't to think of
them as defining functions on numbers, but to make use of their "shape": a sum
of certain powers of \(x\), like \(ax^n+bx^m+...\)
{{< /message >}}
{{< message "question" "reader" >}}
So we won't be plugging numbers in, or trying to graph the polynomials in
this section?
{{< /message >}}
{{< message "answer" "Daniel" >}}
That's right, we won't be. The sort of thing we're doing here is a bit
closer to <a href="https://en.wikipedia.org/wiki/Abstract_algebra">abstract algebra</a>
than to high school math. Don't worry if you're not familiar with the
subject, though: I'm trying to explain everything from first principles.
{{< /message >}}
{{< /dialog >}}

Well, it's a neat little thing that tracking trips corresponds to adding
and mulitpying polynomials like that. We can push this observation a bit
further, though. Since our trick relies on multiplying two polynomials,
we'll need to better understand what that multiplication needs to behave as we
expect. In particular, we'll need to know what the "bare minimum" is for
working with polynomial: what arithmetic properties must we bring to the table?
Let's take a look at that next.

### Polynomials over Semirings
Let's watch what happens when we multiply two binomials, paying really close
attention to the operations we're performing. The following (concrete)
example should do.

{{< latex >}}
\begin{aligned}
    & (x+1)(1-x)\\
    =\ & (x+1)1+(x+1)(-x)\\
    =\ & x+1-x^2-x \\
    =\ & x-x+1-x^2 \\
    =\ & 1-x^2
\end{aligned}
{{< /latex >}}

The first thing we do is _distribute_ the multiplication over the addition, on
the left. We then do that again, on the right this time. After this, we finally
get some terms, but they aren't properly grouped together; an \(x\) is at the
front, and a \(-x\) is at the very back. We use the fact that addition is
_commutative_ (\(a+b=b+a\)) and _associative_ (\(a+(b+c)=(a+b)+c\)) to
rearrange the equation, grouping the \(x\) and its negation together. This
gives us \((1-1)x=0x=0\). That last step is important: we've used the fact
that multiplication by zero gives zero. Another important property (though
we didn't use it here) is that multiplication has to be associative, too.

So, what if we didn't use numbers, but rather any _thing_ with two
operations, one kind of like \((\times)\) and one kind of like \((+)\)?

{{< dialog >}}
{{< message "question" "reader" >}}
Here, it seems like you're saying that in the polynomials we've seen so
far, it's numbers themselves that need to be commutative, associative, etc..
{{< /message >}}
{{< message "answer" "Daniel" >}}
That's right, I am saying that. We need the \((+)\) and \((\times)\)
operations on numbers to follow the laws I laid out above.
{{< /message >}}
{{< message "question" "reader" >}}
Okay, but in your equations above, it's not just numbers that were moved
around using commutativity and associativity: it was variables, like \(x\).
Just earlier you said that we're thinking of the polynomials in terms of
their "shape", and not as functions. If that's the case, why we allowed to
blur the lines between polynomial and number like that?
{{< /message >}}
{{< message "answer" "Daniel" >}}
Good question. If you want to get really precise, in the abstract view,
adding numbers is not quite the same as adding polynomials. Because of this,
saying that addition commutes for numbers does not <em>immediately</em> tel
us that it commutes for something like \(x\). However, also in the abstract
view, we define how addition and multiplication on polynomials work
<em>using</em> addition and multiplication numbers. Thus, properties of
numbers make their way into properties of polynomials.
{{< /message >}}
{{< /dialog >}}


As I was saying, what if we used some other kind of _thing_ other than
numbers, together with notions of what it means to "add" and "multiply"
this _thing_? As long as these operations satisfy the properties we have used so far, we
should be able to create polynomials using them, and do this same sort of
"combining paths" we did earlier. Before we get to that, let me just say
that "things with addition and multiplication that work in the way we
described" have an established name in math - they're called semirings.

A __semiring__ is a set equipped with two operations, one called
"multiplicative" (and thus carrying the symbol \(\times)\) and one
called "additive" (and thus written as \(+\)). Both of these operations
need to have an "identity element". The identity element for multiplication
is usually
{{< sidenote "right" "written-as-note" "written as \(1\)," >}}
And I do mean "written as": a semiring need not be over numbers. We could
define one over <a href="https://en.wikipedia.org/wiki/Graph">graphs</a>,
sets, and many other things! Nevertheless, because most of us learn the
properties of addition and multiplication much earlier than we learn about
other more "esoteric" things, using numbers to stand for special elements
seems to help use intuition.
{{< /sidenote >}}
and the identity element for addition is written
as \(0\). Furthermore, a few equations hold. I'll present them in groups.
First, multiplication is associative and multiplying by \(1\) does nothing;
in mathematical terms, the set forms a [monoid](https://mathworld.wolfram.com/Monoid.html)
with multiplication and \(1\).
{{< latex >}}
\begin{array}{cl}
    (a\times b)\times c = a\times(b\times c) & \text{(multiplication associative)}\\
    1\times a = a = a \times 1 & \text{(1 is multiplicative identity)}\\
\end{array}
{{< /latex >}}

Similarly, addition is associative and adding \(0\) does nothing.
Addition must also be commutative; in other words, the set forms a
commutative monoid with addition and \(0\).
{{< latex >}}
\begin{array}{cl}
    (a+b)+c = a+(b+c) & \text{(addition associative)}\\
    0+a = a = a+0 & \text{(0 is additive identity)}\\
    a+b = b+a & \text{(addition is commutative)}\\
\end{array}
{{< /latex >}}

Finally, a few equations determine how addition and multiplication interact.
{{< latex >}}
\begin{array}{cl}
    0\times a = 0 = a \times 0 & \text{(annihilation)}\\
    a\times(b+c) = a\times b + a\times c & \text{(left distribution)}\\
    (a+b)\times c = a\times c + b\times c & \text{(right distribution)}\\
\end{array}
{{< /latex >}}

That's it, we've defined a semiring. First, notice that numbers do indeed
form a semiring; all the equations above should be quite familiar from algebra
class. When using polynomials with numbers to do our city path finding,
we end up tracking how many different ways there are to get from one place to
another in a particular number of hours. There are, however, other semirings
we can use that yield interesting results, even though we continue to add
and multiply polynomials.

One last thing before we look at other semirings: given a semiring \(R\),
the polynomials using that \(R\), and written in terms of the variable
\(x\), are denoted as \(R[x]\).


#### The Semiring of Booleans, \\(\\mathbb{B}\\)
Alright, it's time for our first non-number example. It will be a simple one,
though - booleans (that's right, `true` and `false` from your favorite
programming language!) form a semiring. In this case, addition is the
"or" operation (aka `||`), in which the result is true if either operand
is true, and false otherwise.

{{< latex >}}
\begin{array}{c}
    \text{true} + b = \text{true}\\
    b + \text{true} = \text{true}\\
    \text{false} + \text{false} = \text{false}
\end{array}
{{< /latex >}}

For addition, the identity element -- our \(0\) -- is \(\text{false}\).

Correspondingly, multiplication is the "and" operation (aka `&&`), in which the
result is false if either operand is false, and true otherwise.

{{< latex >}}
\begin{array}{c}
    \text{false} \times b = \text{false}\\
    b \times \text{false} = \text{false}\\
    \text{true} \times \text{true} = \text{true}
\end{array}
{{< /latex >}}

For multiplication, the identity element -- the \(1\) -- is \(\text{true}\).

It's not hard to see that _both_ operations are commutative - the first and
second equations for addition, for instance, can be combined to get
\(\text{true}+b=b+\text{true}\), and the third equation clearly shows
commutativity when both operands are false. The other properties are
easy enough to verify by simple case analysis (there are 8 cases to consider).
The set of booleans is usually denoted as \(\mathbb{B}\), which means
polynomials using booleans are denoted by \(\mathbb{B}[x]\).

Let's try some examples. We can't count how many ways there are to get from
A to B in a certain number of hours anymore: booleans aren't numbers!
Instead, what we _can_ do is track _whether or not_ there is a way to get
from A to B in a certain number of hours (call it \(n\)). If we can,
we write that as \(\text{true}\ x^n = 1x^n = x^n\). If we can't, we write
that as \(\text{false}\ x^n = 0x^n = 0\). The polynomials corresponding
to our introductory problem are \(x^2+x^1\) and \(x^3+x^2\). Multiplying
them out gives:

{{< latex >}}
(x^2+x^1)(x^3+x^2) = x^5 + x^4 + x^4 + x^3 = x^5 + x^4 + x^2
{{< /latex >}}

And that's right; if it's possible to get from A to B in either two hours
or one hour, and then from B to C in either three hours or two hours, then
it's possible to get from A to C in either five, four, or three hours. In a
way, polynomials like this give us
{{< sidenote "right" "homomorphism-note" "less information than our original ones" >}}
In fact, we can construct a semiring homomorphism (kind of like a
<a href="https://en.wikipedia.org/wiki/Ring_homomorphism">ring homomorphism</a>,
but for semirings) from \(\mathbb{N}[x]\) to \(\mathbb{B}[x]\) as follows:

{{< latex >}}
    \sum_{i=0}^n a_ix^i \mapsto \sum_{i=0}^n \text{clamp}(a_i)x^i
{{< /latex >}}

Where the \(\text{clamp}\) function checks if its argument is non-zero.
In the case of city path search, \(\text{clamp}\) asks the questions
"are there any routes at all?".

{{< latex >}}
\text{clamp}(n) = \begin{cases}
    \text{false} & n = 0 \\
    \text{true} & n > 0
\end{cases}
{{< /latex >}}

We can't construct the inverse of the above homomorphism (a mapping
that would undo our clamping, and take polynomials in \(\mathbb{B}[x]\) to
\(\mathbb{N}[x]\)). This fact gives us a more "mathematical" confirmation
that we lost information, rather than gained it, but switching to
boolean polynomials: we can always recover a boolean polynomial from the
natural number one, but not the other way around.
{{< /sidenote >}}
(which were \(\mathbb{N}[x]\), polynomials over natural numbers \(\mathbb{N} = \{ 0, 1, 2, ... \}\)), so it's unclear why we'd prefer them. However,
we're just warming up - there are more interesting semirings for us to
consider!

#### The Semiring of Sets of Paths, \\(\\mathcal{P}(\\Pi)\\)
Until now, we explicitly said that "all paths of the same length are
equivalent". If we're giving directions, though, we might benefit
from knowing not just that there _is_ a way, but what roads that
way is made up of!

To this end, we define the set of paths, \(\Pi\). This set will consist
of the empty path (which we will denote \(\circ\), why not?), street
names (e.g. \(\text{Mullholland Dr.}\) or \(\text{Sunset Blvd.}\)), and
concatenations of paths, written using \(\rightarrow\). For instance,
a path that first takes us on \(\text{Highway}\) and then on
\(\text{Exit 4b}\) will be written as:

{{< latex >}}
\text{Highway}\rightarrow\text{Exit 4b}
{{< /latex >}}

Furthermore, it's not too much of a stretch to say that adding an empty path
to the front or the back of another path doesn't change it. If we use
the letter \(\pi\) to denote a path, this means the following equation:

{{< latex >}}
\circ \rightarrow \pi = \pi = \pi \rightarrow \circ
{{< /latex >}}

{{< sidenote "right" "paths-monoid-note" "So those are paths." 0.25 >}}
Actually, if you clicked through the
<a href="https://mathworld.wolfram.com/Monoid.html">monoid</a>
link earlier, you might be interested to know that paths as defined here
form a monoid with concatenation \(\rightarrow\) and the empty path \(\circ\)
as a unit.
{{< /sidenote >}}
Paths alone, though, aren't enough for our polynomials; we're tracking
different ways to get from one place to another. This is an excellent
use case for sets!

Our next semiring will be that of _sets of paths_. Some example elements
of this semiring are \(\varnothing\), also known as the empty set,
\(\{\circ\}\), the set containing only the empty path, and the set
containing a path via the highway, and another path via the suburbs:

{{< latex >}}
\{\text{Highway}\rightarrow\text{Exit 4b}, \text{Suburb Rd.}\}
{{< /latex >}}

So what are the addition and multiplication on sets of paths? Addition
is the easier one: it's just the union of sets (the "triangle equal sign"
symbol means "defined as"):

{{< latex >}}
A + B \triangleq A \cup B
{{< /latex >}}

It's well known (and not hard to verify) that set union is commutative
and associative. The additive identity \(0\) is simply the empty set
\(\varnothing\). Intuitively, adding "no paths" to another set of
paths doesn't add anything, and thus leaves that other set unchanged.

Multiplication is a little bit more interesting, and uses the path
concatenation operation we defined earlier. We will use this
operation to describe path sequencing; given two sets of paths,
\(A\) and \(B\), we'll create a new set of paths
consisting of each path from \(A\) concatenated with each
path from \(B\):

{{< latex >}}
A \times B \triangleq \{ a \rightarrow b\ |\ a \in A, b \in B \}
{{< /latex >}}

The fact that this definition of multiplication on sets is associative
relies on the associativity of path concatenation; if path concatenation
weren't associative, the second equality below would not hold.

{{< latex >}}
\begin{array}{rcl}
A \times (B \times C) & = & \{ a \rightarrow (b \rightarrow c)\ |\ a \in A, b \in B, c \in C \} \\
    & \stackrel{?}{=} & \{ (a \rightarrow b) \rightarrow c \ |\ a \in A, b \in B, c \in C \} \\
    & = & (A \times B) \times C
\end{array}
{{< /latex >}}

What's the multiplicative identity? Well, since multiplication concatenates
all the combinations of paths from two sets, we could try making a set of
elements that don't do anything when concatenating. Sound familiar? It should,
that's \(\circ\), the empty path element! We thus define our multiplicative
identity as \(\{\circ\}\), and verify that it is indeed the identity:

{{< latex >}}
\begin{gathered}
\{\circ\} \times A = \{ \circ \rightarrow a\ |\ a \in A \} = \{ a \ |\  a \in A \} = A \\
A \times \{\circ\}= \{ a\rightarrow \circ \ |\ a \in A \} = \{ a \ |\  a \in A \} = A
\end{gathered}
{{< /latex >}}

It's not too difficult to verify the annihilation and distribution laws for
sets of paths, either; I won't do that here, though. Finally, let's take
a look at an example. Like before, we'll try make one that corresponds to
our introductory description of paths from A to B and from B to C. Now we need
to be a little bit creative, and come up with names for all these different
roads between our hypothetical cities. Let's say that \(\text{Highway A}\)
and \(\text{Highway B}\) are the two paths from A to B that take two hours
each, and then \(\text{Shortcut}\) is the path that takes one hour. As for
paths from B to C, let's just call them \(\text{Long}\) for the three-hour
path, and \(\text{Short}\) for the two-hour path. Our two polynomials
are then:

{{< latex >}}
\begin{array}{rcl}
P_1 & = & \{\text{Highway A}, \text{Highway B}\}x^2 + \{\text{Shortcut}\}x \\
P_2 & = & \{\text{Long}\}x^3 + \{\text{Short}\}x^2
\end{array}
{{< /latex >}}

Multiplying them gives:
{{< latex >}}
\begin{array}{rl}
    & \{\text{Highway A} \rightarrow \text{Long}, \text{Highway B} \rightarrow \text{Long}\}x^5\\
    + & \{\text{Highway A} \rightarrow \text{Short}, \text{Highway B} \rightarrow \text{Short}, \text{Shortcut} \rightarrow \text{Long}\}x^4\\
    + & \{\text{Shortcut} \rightarrow \text{Short}\}x^3
\end{array}
{{< /latex >}}

This resulting polynomial gives us all the paths from city A to city C,
grouped by their length!

#### The Tropical Semiring, \\(\\mathbb{R}\\)
I only have one last semiring left to show you. It's a fun semiring though,
as even its name might suggest: we'll take a look at a _tropical semiring_.

In this semiring, we go back to numbers; particularly, real numbers (e.g.,
\(1.34\), \(163\), \(e\), that kind of thing). We even use addition --
sort of. In the tropical semiring, addition serves as the _multiplicative_
operation! This is even confusing to write, so I'm going to switch up notation;
in the rest of this section, I'll use \(\otimes\) to represent the
multiplicative operation in semirings, and \(\oplus\) to represent the
additive one. The symbols \(\times\) and \(+\) will be used to represent
the regular operations on real numbers. With that, the operations on our
tropical semiring over real numbers are defined as follows:

{{< latex >}}
\begin{array}{rcl}
x \otimes y & \triangleq & x + y\\
x \oplus y  & \triangleq & \min(x,y)
\end{array}
{{< /latex >}}

What is this new semiring good for? How about this: suppose that in addition to
the duration of the trip, you'd like to track the distance you must travel for
each route (shorter routes do sometimes have more traffic!). Let's watch what
happens when we add and multiply polynomials over this semiring.
When we add terms with the same power but different coefficients, like
\(ax\oplus bx\), we end up with a term \(\min(a,b)x\). In other words,
for each trip duration, we pick the shortest length. When we multiply two
polynomials, like \(ax\otimes bx\), we get \((a+b)x\); in other words,
when sequencing two trips, we add up the distances to get the combined
distance, just like we'd expect.

We can, of course, come up with a polynomial to match our initial example.
Say that the trips from A to B are represented by \(2.0x^2\oplus1.5x\) (the
shortest two-hour trip is \(2\) units of distance long, and the one-hour
trip is \(1.5\) units long), and that the trips from B to C are represented
by \(4.0x^3\oplus1.0x^2\). Multiplying the two polynomials out gives:

{{< latex >}}
\begin{array}{rcl}
(2.0x^2\oplus1.5x)(4.0x^3\oplus1.0x^2) & = & 6.0x^5 \oplus \min(2.0+1.0, 1.5+4.0)x^4 \oplus 2.5x^3 \\
    & = & 6.0x^5 \oplus 3.0x^4 \oplus 2.5x^3
\end{array}
{{< /latex >}}

The only time we used the additive operation in this case was to pick between
two trips of equal druation but different length (two-hour trip from A to B
followed by a two-hour trip from B to C, or one-hour trip from A to C followed
by a three-hour trip from B to C). The first trip wins out, since it requires
only \(3.0\) units of distance.

### Anything but Routes
So far, all I've done can be reduced to variations on a theme: keeping track
of some aspects of a trip between cities, using polynomials for structure.
However, that's just the beginning. This sort of trick can be be made even
more powerful by further relaxing the notion of a "polynomial". By doing so,
we can make our polynomials represent arbitrary _effects_ (in the computer
science sense -- things like errors, logging to a console, storing and
accessing information from a database). Relying for just a little longer on
our example of journeys between cities, we might be able to represent trips
with random variation (traffic can be unpredicatable!), or maybe cities where
you will get stuck. But the point isn't routes: the same approach can be
used to represent traversing a binary tree, performing Prolog-like proof
search, or evaluating a non-deterministic program. The sky's the limit!

Unfortunately, doing so would require even more background and buildup, for
which I just don't have space for in this article. I'll save these
things for next time, though -- stay tuned!