blog-static/content/blog/introducing_highlight/index.md

title	date	tags
Introducing Matrix Highlight	2021-12-13T16:49:42-08:00	Matrix Project Matrix Highlight

I wanted to briefly introduce a project that I've been working on in my spare time over the past couple of months. It's called Matrix Highlight, though this is a working title. However, it does exactly what the title claims: this little project is a browser extension to annotate the web, using Matrix as a communication and storage protocol. My goal with this project is a decentralized, federated, collaborative annotation system for web pages and documents (that can be self-hosted). See the image below for a quick sneak peek at what it looks like.

{{< figure src="mhl_many.png" caption="Text randomly highlighted with Matrix Highlight" class="fullwide" >}}

Project Goals

Decentralized

Quite literally, the word "decentralized" lies in opposition to "centralized". In a centralized application, the data, or computation, or anything really, is controlled by a single entity. An example of this might be Google Docs: Google is in charge of Docs, and no one else. You log in through Google, and Google manages your various documents and edits to them authoritatively.

I don't think that's a good idea. Although convenient, this kind of arrangement shifts control out of your hands as a user, and into the hands of the entity running your software. Google has the power, if they wanted, to ban you from using Google Docs, or to manage your content in ways you don't like. They can (and do) impose storage limits. You are, in a sense, at their mercy. Furthermore, in the (admittedly unlikely) case that Google goes down, Google Docs goes down for everyone. There's a single point of failure. Whereas it's hard to imagine Google itself having any real trouble (although the recent AWS outage proves that such a thing is possible), most services aren't Google.

A decentralized application does not suffer from these problems. The failure of a single server in a decentralized system does not bring it down for everyone. Furthermore, users have the ability to switch between different servers or providers if one becomes abusive (or simply stops existing). Users have more choices, and more control.

For Matrix Highlight specifically, this means not having to rely on one specific group or company for storing and managing your annotations or notes.

Federated

Decentralization by itself does not make for useful software. There might very well be multiple servers providing access to a particular piece of software. However, there's no guarantee that users of one such server can meaningfully interact with users of another server. Microsoft's Office 365 has collaborative document editing, and so does Google Docs. However, users of the two services cannot collaborate with each other.

In a federated system, the various providers establish a way of working together. The Fediverse is a big example of this. Users of various Mastodon servers can see each other's messages and posts, despite residing on servers with differing rules and administration. Users of Matrix can send messages between servers, with only one account.

For Matrix Highlight, this means that users who choose to use different servers or providers are still able to collaboratively highlight and annotate pages together.

Self-Hosted

Self-hosting is the practice of running the various software you use yourself. This allows you yourself to be in charge of your data, instead of any other entity, however trustworthy. A popular self-hosted solution is Nextcloud, which may be used, among other things, as a Google Drive replacement that you run on your own server. With Nextcloud, your files are completely under your own management, rather than that of some other person or company elsewhere.

For Matrix Highlight, this means that users can choose to run all the necessary software themselves, and thus remain in complete control of their annotation and other data.

What it Looks Like

First of all, you can watch a little demo video I recorded here:

{{< youtube Q3h5A0DsE1s >}}

You already got a little taste of Matrix Highlight in the opening screenshot. However, I'd like to show you some more of what I have so far. The most important aspect of the tool is the ability to annotate web pages. The tool can be brought up on any page; I typically test it on my blog, but that's just because it's convenient. Selecting some text brings up a little highlighting tooltip:

{{< figure src="mhl_tooltip.png" caption="A matrix highlighting tooltip appearing over one of the sidenotes in a different article." >}}

Selecting one of the colors in the tooltip creates a new highlight of the text you had selected:

{{< figure src="mhl_highlight.png" caption="The result of clicking a color in the previous screenshot." >}}

Annotations applied in this way are shared across all active instances of a Matrix Highlight page, including those shared with other users.

{{< figure src="mhl_multi.png" caption="Two chrome windows with the same annotations." class="fullwide" >}}

Highlights created by users can also be browsed as a list:

{{< figure src="mhl_quotelist.png" caption="A list of highlights from another page." class="medium" >}}

Highlights are stored in Matrix rooms. Since Matrix rooms are effectively chat rooms, they are built for being shared with other users. Thus, it is very simple to give another user access to the current list of highlights.

{{< figure src="mhl_userlist.png" caption="A list of users for a particular page." class="medium" >}}

This also means that a single page can have multiple independent sets of highlights, allowing you to organize them however you like. For instance, if you're proofreading a page of your own, you may have a highlight set (Matrix room) for every editing pass. The rooms can be switched at a moment's notice:

{{< figure src="mhl_roomlist.png" caption="A list of rooms for a particular page." class="medium" >}}

Current and Planned Features

The following are the current and planned features for Matrix Highlight:

• Current: Create and send website annotations over Matrix.
• Current: Store data in a decentralized and federated manner.
• Current: Share highlights with other users, including those on other servers.
• Current: Group annotations together and create multiple annotation groups
• Planned: Use Matrix's End-to-End encryption to ensure the secure transmission and storage of highlight data.
• Planned: Leverage the new m.thread MSC to allow users to comment on and discuss highlights.
• Planned: Use something like ArchiveBox to cache the current version of a website and prevent annotations from breaking.
• Planned Highlight PDFs in addition to web pages.

Project Status and Conclusion

For the moment, I'm refraining from publishing the project's source or output extensions. This is a hobby project, and I don't want to share something half-baked with the world. However, I fully intend to share the code for the project as soon as I think it's ready (which would probably be when I feel perfectly comfortable using it for my own needs).