notetaker
2026-10-29

In recent years (i.e. since the pandemic) it's become very common for math lectures to be posted online, and there's now a substantial amount of work which is available only via these recordings. I do not find this situation satisfactory, since it's a pain in the ass to comb through a video – or worse, series of videos – to find a specific result, especially compared to the ease of scanning through a document. (Let me not complain too much about this, though, since having these recordings available is still much better than having nothing.)

This was the motivation for notetaker, a tool for autoscribing math lectures from recordings. The way it works is pretty simple: since current AI models are good at math but can't natively ingest video, we just provide them with a transcript of the text, together with timestamped images of the board just before it's switched or wiped clean. (The transcript quality is generally not great, but the models are clever enough that this doesn't seem to be a problem in practice.) There's a bit of extra scaffolding to help with things like writing style, drawing diagrams, and other details, but overall the models do a decent job on their own. I figure it won't be too long before we get mathematically-capable models which can economically watch videos, at which point notetaker will be obsolete, but I hope people find it useful in the meantime.

My first test case (and motivation) for notetaker was Clausen and Scholze's lecture series on analytic stacks. With their permission, I've made the output available at dmanam/analyticstacksnotes. It's clear here that the error rate is higher than a human notetaker's, but I've still found the notes useful. One aspect I find particularly pleasant is the presence of a timestamp alongside every paragraph, something which makes it very easy to refer back to the videos and which would be impractical to include in manually-typed notes.