Thomas Green

Embedded systems · Self-directed

An e-paper display that summarises a paper notebook

I write on paper and rarely look back at it, so I cut a recess into the back cover of a notebook, fitted an e-paper display, and set it up to show summaries of the recent pages.

Context
Personal project, built for my own use
Role
Sole author, whole system
Hardware
Raspberry Pi Pico, Waveshare e-paper
Code
smart-notebook
The inside back cover of a hardback notebook, hollowed out and lined with black moulded trays. Set into it are a lithium-polymer pouch cell, a small blue charging module, a Raspberry Pi Pico and a Pico e-paper driver board, wired together with fine enamelled copper wire, with a ribbon cable running out to the display. Cell Pico USB-C charger Driver board Ribbon to panel
Everything lives in the back board of the notebook: cell, charger, Pico, driver board, and the ribbon out to the panel. The cover still closes.

Why I built it

This one is a personal build. I think better on paper and keep a notebook properly, but a fortnight later the contents may as well not exist, because nothing on a paper page is searchable. Digital notes fix that, but they lose the reason I write on paper in the first place.

The idea was to keep writing on paper and have the notebook do the recall itself: an e-paper display in the cover, showing what the recent pages were about and what the week looks like, updating without the notebook ever being opened.

An early build stage outdoors on a weathered wooden bench: the notebook's board covers split open and hollowed, with filler smoothed into the recess, and a Raspberry Pi Pico and two flat battery cells laid in place to check they fit.
Checking the recess before committing. Most of the difficulty in this build was mechanical rather than electrical.
Close view of the blue Pico e-paper driver board with its GPIO labels visible, a flat ribbon cable running from its connector into the edge of the e-paper panel behind it.
The driver board, and the ribbon into the panel. Three-colour e-paper, so the palette is black, white and red and nothing in between.

How it works

The chain is longer than it looks, because none of the parts talk to each other natively.

System diagram. An open notebook with the 7.5-inch e-paper panel set into the inside cover, facing a handwritten page, and the driver board, Pico, USB-C controller and cell in the back board. Pages pass through a capture stage producing plain text, a compile stage of four language-model calls, a render stage producing a one-bit image, and a display stage where the Pico drives the panel, with a return arrow carrying the finished summary back to the e-paper.
The chain from a handwritten page to the panel in the cover. Four separate language-model calls do the reading, the browser does the typesetting, and the Pico only ever sees a finished one-bit image.
  • Pages get captured and turned into plain text, dropped into a watched folder.
  • A file watcher notices the new file and triggers the compiler.
  • Four separate language-model calls run over the text: correct the spelling, produce a short title, summarise in under forty words, and pull out a date. Splitting them up rather than asking for all four at once made each one far more reliable.
  • The results append to a JSON store, which a small local web page renders alongside a weather forecast.
  • That page gets screenshotted headlessly at exactly the panel's resolution, then reduced to a one-bit-per-channel image.
  • The Pico pulls the image and drives the panel. E-paper holds its state with no power, so the display only costs anything at the moment it changes.

Rendering to a web page and screenshotting it is the decision I would keep. Laying out mixed text and icons directly on a microcontroller is painful, and HTML and CSS already do it well. Letting the browser do the typesetting and treating the Pico as a dumb display removed most of that work.

Getting the image onto the microcontroller

The awkward constraint is memory. A full-panel image is larger than the Pico comfortably wants to hold, so a chunk of this project is a set of compression experiments written in C, trying to find a format that is small enough to move and cheap enough to decode on a device with almost no memory to spare. That directory taught me more than anything else in the repository.

The e-paper display in the notebook cover, showing a seven-day weather forecast in red across the top with icons and temperatures, and below it a generated title and summary paragraph about recent news, with a date and page number in the margin.
The output, straight off the panel. Forecast across the top, generated title and summary underneath.

Where it fails

The photograph above shows the failure modes rather than a staged best case. The date reads 01/01/1999, which is the sentinel the code returns when the extraction step can't find a date and gives up. The page number next to it is wrong. And the title is not the short one the prompt asked for, it is most of a paragraph.

All three are the same class of problem: a language model asked for structured output in a fixed format, complying most of the time but not always, with a regex behind it doing the actual parsing. The fallbacks work, so the display renders something sensible instead of crashing, but what is on the panel in that photograph is the fallbacks firing. If I rebuilt it now I'd use constrained decoding for the fields that have to be machine-readable and stop trying to parse prose.

← All projects