Back to Blog

Building incremental rebuilds for a Notion-to-site generator

· 6 min read
engineering performance static-sites notion cloudflare bullet-so

The problem started showing up around the time some Bullet.so customers crossed a couple hundred pages.

A publish meant: fetch every page from Notion, render each to HTML, stamp them into a Hugo template, build the site, and deploy it to Cloudflare Pages with wrangler. For small sites this took seconds. For a few hundred pages, it crept past a minute. For the larger ones, several minutes — and the failure rate climbed because each step added more chances for the unofficial Notion API to flake.

What we actually wanted was the obvious thing: only rebuild what changed. This post is about the path to that and the gotchas along the way.

Why naive caching doesn’t work

The naive plan is: cache the rendered HTML of each page, keyed by something, and on republish, skip pages whose key hasn’t changed.

But what’s the key?

  • Page content hash — requires fetching the page first, which defeats the purpose.
  • Last-modified timestamp from Notion — the right idea, but with caveats (more on this in a minute).
  • Whatever the user said changed — Bullet doesn’t get told what changed; a publish is a single “publish my site” button.

Notion exposes a last_edited_time on every block. That’s the timestamp to lean on — but it has interesting properties:

  • It updates when the page itself is edited, not when a referenced page changes.
  • It’s coarse — minute-level.
  • It’s not a strict cache key on its own: a page can contain link_to_page references whose targets were rewritten elsewhere, or sync-block children that changed in a different page.

So last_edited_time is necessary but not sufficient. The cache has to remember enough to detect indirect changes too.

The recordMap cache

The unofficial Notion API returns a thing called a recordMap — a flat collection of every block, collection, and space you touched, keyed by ID, with each one’s last_edited_time. It’s the same structure react-notion-x consumes when it renders.

The idea: cache the entire recordMap from the last successful publish, per site. On the next publish, fetch the new recordMap and diff it.

What lives in the on-disk cache per site:

  • The recordMap from the last build (block IDs → block content + last_edited_time).
  • The list of page IDs that were in the published site (so we can detect deletions).
  • The mapping from Notion page IDs → the slugified path Hugo used.

The change-detection pass

Per publish:

  1. Walk the page tree from the site’s root page and produce the new list of pages to publish, respecting publish properties on each page (unpublished pages and their descendants are skipped).
  2. Fetch the recordMap for every page that’s currently in scope.
  3. For each page in scope:
    • Not in the cached recordMap → new page, rebuild.
    • Its last_edited_time is newer than cache → changed, rebuild.
    • Any block it transitively references has a newer last_edited_time → indirect change, rebuild.
    • Otherwise → reuse the previously rendered HTML.
  4. For each page in the cache but no longer in scope → mark as deleted; tell Hugo to remove the corresponding .md file.

The third bullet is the subtle one. A page can contain link_to_page references whose target was renamed; the page itself didn’t change, but its rendered output should. We catch that by walking the block tree and checking the latest last_edited_time across all blocks that the page transitively touches in the recordMap.

What it actually skips

Cloudflare Pages’ build step does its own work — Hugo will still walk the content directory, evaluate templates, build the navigation, copy assets. So “skipping” a page doesn’t skip a build entirely; it skips the Notion-fetch and the HTML-render pass for that page.

In practice on bigger sites that meant the publish went from “fetch and re-render N pages” to “fetch recordMaps, diff, re-render only a handful.” The dominant cost dropped from page-rendering to the Cloudflare deploy step itself — which is fine, because that’s bound by the static-output size, not the page count.

Edge cases worth knowing

Cache invalidation triggers. Theme changes, custom-code changes, navigation changes, and plan-level changes all bypass the incremental path and force a full rebuild — because they affect every page, not specific pages. The publish request from the dashboard carries an isRapid flag that we only honor when none of those have changed.

Notion API flakiness. If a recordMap fetch fails, the previous-build recordMap is still on disk. We have a fallback path: serve the cached recordMap for that page and log the divergence. This was added after a 429 storm took down a publish.

Force-rebuild as a debug tool. Customer support kept hitting “stale page” reports, almost always traced to one of: a sync-block, a child-page rename, or a custom-code edit. We added a “force full rebuild” toggle in the dashboard — used rarely, but a release valve.

Cache size. Per-site recordMap can be a few MB on bigger sites; multiply by the number of sites and the cache directory adds up. We keep it on the build server’s local disk and ship a periodic backup of the small bits (the path mapping, the published-pages list) to R2.

Why this approach instead of webhooks

Notion doesn’t offer a usable change-webhook for the spaces these sites publish from. (There’s a Notion API webhook now, but it didn’t exist when this was built, and even today it’s tied to the official integration model that doesn’t fit a multi-tenant SaaS like this.) That ruled out push-based invalidation. Pull-based diffing on the recordMap was the next-best thing — and has the upside that “publish” is the natural sync point: users press a button, expect changes to be live, and don’t expect background activity.

What I’d do differently

I’d version the cache schema from day one. We’ve changed what we store a couple of times, and each migration required a one-shot script that rebuilt the cache for every site. Versioned schema + a migration runner would have made each iteration cheaper.

The “indirect change” check is currently a full walk of the relevant subtree per page. For large sites with deeply nested references, that’s the slow part of the diff. Hash-summarizing each subtree at cache-write time would let the diff be a single hash compare — at the cost of more storage and more careful invalidation when the hashing algorithm changes.

The whole feature shipped behind the isRapid flag so it could be turned off site-by-site. A few months in, it’s the default and the slow path is the fallback — which is the test that matters.