This converts Markdown from md/ to a static site in out/. The normal workflow uses the pinned Podman image defined by Containerfile.
AI helpers: read
AGENTS.mdbefore making changes.
For a normal Markdown change:
md/, or create md/PageName.md with a level-one title../OtherPage.md; the generator rewrites them for the published site.md/PageName/ and reference them as ./PageName/file.png.make container-build.out/Index.html and review the generated page.make container-build builds the image from Containerfile when needed, runs the complete build in that image, and writes the generated site to out/. It includes the pinned Markdown, syntax-highlighting, and diagram-rendering tools, so no site-build tools need to be installed on the host.
make container-build
xdg-open out/Index.html # or open on macOS
The builder uses rootless Podman and leaves out/ owned by the invoking user. Markdown, media, and archives stay outside the image build context; the repository is mounted when rendering runs. It builds into a fresh staging directory, normalizes timestamps and permissions, and then replaces the generated tree so stale files cannot survive between builds.
tools/sitegen.py performs one deterministic site-wide pass: it parses each Markdown document once for backlinks, then writes the pages, index, and RSS feed. The build fails when generated HTML refers to a missing local page or asset.
Deployment assets stored alongside the Markdown, including .htaccess, are copied into the generated tree. The Apache configuration makes Home.html the directory landing page.
Lossless masters for optimized site images live in source-assets/. They are retained in Git but excluded from both the container context and published site.
For a new article, also add a dated entry near the top of md/Home.md's Updates table. To run the full checks before committing:
make test
make verify-reproducible
The Containerfile pins both the Debian base-image digest and a dated Debian package snapshot. Update those two values together only when intentionally refreshing the toolchain.
Fenced code blocks use their language label; inline code keeps the normal page styling.
| Fence language | Output |
|---|---|
clojure or clj |
Clojure highlighted by Pygments |
| Other languages supported by Pygments | Highlighted with the same restrained palette |
mermaid |
A static SVG diagram |
| Unknown or omitted | Plain code, unchanged |
tools/codeblocks.py processes the code blocks emitted by cmark. It renders identical diagrams once per build, using one Chromium process for the batch. Mermaid embeds the fonts in each SVG; stable diagram IDs and content-hashed asset names keep output reproducible and invalidate browser caches when a diagram's appearance changes. Malformed diagrams fail the build with the source filename.
Give diagrams an accTitle and, when useful, an accDescr. The title supplies the image's alternative text. The renderer's theme lives in tools/mermaid/render.mjs; code colors live in tpl/style.css. Only HTML, CSS, and SVG are deployed.
| Build dependency | Pin |
|---|---|
| Python, Pygments, Chromium, fallback fonts, cmark | Debian image digest and package snapshot in Containerfile |
| Node | Separate image digest in Containerfile |
| Mermaid CLI, Puppeteer, Mermaid, bundled fonts | tools/mermaid/package.json and the complete npm lockfile beside it |
tools/mermaid/stable-css.patch makes Mermaid CLI 12.2 load font stylesheets in a fixed order; concurrent insertion otherwise changes the embedded SVG bytes intermittently. The image build applies the patch and fails if it no longer fits. Revisit it when upgrading Mermaid.
After changing the npm dependencies, refresh their lockfile with npm install --package-lock-only --ignore-scripts --prefix tools/mermaid. Rebuild the image and run the normal tests and reproducibility check. Diagram rendering itself works offline.
| Source | Generated page | Discovery |
|---|---|---|
drafts/Page.md |
None | Kept outside the website |
md/_Page.md |
_Page.html |
Shareable by URL; omitted from the index, feed, and backlinks |
md/Page.md |
Page.html |
Listed in the index; announce it in Home's Updates table when ready |
An underscore-prefixed directory also makes its pages unlisted: md/_previews/Page.md becomes _previews/Page.html. Keep the normal level-one title; the underscore belongs only in the path. Editor backups and lock files remain excluded.
Build with make container-build, inspect the HTML in out/, and deploy through the manual Deploy workflow in Forgejo. For example, md/_Example.md will be available at https://tailrecursion.com/~alan/_Example.html after deployment. This uses the ordinary static-site build and deployment, with no separate preview server. The manual deployment publishes both the website and its source mirror.
Previews can link to published pages and to one another. They do not add backlinks to those pages. The build rejects links from listed pages to previews, including links in Home's Updates table, so previews cannot accidentally appear in the updates feed or Explore tiles. To publish a preview normally, remove the underscore and add its new path to the Updates table.
Preview pages include noindex metadata for search engines. They are unlisted, not private: anyone with the URL can read them. Keep work that should not be deployed in drafts/.
Forgejo CI uses the pinned builder to run the tests, compare two independent builds, and build the website. It downloads only the LFS media needed by the website. Pushes and pull requests run checks only. The manual Deploy workflow publishes the website and requests publication of its source at tailrecursion.com/repos/.
git clone https://tailrecursion.com/repos/git/homepage.git
The public mirror includes LFS objects referenced by the published history. For a smaller checkout without archive media, use GIT_LFS_SKIP_SMUDGE=1 git clone ..., followed by git lfs pull --include='md/**' --exclude='' when building the website. The source mirror is built by the private alan/repos-publisher workflow after the website upload succeeds.
The host build is optional. It requires cmark-gfm, Python 3.11 or newer with Pygments, rsync, Make, and Git. Pages with Mermaid diagrams also require Node 22.13 or newer and Chromium:
npm ci --ignore-scripts --prefix tools/mermaid
git apply --directory=tools/mermaid/node_modules/@mermaid-js/mermaid-cli tools/mermaid/stable-css.patch
export MERMAID_RENDERER="$PWD/tools/mermaid/render.mjs"
make assets && make
The renderer uses /usr/bin/chromium by default; set PUPPETEER_EXECUTABLE_PATH if it is elsewhere. The container workflow supplies all of these tools automatically.
Set MD2HTML if the local renderer is not on PATH:
make MD2HTML=bin/cmark-gfm
Outbound links can be checked with the containerized toolchain:
make container-check-links
This check contacts external sites and is diagnostic; it is not part of the byte-for-byte build guarantee.
In Forgejo Actions, run Deploy on master. The latest Check run must have succeeded for that exact commit. Select dry_run to build and preview the rsync changes without publishing either destination.
The deployment reuses the CI build workflow and checks eligibility both before building and immediately before uploading. It publishes out/ to the website, then requests source publication under /repos/. The private publisher independently rechecks CI and finishes the mirror update in its own workflow. These are sequential deployments, not one atomic transaction; if source publication fails, the website may already be updated. Rerun Deploy after resolving the failure.
The SITE_DEPLOY_KEY repository secret is restricted by the server's rrsync command to writes within the website directory. .forgejo/known_hosts pins the server's SSH host key. PUBLISH_TOKEN can dispatch the private source publisher. Build and test containers receive neither deployment credential; only the separate media-fetch container receives the short-lived checkout token.
For a local website-only deployment, make deploy remains available. It runs the pinned container build, then uses host-side rsync and Git credentials to publish to DEPLOY_HOST. It requires a clean worktree with HEAD matching its upstream branch and does not request source publication.
The pinned builder image also includes yt-dlp, ffmpeg, and git-lfs for preserving media associated with the TechWorks archive. They are archival tools, not requirements for the local fallback build. Media should be downloaded only when redistribution is appropriate; otherwise preserve metadata, captions, and durable source links.
Full-resolution MP4 masters under md/TechWorks/media/video/ are tracked with Git LFS. Initialize Git LFS before checking out or restoring them, then verify the local object store:
git lfs install
git lfs pull
git lfs fsck
The DreamHost upstream uses Git LFS's pure-SSH transport. Maintainer remotes must use standard ssh://user@host/absolute/path syntax; a colon between the host and absolute path prevents Git LFS from identifying the endpoint.
Preservation-only material lives under archive/, outside the generated website. archive/techworks/ holds event media and source snapshots; archive/open-source/ holds complete repository and gist bundles; and archive/forums/ holds repository-only captures of the wooby accounts on Hacker News, Slashdot, and Halfbakery. ProblemScan and recovered personal-site history have their own adjacent directories.
Verify the complete corpus with make container-archive-audit (or make archive-audit when FFmpeg and Poppler are installed locally). Refresh checksums after an intentional capture with make archive-checksums. To prepare a checksummed recovery directory containing Git history, LFS objects, repository bundles, and preserved files, run make archive-export DEST=/absolute/path.
Network capture is always explicit and resumable:
make archive-fetch-forums
make archive-fetch-github-inventory
make archive-bundle-github
make archive-fetch-archive-org
make archive-fetch-gist-footprint
make archive-fetch-sourceforge-jd4x
make archive-fetch-open-source-evidence
make archive-checksums
The maximum-preservation Git corpus is several gigabytes. Git bundles, WARCs, release archives, and large media below archive/ use Git LFS. A metadata-only checkout can skip automatic LFS downloads with GIT_LFS_SKIP_SMUDGE=1.
The Archive.org fetch maps every archive link in public Markdown to a local WARC response; linked book items retain their public pages and metadata, not restricted book files. The gist-footprint fetch is resumable and records API context for all inventoried gists or selected --gist-id values.