Rough Roadmap Of Moving The Blog

I am not updating the blog as much, because the few brain cycles I have left for doing "blog things" are spent moving the blog off of Cloudflare and onto OVHCloud (previously). But I thought I create a little post noting down how I hope this can go, although, as we all know: Life is what happens to us while we are making other plans.

• Deadline: I would love to have the blog moved by end of year. That leaves 7 weekends counting this one to get it done.
• What needs to be moved exactly: The static site, the media assets (mostly images), the api, the database (and, technically, the kv storage).
• KISS: I am not trying to change to many bits at the same time here. I could see using a small laravel app for the API, etc. But I am not touching this before the move is done.
• Local/Prod parity: I wanted to run my local blog dev env close to what gets deployed later.

The following is a status quo snapshot.

Status Quo

Static Site

This is already done! I have managed to setup coolify in such a way that it builds the static bits of my blog on push just as Cloudflare does it.^[1] Coolify provides an app for Github^[2], which basically does all of the heavy lifting here. I think I'm getting to the point where using Eleventy-Img to resize images on build becomes too heavy - although I have to investigate more, why my caches are seemingly not eternal.^[3]

API

This was a little reimplementation project. Most of this was done using Github Copilot.^[4] Not too hard, but since the DB has changed (from sqlite-esque D1 to Postgres) and the runtime has changed, too (from CF's own worker runtime to node) there was some work.

I am still using hono (now with the node adapter). The reason I am using postgres, is because I don't know a lot about the js ecosystem, had the need for a db migration tool, saw that node-pg-migrate is well maintained and I was also curious to use Postgres in a project.^[5]

DB

Done! Or at least done as long as no one is adding new favs/likes/guestbook entries. Hehe. But there exists now a robust workflow to export from D1, convert to Postgres (using pgloader) compatible sql and using node-pg-migrate to create a "seed" migration with all the data from the Cloudflare db (so it gets imported once).

KV

Not sure that this actually needed for the size of my blog. But I have some rate limiting, some caching and similar things in place and spun up a Redis for this, now that Redis is maybe not evil anymore? Did I get that right?

Object Storage/Assets

The big reason I was having trouble finding a hosting company I liked was that I didn't want to handle my images. Having compatibility with S3 is great. As it allows using plugins for Obsidian to upload pictures via drag and drop. I also have a crazy iOS shortcut that relies on the fact that I can just upload an image to an S3 bucket. There is also minio, but that will be a quest for another time. So I just synced the files from one bucket to the other using Cyberduck. Didn't even need to make my hands dirty for that one.

Setup

The whole thing is a relatively simple pair of docker-compose files (one for the local env, one for coolify/prod) and some Dockerfiles (for the api, the blog (on prod), the rest is all coming from the public registry).

One important piece of the new setup is, that I needed to emulate pages functions which I relied upon to make my images and my api available etc. If you don't know: A pages function is a filesystem based routing system, where you can put a file under a path like functions/media/[[all]].js and in there it could look like this:

export async function onRequestGet(ctx) {
	const path = new URL(ctx.request.url).pathname.replace("/media/", "");
	const file = await ctx.env.MEDIA.get(path);
	if (!file) return new Response(null, { status: 404 });
	return new Response(file.body, {
		headers: { "Content-Type": file.httpMetadata.contentType },
	});
}

What that did, was to accept a URL like https://blog.martin-haehnel.de/media/bla.png and turn it into a request towards the media bucket. In this way the bucket itself was never exposed and everything was delivered coming from the same domain. Somewhat similar things were done for the api (a separate Cloudflare worker) as well.

To make the same routes available using my new setup I had to put the static site and the api behind a reverse proxy (Caddy) that deferred to the api or the static site container depending on the route.

:80 {
	# API routes - proxy to blog-api
	handle /api/* {
		reverse_proxy blog-api:8080
	}

	# Media routes - proxy to blog-api
	handle /media/* {
		reverse_proxy blog-api:8080
	}

	# Static files
	handle /* {
		root * /srv
		try_files {path} {path}/ /404.html
		file_server
	}

	# Logging
	log {
		output stdout
		format console
	}
}

In the new system the media routes are also handled by the api which in turns talks to OVH's S3 bucket. In the prod env Coolify has its own proxy for all its services handling certificates (so that's why it reads port 80 up there).

At the moment the VPS version of the blog runs on a temporary subdomain in parallel with the legacy deployment.

So what's missing?

• More testing and a lot more cleanup
• Cost check - I want to wait for my first monthly bill, so I have a no-load-on-the-blog baseline (I opted for a pretty beefy VPS that has predictable pricing but I don't know yet what the object storage will cost me)
• DNS cutover - make the VPS available under its intended domain

So not that much! I am hopeful I can get this done very soon. Amazing to think that I will actually host my own stuff again instead of relying on Cloudflare.

• ← Previous
  DailyDogo 1448 🐶