Tales from this website's update
I spent half of November (and then a few days on top of that) working on my website. Yes, the very one that you're reading this on! While the setup that I have is very specific to my preferences and presents no value to others, I have experienced a few... revelations in the process that can be interesting. So I want to share.
The goals of this update
So, just to set the scene, why was I working on the website and what problems was I trying to address? Like a barber who ends up having a bad haircut, the old version of this website was a bad representation of myself and my work, despite me being a web developer in the past life. A professional page should be like your business card, and mine was failing at giving people the information about who I am at this point in my life, what I do, what I've done, and how to reach me.
On top of that, the entire website was rendered client-side, using content fetched from my CMS. Pages were but shells for data to be loaded afterwards. This means that whatever content I put on these pages would not get indexed properly by search engines. And sharing links would be problematic, as no custom open graph markup (used by all platforms for embeds) could be provided this way. And the thing is, I wanted to start using my blog again.
While I post about my work on Patreon, it has absolutely awful text editing tools and presentation. I can't even embed code snippets, which as a developer is an absolute turndown. Your own blog gives you unlimited capabilities with presentation, and so I needed to make sure I can use mine for more than just occasional musings.
So, to wrap this intro up, I have achieved all my goals. I even ported the three articles from this year back here:
- How to rewrite someone else's project (in another language)
- How to create the core mechanic from The Golden Idol games
- Shader-styled UI (in Godot) (has code snippets now!)
So what did I learn?
Shadow DOM is ready for server-side rendering
This website uses web components as a simple and native way to define custom elements on the page and attach logic to them. I like the component approach to web development, but don't want to get trapped by the constant update cycle of React and other such libraries. So I chose web components and lit-html as a light wrapper over JavaScript classes to make the process of authoring components easier. My main goal here was to enable server-side rendering without reworking the entire website. I quite like my setup!
So how do you take a Shadow DOM driven web page and make it into a ready-to-use HTML file?
When I was first setting this up, Shadow DOM was relatively new, but in the process of being widely adopted. So I decided that it's okay if some older browsers can't handle it. Chances are, people who would be more likely to open this website are using up-to-date apps. Things are quite similar today if you want to use Shadow DOM with server-side rendering. The standard is called "Declarative Shadow DOM", and it allows you to define web components directly in HTML, making it possible to pre-render the entire layout before sending it to the client. And it should work without JavaScript too, which is always a nice bonus!
As of January 2024 Declarative Shadow DOM is supported by all major vendors, which makes it fine to use in my book. But I don't really want to leave people with slightly outdated browsers without access to my brilliant content. Worse yet, I have a 10-year-old iPad, which cannot go past iPadOS 15.8, and that specific version doesn't get to enjoy DSDOM. Can you imagine not being able to open your own website?
The library that I use, lit-html, comes with experimental support for server-side rendering, utilizing the new and cool Declarative Shadow DOM approach. Good, means I should be able to just slap it into my build system and it'll all "just work". Right? Not quite, but more on that below. For now, let's finish with the setup. Say, we do have it working. So what about old browsers?
Well, there is this ponyfill that converts components with template into proper Shadow DOM. The library even recommends it! But be careful with these instructions, as they are outdated. At some point, between being an experimental feature and a standard the property used to detect the feature was renamed. There is this good guide about DSDOM in general, but the gist of it is, do this:
if (!HTMLTemplateElement.prototype.hasOwnProperty('shadowRootMode')) {
hydrateShadowRoots(document.body);
}
There, my old iPad can open my website now!
Client and server need the same context
When you use components with server-side rendering, you want the same scripts to be able to "attach" themselves to the page elements which are written in explicit HTML markup. That's called hydration, a process of reassociation of the fixed data and the dynamic script. I've relied on such things before, of course, but not in my own janky setup. Here I completely forgot an obvious fact that for this to happen both server and client need to have the same idea about the page being rendered.
In other words, although we don't need to render the page on the client initially anymore, it still needs to be able to perfectly reconstruct itself as if there was no server-side rendering at all. If you design your components well, it shouldn't be that much of an issue. In most places you pass data from somewhere on top and just render it deterministically. That passed data doesn't need to be stored in the rendered markup in any way, as it comes from higher-order components.
But at some point you do reach the component that is the source of this data. In the old version, it would just fetch the data on the client and render it. Now the data must come from the build system, and then somehow we need to tell the client about it without making extra requests. So I had to come up with a way to serialize it. My first successful attempt was a ridiculous one in the hindsight.
Say, your page is constructed from blocks, and you have a list of them, and then render that list using unique components for each block. I would set everything up in a way where each block's component serializes data for its component. By the way, forget about complex data. No objects or arrays (at least in their raw form). You have to turn them into some sort of strings. So I had to rewrite parts of these components to make them much more annoying to use. Oh-oh! On top of that, I still needed to let the parent element to know about its state by reparsing these components during the first render and reconstructing its internal data based on that. I called the process adopting, so mad and so beautiful...
Wrong idea, though. And one good rubber-ducking session with a friend later I realized that I should just serialize the entire page data object wholesale, and then I only need to decode it at the very top of the entry component. Everything else would render like before, without ugly hacks and tricks. No data adoption, with objects and arrays. Second time's the charm.
The serialization logic? Ah, just this Base64-inspired beauty:
function stringifyObject(object) {
return btoa(encodeURIComponent(JSON.stringify(object)));
}
function parseObjectString(str) {
return JSON.parse(decodeURIComponent(atob(str)));
}
// Hack for the server side.
if (typeof window === 'undefined' && typeof Buffer !== 'undefined') {
global.btoa = (str) => {
return Buffer.from(str, 'binary').toString('base64');
}
global.atob = (buff) => {
return Buffer.from(buff, 'base64').toString('binary');
}
}
So we have a clean way to achieve parity now. But...
Some libraries are stupid (DOMPurify vs sanitize-html)
Hydration wouldn't always work still. The main source of these problems was the element responsible for rendering Markdown. Being the kind of person who goes extra step to make things proper, I was sanitizing the output of the Markdown renderer. Highly unlikely to be an issue, but why not, thought I. With my new server-side setup, however, a problem manifested. The library that I used for sanitization, DOMPurify, was client-side only. Sure you could hack around this limitation by using something like jsdom, a facade of APIs that imitate what is available in the browser on the Node side of things. But that felt extreme.
No worries, there is a server-side Node module called sanitize-html! It used to run on clients too, but support for that was removed. You could still try to do that, but you'd need to carry a whole bunch of dependencies to the client side, which, again, felt extreme. So I opted to use both. At first glance, both do a very similar job. But the devil is, always, in the details.
Did you know that DOMPurify consistently reverts the order of arguments in HTML elements? This has been brought up a few times (1, 2), but maintainers decided against fixing the issue. While I empathize with open-source maintainers, the arguments given are very frustrating. They make a point that browsers already can mess up the order, so there is no point fixing it on the DOMPurify side.
So, how to work around this unsolvable problem? Just run the sanitization twice. Because that will revert the order twice, giving you the original order. Can some browsers still break it? Sure, I suppose. But it's nice to at least eliminate the issue introduced by the library itself, you know?
DOMPurify.sanitize(DOMPurify.sanitize(dirtyHtml));
Another fun problem is that DOMPurify is enforcing a standard-complying behavior when it comes to self-closing tags, but sanitize-html enforces the exact opposite. As you may know, there is no such thing as self-closing tags in HTML. But historically there was a side-set of HTML, XHTML, that tried to merge principles of XML and HTML together. XML does have a concept of self-closing tags. If your tag has no content, you can omit the closing counterpart and just write <tag />.
And so this became a preference/habit of some web developers to write <img />, <input />, or <br /> the same way. And browsers support it too, and forever will! But ideally, you should write <img>, and <input> and <br>. And so, our two libraries ended up on the opposite sides of the barricade on this issue. And neither of them is configurable, with both of them enforcing their preferred behavior for all input values. Well, nothing a replace call cannot fix.
NVM is bad with servers
When you have a web server with many services, you inevitably want to have granular control over the stack each service uses. In our case, that would be the version of Node.JS. Normally, I'd just install one version from NodeSource distros, but that one is always installed globally. So to the rescue comes the recommended way to manage your Node these days — NVM, Node Version Manager.
On the surface, it's very neat. You use command line to install as many versions of Node.JS as you want. They all get installed for your specific account, and you can switch between them quickly. It has a dark secret, though. It's not an application. It's a shell script, with a bunch of tricks to force the environment for your interactive shell. It works for you. But it doesn't work for an account that has no interactive shell.
Enter, systemd services. The environment in which the service runs is not an interactive shell. So changes that NVM would make to enable itself are never made. Which is fine, you can specify the full path to the version of node that you want to use. But it all breaks apart when something inside of your scripts, or scripts from the modules that you use, tries to interact with the node or npm executable. Suddenly, it all points in a wrong direction. You may have some luck with manipulating the environment variables, like PATH, for the service.
EnvironmentFile=/path/to/service.env
ExecStart=/home/username/.nvm/versions/node/v22.12.0/bin/node /path/to/script.js
But do you know what else isn't an interactive shell? This:
sudo -H -u username bash -c 'node -v'
This command does not load the bash profile for the specified user. Experts say there is a very good reason for it, interactive shells have different expectations. So, it is also oblivious to NVM. If you enter interactive mode NVM works (provided it is installed for that specific user), if you don't — you get system node. The reason why this is so relevant for me, is because each server has its own non-loginnable user. And this user owns the files, so you have to do some work on its behalf.
~$ sudo -H -u username bash -c 'which node'
/usr/bin/node
~$ sudo -H -u username bash -c -i 'which node'
/home/username/.nvm/versions/node/v22.11.0/bin/node
By the way, did you know that when using child_process.spawn various system commands can behave differently depending on whether you set the shell flag? For example, cp will not find any files unless you set it to true, but ls/dir will do directory listing for the same path just fine.
import { spawn } from 'child_process';
spawn('ls', ['-la', './out'], { stdio: 'inherit' });
// Prints a listing for the out directory.
spawn('cp', ['-rv', './out/*', process.env.BUILD_COPY_DESTINATION], { stdio: 'inherit' });
// What out directory?
spawn('cp', ['-rv', './out/*', process.env.BUILD_COPY_DESTINATION], { stdio: 'inherit', shell: true });
// Ah, _this_ out directory? Yeah, I know it.
You can use social media as comments
Finally, something really cool! I got this idea from @passivestar. He spotted a library that allows you to add a comment section to your blog by matching one of your Bluesky posts to the article and displaying all replies and reactions that you receive. Basically, you post, you share that post, and the widget fetches interactions from Bluesky to you sharing.
I was in love. I felt like my website is missing user interactions, especially if I'd post something more technical. But I didn't want to integrate some service or set up some custom platform and then moderate it. But this, this is a brilliant idea! Your blogposts become alive, and all you need to do is link them to your social media which you already moderate and actively use. Positive feedback that you receive (or questions and criticism, equally) are immediately accessible right next to the article.
What I didn't like is that the library that passivestar used (this one!) carried the entirety of React with it. That's a hefty price to pay for a small comment section at the bottom of a page. And I was also eyeing a possibility to include Mastodon into the mix. Both social media platforms provide open and public API that serves data without authentication. And that's the beauty of it all — requests are made by users who visit your blog, the load is distributed and is unlikely to get rate limited.
So this Sunday I sat down and made my own comment section, that hooks into both Bluesky and Mastodon. The cool part of it is being able to mix them together, and provide support for unique features, like Mastodon's custom emotes and animated avatars. I tailored the whole thing to my needs, but let me know if you would like to know more about integrating either of the platforms!
Thanks for reading!
I could've told many more things about the process, but we're running out of ink! Frankly, web development is equally frustrating and exciting, and boy how excited I am now that it's all done. wink wink
Please let me know what you think! And expect more blogging from me :)
Cheers!
Loading comments and feedback...