tl;dr: The main struggle in tech today is adopting tools because big tech uses them, because they’re cool.
What scale do we need?
Inefficient technologies drive massive hardware costs. For most usecases, a businesses could run at 1/10th the cost. Standard Ebooks serves 1.2 million req/month, just under a .5 req/sec microfortnight, on a 2GB VPS.
The typical microservice PWA has dozens of internal services pinging each other thousands of times more often, with no benefit to the end user. With Standard Ebooks, if a request fails, you just refresh with no issue and no struggle for availability. They use PHP with FastCGI, an in memory cache and search via APCu cache, which should let them scale 1000x to thousands of requests/second on this same VPS.
Many personal blogs receive thousands of requests a month, easily hostable on any machine with the most inefficient code imaginable. Actually hugging a site to death (slashdot effect, flashcrowd) seems infeasible today - 300 req/second sustained over a month is a visit from literally every American. (I exaggerate. I believe my hosting will fail after 2 million views/month. If I bothered to CDN my images, that should be ~5 million.) Few actually needs to scale.
This leads to an interesting gap in offerings. Every provider’s optimized towards huge customers with insane throughput (who are so big it’s worth running their own infrastructure.) Now imagine a cloud provider giving you limited bandwidth for a month, for $.10. (The closest is Hetzner, €3.79 for a shared vCPU and 20 TB of bandwidth.)
But resume driven development (RDD) is fun and occasionally lucrative! Why else would a company receiving a couple req/sec have a Kubernetes cluster, CI/CD, frameworks on the front and backend and multiple teams of engineers maintaining all those microservices? Answer: Bad CFOs and accountants
Coordination issues also occur. As a organization grows, a platform team makes sense. But goals get misaligned. The platform team’s manager wants to build more tooling to get promoted. The platform engineers lose sight of developer needs, get distracted by fun problems. Example: Me. I first saw this complaint lodged at Ruby’s development team, who write C, who are more familiar with VMs, interpreters etc. than web development. They do a fine job, but not everyone can.
The Frontend Revolt
Many of us are tired. Us = People who only want ‘good enough’ UI, whose differentiators are on the backend or content, not polished interactive frontends. But a new hypecycle is here. Now the “new hotness” is htmx. Easily add interactiveness without reloading, without a full framework, just return html from the server! But isn’t this just the MPA vs. SPA debate? JS vs J-less? And more!
Perhaps the urgoal of hyperlinked static pages is a memory and many sites wouldn’t work without JS, but why should one tool set solve everything? For many usecases, server rendered HTML is actually optimal. Others e.g. enterprise SPA, Figma thinkorswim etc. do demand other approaches.
This dissension leads to disparate places. Some speak against all clientside rendering. They want apps to run in the OS, their browser is for HTML alone. A bit extreme. The key is to temper RDD and use the best, the simplest tool possible for the job, a minimal viable product iterated with customer feedback. We best decide by asking whether our interaction needs the network. Lower latency e.g. GoogleDocs where a button makes text bold doesn’t belong on the network. Generally, formatting input requires clientside rendering. Simpler solutions require less maintenance, less development time, less hosting costs etc. It’s just good engineering.
How did we get here?
We started with MPAs but struggled with interactivity. Frameworks appeared solving interactivity, but we struggled to integrate them with our templating systems. We made the frameworks template the whole site too, unifying our tooling and birthing SPAs.
But Google couldn’t index them, performance dragged, so we added server side rendering to the frameworks, letting them hydrate HTML to the SPA. shudder Fell hydration errors… Now islands mix approaches. But we got stuck in the framework. To actually deploy these beasts, at scale, around the world, Kubernetes etc. appeared.
Also see MDI and SDI: Window as application or window(s) written by application Actually making an SPA work correctly’s quite the struggle, the developer works with limited browser support. While the common libraries help, it’s obvious this isn’t the happy path. The browser provides so much, so why not leverage that? That said, as most tasks are CRUD, our tools should focus on this (with ways to add interactivity.)
Ultimately, the simpler it is for anyone to build something, the more democratic the web, weakening social media’s stranglehold.
To verge away from simple, into megalomaniacal ideas, imagine building server systems on a chip, with a fresh architecture aimed at cloud native workloads, instead of emulating a PDP-11…
Some Examples of RDD
- Wordpress on Kubernetes: Give up control compared to headless, for sweet K8s xp! Now you can’t control how plugins autoupdate, so deploy and configure caching on prod, copy prod, update content, redeploy on prod for every new article!
- SQL vs. NoSQL: Reading cutting edge blog posts about cool problems, engineers started using MongoDB, Cassandra etc. for CRUD, the wrong tool for their domain.
- SPA vs. MPA: SPA’s excel at complex tasks, while SPA’s are too easy…
- Embedded web assets vs. Node dashboards: Admin passages were trivial, but are now separate services (because they scale!)