What a month!

This is going to be a new type of post in the practitioner-journey series. It's a rest between tasks, where I'll try to fill in some of the gaps in what we've done so far and describe the steps ahead. I'm calling these posts the Practitioner Inn.

If you've made it through the first two exercises successfully, you've done pretty much everything it takes to run your own cloud infrastructure^[1]. In one sense, you're kinda done already--if you want to take a break for about six months or so, when you get back I should have some really cool stuff that you can deploy just like the static website or the billing alert. But if you'd like to stick around, we should start to look at the general principles that guide us on our way. Let's start with the shape of the system as a whole:

This diagram is a functional view--it shows the different elements in terms of what they're for. We see the website, the visibility system, and the content management system (CMS). To put this in perspective of a system like instagram, you're interacting with the website when you're looking at your friends' posts; you're interacting with the CMS when you're authoring a new post; and you're interacting with the visibility system when you see the number of likes that your post gets. Obviously these different systems can take many forms and they can overlap with each other-- platforms like Wordpress and facebook embed CMS functions directly in their website systems. We're allowed to make those kinds of structural decisions based on the situation and our goals--there isn't an obvious best choice for every situation.

Let's imagine that we've built this system and started using it. It has a similar feature set as facebook--we can post text, images, and videos. We can comment on posted items. We can exchange direct messages with people^[2]. We've been using this system for six months or a year. Over that time, we've posted a lot of things, had a lot of interactions, and we're starting to get a sense of which parts we like and which parts we don't. There are some features we rely on, some that we don't ever use, and others that have turned out to be more trouble than they're worth. We've also seen new systems--ones that have come out since we finished ours--that have cool features that we want but don't have yet. It's time to do a refresh.

Let's take that perspective--"we want to do a refresh"--and try to use the functional diagram above to decide what we're going to do. Where can we make changes? Where is it risky to make changes? If we want to experiment with a new idea but we don't want it to take up all our time, or cost a bunch of money, how should we design it and where should we put it?

That diagram is pretty bad at answering those questions, isn't it? It seems likely that no matter what changes we want to make, the functional diagram of the system stays pretty much the same. We'll still want to be able to author things, publish things, and interact with people. We might want to change how the system enables us to do that, but these basic goals haven't changed since Myspace and Geocities. And since those early services, the upgrade path available to an average person has included two options: you can take whatever new developments the service operators decide to offer, or you can migrate to a newer service. If you want to migrate, you can either leave everything behind or you can try to find a way to bring it with you. Often the service operator deliberately makes it difficult to bring things with you because you're important to their revenue stream. Even if you were using something like Wordpress, the situation wasn't much better. Anyone who's done it can tell you that after a year or so of fiddling with plugins, a site would get harder to upgrade and modify. And since Wordpress is an all-or-nothing deal--it's either running on a server or it isn't--the question of how to maintain an evolving site can be tricky.

Before we accept this dynamic as a fundamental law of computer systems, let's see if we can imagine something better. Is there a view of the system that would make it easier to answer questions about upgrades and modifications like I raised earlier? Let's look at another way of diagramming the system--at the same high level of generality--and see if that helps.

This diagram focuses on state rather than function. Instead of dividing the system into different parts based on what each part does, this view describes the system in terms of what each thing is. It also helps us answer the upgrade questions much more easily. I claim that, when we use this state view of the system, everything in the top box is basically irreplaceable, and should be preserved with the greatest care. Moving down to the other boxes, things get progressively less critical--if we lose all our HTML pages or all our resized images, we can just regenerate them. It would suck to lose all our log data, but we'd get over it. If some smug CS grad invents a better function for rendering markdown into HTML, we can swap that in without worrying too much. If Amazon keeps union-busting, we should think nothing of moving all our networking (and our money) to a more ethical company.

This leads us to the first principle that will guide all of our design efforts: state floats. We will always identify which things within the system are artifacts of our attention--the things that we make ourselves as humans. Protecting and preserving those things is our top priority. If our tastes change--if we write a bunch of Twilight fanfiction that we end up embarassed about--we should be able to delete all the HTML pages that reference it without having to permanently delete the original text sources and disown the person we were when we wrote it. If we want to try out a new feature, we should be able to set it up alongside the existing system without getting our shoelaces tangled.

The next few exercises in this series are going to introduce an example blogging system--the one that powers this very website. We're going to see how both of the diagrams in this post--the functional one up top and the state-focused one we've just seen-- are useful ways to think about these systems in different contexts. We'll get into the specific building blocks out of which these systems are made, and we'll pay special attention to techniques for making incremental progress--ways of improving our system slowly over time without having to blow it up and start over. If we're lucky, we'll hit a tipping point where the systems we're making stop being purely experimental and become real, useful parts of our online presence.