Anchoring Our Mental Model

I remember being on a high floor down in one of our Boston buildings, looking out at the sunny blue sky and having a coffee with one of my favorite people to exchange crazy ideas with, Niko. I went on another one of my exploratory thought exercises: how the heck are we going to manage our mental model if AI is going to write all our code from Markdown files? It feels so unstructured and so different from how we have thought about coding thus far.

Back in the Butter-Churning Days

Put yourself back in the era of artisanal coding (I know, hot-topic term, but I like that it makes hand-written code sound like churning butter).

Imagine you’re a lead engineer and you’ve got a system to build. When you do this, you go through a series of steps. You’ve got a starting point, which is understanding the problem you’re solving, then designing the system you will build. You then decompose the design, figure out how to layer the work, and divide the work of building the system. This whole process is not done in one day; you work through it over time.

As you work through it, you build the mental model in your head as you build it in your documentation. The slow human process is on your side to give you time and exposure to absorb it. By the time you get to coding, you have a working mental model in your head and in your documentation, and can confidently move forward. Team members then pick up these design and coding tasks in parallel and the team iterates until the whole system is built.

The mental model of the lead engineer is a powerful tool. Best case, multiple team members share the full mental model. However, not everyone does or has had to. A person’s mental model is scoped to the size of the work they own. For example, someone starting as a junior engineer on the team may have a feature-level mental model. Compare that to someone who has been on the team longer and owns work across systems; their mental model reflects that.

A Strong Mental Model Is a Shared Model

What makes a shared model possible is the application of distributed cognition to system design. In the words of Hollan, Hutchins, and Kirsh, who brought the idea into human-computer interaction research:

“Cognitive processes may be distributed across the members of a social group. Cognitive processes may involve coordination between internal and external (material or environmental) structure.”¹

An example is when we draw a diagram and make the relationships between system components clear and available to everyone without anyone having to hold it all in their head. Everyone brings a different mental model to the game, and another engineer could look at it and identify a missing consequence of a relationship that the author missed. The group works through contradictions, tensions, and alignments together while using the diagram and documentation to maintain and track the reasoning. Going through this process refines each participant’s mental model; it connects shared reasoning back to our own understanding. The act of reasoning, assessing whether a choice is acceptable, is our human judgment. Its results and details are key elements of design documentation.

And when we code, iterating, debugging, and fixing add to our mental model. We make choices and fold them back into the system, baking our mental model and judgment into the code base.

Help, Agents Killed Our Mental Models!

Agents showed up, and engineers have been doing all kinds of things ever since to try to figure out how to maintain control over development. We’re all supposed to move fast, but more of the rules of the game changed than speed. For many, the message landed that we are slow at coding, so let agents code instead, but what does that mean for the rest of the process, since building is more than coding?

The frontier models are excellent at writing full-stack systems and building them in a fraction of the time it took us slow humans. Not only would looking at every line of code be a huge time sink, but we also can’t be sure what design the AI came up with, if it solves the problem, and if there’s anything in there that would be a bad surprise. Because a lot of it works, though, it can give the illusion that we can be more hands-off, but the key is where we decide to be hands-off.

Even if we’ve built the design through the process outlined above, giving us a strong starting mental model, we could lose touch with it once the agent does all the coding, testing, and debugging, depending on how we convey to the agents what that mental model is. Aside from the human-to-agent design hand-off ambiguity, we also lose the hands-on aspect. It was always a key learning loop for engineers for iterating on building and refining design.

Instead, we have agents making decisions and building in tight loops since we want them to move quickly. But where are these decisions and choices going? Since our diagrams and documents are traditionally human-managed, we can’t update them with context we don’t have, letting them drift as fast as agents can build. We can use AI to analyze the code base and produce reasoning for updating our human-managed documents, but this adds extra layers of work instead of giving us less.

Bringing us to this core question: How do we have a mental model for our system if we are not the ones writing the code?

Putting Our Mental Model Back Where It Belongs

I see a pattern of people putting the mental model at the end of the whole process. One example is attaching design context or specs to code reviews so that humans can reconstruct the agent’s judgment after the fact. While this is an improvement on the approach of meticulously manually reviewing agent-written code, it is slow, and both are backwards to me.

Inserting yourself into the process this way is like stepping in to inspect butter created by a butter plant to figure out what kind of milk the plant used because you didn’t churn it yourself, and checking it for bugs before it gets wrapped and shipped.

Instead, let’s define up front what milk to use, and build our infrastructure so that bugs don’t end up in the resulting butter.

This is one of the main areas I address with Anchor Engineering: I put the mental model at the front of the process with the Anchor Book, and make it part of each subsequent step.

We are still building our mental model as we have before, now in a compressed timeline, working in tight loops with each other the same way we work with our agents. When we do hands-on work together where everyone who needs to weigh in is in the room, we can see gaps and correct them quickly, align our mental models, and put the results back in the Anchor Book. The mental model stays up to date. The agents work the same way: when an agent makes a decision during implementation, it records it in the Anchor Book. When it hits a conflict or something that needs our judgment, it surfaces the decision to us and codifies our answer. The choices that used to disappear into agent sessions land back in the shared mental model.

Now here is something really different from before. As the team builds out the design and works through alignment and judgment, developing and saving their mental model, they are doing this as a group. Everyone from team lead to junior engineer is part of the same design loop, so everyone develops a mental model of the whole system and can own larger pieces than they otherwise could have. Not only that, but if people in the room are more experienced at design and the team is constantly designing as a group, it becomes a crash course in design.

As an industry, we have been so worried about how we develop the next generation of engineers if they don’t code. Coding was only ever part of the job; the bigger part was always building mental models, learning, abstraction, and applying what you learn. I wrote about this before AI was writing our code.

We don’t build with less understanding; we get to understanding faster.

Our Judgment Is the Anchor

The Anchor Book is the design. It is the shared mental model with your team, and it evolves as team members build and share their mental models. It becomes the repository for the collective distributed mental model of the team, as well as a coding artifact that the agents use as their mental model. In a sense the agents and the team members are forming this mental model together, putting them all on the same plane.

However, the agents and the humans differ in their contributions; the humans define the judgment in the Anchor Book, and the agents define the implementation and updated status of the code base. An anchor is the differentiator: our judgment.

In the butter example, the human judgment of what milk should be used can be represented as an anchor for the AI to ensure remains true in the resulting system, meaning it builds tests and validates that it holds. The goal is to be able to trust your system to apply your judgment correctly, so that you don’t need to go poke around yourself to figure out if your system picked the right milk or not.

A Stepping Stone

Anchors and the Anchor Book add structure to Markdown. Structure lets us check, with evidence, that what must hold actually holds. I am hoping this is a stepping stone towards an AI-native modeling language: one that people reason through, agents build from, and the system can verify.