The end of software architecture?

The decisive factor: context

You could almost say that AI agents are human, too. Their performance does not depend only on the size of the brain or the model. Just like us, they become overwhelmed when the context grows too large. If you feed them a lot of information without making clear what matters and what does not, they pile it into a diffuse mass of knowledge that works against good problem-solving.

An AI system’s context capacity is limited. Depending on the model and provider, it currently ranges from about 128k tokens to 1 million. Beyond the size itself, the current fill level is a crucial indicator. It shows how much cognitive stress the model is under. If the context is moderately filled, the agent has a stable base to work from. If the context is nearly maxed out and the model has to keep too much in play at once, it comes under stress and starts making mistakes. In the worst case, the context is not just full, but full of things that do not matter.

Modern agents already make it possible to actively manage this context through analysis, monitoring, or deliberate condensation, sometimes called controlled compaction.

Lost in the middle

One fascinating effect in working with AI agents is the phenomenon known as lost in the middle. Information inside a long context is not weighted evenly.

Models typically prioritize:

• information at the beginning of a context, such as instructions,
• as well as the most recently seen content, the recency effect.

Information in the middle tends to lose relative importance and is more often overlooked or weighted less strongly.

This is primarily a byproduct of large language model architecture. At the same time, training data reinforces the pattern, because tasks often appear at the beginning while relevant outcomes appear at the end. That is why this behavior can sometimes look surprisingly intelligent. It mirrors a kind of prioritization that humans also show.

A simple example: when stopping at a red light, the general traffic rules, comparable to long-term knowledge, and the current state of the light are what matter. Which traffic lights you saw before is irrelevant.

In that sense, the implicit prioritization of beginning and end can actually be useful in some contexts. What matters far more is that you can steer this effect deliberately through context management.

Context management as a core skill

This leads to a few clear principles for working well with agents:

• structured information instead of uncontrolled volumes of data, so the context does not overflow
• small, iterative work steps, because they require less context
• a clearly defined work context, with small well-structured modules directly related to the current task
• regular recalibration of the context, through controlled compaction or a context reset via a fresh chat
• put foundational information first, and give explicit hints when something important sits in the middle, so you can intentionally counter the lost-in-the-middle effect

That makes context management a central capability in working productively with AI agents.

And what does this have to do with software architecture?

A great deal. Good software architecture does exactly what an agent needs: it structures context.

It separates:

• stable parts from variable ones
• core logic from details
• foundational information from less important information

On well-structured software, an agent can work with precision and efficiency.

In a poorly cut monolith, by contrast, an agent has to keep large parts of the system in view at the same time. Redundant code, for example, takes up context space for no reason.

That leads to:

• higher context consumption, longer reasoning time, and higher cost
• lower precision
• more errors

The illusion of quick wins

AI can temporarily conceal bad architecture. An agent finds duplicates, fixes bugs, and produces visible progress quickly, especially where humans are already overwhelmed.

That tempts people into the dangerous conclusion that architecture is losing importance.

In the long term, the opposite happens. Without structure, complexity rises disproportionately and the agent quickly reaches its limits. The quality of the results begins to drop. And once you get there, you have a real problem: the software structure is no longer opaque only for humans, but for the AI as well.

The code gets “improved” into something worse.

The real consequence

Architecture is not disappearing. Its role is changing. Instead of being only a communication tool within the team or community, it is becoming the foundation for efficient AI-supported development.

In the future, good architecture defines the context of the AI. It determines whether an agent can work effectively or fail. An AI agent can follow a given architecture surprisingly well. It will rarely violate architectural rules when those rules are clearly visible, especially when they are communicated upfront.

For mature legacy code, it often makes more sense to use AI to move it toward a cleaner architecture first, instead of unleashing agents on it immediately to implement new features.

Conclusion

AI will not make good software architecture obsolete any time soon. Anyone who wants to use AI agents intelligently and efficiently needs an architecture that tailors context to the agent. Agents make good architecture visible and operate with precision and efficiency when the structure is sound.

Sources

• Lost in the Middle: How Language Models Use Long Contexts