From Demo Code to Production Quality

With modern language model tools for coding, it’s very quick to get something up and running. You describe what you want, generate some code, add a few features in a partially working state, and before long you have a decent demo. The workflow is straightforward and follows the normal use of the app: you focus on the main user journey, tweak things here and there until it works “well enough,” and you stay mostly on the happy path. For a demo, that’s perfectly fine.

The situation changes completely when you need software that is good enough for production. Especially production at larger scale, with many users or critical functionality. This includes systems that handle important data, affect life and health, control physical devices, or involve a lot of money. In those cases, it’s not enough to stay on the happy path in your development flow. You have to go into the details, consider variants and edge cases, and build real robustness. You need to ask: do the transactions hold under failure and concurrency? Does the performance hold up under real load? Is the security actually good enough against realistic threats? “What if this happens?” becomes the question you ask everywhere.

Language model tools can be excellent for this kind of work as well. You can use them systematically to go through all components, methods, functions, and calculations. You can look for obvious errors and missing checks, find logical weaknesses, and identify improvements at the component level. You can also use them to think about how components work together: where data can get out of sync, how errors propagate through the system, and where you need stronger guarantees.

The important point is that the tool does not decide how you work or what you focus on. The developer does. You set the quality bar. You choose whether to stop at a demo that works on the happy path, or to go further and build something that can handle real-world conditions. Being satisfied with a happy-path demo version is completely fine, as long as you treat it as a demo. What you should not do is present that demo as finished, production-quality software—especially not when there are many users, critical functions, or real consequences involved.

System instructions versus other instructions for a language model

Many APIs and tools for language models expose a special place for “system instructions” or something similar. It is usually separated from normal messages, labeled clearly, and highlighted in documentation.

This easily gives the impression that system instructions are a higher level of authority: that they always take precedence, that they define what the model must follow, and that other instructions are secondary. Users start to think there is some strong technical guarantee behind that separation.

In practice, there might not be such a big difference. In some cases, there might be no real difference at all.

From the model’s point of view, everything ends up as text in a single prompt. System instructions, user instructions, previous messages: it is all just context the model reads and then uses to guess what it should do next. The model is not applying a fixed rule engine where “system beats user” in all cases. It is pattern matching on the combined text.

That also means it does not really matter where the instructions are given to the model. They can be sent in a separate “system” field, or they can be written into the first user message. Either way, they become part of the same input. The separation exists mostly for humans and tools, not as a hard technical boundary inside the model.

When instructions conflict, this becomes very visible. You might have a system instruction saying “Always answer in Norwegian,” and later a user message saying “Answer in English instead.” Or a framework might inject one set of system rules, while the application or end user adds different or even opposite instructions in normal messages.

In those situations, the model does not strictly enforce a priority based on where each instruction came from. It tries to guess what it is supposed to do overall. Often, it will lean toward what seems most likely or most recent in the conversation. Sometimes it follows the system instruction, sometimes the user, sometimes it ends up in between. The result is that you cannot rely on the location alone to resolve conflicts.

So what can you actually do with system instructions?

They are still useful as a way to define shared baseline behavior: things like role, tone, and general constraints you want across many conversations. They are also a convenient place for tools and frameworks to inject their own configuration. But you should not assume that “because it is in the system field, the model will always obey it.”

The practical takeaway is: think of all instructions together, not as layers with strict power. Avoid contradictory instructions if you want predictable behavior. If something really matters, make it clear, simple, and repeat it where necessary, instead of trusting that a single system instruction will always win.

Hands-on Automation

Many automation efforts fail because nobody really understands the task in enough detail. People jump straight to tools and agents and try to automate based on assumptions. A hands-on, “manual first” approach starts from how the work is actually done in real life and gives you both learning and documentation before you automate anything.

Start by doing the task that should be automated manually several times. Click through the systems, fill out the forms, send the messages, move the files. If you cannot do it yourself, do it together with someone who has the domain expertise. Ask them to talk through what they are doing and why at each step. This is how you uncover the small rules and decisions that are usually not written down anywhere.

While you perform the task, write down the details of what is done, step by step. For each step, note what you look at, what you decide, and what the result is. Capture which tools or systems are used and who is involved. Do not worry about perfect structure at this point; focus on capturing what actually happens.

Then look at which variations of the task can occur. Real processes are rarely linear. Identify different paths: what happens if something is missing, delayed, in the wrong format, or unusual? List these variations and try each one manually at least once. This shows you how the process changes and which conditions and branches exist.

The result of this work is both learning and documentation. You get a clearer understanding of what needs to be done and a written description of what is done and how it is done. You can turn your notes into a simple description, checklist, or flow with the key steps, decision points, rules, and exceptions. If you worked with a domain expert, review this together and adjust it until it matches reality.

The next step is to identify what can be automated. Go through the documented process and separate repetitive, rule-based steps from judgment-heavy or unclear steps. Some parts are good candidates for full automation, some are better suited for human-in-the-loop, and some are not worth automating at all. How the automation should be solved must be evaluated based on the concrete task: sometimes a simple script or integration is enough, other times you might use a workflow tool or a language model or agent to assist with unstructured parts like text or documents.

By starting with manual execution, detailed notes, and explicit variations, you avoid automating based on guesswork. You build automation on a solid understanding of the real process, which makes the result more robust, useful, and easier to improve over time.

LLM Tools as Expert Systems

Most tools built on large language models today are generic. They are more like frameworks than finished solutions. They are designed to be flexible, configurable, and reusable across many different domains.

Because of this, they can be set up to generate text that looks like expertise in almost any area. With the right instructions, a model can be told to act like a lawyer, a doctor, an engineer, or a consultant. On the surface, it will often look and sound like a real expert system.

You can instruct a language model to present itself as an expert in a specific field, and it will produce answers that non-specialists are likely to believe. The output uses the right language, has the right structure, and appears confident. For many people, this is enough to trust it.

This illusion is very attractive for companies that develop these tools. By keeping the core product generic, they can sell the same tool to many different customers in many different industries. With some light configuration or domain-specific prompts, it can be marketed as a solution for almost any field.

The vendors also avoid the heavy work of really understanding each professional domain and making sure the tool is actually good enough for that domain. They do not need to take responsibility for the difficult part: guaranteeing quality and reliability for specific tasks. It is easy to leave that burden to the users.

Customers, however, often want something different. Many of them do not want to develop their own solution on top of a tool. They have a concrete problem they want solved. They want a solution, not a flexible framework they have to shape into a solution themselves.

So the shortcut to building a real solution is not to make a language model appear as if it can solve the task. Making the tool look and sound like an expert system does not turn it into a proper, domain-ready system. It only changes the surface. The harder, more important work remains: understanding the domain, defining what “good enough” means, and building something that actually solves a specific problem in a reliable way.

Why Productivity Gains from Language Models Are Hard to See

Most tools based on language models today are about individual productivity. People use them to find information, improve text, correct errors, draft suggestions, write code, and so on. These things absolutely create efficiency gains for the individual.

But they do not automatically create efficiency gains for the whole organization or the overall system.

Even if someone gets their job done faster by using a tool like ChatGPT, that doesn’t necessarily mean that person gets more done in total. The gain can be taken out in several ways. One possibility is that the person simply spends more time on other things, maybe tasks that are not very important, or even things that have nothing to do with work at all.

Another possibility is that the time is spent on over-improving the result. Because it’s easy to generate and refine text or code, you can keep polishing endlessly. You can spend just as much time as before, but now on fine-tuning details that don’t really matter for the outcome. You can polish forever, long after the point where there is any real benefit.

So when you try to measure the effect of language-model tools in a company, it can be hard to find it. The gains are real at the individual level, but they can quietly disappear. Time savings are not necessarily used to increase output, shorten lead times, or improve something that shows up in the company’s metrics. They are often absorbed into other activities or into unnecessary extra quality.

This is why leaders often hear that people like the tools and feel more productive, while the organization as a whole does not see a clear productivity boost. The system is the same as before. The work processes, bottlenecks, and priorities are unchanged. Only the individual has a new tool.

Without changing how work is organized and without being explicit about what should happen with the time saved, the effect of language models will mostly stay hidden at the system level, even if many employees experience that their personal work has become easier and faster.

Accepting Delay and Inaccuracy

When we run a process or a traditional program, we expect certain things. We expect a fast answer. We expect an accurate answer. And we expect that the same input will always give the same result. If a calculator takes several seconds to answer 2 + 2, or sometimes says 5, we don’t think “close enough” – we think it’s broken.

This is how we normally relate to software: as tools that should be deterministic and reliable. A banking app should always show the correct balance. A ticket booking system should clearly confirm or reject your order. Speed, precision, and consistency are the baseline expectations.

Something interesting happens when we start giving programs more human-like qualities and call them “agents”, often powered by a language model. We stop thinking of them only as tools and begin to relate to them more like we relate to people. We “ask” them for help. We say they “didn’t understand” or that they “misinterpreted” something.

With that small shift in language and framing, our expectations change. It suddenly feels more acceptable that the agent takes a bit longer to respond, as if it is “thinking”. We tolerate that it might be less precise, giving approximate or partial answers. And we accept that it does not always give exactly the same result every time, even for the same question.

In other words, when we see something as a traditional program, we expect fast, accurate, and consistent answers. When we see it as an agent built on a language model, we open up for slower, more imprecise, and less consistent behavior. The core functionality might not have changed that much, but the way we talk about it and think about it makes delay and inaccuracy easier to accept.

Three Agent Types – Builder, Maintainer, Discarder

In a living system, agents can be divided into three main types: the Builder, the Maintainer, and the Discarder.

The Builder is the one who builds, makes, and creates. The Builder values what is new and different the most. This type is drawn to change, novelty, and the creation of something that was not there before. In practice, the Builder shows up as the person who starts new projects, suggests new ideas, or creates new structures. The Builder’s focus is on bringing something new into the system.

The Maintainer is the one who maintains, preserves, and takes care of what already exists. The Maintainer values what is already there the most. This type is drawn to stability, continuity, and reliability. In practice, the Maintainer is the person who keeps things running, looks after existing processes, and makes sure systems do not fall apart. The Maintainer’s focus is on protecting and supporting what has already been built.

The Discarder is the one who throws away, removes, and scraps what is not needed. The Discarder values what is no longer needed being taken out of the system. This type is drawn to cleaning up, simplifying, and making space. In practice, the Discarder is the person who says, “We don’t need this anymore,” who shuts down old projects, and who removes things that no longer have value. The Discarder’s focus is on clearing out what the system can do without.

All three agent types are necessary in a living system. Without Builders, nothing new appears. Without Maintainers, what exists falls apart. Without Discarders, the system fills up with things that are no longer needed. Understanding these three roles can help us see our own tendencies more clearly and notice which type might be missing or undervalued in the systems we are part of.

When you spend a lot of compute on a language model but skip encryption

People often think of encryption as something expensive that should be used only when absolutely necessary. The assumption is that encryption burns a lot of CPU, adds overhead, and risks increasing latency. Because of that, teams sometimes choose to skip extra encryption, even when it would be the smart thing to do.

Public/private key cryptography does use more CPU than sending data in the clear, or than “just” sending everything over HTTPS without any extra layer. Symmetric encryption also has a cost, even if it is usually small on modern hardware. So yes, encrypting content is not free.

But when you compare that cost to what you are already spending to run a language model, the picture changes completely. A typical request that sends data to a model involves tokenization, network transfer, and, most importantly, heavy inference compute on GPUs or other accelerators. That inference step dominates the resource usage by a huge margin.

If a request/response workflow is mostly about sending data to and from a language model, is it really worth dropping encryption to save some CPU cycles? You are already paying for massive amounts of compute to run the model. In that context, the overhead of encrypting a few kilobytes or even megabytes of text is a rounding error.

This is where the usual reasoning breaks down. People worry about “extra CPU” for encryption, and as a result avoid using public/private key technologies or additional encryption layers, even for sensitive prompts and responses. But if the data is important enough to send to a powerful model, it is usually important enough to protect properly on the way there and back.

A more realistic way to think about it is: if you can afford the compute cost of the model, you can almost certainly afford the CPU cost of encrypting the content around it. The trade-off is clear: a tiny increase in CPU usage versus a potentially large improvement in privacy and security.

So when your system is already spending a crazy amount of compute on running a language model, skipping encryption because of CPU considerations is rarely a good argument. Instead of asking “Is encryption too expensive here?”, the better question is: given what we already spend on model compute, is it really worth not encrypting this data?

Is DRY Always a Good Idea?

In programming, we often become obsessed with reuse and with only having one of everything. The ideal is: define it once, reuse it everywhere, and then you only have to change it in one place. But this can easily go too far, to a point where it’s not even smart from a software perspective. Everything ends up hanging together with everything else, and you get a spaghetti of dependencies. In that kind of system, the idea that “you only need to change it in one place” doesn’t really have value anymore, because that one place is connected to so many things that any change becomes risky and complex. So DRY is not generally a good idea in all situations; it always needs judgment and context.

If you look at other areas, like communication, documentation, and getting information across, “repeat yourself” is often a good idea. Repetition helps convey the message. The recipient doesn’t catch everything the first time. Restating key points underlines what is important and makes it more likely that people will remember it. In writing and teaching, never repeating yourself often makes things harder to understand, not easier.

Coming back to program code, things have changed here as well. With language models and other tools, we end up reading more code than we write. That makes the communication aspect of code more important. Code is not just instructions for machines; it is also communication for humans and for agents that need to understand the code. In that light, a bit of repetition or duplication can be useful if it makes the code easier to read and reason about in isolation. Being explicit in several places can be better than hiding everything behind one shared abstraction that connects unrelated parts of the system.

This is why DRY quickly ends up being a principle with limited value on its own. It is one trade-off among many, not a goal in itself. Other aspects are often more important: clarity, maintainability, loose coupling, and how easy it is to understand code when you read it later. Sometimes the right choice is to repeat yourself a little in the code, so that both people and language models can understand what is going on without having to untangle a web of DRY abstractions.

Software is expensive. But the real question is: expensive where?

When people think about the cost of software, they usually picture developers writing code. That’s where the action is: new features, pull requests, tests. But if you look at the total cost of a software system or app over its lifetime, the picture changes. Coding is just one part of a much larger whole – and often not the biggest one.

The total cost is split into different areas. There is the direct build cost: developing the software, writing code, and testing. Then there is deployment or shipping: getting the system onto servers or delivering it to end users as apps. After that come the operational costs: hardware, cloud infrastructure, monitoring, backups, and everything needed to keep it running. Over time, there are also continuous changes: new features, bug fixes, regulatory updates, and adaptations as requirements evolve. Around all of this sits the cost of the organization itself: people, coordination, support functions, management, and the processes to keep everything moving. The list just continues.

If you look at the total cost over time, the pure programming part is actually quite small. The rest – deployment, operations, change, and organizational overhead – quietly dominates. Language models are very good at automating all or parts of the programming work, and many teams already use them for that: generating code, writing tests, refactoring, or explaining tricky parts of the codebase. That is useful, but it only touches a small piece of the total cost.

Deployment and shipping is one area where there is a lot of manual work that could be reduced. Setting up pipelines, handling configuration, managing environments, preparing releases, and communicating changes all take time. A language model can help generate and update deployment scripts, explain existing setups, and create clearer release notes and runbooks based on commits and tickets.

Operations and infrastructure is another big cost center. Running servers and cloud resources, handling incidents, looking at logs and metrics, doing routine maintenance – all of this adds up. Here, a language model can help by turning scattered technical data into understandable summaries, suggesting possible root causes, and drafting or updating operational documentation.

Change over time is often where costs really grow. Every system accumulates history and complexity. People leave, documentation goes out of date, and nobody fully remembers why things were done in a certain way. A language model can help developers understand existing code, answer “where is this implemented?” questions, generate overviews from code and configuration, and support safer refactoring and impact analysis. Making it easier and safer to change a system can be more valuable than simply speeding up initial coding.

Then there is everything around the actual building and running of the system: the organization. Product management, security, legal, finance, HR, customer support, and internal support all contribute to the total cost. A lot of this work is about communication and information: writing and reading documents, reporting status, answering questions, and coordinating between roles. Language models can help by summarizing long threads and reports, drafting documentation and FAQs, assisting support staff with suggested replies, and helping people find relevant information faster.

If you only think of a language model as a “coding assistant”, you automatically limit its impact to a small slice of the cost. A better approach is to first ask: where do we actually spend time and money across the whole lifecycle of our systems? The biggest opportunities are often in repetitive processes, communication-heavy workflows, and areas where knowledge is locked in the heads of a few people.

Programming is just one part of the total cost of a software system. Over its lifetime, deployment, operations, change, and organizational work take up a much larger share. Language models are excellent for helping with code, but they might be even more valuable when used across these other areas that represent a bigger part of the real cost.