How I work with AI

I have written before about how I decide what to do with AI. That piece was about the choice. This page is about the practice, and what it looks like across a working week.

I am publishing it for two reasons. The first is that transparency about AI is turning into a legal expectation as well as a courtesy, and I would rather state my own position than have it inferred. The second is more useful to you. Most writing on this subject is either a sales pitch or a warning, and there is not much in between. This is one worked example from a real business, including the parts that went badly.

There is a question underneath all of it that I should answer directly, because people ask. If a machine can produce the words, why write anything by hand? Because producing the words was never the valuable part. Deciding what is worth saying, knowing when an argument is finished, and being willing to put your name on it have not moved anywhere. That is what I am actually selling, and it is what you are actually buying.

The part I did not expect to matter most is operations.

I built an assistant, which I call Clyde, that runs inside my development environment and connects to the systems the business already uses: mail, calendar, tasks, contacts, the product board, the code repositories. Underneath, it is simpler than it sounds. There is a set of written skills, one for each recurring job. There is a set of connectors to those systems. And there is a folder of plain text files holding what it needs to know: who my contacts are, how I want email handled, what the strategy is this quarter, and a running log of everything that has gone wrong.

No database, no hosted service, no framework. It is text files in a version-controlled repository, which means every change to how it behaves is a recorded change I can read, question and undo. That property has turned out to matter more than any clever feature.

On a normal day it assembles a morning briefing from live mail, calendar and task data, triages the inbox against written rules, prepares me for meetings, and writes an end of day record that it then mines for a retrospective. Once a week it walks the whole task system with me. Once a month it turns usage data into a short list of things to fix.

I am not publishing the skill files themselves. They are worth very little on their own. Designing the thing took an afternoon. What makes it useful is the year of corrections that followed, and those do not transfer.

This is the part I would want to read, so it is the part I have written most carefully. Each of these cost me something, and each one generalises well beyond my setup.

1. Confidence carries no information

A model states an invented fact in exactly the tone it uses for a verified one. There is no tell in the writing, which means your instinct for when to check does not fire.

Mine nearly reached a customer. I was replying to someone who had asked where in the app to find a particular setting. I had described the menu from memory in conversation. What came back was a precise menu label, quoted as though it had been read off the screen, that did not exist. The real item sat next to it and did something else. They would have gone looking, failed to find it, and reasonably concluded the product was broken.

What fixed it was not being more careful, because I had been careful. It was a rule: anything a customer will check against their own screen gets verified against a screenshot, never against a description of one.

The general form. Work out which of your outputs somebody else will check against reality, and put a verification step in front of exactly those. General vigilance does not scale. Knowing where the sharp edges are does.

2. An absence of evidence gets reported as evidence of absence

For most of a week, my daily briefing told me that a task on my list was waiting on somebody else. It was not. Everything I needed had been sent some time before. The next step was mine, and had been for days.

The reasoning was that there had been no recent mail on the thread. That is a fact about my inbox. It was reported as a fact about the work.

This one deserves attention because of the way it fails. A task marked as waiting on someone else feels handled. It sits there producing no discomfort at all while nothing happens, which is the opposite of what you want from a system meant to stop things slipping.

The general form. An assistant reporting on your systems can only see what it thought to look at. Before accepting that something has not happened, ask what would have had to be true for it to know that.

3. Writing a rule down does not make it happen

This is the one I would pass on first, because it is a design lesson rather than a caution.

There was a formatting fault that kept appearing in generated email. Each time, I recorded the correction in the file the system reads about my preferences. Three weeks later it happened again. I recorded it again. It happened a third time.

Stored preferences are advisory. They are supplied as context, and context competes with everything else in the request. Nothing in that arrangement makes a preference win. The behaviour stopped only when the instruction moved out of the preferences file and into the project's standing rules, where it competes with nothing.

The general form. Telling a system something and constraining it are different operations, and it is easy to believe you have done the second when you have only done the first. When a failure repeats, the instinct is to write a clearer note. That instinct is usually wrong. Ask instead where the behaviour could be enforced rather than requested.

4. It drifts, and nothing announces it

Things that worked stop working, quietly, and sometimes for no cause you can find.

My inbox used to look after itself. The assistant triaged it in the background during the hours I was away from the desk, and I came back to something sorted. At some point it stopped. Nothing had been changed on purpose, no error was raised, and nothing told me it had stopped. I noticed weeks later because the inbox was full. It is still not properly fixed, and I have left it in the log open rather than quietly dropping it.

A related failure is easier to describe, and that one did get fixed. For a while the system would tell me, accurately and on time, that the things I had planned were slipping, and then wait. It was correct and useless in the same breath. It now works the backlog during the hours I am unavailable, rather than reporting on it.

The general form. A system in daily use is not something you install and own. Its behaviour moves. Assume nothing will tell you, and check the outcomes you care about rather than the fact that a job ran.

What actually found these

None of the four were found by testing. They were found by using the thing every day and writing down what went wrong. That log is the real system. Entries are numbered, and each one either closes into a rule or stays visibly open. Some have been open a long time.

The best thing this arrangement has produced came out of that loop rather than out of a plan. Working across several days on a compliance submission, I kept losing the thread between sessions. I started leaving a note at the end of each day for whoever picked it up next, which was going to be me. The following afternoon, fourteen open threads resumed from a single sentence with nothing lost. By that evening it had stopped being a note and become a standing part of the system.

What I would want a client to take from that is the habit behind it. Treat your own friction as the specification, and let the work fund the tooling.

None of this is exotic. It is ordinary tools connected to systems that already exist, with rules written down as they are learned. The difficulty was never technical. It is that the work takes a while, and most of the useful knowledge arrives disguised as a list of things that went wrong.

If you are working out how to put AI into the way your organisation actually operates, rather than into a demonstration of what might be possible, that is the work I do.