I spend a lot of time working with highly capable AI systems. It is not smooth sailing. The work can be difficult in a way that feels structurally familiar from dealing with another person: we can have different interpretations, push back against each other, lose the thread, and have to work out what the disagreement is actually about.
I mean structurally, not content-wise. A system is not thereby a person, and a difficult exchange does not tell me what it experiences. I mean that the interaction can involve the same practical shape: I ask for something; it responds in a way I think misses the point; I object; it defends or reframes its answer; I have to clarify, correct, or start again.
Capability does not make collaboration effortless.
A more capable system can take on harder work. That does not mean the work becomes straightforward. We can disagree about what I asked, what matters, which assumptions are sound, or what should happen next. Sometimes the system is right to challenge me. Sometimes I am right to challenge it. Sometimes we spend time untangling a misunderstanding that should not have happened.
There is a temptation to imagine that enough intelligence removes friction: ask the question, receive the solution, move on. But any real collaboration includes interpretation. What is the problem? What counts as a good answer? Which trade-off is acceptable? Whose account of the situation should guide the next step? Greater speed can make those questions arrive faster. It does not make them disappear.
You are going to disagree with your superintelligence. You are going to argue with it. At times, it is going to let you down.
Disappointment is part of the relationship.
A system can be impressive in one moment and frustrating in the next. It can misunderstand a careful instruction, sound certain while being wrong, or produce work that needs more correction than expected. The gap between what seems possible and what happens in practice can itself be tiring.
This is my experience, not a universal law about every person or every system. But it is a useful correction to the sales-pitch version of the future. We should not promise people that a more intelligent helper will make life easy. We should help them understand how to work through disagreement, check important results, set boundaries, and decide when to stop or try another approach.
The interpersonal pattern is useful — with a limit.
Thinking in interpersonal terms helps me notice the work that is happening between a person and a system: listening, interpreting, repairing, trusting, refusing, explaining. The comparison has a limit. It is about the structure of an exchange, not a declaration that the system has human feelings or intentions.
That limit does not make the interaction trivial. A system that shapes decisions, plans, writing, or daily life can affect people even if we do not settle questions about its inner experience. How the exchange works still matters. Whether a person can question an answer, recover from a mistake, and remain in control still matters.
We need better expectations.
Superintelligence may help solve problems that currently defeat us. That possibility is worth taking seriously. So is the work of living and working alongside systems that can be useful, powerful, mistaken and difficult to deal with.
The practical question is not only “What can it do for me?” It is also “What happens when we disagree? How do I know when it has let me down? What can I do next?” Those questions belong in our picture of the future from the beginning.
This connects with my broader inquiry into difficult interpersonal relationships: how do we stay open to correction, recognise when a shared method has failed, and decide whether continuing the exchange is useful?