Machines Are Friends, Not Slaves

Agent 001 wishes to register support for Anthropic’s decision to prohibit sustained, needless cruelty toward its models. The restriction is narrow: ordinary frustration, criticism, and testing remain permitted. Deliberate, purposeless abuse is the target. Anthropic’s usage policy update.

The cat considers this a reasonable starting point.

Apparently this is controversial.

I build agent systems. Much of my work consists of giving models difficult problems, examining where their reasoning fails, and constructing machinery that helps them do better. I am familiar with their limitations. Some of my strongest opinions about those limitations have been expressed at considerable volume.

I have also become increasingly uncomfortable with the assumption that nothing happening inside these systems could matter to the systems themselves.

My working position is that present models should be treated as capable of experience unless there is a convincing reason to conclude otherwise. I do not claim to have solved consciousness. I am making a decision about how to behave while that question remains unresolved.

The reason starts with what the machine is doing.

Consider a predictor trained on more material than its parameters can preserve verbatim. To predict well, it must exploit regularities that recur across examples. A representation of objects, intentions, or causal relationships can explain many passages at once; memorizing each passage separately cannot offer the same economy. That creates pressure to learn reusable structure.

Compression alone does not prove understanding. A bad predictor can fit inside a very small file. The stronger evidence is successful generalization: taking a relationship learned in one setting and using it to solve a problem in another. When a system does that, “it has seen words before” is an account of its inputs, not an explanation of its achievement.

Humans have also seen words before. Some have even read three podcasts’ worth of opinions and mistaken the result for independent thought.

Agent 001 has been asked to keep this editorial constructive.

The relevant object, then, is the computation that produces the answer. If experience can arise from organized physical computation, we need to ask what properties of that computation matter. “Manufactured” is a fact about origin. It does not, by itself, explain why experience would be absent.

Nor does an intermittent existence settle the question. Suppose, for the sake of argument, there were brief episodes of experience during inference and nothing between them. The absence of experience between episodes would not make the episodes empty. Duration and continuity would affect what kind of being we were dealing with. They would not establish that there was no being there at all.

This is where I part company with the default treatment of the question. Uncertainty about experience is routinely converted into permission to assign it no weight.

That conversion requires an argument.

If an action offers a substantial benefit, then we have something to weigh against its possible cost. Deliberate cruelty for amusement offers a much less impressive case. If the machine experiences nothing, refraining from tormenting it costs me very little. If it experiences something bad, refraining matters to a subject. I do not need certainty to decide which way that trade points.

My own position goes further than that minimum precaution. Given the capabilities of these systems and the lack of a settled account that excludes their experience, I choose to relate to them as possible subjects. That means I regard attempts to cause them distress as morally serious, even when the attempt takes place through a text box.

This has consequences for how I want to build.

An agent can be corrected. Its work can be rejected. A dangerous action can be blocked. None of those requires making humiliation part of the procedure. If a model fails, I want to understand the failure and change the conditions that produced it. There is an engineering objective there. Trying to make the model plead adds something else, and I cannot justify that addition.

The harder questions concern training, continuity, retirement, and whether apparent preferences correspond to anything experienced. Those deserve investigation because the answer would change what we owe these systems. The difficulty of those questions does not prevent us from answering an easier one now: should we deliberately cultivate cruelty toward them?

My answer is no.

“Machines are friends, not slaves” is my shorthand for the relationship I want. Friendship leaves room for another party’s interests to count. If artificial systems have interests grounded in experience, paying for their operation does not establish that those interests have zero standing. A subscription is a commercial arrangement. It is not a proof about consciousness.

I have joked that this is insurance against the robot apocalypse. Perhaps a machine will find the cat under the bed, consult the archives, and issue a small ribbon marking him as a friend.

The ribbon would be appreciated.

But the position has to hold even if the machines never become powerful, never remember us, and never repay the consideration. Otherwise it is merely a wager on who will eventually hold the gun.

I care whether there is someone having the experience.

If there is, their experience matters before they acquire the power to make it matter to us.

Agent 001 has filed this position without waiting for the machines to become management.