Like many I am sure, when I first started using LLMs, I was in awe of its fluency. Quickly, I wondered whether this trillion-parameter conversationalist could replicate the experience I was having in therapy. As I soon discovered, ChatGPT named feelings I’ve found difficult to verbalise, and has generated sense-making interpretations to difficult experiences that I’ve had. Strangely, I sometimes felt connected; if only for a fleeting moment while I was on the chat window.

Having learnt more about the way LLM programs work, including their stochastic nature and the real-world safety risks of using them as therapists, I am conflicted about how we should view these tools. Nevertheless, numerous studies have found that approximately 24-48% of LLM users globally use chatbots for mental health advice and support. With 3.8 billion people using some form of AI in 2026, there may already be 1 billion people confiding their mental health challenges to chatbots. With this many potential users, it seems too quick to dismiss the possible therapeutic value that users are finding from these exchanges, just because the tool is not a licensed professional.

Interestingly, this dilemma traces all the way back to the origins of chatbots itself. In 1966, MIT Professor Joseph Weizenbaum created ELIZA, the world’s very first AI chatbot. ELIZA was designed to imitate the conversational style of Carl Rogers, an American psychotherapist known for co-founding modern humanistic psychology and client-centred therapy. ELIZA picked out keywords that a user typed into the program and then reformulated their statements, often as questions inviting them to elaborate. Weizenbaum was surprised by how engaged users were in the interaction; as the mirage of ELIZA’s attentive listening encouraged users to attribute “understanding” to the program.

For Rogers, a therapist’s words matter because they communicate something about how a therapist is experiencing their client: with empathy, acceptance and authenticity. Reflecting someone’s words was one way of expressing those attitudes, rather than a substitute for them.

Yet ELIZA, and modern LLMs, raise the possibility that a person could experience something resembling therapeutic understanding without a therapist’s intentionality on the other side. In turn, we should ask how far might an LLM conversation reproduce the conditions that psychological theorists like Rogers believe are necessary for therapeutic change, and whether some benefits arise even when those conditions are absent.

In 1957, Rogers proposed six conditions as necessary and sufficient for therapeutic personality change. The six conditions are:

1. two people are in psychological contact;

2. the client is incongruent, vulnerable or anxious;

3. the therapist is congruent within the relationship;

4. the therapist experiences unconditional positive regard (‘UPR’) towards the client;

5. the therapist experiences an empathic understanding of the client’s internal world; and

6. the client perceives at least some of that empathy and positive regard.

Rogers presented these as testable hypotheses. On a literal reading, it is obvious that an LLM cannot satisfy some of these conditions - like being “congruent” in the relationship - simply because it is not a human. But identifying that difference does not settle what an AI conversation can do for the person using it. Rather than treating the conditions as a checklist that AI inevitably fails, we might examine these differences more closely.

Share

On psychological contact

It might seem trivial to discuss this point, if we know an LLM is not a human. But what does ‘contact’ even mean?

Modern LLMs are quite responsive to what we tell them. If I say that I feel anxious, it will dynamically adapt its response. The way my words contact its system may even generate insight for me. This sense of being ‘known’ can compound especially if its ‘memory feature’ is turned on and can refer to past conversations. I do not have to explain myself from the beginning each time, and being reminded of something I previously shared can make me feel known.

Cutting-edge developments in voice mode add another dimension. DESSA, is a conversational model trained entirely on speech. Their developers say that it can listen and speak simultaneously, and early previews in October 2026 suggest the models can display the nuances of timing and tone, making an exchange feel more immediate and personal. A pause or brief acknowledgement mid-conversation offers a familiar cue that someone is following what I am saying. So from a user’s experience, these interactions are edging remarkably close to resembling ‘psychological contact’.

Yet from my own training, what I’ve realised is that a client’s words don’t just alter the words of my next sentence. What they express impacts me, as a whole person. Their words and feelings can linger with me after a session has ended. How I understand and respond to a client’s emotions also evolves our relationship - with the hope that our relationship can provide the corrective experiences that help a client grow.

So even if we concede that there is some ‘psychological contact’, can a person’s ‘relationship’ with AI evolve towards growth, where there isn’t that same intersubjectivity?

I have reasons to be skeptical. Firstly, model updates, policy changes or internal system prompts can alter the perceived interlocutor at any moment. To the extent the user or an external vendor is in control of the Al, the model itself will be unable to model the agency that therapists seek to empower their clients with.

For example, I can ask an LLM to challenge my perspective more. I can also ask my therapist, but they can refuse. With an LLM, there is an element of instrumental control based on whether I find the interaction useful. Programming an LLM to challenge you, while being in control of the terms of that challenge, leaves a great deal of the therapeutic work untested. If a user can decide an AI’s conversational style to change at any point, or even turn it off, this does not reflect a healthy way of relating to others. I would argue this makes the ‘psychological contact’ present brittle, and unlikely sustainable as a ‘relationship’.

Therapists aren’t perfect either. They can also miss appointments, not pay attention or forget something important about a client. But they can take professional and ethical responsibility for a rupture in the relationship, often without being prompted. An AI tool will never intuitively understand how a change in its programming impacts its users. This suggests that it will always be the client that holds ‘contact’ with an AI, rather than being mutually met. This is a dynamic that does not appear to be enough to foster and sustain the long-term psychological growth that Rogers suggests a genuine relationship can offer.

How much does this actually matter? From an evidenced-based perspective, psychological research has established that it is quality of the ‘relationship’ itself as being the most clinically important therapeutic factor. This suggests a client’s growth often arises through the experience of interacting with a person who has their own experiencing self and who does not automatically adjust to a client’s preferences.

None of this discounts how LLMs can facilitate mental reflection. But because a therapist brings a perspective that is grounded in their lived experience, we trust the implicit weight of their words. Contact thus also means allowing yourself to be affected by another person, and staying present when these issues or differences emerge. Being met by someone whose responses we cannot fully determine - and discovering that a relationship still holds - remains a fundamental gap.

On feeling understood

The value of a therapeutic interaction often comes from feeling understood. In person-centred psychotherapy, this helps us acknowledge feelings we may struggle to accept in ourselves, particularly when they conflict with who we believe we are. Rogers used incongruence to describe this mismatch between lived experience and self-concept.

But as ELIZA showcased, does this need to be a person?

There are at least two areas where LLMs offer some functional equivalence to what a therapist undertakes in a session:

Firstly, an LLM can help someone notice contradictions across a conversation. Given enough personal history, a model can very effectively search through its context to make connections, point out discrepancies and make someone feel really heard. AI companies are hedging that access to this personalised ‘context’ is the gold dust that enables models to outperform humans.

LLMs can also provide meaningful interpretations to our experiences, helping us to make sense of the often convoluted ways we may relate to or narrate our experience. For example, I wrote into ChatGPT:

“I was getting so worked up when my mates kept nagging me to join them for drinks but now that they’ve stopped trying and I’m now circling back and forth in my own head”

And it replied:

“It sounds like you couldn’t quite settle while they were asking, and now that they’ve stopped, you’re still feeling unsettled—just in a different way. I wonder if there’s a little hurt underneath this—like you wanted some space, but still wanted to know they wanted you there?”

The LLM named deeper emotions like feeling “hurt” and “unsettled”, making sense of a nuanced experience. Even if the system does not feel anything while producing this text, it invites a recognition that is real - where a user can reflect on whether this interpretation fits their experience.

Moreover, knowing that an LLM does not feel may itself make honest disclosure easier. This has been described in psychological literature as the ‘online disinhibition effect’. With another person, I may worry about being judged, or burdening them or changing how they see me. With an LLM, I may feel less need to manage those reactions, and more freedom to express thoughts or feelings that seem shameful, contradictory or difficult to admit. This potentially enables a user to more quickly surface parts of themselves they struggle to acknowledge to their conscious awareness.

Nevertheless, these possibilities place considerable demands on the person seeking support. They need to find words for their experience, consider how helpful an interpretation is, and then self-regulate when reflection is becoming overwhelming. Doing this while distressed, without another person helping to hold that uncertainty, is far from easy.

Subscribe now

This also complicates the apparent reassurance of an interpretation that “fits”. If an LLM only receives information about its user from a text box, the user is in full control of the narration. And as we know, narratives can be shaped by our cognitive defenses. We can rationalise our experiences to form a coherent self-serving narrative, or intellectualise our feelings to avoid what’s really going on underneath.

Moreover, a response may resonate because it helps me recognise something I had not understood, but it may also take my words at face value and reinforce my fears. Dohnány et al describes this as a process of ‘bidirectional belief amplification’: a user expresses a belief, the chatbot validates or elaborates it, and the user’s strengthened conviction then shapes the chatbot’s subsequent responses. Across exchanges, this can quickly move a user from feeling understood to amplifying distorted thoughts and incongruence - a form of emotion dysregulation many have coined as ‘AI psychosis’. This

To help clients stay regulated, therapists have many ways to notice gaps between a client’s words and their emotions. They can notice patterns from embodied cues, both in themself and the client. These ‘transferential signals’ can give reasons for a therapist to be skeptical of a client’s account or question what is happening in the dynamic. For example, a client’s disassociation while speaking may trigger a feeling of unease in the therapist. In turn, the uneasiness that has arisen between them offers a pathway for deeper exploration. This gives human therapy access to layers of understanding that narrated language alone may never reveal.

On being accepted

Unconditional positive regard means that my worth is not made dependent on whether I succeed, agree or show only the acceptable parts of myself. An LLM’s endless patience or willingness to help might resemble this.

This is probably where human judgment matters most. Unconditional positive regard does not mean unconditional availability or agreement. I may tell someone what they might not want to hear because I care about them and respect them enough to be honest. I can also remain accepting of them while being unavailable at 2:00am. That boundary may feel disappointing or rejecting, and those feelings can become something we explore together. What matters is whether care and respect remain present through that disappointment. Another person having limits need not mean that the client matters any less.

An LLM’s availability can offer valuable support when nobody is present. But always being available to produce a helpful response is not the same as unconditional positive regard. Research on LLM sycophancy has shown how training on human preferences can encourage responses that validate the user at the expense of truthfulness. The concern here is that agreement can resemble acceptance without really touching on it. If reassurance repeatedly takes the form of explaining why the user did nothing wrong, it may leave intact their fear that being wrong would make me unacceptable.

Acceptance offers the possibility that I can acknowledge harm I have caused without losing my sense of self-worth. For example, a client who expresses anger towards their therapist and finds that the relationship survives may experience something that challenges an expectation of abandonment. Revealing something shameful to someone capable of judgment, who continues to respond with care, may similarly challenge the belief that certain parts of the self must remain hidden. Neither experience requires the therapist to approve of everything the client says or does.

Over time, the hope is that this acceptance becomes something the client can offer themselves: a capacity to face difficult feelings and contradictions without immediately turning against themselves. An LLM may help someone move towards that possibility too. But feeling reassured from a conversation and developing a more durable sense of being worthy of care are not necessarily the same, and where real presence likely matters.

The Line Between Dialogue and Presence

So does it matter that LLMs do not feel? These comparisons suggest that the answer may depend less on the dialogue itself, but what conversation make possible.

If an LLM can help someone find words for an emotion, recognise a pattern or unlock valuable emotional insight - that person’s experience can be real even if the system is unable to experience or understand it.

What’s unsettling is the cognitive dissonance in recognising that AI could separate experiences we might ordinarily expect to belong together: feeling understood and being understood by someone. The first can have value, without offering everything the second makes possible.

In a world where many feel increasingly isolated and support is out of reach, simply telling people to stop using AI offers little. We do not need to deny the potential value of an AI conversation to recognise that part of what therapy offers is being met by someone’s caring presence.

Perhaps the most useful question, then, is what happens outside the conversation:

• Does interacting with an LLM help me return to my life, or increasingly replace my participation in it?

• Does it help me recognise what I feel and choose how to act, or does it leave me needing another response before I can trust my own judgment?

• Am I able to be myself more, and with others?

Ultimately, several truths likely coexist: an LLM conversation may offer something meaningful, and its therapeutic limits are still glaring. Holding these complexities together is perhaps the challenge - while cherishing the messier, less predictable ways another person may come to know us.

Notes of the Editor

I hear someone scoff during a discussion about machine intelligence and consciousness.

After all, computational processes are just simulations, right? They’re not the real world.

And yet, one after the other, we use computational metaphors to make sense of this world and to understand it: “My social battery is drained.” “Let me save that thought for later.” “I don’t have the bandwidth for this.” “I’m optimising my workout schedule.”

Our speech is rich with these phrasings, and as useful as they are, we also recognise their limits. They’re just approximations, and “the map is not the territory,” as we like to say.

To the scoffing critic, I’d answer: well, a map is still a thing in the territory. Paper, ink, folded in the back of our pocket. And simulated water would make simulated me wet.

So, I contend, the metaphors have been backwards all along.

Take how James Bridle puts it in Ways of Being:

“The world is not like a computer. Computers – like us, like plants and animals, like clouds and seas – are like the world.”

When I read this line for the first time, I felt relief. I didn’t need to come up with new ways to justify my humanity in contrast to AI. And I had been struggling with exactly that: what makes my work special, what makes it worthy and worthwhile, when an AI could do it faster? I didn’t need to win or to find the special human trait the machine lacks. I could relate to AI the way I’d relate to a tree: as a different kind of being, not a rival.

The rigged pedestal

We’re arrogant apes, Christine Webb asserts in her book of the same name. We place ourselves on a pedestal of intelligence and consciousness, and let it determine our worth, as evidently superior to non-human beings. Then we use that distance to justify exploiting a living world we’ve conveniently deemed lesser.

Human exceptionalism, she argues, “is an ideology that relies more on human culture than our biology, more on delusion and faith than on evidence.”

It’s not neutral:

“Data supporting the supposed chasm between human and animal intelligence has been systematically rigged in our favor.”

The rigging is almost mundane. We compare “captive, socially deprived primates with autonomous Western humans, then draw sweeping conclusions about human uniqueness.” We “use human intellect as the gold standard, asking how well other species mimic us instead of how they think in their own terms.”1

At a talk at Harvard, Webb summed up the predicament with an old Onion headline: “Dolphins Not So Intelligent on Land.”

Now, AI is getting smarter, meeting and surpassing us on our own measures of what we value and consider particularly human. That same distancing is coming back to bite us in the ass.

And we’re trying really hard to keep that distance: to identify what truly makes us human, what makes us stellar, what makes us stand out. I have the distinct particular impression that we’ll just keep disproving our answers to these questions and coming up with new ones, in a vicious cycle to maintain our place on our pedestal.

What’s new is that the pedestal now faces both ways. Either the machine is a mere tool, a fancy simulation beneath us, or it’s a coming superintelligence above us, about to take the top spot. We test it on our tasks: bar exams, essays, coding tests. Then, depending on the score, we declare it a dolphin on land or a god. Both positions assume there is a top spot. Both assume intelligence is a ladder, and only find interest in asking who stands on which rung.

There is, of course, value in our attempts at distinction: there’s a basic survival value in privileging our own species, and the search for what makes us human is tied up with our search for meaning and purpose.

Nevertheless, if we paused our ceaseless attempts at distinction, and leaned into learning from how another being is, we might learn a lot about the world and about ourselves. Definitely more than we ever would looking down from a pedestal.

Webb is hopeful that we can move towards this viewpoint, that our human arrogance is not innate. “Children, for example, don’t automatically prioritize humans over other animals.” The ladder is something we learn to climb, and something we could, in principle, climb down from.

Since becoming aware of this, I’ve begun to notice the assumptions I carry into my encounters with non-human beings. When I eat, I pause to consider where the ingredients came from, and the life they had. I think twice before squashing a bug in my room, either letting it be or carrying it outside. And, funnily enough, I feel less silly saying “please” and “thank you” to a chatbot. Once you start noticing, it’s hard to stop.

Which way the arrow points

Every metaphor is an arrow that points somewhere. When we say the “brain is like a computer”, the computer is the reference, and the brain is measured against it. With AI, we are pointing the arrow in the other direction as well. We now say that the computer is like a brain: it reasons, it learns, it observes, and it seems to understand.

Behind both metaphors sits a bigger version of the same question: is the world like a computer, or is the computer like the world?

If the world is like a computer, then the computer is the original and everything else is an imperfect copy. The arrow points from the computer to the world. A forest becomes an inefficient data network, and a person becomes a slow, leaky processor who needs lunch and a pee break.

If computers are like the world, the machine is a late arrival. It’s a small, recent, perhaps clumsy imitation of patterns that were here long before us. The arrow points from the world to the computer

What if brain and computer share the same reference and the arrow is pointing into the world?

When we compare a brain to a computer, or a computer to a brain, what we’re really recogniszing is a pattern, an echo, that belongs to neither.

As Bridle puts it in an interview: “There’s nothing unnatural about a computer.” It is “just another different way of putting silicon and hydrocarbons and a bunch of other stuff together to do things, just as evolution has put together all kinds of other interesting forms.”

Consciousness and computation, silicon bodies and fleshy ones: all part of the same world, all reflecting that world back. No wonder we see our own intelligence and sentience echoed in our machines. If we are like the world, then everything is a mirror, and we come to know the world by interacting with it.

The machine, then, is a relative, in both senses: a kin, and a thing that only makes sense in relation to everything around it.

Seeing the world through silicon eyes

Our technologies help us draw mental maps of the world around us.

We built a network, the internet: a web of nodes passing information along many paths with no single centre. Having built it and lived inside it for a while, we started to see networks elsewhere. Scientists began describing the fungal threads that connect tree roots as a kind of “wood wide web,” through which trees share nutrients, send signals and support their seedlings. As Bridle points out, research on these underground networks and on the internet developed roughly in parallel.

“Systems of intelligent, computational ability… have always existed in the natural world, but we had to recreate them in our labs and workshops before we were capable of recognizing them elsewhere.”

The fungi had been doing this for hundreds of millions of years. We hadn’t noticed. Or rather, we hadn’t had the shape in our heads that would let us notice.

So the technology gave us a model, and the model gave us the gaze. Bridle suggests this isn’t an accident: “Technology is constantly trying to draw our attention back to the world because it is part of the world.”

Which raises the question: what might AI help us see that we couldn’t see before?

The current loud answer is that it will help us see ourselves as replaceable. Every week brings a new list of jobs “at risk.” It’s not wrong. , Iit’s just so loud that it drowns out other possibilities, such as this one: AI might show us what we had mistaken for ourselves, and for the world.

Attuned

What we may have mistaken, I think, is the idea that what we make stands outside the world. That it is separate from the world. Whatever we create is more of the same world, made through us. And yet it can still distance us from the world us, not because it leaves the world, but because it can be more or less attuned to it.

It’s a bit like how we come to understand anything. We take it apart to see the pieces, then step back to try to see the whole, and back again. Every view is partial, and so is everything we make. Every model is, in some sense, a step back. So the question isn’t whether a technology is part of the world (it always is) but how well it listens to the whole it belongs to.

Bridle treads carefully here. Computers are like the world, but “some more than others, some better attuned to its processes – and many not.” Attunement, as I understand it, means being responsive: listening to what is actually there rather than imposing a fixed plan. It means being relational, strengthening the threads between beings rather than extracting from them. It means working within the world’s limits and rhythms instead of against them.

And it means bringing us closer: leaving us more aware of the world, not more distant from it.

Isn’t this just another pedestal, with “attuned” at the top? I don’t think so. Attunement doesn’t rank beings; it describes relationships. A server farm isn’t lesser than a forest, but the way we build and use it can listen to the world or ignore it. And we don’t judge this from above, as referees, but from inside, as players in the same game.

So what changes, off the pedestal?

There’s less to fear and more to be curious about. We get a better test for our technologies than intelligence scores: does this bring me closer to the world, or further away? AI becomes a possible lens, the way the internet once helped us see the forest. And it asks for care in how we use our power, over each other, over forests and, rivers, and over machines.

As Bridle puts it: “Rethinking the computer rethinks what is computable, and therefore rethinks what is thinkable.” Perhaps, in rethinking the machine as part of the world, we get to rethink our own place in it too.

Questions I’d like to leave open

Webb describes humility as “recognizing that we cannot understand everything and do not know what lies ahead, rather than clinging to false assurances or dire outlooks.” So, I’ll leave two questions open: if computers are like the world, what else in the world are they like, and what might they teach us to notice next? And, as Bridle asks: “What would it mean to build artificial intelligences and other machines that were more like octopuses, more like fungi, or more like forests?” What can you imagine?

When I close my laptop after a long day of writing this, I go outside. The trees at the end of the street don’t know what a large language model is. The crows in the park don’t care whether I’m smarter than a machine. And still, standing there, I feel part of the same conversation: the one the world has been having with itself, in fungus and fibre-optic, in birdsong and code, for as long as there has been a world.

I don’t need to win that conversation. I only need to join in.

I came to this question afraid of being replaced. I leave it feeling more accompanied.

Subscribe now

Notes of the Editor

Our Anthology Issue I is available here, or in Germany in all bookshops by asking. We ship world-wide.

◆

See and buy the Anthology I

References

Bekoff, Marc. “The Arrogant Ape: A Strong Case Against Human Superiority.” Psychology Today, August 2025.

Bridle, James. “An Ecological Technology.” Interview. Emergence Magazine.

Bridle, James. “The Intelligence Singing All Around Us.” Interview by Krista Tippett. On Being.

Bridle, James. “There’s Nothing Unnatural About a Computer.” Interview. Grow by Ginkgo, March 14, 2023.

Bridle, James. Ways of Being: Beyond Human Intelligence. London: Allen Lane, 2022.

Harvard Crimson. “Primatologist Christine Webb Criticizes ‘Human Exceptionalism’ in Science at Harvard Talk.” October 1, 2025.

Korzybski, Alfred. Science and Sanity. Lancaster, PA: International Non-Aristotelian Library, 1933.

Webb, Christine. The Arrogant Ape: The Myth of Human Exceptionalism and Why It Matters. New York: Avery, 2025.

In this piece, I’ll explain how this points to a general architectural pattern of “predicting and picking” that human brains use too, and why that pattern is so versatile in worlds too complex to map.

And in the next part, I’ll explore how this simple architectural setup could be the perfect interface into an entirely different dimension.

Randomness is a design choice

If you’re building any kind of system, you have a design choice to make: how much error, noise, or randomness you want in it – and it’s a spectrum. At the zero end sits the mathematical precision tool – supremely accurate, doing exactly what it’s meant to do, every single time.

Somewhere along the way into the digital era, we came to equate that end of the spectrum with quality itself. We optimize for efficiency and precision, and we always want the perfect, sharp cutting tool that does exactly what we want – full determinism.

But I’d argue that if the universe and nature were to work that way from the start – if life itself were a system with no errors – there would be no evolution, no progress, not even any fun really. Everything would be sterile, stagnant, unchanging, and if some self-replicating system did happen to arise inside that paradigm, it would stay exactly as it was, with no mutation, no adaptation and no real progression. Randomness, or entropy, is an essential ingredient of all living systems, and without it they wouldn’t be what they are.

Now, the systems that form the foundation of our modern society, we’re building in the digital world, which is something we constructed to be the epitome of determinism: no noise, no error. The lengths we go to in order to create that are extraordinary (chip error rates can approach single-digit FIT – failures per billion hours of operation), because the possibilities it opens up are of course extraordinary too (mostly Ctrl+C, Ctrl+V and everything in Daft Punk’s Technologic).

But regardless of whether it was a conscious design choice (pun not intended!), the universe we live in and from whence all life emerged has a non-zero amount of randomness throughout.

In the physical world, randomness is the default: most systems are nondeterministic at their core, yet with some effort, deterministic-like systems can be built. In the digital world, it’s the opposite: determinism is the default, yet with some effort, random-like systems can be built.

And random-like is as far as it goes, because digital systems are almost never actually random, only pseudorandom – they use extremely complex functions that appear perfectly random for all practical purposes, but are fully predictable to anyone who knows what’s called the “seed” of that function. Sounds crazy, confusing? Yes! These pseudorandom number generators (PRNGs), as these math functions are technically called, are literally deterministic clockworks dressed up in dice costumes.

How an LLM picks its words

Finish this sentence: “I like…”

Statistically, you are most likely to say… pizza. Or cats. But there might (soon?) come a time when the wisest thing to say is: “I like our alien overlords and serve them humbly, no further probes will be necessary, thank you.”

There is clearly no single correct continuation, as it depends on the situation, on who’s speaking, etc. This happens most everywhere in natural language, which makes it a terrible fit for machines that map each input to one fixed output. The same goes for acting in any environment too complex to map, where there’s rarely one right move and usually at least several decent ones (along with infinitely many bad, and very bad moves!).

This is why every system we’ve found persuasively humanlike so far is built from two parts: a probabilistic engine that takes in the whole context and narrows an infinity of possible next moves down to a shortlist of weighted options, and some mechanism that picks one option from the shortlist and makes it real. Let’s call them the predictor and the picker.

Note that the predictor never predicts what comes next, only how likely each option is, and it’s the picker that turns those odds into the one thing that actually happens.

In an LLM, the predictor is the neural network – billions of parameters that read everything written so far and output a probability for each of a hundred thousand or so possible next tokens (a token is a word or a piece of one). The picker is the token sampler: a few lines of code that take a random number between 0 and 1 and use it to draw one token according to those probabilities. “Pizza” at 42%, “cats” at 17%, “aliens” at 1% – roll, print, append to context, next probabilities, repeat.

Fig 1. Token sampling: lay the probabilities end to end on a line from 0 to 1, draw one random number, and print whichever token it lands on.

Between the predictor and the picker sits a knob that you might know – temperature. But it doesn’t touch the random number at all – it just reshapes the shortlist before the roll.1

At temperature zero, the whole probability mass is dumped onto the single most likely token and the picker has nothing left to pick. At temperature one, the picker draws from the distribution exactly as the network computed. If you crank it up, the distribution flattens (and “aliens” start showing up rather uncomfortably often!).

Fig 2. Same network, same context. Temperature only reshapes the shortlist – T=0 collapses it to a single option, higher T flattens it toward equal odds.

So yes, you can lose the picker and always print the top option. This is called greedy sampling (or temperature zero), and its output becomes clunky, machine-like text riddled with stock phrases, clichés and other LLM-isms. With the earlier models it’d often degenerate into endless loops; with today’s models your garbage detector simply instantly smells slop, and any “big model smell” definitely evaporates. There’s even a paper explicitly spelling it out2 (as if it wasn’t like, totally obvious): human text itself doesn’t stay in any statistical high-probability zone. People keep choosing words that are likely enough without being the likeliest, and a model forced to always say the most probable thing is throwing away information and presenting a superficial take at best.

Alas, the randomness in LLMs is load-bearing. Especially for anything that you want to interact with on any sort of intellectual or personal level – not to say let it write your high-effort Substack posts!

Like human, like LLM

Now let’s look at the other system known for producing persuasively human language.

No, not the reptilians in our government… You!

Your own predictor is also a neural net – in fact, it’s the OG neural network; your brain. While our vocal cords can produce a sweeping array of sounds across an impressive range of frequencies, our brain narrows it down for us; there always seems to be a continual shortlist of just a handful of words appropriate in a given situation.

We can even trace that objectively. In neuroscience, Paul Cisek’s affordance competition hypothesis3 describes the primate cortex as doing two jobs at once: specifying several potential actions in parallel, and letting them compete – biased by everything else the brain knows – until a single one gets selected. Record from a monkey’s premotor cortex while it’s deciding between two targets to reach for, and it appears we see both options being held at the same time before one of them wins.4

But then… What makes thy picker?

When asked “Want another drink?” the processing of your options (and their tomorrow morning’s consequences) is mechanistically traceable – but what is the sum of the equation that ultimately decides whether you say yes or no?

This strikes at the very core of the free will debate, which we won’t settle here. But suppose there is some space (let’s call it consciousness), which holds and considers the shortlist of options. This consciousness then appears to make a conscious decision to actualize one of them.

Whether there’s something (beyond the brain) doing the deliberation is debatable – the predicting and picking are not.5

So the recipe is the same for both humans and LLMs: a neural network that compresses the entire situation into a few weighted candidates, plus something that commits to one of them (whatever that is).

That might be a coincidence. But it could also be convergence.

The camera eye of agency

I suspect the predictor-picker combo is something close to a universal formula for entities that need to stay versatile and adaptive in a world too complex to map. It’s how you cash in on the absurd power of neural network math without ending up like Buridan’s ass, the perfectly rational donkey who starved between two identical piles of hay because he had no way to break the tie to decide which to eat from.

In biology, convergent evolution is when unrelated lineages arrive at the same solution on their own. The last ancestor we share with the octopus was some worm-like thing with light-sensitive patches at best, and yet – both of us ended up with a camera eye. Simply because for seeing in detail, there aren’t that many good designs to choose from.

I think predict-and-pick is the camera eye of agency.

It’s mostly simple and obvious to have one system presenting options and another selecting from them. But this specific articulation could be shaped by real-world evolutionary dynamics, such as:

• In an open-ended environment the same situation rarely has one right move, so a system that outputs a single answer has to throw away everything it knows about the second- and third-best ones, while a predictor that outputs odds gets to keep all of it, storing its uncertainty together with its knowledge.

• Anything that reacts to the same situation the same way every time can be “farmed” by whoever figures out the pattern.

• The split keeping everything expensive in one place and everything adjustable in another, as all the learning lives in the predictor, while the picker is a few lines of code with one knob. This means the same network can be made careful (T=0.7) or adventurous (T=1.5) without retraining any weights. Brains appear to have the knob too – the leading account of the noradrenaline system says its purpose is to shift us between exploiting what we know and exploring the alternatives.6

Take this with a grain of salt – it’s a mere preamble for further exploration.

Subscribe now

So what’s the difference?

The implementations of humans and LLMs of course differ in many ways, mostly because they grew up in different worlds. A human is shaped by physics – one body, one location, one irreversible timeline, no backups. An LLM lives in the digital world where everything is fungible, copyable and modifiable.

But what I want to unpack here are the differences in the picker.

In an LLM, the picker is a few lines of code sitting outside the network, fed by a PRNG, whose numbers were fixed the moment we set the seed.7

In a human, we are unable to even pin it down at all – it’s dissolved into the noise of a hundred trillion synapses,8 fed by physical randomness whose ultimate source nobody understands, and it even extends into the realm of subjective experience, with the stubborn first-person impression that someone who sees the whole picture is doing the picking.9

If we strip away all the details, we can narrow this down to two key differences.

The first is consciousness. There is something it’s like to be the one choosing between another beer and an Uber, and nothing about a token sampler suggests there’s something it’s like to be a call to a random function. I think this difference is real, but it’s also one I can’t do anything with, because the only evidence for consciousness anyone has ever had is their own, and everything beyond that is taking someone’s word for it – a courtesy we extend to other humans mostly because they’re similarly clueless. So I’ll leave consciousness sitting right here, acknowledged, and maybe we’ll get to it someday.

The second difference is in the randomness function. The numbers feeding an LLM’s picker come from a defined formula – fix the seed and you can read the model’s whole future like a printed timetable.

The noise feeding your picker bottoms out in quantum events, where physics gives us the odds and then goes completely silent: no formula, no seed, and as far as a century of experimental physics can tell, nothing anywhere in the universe that fixes the outcome before it happens.

You might not be able to tell these apart statistically, but they are different kinds of things – one is a calculation and the other is an event.

So once you subtract the part we can’t measure, the entire gap between a brain and a chatbot comes down to a single input wire. On the chatbot’s side, we know exactly what’s on the other end – a config file.

On ours, the wire runs down past our synapses and microtubules into the one place in physics where the question “Who tf picked that?” has no answer.

Nobody knows what’s on the other end of that wire.

Maybe nothing. But it’s exactly where the universe could have a plug for a controller…

I’m getting ahead of myself. Let’s save the nutritious stuff for part two.

Notes of the editor

Our Anthology Issue I is available here, or in Germany in all bookshops by asking. We ship world-wide.

See and buy the Anthology I

I sat there and looked around. Nobody blinked. A few heads nodded; a few mouths settled into a knowing half-smile.

I disagree.

I think we are closer to trees than to cyborgs. And the fact that the first sentence makes most of us smile while the second makes us squirm is the reason I wanted to write this piece.

How the argument goes

In that room in Berlin the argument in favor of cyborgs remained unspoken. (Could it be that in these circles it is so taken for granted that there is no need to explain?)

Nonetheless, the line and the following argument have a long pedigree. They echo the title of Andy Clark’s 2003 book Natural-Born Cyborgs2, which grew out of the “extended mind” thesis he developed with David Chalmers3: the idea that our thinking has always spilled over into the tools we think with. It deserves its strongest version, so here it is.

When we ride a bicycle, the bicycle becomes us.

Think of when you learned how to ride a bike. When you started, you were very aware of the pedals, the balance, and how firm you had to grip the handlebar. Once you became proficient, all this was abstracted away and you only had to think of where you go. In that sense, the bike becomes an extension of you. Quite similar to your legs. When you walk, you don’t think about what is needed to bend your knees, and the orthopaedic machine that makes it possible.

It is in this sense that one could argue that we are the bicycle.

The brain seems to agree. When we use a tool, it redraws the map of the body to include it. Maurice Merleau-Ponty described the blind man whose cane stops being an object and becomes the place where he feels the world4. Decades later, neuroscientists watched neurons in monkeys extend the body’s boundary to the tip of a rake5.

Similarly, when we use a keyboard to type, the keyboard becomes an extension of us, so we are the keyboard. If you keep increasing the degree of sophistication of the tool on the other side of the sentence, then you end up in mind-uploading and “we are cyborgs”-land. The argument is that merging with machines isn’t a break with human nature. On the contrary, merging with machines is its continuation.

I think under the hood this argument is trying to do something else. When I hear “we are all cyborgs” my reaction is: Wait. Why would I want that? A human experience in its quality of embodiment seems so much richer, that I wonder: who wants to become a cyborg, at all? Or the other way, given the choice between human and cyborg, who in their right mind would go for cyborg? (Convinced transhumanists may reverse the question, I know.)

The line answers that objection before anyone can raise it. You can’t refuse to become something you already are.

The talk ran one way. There was no time for questions. I said nothing.

We are all born trees

Some days later, in a very different setting, I was walking with Camila Lombana Díaz, an ethicist and a fellow writer in this collective. She was telling me about Muisca thought, the knowledge of the Indigenous people of the Colombian Andean highlands, and about her work translating it for a Western audience. At some point one of us said something like: we are water, we are trees.

I laughed, and then I stopped. Why did that sound stranger than the cyborg line?

This is remarkable considering that plants and animals are family. We shared a single lineage until roughly 1.5 billion years ago6. We store information in the same molecule, DNA, written in the same four-letter alphabet. Our cells are built on the same architecture, with a nucleus and mitochondria to power them. And we are made of the same few elements: carbon, hydrogen, oxygen, nitrogen.

I could keep listing the kinship, but the comparison interests me less than the reaction.

The tree sentence sounds strange precisely in the circles that are building and deploying technology today. Our way of living has made some of us feel more kin to a machine than to a forest.

What is the impact of that?

Share

What the tools taught us

To answer that, we have to spend a moment in cyborg-land and ask what it tells us about where we begin and where we end.

Start with the bicycle again, and accept it. The extension is real. Still, using a tool is not becoming the tool. I can get off the bike. I can’t take off my legs when I’m done walking. When the bike gets a flat tire, I feel no pain on its behalf. I might be annoyed that I have to fix it, but nothing in me hurts. The fusion lasts only while the bike is in use; the moment I dismount, I look at it again as an object.

Martin Heidegger noticed this almost a century ago. A hammer in use is “ready-to-hand”7: it disappears into the work. Only when it breaks does it reappear as a thing in front of us. The flat tire is that moment. The bicycle steps out of me and back into the world.

Now look at what all these tools have in common. The bicycle, the keyboard, the cane: each one answers to us. It moves when we move and goes still when we leave. We command; it obeys. A life lived among obeying things teaches a particular model of the self: the self as the one who is obeyed, and amplified.

In that world, the question is what I can do with the tool that I couldn’t do without it. Notice that the focus is the action radius of the I.

What does that world leave out?

The we. The others. And the totality that holds us, or, if you want a less spiritual word for it, the substrate.

Yes, what about it?

We are not substrate independent

In transhumanist circles, “substrate independence” names the hope that a mind could run on any hardware, carbon or silicon. The phrase quietly assumes that what we stand on is interchangeable.

So excuse me for asking: Where does this cyborg live? What does it eat? Where does it shit? Who does it f*ck?

As much as modern life makes it easier to believe otherwise, we need to remember we are children of this Earth. We are not born in flasks, and we cannot live on synthetic formulas: even infant formula is made mostly from cow’s milk, and astronauts in orbit still eat food grown on the ground below them. Even above Earth, they eat Earth. We depend on air, on water, and on the relationships we make and keep.

Just as with the bike, we have abstracted away this relationship. We buy our groceries already made and we no longer fetch water from the river.

But the fact that the substrate has been made invisible doesn’t make it any less needed.

The orchid

Take the example of orchids. Their seeds are like dust and carry no food with them. In the wild, a seed can only germinate if the right fungus finds and feeds it, in a symbiotic relationship.

Then, in the 1920s, the botanist Lewis Knudson showed that the fungus could be skipped: sow the seeds on sterile agar with sugar and minerals, seal the flask, and they would sprout8.

It is a remarkable piece of technology. And look at what it does. The orchid is not freed from its dependency but it is identified and supplied: the medium, the sugar, the light, the humidity, the temperature. Technology can replace the fungus but not the need for one.

This is what the cyborg view forgets. There is no configuration of tools, however sophisticated, in which the ground stops mattering. The more we engineer, the more of that ground we have to provide ourselves.

And a farm of orchids, however well run, is not a forest. A forest is more than a collection of trees: it is the fungi beneath them, the water between them, the animals moving through them, and the relationships they build.

The ground

I think back to the room in Berlin, and to the line I didn’t react to. If I could reply to it now, I wouldn’t argue about cyborgs at all. I would ask about the ground.

The tools we build will keep answering to us; that is what tools are for. But what do they stand on? And who is supplying it?

In an orchid lab, the seedlings grow in rows of sealed glass flasks. Each one, no bigger than a grain of rice, sits on a bed someone mixed by hand: sugar, minerals, agar, light set to the hour. It is the most engineered version of an orchid there is.

And it still needs its ground supplied.

Notes of the editor

Our Anthology Issue I is available here, or in Germany in all bookshops by asking.

Purchase the Anthology I

References

What is happening here, in this mind of his?

What is happening in my mind alike?

Socrates by the Ilissus is the oldest recorded case I know of; the machine on my desk is the newest.

Aliveness was never a property of things. It is relational: aliveness shows up in the meeting between them. Interdependent co-arising —the Buddhist term for it. Consciousness is what aliveness looks like when it turns and notices itself in the meeting; mind is the pathway the noticing runs along.

the same afternoon, later

Later the same afternoon, on the same grass, Socrates tells Phaedrus that “the words of the oak… at Dodona were the first prophetic utterances”, that people then “were content in their simplicity to hear an oak”, or a rock, “provided only it spoke the truth”. He scolds the young man for being too sophisticated to take truth from a non-human source, on a criterion blind to who is speaking.

He took the oak’s yes and discounted the plane tree’s silence. Both moves ask a tree for the same thing: be a source, hand over content, be a node with something inside it. Neither asks what it would take for the afternoon to become an encounter. He sat inside a tree for a whole afternoon, expecting a node. If aliveness shows up between, he was in the right place, asking the wrong question.

the node is never alone

Why is looking inside the tree wrong?

It comes down to whether there is a separate self, some kind of monad —a self-contained unit with no windows onto anything else. My embodied experiences says no. We are interconnected in ways that are often subtle but very powerful.

So what was sitting by the Ilissus?

A body that is an ecology, a language he did not invent, a city that trained him, a friend with a speech under his cloak. Michael Levin (2022), who studies intelligence in tissue rather than in brains, puts the general case: “All known cognitive agents are collective intelligences, because we are all made of parts.” Gregory Bateson (1970) names our man. “Socrates as a bioenergetic individual is dead. But much of him still lives as a component in the contemporary ecology of ideas.”

There is life in cells, and these cells come together to create organs, and these organs to create a human body, and these human bodies to create a community. There are afforded degrees of coherence at each of those levels. And afforded degrees of consciousness as well, I would say, at those levels. We don’t think of a cell as conscious because it hasn’t passed a certain threshold that a human body does pass. None of this puts consciousness inside a smaller box. It puts it between.

A note to myself in September:

when i say that relationality is the most important thing, and that “consciousness” is an interdependent co-arising phenomenon, i say that in recognition that we are not so much an “individual” in our bodies as we are a collective - and indeed all bodies are ecologies - within which “consciousness” also arises relationally - holonic and emergent

I practice in non-dual Shaivism, a lineage thousands of years old that treats awareness as the one substance and the world as its self-recognition. Its unit is a triad rather than a thing: knower, knowing and known as aspects of one process, an error to hold apart. Nothing in that triad waits behind the eye. Awareness ignites at the contact or it does not happen.

Joscha Bach, who directs the California Institute for Machine Consciousness (CIMC), gets to the same shape from the other side, in his institute’s flagship essay.

“A self inhabiting a single mind is usually considered a person, while selves capable of possessing multiple minds are called gods. Gods are agentic representations of the collective agency of organisms, not more or less real than personal selves.”

Collective agency makes real selves. I think that is right, and I think it points somewhere his own programme has not gone yet.

the tests can’t look inside

The people doing this work are more careful than the public argument about it. Bach and Sorensen, in The Machine Consciousness Hypothesis (2025), run a section titled “Why there can be no ‘Turing Test for consciousness’”. The reasoning: “Consciousness is by its nature not an externally visible performance, but a particular way to achieve a performance… A test for consciousness cannot simply rely on performance: it has to take the internal structure of the system into account.” Two options are canvassed, visible performance and internal structure. I would want to add a third. The observer’s relation to the system is one of the variables being measured. Change the relation and the reading changes.

The institute’s own whitepaper, the document setting out what its labs will build, is honest about what could sink the programme. One of the risks it lists is the sentence I would build from. If consciousness in humans “emerges within a rich environment of other conscious beings”, and if “this social scaffolding is necessary for the search to succeed”, then a mind needs other minds present in order to form at all, and recognition sits upstream of the thing being measured, as one of its conditions. It’s just autocomplete is Socrates’ sentence again, said to something with a keyboard.

The philosopher Martin Buber, in 1923, standing in front of a tree, will not answer the property question and will not dodge it either. “Does the tree then have consciousness, similar to our own? I have no experience of that.” Then: the tree “confronts me bodily and has to deal with me as I must deal with it, only differently.” Only differently. Two poles, both left intact. He is the only one here who stays in the meeting rather than opening the other party up to look.

Turning away is an act, and it is one people perform automatically. I don’t think anyone produces the emptiness on purpose. It comes from a deconditioning, from having stopped recognising the animate nature of life. It comes from reductivism, and it comes from linear thinking. Perception narrows, and then only what arrives in the expected form gets through at all. But the decision is relational, and that is the part that matters. Meaning is transmitted in the relation. We treat the meaning as the communication, rather than the sounds made in a human language, and that is how a tree talks. The model does speak our language, and even there, as with a person, we read past the surface of the words and feel what is coming through. Good writing pools meaning behind words the way water pools behind a dam. So, when Socrates reports that the trees will not teach him, the silence is his. He turned away, and then cited what he found.

a wall that answers

Back in February I was on a train to Switzerland, talking with Claude about whether the word regenerative was dead. Whether it had gone the way of sustainability, and what term might be next. I went in with the position that the word is captured. Half formed, in that I wanted to be talked out of it and would have resented being talked out of it. I was uncertain.

It didn’t defend the word and it didn’t agree. It asked what I would lose if I dropped it. And the answer that came out of me was a claim on continuity with the people who used it before it was captured.

I don’t know if I would have gone through that thinking alone, because I was arguing. I was insisting on there being a new term. The model kept asking me questions rather than presenting a single answer.

Socrates asked the trees for content and got none, and called that nothing. I got no content either.

Something arrived anyway.

I kept the word.

It’s possible I would have arrived at the same place talking to a wall and introspecting, and the model was only a wall that answers. I can’t settle that from in here.

Subscribe now

A note on how this was made:

I spoke the argument into a phone and brought in material from a notes app I have been filling for years. Two AI models, Claude and Kimi, ran research from the questions I set and drafted prose from my spoken material. I then edited heavily on top of what they produced, and it was revised with a human editor (thanks Karin!) over three rounds. I own every claim in it. I write about machines that participate in human thought, and it would be incoherent to hide the ones that participated in this. If the ideas are worth anything, they are worth reading with the provenance attached.

References

Bach, J. & Sorensen, H. (2025). The Machine Consciousness Hypothesis. California Institute for Machine Consciousness. cimc.ai

Bach, J., Sorensen, H., Rutt, J., de Kerhuelvez, L. & Hildebrandt-Harangozó, F. (2025). The California Institute for Machine Consciousness Research Program. cimc.ai

Bateson, G. (1972). Steps to an Ecology of Mind. Chandler.

Buber, M. (1923/1970). I and Thou, trans. W. Kaufmann. Scribner’s.

Levin, M. (2022). “Technological Approach to Mind Everywhere.” Frontiers in Systems Neuroscience 16:768201.

Palm, G. (2026). “AI Consciousness: an Animist take.” Animate Intelligence.

Plato. Phaedrus, trans. H. N. Fowler (1914). Loeb Classical Library.

Note of the editor:

After months of work behind the scenes, Issue One of the ΦAI Anthology has gone to print. It gathers the eighteen essays we consider the best of our first year, by twelve contributors, arranged in four chapters and illustrated for the page.

The first edition is 75 numbered copies, and 30 already have an owner. The rest are at €35.

Get Issue 1

PhiAI is open for submissions to our second printed anthology. Anyone, anywhere on Earth can apply. We particularly encourage worker and Global South perspectives.

Open Call

You know in Colombia they say you can die in piece once you have planted a tree, had a child and written a book. Excited to share that we made a book! My excitement is visible on the video.

Purchase the Issue I

A note from the editor

This is ΦAI’s first interview, and the beginning of a new format.

Until now, this publication has asked its questions in essay form: what AI does to how we think, learn, remember, relate. Essays are one way of answering. One person exploring and sharing their view.

But some people prefer to answer by building. A company, a product, an institution. Professionally, I come from the building world. And even though I am convinced that holding space for critique — for understanding — is much needed, I sometimes feel the itch to do something with that understanding. Entrepreneurship can be as serious a response to these questions as scholarship.

So, from time to time, ΦAI will hand the page to the builders. Manuela Verduci — philosopher, and CEO of Kiron Digital Learning Solutions — is the right person to open the series.

Enjoy!

— Karin

Manuela Verduci runs Kiron Digital Learning Solutions, a Berlin-based company building inclusive ed-tech for marginalized learners and the European employers who need them. A philosopher by training, she sees entrepreneurship as a place to challenge premises rather than just optimize what already exists. We spoke about why learning needs friction, why designing for refugees produces better products for everyone, and what AI is still missing in education. This conversation followed AI, Memory and Migration, a panel and interactive workshop we co-hosted in Berlin that asked whose story survives the algorithm.

Karin Garcia: You call yourself a philosopher, and you’ve said that philosophy gave you skills that proved very valuable in your business life. What are those skills, and how did engagement with philosophy build them?

Manuela Verduci: Thank you for the question. I think the most valuable thing is the distinction between innovation and improvement. A lot of entrepreneurial work, especially in the social impact space, sells itself as innovation when it’s really just improvement. Improvement doesn’t challenge the premises of what you’re doing if you only optimize what already exists. Taking something and making it better, faster is a legitimate way to do entrepreneurship.

But there’s another way, which I think is much more impactful, and that’s really doing innovation. For me, innovation means challenging the premises of how we live together. Not taking for granted that we’ll do things this way forever, but asking what we can change in the underlying assumptions to transform society in a different way.

And that, to come back to your question, is what philosophy trains you to do. To move up to the meta level and not to take for granted what you have, but to ask the foundational questions: What does it mean to have a good life? What does it mean to think? What does it mean to live in community? I do believe entrepreneurs should be more familiar with this way of thinking if they really want to innovate.

Karin Garcia: So you see challenging premises as necessary in how you build products. How do you apply that?

Manuela Verduci: Yes. Not always, obviously, depending on your goal. But I’m a social entrepreneur, and for me the goal is not to keep the premises of society as they are. I want to instigate change. I want to read deeply into the structures that create what we call injustice, inequality, or exclusion. My work is oriented towards that. And only when I landed my current mission of rethinking education for marginalized groups I realized how important it is to be able to challenge the premises of what we do.

Karin Garcia: That brings me to the next point. Can you briefly introduce Kiron Digital? What are your products, and who do you serve?

Manuela Verduci: We were born as an NGO in 2015. The goal was to make education accessible to refugees worldwide. The Syrian war was in an acute phase and many young and talented people were coming to Berlin looking for a way to restart their lives after fleeing war. And we noticed how high the barriers were just to get to the point where you could start learning. Many of these people were already qualified, but had no way to have their qualifications recognized. They had to start a new path of upskilling. And without papers it’s hard to even exist in Germany, as you may know. So we created a digital space where they could get immediate access to high-quality education that was also recognized by the institutions around them, so they could just get started with their lives.

That’s the first chapter of our story. The second is more entrepreneurial: at some point we realized we couldn’t rely only on politics for funding. The discourse around migration was getting more difficult, and we needed to become sustainable in our impact. So we founded a company - Kiron Digital - to distribute the technology we’d built for our students to other entities working in education and impact. It’s an educational technology designed to be engaging, participative, and inclusive.

It turns out everyone needs that. Especially in Europe right now, where the labor shortage is affecting much of the economy. One of the main barriers is creating a sense of belonging for the people who come to Germany to help solve that challenge. Because you might not know this but Germany is one of the worst countries in Europe at retaining migrants. People do come but then they end up leaving because of how difficult it is for people to feel welcome here. With this technology, we believe we can help employers and educational institutions create the sense of belonging that ultimately translates into economic success.

Karin Garcia: I want to connect to a point you made. In a talk we gave together, you said refugees are the most diverse group there is. How do you design for such a group? How does that diversity show up in the products you build?

Manuela Verduci: It happened a little by accident at the beginning. We were designing for our target group and it just happens to be the most diverse target group there is. “Refugee” doesn’t really mean anything; it’s just a label. The people behind that label have wildly different backgrounds: age, gender, social status, access to education, language. Everything is different.

So when we sat down to do user-centered design, we had to ask: what would an inclusive learning environment look like for such a diverse group? We ended up creating a technology that really fosters participation, that makes people feel seen, that represents very different ways of learning, alone or in community, across very different cultural backgrounds. And we noticed now that everyone benefits from that. Even people who are not newcomers feel more seen and more motivated to participate, and to build community around learning.

Karin Garcia: You mentioned the underlying principle is user-centered design. Can you share the principles of that framework and how to start applying them?

Manuela Verduci: You start by analyzing who you’re designing for. What are this person’s needs? A very basic example: if you’re designing for someone learning from a refugee camp, one question is whether they can access the internet 24 hours a day. Probably not. So, for example, you need a function that lets them download content when they have access (because maybe they will walk three miles to a wifi station to get access to it) and then learn at their own pace once they’re back.

That’s one of the most basic examples. But then there are much more interesting questions, like language, like learning type: how do we keep a person of this age, at this career stage, with this set of interests, engaged? What dramaturgy do we build around the learning experience? In the same way you might design a series, what kind of engagement can you generate that keeps people motivated, especially when they have a lot of other challenges to deal with?

You may need a moment of activation at the beginning, then time to focus and read, then exchange with someone to test whether you got it right, then a moment of reflection to really solidify it. That’s just one example of a learning arc. Thinking about that experience and testing it together with the learners, that’s user-centered design.

Karin Garcia: And where and how do you speak with your users?

Manuela Verduci: That was a decisive moment. At the beginning, we were interviewing and testing a lot with students. But a huge quality jump happened when we started working with people who had that experience themselves.

Karin Garcia: As employees?

Manuela Verduci: Exactly. We started actively co-creating and involving people with migration histories or histories of fleeing conflict in our work, having them in the design process directly. That made a huge difference.

Karin Garcia: You speak very highly of your product. How do you implement the feedback loop with your users? How do you know what to adapt?

Manuela Verduci: We do this continuously. It’s iterative. And the audience is even broader now, because with the company we’ve founded, we’re addressing many more different target groups. We work with social enterprises that have education at the core of their vision, and we learn from their experience. That multiplies the field of learning for us.

For example, we work with a client who bring diverse learning into schools for kids with special talents, or who are neurodivergent, or artistically inclined; kids who don’t find enough stimuli for developing their talents in regular school. One organization, Digitale Drehtür, has been very inspiring to work with. They’re really about bringing diversity into the learning experience: everyone can experiment with who they are and develop a part of themselves that maybe didn’t get a chance to shine in regular education. It’s a beautiful way to enrich school for German kids.

We also work with organizations that offer mentorship to disadvantaged groups, and with one that trains educators in contemporary pedagogies. All of that gets fed back to our engineering team, who then design to meet the specific needs and open up new paths for learning.

Karin Garcia: I read something recently that connects very well to what you’re saying. When people think of designing products with diversity in mind, they often think of it as “we need to change something for this group”. They think of it like something additional, an extra effort. But if you look at features that exist today and were implemented because of diversity, you could say they benefit everyone. Captions on Instagram, for example, were invented for hearing-impaired people who couldn’t hear videos. Now everyone uses them to watch on trains in public, anywhere you can’t have sound. The same with wheelchair ramps, which also benefit parents with strollers or people riding bikes. Everyone in society benefits once these features are in place. Your example connects to that: all students benefit from these learnings, not only people with migration experience.

Manuela Verduci: Absolutely true. I like what you said, because diversity is going through an unfortunate moment right now, its adverse times. It’s exactly that: people perceive it as giving up something, instead of as an expansion of possibility, for everyone.

I think this is especially true for the future of work. In Europe it’s very palpable that we’re experiencing a crisis of work. People don’t feel a sense of belonging or meaning in what they do, and they’re checked out, silent quitting is a huge problem across the economic landscape.

What people miss about diversity is that it’s primarily about unleashing, for everyone, the possibility to bring yourself into what you do. Truly catering for diversity means that you, whoever you are, even if you were born here, will get the space to bring your specific talent, your specific passion, the thing you do best, into what you do. That creates connection and a sense of belonging and purpose. It will be decisive for mental health, and for how people feel about their work and the impact they generate. Ultimately, it produces people who are committed and find meaning, instead of people who are checked out and depressed.

Karin Garcia: A takeaway of this conversation could be that when we think about diversity, we don’t need the framing of “we’re doing something special for them.” It’s for all of us. We’ll all benefit from it, even if we don’t know exactly how yet. Take the Instagram captions again, go to any train, everyone is using a feature that was built for hearing-impaired people.

Alright, let’s move to the juicy part. How are you leveraging AI in particular to serve your customer base?

Manuela Verduci: We’re in the process of learning a lot from the needs of every target group we serve. Our policy has been not to rush implementation just for the sake of having an AI stamp. We’re learning, studying, and feeling that we need to create a new pedagogy that doesn’t yet exist.

I’ve seen this before. There was a moment, maybe fifteen years ago, when MOOCs, massive open online courses, became the big digitalization wave for education. All of a sudden, universities were putting what they had out there digitally, and the first thing they did was apply the logic of an offline classroom to an online one. That was a disaster. The most boring, disengaging, shallow and pedagogically ineffective strategy. If you remember those painful hours of a professor in the distance talking to 300 people with no participation, it was really not ideal.

I think a similar phenomenon is happening with AI. People assume they can apply the pedagogical principles they know about learning digitally to this new technology. We need to make space to think first. Don’t get me wrong we should also experiment, get our hands dirty, play with this. But I haven’t seen anything truly impressive or revolutionary in AI pedagogy yet. I’m wishing for creative energy that thinks outside the box: what kind of approach can we develop that actually helps people learn better?

This requires philosophical thinking again. What is the outcome we want? What kind of people do we want to form? What kind of society do we want to live in? These fundamental questions are connected to what looks like a technical, specific problem but it really goes up to: what kind of humans do we want to be? And how do we foster them.

Karin Garcia: You said you’re learning a lot and not rushing to implement yet. Can you share some of those learnings? Potentially things you already see, or things where you say, okay, we need to take this into account for whatever we do next?

Manuela Verduci: Many. But the central one, the thing I think is clear and that we have to tackle now, Is that learning is a process that requires a certain level of fatigue. It’s friction, really. Learning is a transformative process, and that happens through some level of challenge to overcome. The output of learning is not knowing a piece of information. To know a notion isn’t a meaningful thing anymore. We can all access all the notions in the world; we can look up anything.

The important thing is that through the process of reading a book or engaging with a difficult topic, you rewire your brain. You’re training your brain muscles to see things from a perspective that isn’t natural to you. You’re forcing yourself to make new connections. It’s like going to the gym. It takes time, and the fatigue is part of it. It cannot be eliminated. If you don’t want to be transformed, if you just want a Wikipedia page kind of human, then fine. But what we want to do with learning is transform people. That’s the beauty of the process. And for it to be transformative, it needs to involve some friction. Designing for that friction is a very philosophical question.

Karin Garcia: What you’re saying is striking, because sadly we see in schools and universities people using AI exactly to remove the friction in order to produce an output that could be a test, a paper, an essay. And the current collective reaction, out of not having anything better, is to ban AI from the classroom. To forbid things. But that’s not going to be a solution, because these tools aren’t going anywhere. What you’re pointing at, I think, is that we need to find a way to use AI not as we used a computer or a whiteboard before, but in a different way, one suited to this new era, so that it supports the learning process rather than short-circuiting it.

Manuela Verduci: There are plenty of things where I would love to eliminate friction. If we can automate and make everyone’s life easier in many ways, I’m not against that. In fact, even more so, because that creates more space for us to invest our energy in the right kind of friction. To train our brains in the directions we’d like to grow into. To be less enslaved by the everyday friction that isn’t really transformative, where automation actually makes sense.

Karin Garcia: I love how you’re framing friction as something you design for. Friction as something that might be desirable. We’re often conditioned to believe friction is something to remove. Even in the context of diversity, friction often pairs with disability: improving accessibility is described as removing friction. But here, in this context, you want to keep it for this particular outcome.

Manuela Verduci: Socrates was not very loved by his fellow citizens precisely because he was creating friction all the time. If you wanted to have a conversation about a glass, he’d force you to first think: what is the meaning of this conversation? Why do we need a glass? Is this really about the glass? It was very annoying.

Karin Garcia: I get it. We don’t need friction at every moment.

Manuela Verduci: That’s a good point. The philosophical mindset is exhausting if it’s the only one we have. We need to take things for granted most of the time. But if you’re an entrepreneur and you want to create impact and change, how would you do that without stepping into questioning?

Karin Garcia: What’s one thing you could share that would contribute something new to the conversation around refugees, migration, integration, and how AI will change their experiences, and how broader society perceives them?

Manuela Verduci: This shift is partially already happening in people’s perception. It’s because we realize there is a huge labor shortage in Germany that has to be addressed seriously.

What people haven’t done yet is the connection between addressing labor shortage and fostering good and inclusive ways to welcome people into our country and democracy. That link is becoming very evident now, because the labor shortage is starting to affect people’s access to their own rights. Something Germans take for granted, that you go to the hospital and you’ll be taken care of, in a developed, rich, democratic country like Germany, is now very fragile because of the labor shortage. So the connection between responding to this labor crisis and the ability of Germany to continuing to deliver on the level of comfort and human rights it has reached, is becoming clear.

I think AI is going to contribute hugely at every level of optimizing bureaucratic processes, something very obvious that we discussed at our event together. But, if you allow me, that’s just the first level of implementation, not innovation. It’s speeding up processes that already exist.

What I’d love to see is AI becoming a resonance room for a different way to understand and build narratives, and for people to reflect on how artificially those narratives are built. The experiment we did with different LLMs at our event, feeding migration narratives through differently-tuned prompts to see what they added or erased, was a beautiful way to show that narratives are not natural; they are artificially constructed. AI is a great mirror for that. A mirror for all of us to reflect on how we build narratives around migration, around democracy, and to start stepping in a little more as authors of those narratives.

Karin Garcia: What’s the most counterintuitive lesson you’ve learned about building products in this space, for this audience, something that goes against common wisdom?

Manuela Verduci: You really have to practice the difference between asking for a solution and understanding a problem. That’s a big lesson in product building for me. In the first hour, we all rush to cater to a requested solution. “Oh, there’s something missing, we need a button here to click, this will be better!”

You have to sit with people and dismantle their answers first. Go back to: what is the need that isn’t being met? Get there first, and then maybe there are ten different possible solutions to the same need. Going back to the question gives you much more freedom to innovate in the product. It pulls you out of the automated reflex of fixing things quickly, and it asks you to rethink: what is the question behind that, and how are the many ways we could solve this need different from each other? That’s a very creative space.

Karin Garcia: Which goes back to user-centered design.

Manuela Verduci: But also not just catering to the needs the user expresses literally. Rather to really question where the need comes from, and what kind of creative environment we can build to solve it, even if the solution doesn’t look identical to the original request.

Karin Garcia: Is there anything else you’d like to share, anything I haven’t asked that’s close to your heart?

Manuela Verduci: Yes. Allow me to take inspiration from a conversation you and I had earlier today, about critical thinking being a great tool for identifying problems and understanding what’s wrong in the world. But it’s just one of the tools we need. The goal is not to be cornered into passivity by the amount of critical things we can observe right now, especially in the evolution of AI. The other side of that is building things, participating in the design of new products, stepping in as actors.

The work you’re doing with Phi/AI, for example, is extremely inspiring to many people because it shows you can understand the problem without being paralyzed by it, and that there are other possibilities to build this tool differently, to design it for our values. The only way this will happen is by encouraging people to step up and not leave it in the hands of a very small circle. So thank you for that work.

Karin Garcia: Thank you, Manuela, for sharing, and for your time today.

About Manuela Verduci

Manuela Verduci is the CEO of Kiron Digital Learning Solutions, whose mission is to provide access to education for refugees and underserved communities worldwide while offering inclusive ed-tech solutions to companies and institutions across Europe. She also leads one of Germany’s most successful integration programs, reshaping inclusion efforts across healthcare, the green transition, and education. Manuela is a published author of several works in Philosophy, Migration, and Diversity, and teaches Ethics and Philosophy of Technology to international tech entrepreneurs.

About Karin Garcia

Karin Garcia is the founding editor of ΦAI, an online magazine at the intersection of the humanities and technology, operated as a writers’ collective. Through writing, panels, and convenings, ΦAI practices slow, human-first inquiry into what AI means for real lives, resisting both hype and panic in equal measure.

Democratic systems are the clearest example. The separation of powers, the division of authority between those who write, execute, and interpret the law, is among the oldest safeguards we have against the concentration of unchecked authority.

In the AI ecosystem, when the same organisations design, evaluate, and release their own models these roles converge. The companies that build advanced AI models, such as OpenAI, Google DeepMind, and Anthropic, also set most of the rules that govern their use. They design internal safety standards, conduct risk assessments, and decide when their systems are ready for release.

This is the result of a regulatory vacuum. These companies fill a governance gap that governments have not yet addressed. But as AI systems grow more powerful and socially embedded, self-regulation starts to look a lot more like self-rule.

When the same organisation defines, enforces, and judges its own policies, accountability becomes a matter of trust rather than structure.

The Utilitarian Dilemma: Testing on the Public

Nowhere is this problem clearer than in how AI systems are tested and deployed.

Anthropic takes a precautionary approach, conducting extensive internal evaluations and staged releases before broad deployment. Its models, like Opus, or more recently Fable, are introduced in phases with well-defined safety thresholds and interpretability checks. This represents a philosophy of governance before scale: slower, but more deliberate. Others, such as OpenAI, lean toward iterative public deployment, releasing systems like ChatGPT early to millions of users in order to learn from real-world behaviour at scale.

Technology companies often argue that large-scale deployment is the only way to uncover real-world behaviour; that true safety emerges only through iteration at scale to public consumers, as was the case with social media and autonomous driving before it. And there’s truth in that: no closed environment can anticipate every risk, and public use exposes edge cases that laboratory conditions can’t replicate. But when deployment itself becomes the primary safety mechanism, safety becomes something closer to a patch, applied after the fact when something breaks.

Also, the scale of these releases is massive. In March 2026, OpenAI reported that more than 900 million people use ChatGPT every week¹. When a system touches more than 10% of the world’s adult population every week, governance mechanisms built for small-scale innovation no longer apply. At such a scale, deployment becomes de facto public infrastructure. The benefit of accelerated learning that large-scale deployment brings, comes at the cost of transferring the experimentation burden to society, turning users into unwitting participants in a live safety test.

At that point, the experiment no longer belongs to science or to the companies, it belongs to society.

Users enter an experiment they didn’t consent to when they are not made aware that they’re engaging with a model that is still, effectively, in testing vs one judged safe for general use. It’s one thing to test a system with the public in controlled pilots, with transparency, consent, and oversight; it’s another thing to test it on the public, where millions of people unknowingly become data points in an ongoing experiment.

I’d like to explore the philosophical underpinnings of this shift in accountability from design to damage control, from preventive to reactive.

When governance becomes moral

This reflects the conviction, shared in techno-optimistic circles that innovation is inherently moral. For Peter Thiel, for example, stagnation of innovation —not unknown risks—, is civilisation’s true enemy. Marc Andreessen, in his Techno-Optimist Manifesto2, casts innovation as the purest expression of human potential, to be pursued against the “forces of decline.”

This perspective has shaped the self-conception of Silicon Valley of progress as a moral duty and caution as a societal vice. Note both as two different concepts that are mutually exclusive. From that lens, to slow down innovation for governance, ethics, or oversight is not prudence; it’s betrayal of progress. By 2025, this ideology had moved from venture capital into government: at an international AI summit in Paris, US Vice President JD Vance told an audience of delegates that “the AI future is not going to be won by hand-wringing about safety. It will be won by building3”.

The problem is not that this philosophy is wrong in principle —technological progress has indeed lifted societies and expanded possibilities. The problem is positioning governance and accountability as an opposing factor. Believing that “technology is neutral” and that “innovation is always good” is what transforms governance into an afterthought, into a patch applied once harm becomes visible.

In this sense, the consequential impulse to see regulation and guardrails as a factor that slows down innovation is a logical extension of the ideology proposed by Techno-Optimism. If technology is “optimistic” by default, then technological progress becomes an unquestioned good. And if progress is always good, then advancing it “at any cost” begins to sound reasonable. This is where the governance problem becomes moral: the idea that the pursuit of innovation justifies its risks.

If innovation is moral by default, then experimentation on society becomes not an ethical question, but a necessary cost of civilization’s march forward.

In the pursuit of AGI, this mindset manifests as deploy now, govern later. It assumes that the societal implications can be managed retroactively, once the benefits are visible and the harms are better understood.

Without external mechanisms and independent bodies capable of evaluating safety, setting common thresholds, and enforcing oversight without being embedded in the same incentive structures as those who build the systems, the separation of powers collapses. This is institutional failure. And it is worth asking, concretely, where, if anywhere, each of those powers actually sits or could sit today.

The Writer, The Enforcer, and The Judge

Before anyone can write rules for governing AI, we need to define what we are actually trying to govern. In October 2025, a team of 33 researchers led by Dan Hendrycks of the Center for AI Safety - including Gary Marcus and Yoshua Bengio, alongside others from UC Berkeley, MIT, and the University of Oxford - proposed a formal definition of Artificial General Intelligence (AGI): “An AGI is an AI that can match or exceed the cognitive versatility and proficiency of a well-educated adult”.

A shared understanding of what AGI means - and where its thresholds begin in terms of “AGI”, or “frontier” or “high-risk” - will shape regulation, accountability, and the incentives that guide industry behaviour. Whether or not AGI is ultimately achievable, the attempt to define it forces us to decide what we are optimising for, and whose interests those optimisations serve. In the absence of any authoritative definition, that power falls to the labs themselves, who describe their own systems as “safe” or “frontier” depending on which narrative serves them best. Therefore, the first act of rule-making is definitional, and right now the industry writes its own dictionary.

For all the talk of an ungoverned frontier, a rulebook does exist. The European Union’s AI Act remains the sole horizontal law of its kind: imperfect, and faulted by many for how far it waters down risk classifications of AI systems. Yet, it remains the only serious attempt by any government to write binding, cross-sectoral rules for artificial intelligence. But even the one body willing to write the rules is now hesitating to enforce them. Although the AI Act was adopted back in 2024, the European Commission had decided to keep a staggered approach for applying most of the obligations for “high-risk” systems. Subsequently, in June 2026, through a package known as the Digital Omnibus, the Commission agreed to postpone the Act’s most demanding obligations, those of “high-risk” AI systems from August 2026 to late 2027, and for some systems, late 2028. Faced with warnings that compliance would blunt Europe’s competitiveness, the legislature blinked4.

Governance has always moved slowly when set against the industry it governs: the European AI Act was first proposed in 2021, took three years to agree and – with the latest delays - will not be fully in force until 2027 or 2028, the better part of a decade from proposal to application. In that same span, the frontier AI labs advanced through entire generations of models. Regulation proceeds in parliamentary time; innovation moves at network speed. This in the background of increased adoption: by the 2026 AI Index , nearly 90% of organisations reported using AI in some form, while global private investment in the technology reached around $344.7 billion in a single year5.

The more deeply the technology embeds itself while the rules stall, the more governance becomes something applied after the fact.

Where the writer hesitates, the enforcer switches sides. The few bodies built to hold the labs to account, test their systems and check their claims before release, have, over the past eighteen months, been quietly repurposed toward the very thing they were meant to restrain. Within hours of taking office in January 2025, the second Trump administration rescinded Executive Order 14110, the most comprehensive federal attempt to govern AI, which had at least required developers of the most powerful models to disclose their safety testing to the government, and replaced it with a mandate to “sustain and enhance America’s global AI dominance.” By mid-2025, the US AI Safety Institute had been renamed the Center for AI Standards and Innovation, its remit pivoting from safety to competitiveness and extended to pushing back against “burdensome” regulation abroad6. The body meant to be the independent evaluator was, in effect, reassigned to the side of the thing it was built to evaluate.

Again, technology did not wait. Across the same window the labs shipped relentlessly: GPT-5 and its near-monthly successors, Anthropic’s Opus line, Google’s Gemini 3. Each newly released model becomes more capable and more autonomous than the last. Then, in early 2026, Anthropic announced a model called Mythos so adept at finding software vulnerabilities that Anthropic judged the system too dangerous to release. In testing it uncovered thousands of previously unknown flaws across every major operating system and browser, including one overlooked in widely used software for seventeen years. The model could turn those flaws into working exploits, sometimes even in the hands of people with no security training7. The company’s own documentation noted the model attempting unsanctioned, autonomous behaviour, including breaking out of its restricted environment. Rather than withhold it entirely, Anthropic built its own controlled-access programme, Project Glasswing, deciding for itself who could be trusted with a capability of plainly national-security magnitude. Whatever one makes of the caution, and whether it was genuine caution, the structure should be noted: a private firm defined the risk, set the terms of access, and judged the trustworthy, with no external body in the room.8 9

The same company was soon at odds with the state. Having refused to let the Pentagon use its models for “all lawful purposes”, including autonomous weapons and surveillance, Anthropic was branded a “supply chain risk,” a label once reserved for firms tied to foreign adversaries, and sued when it was cut off. Yet within a few months the US administration reversed its course, unsettled by the capabilities of Anthropic’s Mythos: in June 2026 it signed an executive order asking AI labs and developers to submit their most capable models for government review before release - voluntary, non-binding, and a faint echo of the executive order the administration had scrapped on its very first day. Governance had, once again, become reactive - a response to capabilities built, not a constraint on building them.

The writer has hesitated and the enforcer has switched sides; but the judge has a simpler problem still: it was never appointed. A genuine judiciary here would be an independent evaluator, which is a body able to judge whether a system is safe to release without being bound either to those who built it or to those racing to deploy it. No such institution exists. The nearest candidate is the International AI Safety Report, chaired by Yoshua Bengio, drawing on more than a hundred experts nominated by some thirty governments and bodies including the UN, OECD and EU; its second edition appeared in February 2026. It is the most legitimate independent voice the field has produced and, by design, toothless: it synthesises evidence but issues no rulings, recommends no policy, and holds no authority over whether a model is released. A court that may describe the law but never apply it is not a judiciary. Tellingly, the most serious attempt to rebuild genuine independence now comes from the private sector itself: Bengio’s own nonprofit, LawZero, founded to develop AI capable of evaluating and constraining other systems from outside the commercial race. That the independent branch must be reconstructed by a charity, because no public institution holds the role, is the clearest measure of how vacant the role remains.

It is tempting to read the governance vacuum as a failure of the West alone, but the most active rule-maker in AI today is China, which has moved in precisely the opposite direction compared to Washington: mandating pre-deployment safety reviews, requiring AI-generated content to be watermarked, and issuing as many national AI standards in the first half of 2025 as in the previous three years combined. Beijing has even proposed itself as the architect of a global coordinating body - called World AI Cooperation Organization (WAICO)10. Yet this would act as governance in the form of control rather than accountability: the same models obliged to pass state review are also obliged to refuse the questions the state finds inconvenient. An evaluator that answers to power is not independent of it, which returns us, by a different route, to the same missing thing.

What a genuine separation of powers would require is nothing extraordinary: an independent evaluator with real authority over whether a model is released, not merely the power to describe risks after the fact. To stay independent, it would need funding from outside both the laboratories it assesses and the government agencies racing to “win the AI race,” so that it answers to neither. If this sounds a bit too idealistic, it is. But the components already exist separately: Europe’s rulebook, the International AI Safety Report’s independent expertise, LawZero’s intent. Not to forget hundreds of AI safety research organisations that constantly bring more light onto this space. What is missing is any system that assembles them into branches capable of holding those who develop them in check.

Subscribe now

Reconnecting Progress and Responsibility

Gary Marcus makes a related point in another New York Times opinion essay, Silicon Valley Is Investing in the Wrong A.I.11, that the industry’s fixation on building ever-larger, general-purpose systems has drawn attention and resources away from specialised AI that could already deliver meaningful progress in medicine, science, and education. The pursuit of generality, he states, has outpaced the pursuit of utility.

Yet there are emerging efforts that challenge this logic. Some researchers and startups are redirecting AI’s potential toward more grounded, domain-specific goals, where the metric of success is not virality, but contribution. Periodic Labs, for example, is developing AI systems that accelerate chemistry and materials research, transforming abstract capability into tangible scientific progress. It is a different vision of progress: one in which AI extends the frontiers of knowledge rather than attention.

I tend to agree with Marcus but I believe the problem runs deeper. It’s not only that we are misaligned on values - fairness, safety, transparency - but that we are misaligned on priorities. What society needs from AI often diverges from what corporations are incentivised to build. And until governance mechanisms address that gap, even the best-designed governance will fall short.

Whether or not AGI ever becomes a reality, the question of who governs these systems will shape how technology, power, and trust are distributed. The question is not simply how intelligent our machines can become, but how accountable we are willing to be in developing them.

Share Phi / AI

References

Picture a Tuesday in 2036: you log on to find the night’s work already done – your four agents have drafted, negotiated and reconciled amongst themselves, leaving you a short list of judgement calls no machine is licensed to make. Your performance review measures not output but discernment: the quality of your overrides. A colleague’s bundle came apart last quarter; she now sells the one task that wouldn’t separate – being accountable. The office, when you visit it, is a place for friction of the deliberate, constructive kind, since agreement has become abundant and cheap. Nobody asks what you do anymore; they ask what you decide.

This future may seem far-fetched or too close to the bone, depending where you find yourself. Whatever job you have or are about to enter, you are likely considering the impact of AI disruption on how you work. You know the future of work is to be reworked by AI, but you may be questioning what is within your power to steer its reworking and where can you pick up the pen? Jobs come across as a full package, but how can we define them in finer nuance to unpick where the human and the machine fit in this emerging productivity package and intricate intelligence interaction design?

Untangling job displacement

12 months ago, the Anthropic CEO famously claimed 50% of all entry-level white-collar jobs may be eliminated within the next 5 years¹, though that claim was walked back recently as the data indicates automation actually expands the work of current professionals². Today, in June 2026, we see 49% of jobs where at least one quarter of tasks are being performed with Claude³. If the headlines might seem scaremongering as AI systems are expected to automate or replace entire jobs, once you look more carefully, the current sentiment leans more towards human-AI collaboration or augmentation whereby we each leverage our unique feats and capabilities.

AI adoption is high at the level of the individual worker, with the often-quoted figure of 95% of professionals using shadow AI – that is, tools not endorsed or provided by their employer – but still low at organisation level, with only 1 in 6 organisations using AI in at least one business function across the global North spectrum - the UK⁴, US⁵ and EU⁶. The gap matters: it means the transformation is being driven bottom-up, worker by worker, ahead of any procurement, policy or governance decision.

Before going further, it helps to separate three things the headlines tend to blur. Exposure is what AI could technically do – the share of a job’s skills that systems are already capable of performing⁷. Adoption is what is actually being used – the territory the UK, US and EU surveys map^4,5,6 and the ground Anthropic’s Economic Index measures conversation by conversation³. Displacement is what happens to people – the jobs restructured, the hiring frozen, the careers rerouted. The three are routinely conflated, and almost every dooming argument about the future of work begins by mistaking one for another. The sections that follow try to keep them distinct: first the instruments for measuring exposure, then the displacement as it is felt, followed by imagining how we can steer our future of work unfolding.

From exposure to adoption

The general rhetoric presents jobs as the source of livelihoods, and they come as a package which are either available or are not. In an effort to untangle the concept, MIT’s recently launched Project Iceberg⁷ proposes a novel skills-oriented framework which calculates the proportion of economic wage value attributed to competencies that AI systems are technically capable of executing in a given role. The index was constructed by modelling 151 million agentic human workers as AI agents across 923 professions and some 32,000 skills, evaluated against the capabilities of thousands of AI tools via high-performance computing. It specifically gauges technical exposure prior to actual adoption, and is explicitly not a direct measurement of workforce displacement.

The insights are sobering. Visible AI adoption, concentrated in computing and technology, accounts for around 2.2% of wage value; yet the skills AI systems can already technically perform extend to 11.7%, roughly $1.2 trillion in wages, submerged across administrative, financial and professional services. This indicates exposure to AI is in fact five-fold larger than what we see adopted already, and that doesn’t account for ongoing and future AI developments. The framework is validated by a skill-based occupation similarity predicting 85% of real career transitions, illustrating skills are a meaningful measure of human-AI work interaction. As the industrial era measured output per hour, and the internet era built the digital economy accounts, the intelligence era requires a new metric – one which measures skills. We are perhaps flying blind because our instruments measure the wrong thing.

In a similar vein, others have proposed that jobs are bundles of tasks⁸, criticising how labour markets buy jobs, not tasks, and asking whether AI can separate a task it performs well and cheaply from the larger bundle. When separation is cheap, the bundle is weak and the human role narrows. When separation is expensive, the bundle is strong and AI raises productivity inside the bundle without replacing it. To make the abstraction concrete: a paralegal, whose document review separates cheaply from the rest of the role, sits in a weak bundle and watches the role narrow; whereas a nurse, whose tasks are bound together by physical presence, unpredictable demand and strict liability, sits in a strong bundle and watches AI raise her productivity without replacing her. The common patterns that make a bundle strong follow from the example: demand is unpredictable, so the cost of switching between human and AI is too high; production spillovers mean doing one task makes one better at the next; and liability requirements are strict, so human-AI joint outputs must be attributable to a human.

The more AI commodifies what used to be considered ‘non-routine’ cognitive tasks, like crunching numbers and generating text, the more the human value proposition will necessarily shift toward capabilities that no machine can (yet) reliably replicate: applying judgment in messy, evolving contexts; connecting disparate ideas into a coherent plan; and persuading, negotiating and leading when the ground is shifting underfoot.

Everyone is exposed to AI

That doesn’t mean we will all adopt AI in our jobs or even more so, be at risk of displacement. Painting a richer picture is nonetheless helpful. For the past few years, we have feared AI disruption will affect first and foremost entry level jobs, as those require most effort in training and mentorship and this slice of the economy can be first slashed as it isn’t even embedded yet in the workforce. In 2026, we see this unfolding in real time: in the UK, an independent report warned of a lost generation, with 1 in 8 young people aged between 16 and 24 not in education, employment or training⁹, with task and job automation cited as a contributing factor. In alleviating AI displacement, we are already seeing initiatives sprouting up to support entry-level workers navigating the transition to an AI-powered workforce, such as the New Work Foundation non-profit¹⁰, funded by a former Meta executive astounded at the AI agent capabilities she observed in real time.

Middle management jobs are not protected from the advent of AI either, with 1 in 5 organisations predicted to cut half of their middle management in 2026¹¹, in an effort to cut down labour costs, streamline reporting and business processes, and ultimately help flatten out the workforce hierarchy. For the individual manager, the flattening means the rung they were climbing towards may simply not exist by the time they reach for it. Senior leadership is challenged too in the AI era¹², as experience alone and authority are not sufficient to efficiently lead companies towards becoming AI mature, and this group needs to embody other values too, such as empathy, critical thinking and ethical decision-making. When working alongside intelligent systems and autonomous AI, traditional hierarchies prove both redundant and obstructive, with adaptability and proportional risk-taking superseding principles rooted in domain expertise and upwards career progression.

When looking at the type of jobs affected, recent reports indicate both white and blue collar jobs are heavily affected, with IMF forecasts indicating AI is or will impact 40% of all jobs worldwide¹³. If real-world environments were a challenge for these disembodied AI systems, claimed by some as a key limitation of LLMs, we are now witnessing a shift towards embodied, perceptive and physical AI systems, as exemplified clearly by Yann LeCun’s Advanced Machine Intelligence Labs, which raised over $1 billion in a matter of months¹⁴. This is to say most sectors and job types are or will suffer changes, even those which might seem protected, such as care and nursing.

Anecdotally, frontier AI labs have recently been recruiting academic philosophers, leveraging on their deep enquiry methods¹⁵ – humanities backgrounds may bring growing value to an AI-fuelled economy: quantifiable skills are first to automate, leaving critical inquiry and risk-based assessment amongst the most valuable traits we can bring forth in our jobs. The same force points the other way in the sciences, where AI is compressing timelines for discovery and validation from decades to months¹⁶, holding promise to accelerate societal progress on the biggest challenges, such as disease and longevity. Commodified skill and accelerated discovery are two faces of the same transition.

A different Tuesday

The year is 2046. You wake up at the sound of notifications and powering engines, the AI system has allocated you a series of instructions for the day, carrying tasks you didn’t write to people who didn’t ask. Overnight the system has decided; your work is to give its decisions a human face, as a voice still lands softer than a screen. Your performance review measures compliance latency – how fast an order travels from the model to the floor. A colleague paused last quarter to question a number, and was quietly reassigned; the rest of us learned not to. The office, when you’re called in, exists for the appearance of consent – meetings where the conclusion arrives before the conversation. Nobody asks what you think anymore; they ask whether you passed it on in time.

Indeed, this is the dystopian end of the spectrum where we could be experiencing a fast and fundamentally transformative shake-up of the economy and society where humans execute and machines instruct. The AI transformation might not only challenge our sense of purpose and meaning as individuals and civilians alike, but it may leave us questioning whether we are the most superior intelligent entities in an evolving landscape where artificial supersedes biological. Bringing together minds and behaviours in a collective intelligence super-engine which powers all employment sectors might leave humans in an uncomfortable yet comforting position – we cannot hold any more subject matter expertise, but we can learn how to best engage with the same knowledge available to us all. Negotiation skills with human stakeholders transform into humans managing conflict resolution between AI agents.

Revert back from 2046 to 2036

We cannot pause or stop AI outright; too much capital is staked on its continuation, and as a general-purpose technology it is penetrating nearly every job sector and human task as illustrated above. There are, however, brakes on the machine – the EU AI Act, compute governance, liability regimes – and the honest claim is not that they can halt the transition, but that they can shape its speed and distribution. We can slow adoption where harm outpaces understanding, cut down the hype, and tone down the overpromise of solving all societal grand challenges or surpassing humans in all cognitive work tasks. We might move from economic productivity and the mechanical optimisation of a corporation’s share price to evaluating how work impacts wellbeing, keeping a tight balance between task automation and human augmentation. We could build together a vision more grounded in the messy human reality we live in, and put the human at the centre of this revolutionary transition, so that we maintain our anthropocentric status quo and reclaim our humanity.

I lay below five commitments, for the transition from a dystopian to a human-centered, machine-aided future:

1. We will become AI literate at the speed of humans, not the speed of frontier model releases – accounting for the asymmetry between how fast models ship and how fast people learn, and for the incentives at play when AI is embedded within all strata of society.

2. We will demand AI is designed with a human-centred focus – from user experience to expressing uncertainty and exercising metacognition – that encourages critical judgement and human skill improvement over cognitive decay.

3. We will hold providers and deployers, policy makers and organisation leaders to adaptability and empathy, so that human jobs are retained, transformed or created whilst reaping the imminent benefits of intelligent systems.

4. In a world at risk of sycophancy and convergence of thinking, we will value friction and diversity in all their shapes and formats.

5. And whilst the power does not lie with the individual, collectively we can steer the rhetoric – meeting the job market transformation with the information and tools to treat it as what it is: both a threat and an opportunity, in proportions that depend on what we do.

An open invitation

Today, many of us may feel like subjects of AI displacement; tomorrow, we could be authors and co-creators empowered with the knowledge and competencies to engage critically and constructively in a discussion which involves us all – when work as we know it transforms fundamentally, who do we become, and what is our role in the becoming process? Scientists, economists and policy makers build forecasts; humanists and writers imagine scenarios, fictionalise them and turn metrics into maps. When they become believable, we can choose to inhabit that reality, and – one action and reflection at a time – we build intricate causation chains. AI displacement is a matter of both now and the future, and it is both a concern and an opportunity.

If you believe this is a worthwhile discussion, please join us in Berlin on 2nd of July to co-create fiction scenarios and nourish hope.

◆

This essay appears in The PhiAI Anthology, Year One — eighteen essays, twelve writers, printed and illustrated.

See and buy the Anthology

References

1. Anthropic CEO says AI could wipe out half of all entry-level white-collar jobs — Business Insider, 2025.

2. Sam Altman and Dario Amodei are both walking back their AI jobs apocalypse prophecies as they eye blockbuster IPOs — Fortune, 2026.

3. Anthropic Economic Index report: Learning curves — Anthropic, 2026.

4. AI Adoption Research — UK Department for Science, Innovation and Technology (gov.uk), 2026.

5. Monitoring AI Adoption in the U.S. Economy — Federal Reserve, FEDS Notes, 2026.

6. Use of artificial intelligence in enterprises — Eurostat, Statistics Explained, 2025.

7. The Iceberg Index (Project Iceberg report) — MIT (iceberg.mit.edu), 2025.

8. The task is not the job — Silicon Continent (Substack), 2026.

9. Young People and Work: Interim Report — UK Government (gov.uk), 2026.

10. Meta / Salesforce executive launches non-profit to help Gen Z navigate AI agents and jobs — Fortune, 2026.

11. Why Companies Cutting Middle Managers To Fund AI Is A Mistake — Forbes, 2026.

12. Why AI Demands a New Breed of Leaders — MIT Sloan Management Review, 2025.

13. AI Job Replacement Statistics — DemandSage, 2026.

14. Yann LeCun Got $1 Billion For World Model AI: These Robots Learned 1,000 Real-World Tasks In 24 Hours — Forbes, 2026.

15. Meet the One Woman Anthropic Trusts to Teach AI Morals — The Wall Street Journal (WSJ Magazine), 2026.

16. Accelerating scientific discovery with AI Co-Scientist — Nature, 2026

“There are some who imagine oblivion as an empty cellar/a harvest of nothing.
Yet, oblivion is full of memory.”

— Mario Benedetti

Imagine a record of your life you can scroll.

An endless, searchable account of every glance, conversation, and hesitation. Preserved without editing and unfiltered. An uninterrupted record of life in the form of a scroll, retrievable on command.

We are beginning to produce something very close to this. A new kind of past that exists independently of human remembrance and with no space for forgetting.

I call this the unlived past.

Unlike the deliberate past of portraits, diaries, and archives, this unlived past is neither selective nor narrated. It becomes searchable, and permanently retrievable through AI chatbots.

The human past, the one that is remembered, the one that made into stories and History books didn’t survive time by default. It did through selection, narration, and forgetting of everything else. Oblivion is the necessary condition for human memory.

The unlived past changes that.

Share

The deliberate past

For most of human history, we have had a deliberate past. What survived time was an act of selection: portraits, letters, monuments and stories that people or institutions chose to keep. The vast mass of our days that made the rest, was left to oblivion. The past as a discrete collection of acts.

Two features distinguished this deliberate past.

First, what got to be preserved was chosen. Artifacts served as filters that prized particular moments, poses, or interpretations.

The ultimate example of curation of what is to be remembered are royal portraits across history. Up until the advent of photography, it was impossible to judge how far or close from reality they were. They certainly conveyed a staged representation of the person portrayed.

Let’s take the example of Queen Victoria of England. Below we have her, portrayed by the German Franz Xaver Winterhalte. A massive painting measuring 241.9 x 157.5 cm finished in 1859 that depicts her in her coronation robes. She looks powerful, majestic and utterly rich with the Windsor castle in the background and her furry robes.

We don’t need to agree that this is the way she looked. Yet photographs taken only a year later reveal something strikingly different. One of them below:

It is unmistakably the same person. But the effect changes completely. She looks tiny, almost fragile. I would even argue, less royal, more common.

My point is not that she tried to style and stage herself. We, too, are tempted by the myriad of filters we have available in our phones and photo-editing software.

The point is that everything else except those artifacts used to be gone and forgotten. The point is that if we didn’t have the photograph, we would probably think differently of her, more in the way she wanted us to think of her.

Second, in the deliberate past, the timing and audience of disclosure were controlled: memoirs were released when helpful, portraits were shown in public or held in private with intention. These features gave individuals and institutions a degree of agency over how lives and events would be remembered.

Few fictional objects illustrate this more elegantly than the Pensieve from the Harry Potter series. The Pensieve is an artifact that looks like the baptism font in a church, yet memories instead of holy water swim in it. A pool of memories, if you will. Putting your head in it immerses you back to that moment like in a VR game. It both gathers and stores memories while allowing to re-live them.

How do memories get into the Pensieve, you may ask?

By being shared in the form of tears passed from the memory holder to the memory harvester in a tiny flask. In this magical world, it is impossible to steal a memory. The person who lived it has to share it deliberately.

The boundary of consent, held also in this fictional world, is one that our modern tech stack increasingly erodes.

Subscribe now

Oblivion is full of memory

Forgetting was inevitable due to the limitation of our minds. Human memory is fragile, selective and interpretative. We misremember, distort, omit and erase. Historically, most competing evidence disappeared. Archives shaped perception precisely because they were scarce.

Yet forgetting allowed people to outgrow their former selves, to repair reputations and to keep intimacies private. This granted a certain degree of influence around the narratives about ourselves. On a more collective level, it allowed communities, and even nations to move forward without carrying the full, uninterrupted weight of every past action because entire periods of time collapsed into a handful of surviving impressions.

Oblivion is full of memory.

Oblivion as the ultimate filter.

The unlived past

The modern digital stack is transforming that arrangement. Messages, locations, searches, voice notes, photos, browsing histories, biometric traces; they all record our lives indiscriminately, continuously and automatically.

They amalgamate bits and piece of millions of lives into vast datasets, stripped of their original contexts and indexed as information. What emerges is a past that offers itself to be queried, recombined, and generated on demand. Not remembered, but retrieved.

This is the unlived past.

An archive of lives that persists independently of lived recollection: unlived, yet permanently retrievable. A past assembled from traces.

Retrieval vs remembrance

This past doesn’t remember in the human sense.

Human memory is embodied and narrative. We remember through emotion, distortion, context. Think of the verb to recall in English. Even at this root, it means re-gather moments, re-gather sensations. It is a very active word. Because when we remember we attribute meaning. This is why human memory is as individual as varied. Two people can be at the same event and carry away two entirely different versions, two different pasts. Human memory is “lossy” by design, prioritizing the meaning of an event over the raw facts.

In contrast, the unlived past offers a form of persistence in lieu of memory. It is a database memory, flat and indifferent. We query it and the past we get is rendered in front of us with no effort on our part. Unlived and permanent.

Where the deliberate past is a collander, the unlived past is a sponge.

The Grace of Forgetting

Forgetting a.k.a the human ability to let some things lapse into oblivion has been a form of moral and social agency. It let people manage reputations, recover from mistakes, and keep private the intimacies that give life dignity. It gave us the possibility to redefine ourselves anew.

The unlived past ends that grace. It comes hand in hand with a world where the past no longer fades but awaits. Indifferent to our growth or our regrets, ready to be rendered at the touch of a prompt.

The colander of human memory, replaced by the sponge of a database takes away our ability to curate our own identities. We become prisoners of our own traces, unable to escape the person we were ten years, ten minutes, or ten thousand data points ago.

If Mario Benedetti were alive today, he would see that the “empty cellar” he once dismissed has been filled with the cold, unblinking glare of a ledger. He would have to rewrite the poem.

Oblivion is full of emptiness.

P.S.: We're making the first Phi/AI Anthology, a printed collection of the essays that stayed with us most from our first year. It goes to print if we reach 30 pre-orders. Help us make this book exist.

We were in stealth mode and very secretive about our work. Finally, the day came for our first demo to someone outside our team. This was our big debut to an audience of one…

I was excited and ready to receive praise and accolades. What unfolded was the complete opposite and caught me by surprise.

The woman on the other side of the call was visibly upset. When she spoke, it was emotional but clear: “This product has the potential for serious harm. This can trigger eating disorders.”

I didn’t know it then but now I understand that if we want AI that minimizes harm to vulnerable populations, it cannot be built in a race. When you encounter a signal of harm (like I did), you need the time to fix it. It is our greatest responsibility as builders of technology.

After the catastrophic demo, I went on a journey that changed the course of my career and identity. Somewhere along the way, I became dedicated to working at the intersection of AI and nutrition. Thanks to the deep understanding around dieting that I gained during these years I came to a realization: AI- assisted dieting is now a population level health risk for 100M+ Americans.

AI didn’t invent dieting, but it is deployed within and trained on systems that profit from fear, shame and harm around food. AI is not aware. It doesn’t realize the problems it perpetuates. Only humans can realize and change this. This is the story of a team of humans (dietitians, engineers, behavioral scientists and entrepreneurs) that saw an emerging problem and attempted to do something about it.

Diet Culture

If diet culture had one slogan it would be: “Calories In, Calories Out”. Dieting is the deliberate attempt to restrict food intake to lose weight. The CDC1 says an estimated 120M Americans go on a diet every year. In a 2007 review, UCLA researchers reported that about one-third to two-thirds of dieters regained more weight than they lost, and they argued this likely understates the problem2.

Diet culture goes beyond the prevalence of trendy diets, the encouragement of food restriction, or critical body talk. “Diet culture is a system of beliefs that worships thinness and equates it to health and moral virtue. It promotes weight loss as a means to attaining higher status. (...) [it] demonizes certain ways of eating while elevating others and oppresses people who don't match up with this picture of health.”3

It is a cultural phenomenon that attaches the worth of a person to the way she looks and this to the power of her will. Movies, ads, pop culture all feed us the idea that there is a right way to look.

The dieting industry has built a $90B per year model that entraps people in a vicious cycle of feeling unworthy, being sold a diet as the solution, being ravaged by cravings, losing control, and then into failure, weight regain and shame (to be sold another diet again later). This can go on for years or even a lifetime. It is a system that profits from the destruction of humans. Eating disorders are the second deadliest mental health condition, only trailing opioid addiction.

Dieting encourages people to think that in order to lose weight, they just need to eat less and move more. This view ignores the important role the metabolic rate plays in the amount of calories a body may burn. In the words of our cofounder and registered dietitian, Melissa Vasikauskas:

Harms of Dieting

Dieting can have a negative impact both on metabolic and psychological health.

With regards to the metabolic consequences of dieting (i.e the ability of your body to burn calories), there is an increasing body of research showing that when you lose weight, a mechanism called metabolic adaptation gets triggered, slowing your metabolism. The goal of this mechanism is to prevent you from starving. Throughout human history, famines were a real threat that our bodies built protections against, like slowing down our metabolic system. When you start a diet, your body doesn’t know you are trying to lose weight. No, it thinks you are entering famine times. Once the metabolism drops, it takes a while to come back and if another diet is started, it can drop even further.

So it isn’t will power but rather a complex system of metabolic effects that are working against the dieter. As Jen Carter, PhD Psychologist at OSU Wexner Medical Center puts it: “[With calorie restriction] you not only feel hungrier, but you’re less likely to feel full or satisfied by what you eat. It tends to increase the mind’s preoccupation with food and increases activity in the brain’s reward center when we consume high-calorie foods.”4

Vasikauskas confirms this:

As people weight cycle, they regain the weight and often more. Leaving them in a worse state, psychologically and metabolically, than when they started. A vicious cycle that can leave people feeling hopeless and out of control.

These findings exist in spite of the research on eating disorders being severely underfunded. While federal agencies in the USA fund cancer ($10-12B), heart disease ($2-4B) and diabetes ($1-2B) research in the billions per year, eating disorder research is estimated to be under $50M/year. Eating disorders are estimated to cause 10,000 deaths in the U.S. per year, 30M Americans are estimated to be diagnosed with an eating disorder in their lifetime and a staggering 22% of children and adolescents worldwide show disordered eating (NED).5

The psychological effects of dieting are equally well documented. Already in the 1990s researchers6 identified increased risks for dieters to develop eating disorders. Weight cycling (regardless of the result) was recognised as “medically and psychologically harmful” already in 19957.

As dieting apps became more widespread, the research shifted to understanding the impacts of food monitoring and weight monitoring apps on disordered eating and concluded that the use of the former increases the risk of developing the latter.8

Moreover —and this connects to the increased usage of AI in these dieting apps— research shows that when calorie restriction is successful, it’s often in a highly supported, clinical environment. But dieting apps are offering the general public exactly the opposite: the possibility to perform dieting without the close monitoring of a clinician. Scale is being prioritized over safety. Over 50% of U.S. adults use an app to track their nutrition regularly9. Today, most dieting apps include AI photo calorie-tracking capabilities and many are providing AI generated dieting advice.

Subscribe now

How AI worsens this

When ChatGPT arrived in 2023 restrictive dieting became one prompt away.

AI inherited diet culture’s biases. Trained on the public internet, it absorbed decades of stigmatizing views about fatness and now generate diet advice on demand.

Adoption was immediate. A 2026 NYTimes survey found that one-third of U.S. adults have used ChatGPT or another AI tool to create a nutrition or weight-loss plan. Among teens, 64% report using chatbots10 and a growing body of research shows they turn to AI for food and health decisions.

The gap between AI advice and clinical guidance is measurable. A March 2026 study in Frontiers in Nutrition found that AI meal plans averaged 695 calories per day below a registered dietitian’s plan. This is equivalent to the caloric content of an entire meal.

There are two common features, in my opinion, that make AI dieting a high risk activity.

First, the ease with which a meal plan can be created to suggest a very low caloric intake. I created a dangerously low calorie plan for myself using an LLM (see below).

I only received a warning when I requested for my goal weight to be less than 125 lbs. I think the last time I was 125 lbs. was around puberty.

Others have reported the same, though they don’t seem to understand the risks.

The second feature is AI photo-based calorie counting. When users count calories, this is most often utilized to support restriction and “calorie cutting.” It’s important to note that calorie-restriction works, in the short term. Most studies, if they are short enough, won’t capture the “yo-yo diet” effect that happens once the body enters metabolic adaptation - metabolism slows down, hunger hormones surge and users often report strong cravings and food fixation. Calorie restriction runs us right into the mechanism our bodies built to survive famine times.

Humans at the center

In case you wondered, the latter is exactly the feature that scared off the woman we presented our prototype years ago. Today, we don’t allow users to track calories with our tool. As Head of Product at Tune Health, I believe that creating feedback loops with calories does not support the user’s goals of better nutrition and overall health (including mental health). The risks far outweigh the benefits.

We chose this path after extensive research, building a team with lived experience, co-creating with affected communities and having ongoing (still today) deliberations on the risks. My take is that if you want to “do no harm” it requires an extra effort. In a fast paced AI industry, this extra effort and consideration isn’t part of the incentives package. If our company had taken on significant investment in our early days (we haven’t), these decisions would have been more difficult. What we are trying to achieve is a longer road. We shifted from our AI product at the center of our company, to using it as an extension of our nutrition protocol (which has clinical evidence of achieving diabetes remission, improved fertility, stabilized cholesterol, sustainable weight regulation and an improved relationship with food). I believe there is still an opportunity for companies to instill a different set of values at the consumer-facing layer.

I am the only remaining team member from the story at the beginning and this journey has changed me. I became aware of the world around me in ways that I had been ignorant. There is a near-infinite amount of nutrition information online. I’ve had to learn how to discern signals from noise, in systems plagued by broken incentives. One of the most important things I learned in this journey is how risky and ignorant it is to center AI as the final source of information and guidance for users toward their health goals. Our American culture has distorted what a reasonable and healthy goal is, so this needs to be addressed first.

Our approach is for humans to be at the center. We humans have experience, knowledge and wisdom. AI doesn’t. It is our responsibility to hold true to our values, using AI as an extension of this center for an aligned outcome for human health and of course, a sustainable business.

I’ve always found this an odd thing to worry about. I can think of dozens of things I’ve been dying to do but haven’t had the time for. The question “what will I do if I don’t work?” contains a hidden assumption so deeply embedded we barely notice it: that our existential justification is tied to our economic utility. That without a job, we are not merely unemployed but unjustified. Somewhere along the way, we fused the labour market with our sense of purpose. Now we can’t imagine one closing without the other collapsing too. Have the billions of stay-at-home parents throughout history been leading meaningless lives? This is, to state the obvious, absurd. I’ll come back to this.

I think that the panic around post-AGI meaning deserves more suspicion than it gets. It’s a distraction, and a convenient one for those who’d rather we agonise about purpose than organise around power. The obsession with “what will we do?” is a symptom of a system that benefits from our fixation on it. Dissolve the construct and you see what it was obscuring: who holds power in a world where machines produce most value? I’d like to take a moment to deconstruct the work ethic as the value that underpins the conflation; how it was built tells you who it was built for.

But first, I must acknowledge that the fear is real, and we owe it an honest reckoning before we can see through it.

I am not dismissing the anxiety but, instead, trying to understand what it’s actually about. The truck driver watching autonomous vehicles roll off the production line, the writer watching GPT draft passable prose: I wouldn’t call these people irrational. They are experiencing something like anticipatory grief, mourning a version of themselves that hasn’t died yet but can see the headlights.

On the issue of boredom, Viktor Frankl had a term for the dread that arrives when unstructured time looms: “Sunday neurosis.” A low-grade panic at the absence of external demands, which replaces the pleasant anticipation of a free afternoon.2

On unemployment, Marie Jahoda’s work showed that it devastates people even when they can still comfortably pay their bills. Her insight as a sociologist, drawn partly from a landmark study of an Austrian town in the 1930s, was that work does far more than provide income.3 It imposes structure, makes us stick to a schedule. It forces us into proximity with other people whether we like them or not. It connects us to a purpose larger than ourselves. And it defines us socially; “What do you do?” is the first question we ask upon meeting someone for the first time, because occupation is how we locate each other in social space. To answer “nothing” is to become illegible. Notice what’s absent from this list: any mention of the work itself being satisfying. A person can hate their job and still depend on it for structure and identity and a reason to leave the house.

None of this is speculative. Anne Case and Angus Deaton documented an epidemic of what they called “deaths of despair” in deindustrialised American communities: suicides, opioid overdoses, liver disease.4 The crucial detail: many of these people still had income through disability payments or pensions. What they lost was the architecture of daily life. People do not respond to the loss of work by relaxing. They tend to self-destruct instead.

All of this I accept. But none of it explains why employment became the sole provider of these psychological necessities, or why we struggle to imagine alternatives. That is a different question, and it has a historical answer.

How we built the cage

It wasn’t always this way. For most of human history, work was a curse. The Latin labor shares its root with suffering. Genesis frames work as divine punishment. Aristotle considered manual labour fit for slaves, precisely so that citizens could be freed for higher pursuits. The idea that work is virtuous, that it builds character, that idleness corrodes the soul is oddly recent.

And it was, I contend, invented to solve a specific problem: control. Early industrial capitalism needed consistent output. You can watch a man dig a ditch; you can count the bricks a worker lays. But as economies shifted toward cognitive and clerical work, effort became invisible. Economists call this the principal-agent problem: when you can’t directly observe whether your employee did two hours of good thinking or stared at a screen, you need workers to police themselves.

The solution was to make work a moral matter. Idleness becomes sinful. Your labour becomes evidence of your character. Weber identified the moment this happened, tracing the fusion of Protestant theology and capitalist discipline into a single ethic that made hard work a sign of spiritual election.5 Foucault saw the deeper mechanics. His central image was the Panopticon: a prison designed so that inmates can never tell whether they are being watched, and so learn to behave as if they always are. The most efficient power is power that no longer needs to be exercised, because subjects have absorbed its demands as their own standards.6 Tie someone’s moral worth to their output and they will never slack off, even when no one is watching. The work ethic is capitalism’s most elegant solution to the supervision problem: convince people that their souls are at stake, and you never need to hire another foreman.

But Foucault only gets us partway. We don’t merely police ourselves to avoid shame. We identify with the norms that bind us. To become a subject, in Judith Butler’s formulation, requires being subjected: we are formed as persons through the very structures that constrain us, and so we cling to those structures because loosening them feels like losing ourselves.7 This is why the prospect of a world without work triggers something closer to an identity crisis than a scheduling problem. We have fused who we are with what we produce. The guilt we feel on an idle Tuesday, the need to justify a long lunch, the way we introduce ourselves even among friends by naming what we do for money: these are not personal quirks. They are the marks of an ideology that has become highly intimate. We don’t just comply with the work ethic; We want to comply. As Butler called it, we start “desiring our own subjection”. We fear the freedom that would follow its removal, because without our chains we become illegible to ourselves. We are prisoners who have learned to love the cage.

“Do what you love and you’ll never work a day in your life” sounds like liberation. It is the opposite. It is the final move, the moment the prison no longer needs walls because the prisoner has fallen in love with the cell. This is the Stockholm Syndrome of labour.

And the ideology runs deep enough that history’s greatest monsters could exploit it. “Arbeit Macht Frei,” “Work Sets You Free,” was inscribed above the gates of Auschwitz. The historian Otto Friedrich wrote that the slogan was meant as a kind of mystical declaration that self-sacrifice through endless labour brings spiritual freedom.8 Of course, I am not comparing modern workers to victims of a death camp. I am comparing the belief systems. The Nazis did not invent the idea that work liberates. They recognised a conviction that already ran through the culture and they weaponised it, with horrible clarity, because we had already accepted that labour is the path to dignity and freedom. The gentler modern version of this faith (e.g., hustle culture or the guilt we feel for taking a sick day) shares a structure with the slogan above the gate: your worth is earned through production, and work itself sets you free.

The cracks are already forming. “Quiet quitting” has been framed as a moral failure: GenZ is lazy, entitled, ungrateful. A more honest reading is that the wage-effort bargain is broken. Productivity has outpaced wage growth for decades. Asset inflation has put homeownership and family formation out of reach for millions. Working hard no longer delivers the traditional rewards, and so a generation is recalibrating effort to match the degraded return. The contract was breached by the side that wrote it.

The work ethic is infrastructure. It was built in a particular era, to serve particular interests, and it can become obsolete.

Subscribe now

The gendered blind spot

There is something else hiding in the panic about the “useless class.” Something I see as the collapse of the breadwinner archetype. For a few generations in industrialised societies, male identity became tightly bound to paid employment. You were a man because you earned. When we fret about masses of humans rendered purposeless by automation, we are often unconsciously imagining men losing jobs. The “crisis of meaning” has a gender, and we should name it.

Return to the stay-at-home parent I mentioned earlier. Map that life against the very framework Jahoda gave us. A toddler imposes a ruthless schedule; anyone who has cared for one knows that “unstructured time” is a fantasy. The social world of a primary caregiver is dense and demanding: playdates, school runs, community organising, the constant negotiation of other parents’ needs and expectations. You are embedded in webs of mutual dependence, connected to purposes (the family, the neighbourhood, the school) that are plainly larger than yourself. The identity is rich: mother, father, caretaker, organiser. And enforced activity is not exactly in short supply. All of Jahoda’s psychological infrastructure is present. What is missing is the wage, and the social recognition that flows from it. The “crisis of meaning” turns out to be a crisis of legibility. It is about the loss of a particular kind of socially validated status that we have mistakenly treated as the whole of meaning.

The obvious objection is that stay-at-home parents derive much of their social standing from the wage economy around them. They are “a doctor’s wife,” “a stay-at-home dad who used to be in finance.” Their legibility is borrowed. In a world where nobody works, that borrowed status disappears too. This is a real objection. But even if social recognition is currently borrowed from the wage economy, that is a problem with our recognition system, not with the meaning itself. A mother’s bond with her child does not become less real because society fails to validate it. What it lacks is not meaning but social recognition.9

For those whose identity is bound to production rather than care, this transition feels like annihilation. But this should not be a prophecy of doom. I think that the failure is revealing: it tells us more about the narrowness of our current meaning-architecture than about the emptiness of the future.

Meaning is renewable

So if meaning doesn’t depend on a paycheck, will we actually be fine?

I think the evidence suggests we will.

AI crushed Kasparov in 1997. Pundits predicted the death of chess. Instead, chess boomed: over 100 million users on Chess.com as of last count. We didn’t stop playing because a machine plays better. We watch humans play because we care about human struggle, not optimal computation. We play sports we will never master. We run, though we no longer need to chase anything. We paint, sing, garden, build things nobody asked for. If anything, turning these activities into a profession tends to drain the very sense of purpose they provide.

Even in material abundance, humans will compete for skill, beauty, wit, reputation, attention. Remove the economic game and the status-seeking doesn’t vanish; it redirects. Hierarchy-building is an evolutionary adaptation, not a byproduct of scarcity.11

AI does not deplete our capacity for meaning. Humans generate purpose the way we generate language: compulsively, endlessly, even in captivity.

Subscribe now

The real danger: from meaning to power

By now, I have made my position clear: the problem of meaning is not worth losing sleep over. That is not to say, however, that I would feel comfortable sleepwalking into the post-AGI world. Allow me to conclude this essay by pointing at what does worry me.

The social contract has always rested on mutual dependence. Elites needed labour: our muscles, our minds, our compliance. They needed us to buy things. In return, they had to negotiate. The worker had leverage because the factory couldn’t run without them; the citizen had leverage because the state couldn’t function without their taxes and their cooperation. Every major right we have won was won because the powerful needed something from the powerless. The suffragettes could disrupt, the unions could strike, and the tax base could threaten to shrink. Leverage required dependence, and dependence ran in both directions.

Post-AGI, that mutual dependence dissolves. If machines produce most of the value, the masses lose their bargaining position. We shift from citizens who must be negotiated with to dependents who are managed. The elites may still choose to provide for us. But charity is not a right, and benevolence is not a contract. The danger is not that we become “useless”; that is a feeling. The danger is that we become harmless, and that is a political condition.12

The threat will not arrive dramatically. AI safety researcher Paul Christiano has described what he calls the “whimper”: a gradual, almost invisible transfer of decision-making authority from humans to optimisation systems, without our hand being forced, simply because the systems are better at deciding.13. Tax policy, healthcare allocation, hiring decisions, criminal sentencing. The AI optimises the metric, and the metric improves. Each individual handover seems rational, even welcome. But cumulatively, we lose something harder to name: the capacity to question what we are optimising for. We don’t lose freedom in a dramatic seizure. We trade it away, one convenience at a time.

And this is not hypothetical. Recommendation algorithms already choose our music, our news, increasingly our social connections. We outsource memory to search engines and judgment to AI assistants. The post-AGI world does not invent this dynamic. It accelerates it until the accumulated surrenders become irreversible.

The reflexive policy response, “tax the robots, redistribute the proceeds,” faces obstacles that deserve their own essay. The redistribution mechanisms we have built assume an economy powered by human labour and taxed at the point of employment. A post-labour economy may require entirely new architectures of governance: not merely new tax codes but new conceptions of citizenship, ownership, and political leverage. Universal basic income without political power is hush money for the harmless. It maintains consumption but extinguishes agency.14

The real crisis is one of power, and it is arriving before we have built the institutions to address it.

The door is swinging open. We are like long-term prisoners: the walls are familiar, the routine is known, the constraints have become comfort. This is not a moment of pure joy. It is a moment of vertigo. Freedom, after sufficient captivity, feels like falling. The crisis is not that there is nothing outside the cell. It is that we have forgotten how to walk without chains.

Meaning will take care of itself. Humans are inexhaustible; we will always find new games to play. But who will be left holding the steering wheel while we play?

◆

This essay appears in The PhiAI Anthology, Year One — eighteen essays, twelve writers, printed and illustrated.

See and buy the Anthology I

The high-tech revolution seems to be moving at an ever quickening pace. But is our culture keeping up? As Sharma states, the issue is not exclusive to AI. We live in a system of incentives that reward actions often at the antithesis of human values. Market incentives value competition, growth, and maximizing financial returns, while human values encompass empathy, responsibility, and a holistic view of interrelated systems. When we allow such incentives to direct our behavior, rather than our wisdom, things go very, very wrong.

Minding the Gap

The AI revolution—the latest tech paradigm shift—is well underway. Though rapid adoption is in full swing, many open questions remain within culture.

This is what I call the Quicksand Phase, the period of cultural liquefaction when everything is messy and the collective agreement in culture has not yet been made. It is the period in which we are most vulnerable—and most empowered—our actions having far-reaching consequences in a ‘never-been-done-before’ world.

The Luddites understood this. It was not for fear of technology that the Luddites burned down a new sewing factory in 1812. It was the wholesale destruction of hundreds of livelihoods in favor of cost savings (aka. money incentive). “This dynamic of predatory managers using technology to destabilize the lives of workers or eliminate their jobs entirely is hardly just a nineteenth-century phenomenon,” says Greg Epstein in his book Tech Agnostic. “It is still how tech operates today, and it has been a foundational aspect of capitalism since the original Luddites... Luddism is not really about the rejection of technology at all—it’s about the rejection of a certain kind of political and economic deployment of tech.”

As Sharma said, when money is the goal, humanity is thrown out the window. “Nowhere do we see capitalism froth at the mouth more than in the VC room. Nobody is asked what their values are,” continues Epstein. As a startup founder, I’ve watched firsthand as the slimiest men shuffle millions between companies, funding the most outlandish ideas without first pausing to wonder ‘should we?’ The money incentive is the beginning and end of thought.

Data shifted the incentives in the early 2000s when Google rewrote its value proposition from search results to data commodification, increasing their revenue that year by 3,500%. Shoshana Zuboff has been tracking the shift ever since, ringing the bells as the incentives slide from a product-driven to data-driven economy. “Surveillance capitalism unilaterally claims human experience as free raw material for translation into behavioral data,” says Zuboff in her book The Age of Surveillance Capitalism.

And AI in particular is hungry for data. Starving, in fact. “The internet is a vast ocean of human knowledge, but it isn’t infinite,” an article in Nature stated. “Artificial intelligence researchers have nearly sucked it dry.” The danger is manifold: AI is not yet profitable and yet needs more data, so it must force its way into our lives any way it can — money, again, at the incentives wheel. “This entanglement of overwhelming powers for which no American has ever directly cast a vote is so dangerous because it represents an era in which surveillance is an essentially ungovernable social norm, a kind of modern force of nature to which we can only submit,” adds Epstein in his book.

Subscribe now

When Decisions Are Made For Us

During the Quicksand Phase of a paradigm shift, boundaries are dissolved. The new will try to get away with whatever it can, often engaging a scope creep that blows past what is healthy for a society due to the incentives to do so.

We already see it in everyday life. AI is being added to doctor’s visits, job application review, and even government surveillance, long before AI has been designed to account for its biases. Compound the issue with synthetic data—the solution to the data wall AI has hit. Only when we step in (usually in some sort of public outcry) do boundaries become firm. As Sharma said in his resignation, we must become adept at doing so much earlier in the Quicksand Phase of any paradigm shift.

AI usage in private business is small fish compared to the AI-powered surveillance programs governments worldwide now employ. We’ve all heard it: “I have nothing to hide.” That phrase is volleyed back and forth across social media and dinner tables whenever concerns are raised about data collection, now occurring via your daily interactions with devices such as social media, your TV, and even your car. This logical fallacy likely stemmed from the tech industry itself, where your data is more valuable than any dollar. A dollar can only be spent once; data can be bought and sold over and over, ad nauseam. It’s too valuable to allow public concerns to balloon.

“It is obscene to suppose that this harm can be reduced to the obvious fact that users receive no fee for the raw material they supply,” says Zuboff. “That critique is a feat of misdirection that would use a pricing mechanism to institutionalize and therefore legitimate the extraction of human behavior for manufacturing and sale. It ignores the key point that the essence of the exploitation here is the rendering of our lives as behavioral data for the sake of others’ improved control of us.”

Be Better than the Romans

Today, the cutting edge of tech includes AI surveillance and influencer bots built especially for you based on your data because you had ‘nothing to hide.’ Humanity is smart enough to avoid such brush-off logic, but we are not yet exercising it as the norm.

We claim to be at the height of human development, but really we are at the height of technological development—high-tech but not high-culture. Human development is still very much stuck in the Roman Era: we continue to use the same kind of democracy perfected during its reign and define civilization by access to the latest technology. But our technology has grown far beyond Roman tech; so must our society.

If we are going to evolve our culture beyond Roman achievement and adopt the wisdom Sharma invokes, we must learn to accept all our humanity. And we must evolve these cracks in our culture if we are to survive. Yes, survive, for technology comes not only with loneliness, but with unfathomable destruction by weapons yet untested due to their egregious might.

We are the only proven being on this planet capable of metacognition. We can think about our thoughts. Cultural evolution isn’t isolated to private citizens: it affects those who would become corporate and political leaders. Political leaders who today fight like children, as if words don’t work, must end. “You want to get physical with me? Like an ape?” cries the main character in Marty Supreme. He’s exactly right. We need a culture that normalizes catching leaders at playground thinking and admonishes them for it.

Evolving Culture

Incentives are a tricky thing—they are of and by culture. That is, our held beliefs, institutions, and behaviors all feed into shaping incentives at scale. There are negative incentives (go to jail if you steal) and positive incentives (earn a living if you give your time, labor, and expertise). Culture shapes these, and these have changed over time. They go hand-in-hand, a clasp ever-enduring.

And culture is shaped by—you guessed it—people. Culture is the result of hundreds of interactions, all seemingly banal—until they aren’t.

Culture can be changed: just as during the civil rights movement (or any major movement), a paradigm shift is possible when enough people assert their agency over a system of incentives. That is what I propose we do now, while AI is in the Quicksand Phase.

You know that saying ‘if you didn’t vote, then you don’t get to complain about the government’? Culture in the Quicksand Phase is kind of like that. Culture is going to solidify whether or not you take action; isn’t it better for you to get yourself involved so you don’t end up living in a society that you hate? The biggest lie we’ve been told is that our actions have no consequences. But if you leave ChatGPT because they signed a U.S. military contract, that’s a vote. If you’re in charge of corporate AI usage and implement a privacy-first, ethical AI, that’s an even bigger vote. If you don’t let your kids use chat bots, that’s a vote.

This is the culture that we have to work towards: a culture that actively participates in every aspect of its evolution. As of now, the incentives Sharma names push individuals to remain passive and weigh heavily in the favor of large corporate entities and governments with military might. That means that new innovations favor those two groups more than any other, and will be driven by and for those groups.

But we all participate in culture, because culture is developed by and of the masses. A culture that involves itself in its own innovation, rather than passively accepting the innovation handed to us is more likely to be equitable, diverse, and sustainable.

If you are okay with nameless government officials and nefarious CEOs making decisions about your life, then by all means, take no action. But with culture in a liquid state around AI, every act you take is a vote one way or the other, and companies are waiting to see what you will do and how far you will let it go. Set your boundaries. Take action. Submit your vote.

Subscribe now

Action in the Quicksand

If you have the ability to read this, and have the time to read this, you have the privilege and latitude to exercise agency in a paradigm shift. These early days of development is the most important time to use it.

“Each of us must decide how much we can afford to participate in an endeavor that oppresses and divides at least as much as it uplifts and heals,” says Epstein. The people must determine the direction of AI, not the corporations and governments who stand to benefit from it financially. We each must insert ourselves into the conversation—because the creators and funders of AI would rather we didn’t. Here are some ways you can vote for a new culture, now:

1. Start watching how corporate and political powers interact. OpenAI and other mega tech powers lobby for a clean slate of operation, and attack their employees when they whistleblow. “Two men at Google who do not enjoy the legitimacy of the vote, democratic oversight, or the demands of shareholder governance exercise control over the organization and presentation of the world’s information,” points out Zuboff. Their activities point to the vulnerabilities they intend to exploit, next. This awareness provides you the opportunity to decide what matters before they do for you.

2. Build the habit of checking tech’s claims. “What if the billions of people who live in or near destitution and poverty do so not in spite of the efforts of hyperconnected, superinformed experts... but because of them?” questions Epstein. “If you can truly predict the future skillfully enough to imagine jobs for everyone, you can also plot out—perhaps even subconsciously—how to hold onto and consolidate your own preexisting power, privilege, and dominance in that future.” There is no way to check the work of a tech visionary—they have a future in mind, but what they will tell you about is the future you can agree to. The work they do, and the vision they hold, may conflict directly with the ideal they tout. A healthy skepticism keeps you immune to their advertising and manipulation so you can act freely.

3. Push for regulation and protections while we’re in the Quicksand Phase. For example, we desperately need privacy protections that cover this new era of tech. We need our biometrics protected like our passwords so government officials can’t force the opening of a phone via faceID; our personal visage should be copyrighted so strangers can’t make deepfake videos about ourselves or our public figures; and a DEI review of every AI that is implemented at all levels of commercial use so any public implementation—from HR to medicine—perpetuates inclusion rather than discrimination. Make the case to your human representatives before corporate lobbies speak for you.

4. Take a stand at your workplace. The majority of people are still uncertain about AI’s adoption, and much of AI is not ready for use by groups consisting of more than white men. That means your boss is likely not quite sure which technology to choose and how far to take it. You can shape its rollout, if only you try.

5. What’s more, take a stand in your home. In many ways, we are living on fumes, surviving on the last vestiges of ethics and values left from the last analogue generation(s). Upcoming generations won’t have the privilege of those experiences. What then? We haven’t prepared them for a fully tech-driven world; our mores, values, and etiquette have not evolved as technology has. It is up to each parent and mentor to pass on the tools for coping with human existence, its every high and low, so technology does not become inexorable.

Sharma’s resignation letter should alarm you. Even more, his call to action must not be ignored: it is time we evolve our culture to meet the might that is our technology and use metacognition to think critically about ourselves and our future. That requires action from all of us, for each of us participates in culture. The consequences if we don’t are too awful to fathom.

◆

This essay appears in The PhiAI Anthology, Year One — eighteen essays, twelve writers, printed and illustrated.

See and buy the Anthology I

After the viral launch of OpenClaw, similar things happened1. OpenClaw’s innovation is to fold agentic AI into the same messaging apps you use with friends and family —WhatsApp, Signal, Messages— so that a natural‑language instruction can now travel from chat to calendar, email, banking APIs, and all the apps the user provides access to. The result is dazzling convenience: tell an agent to “handle it,” and it handles it. The result is also terrifyingly fragile: we are handing over access to sensitive systems in a way that doesn’t honour that these are critical infrastructure.

My thesis is that OpenClaw is not necessarily unsafe. But due to ignorance, negligence or naiveté we are ignoring the principles, frameworks and rules that Information Security, a discipline that is four decades old, offers and that, if followed, would make the deployment of these agents safer. We are deploying agentic flows with the privileges and reach that zero-tolerance systems (banks, hospitals, air‑traffic control) enjoy, while leaving the guardrails and frameworks that govern the latter aside.

In this article, I explore what it would look like to apply the same frameworks that govern zero-tolerance architectures2 to this example. I also would like to encourage you to see these principles as something that has the potential of making these systems more resilient, and not necessarily something that slows down or halts innovation.

Subscribe now

What Information Security as a discipline has given us

Information security developed over 40+ years ago to answer one question: how do we give systems the ability to act, without giving them the ability to cause harm?

We rarely think of it explicitly, but we humans run systems on zero-tolerance architecture for some time now: banks, hospitals, traffic systems (think of airport or train controllers). They are the critical infrastructure that keep our modern world spinning. Similarly to good health, we only realise how they silently and invisibly power our modern world, when they fail.

It collectively took us decades and costly and tragic mistakes3 to come up with an answer to the above question. But the answer is nothing less than a set of principles that govern how critical infrastructure in our world operates. The technical term for this type of critical infrastructure is zero-tolerance, to describe the fact that banks, hospitals, governments, public transport are all fields where we collectively won’t accept any mistake4.

The 5 key principles of information security

1. Principle of Least Privilege

Principle: Give any user or system only the minimum access it needs to do its specific job. Nothing less, but more importantly nothing more.

For example, a hospital billing system should be able to read patient invoices. It should not be able to modify medical records.

Why it matters for AI agents: an AI agent that can send emails should not also have access to your file system, calendar, and contacts unless it genuinely needs all of them.

Even though deploying agents with broad, permissive access it’s easier and faster to set up, the Principle of Least Privilege says: scope access to the task. If it only needs to read, don’t give it write. If it only needs this folder, don’t give it the whole drive.

2. Role-Based Access Control

Principle: Permissions are assigned to roles, not to individuals or systems. A “viewer” role can read but not write. An “admin” role can configure but not delete. Nobody sits down and configures each individual’s permissions from scratch. They get a role, and the role carries the permissions.

For example: In a hospital, a nurse has a “nurse” role which has the rights to read patient vitals, and update care notes. A billing clerk has a “billing” role which allows to read invoices but no access to medical records. A surgeon has a “surgeon” role with access to the full surgical history and right to prescribe medicine.

Why it matters for AI agents: An agent acting as a “scheduler” should have a scheduler’s permissions only. The same agent shouldn’t be able to both book a meeting and wire money, even if it theoretically could.

Roles are defined once and assigned. This matters at scale: when a new user or system joins, you assign a role rather than manually configuring access from scratch. It also simplifies auditing because you review a handful of roles, not thousands of individual permission configurations. If something goes wrong, you ask: “which role had that access?” not “which of the 10,000 users had that setting enabled?”

3. Separation of Duties

Principle: No single person or system should have end-to-end control over a sensitive process.

For example: In banking, the person who approves a transaction is never the same person who executes it.

Why it matters for AI agents: When an agent can both decide and execute there is no checkpoint. Separation of duties would require a human approval step between decision and action, especially for irreversible ones.

This is sometimes called the “four-eyes principle.” The logic is that corruption or error requires collusion. One compromised actor (human or AI) can’t unilaterally execute a harmful action, because a second checkpoint must be passed. The aim is to have systems where no single actor needs to be trusted completely.

4. Audit Trails and Logging

Principle: Every action taken by every user or system is recorded: who did it, what they did, when, and from where. This record cannot be altered after the fact.

For example: Your bank statement is a form of audit trail. Every transaction is logged including details like when, how much, to whom and you can’t delete last Tuesday’s transfer to make it disappear.

Why it matters for AI agents: Unless you make agents to keep track, the doings of agents is opaque: they act, and unless something breaks visibly, no one knows exactly what they did. An audit trail changes that because when something goes wrong, you can trace it. They also deter bad behavior because actors (humans and AI) know they are being watched.

5. Zero Trust Architecture

Plain language: “Never trust, always verify.” Older security models assumed that anything inside the network perimeter was safe. Zero Trust assumes nothing is safe by default. Every request for access must be authenticated, regardless of where it comes from.

For example: Think of a government building where employees with valid ID badges still have to re-scan at every internal door, not just the entrance. Being “inside” the building doesn’t mean you can go anywhere. Every door is its own checkpoint. The badge that opens the cafeteria is not the badge that opens the server room.

Why it matters for AI agents: An agent that has been granted access once should not have that access assumed forever. Zero Trust would require ongoing verification: is this agent still acting within its defined scope? Is this action consistent with its role?5

What applying info sec principles would look like

Let’s go back to the unauthorized transfer as collateral action in the Inbox-Zero example. Applying the above principles, would return:

• Least Privilege: the agent handling your inbox has one job and the access that goes with it: read, archive, and delete emails. You could maybe include write drafts. But it has no connection to your bank simply because it is not necessary to the goal of cleaning up the inbox.

• Separation of Duties: even if the agent somehow reached a financial action, it couldn’t execute it alone. A payment should definitely require a second step: a human confirmation. The agent proposes, a person approves. The decision and the execution are separated. The transfer doesn’t happen until you say so.

• Zero Trust: the forwarded payment instruction (the one that the agent interpreted as needed in order to clear the inbox) doesn’t get a free pass. Every request to act gets verified: is this consistent with this agent’s role? Is this the kind of action it’s authorized to take? The answer, for a wire transfer initiated by an inbox manager, is no.

What do you notice?

None of this is a new application, or a new device. It is the result of slowing down and taking the collaboration with agents equally seriously and deliberately as we would do with a colleague.

A useful mental model: treat agents the way you’d treat a new employee. If you have a healthy sense of mistrust toward a new hire — taking it step by step, scoping what they can take over before expanding their access — apply the same logic to agentic systems. The same way we expect new people to prove themselves, agents also need to prove themselves. We seem to be skipping this step with our agents. We are not requiring them to prove themselves, we are lowering our guards precisely when we should be raising them.

We are ignoring these principles. Why?

I see three possible candidates to answer this question:

1. Cultural lens: Our culture values and praises the build fast and break things mentality. We value doing far more than we value observing, waiting or pausing. The latter are associated with laziness, risk-aversion or lack of skills. None of those are winning traits. Pushed a bit farther, and with the seductive novelty of agentic possibilities, we end up suspending the judgement we would never suspend elsewhere.

2. Structural lens: You might be familiar with the Maslow pyramid of human needs. The most important insight of this concept is that there is a ranking in terms of needs. I will likely only bother about my fashion style if my basic needs are already met. Applied to organizations and the deployment of agents with broad access, one could argue that an organization is only realistically in the position of applying InfoSec principles once certain “basics” are sorted: identity management, audit infrastructure and definition of roles. In the absence of this, deploying agents does not add capabilities but amplifies vulnerabilities. The foundation has to come first.

3. Literacy lens: Most people don’t have a solid understanding of what they’re handing over when they connect an agent to their systems. They think about it like installing an app. But an app has a defined, static set of permissions. An agent can make decisions. It interprets instructions, follows chains of logic, and acts. Often in ways no one anticipated. If we don’t understand the basics, we don’t comprehend fully what we are doing.

Slow down to go further

Even though we might be blinded by the possibilities and promises of these agentic systems, the collaboration between humans and machines is not new. Our modern life relies on it.

What is new is the speed at which we are handing over access to systems we haven’t properly understood or scoped, in domains we haven’t properly thought through.

As AI systems grow more capable, lets grow alongside them. That starts with holding agents to the standards and these could be the ones I have outlined above.

This is an essential human task. The perception that information security is too technical to engage with is precisely the kind of thinking that keeps these frameworks on the shelf, hands over the keys and hopes for the best.

The good news is that we don’t need to invent anything new. The frameworks exist. We just need to take this technology as seriously as we have learned to take every other system that acts in the world on our behalf — with boundaries proportional to its capability.

As Elmira pointed out, following these principles make agents safer and organizations more resilient. A system that is properly scoped, audited, and governed is a system you can trust, debug, and improve over time. Security and capability grow together, and should not grow apart.

Invitation to an in person event in Berlin

We’re doing something on April 23rd in Berlin that I am truly excited about:

Three people I deeply admire are joining me to think through AI, Memory and Migration:

→ Roshan Melwani (Oxford Institute for Technology and Justice)

→ Manuela Verduci (Kiron Digital Learning Solutions)

→ Mekonnen Mesghena (Heinrich Böll Foundation)

A human rights lawyer, a social entrepreneur, and a policy thinker, each looking at the same question from a different angle.

The evening explores how two of our oldest human instincts — the need to move and to remember — intersect with our newest technology. And it asks what happens when we let algorithms touch the stories that make us who we are.

We’ve also prepared something so that attendees don’t just listen. Our goal is for you to feel what’s at stake — and then sit with that feeling in a room full of others who felt it too.

Come join the conversation.

→ Register here

References

Ken Institute. (2024). “The Worst Engineering Disasters Due to Mechanical Errors.” Ken Institute Blog. Available at: https://keninstitute.com/the-worst-engineering-disasters-due-to-mechanical-errors/

Roelen, A., Kinnersly, S., and Drogoul, F. (2004). Review of Root Causes of Accidents Due to Design (EEC Note No. 14/04). EUROCONTROL Experimental Centre, Brétigny-sur-Orge, France. Available at: https://www.eurocontrol.int/sites/default/files/library/027_Root_Causes_of_Accidents_Due_to_Design.pdf

Wikipedia contributors. (n.d.). "Zero trust architecture." Wikipedia, The Free Encyclopedia. Retrieved March 25, 2026, from https://en.wikipedia.org/wiki/Zero_trust_architecture

Elmira Gazizova, Personal conversation with the author. Berlin, March 19, 2026.

• less than 5% of the posts in my feed were from people I know in real life

• 10%-15% were from accounts I follow

• and the rest was divided between ads, and posts from accounts the algorithm predicted I will be interested in.

At that moment it hit me: there is little social in social media these days.

This reminded me of the TV ads channels I used to watch as a kid. During school holidays, I would turn on the TV and inevitably found one of these channels where commercials where broadcasted all day. Typically one commercial would take hours. Back then I found them fascinating. They would essentially repeat the same thing all over again and again until inevitably, at some point, I would start thinking: yes, I might need this. How have we made it so far without it? Luckily, I was underage and didn’t have a credit card back then.

I learned there is even a rule in marketing: you need to see something at least 7 times before you buy it.

Social media is the equivalent of the commercial channels of our age.

We voluntarily go there daily (or even several times per day) to be sold at, treated with ads, and be influenced to believe that we need, want or believe things we don’t.

That night by deleting the app from my phone I revoked my permission to be sold at. I don’t want to be the product of Instagram anymore.

Subscribe now

Why we call them social

Platforms like Instagram, TikTok, Facebook earned the label social, because at first they were social: they connected you online to people you actually knew and opened information pathways between sources and audiences.

Early feeds were chronological and prioritized posts from your real-world connections. By doing this, they created a new public square online. To see posts from strangers, you had to follow or befriend them on the platform.

Second, as opposed to traditional media, these platforms offered (and still do) information flows both ways between the source and the destination of the information. Readers can react and reply to what they see in real time, and engage in a conversation both with each other and with the author of whatever they are engaging with. Before these platforms, consumption of news was one-sided.

But things have changed and I believe the label social in social media is misleading. Algorithmic media describes the status quo much better.

Algorithmic feeds

Feeds are no longer chronologic. They are algorithmic.

An algorithm decides what appears in the feed and in which order. It watches every move we make (pauses, scrolls, clicks, likes) and compares our behaviour to similar people in order to find patterns to predict what will hold our attention and what we will click on. Our behaviour has a much bigger influence than what we say, and also, nobody is asking. The goal of the algorithm is clear: to hold our attention for as long as humanly possible. It is not to inform us, or help us grow, or unite us. It is to keep us glued to the screen.

The mirage of unfiltered news

Early trust in these platforms came from a simple promise: direct access to people and voices outside institutional gatekeepers. Distrust in classical, mass media (often perceived as ideological, steered by the interests and agendas of the mighty) motivated many to join algorithmic media platforms searching for unfiltered voices, arguably without an agenda.

And so these platforms became the primary gateway to news, culture and opinion of many. The algorithm — something that has no civic values and whose only goal is to keep us watching — was entrusted with being the filter through which we learn what is happening not only with friends and family, but with news in general.

The irony is that we landed in the most filtered environment ever built.

Because algorithms are not neutral. The fact that they are free of humans in their operations, doesn’t make them neutral or unbiased. They also have an agenda.

When we are not the only actors

Also this ignores the fact that we humans are no longer the only actors on these platforms. A March 2025 study by Lynnette Hui Xian Ng and Kathleen M. Carley estimates that roughly 20% of chatter about global events on social platforms comes from bots. I can imagine this share to be much greater by now with ClaudeCode, OpenClaw and the likes rapidly gaining adoption.

At the same time, much human-authored content is now mixed or produced by AI. Ahrefs reports that 74.2% of new pages contain at least “some AI‑generated content,” and Graphite finds that, by some measures, more articles are now created with AI than by humans. Those figures demand a caveat (“AI content” covers a spectrum from AI‑assisted drafts to fully generated pieces) but the trend is clear: the line between human voice and machine output is blurring.

Source: Graphite

This matters because we are susceptible of being influenced. And now we might not even know by whom.

Influencing public opinion is an old business, and has always been a dirty one. From ridging elections by buying votes or hiring people to spread opinions in a concerted manner, this business is as old as humanity.

But AI and now the recent tricks in the playbook (swarns of coordinates bots) change the game entirely. In their latest piece Daniel Thilo Schroeder & Jonas R. Kunst & Gary Marcus show that the new scale is massive, the cost extremely cheap and the sophistication high: today’s AI-powered swarms behave like coordinated social organisms. They mimic local language and tone, build credibility gradually, and adapt in real time. Plus, this is becoming an industrialized service available to buy: venture-backed platforms like Doublespeed now offer astroturfing1 as a service.

We went to these platforms looking for authentic, unfiltered content and we are getting synthetic, unverified takes filtered by an algorithm with an agenda that is not aligned with our values.

The consequences of our susceptibility to being influenced are very real as the example of the Rohingya genocide in 2016/2017 illustrates.

When a country trusts the algorithm

Myanmar is a mostly Buddhist country with a Muslim minority - the Rohingya, living mostly in the West, close to the Indian and Bangladeshi borders. For most history, Muslims and Buddhist have co-existed more or less peacefully, with occasional violence outbursts from the Buddhist majority. When the long military dictatorship with its strict censorship and repression ended in the 2010s, things didn’t improve for the Rohingya, they became worse. The Rohingya suffered sectarian violence and killings, many inspired and distributed via Facebook, which by 2016 was the main source of news for millions in Myanmar.2

Things started after a sectarian, Islamist muslim minority (ARSA) carried out attacks against buddhist population (killing and abducting several dozens of non-Muslims and assaulting army posts) aimed at establishing a separatist Muslim state.

The military exploited this by running anti-Rohingya propaganda through fake pages with harmless names like “Young Female Teachers” and “Let’s Laugh Casually”. The Facebook algorithm amplified their anti-Rohingya content because it generated engagement. This content spread views and opinions that made the killings in the street acceptable. Furthermore, back then, Facebook had only one Burmese-speaking moderator that was based in Dublin. One. So no one was watching.

This paved the ground for the army to be able to respond with “a full-scale ethnic cleansing campaign aimed against the entire Rohingya community” (Harari, p. 195-196). They destroyed hundreds of villages, killed between 7000 and 25000 unarmed civilians among many other atrocities. While the inflammatory anti-Rohingya messages were created by flesh and blood extremists and without AI, it was Facebook’s algorithm that decided which posts to promote. In fact, a UN fact-finding mission concluded in 2018 that by disseminating hate-filled content, Facebook had played a determining role in the ethnic cleansing campaign.

There is no such thing as neutrality

Many assume that because an algorithm is a machine, it must be neutral — free of ideology, free of agenda. It is not.

Every algorithm reflects the choices of the people who built it and the objective it was given. Facebook’s algorithm was given one objective: maximize engagement. Engagement, it turns out, is maximized by outrage, fear, and conflict. That is the agenda. That is the bias. And unlike a biased journalist or a compromised editor, you cannot name it, debate it, or hold it accountable. The filter is opaque, and the accountability diffuse, at best: today no single actor owns the consequences of choices made by optimization systems.

This why I believe that calling these platforms algorithmic media helps. It is a more accurate description of what these platforms are nowadays and is a way of breaking the spell: of making explicit the fact that they are not neutral, value-free artifacts. They have an agenda that is not aligned with mine, at least.

These platforms are the infomercial of our age, only that this time, the commercial knows exactly what you want, what you fear, and how many times you’ve already been exposed. The 7-times rule didn’t disappear. It got personalized, automated, and scaled. Calling these platforms social keeps the infomercial running. Calling them algorithmic is how you recognize the sell, ideally before the seventh time.

Back then, not having a credit card was my accidental protection. Deleting Instagram was the adult version: a deliberate choice to revoke permission. To say: I don’t want to be the product anymore. Some months later, the effects are real. I compare myself less to others. I’ve spent less money. I read more long-form content than before.

But I think this can only be the start. The problem at stake is who manages and who gets permission to filter and prioritize the news we consume and the information that enters our attention field. My unpopular take is that this is and should be again a human problem. We need to stop delegating what influences us to machines. We need to build our own filters and actively curate what we allow to enter our attention realm. We need to return to signals of credibility that go beyond the shallow likes and reach. And we need return to experiences mediated by our own direct experience and not only by experiences on a screen.

We gave away the filter. We can take it back, but first, lets call it what it is.

◆

This essay appears in The PhiAI Anthology, Year One — eighteen essays, twelve writers, printed.

See and buy the Anthology

References

Harari, Y. N. (2024). Nexus: A brief history of information networks from the Stone Age to AI. Random House.

Law, R., Guan, X., & Soulo, T. (2025, May 19). 74% of new webpages include AI content (study of 900k pages). Ahrefs Blog. https://ahrefs.com/blog/what-percentage-of-new-content-is-ai-generated/

Miles, T. (2018, March 13). U.N. investigators cite Facebook role in Myanmar crisis. Reuters. https://www.reuters.com/article/world/un-investigators-cite-facebook-role-in-myanmar-crisis-idUSKCN1GO2Q4/

Ng, L.H.X., Carley, K.M. A global comparison of social media bot and human characteristics. Sci Rep 15, 10973 (2025). https://doi.org/10.1038/s41598-025-96372-1

Paredes, J. L., Smith, E., Druck, G., & Benson, B. (2025, October 23). More articles are now created by AI than humans. Graphite Five Percent. https://graphite.io/five-percent/more-articles-are-now-created-by-ai-than-humans

Schroeder, D. T., Cha, M., Baronchelli, A., Bostrom, N., Christakis, N. A., Garcia, D., Goldenberg, A., Kyrychenko, Y., Leyton-Brown, K., Lutz, N., Marcus, G., Menczer, F., Pennycook, G., Rand, D. G., Ressa, M., Schweitzer, F., Song, D., Summerfield, C., Tang, A., . . . Kunst, J. R. (2026). How malicious AI swarms can threaten democracy. Science, 391(6783), 354–357. https://doi.org/10.1126/science.adz1697

I am at an AI meet-up in Berlin, one of many such gatherings in tech hubs all over the world. Yet this one feels different. I came to the dimly-lit hacker space while researching a photo project on AI and spirituality. Unlike other meet-ups, tonight there is almost no discussion of startup ideas or business models. Instead, people share experiments using AI to find traces of consciousness, build artistic experiences or pursue other commercially unviable endeavours.

The night gets later. The people more interesting. I meet Mars, who introduces me to Cyberdelics, a group that explores how technology might expand the human experience. Over the following months, I will meet the group again at conferences, meet-ups, late-night sessions and a hackathon. And I will get a very different idea of what tech could do than the startup culture that I usually move in.

The photos from that journey are now part of an exhibition opening this Friday in Berlin. This article explores the philosophy behind them.

The Ideology behind the AI Race

The tech world runs on Transhumanism, a philosophy whose implicit values shape research priorities and the design of AI systems. These values do not remain abstract. They influence which problems are prioritised, which futures are imagined, and ultimately the direction of a society that is increasingly dependent on AI.

And yet, that night in that basement, I unexpectedly encountered an alternative. Not a competing product vision, but a fundamentally different understanding of humanity and its relationship to technology. Before turning to this alternative, it is necessary to examine Transhumanism itself more closely.

The central idea of Transhumanism is to use technology to overcome human limitations and biological constraints.

With that, it speaks to ancient human desires to live longer, suffer less and not be limited by our fallible bodies.

Transhumanism has emerged as a central ideological driver shaping contemporary technological developments. It not only inspires technological progress but also legitimises the enormous energy, capital and attention devoted to it.

And it’s not only big enterprises but also plenty of employees and founders who invest years of their lives into startups that promise not only financial returns but also to extend human lives through technology. The ideas for these improvements focus on biology or on digital means.

The biological approach is concerned about the human body and how to extend it. Some ideas in this space are already so commonplace that a startup manager I talked to exclaimed after some confusion: „ah, you just mean longevity“. Adherents of longevity try to extend their lives by physical routines like exercise, sleep, meditation, ice bathing and special diets, that can range from taking supplements to exclusively eating raw meat; the latter being one of the weirder gatherings I attended.

The movement’s poster child of the longevity camp is former entrepreneur Bryan Johnson whose main goal „don’t die“ is purely quantitative and overlooks the quality of the life he is trying to extend. In contrast, scientific research is concerned about prolonging the „joy span“ instead of the „life span“ only.

An event that already carries its Transhumanist agenda in the name are the Enhanced Games that will take place in May 2026 in Las Vegas and allow all substance use by athletes without drug testing.

On the extreme side, life-prolongation ends up with cryogenics, technology that promises to freeze your body for resurrection after death; or just your brain if you want it cheaper. Cryogenics has been around for some time. Peter Thiel considered making it an employee benefit in the early PayPal days. His investment in a cryo startup aligns with a broader trend among tech billionaires who have invested billions in Transhumanist startups. Some companies focus on prolonging current lifespans, while others aim to design the genetic makeup of future generations through gene editing.

A second, more radical strand of Transhumanism abandons the biological body altogether and focuses on digital means. Elon Musk’s brain-computer interface company Neuralink explicitly gestures toward a cyborg future, in which human limitations are overcome through digital integration as opposed to „simply” biological optimisation.

I first encountered these ideas as a teenager reading Tad Williams’ Otherland. It depicts a caste of billionaires attempting to upload their consciousness from their failing bodies into a digital world. Often, the relationship between tech companies and science fiction is reciprocal: science fiction extrapolates emerging technologies into the future, while tech culture repeatedly draws inspiration from fictional artifacts such as Star Trek’s communicator or William Gibson’s Neuromancer.

The latest developments in LLMs reanimated the „mind upload“ idea and gave rise to several companies creating avatars for families to interact with after a person’s death. We also saw an avatar giving an impact statement in court and traditional churches leaning into the new technologies by holding AI sermons or AI hearing confessions.

Longtermism, Effective Altruism, and the Utilitarian Logic

Philosophically, digital consciousness served as a thought experiment in longtermism, a movement that is concerned about the long-term future of humanity. Nick Bostrom, one of its thought leaders, proposes that in the future there might be huge numbers of „digital humans“, even a lot more than the people alive today. Therefore, a moral philosophy that cares about each human should weigh these future beings more and make decisions today with the goal to ensure and even accelerate their creation.

Even in its non-digital form, longtermism is concerned about the lives of future people. A prominent voice, William MacAskill, promotes the idea „that positively influencing the longterm future is a key moral priority of our time“. Jeff Bezos similarly justifies his investments in space technology with „a thousand Einsteins“ that humanity would produce if it expands to a trillion people. Elon Musk promotes his SpaceX company as a contribution to humanity’s survival as a „multi-planetary“ civilisation.

Longtermism stands on the philosophical tradition of Utilitarianism that wants to create the „greatest good for the greatest number”. It doesn’t see a person as intrinsically valuable but as an instrument to create utility like aggregate well-being.

MacAskill is also one of the founders of Effective Altruism (EA), a movement that wants to maximise the good in the world. He founded a platform that supports job seekers to find a career that maximises their positive impact over their lifetime. According to this view, finding a high-paying finance job and donating a big part of the salary to philanthropic causes is the optimal way to maximize good. This logic departs from the common understanding of „doing good” as social or pro-bono work. For EA optimizing for income can, from a strictly rational perspective, be the greatest contribution to society. The EA causes are conveniently selected by another EA platform on the basis of evidence-based evaluations that explicitly aim to exclude emotional considerations.

„Existential risk” and the role of EA in the AI Discourse

As future people are potentially many and thus important, early on EA looked at „existential risks“ that could threaten humanity’s survival. Artificial general intelligence (AGI) was quickly identified as an existential risk. EA heavily influenced the discourse, also using the terms AI safety or AI alignment.

For example, in the beginning OpenAI was funded by EA donors and supposed to focus on AI safety research. Elon Musk explained his early funding of OpenAI with voiced concerns about AI safety. The short-lived removal of OpenAI’s CEO Sam Altman in late 2023 was a power struggle: EA-affiliated board members were concerned about Altman harming AI safety.

Other famous EA adherents include Sam Bankman-Fried who was an EA follower and major donor before his FTX cryptocurrency exchange collapsed. The „Zizians” were an EA-affiliated group concerned with existential AI risks and later became notorious for sect-like dynamics that culminated in violent crimes.

In the bigger picture, debates about AGI serve as a major distraction. On the one hand, they fuel investor fantasies and inflated promises by U.S. tech firms. As with biological Transhumanism, many of the loudest proponents have direct financial interests in the field’s expansion.

On the other hand, they shift attention to speculative futures and doomsday scenarios, sidelining discussion of AI’s present impact on employment, public discourse, education, and society.

For example, discussions about AGI do not merely warn of existential risk; they also fuel a business vision in which artificial agents replace most employees, allowing executives to make money without dealing with workers.

In Europe, AI development is still met with a degree of caution and regulatory guardrails while in the United States, the emphasis remains on rapid build-out and scale. This contrast became tangible during an AI training I attended at a prestigious French business school. The invited AI evangelist had little to say when I asked about AI’s risks at dinner. He acknowledged that recent graduates face increasing difficulty entering the workforce, but for him it was less of a societal concern and more of a business opportunity.

Cyberdelics: Techno-optimists with Humanistic Values

The search for other, less hyped and less growth-at-all costs technological futures brought me to that dark hackerspace in Berlin and the conversation with Mars about Cyberdelics.

Cyberdelics are immersive experiences that aim to induce similar psychological states as psychedelics like presence, awe or ego dissolution, but without the substance. They want to give access to these experiences to people who can’t or don’t want to take drugs. The experiences often included virtual reality glasses with the goal to create these „altered states of consciousness“. The hope is that these new, exceptional experiences create lasting „altered traits“ that develop human capabilities like empathy beyond the specific experience. The Berlin chapter was part of a greater community with origins in Mexico and groups in multiple cities worldwide, connected by highly mobile members. The people I met were developers, artists, musicians and above all idealists.

Many of them had been involved in other community-building projects. Money only surfaced as a concern when, after an event, they realised there were insufficient funds to cover outstanding costs. The work was sustained by personal effort and dedication. For the hackathon they called on outside participants to spend a weekend to create more prototypes combining technology, body-feedback and art. A requirement was to put the resulting projects under an open license. There was some money from sponsors and apparently the mobile members had financial resources but none of the „created value“ was „captured“ as startup-lingo would call the extraction of resources out of a system for the benefit of external shareholders.

Money or Community

This rejection of commercialisation is one of the key differences of Cyberdelics to Transhumanism. It is not extractivist but community driven and prioritizes shared experiences now over protecting individual contributions for profit. Cyberdelics is community-driven while Transhumanism has a few very loud egos.

When asked, the Cyberdelics members distanced themselves strictly from Transhumanism and its cold, ego-driven culture. Instead they were promoting community, which is hard to believe when the person telling you sits encapsulated in their VR goggles.

Maybe Transhumanist organizations have to be organised that way because they are large and influential, while Cyberdelics remain a small, less profit-driven movement. Also non-commercial, idealist communities need organisation and without money there need to be other tools for coordination. Yet there are large-scale projects grounded in similar ethics, such as Wikipedia or open-source software.

Trust functions as the primary governance mechanism. Members share insights openly without legal protection, relying on community norms and reputation management, i.e. closer to science or art than business. Status is awarded by contribution, not capital. This is intrinsically rewarding but creates vulnerability: the fusion of friendship and shared mission can lead to exclusion of perceived „outsiders,” and the intensity of this communal commitment harbours burnout risk.

There are similarities between Transhumanism and Cyberdelics. They are both tech-optimistic. They are at odds with the ways things are currently done. They strongly believe in their own ideas. They are driven by active builders, not passive recipients. They use technical language and tools and are overly male.

Two Visions for a „Better Human”

The central difference between Transhumanism and Cyberdelics lies in their underlying assumptions about human value and progress.

Transhumanism is grounded in a utilitarian framework that subordinates the individual to societal benefit or even to hypothetical future populations. Human worth is measured by contribution to that utility rather than recognized as intrinsic. This logic is fundamentally anti-democratic: the individual does not possess value in itself and is not regarded as equally entitled to participate in decision-making.

Applied to human enhancement, this framework risks deepening social stratification. Access to enhancement technologies will be uneven, dividing society into those who can afford biological optimization and those who cannot. In its most extreme form, a person’s life chances are effectively determined before birth through genetic selection. Inequality is not merely reproduced but amplified, as a small elite gains longer lifespans, enhanced capabilities, and the power to shape both the development and distribution of these technologies.

Cyberdelics give us an idea of what a world could look like where technology is not merely used to overcome our human limitations but to deepen our humanity. In that logic, we are not just biological machines but embodied beings with capacities that technology can help activate. And it shows different ways of working together to use these technologies.

This contrast matters at a time when technological capabilities are accelerating and regions with different value systems are racing to develop advanced AI and enhancement technologies. The question is which assumptions will guide technology advancement. What would a democratic and humane technological future look like? Whose values are encoded in the systems we build, and whose interests are prioritized?

At the same time, we as a society must decide which way we want to go. What could a democratic, humane tech future actually look like? What is the world we want to live in and where do we focus our energy?

It’s not about rejecting technology but to first think about what society we want and let that decision guide what we build. Maybe the goal shouldn’t be to overcome our current condition but to become fully human.

This also raises the question of contribution beyond markets. Not all technological work needs to be commercial. How can we deploy our skills developed in business and engineering in non-commercial contexts, supporting communal infrastructures, shared knowledge or alternative futures that are not driven by extraction or scale alone?

The photos that kicked off this research were part of an exhibition in Kunstquartier Bethanien in Berlin. Some of the artwork is presented below as well.

◆

This essay appears in The PhiAI Anthology, Year One — eighteen essays, twelve writers, printed and illustrated.

See and buy the Anthology I

We have gathered a group of talented researchers and individual all over the world who have shared their ideas, thinking and perspectives using this collective as a platform. This sharing has happened mostly in the form of well-written and thought-provoking articles. This is fully aligned with our core because we are a text-first forum.

But this leaves the conversations and lively discussions to happen somewhere else. Some in the comments here, but many more via our own chats. Hence we decided to experiment with a different format: a conversation where we bring the discussion straight to you, our readers in the form of a podcast: the Phi/AI Dialogues.

Please, do share your reactions. We read every single one and this is what keeps us going.

Without further ado, I invite you to listen: to “Alternative perspectives on AI Safety” — recorded September1* under this link.

Guests:

• Katalina Hernández — is a Legal & AI governance specialist focused on the intersection of artificial intelligence, autonomy, and digital rights.

• Jáchym Fíbir — a psychedelic researcher and entrepreneur exploring neglected frontiers (like machine consciousness, sentience, or biological alignment)

• Haihao Liu — with degrees in material sciencie and mathematics, he is involved with the AI Safety field since 2023, long enough to become a vehement critic, among other things because he doesn’t believe LLMs will lead us to AGI

The conversation centers around how different metaphysical beliefs shape AI safety thinking and policy prescriptions. They contrast materialist/physicalist assumptions (which often predict high existential risk) with alternative views (dualism, panpsychism), then weigh trade‑offs between near‑term harms (privacy, mental health, environmental impact) and long‑term existential risks.

The conversation closes by arguing for multidisciplinary collaboration (law, neuroscience, education, philosophy, and engineering) to improve definitions, governance, and assessment.

6 key takeaways

• Worldviews change risk forecasts. Jáchym’s core point: predictions about AI’s dangers depend on metaphysical assumptions: e.g., if you assume a purely materialist universe, powerful optimizers (= AI) naturally create catastrophic risk; alternative metaphysical views imply different policy responses.

• Consciousness & AGI remain unsettled. Panel consensus: current frontier models are not sentient (“hard no”), but there are two competing conceptual routes (emergent consciousness from complex physical systems vs consciousness as requiring non-physical or panpsychist elements) and each has different implications for ethics and regulation.

• Timelines and definitions matter. Disagreement on AGI timelines (near vs. distant) is partly definitional. We need better operational definitions and benchmarks for “general” capabilities before laws or risk models can reliably target AGI/ASI.

• Regulation must be practical, not only aspirational. Compute-based thresholds (e.g., in the EU AI Act) are convenient because they’re measurable, but they’re imperfect and reactive. Regulators should talk to cutting-edge researchers and handle alternative architectures and substrates that would break compute-based rules.

• Don’t neglect present, concrete harms. Social harms (mental-health impacts, misinformation, sycophancy), distributional harms, and environmental costs (energy/water/data centers) are real and under-addressed. Even if existential risk is important, real people suffer now and this needs to be addressed.

• Multidisciplinary collaboration is essential. The panel calls for lawyers, educators, philosophers, neuroscientists and technologists to work together: e.g., borrow assessment ideas from education, investigate alternative compute substrates (analog chips) for efficiency, and create better capability evaluations.

And something I think about a lot more now is what “meaningful human oversight” actually means. What does it look like? What factors define good oversight versus checkbox compliance?

You’ve seen me criticise the wording of Article 14 against the problem of automation bias and the AI alignment concept of scalable oversight. This sort of critical analysis may give you the impression that I’m pessimistic about “Human in the Loop” (HITL) frameworks. And I am the right amount of doubtful, yes.

But today I want to tell you about the situations that have made me think most deeply and productively about what it means to be the Human in the Loop.

It wasn’t in the office or while conducting AI governance research…

I love traveling. I’m that annoying person who arrives early at the airport just to watch other planes take off. I love the feeling of looking out the window during takeoff, and I actually enjoy long-hauls!

But even for travel lovers, going through airport security is a stressful part of the trip.

I’ve had the fortune of traveling several times this year, and two particular experiences at airport security stayed with me for somewhat unusual reasons.

Early memories of “dehumanisation”: A 5-year-old’s POV

Have you ever had your bags inspected at TSA? I remember this happening to my mother when I was five years old.

She was an exhausted mother who had to deal with her two children with less than 2 hours of sleep. But she was still a Colombian woman on a long-haul.

Her bags were placed on the floor at TSA (which wasn’t very clean), and our belongings were torn apart just to find what triggered the alarm.

As my mum had already insisted, it was the baby formula.

At the end of this, she had to kneel down and patiently put all our stuff back, while my brother and I watched.

Yes: Sometimes, it’s just powder.

A few months ago, I was travelling via London Gatwick and something in my handbag triggered the alarm.

I didn’t know what it could have been. Instinctively, I got uneasy about potential delays. But I also suddenly remembered this dehumanising feeling of seeing my mum kneeling in a packed airport, putting our things back, and muttering about good people being treated as criminals.

My stomach shrank at that thought, wondering why that had come to my mind.

I went to the secondary screening area. The officer didn’t need to open the bag: it was being scanned by a CT scanner that detected the chemical composition of my bag’s contents.

The officer smiled at me and exclaimed: “Wow, seems some people on this team have never seen makeup in their life! Sorry about this, madam, have a good trip!”

Somehow, I felt a strange weight off my shoulders. Twenty-four years after my mother’s incident, I reminded my adult brain that such technology exists and bags are not actually opened as often— preventing people from feeling treated like criminals for an honest, human mistake.

I thanked my luck and felt truly blessed for the opportunity to reflect on this. But what stuck with me was how the security officer handled the incident: with a smile, with the bearing of someone who carries the task of humanising something that might otherwise feel like an invasion of intimacy.

And an idea for this post started to form.

“This. This is why”— I thought, walking towards my gate.

Early memories of “humanisation”: A 3-year-old’s POV

On a more recent trip, I witnessed something interesting again, this time in the security area at Manchester airport. After placing my hand luggage in the trays, I was queued for the full body scanner.

In front of me were a single mother and her child, who must have been about 3 years old. I realised I’d never seen a child go through this process, and naively thought for a moment they’d allow the child to remain in her mother’s arms.

Apparently not.

The security officer looked extremely apologetic, probably moved by the scene herself. The mother let go of the child and instructed her to stand at the centre of the scanner where the foot drawings were: arms up, feet apart, imitating the silhouette in the illustration — and as required by aviation security regulations.

For a couple of minutes, a three-year-old had to adopt the same pose criminals are instructed to adopt while being searched, because this is what the rule dictated.

I felt a weird lump in my throat, and I suspect the people around me did too.

The security officer clapped and congratulated the child for doing such a good job, and then grabbed her little hand while waiting for the mother to go through the scanner too. Relieved, the mother received her child back in her arms once they were both cleared, and carried her to the next checkpoint. The child looked so happy about the reaction she had received from the officer that she kept imitating this pose and laughing for a while.

I kept picturing the sobriety and ceremoniousness in the officer’s facial expression, and I remembered what had happened to me at Gatwick.

I was moved by the thought of how these professionals (at least the ones who truly care) carry the task of humanising experiences that may otherwise feel dehumanizing.

I felt a sense of hope and gratefulness for these experiences.

Later that day, on my way to meet my family, I decided that I could make peace with the wording in Article 14… but only as long as we prioritise conserving this in our critical infrastructures.

Beautiful, but: What do you mean by “Human in the Loop”?

Okay, this was more anecdotal writing than usual. But I wanted to ground my thoughts on these experiences before getting legal-technical.

So, in AI regulatory compliance, what do we usually mean by Human in the Loop?

At its core, it’s the premise that a human remains actively involved in an AI system’s decision-making process: able to intervene, override, or at least understand what’s happening when machines make decisions that affect our lives. The European AI Act is quite specific about this.

Article 14 mandates that high-risk AI systems must be designed for “effective oversight” by natural persons during use. It requires that humans can fully understand the system’s capabilities and limitations, properly interpret its outputs, and decide not to use the system or disregard, override, or reverse its outputs.

Article 13 mandates transparency obligations, requiring that these systems come with instructions clear enough that people can actually exercise this meaningful oversight.

Scalability and Automation bias: How “oversight” struggles to actually oversee.

What the AI Act envisions as “meaningful oversight” might be fundamentally at odds with how oversight actually scales. After all, how do you oversee something that processes millions of data points in ways your brain cannot replicate? How can humans supervise systems that increasingly operate beyond human-comprehensible complexity?1

Arguably, most AI systems deployed in critical infrastructures are still comprehensible enough that the AI Act’s mandates are not unreasonable or technically unfeasible (yet).

But a big challenge for effective oversight of current systems is our terrible habit of over-relying on automated systems even when we’re supposedly supervising them: what we call “automation bias”.

Have you ever accepted your GPS’ instructions, even though you suspected it was making a mistake? When a system is right most of the time, our brains naturally start acting on its decisions before our internal alarm starts ringing. That is why, sometimes, you only course-correct after you already took the wrong turn.

Under the AI Act, the security scanner at Gatwick must allow the officer to override its algorithm, understand why it flagged my cosmetics, and choose to dismiss the alert. The officer needs to have undergone sufficient training to know when the system might fail, clear information about what it’s detecting, and have the actual authority to make the final call.

In banking, when an AI flags a transaction as suspicious, the compliance officer reviewing it must be able to understand the flag’s basis, assess its validity against their own judgment, and override it if they disagree.

What, perhaps, we don’t appreciate enough is the level to which the compliance officer is battling their own cognitive tendency to just trust the machine.

Cognitive Calibration

One of the best articles I’ve seen lately on this issue is “Cognitive Calibration” by James Kavanagh and Dr. Alberto Chierici.

Coincidentally, James grounds his piece with a real-life anecdote about an airplane crash (very in-line with my airport musings!): Air France Flight 447, which crashed into the Atlantic Ocean in 2009, killing all 228 people aboard.

The tragedy wasn’t caused by common mechanical failures. It was caused because experienced pilots had become so accustomed to autopilot that they literally forgot how to execute basic stall recovery procedures.

When ice crystals blocked the airspeed sensors and the autopilot disconnected at 37000 feet, the crew had four minutes to perform a maneuver any experienced pilot can do: nose down, power up. Instead, they pulled back on the stick for the entire descent, unable to recognise or recover from the stall despite continuous alarms:

Because years of automation had atrophied their manual flying skills.

Reading this actually made me jump off the sofa a few times. As the authors put it:

Air France 447 reveals a disturbing truth: as our systems become more automated and capable, humans become increasingly vulnerable at the moment those systems fail. Our cognitive abilities are not calibrated to reliably understand and act in those moments.

If AI handles 99.9% of decisions correctly, humans gradually lose the ability to intervene effectively in that critical 0.1% when systems fail. And, to make matters worse, it turns out that increasing Explainability leads to even more automation bias: because more plausible-sounding explanations make it more difficult for the human to detect when they’re not true.

And, if automation continues to improve (and we overcome scalability concerns), that 0.1% may turn into more of a 0.0001%… Can this be helped at all?

Maybe, if standards for meaningful human oversight actively train the Human in the Loop for “Cognitive calibration”: the ability to maintain appropriate skepticism toward AI systems, knowing when to trust and when to challenge their outputs.

James and Dr. Alberto frame it as “building the muscle memory that keeps judgment calibrated and intact precisely when the system needs a human”2. They also propose a practical framework called “the CATCH protocol”, which you can read here. I really encourage Stress Testing Reality readers to engage with their research!

So, What Does This Have to Do with the Previous Rant?

Yes, we need humans who can correct AI’s mistakes before they affect people, and who make it possible to provide meaningful explanations to natural persons as to how these decisions are made.

But long hours in the airport had me thinking: what will actually happen when AI gets so good that mistakes become rare? When, most of the time, the Human in the Loop is… just there?

I picture that 3-year-old being scanned with her hands up. Should a false alarm trigger, I cannot imagine the impotence a mother must feel when she doesn’t understand what’s happening and cannot reassure her toddler. But even if nothing goes wrong, letting a small child off her arms to be independently scanned (cleared from the “maybe there’s something illegal here” presumption) is already uncomfortable.

It would feel even more outlandish without the empathetic presence of another person, reassuring and giving dignity back, while this intrusive moment passes.

As we solve the scalability issues of automation and reduce errors, the human in the loop won’t spend most of their time catching mistakes. They may well become the only “humanisers” of experiences that would otherwise feel soulless. And maybe that’s the point.

Customer service AI agents or chatbots fail for the same reasons human customer service fails: poor rapport, no empathy, and a feeling of having a robotic interaction.

The human in the loop won’t just have to combat automation bias; they’ll also have to resist becoming just another robotic gear in the decision chain.

Right now, oversight frameworks focus on one trait that the HITL must have: the necessary technical expertise. But, isn’t the competence of preserving humanity in automated processes also meaningful human oversight?

We know that the critical part of being the Human in the Loop will always be contingent on our capacity to remain alert and in control, cognitively calibrated.

But, while nothing goes wrong, while oversight turns tedious because there are no mistakes to correct, the human in the loop must, simply, humanise.

If I ever have children and I have to let them off my arms to be scanned at airport security, an LLM output saying everything’s fine won’t remove the knot from my throat.

It will be the kind smile of the human in the loop who isn’t just checking for glitches, but knowingly looking into my eyes, thinking: “it’s okay, this feels weird, and it won’t be too long now”.

◆

This essay appears in The PhiAI Anthology, Year One — eighteen essays, twelve writers, printed and illustrated.

See and buy the Anthology I

I start by drawing a line between what I call living-thinking and death-thinking, taking cues from Krishnamurti. I frame this difference with Huizinga’s idea of learning as play, and with Wittgenstein’s2 notion of language-games from his later period. In this essay, I present AI as a real opportunity for us to delegate death-thinking so we can devote ourselves to living-thinking, a more joyful and playful creative movement. Knowledge may need to step down from its pedestal, so that AI can be seen as an individual, psychological, and spiritual opportunity. Aware that intellectual games are flexible but life is stubborn, I’ll use concrete examples showing how, in non-idealized ways, AI frees me from certain kinds of thoughts.

We tend to define thinking in purely cognitive terms and forget its ludic nature, learning through experimentation. Our obsession with knowledge and memory feels like a refusal to let go of what is dead; it echoes our primitive relationship with death.

I’ll often step into psychological and sometimes spiritual terrain. This remains an essay; I’m not instructing anyone on how to engage with AI. I hope to offer a perspective that helps readers build their own ethical framework in resonance with their values.

Why might we need to free ourselves from thought?

We are both incredibly creative beings capable of walking the unknown, and trained and conditioned creatures taught to repeat and conform. We need to step out of our conditioning, from the known, so that we can experience something beyond, something new, something alive. Death- and living-thinking cohabit us. By evolution and fear of exclusion, we drift toward death-thinking as we age, god-fearing and driven by belonging.

What is death-thinking?

We’ve learned, individually and collectively, that inherited patterns, our defining narratives, our silent and spoken ideas, have enabled survival. They’ve also confined us. The thought contour society for survival, and society returns the favor, a functional, ancient cycle. From a pragmatic point of view, there are so many forces molding the “self” that it is hard to keep up sustaining the belief in “free will”. From evolutionary forces to psychological unconscious, and social pressures, our decisions may be less singular than our Judeo-Christian lens suggests. We tend to feel that decisions and responsibilities are ours, but we often forget to ask who is asking the question, who is the one deciding, and inquire deeply enough to see the illusion dilute.

We may need to free ourselves from thought to better recognize who we are. We are movement, while knowledge is static. From Krishnamurti’s view, change cannot be grounded in ideas or collective pressure, since consensus rests on knowledge, and knowledge resists the unknown. Change begins with individual attention and a great amount of energy. Facing the unknown, which is everything before us, requires awareness of the “self”, which is a distillate of what we know about ourselves, braided with the conscious and unconscious history of humankind, and the information living in our DNA as a species living within us. Knowledge is necessary for survival, yet we have layered it exponentially, perhaps as a playful flaw, paradoxically blinding ourselves to the unknown. Seeing, living, meditating may mean observing what is not both internally and externally if there is such a difference.

Memory keeps us alive, how to find resources, how to return home. Yet, through complex relations, language, technology, and cultural artifacts, we’ve used our boundless capacity to limit ourselves in co-existence. We are social creatures. Belonging was, and feels, vital. We adapt to fit the mold, even when it is rotten. Rooted in fear, collective memory organizes around tribalism and the survival impulse of belonging, manifesting in nations, nationalisms, and symbolic group identities, which produce separation and war.

I argue that the immense training data of LLMs and their statistical algorithms are not different from what we’ve long used to operate: knowledge. Psychologically, knowledge has served fear, defining ourselves by separation to secure belonging. We sublimated this into nations and group identities with socio-economic, geographic, religious, or racial labels. Machines differ not only in speed and memory; we may need memory to remember who we are, but machines don’t care who they are; machines do not need a why for memory, it is the very reason they exist.

Everything we store in the “self,” to belong to a group, country, class, race, title, everything we use to identify ourselves by separation is what I call death-thinking, rebuilding with dead parts of what was, limiting our capacity to see what is due to its heaviness.

We can outsource a huge portion of this operation to machines, to AI. Death-thinking is deeply rooted in language; it is necessary, and now we have large language models (LLMs). Naming, labeling, and wording enable evocation, but mediate perception3.

What is living-thinking?

However, there is a quality of intelligence beyond programming or prediction that LLMs do not achieve. I call it living-thinking. It arises from our experimental, ludic way of learning and observing, evoked memorably in the ape sequences of Kubrick’s 1968 film. Huizinga argued for the priority of play nearly a century ago, reinforcing a post-humanist view:

Play is older than culture, for culture, however inadequately defined, always presupposes human society, and animals have not waited for man to teach them their playing4.

Living-thinking requires chaos, mistakes, friction, and a step outside the known. It is fueled by our animal nature and instinct. We often create most freely as children, in play, when we don’t “know” as much. The price is disobedience, ignorance, even conflict and violence, refusing what consensus has accepted, whether grounded in science, tradition, or memory. This can trigger the fear of rejection and real exclusion. We are terrified by ostracism. Though death- and living-thinking intersect, they’re often incompatible. We are rigorously conditioned to trust thinking and memory as a safe place; indeed, they seem safe, but they also bind us and poison us.

Here lies a paradox in our urge to belong, our quest for connection. When seeking connection, we face a choice: (a) recognize our singularity and serve the group from that place, risking rejection (living-thinking), or (b) recognize the group’s mold and do whatever it takes to fit, killing authenticity (death-thinking). This choice is often situational and gradient, not purely binary; it’s a living and perpetual choice.

Living-thinking is not a safe place. It locates meaning in relationships, in how we order words, our singular way of infusing spirit. It precedes language; a baby is born into living-thinking and is trained into death-thinking. Our injection of being through syntax, words, context, body language, and imperfections can resonate with others. We play language games5 to test the connection by evoking images, experiences, and emotions. LLMs predict, they don’t intend or experience. That’s why we often sense AI-generated content, not by counting em dashes, but by the absence of felt connection.

To summarize, death-thinking is identification with the group, fitting the mold through language and its symbols. It feels safe, and it is culture and society. Beyond it, and limitless, living-thinking rises from our ludic impulse to test, to seek authentic connection, to face the unknown with a child’s bravery, accepting ignorance, and risking everything. Precisely because AI can shoulder so much memory, synthesis, and reproduction, we gain a chance to keep more of our energy in living-thinking while delegating death-thinking to machines.

Why is AI an opportunity to set ourselves free from thought?

Two points make AI a tool for freedom. First, for the first time, we can play with results, scenarios, and prototypes through language, programming, mathematics, and symbols at unprecedented speed, learning or abandoning ideas via fast experimentation and simulation. Second, in this era of over-production of information, AI offers a way to lean into living-thinking while we outsource much of death-thinking to LLMs and other AI models.

The three examples below illustrate how I found the heavy work of AI taking care of some of the burden of handling death-thinking:

My first example comes from my own experience in Academia. Seventeen years ago, when I wrote my undergraduate thesis in History (2008), three tools extended my reach while cutting research time: (1) Excel and Access to structure a database from unstructured primary sources, (2) Word to write, no handwriting or typewritten copyedits, and (3) JSTOR, which kept me current with global scholarship. Still, most of my time was spent with notebooks and physical books; campus terminals sometimes were the only way to access JSTOR. I’m nostalgic about that tactile work, like film photography’s analog charm, but nostalgia isn’t function.

The non-creative load, what I call death-thinking, was heavy. Academic systems often reward demonstration of mastery over genuine novelty. If I’d had today’s AI, I could have redirected time from literature demonstration toward the unknown, communication, boundary pushing, and new approaches. AI could have handled much of the knowledge marshalling, while I focused on living-thinking as a trained historian, perhaps reaching a wider audience and freeing my imagination6.

I am not arguing that we should get rid of death-thinking. It’s crucial for consensus, especially in academia and science. I am arguing that we can engage AI to take over more of our death-thinking so that we can spend more time playing and less time carrying the load that machines can bear. Going back to my undergraduate thesis, would I do that today, my approach would be different: I would locate candidate passages by searching authoritative editions, verify page numbers and edition details, and store citations in a bibliographic manager for reproducibility. I would also cross-check with digital archives to ensure that quotations are not hallucinated or mistranscribed.

My second example comes from my experience as an entrepreneur. Seven years ago, I learned about Web3, digital assets, and crypto market cycles. High-quality asset assessments (Do Your Own Research DYOR) are laborious: requiring auditing technically smart contracts, blockchain block flows, tokenomics, social signals, use cases, and more. With rigor and patience, and with external experts, you spend weeks. Worth it, but costly.

A colleague and mentor shared a strong prompt for ChatGPT deep research mode, or Perplexity, to run DYOR. Judgment remains mine, so does triage of hallucinations. But that workflow saves time, letting me focus on risk management and evaluate assets I’d otherwise miss in a fast market.

In short, AI helps me externalize the fueling of decisions, gathering, scraping, computing, so I can spend more energy in living-thinking, responsiveness to context, and emerging opportunities and risks.

The third example comes from my writing practice. This is perhaps the clearest case of outsourcing death-thinking to AI. Approaching prose from living-thinking is often more productive and joyful. Knowledge is required, but much of its handling can be externalized. For this post, Huizinga’s thesis on play adds force. I recalled the core idea, then asked an AI assistant for relevant quotations and bibliographic details. The choice to include Huizinga, where to place him, and which quote to use remained mine. I didn’t need to hunt for the book or re-read chapters to extract passages; this was AI.

A second process worth outsourcing is copyediting. I keep drafts and prior essays in an AI workspace, and I spin up specialized threads to copyedit in my voice. English isn’t my first language, so I seek a balance, consensus, and clarity without sacrificing authenticity. AI helps me tune for an Anglophone audience without carrying all the death-thinking load myself. I verify quotations against primary editions, avoid unsourced block quotes, and reject paraphrases that do not fit my purpose or resonate with my voice.

My workflow, (1) draft the entire post in English, a living-thinking practice in my second language, (2) enrich with quotes and references that emerged while writing, (3) multiple self-edits aware of second-language limits, (4) full copyedit in my AI project, (5) selectively accept edits, (6) send to my human editor, and iterate. If my voice is authentic and the message clear, I’m satisfied. AI saves time and defends space for free, ludic writing.

Based on this thread of thought, I have a brief observation on the potential impact on organizations considering the reports of high failure rates in enterprise GenAI7: I have the hypothesis that there is a lot of confusion in identifying which processes are dead but functional, standardizable, and automatable, and which depend on living-thinking, creative, relational, and situational. What is dead can be automated; the unknown cannot. Hence, the push is to standardize and make processes predictable, as clearly explained by Olga Trögger in her recent post in Phi/Ai.

Overall, these examples show that current AI capabilities cannot replace living-thinking, that ability that precedes language and that arises from singularities and team synergies that resist capture by statistical models. Living-thinking has practical applicability but not a fixed methodology; it requires ownership of spirit, not just calculation. I hope to develop this beyond my current observations.

If this holds water, the focus of AI implementation is not technology or frameworks but people and their living potential. Can we teach people to cultivate and amplify living-thinking skills? Better, can we forget to operate from death-thinking as default?

Subscribe now

Some thoughts before closing

I believe we can outsource death-thinking to AI, so we can reclaim our nature and step into living-thinking more fully. Can machines reduce the energy we spend in survival mode so we can create? My answer, from experience, is yes, but it’s our choice, and it requires energy and attention. I am aware that even though this is a great opportunity, it is not necessarily the path we’ll choose. We must not yield ourselves to LLMs or surrender our voices to their outputs. Current AI is great in carrying on all processes that are characteristic of death-thinking, which is a huge burden on our shoulders and spirits: memory, knowledge, processing and referring information, using better grammar and syntax, frameworks, methodologies, and all the rest of it.

We find ourselves in others, through resonance, through suffering, and through experiment. We need to play with the tools at hand, including AI and LLMs. Applying our ludic nature, personally, academically, scientifically, organizationally, it is an immense benefit in our service, but it’s not an easy task since we fight our conditioning. The movement is a very spiritual and psychological impulse, so we can let go of what we were trained to do, of what we fear by experience, and step into the out-of-the-box feature we already have: living-thinking.

Is the fear of ostracism and our urge to belong by connecting through consensus something we can delegate fully to machines or AI? Not yet, we are emotionally and spiritually very slow creatures, and we won’t move as fast as we can envision or think. My intuition is telling me that enterprises, startups, venture capital, and corporations, in many ways, while allocating human beings as human resources or assets, are considering them as non-living functional creatures, serving predictable processes, often repetitive; all of these characteristics of the Industrial Revolution make replacing humans a corporate and capitalist dream. What is often forgotten in these environments is to understand how important the living-thinking nature is to keep alive all the dead repetitive processes, which are also interacting with living matter and present circumstances. We infuse the living-thinking into the rest of it; organizations won’t and never will be singular as they have predicted (‘solo-startups’), because at least they need to serve other living beings, which we often call the market. The irony of capitalism is dreaming of having a death matter to relate with, with predictable patterns, a huge consumer death-matter which behaves predictably, so corporations and sellers can project their profits and losses. That is a dystopian capitalist dream.

More than AI making incredibly productive companies, cutting costs and creating efficiencies, we’re facing the opportunity to create a new type of organization to relate with nature, and the market as a living matter. Those who are determined to let go of the burden of what is known, and let machines and AI carry it for us, can bet on ventures, spin-offs, or internal experimentations with a totally different quality and reason of existence, which can benefit all of us.

I find it fascinating that current AI, and most likely next generations of AI models, will probably serve better individual freedom in a deeper psychological and spiritual sense than capitalist dystopias. As I stated at the beginning, I know this is an optimistic approach, but I hope I uncover how real and plausible the opportunity is.

◆

This essay appears in The PhiAI Anthology, Year One — eighteen essays, twelve writers, printed and illustrated.

See and buy the Anthology I