AI Safety Starts with Metaphysics: A Podcast Debate

A debate testing whether materialism, dualism, or panpsychism reshape AI risk forecasts and the policies we write — featuring Katalina Hernández, Jáchym Fibír, and Haihao Liu.

Phi/AI was born out of the desire of having and sharing deeper conversations about the meaning of AI and the human consequences of its development and deployment.

We have gathered a group of talented researchers and individual all over the world who have shared their ideas, thinking and perspectives using this collective as a platform. This sharing has happened mostly in the form of well-written and thought-provoking articles. This is fully aligned with our core because we are a text-first forum.

But this leaves the conversations and lively discussions to happen somewhere else. Some in the comments here, but many more via our own chats. Hence we decided to experiment with a different format: a conversation where we bring the discussion straight to you, our readers in the form of a podcast: the Phi/AI Dialogues.

Please, do share your reactions. We read every single one and this is what keeps us going.

Without further ado, I invite you to listen: to “Alternative perspectives on AI Safety” — recorded September¹* under this link.

Guests:

• Katalina Hernández — is a Legal & AI governance specialist focused on the intersection of artificial intelligence, autonomy, and digital rights.

• Jáchym Fíbir — a psychedelic researcher and entrepreneur exploring neglected frontiers (like machine consciousness, sentience, or biological alignment)

• Haihao Liu — with degrees in material sciencie and mathematics, he is involved with the AI Safety field since 2023, long enough to become a vehement critic, among other things because he doesn’t believe LLMs will lead us to AGI

The conversation centers around how different metaphysical beliefs shape AI safety thinking and policy prescriptions. They contrast materialist/physicalist assumptions (which often predict high existential risk) with alternative views (dualism, panpsychism), then weigh trade‑offs between near‑term harms (privacy, mental health, environmental impact) and long‑term existential risks.

The conversation closes by arguing for multidisciplinary collaboration (law, neuroscience, education, philosophy, and engineering) to improve definitions, governance, and assessment.

6 key takeaways

• Worldviews change risk forecasts. Jáchym’s core point: predictions about AI’s dangers depend on metaphysical assumptions: e.g., if you assume a purely materialist universe, powerful optimizers (= AI) naturally create catastrophic risk; alternative metaphysical views imply different policy responses.

• Consciousness & AGI remain unsettled. Panel consensus: current frontier models are not sentient (“hard no”), but there are two competing conceptual routes (emergent consciousness from complex physical systems vs consciousness as requiring non-physical or panpsychist elements) and each has different implications for ethics and regulation.

• Timelines and definitions matter. Disagreement on AGI timelines (near vs. distant) is partly definitional. We need better operational definitions and benchmarks for “general” capabilities before laws or risk models can reliably target AGI/ASI.

• Regulation must be practical, not only aspirational. Compute-based thresholds (e.g., in the EU AI Act) are convenient because they’re measurable, but they’re imperfect and reactive. Regulators should talk to cutting-edge researchers and handle alternative architectures and substrates that would break compute-based rules.

• Don’t neglect present, concrete harms. Social harms (mental-health impacts, misinformation, sycophancy), distributional harms, and environmental costs (energy/water/data centers) are real and under-addressed. Even if existential risk is important, real people suffer now and this needs to be addressed.

• Multidisciplinary collaboration is essential. The panel calls for lawyers, educators, philosophers, neuroscientists and technologists to work together: e.g., borrow assessment ideas from education, investigate alternative compute substrates (analog chips) for efficiency, and create better capability evaluations.

1. It pains me to share that the conversation already took place in September. It is my responsibility that we didn’t share this earlier. We are still here, we learned what we needed to learn and next time we will be better.