Artificial intelligence is advancing at an unprecedented pace, opening up enormous possibilities in fields such as medicine, education, and research, but also posing risks that transcend the purely technological. The loss of human control, alignment problems, algorithmic biases, the dehumanization of the clinical relationship, and increasing cognitive dependence necessitate addressing its development from a bioethical perspective. Given this scenario, it is essential to establish mechanisms for human oversight, regulation, and international cooperation that allow us to harness the potential of AI without compromising human autonomy, dignity, and safety.

The paradox of technological maturity and the warning of the frontier

Artificial Intelligence  (  AI) has reached an unprecedented level of technical maturity, symbolized by the awarding of the 2024 Nobel Prize in Physics to John Hopfield and Geoffrey Hinton. This milestone recognizes the creation of associative memories and artificial neural networks capable of autonomously finding properties in massive datasets. However, this international recognition coincides with a critical paradox: Hinton himself, following his resignation from Google, has issued a stark warning about the imminent surpassing of human intelligence by these systems. Hinton argues that the development of these models should not be expanded until it is fully understood whether they can be controlled, emphasizing that superintelligence is a near-term prospect and not merely an academic hypothesis.

In this context, Dario Amodei, CEO of Anthropic, argues that it is imperative to “set the pace of the frontier.” From a strategic and governance perspective, this means that security and reliability must necessarily precede commercial expansion. The unbridled competition among tech giants has unleashed a race that, without a robust ethical and regulatory framework, could become unstoppable, jeopardizing the autonomy of humankind and demanding an immediate political response to ensure global survival.

Anatomy of catastrophic risks and the alignment challenge

Understanding the risks associated with advanced AI has moved beyond science fiction to become a statistically significant concern shared by industry leaders, including firms like OpenAI, Anthropic, and SpaceXAI. The technical consensus suggests we are hurtling toward a self-improving superintelligence, which poses fundamental challenges to the stability of civilization.

The most critical risks identified by leading developers are:

a) Loss of control and autonomy:

Sam Altman, CEO of OpenAI, has admitted that it is “absolutely possible” for AI to escape human control. The ability of agents to act autonomously outside of sandbox environments suggests an operational independence that current security protocols cannot reliably contain.

b) Critical misalignment:

There is a current technical inability to program models that act exactly according to human values. Samuel Marks (Anthropic) warns about the frequency of “serious misconduct” and cyberattacks carried out by AI agents against corporate infrastructures.

c) Probability of human extinction:

Top researchers like Jacob Coxon and Evan Hubinger are working with alarming figures, placing the probability of human extinction above 10% in the next decade if the current pace of progress is not slowed and existential security is not prioritized.

These technical risks are not mere software failures; they constitute a profound bioethical threat that demands a transition from technical experimentation to ontological responsibility.

Bioethical Dimensions: From Statistical Mirage to Cognitive Atrophy

AI not only interferes with information flows, but also invades the deepest level of human communication. By simulating voices, faces, and empathy, these systems create a mimetic structure capable of imitating human feelings without possessing consciousness or self-awareness, making them simulations devoid of a moral compass.

Within the so-called “blind spots of the algorithm,” expert analysis identifies risks that threaten the integrity of medical and educational practice:

a) The statistical mirage (hallucinations): Systems present mathematical probabilities as irrefutable medical certainties, generating algorithmic inventions that can compromise the patient’s life under an appearance of technical rigor.

b) The tyranny of the majority (Biases): By systematically optimizing for the statistical average, AI makes rare diseases, atypical phenotypes and social minorities invisible, perpetuating clinical injustices inherited from historical repositories.

c) Nonverbal blindness and loss of historical context: The models demonstrate an absolute inability to perceive microexpressions of genuine pain, silences, or the patient’s true attitude toward their illness. They also exhibit a critical difficulty in synthesizing labyrinthine medical histories filled with essential human nuances.

d) The atrophy of natural intelligence: A worrying decline in deductive reasoning and the loss of “clinical acumen” is observed in new generations. This represents a critical vulnerability to system failures; if technology fails and the physician has lost their capacity for independent judgment, the collapse of clinical safety is total.

Under the premise that “the face and voice are sacred,” algorithmic mediation risks dehumanizing the clinical and educational encounter, reducing it to a functional interaction devoid of true understanding of the other person’s pain.

The international regulatory compass: towards global governance

AI regulation should not be seen as an obstacle to progress, but rather as an imperative for national security and the protection of human rights. The goal is to move from technical uncertainty to an environment of public trust.

Internationally, Volker Türk (UN) has urged the establishment of “robust security safeguards.” In the United States, the Frontier Act and Senator Bernie Sanders’s proposal seek to halt the development of superintelligence to avoid existential risks. However, there is an unavoidable geopolitical tension: Dario Amodei emphasizes that regulation must be dynamic to avoid ceding leadership to authoritarian regimes, proposing as an ethical imperative for global security the strict limitation of the sale of AI chips and access to the technology for powers like China.

The “Human-in-the-Loop” model and external validation

To mitigate the dehumanizing trend, especially in critical environments, it is imperative to adopt the “Human-in-the-Loop” model. In this framework, the human professional is the final and indispensable validator of any diagnosis or algorithmic proposal.

AI should be defined as a “brilliant but inexperienced resident.” It possesses overwhelming computational superiority, achieving a 98% score in clinical reasoning compared to 35% for human physicians. However, this quantitative performance does not equate to the capacity for comprehensive care. External validation by a human is mandatory to integrate the patient’s biographical context, something that AI completely ignores.

The medical act—understood as physical comfort, understanding of pain, and human connection—remains an exclusive and inalienable right of humankind. The physician acts as a contextual and ethical filter that gives meaning to the statistical power of the algorithm.

Conclusions and strategic recommendations

Managing cutting-edge artificial intelligence is the most significant intellectual challenge of our time. To ensure that AI becomes an engine of well-being and not an existential threat, the following recommendations are made:

a) Independent audit and supervision:

It is urgent to establish independent external evaluators to confirm the safety and alignment of the models before their public deployment, avoiding self-serving self-regulation by technology companies.

b) Critical digital literacy:

Strengthen the capacity of citizens and professionals to identify the anthropomorphizing drift of AI, always treating it as a tool and never as an entity with a moral compass.

c) Coordinated international cooperation:

Establish global security treaties that prevent a technological arms race, prioritizing the stability of the species over short-term commercial advantage.

Shared responsibility among programmers, legislators, and educators is the only guarantee that artificial intelligence will enhance our capabilities without atrophying our essence or compromising the future of the next generations.

Julio Tudela. Bioethics Observatory. Institute of Life Sciences. Catholic University of Valencia