The Switch in the Machine: The Trolley Problem and the Ethics of Autonomous Vehicles

The Hook: A Thought Experiment Climbs Into a Car

For more than fifty years it was a toy for philosophy seminars. A runaway tram is hurtling toward five people tied to the tracks. You stand at a switch. Throw the lever, and the tram diverts onto a side track — where a single person is tied down. Five dead, or one? Do you pull the lever?

This scenario, the trolley problem, was devised in 1967 by the British philosopher Philippa Foot and later expanded by Judith Jarvis Thomson into a whole family of variants. For decades it was exactly what it appeared to be: a thought experiment. A deliberately artificial construct, built to probe our moral intuitions — a test tube for ethics, not a blueprint for reality. No one ever really stands at that switch.

And then, around 2015, something remarkable happened. The trolley problem climbed out of the seminar room and into a car. For the first time in history, humanity was building machines that could decide over life and death on their own, in fractions of a second: self-driving cars. An autonomous vehicle whose brakes fail while barreling toward a group of pedestrians could, in principle, be programmed to swerve — killing its own occupant. Or not. For the first time, someone must pour the answer to the trolley problem into program code in advance, coolly, at a desk, long before the situation ever arises.

With that, an abstract philosophical finger exercise turned into an urgent question of engineering, law, and society. Who should live, who should die, when a crash is unavoidable? And, far more importantly: who decides that? The programmer? The carmaker? The legislator? The majority of the population? The owner, who after all paid for the thing?

This article takes you along the full length of the debate: from the philosophical origin of the trolley problem, through two groundbreaking empirical studies — the social-dilemma trap and the vast Moral Machine experiment with its forty million moral judgments — to the German Ethics Commission, which gave a surprisingly clear but also contested answer. At the end stands an insight that reaches far beyond cars: it concerns every machine to which we entrust moral decisions.

Part 1: The Philosophical Foundation

Foot's Switch and Thomson's Bridge

When Philippa Foot formulated the trolley problem in 1967, she was not really interested in trams. She wanted to illuminate a moral puzzle: why does it seem permissible, in some cases, to sacrifice one person to save five — yet deeply wrong in others? To sharpen the point, Judith Jarvis Thomson set against the switch case a notorious variant, the footbridge case: again the tram races toward five people. This time you stand on a bridge over the tracks, beside a very heavy stranger. If you push him off, his body will stop the tram and save the five. The arithmetic is identical — one dead, five saved. And yet the overwhelming majority of people feel: throwing the switch is defensible; pushing the stranger is murder.

This asymmetry is the real treasure of the trolley problem. It shows that our morality does not run on pure arithmetic. A strict utilitarian would decide the same in both cases: save the greater number, whatever the cost — consequences are all that count. An adherent of the deontological (duty-based) tradition of Kant, by contrast, sees a categorical difference: to actively use a human being as a mere means — to abuse him as a brake block — violates his dignity in a way that no outcome, however good, can outweigh. The human being is an end in himself, never a mere tool.

Between these two poles — reckoning with outcomes and insisting on inviolable principles — the entire field of normative ethics stretches out. The trolley problem is so powerful precisely because it captures this tension in a single, cruelly simple image.

Why the Thought Experiment Suddenly Became Real

The decisive point for our topic is the difference between a human at the switch and a machine. The human decides in the moment, in shock, with incomplete information, from the gut. We rarely judge that person harshly, however they act — the situation is superhuman. But a machine does not decide in the moment. Its "decision" was fixed months earlier by engineers, at leisure, in full awareness of the stakes, tested, documented, then rolled out thousands of times over. That transforms a tragic split-second reaction into a deliberate policy.

That is exactly what makes machine ethics so unsettling. We force ourselves to answer, in advance and explicitly, what humans were until now permitted to leave to chance and instinct. There is no more looking away. The code must do something, and whatever it does is a conscious moral commitment.

Part 2: The Social-Dilemma Trap

What People Want — and What They Would Buy

The first major empirical investigation of this question came from the behavioral scientists Jean-François Bonnefon, Azim Shariff, and Iyad Rahwan. Their study "The Social Dilemma of Autonomous Vehicles" appeared in 2016 in the journal Science and exposed a nerve that almost no one had expected.

Across several online experiments, the researchers surveyed thousands of people about exactly those crash scenarios. The result split into two contradictory halves. On the one hand: when people were asked, abstractly, how an autonomous car should behave, the large majority answered in a utilitarian way. A car should minimize the total number of victims — if necessary sacrificing its own occupant to save several pedestrians. That seems morally noble and reasonable.

On the other hand, there was an uncomfortable second question: which car would you buy for yourself? And here the attitude flipped into its opposite. The very same people preferred, for themselves, a car that protects its occupants at all costs — that is, themselves and their family. Utilitarianism should apply, but please to everyone else.

The Mechanism of the Dilemma

That is the classic pattern of a social dilemma: what would be best for everyone collectively (utilitarian cars that cause the fewest deaths overall) collides with what is rational for the individual (a car that protects me). It resembles the free-rider problem or the tragedy of the commons: everyone benefits when the others behave self-sacrificingly, yet no one wants to be the sacrifice.

Bonnefon and colleagues drew from this an almost paradoxical policy conclusion. The respondents additionally rejected the idea that the state should mandate utilitarian programming — and said they would be even less willing to buy such a mandated car. The twist: if a legislator forced utilitarian algorithms, that could delay the adoption of self-driving cars, because no one wants to buy them. And since autonomous vehicles are, overall, presumably far safer than human drivers, such regulation could cost more lives on balance than it saves — through prevented or slowed adoption of the safer technology. The morally "cleanest" rule could in reality be the deadliest.

That is a deep lesson: in machine ethics, the good cannot be legislated in isolation. Every rule feeds back into human behavior, and those feedback effects can turn the original moral intention into its opposite.

Part 3: The Moral Machine Experiment

The Largest Survey of Morality Ever Conducted

If people are so divided — can one at least discover which rules the world's population prefers on the whole? That is exactly what the most ambitious project in this debate attempted: the Moral Machine experiment, led by Edmond Awad, Iyad Rahwan, and colleagues at the MIT Media Lab, published in 2018 in Nature.

The idea was as simple as it was ingenious. The researchers built a playful website where every visitor was shown crash scenarios. An autonomous car with failing brakes; one had to decide whom it should spare. Five elderly pedestrians or two young occupants? A pregnant woman or two doctors? A dog and three children or five adults? Pedestrians crossing on red or those waiting lawfully? The site went viral around the globe.

The result shattered every dimension of previous moral research: around 40 million individual decisions from millions of people in 233 countries and territories, in ten languages. It was, by a vast margin, the largest empirical study of human morality ever conducted.

The Three Strongest Global Preferences

Across all cultures, three especially strong, nearly universal preferences crystallized out. First: spare humans over animals. Second: spare more lives over fewer — so sheer numbers do count after all. Third: spare younger people over older; a child was saved far more often than an old person.

Beyond that, weaker but clear tendencies appeared: people of higher social status were spared more readily than those of lower status; the physically fit more than the overweight; those crossing lawfully more than those crossing on red; and, by a slight margin, women more than men. The table below summarizes the strongest patterns.

Note how uncomfortable this list is to read. Some preferences — humans over animals, many over few — seem defensible. Others — the fit over the fat, the rich over the poor, the young over the old — would be a scandal as the official rule of a carmaker. This is one reason why mere majority opinion is a dangerous foundation for ethics. But more on that shortly.

Three Moral Worlds

Perhaps the most fascinating finding was not the global average opinion but its fragmentation. When the researchers grouped countries by the similarity of their moral profiles, they fell into three large clusters, running roughly along geographic-cultural lines.

The Western cluster comprises broadly the Protestant, Catholic, and Orthodox countries of Europe and North America. The Eastern cluster unites countries strongly shaped by Confucian and Islamic traditions, including Japan, Taiwan, Indonesia, Saudi Arabia. The Southern cluster comprises Latin America along with countries under strong French cultural influence.

And these clusters differed systematically. In the Eastern cluster, the preference for the young over the old was markedly weaker — a preference plausibly linked to the greater respect for age in Confucian-influenced societies. The Southern cluster showed an especially strong inclination to spare women and people of higher status. The Western cluster, in turn, leaned toward inaction — not actively wrenching the car's course in another direction — whereas the other clusters favored active intervention.

Statistically, these differences could be correlated with deep cultural and institutional traits: individualistic societies (in the sense of cultural psychology) leaned more strongly toward saving more lives at any cost — an expression of the equal value of each individual. In countries with greater economic inequality, social status was weighted more heavily; in countries with more robust state institutions and rule of law, lawful pedestrians were preferred far more strongly over rule-breakers. Morality, it seems, bears the fingerprint of the society that produces it.

Part 4: Why the Machine Should Not Follow the Survey

The Limits of the Experiment

Impressive as the Moral Machine experiment is, one must be careful about what it actually shows and what it does not. The criticism of it is substantial and important.

First, the scenarios are unrealistically idealized. In the vignettes it is absolutely certain who will die and who will survive, and the identity and attributes of everyone involved (age, fitness, status) are known to the car. In reality a vehicle knows none of this. It sees no "doctor" and no "homeless person," no "pregnant woman"; it sees uncertain probability clouds of sensor data. Real crash decisions are not certain death sentences but distributions of risk under massive uncertainty.

Second, the sample was self-selected — predominantly young, tech-savvy, male internet users who voluntarily played a viral online game. That is not a representative image of humanity.

Third, and philosophically most important: the experiment measures what people prefer — it is purely descriptive. But from what people in fact hold to be right, it does not follow what actually is right. David Hume called this gap the is-ought problem: from a description of what is (the majority prefers X), one can never logically derive what ought to be. Morality by referendum would be a category error.

The Prohibition on Trading Lives Against One Another

And indeed the authors of the Moral Machine experiment never claimed otherwise. They explicitly understood their work as a survey of public expectations — as input for a societal debate, not its outcome. For many of the measured preferences, once turned into explicit rules, would be morally unacceptable. A car programmed to deliberately kill the fat rather than the thin, the poor rather than the rich, the old rather than the young, would institutionalize discrimination by personal traits that every constitutional order forbids. The principle of equality — the equal dignity of every human being — is exactly not up for majority vote.

Here we hit the central tension of this whole field: the empirical question ("What do people want?") and the normative question ("What is right?") must not be confused. The machine should not execute the survey.

Part 5: The German Answer — 20 Rules

The Ethics Commission of 2017

While American researchers were surveying the world's morality, Germany took another path. In 2016 the Federal Ministry of Transport convened an Ethics Commission on Automated and Connected Driving, chaired by former Federal Constitutional Court judge Udo Di Fabio. In 2017 it presented its report with 20 ethical rules — the world's first state-commissioned ethical guideline for autonomous driving. It became the basis for later legislation and ultimately fed into the German Autonomous Driving Act of 2021.

The Commission gave a remarkably clear answer that directly contradicts the crowd-sourced opinion picture on decisive points. The most important principles can be summarized as follows.

The protection of human life has top priority — above property and animal welfare. When a crash is unavoidable, the technology must, in case of doubt, accept property damage in order to protect people.

Decisive, however, is Rule 9: in unavoidable accident situations, any qualification of people by personal characteristics — age, sex, physical or mental constitution — is strictly prohibited. Exactly the characteristics that the Moral Machine experiment identified as humanity's preferences are here declared forbidden by German ethics. No car may be programmed to favor the child over the elder, the fit over the sick. The equal dignity of every human being is not to be offset against anything.

Equally forbidden is the weighing of victims against one another: it is impermissible to actively sacrifice an uninvolved person to save others — the footbridge case of the road. A general programming to reduce the number of personal injuries, by contrast, can be required, so long as it is not tied to the targeted selection of particular victims. The distinction is subtle but fundamentally deontological: minimizing the number of injured in general is permitted; using a concrete human being as a means is not.

The Gulf Between Survey and Principle

The German position thus marks the sharpest conceivable rejection of morality by majority. Where the Moral Machine experiment asks "Whom would you sacrifice?", the Ethics Commission answers: "The machine must not even ask that question." Human dignity (Article 1 of the German Basic Law) is the fixed point behind which no preference, however popular, may reach.

One recognizes here the deontological tradition of Kant in pure form, cast in administrative language: the human being may never be treated as a mere means, his dignity never weighed against a calculus. The German Commission made a deliberate decision against the utilitarianism of case-by-case number-crunching and for an unconditional prohibition of discrimination.

Whether this holds up in practice in every conceivable extreme situation remains contested. Critics object that the rules can become contradictory in genuine edge cases and that they only partly address the real problem — the distribution of risk. But as a statement of principle, the message is unmistakable.

Part 6: From the Trolley Problem to the Ethics of Risk

Why the Wrong Question Shaped the Debate

A growing faction of researchers argues that the whole trolley fixation is a wrong turn — a media-friendly but practically misleading image. A team around Maximilian Geisslinger and Markus Lienkamp at the Technical University of Munich has proposed moving from the ethics of the dilemma to the ethics of risk.

Their argument: the pure trolley case — two clearly defined groups, certain death on both sides, no third option — practically never occurs in reality. It is so rare that it is almost absurd to hang the entire ethics of autonomous driving on it. What an autonomous car does constantly is something else: it continuously distributes risk. How much clearance does it keep from the cyclist? How fast does it drive past the school? How heavily does it weigh the small residual risk to its own occupants against the risk to other road users?

The real ethical work, then, lies not in the spectacular death dilemma but in the unspectacular, permanent distribution of risk — in millions of everyday micro-decisions, long before any emergency arises. A well-designed system solves the trolley problem not by deciding whom it kills, but by ensuring, through anticipatory risk minimization, that the trolley situation never arises in the first place.

Responsibility and Transparency

This brings other values to the foreground: how is the risk distributed fairly? Is the distribution transparent and traceable? Who bears responsibility when something goes wrong anyway — manufacturer, owner, programmer, regulator? These questions are less dramatic than "child or elder?", but they are the ones that truly matter in practice. They tie the ethics of autonomous vehicles directly to the large regulatory frameworks now emerging — such as the requirements for high-risk AI systems under European law.

The Central Takeaway

The real lesson of the debate over autonomous vehicles is one about ourselves, not about cars. As soon as we build a machine that acts on programming set in advance, it forces us to make our values explicit — to write down what we otherwise leave to instinct, chance, and forgetting. And in that moment of explication, it becomes clear how divided, how contradictory, and how culturally shaped our morality really is. People demand utilitarian cars for others and self-protecting ones for themselves; they favor the young, the fit, the rich when asked anonymously — and forbid exactly that when, as legislators, they write down their principles.

For practice, three robust principles can be held fast. First: separate the descriptive from the normative. What people prefer (the 40 million judgments) is valuable input, but never itself the answer to what is right. Second: move the ethics upstream. The decisive moral work lies not in the rare death dilemma but in the everyday, transparent, and fair distribution of risk — in the design, long before the crash. Third: keep dignity non-negotiable. The strongest protection against a discriminatory machine is the principle that certain traits — age, sex, status — must simply remain invisible to it.

These principles reach far beyond the car. Every AI system that helps decide on loans, medical priorities, parole, or job applications faces the same basic question: whose values are programmed in, who legitimizes them, and how do we prevent the majority opinion from overruling the dignity of the minority? The autonomous car is only the first, most tangible case of a question that will accompany us with every serious machine.

A Closing Question for Reflection

Imagine you were allowed to set the one basic rule by which all autonomous cars worldwide act in an unavoidable emergency: absolute occupant protection, strict number-minimization, or an unconditional prohibition of discrimination with no offsetting of victims. Which rule would you choose — and would you be willing to sit in a car that, following exactly that rule, would if necessary sacrifice you to stay true to the principle?

Cross-References in the Vault

• The Pyramid of Risk: How the EU AI Act Tames Artificial Intelligence – and Why It Concerns the Whole World – the regulatory framework under which high-risk AI systems (including in transport) will have to be assessed.
• The Two Boxes: Newcomb's Paradox and the Battle Over Rational Choice – the roots of decision theory and the conflict between competing notions of rationality.
• The Chinese Room: Searle and the Question of Whether Machines Can Understand – whether a machine that acts morally also morally understands.
• What Mary Did Not Know: The Knowledge Argument and the Riddle of Consciousness – the question of experience and subjective value that underlies any machine ethics.
• The Ghost in the Machine: How to Read a Neural Network From the Inside – how one can even trace by what "values" a neural network actually decides.

Sources

• Bonnefon, J.-F., Shariff, A., & Rahwan, I. (2016): The social dilemma of autonomous vehicles. Science 352(6293), 1573–1576. https://www.science.org/doi/10.1126/science.aaf2654
• Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.-F., & Rahwan, I. (2018): The Moral Machine experiment. Nature 563, 59–64. https://www.nature.com/articles/s41586-018-0637-6
• MIT open-access version of the Moral Machine paper. https://dspace.mit.edu/bitstreams/750c4524-46a2-4013-916a-c8131d726b56/download
• Ethics Commission on Automated and Connected Driving (2017), German Federal Ministry of Transport and Digital Infrastructure; summary e.g. at the World Economic Forum: https://www.weforum.org/agenda/2017/08/germany-has-developed-a-set-of-ethical-guidelines-for-self-driving-cars
• Weber, N. et al. (2021): How to Regulate Moral Dilemmas Involving Self-Driving Cars: The 2021 German Act on Autonomous Driving, the Trolley Problem, and the Search for a Role Model. German Law Journal. https://www.cambridge.org/core/journals/german-law-journal/article/how-to-regulate-moral-dilemmas-involving-selfdriving-cars-the-2021-german-act-on-autonomous-driving-the-trolley-problem-and-the-search-for-a-role-model/A57B334A189F72B16A65150EE38B0008
• Geisslinger, M., Poszler, F., Lienkamp, M. et al.: Autonomous Driving Ethics: from Trolley Problem to Ethics of Risk. https://www.semanticscholar.org/paper/eb8c53fbe9a339c96d418dd465878d4681af2f38

Note on scientific grounding: The empirical findings cited (Bonnefon et al. 2016; Awad et al. 2018) are peer-reviewed and widely cited. The cultural correlations are associations, not causal proofs. Where this article evaluates beyond the study findings — for instance in judging that morality by majority is a category error — this is a reasoned philosophical position, not an empirical fact.

← All articles