Moral Machine

With the rapid development of artificial intelligence, concerns have arisen about the ethics of the decisions that intelligent devices will have to make. With regard to autonomous cars, for example, one of the most important applications of AI, MIT researchers have tried to find an answer to a crucial question: if an autonomous car were to make an unavoidable accident, who should be sacrificed first?

This is the famous tramway dilemma, the original version of which can be formulated as follows: imagine the driver of an out of control tramway who has to choose his route between two possible tracks: five men work on the first track and one man is located on the other. The track taken by the tramway will automatically result in the death of the people on it. What should be done then? Take the second path to sacrifice one life to save five? But is this choice really moral?

MIT Media Lab researchers used this classic problem to test nine different situations: should the autonomous car spare humans rather than pets, passengers instead of pedestrians, women instead of men, young rather than old, people with high social status instead of those with low status, people who respect the law rather than criminals, people who are unhealthy instead of those in good health? Should the car deviate (act) or do nothing or try to save as many lives as possible?

To do this, in 2014 they designed an experiment they called Moral Machine, a platform that allowed them to gather public opinions on what to sacrifice or save as a priority if an autonomous car were to crash inevitably. But rather than offering individual comparisons, the experiment presented participants with various combinations. Four years after the platform went online, millions of people in 233 countries and territories have recorded 40 million moral decisions, making it one of the largest studies ever conducted on global moral preferences.

A new article published in the scientific journal Nature presents the analysis of these data. The study reveals that, overall, people agree that people should spare humans rather than animals or preserve children’s lives as a priority rather than adults’. Not surprisingly, people also believe that the least costly scenario in terms of number of lives should be preferred.

However, there are significant differences according to culture, economy and geographical location. The study shows, for example, that countries with more individualistic cultures are more likely to spare the young rather than the elderly – and France leads this group of countries. But in contrast, participants from collectivist cultures such as China and Japan are less inclined to spare young people at the expense of older ones. The researchers assume that this is probably because of the increased importance given to respect for older people in these countries. This is illustrated by the following figure comparing countries that test autonomous cars: a score closer to 1 means that respondents are more willing to save young people than older ones. And a score closer to -1 means that respondents are more supportive of preserving the lives of older people; 0 being the world average.

The results also showed that participants from individualistic cultures, such as the United Kingdom and the United States, were more willing to save more lives given all the other choices. France also leads this group, as shown in the figure below. The closer the score is to 1, the more respondents believe that more lives should be saved. The closer the score is to -1, the less emphasis respondents place on saving more lives; 0 being the global average.

Similarly, participants in poor countries with weaker institutions are more tolerant of pedestrians walking on the roadway when it comes to choosing between them and pedestrians crossing legally. And participants from countries with a high level of economic inequality show larger gaps between the treatment of individuals with high social status and those with low status.

Should pedestrians be spared rather than passengers? It was also an important question, the answer to which varies from country to country. Japan ranked as the first country where respondents prefer that pedestrian lives be spared at the expense of passenger lives. In contrast to Japan, there is China, where respondents made it clear that they prefer not to save passengers’ lives, and therefore risk pedestrian lives if necessary. Compared to the world average, France is on the side of those who want us to spare passengers’ lives rather than pedestrians’.

The study has interesting implications for countries currently testing autonomous cars, as these preferences could play a role in the design and regulation of these vehicles. Car manufacturers may, for example, find that Chinese consumers would find it easier to get into a car that protects them at the expense of pedestrians. But the authors of the study stressed that the results are not intended to dictate how countries should act. Indeed, in some cases, the authors felt that technologists and decision-makers should not take into account collective public opinion.

On the moral decision based on social status, for example, “it is worrisome that people have found, to a significant degree, that it is good to spare people with high social status at the expense of those with low status,” says Edmond Awad, one of the study’s authors. He therefore believes that the results should be used more by industry and government as a basis for understanding how the public would react to the ethics of different design and policy decisions.

Finally, it should be noted that this study is only relevant if the objective is to start discussing now what could be a problem in 10 or 20 years’ time. Indeed, autonomous cars are not currently as autonomous and efficient as they are to identify all the characteristics of objects or humans that enter their field of vision. Because distinguishing a young person from an elderly person or a criminal from a law-abiding person does not just require sophisticated sensors, but a real facial recognition system and all the infrastructure that goes with it.

Leave a Reply

Your email address will not be published. Required fields are marked *

13 − 9 =