The Importance of Ethical Applications in Society
Coded Bias¹, currently shown on Netflix, is a must-see film. The documentary shows how software applications, based on Artificial Intelligent (AI) algorithms, perpetuate inequality and discrimination. Facial recognition applications are failing to recognize women of color². Police surveillance systems flag innocent bystanders as wanted criminals³. These AI-applications have caused a lot of uproar in the AI-community, but also in general society. Do we really want computer systems that make decisions on our behalf without knowing if those decisions are fair?
“We are all grappling every day with algorithmic determinism. We are all being scored.” ⁴ (Amy Webb in Coded Bias)
There are many applications that influence the lives of both customers and citizens. So much so, that software applications decide whether you are eligible for loans, parole, or social benefits. Because these decisions can have severe personal repercussions, evaluating the ethics of application development becomes crucial. As these applications become more prominent, we need to establish some form of control. How can we assess that these decisions are fair and balanced? Well, the assessment starts with the design of the application itself.
It is the purpose and use of the software that causes the ethical problems. The questions we need to ask ourselves are: is the purpose of the application ethically sound? Are the consequences of the application acceptable for individuals, minorities, the poor, and society? Are our fundamental human rights, like freedom and justice, ensured when using these applications?
Ethics of software applications
Software applications are human products, as are any of the other artefacts we’ve built into modern society. The difference being that software products are far more complex than simple household utensils. The more intrusive and impactful a product is, the more intense the discussions need to be. This can make the ethical discussion complicated and volatile.
“Customers expect transparent and fair [algorithmic] interactions, and with clear accountability.” ⁵ (Capgemini)
We see this in the ongoing discussions about the negative impact of social media. To begin with, social platforms were praised for their ability to bring people together, but now the discussion has shifted focus to the power and influence of Big Tech on society and its values.
“While consumers may not be changing behavior en masse, they’re starting to recognize the scope of the challenge for themselves and for society”, says Eric Turkington from Fahrenheit212⁶. Turkington argues that “responsible use” is something that tech companies should actively be discussing, and even advocating.
Technology is not value neutral
There is a consensus among philosophers that technology isn’t value neutral. Until fifty years ago we believed that the user of a product embodied values, not the product itself. Like that old phrase, “guns don’t kill people, people do.” This is now being regarded as outdated. Products embody values because the use of a product makes it possible to realize values. But a product may also have negatives which violate those morals or ethics.
For example: a car may offer freedom to its driver, but it exhausts harmful gases into the environment. To some extent car drivers are responsible for the use of their cars and the emissions this use causes. But it is not the user, i.e., the driver or owner, that has decided the car would have those consequences. He might be aware of the consequences and stop using the car. But it is the designer of the vehicle that has accepted the positives and negatives as morally acceptable in the first place.
“Developers must think about to whom their product or service is relevant, how the rights of these people are protected, and what consequences they may face.” ⁷ (Anna-Mari Rusanen, University of Helsinki)
To put it bluntly: it is the designer that created the artefact in the first place. And he or she, as a moral agent, should have made a conscious decision whether normal use of his design would be morally acceptable. Moreover, the designer should take all imaginable consequences into account, including the possible misuse of the product or software application, for that matter. “In fact, design is an inherently ethical activity,” writes Jet Gispen, designer at A/BZ⁸.
↑ The new quality characteristics of intelligent machines (Source: Sogeti)
Ethics-by-design, privacy-by-design, and security-by-design
Over the last few decades, we have developed methods to incorporate security and privacy requirements into the application development processes. Basically, the starting point is that the project should take those requirements into account from the very beginning. Security and privacy considerations must be part of the vision or conceptualization of the application.
With ethics, the same way-of-working applies. You should incorporate ethics from the start of the design process. Some experts even say that ethics by design needs to encompass security and privacy, because both are ethical values too. Ethical consideration should also be part of the design of the application. You cannot apply ethics as an afterthought unless you’re prepared to shred all the work you have done and start all over again. Ethics is an inherent quality of an application, the attributes of which can be tested like any other feature⁹.
Ethical guidelines are not sufficient
But the question remains, where do we get the quality attributes for the ethics of applications? In the area of Artificial Intelligence, new guidelines, rules and regulations are drafted. These documents describe, in global terms, what features an AI-application must have in order to become ethical. But the rules have some limitations:
- They describe the requirements in broad terms, only the why and the what. How these rules can be translated into measurable requirements is up to the designer and its environment. At this moment, experts are warning that too loose of an interpretation of the rules does not meet the criteria of being truly ethical.
- The rules only reflect societal norms and values. Your own company ethics are not considered. So, designers have to also research the culture, purpose, and governance of the organization. All to obtain a full set of ethical requirements for application development.
It is the domain of business ethics that tries to create frameworks for ethics within organizations and businesses. And, in my opinion, ethics for applications will only work when these ethics comply with or are embedded into the company ethics.
“Beyond the ethical implications for businesses to become more purpose-driven, consumers are also increasingly evaluating companies based on their environmental, social, and corporate governance factors.” ¹⁰ (Rhea Cai, Capgemini)
Value sensitive design
Besides the required function of a system (the functional requirements), we also need to consider the quality of how well it performs that function (non-functional requirements), such as performance or ease of use. But there is a third type of requirement that is equally important to success: the moral requirements. Moral requirements are about how the design affects individuals and society: the moral impact.
Value sensitive design (VSD) has been developed to cater for all three kinds of requirements, especially the ethical ones. “Value sensitive design (VSD) is a theoretically grounded approach to the design of technology that accounts for human values in a principled and comprehensive manner. VSD originated to address design issues within the fields by emphasizing the ethical values of direct and indirect stakeholders”, as stated by Wikipedia¹¹.
Value Sensitive Design is one of the methods to systematically investigate the values of the stakeholders, both internally and externally to an organization. But how can we create a shared set of values, when every individual may have different value sets? How do we avoid those conflicts? VSD proposes dialogues to find common values and to build on those.
“Respecting human values in a design is not always straightforward. It may require creative thinking to find solutions that reduce potential negative impacts on stakeholders.” ¹² (Marlies van Steenbergen et al., Sogeti)
For large projects, like building a dam or a nuclear power plant, the size of the projects allows us to elaborate on values extensively. But for application development, most projects are too small to thoroughly evaluate values to be incorporated into the application from scratch every time. In my opinion, you should have a set of values for application development ready at hand.
By using this method, you can re-evaluate these values for each project according to the specific purpose of the application. It also allows the project team to become familiar with these values, and it becomes habitual to incorporate them into the application design.
So, the question to ask yourself is: how do I obtain a set of shared values within my organization? In 2020 Capgemini started a program of making companies aware of their purpose. This leads to a purpose aware culture, where ethics is an intrinsic part. But within purposeled companies, ethics are not just a compliance issue, limiting the business possibilities. On the contrary, ethics can create new opportunities. “Creating a purpose-driven culture across all levels of an organization begins with recognizing that social impact and profitability are far from mutually exclusive concepts.” writes Rhea Cai, consultant at Capgemini¹⁰.
By being aware of the purpose of your organization, it’s societal relevance and ethical position, an ethical framework that makes sense can be derived. And of course, societal norms and values are part of this framework because the organization is also part of society. This ethical framework should inform the guidelines and requirements for application development.
“We follow our Code of Business Ethics no matter where we operate and whatever the economic circumstances.” ¹³ (Capgemini)
Building software applications with ethical requirements and guidelines – next to the functional and nonfunctional requirements – creates better and more relevant applications for consumers, users, and society. And this is not only necessary from a purpose-led perspective. Capgemini is convinced that people, purpose and profit are intertwined in ways that allow for companies to become significant drivers of positive change, while also driving performance and growth.
Consumers are increasingly evaluating companies based on their environmental, social, and corporate governance factors, meaning the time for stricter ethical guidelines in application development is long overdue.
This article has been previously published on Capgemini for the report “Applications Unleashed” (23 September 2021).
Photo by © Capgemini
|¹||Coded Bias, an American documentary film directed by Shalini Kantayya; https://www.codedbias.com|
|²||Antoaneta Roussi: Resisting the rise of facial recognition; Nature; 18-11-2020; https://www.nature.com/articles/d41586-020-03188-2|
|³||Fieke Jansen & Sarah Chander: Why EU needs to be wary that AI will increase racial profiling; EU Observer; 19-4-2021; https://euobserver.com/opinion/151556|
|⁴||Coded Bias on Facebook: https://www.facebook.com/codedbiasmovie/posts/we-are-all-grappling-every-day-with-algorithmic-determinism-we-are-all-beingsco/235819614631873/|
|⁵||AI and the Ethical Conundrum; Capgemini; 2020; https://www.capgemini.com/research/ai-and-the-ethical-conundrum/|
|⁶||Eric Turkington: Should My Phone Have a Surgeon General’s Warning?; Fahrenheit212; 2021; https://www.fahrenheit-212.com/boiling-point/should-my-phone-have-a-surgeon-generals-warning|
|⁷||Sandra Theres Dony: The Ethics of Artificial Intelligence: Helsinki University; Techstory, 04-04-2021; https://techstory.in/the-ethics-of-artificial-intelligence-helsinki-university/|
|⁸||Jet Gispen: Ethics for designers; 2017; https://www.ethicsfordesigners.com|
|⁹||Paul Saunders: Testing of artificial intelligence – AI Quality Engineering Skills; Sogeti; 27-02-2020; https://www.sogeti.com/explore/blog/testing-of-artificial-intelligence/|
|¹⁰||Rhea Cai: Why purpose, profit and people are now inseparable; Capgemini; 18-10-2020; https://www.capgemini.com/2020/10/why-purpose-profit-and-people-are-now-inseparable/|
|¹¹||Value sensitive design; Wikipedia; last edited 23-03-2021; https://en.wikipedia.org/wiki/Value_sensitive_design|
|¹²||Marlies van Steenbergen et al.: Value sensitive architecture; Sogeti, 02-09-2020; https://labs.sogeti.com/whitepaper-value-sensitive-architecture/|
|¹³||Our Code of Business Ethics; Capgemini; https://www.capgemini.com/our-company/values-ethics/our-code-of-business-ethics/|