It’s beyond time to put ethics at the core of UX Research
Part 1 of 2: How do we start incorporating ethical research practices in industry?
Coauthor: Lexi Neigel, PhD
UX researchers are in many ways customer success strategists. We are the voice of the customer and are hired to speak on the user’s behalf when critical engineering and design decisions are being made. That said, a researcher’s work is conducted in the context of increasing profit and reducing user churn. In practice, that can sometimes happen at the expense of the users’ experience.
This dichotomy of our responsibilities as researchers can create an ethical conundrum. On the one end of the spectrum, we are first and foremost the users’ advocate. But our success is anchored to the other end of the spectrum, increasing our employers’ bottom line. In prioritizing the latter, we secure our career and potentially move up the corporate hierarchy. Championing the former, though, a researcher or scientist may face forced removal, blatant firing, and significant backlash.
These two ends of the spectrum may seem like they paint an extreme example of the ethical conflicts researchers and scientists face when working in industry, but as more continue to speak up about the ethics of their work and how it is negatively impacting the end-users, we may find that we are only seeing the tip of the iceberg.
For a lot of researchers, navigating this position is breaking into unknown territory. Few have exposure to applied research ethics in formal or on-the-job training. In some cases, researchers strive for ethical research practices, but they work in companies that prioritize unethical products or experiences. How do we balance the decision to increase company profits while being ethically mindful of the users? The secondary issue is how do we make ethics work with the speed at which corporate tech acts and makes decisions?
This is the first article in a series about these two pressing issues.
To start, we find it imperative to take a trauma-informed stance when discussing this topic. Ethics in human research stems from a long and violent past. Having a working knowledge of this general history and the atrocities that occurred provides an important context for why ethical violations that happen within user research today need to be met with strict rebuke. We also discuss what we can do about this at the intersection of user research in technology.
An Abridged History of Ethical Violations in UX Research
When teaching formal research methods courses in higher education, we reference many historical moments of trauma in research and ethics. These examples lay the groundwork for why research involving human subjects needs to follow established ethical practices in both basic and applied research, such as UX research.
It may seem like outright abuse in human research is curtailed in modern times (more on that in a moment) and that the examples highlighted in formal education are ancient history — ethical violations persist. This is in addition to significant, systemic racial inequalities and disparities in human subjects research.
The abuse of communities and people has simply taken on a new form in corporate tech.
For instance, one search engine returns over 120 links to disinformation and misinformation, while another returns less than 15. Differences like this, in how we access and are exposed to information, makes it difficult for users to determine fact from fiction.
One of the largest social media networks used today has been in and out of the news for years about engaging in unethical business practices, unethical experimentation on users, and purposeful manipulation of negative content to trigger addictive behaviors; all to increase company revenue at the expense of the users.
In the case of one dating site, a large-scale dataset that was not anonymized fell into the public, searchable domain back in 2016. Millions of users’ private information was made public without consent. When researchers were asked why they did not remove identifiable information, they responded with “it’s already public.” This is not the first nor last time this excuse was used.
As with many areas of life, Silence ≠ Consent.
In just the last 5 years:
- 2017: Researchers did not use a trauma-informed approach when creating an AI-generated “gaydar” causing additional harm to the LGBTQ+ communities
- 2017: Major social media network exploits consumer behaviors using dopamine-driven feedback loops to drive user engagement
- 2018: digital consultants to a major political campaign misused the data of millions of social media users without their knowledge
- 2018: Deceitful AI used in robocalls purposefully designed to mimic human speech patterns so users would not know there were talking to a computer instead of a person
- 2019: Digital trespassing, falsifying data, and misrepresentation of researchers in online medical community without informed consent
- 2019: Superhuman shares geographic locations of users without their knowledge
- 2020: Immigrant rights groups have criticized Silicon Valley for working with the Department of Homeland Security, which includes ICE and U.S. Customs and Border Protection
- 2021: Ivermectin (an anti-parasitic commonly used in livestock) tested on Arkansas US prisoners without informed consent due to a widespread disinformation campaign on major social media and news aggregators
- 2020: Infinite scroll creator goes on apology tour to bring awareness to the psychological and societal ills of addictive technology
- 2021: Major search engine fires world-renowned AI ethics researcher for identifying flaws in novel AI natural language processing and the environmental cost of large-scale computing systems used to power natural language processing programs
And this list is by no means exhaustive!
All of this begs the question — if we’re proponents and champions for the end users as researchers, is it our responsibility to shift corporate culture to better treat and protect them?
Yes, most definitely yes.
Improving ethics within major technology corporations is a team sport, and needs broader societal support for change to occur. It may feel like we are fighting an uphill battle, and change is not happening fast enough. People far above us in the hierarchy need to be as invested in ethical business and research practices as we are.
So what can we do?
Applying Ethical Standards in Research for Corporate Tech
Listening to Frances Haugen testify before the Senate Commerce Committee about the major ethical violations of a large social media network in October 2021 highlights why user research must undergo an ethical overhaul and refresh. There is much to be done.
Ethics can no longer be viewed as a ‘nice-to-have’ or a side hustle.
Instead, we challenge corporations to take immediate, actionable steps to better protect users, their rights, their data, and their being:
- Establish and formalize ethics reviews and committees for user research (we discuss this further in Part 2 of this series)
- Implement ethics review processes for user research
- Create a safe environment for employees to speak up without fear of retribution
- Protect and normalize whistleblowing
- Speak up, and encourage others to speak up, when abuse and ethical violations occur
- Form unions to protect researchers — FTEs or contract employees (the latter of which have little to any protection and yet comprise the bulk of corporate tech)
- Establish and work from ethical design and engineering principles
- Establish accessible, inclusive ethics training and education for employees
- Review and understand the importance of the ethical pillars laid out in the Belmont Report
- Integrate ethics into how we work today (In Part 3 of this series, we dive deep into how ethics can be integrated into Agile frameworks.)
- Utilize external review boards to evaluate internal best practices as well as societal impact of products offered by the company; action on the feedback
Furthermore, researchers and their partners can begin auditing themselves. Answers to the following questions (centered around the Belmont Report) should be documented and retained. Future audits by internal and external review boards will need this documentation when considering to what extent companies are engaging in ethical best practices:
- Respect for Persons: Do you select users that represent all segments or meaningful subsegments of users that interact with the product you are testing? Do you understand the history and trauma that a particular group of users may have experienced? Is that accounted for in your research?
- Respect for Persons: Do you have participants review a comprehensive informed consent document that details information about the study and explicitly states the voluntary nature of the study?
- Beneficence: Do you clearly outline the potential benefits AND potential negative outcomes of participating in the study? This should be done explicitly at the personal level, but may also include information about the potential impact to the larger community.
- Beneficence: For stakeholders inside the company, have you outlined the potential benefits and potential negative consequences of the product and how it will impact different segments of users?
- Privacy: Do you collect personally identifiable information? If so, what protections are in place to ensure the data is protected from unauthorized access both inside and outside of your company? Can participants easily and seamlessly delete personal data obtained during a research study?
- Justice: Are all participants within the study treated equitably? Does the manipulation in your study disproportionately affect one group more than another? In other words, are you accidentally hurting someone with your work through the manipulation of variables?
- From a 10,000 ft lens, critically evaluate your work and ask yourself: Is your research harmful? Is your company doing harmful things with your work?
Have additional ideas or questions, or thoughts, comments, or perspectives you would like to share? Please connect with us — we’d love to know more about you and how you’re applying ethics to your work in corporate tech.
