The AI Ethics Brief #59: Carbon-accounting for AI, filtered dating, dark patterns, and more ...

How permissive are we with being uncertain in AI ethics work?

Welcome to another edition of the Montreal AI Ethics Institute’s weekly AI Ethics Brief that will help you keep up with the fast-changing world of AI Ethics! Every week we summarize the best of AI Ethics research and reporting, along with some commentary. More about us at montrealethics.ai/about.

This week’s Brief is a ~12-minute read.


Support our work through Substack

💖 To keep our content free for everyone, we ask those who can, to support us: become a paying subscriber for as little as $5/month. If you’d prefer to make a one-time donation, visit our donation page. We use this Wikipedia-style tipping model to support the democratization of AI ethics literacy and to ensure we can continue to serve our community.

*NOTE: When you hit the subscribe button, you may end up on the Substack page where if you’re not already signed into Substack, you’ll be asked to enter your email address again. Please do so and you’ll be directed to the page where you can purchase the subscription.


This week’s overview:

✍️ What we’re thinking:

  • The current state of affairs and a roadmap for effective carbon-accounting tooling in AI

  • Permission To Be Uncertain: Ethics & Social Responsibility in AI Research & Innovation

🔬 Research summaries:

  • Dating through the filters

  • Quantifying the carbon emissions of machine learning

📰 Article summaries:

  • Google’s New Dermatology App Wasn’t Designed for People With Darker Skin (Vice)

  • The New York Times Uses the Very Dark Patterns it Derides (Nir and Far)

  • How a largely untested AI algorithm crept into hundreds of hospitals (Fast Company)

  • Dark Patterns that Mislead Consumers Are All Over the Internet (The Markup)


But first, our call-to-action this week:

Nominate under recognized people in AI ethics to be featured in our next report!

Photo by Tim Mossholder on Unsplash

We are inviting the AI ethics community to nominate researchers, practitioners, advocates, and community members in the domain of AI ethics to be featured in our upcoming State of AI Ethics report.

There is often great work being done in different parts of the world that does not get the attention it deserves due to the state of our information ecosystem and the manner in which platforms surface content. We would like to break that mold and shed some light on the valuable work being done by talented people.

Nominate now


✍️ What we’re thinking:

From the Founder’s Desk:

The current state of affairs and a roadmap for effective carbon-accounting tooling in AI

Digital services consume a lot of energy and it goes without saying that in a world with accelerating climate change, we must be conscious in all parts of life with our carbon footprints. In the case of the software that we write, specifically, the AI systems we build, these considerations become even more important because of the large upfront computational resources that training some large AI models consume, and the subsequent carbon emissions resulting from it. Thus, effective carbon accounting for artificial intelligence systems is critical!

To delve deeper, read the full article here.

Permission To Be Uncertain: Ethics & Social Responsibility in AI Research & Innovation

The interviews in this series explore how today’s AI practitioners, entrepreneurs, policy makers, and industry leaders are thinking about the ethical implications of their work, as individuals and as professionals. My goal is to reveal the paradoxes, contradictions, ironies, and uncertainties in the ethics and responsibility debates in the growing field of AI. 

To delve deeper, read the full article here.


🔬 Research summaries:

Dating through the filters

This essay explores ethical considerations that might arise from the use of collaborative filtering algorithms on dating apps. Collaborative filtering algorithms learn from behavior patterns of users generally to predict preferences and build recommendations for a target user. But since users on dating apps show deep racial bias in their own preferences, collaborative filtering can exacerbate biased sexual and romantic behavior. Maybe something as intimate as sexual and romantic preferences should not be the subject of algorithmic control.

To delve deeper, read the full summary here.

Quantifying the carbon emissions of machine learning

As discussions on the environmental impacts of AI heat up, what are some of the core metrics that we should look at to make this assessment? This paper proposes the location of the training server, the energy grid that the server uses, the training duration, and the make and model of the hardware as key metrics. It also describes the features offered by the ML CO2 calculator tool that they have built to aid practitioners in making assessments using these metrics.

To delve deeper, read the full summary here.


📰 Article summaries:

Google’s New Dermatology App Wasn’t Designed for People With Darker Skin (Vice)

  1. What happened: In an app that is used to assist doctors to do dermatological analysis for skin diseases, it was found that the app had severe limitations in the outcomes from the app for those with darker skin tones. In particular, in a paper that was published in Nature Medicine some time ago, the results from the app were popularized as performing well on people of different ethnicities, much more so than previous solutions that attempted to find a computational solution to detecting skin diseases. 

  2. Why it matters: Something that readers should pay attention to is that the results for various minority groups were based on self-identified ethnicities rather than Fitzpatrick skin types, in particular Type V and Type VI which were severely underrepresented or absent in the dataset used to train the system. For something where skin type can have significant impacts on the outcomes, relying on self-identified ethnicities doesn’t serve as any meaningful proxy for the skin type and can severely overestimate the efficacy of the solution on the non-majority demographics. 

  3. Between the lines: A problem that continues to pervade in dermatology research is the lack of sufficient or comparable datasets for darker skin tones compared to lighter skin tones. And this only gets amplified in computational approaches as well. In particular, without deep investments in building up more representative datasets first, any future research in applying these methods will continue to suffer more similar failures and ink will be spilled (here and elsewhere!) pointing out the same errors again and again.

The New York Times Uses the Very Dark Patterns it Derides (Nir and Far)

  1. What happened: Nir Eyal, author of Hooked and Indistractable, highlights how the NYT uses a dark pattern for its unsubscribe workflow. Dark patterns are something that their journalists have chided other companies for yet at this point it is quite well known that NYT makes it incredibly difficult for subscribers to get out. Eyal positions this as the roach motel model and through screenshots demonstrates the ugliness embodied by the NYT subscriptions team in how they handle their customers. 

  2. Why it matters: Eyal provides recommendations like usability testing and the “regret test” which can help companies get a sense for whether what they are doing follows ethical practices or not. A regret test basically seeks an answer to the question if the user would take an action knowing everything that the designer knows about the product or service. This is a great smoke test to get the basics right in ethical product building. 

  3. Between the lines: The NYT is a long-time offender in the use of this particular dark pattern. Yet, they don’t change their ways but offer a cautionary tale to other companies who engage in such practices that over time, all does come to light and can hurt their long-run sustainability. As Eyal points out in the article, it is perhaps just a function of a large organization with misaligned incentives where someone in charge of subscriptions and revenue decided that such a dark pattern was ok to meet their quotas, disregarding the negative ethical implications that the use of such a pattern has.

How a largely untested AI algorithm crept into hundreds of hospitals (Fast Company)

  1. What happened: Epic, one of the largest health data companies in the US, deployed a non-peer-reviewed system called the Deterioration Index across many hospitals amidst the rush unleashed because of the pandemic. In particular, this system is used to aid doctors in triaging a patient to allocate intensive care beds. The typical workflow in the case of a medical system is to subject the system through rigorous peer-review before allowing it to be used in live settings. 

  2. Why it matters: The biggest flaw emerging from the system, in light of all that we know about the huge issue of bias in medicine, is that it is proprietary. While the doctors are given some guidance on the importance of different factors that go into arriving at the final recommendation from the system, they are not allowed under the hood. This has tremendous potential to amplify pre-existing biases along the lines of race and gender. 

  3. Between the lines: On the one hand, it is not surprising that a system was rolled out hastily without the regular review process given the enormous strains that medical institutions have faced in the wake of the pandemic. But, as has been articulated in many pandemic playbooks before, this should not be used as an excuse for releasing and using untested technology, especially when it can have significant impacts on the lives of people. A similar argument has been made in the case of facial recognition technology as well as many states rolled that out in a hurry to monitor and enforce social distancing rules among other use cases.

Dark Patterns that Mislead Consumers Are All Over the Internet (The Markup)

  1. What happened: Building on the NYT article, the problem of dark patterns continues to plague us and is seemingly all around us. In this piece by The Markup, the article talks about ABCMouse utilizing automatic subscriptions after a free trial to trick customers into giving them money. This is a very common pattern (and the article has a quiz that you can take to test your skills at spotting these) that just doesn’t seem to go away. Other companies are also named in the article including Amazon that makes it hard to cancel Prime subscriptions. 

  2. Why it matters: Dark patterns essentially nudge users into taking actions that they wouldn’t take otherwise. This is a huge problem, especially when you have users who are not that tech-savvy who can fall into these traps. There have been attempts in the past to regulate these dark patterns like the DETOUR Act but there needs to be a systematic effort to root these out. A website linked in the article documents many other cases where this takes place. 

  3. Between the lines: Ethical design practices should be something that is ingrained at a very early stage in the education of designers. More so, this should be reinforced at organizations by way of correctly setting up incentives in the design practice so that the chance of stumbling into, or intentionally practicing, dark patterns becomes minimized.


From our Living Dictionary:

‘Automation bias’

Automation bias is the propensity for humans to favor suggestions from & decisions made by automated decision-making systems.

👇 Learn more about why it matters in AI Ethics via our Living Dictionary.

Explore the Living Dictionary!

From elsewhere on the web:

ICANN open letter on content moderation

“Specifically, we are concerned that the lack of proper action to address the dangers posed by Registry Voluntary Commitments (RVCs), which represent a significant challenge not only to the integrity of the Final Report, but to ICANN’s mission as a whole…

Delegating the decision-making to a third-party arbiter would not absolve ICANN over responsibility for the outcomes of these decisions any more than the Facebook Oversight Board has relieved Facebook of the intense scrutiny that follows its decisions. Is this really the future that ICANN wants?

To delve deeper, read the full letter here, which we (MAIEI) have signed on to.

In case you missed it:

The algorithmic imaginary: exploring the ordinary affects of Facebook algorithms

Bucher explores the spaces where humans and algorithms meet. Using Facebook as a case study, she examines the platform users’ thoughts and feelings about how the Facebook algorithm impacts them in their daily lives. She concludes that, despite not knowing exactly how the algorithm works, users imagine how it works. The algorithm, even if indirectly, not only produces emotions (often negative) but also alters online behaviour, thus exerting social power back onto the algorithm in a human-algorithm interaction feedback loop.

To delve deeper, read the full report here.


Take Action:

Events:

The Triangle of Trust in Conversational Ethics and Design: Where Bots, Language and AI Intersect

We’re partnering with Salesforce to host a discussion about conversational ethics and design.

​Conversational AI enables people to communicate via text or voice with automated systems like smart speakers, virtual assistants, and chatbots. Leveraging Automatic Speech Recognition (ASR) and Natural Language Processing (NLP), these systems can recognize speech, understand context, remember previous dialogue, access external knowledge, and generate text or speech responses.

​However, conversational AI may not work equally well for everyone, and may even cause harm due to known or unknown bias and toxicity. Additionally, generating “personalities” for bots or virtual assistants creates risks of appearing inauthentic, manipulative, or offensive. In this workshop, we will discuss the issues of bias, harm, and trust where bots, language, and AI intersect.

📅 June 10th (Thursday)
🕛 12:00PM – 1:30PM EST
🎫 Get free tickets


Nominate under recognized people in AI ethics to be featured in our next report!

We are inviting the AI ethics community to nominate researchers, practitioners, advocates, and community members in the domain of AI ethics to be featured in our upcoming State of AI Ethics report.

There is often great work being done in different parts of the world that does not get the attention it deserves due to the state of our information ecosystem and the manner in which platforms surface content. We would like to break that mold and shed some light on the valuable work being done by talented people.

Nominate now