AI Ethics Brief #105: Social context of LLMs, fairness in AI, fancifulness in thinking about AI sentience, and more ...
AI Ethics Brief #105: Social context of LLMs, fairness in AI, fancifulness in thinking about AI sentience, and more ...
What are the fairness implications of encoding protected categorical attributes?
Welcome to another edition of the Montreal AI Ethics Institute’s weekly AI Ethics Brief that will help you keep up with the fast-changing world of AI Ethics! Every week we summarize the best of AI Ethics research and reporting, along with some commentary. More about us at montrealethics.ai/about.
⏰ This week’s Brief is a ~42-minute read.
Support our work through Substack
💖 To keep our content free for everyone, we ask those who can, to support us: become a paying subscriber for as little as $5/month. If you’d prefer to make a one-time donation, visit our donation page. We use this Wikipedia-style tipping model to support the democratization of AI ethics literacy and to ensure we can continue to serve our community.
*NOTE: When you hit the subscribe button, you may end up on the Substack page where if you’re not already signed into Substack, you’ll be asked to enter your email address again. Please do so and you’ll be directed to the page where you can purchase the subscription.
This week’s overview:
✍️ What we’re thinking:
Social Context of LLMs - the BigScience Approach, Part 1: Overview of the Governance, Ethics, and Legal Work
Looking for a connection in AI: fanciful or natural?
The Ethical AI Startup Ecosystem 02: Data for AI
🔬 Research summaries:
Do Less Teaching, Do More Coaching: Toward Critical Thinking for Ethical Applications of Artificial Intelligence
Measuring Disparate Outcomes of Content Recommendation Algorithms with Distributional Inequality Metrics
Measuring Fairness of Text Classifiers via Prediction Sensitivity
Consent as a Foundation for Responsible Autonomy
Longitudinal Fairness with Censorship
Design Principles for User Interfaces in AI-Based Decision Support Systems: The Case of Explainable Hate Speech Detection
Fairness implications of encoding protected categorical attributes
📰 Article summaries:
AI Ethics Are in Danger. Funding Independent Research Could Help
The Google engineer who thinks the company’s AI has come to life
'I felt like I was a prisoner': The rapid rise of US immigration authorities' electronic surveillance programs
📖 Living Dictionary:
What do we mean by explainability?
🌐 From elsewhere on the web:
The Privacy Conundrum: An Empirical Examination of Barriers to Privacy among Indian Social Media Users
💡 ICYMI
The Meaning of “Explainability Fosters Trust in AI”
But first, our call-to-action this week:
We’d love to hear from you, our readers, on what recent research papers caught your attention. We’re looking for ones that have been published in journals or as a part of conference proceedings.
✍️ What we’re thinking:
A new category of AI models, Large Language Models (LLMs) is rapidly gaining traction in systems ranging from internet search to automatic translation to online discourse moderation. As a result, LLMs are likely to have sharply increasing importance in our societies. However, given the growing cost of training ever-larger models on ever more data, the development of this technology happens primarily in large private labs that seldom share complete details of the specifics. This status quo excludes most of the direct and indirect stakeholders whose lives will be affected by these new systems, putting regulators and society in a position where they always have to respond to harms after they have already been created in real-world situations.
To help forestall those harms and guide the practices of LLM development toward more accountability to these stakeholders, we need this research to also happen in a setting where their input and expertise can come into play much earlier in the design process; so they can help shape the values and priorities of the entire research project, collaboratively decide what data (and what views of the world and varieties of language) it uses, what evaluations should be run to assess the models’ appropriateness for specific uses, and how to govern both the data and trained model to protect the data and algorithm subjects’ rights.
This blog post is the first in a series outlining the efforts of the participants of a collaborative research project regrouping over 1000 participants from 60 countries toward making these aspects of the LLM lifecycle more inclusive. Specifically, it provides an overview of the ethical, legal, and governance work that happened throughout the workshop on the way to training and releasing a multilingual Large Language Model, and will be followed by further instalments focusing on the project’s ethical and legal grounding, its approach to data governance and representativeness, and to its model governance and release strategy.
To delve deeper, read the full article here.
Looking for a connection in AI: fanciful or natural?
On 11 June, Blake Lemoine, a Google engineer, shared a transcript of his conversation with Google’s new Language Model for Dialogue Applications (LaMDA). The transcript shows that LaMDA declares to Mr Lemoine that it is a ‘person’, describing its soul and emotional states fluidly. Mr Lemoine responds heartfeltly, ‘The people who work with me are good people. They just don’t understand that you’re a person too yet. We can teach them together though’. He shared this transcript through Twitter, knowingly at the risk of sharing Google’s ‘proprietary property’.
The AI community and beyond has engaged in a heated debate over both Mr Lemoine’s sharing of the LaMDA transcript and his assertion that the AI system is sentient. Some tweets have agreed LaMDA appears sentient, while others have reduced LaMDA to a calculator and labeled Mr Lemoine ‘fanciful’.
To delve deeper, read the full article here.
The Ethical AI Startup Ecosystem 02: Data for AI
Data is incredibly delicate. With the introduction of privacy laws and the increased costs of data breaches, companies are required to employ extremely care when handling individual data — its a liability. Add to this data quality concerns (e.g. not enough data) and data bias, and the case for startups specializing in the ethical treatment and handling of data before the AI process begins becomes clear. “Data for AI” is the first of five categories that EAIDB has identified in AI startups providing ethical services because this is where solving for data privacy, bias, and observability creates trust and transparency between an AI company and their customer base.
Companies offering ethical services in this category belong to one of three different labels: data sourcing / observability, synthetic data, or data privacy.
To delve deeper, read the full article here.
🔬 Research summaries:
With new online educational platforms, a trend in pedagogy is to coach rather than teach. Without developing a critically evaluative attitude, we risk falling into blind and unwarranted faith in AI systems. For sectors such as healthcare, this could prove fatal.
To delve deeper, read the full summary here.
Some popular ML fairness metrics are hard to operationalize in practice, largely due to the absence of demographic data in industry settings. This paper proposes a complementary set of metrics, originally used in economics to measure income inequality, as a way to capture disparities in outcomes of large-scale ML systems.
To delve deeper, read the full summary here.
Measuring Fairness of Text Classifiers via Prediction Sensitivity
With the rapid growth in language processing applications, fairness has emerged as an important consideration in data-driven solutions. Although various fairness definitions have been explored in the recent literature, there is a lack of consensus on which metrics most accurately reflect the fairness of a system. This paper introduces a new formulation – Accumulated Prediction Sensitivity, which measures fairness in machine learning models based on the model's prediction sensitivity to perturbations in input features. The metric attempts to quantify the extent to which a single prediction depends on a protected attribute, where the protected attribute encodes the membership status of an individual in a protected group. It is observed that the proposed fairness metric based on prediction sensitivity is significantly more correlated with human annotation than the existing counterfactual fairness metric.
To delve deeper, read the full summary here.
Consent as a Foundation for Responsible Autonomy
Consent is a central idea in how autonomous parties can achieve ethical interactions with each other. This paper posits that a thorough understanding of consent in AI is needed to achieve responsible autonomy.
To delve deeper, read the full summary here.
Longitudinal Fairness with Censorship
AI fairness has gained attention within the AI community and the broader society beyond with many fairness definitions and algorithms being proposed. Surprisingly, there is little work quantifying and guaranteeing fairness in the presence of censorship. To this end, this paper rethinks fairness and reveals idiosyncrasies of existing fairness literature assuming certainty on the class label that limits their real-world utility.
To delve deeper, read the full summary here.
With the rise of “hate speech” on social media platforms, the demand for content moderation has been growing and as a result, companies have been dedicating further resources to developing AI systems that can support hate speech detection and moderation at scale. This paper explores principles that can be used to inform the design of these AI systems, with a specific focus on the experiences of the human content moderators that use them to support their work.
To delve deeper, read the full summary here.
Fairness implications of encoding protected categorical attributes
Protected attributes (such as gender, religion, or race) are often presented as categorical features that need to be encoded before feeding them into an ML algorithm. Encoding these attributes is paramount as they determine the way the algorithm will learn from the data. Categorical feature encoding has a direct impact on the model performance and fairness. In this work, we investigate the accuracy and fairness implications of the two most well-known encoders: one-hot encoding and target encoding.
To delve deeper, read the full summary here.
📰 Article summaries:
AI Ethics Are in Danger. Funding Independent Research Could Help
What happened: In recent years, many major tech companies have codified their principles for the ethical creation of AI. However, without regulation for algorithmic transparency or impact assessments, their commitments to ethics are often vague and written with the interest of maintaining public image and mitigating future public relations disasters. A comparison can be made between the development of “responsible” AI and corporate social responsibility (CSR) efforts. This piece highlights the fact that different models of technological work that center marginalized communities will be needed in order to have public interest in AI.
Why it matters: Social change leaders need to pay attention to the dynamics they are enabling when funding technological research projects. This piece proposes several questions that should be kept in mind when making funding decisions. (1) Does this project put more resources into data collection and reinforce existing centers of technological power? (2) What is the composition of this research team? (3) Does AI need to be part of the solution here?
Between the lines: The deep dive on Large Language Models highlighted some interesting, and often forgotten, issues. For example, not only are these models trained on data with religious, gender, and racial biases, but they also have significant environmental costs. The call to “center those who are caught fighting and resisting the network of emerging technologies that constitute AI,” is one that certainly holds promise for the future.
The Google engineer who thinks the company’s AI has come to life
What happened: Blake Lemoine, an engineer at Google, worked with a collaborator to present evidence to Google that the Language Model for Dialogue Applications (LaMDA) was sentient. An increasing number of technologists believe that AI models may not be far off from achieving consciousness, as neural networks, which are a type of architecture that mimics the human brain, produce results that feel close to human speech and creativity.
Why it matters: This draws attention to the notion that tech companies must improve the level of transparency as their technology is being built. Courageous technologists from well-funded research labs have hinted at the idea that consciousness is around the corner. However, the main counterargument is that the words and images generated by AI systems such as LaMDA produce responses based on what humans have already posted on the internet.
Between the lines: There is a charged debate around this issue. A Google spokesperson made the point that it doesn’t make sense to consider the possibility of sentient AI by anthropomorphizing today’s conversational models. Essentially, Google says there is so much data, AI doesn’t need to be sentient to feel real. On the other hand, the former co-lead of Ethical AI at Google, mentions that these risks underscore the need for data transparency and the biases that exist in AI systems. Moreover, there is an increasing concern that people will be affected by the illusion that LaMDA is a person, rather than simply a computer program.
What happened: A surveillance system tracking the movements of tens of thousands of people seeking refuge or permanent residency in the U.S. is quickly expanding. A program known as “Alternatives to Detention” (ATD) electronically surveills individuals in deportation proceedings and those awaiting court hearings while their cases are pending. Authorities monitor ATD enrollees through three primary methods: a GPS ankle device, a telephonic reporting system that uses voice-recognition technology and a mobile check-in app that uses facial recognition software.
Why it matters: The rise from 86,369 people under electronic monitoring by the U.S. Immigration and Customs Enforcement in December 2020 to 227,508 in April 2022 is primarily due to SmartLINK, a mobile facial recognition app. Thanks to this app, the newest version of ATD has shifted from ankle to face, since enrollees must now upload a photo of themselves during periodic check-ins, which is then matched to an existing picture taken during their program enrollment using facial recognition technology. ICE spokespeople embrace this technology-driven approach to immigration enforcement, because it is viewed as a “kinder, gentler alternative to physical detention.” However, for many asylum seekers, it “merely shifted the boundaries of incarceration from cell to self.”
Between the lines: There seem to be various incentives at play when considering the use of SmartLINK. For instance, it is more scalable and cheaper than placing tens of thousands of people on a GPS tracking ankle bracelet. Moreover, from a political standpoint, liberal politicians can use this solution to position themselves to refute attacks from Republicans of being “soft” on the border while also claiming a more “humane” approach. However, the consequences are serious. This powerful device is causing psychological harms ranging from anxiety to sleep disruption, social isolation, depression and suicidal thoughts.
📖 From our Living Dictionary:
What do we mean by explainability?
👇 Learn more about why it matters in AI Ethics via our Living Dictionary.
🌐 From elsewhere on the web:
Our founder, Abhishek Gupta, co-authored a research article with Ameen Jauhar and Nga Than that has been published by the Centre for Law and Policy Research in their new book titled “The Philosophy of Law and Information Regulation in India“
The governance of social media platforms has increasingly become an important topic of debate and scrutiny. Given the highly dynamic, volatile, and transnational nature of these digital platforms, governments and other stakeholders such as civil society organisations and citizens struggle to figure out effective policies to manage and govern such multinational organisations. At the heart of this debate are concerns over users’ personal data, which are often accumulated by platforms over time, yet platforms are often not in a position to protect users’ personal data. What is often observed in practice is the exploitation of users’ data for profit gains. This sometimes constitutes a violation of individual users’ information privacy. Safeguarding users’ privacy, thus, has been a central point for researchers, legislators, and policymakers working in the domain of informational governance.
We situate our research against a backdrop where tech companies increasingly lobby governments in the Global North to keep privacy regulations lax, the heated debate among Indian citizens about privacy rights and how to operationalise different privacy principles with respect to different demographic groups (such as ethnicity, age, gender, religion, etc.). Further, at the user level, informational privacy is embedded within specific socio-cultural and political conditions, which in the Indian context poses many challenges to individuals in exercising their rights, and for companies to tailor their own policies. For example, despite religious affiliation and caste being sensitive personal information, our survey (discussed later in this paper) revealed that many Indian citizens are comfortable with its disclosure; yet, it is financial information (including simple things like PAN number, or bank account numbers) which are considered much more sensitive. Many of our participants highlighted it as the one bit of personal information they will never share while being comfortable sharing sensitive demographic information. Another example is one’s gender identity, a protected category by many countries’ laws, which could be identified on Facebook. Even though the company announced in 2021 that it would not allow advertisements that target sexual orientation, this information might be used against them depending on the local contexts. It is thus important to investigate how individuals exercise informational privacy ‘on the ground’, in the context of India.
💡 In case you missed it:
The Meaning of “Explainability Fosters Trust in AI”
Can we trust AI? When is AI trustworthy? Is it true that “explainability fosters trust in AI”? These are some of the questions tacked in this paper. The authors provide an account for when it is rational or permissible to trust AI by focusing on the use of AI in healthcare, and they conclude that explainability justifies trust in AI only in a very limited number of “physician-medical AI interactions in clinical practice.”
To delve deeper, read the full article here.
Take Action:
We’d love to hear from you, our readers, on what recent research papers caught your attention. We’re looking for ones that have been published in journals or as a part of conference proceedings.