AI Ethics Brief #115: LLMs and the conversational experience, watermarks in AI generated content, hazard analysis framework, and more ...

What can we learn from systems safety to build better AI systems?

Dec 06, 2022

Welcome to another edition of the Montreal AI Ethics Institute’s weekly AI Ethics Brief that will help you keep up with the fast-changing world of AI Ethics! Every week we summarize the best of AI Ethics research and reporting, along with some commentary. More about us at montrealethics.ai/about.

⏰ This week’s Brief is a ~38-minute read.

Support our work through Substack

💖 To keep our content free for everyone, we ask those who can, to support us: become a paying subscriber for as little as $5/month. If you’d prefer to make a one-time donation, visit our donation page. We use this Wikipedia-style tipping model to support the democratization of AI ethics literacy and to ensure we can continue to serve our community.

NOTE: When you hit the subscribe button, you may end up on the Substack page where if you’re not already signed into Substack, you’ll be asked to enter your email address again. Please do so and you’ll be directed to the page where you can purchase the subscription.

This week’s overview:

✍️ What we’re thinking:

Can LLMs Enhance the Conversational AI Experience?
Regulating Artificial Intelligence: The EU AI Act – Part 1

🔬 Research summaries:

The Ethical Need for Watermarks in Machine-Generated Language
A Hazard Analysis Framework for Code Synthesis Large Language Models
A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning
System Safety and Artificial Intelligence

📰 Article summaries:

A New Law Designed for Children’s Internet Safety Will Change the Web for Adults, Too
Digitization, Data, and Distrust in Jamaica
How Google’s Ad Business Funds Disinformation

📖 Living Dictionary:

What is an example of fairness concerns in the use of AI systems?

🌐 From elsewhere on the web:

Scaling AI: Here's why you should first invest in responsible AI
Concordia Continuing Education - Responsible AI - CEAI 1002
AI for Everyone: An expert panel webinar on the importance of non-technical roles in AI

💡 ICYMI

The Challenge of Understanding What Users Want: Inconsistent Preferences and Engagement Optimization

But first, our call-to-action this week:

Come work on my team at BCG - I’m hiring!

I’m hiring for two positions on my Responsible AI team at BCG - if you’re interested and would like a referral as a part of our awesome AI Ethics Brief community, click through below. Information about the positions is also included in the link.

I'd like a referral.

✍️ What we’re thinking:

Can LLMs Enhance the Conversational AI Experience?

During conversations, sometimes people finish each other’s sentences. With the advent of LLMs or large language models, that ability is now available to machines. Large language models are distinct from other language models due to their size, which are typically several gigabytes and billions of training parameters larger than their predecessors. By learning from colossal chunks of data, which could come as text, image, or video, these LLMs are poised to notice a thing or two about how people communicate.

Conversational AI products, such as chatbots and voice assistants, are the prime beneficiaries of this technology. OpenAI’s GPT-3, for example, can generate text or code from short prompts entered by users. OpenAI recently released ChatGPT, a version optimized for dialogue. This is one of many models driving the field of generative AI, where the “text2anything” phenomenon is letting people describe an image or idea in a few words and letting AI output its best guesses. Using this capability, bots and assistants could generate creative, useful responses to anyone conversing.

However, LLMs have their faults. Beyond the lack of transparency in training these models, the costs are typically exorbitant for all but massive enterprises. There are also several instances of them fabricating scientific knowledge and promoting discriminatory ideals. While this technology is promising, designers of conversational AI products must carefully assess what LLMs can do and whether that creates a beneficial user experience.

To delve deeper, read the full article here.

Regulating Artificial Intelligence: The EU AI Act – Part 1

The first-ever legal framework for AI regulation: the Artificial Intelligence Act was proposed by the European Commission on April 21, 2021, with the following specific objectives: (1) Ensure that AI systems placed on the EU market are safe and respectful of fundamental rights and Union values; (2) Ensure legal certainty to facilitate investment and innovation in AI; (3) Enhance governance and enforcement of the law on fundamental rights and safety requirements that apply to AI systems; (4) To facilitate the development of safe and trustworthy AI applications and prevent market fragmentation.

The proposed rules would be enforced through a system at the Member States’ level with a cooperating mechanism at the Union level with the establishment of a European Artificial Intelligence Board. Other measures are proposed to reduce the regulatory burden and support innovation in Small and Medium-sized Enterprises and startups. This proposal is coherent “with the Commission’s overall digital strategy in its contribution to promoting technology that works for people, one of the three main pillars of the policy orientation and objectives announced in the Communication Shaping Europe’s digital future.” The AI proposal is closely linked to the Data Governance Act and the Open Data Directive, which will establish mechanisms and services for using, sharing, and pooling data that are crucial for developing data-driven AI.

To delve deeper, read the full article here.

🔬 Research summaries:

The Ethical Need for Watermarks in Machine-Generated Language

With the ability of large language models to reproduce text becoming more prominent (such as Meta’s Galactica and GPT-3), it becomes increasingly cumbersome to distinguish between machine-generated and human-generated text. Consequently, the authors propose a watermark technique to separate the two to avoid the grave dangers of manipulation.

To delve deeper, read the full summary here.

A Hazard Analysis Framework for Code Synthesis Large Language Models

Code Synthesis Large Language Models (LLMs) such as Codex provide many benefits. Yet, these models have significant limitations, alignment problems, the potential to be misused, and the possibility to destabilize other technical fields. The safety impacts are not yet fully understood or categorized. This paper thus constructs a hazard analysis framework to uncover said hazards and safety risks, informed by a novel evaluation framework that determines the capabilities of advanced code generation techniques.

To delve deeper, read the full summary here.

A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning

Large-scale vision-language models are becoming more pervasive in society. This is concerning given the evidence of the societal biases manifesting in these models and the potential for these biases to be ingrained in society’s perceptions forever – hindering the natural process of norms being challenged and overturned. In this study, we successfully developed efficient but cheap computational methods for debiasing these out-of-the-box models while maintaining performance. This shows promise for empowering individuals to combat these injustices.

To delve deeper, read the full summary here.

System Safety and Artificial Intelligence

The governance of Artificial Intelligence (AI) systems is increasingly important to prevent emerging forms of harm and injustice. This paper presents an integrated system perspective based on lessons from the field of system safety, inspired by the work of system safety pioneer Nancy Leveson. For decades, this field has grappled with ensuring safety in systems subject to various forms of software-based automation. This tradition offers tangible methods and tools to assess and improve safeguards for designing and governing current-day AI systems and to point out open research, policy, and advocacy questions.

To delve deeper, read the full summary here.

📰 Article summaries:

A New Law Designed for Children’s Internet Safety Will Change the Web for Adults, Too

What happened: The California Age Appropriate Design Code Act (Cal-AADC) has been signed into law. It places privacy in the spotlight by shifting the baseline of defaults with new settings. The law requires tools for managing privacy preferences and adjusting behavior manipulations designed to keep children using a product. This law will go through a rulemaking process and will go into effect in 2024.
Why it matters: With research showing that algorithmic systems, such as news feeds and recommendation engines, are leading to addictive behaviors and negatively impacting self-esteem, the bill also touches on these issues. If a company’s defaults cause this harm to children, they must be changed through age verification or by treating all visitors with the baseline defaults established by the law.
Between the lines: This law will significantly impact big tech companies, though educational technology sites and products may also be affected. It can be seen as the first attempt to regulate algorithms from the perspective of health and well-being. It will pressure companies to think of a way to present content that doesn’t worsen the harms that Cal-AADC identifies.

Digitization, Data, and Distrust in Jamaica

What happened: Before being struck down in 2019, Jamaica’s National Identification System (NIDS) would have provided each person with a national identification number and established a secure database of demographic, biographic, and biometric data on all Jamaican citizens. In 2021, Jamaica’s House of Representatives passed a new National Identification and Registration Act, making NIDS voluntary and strengthening the protection of data and identity information.
Why it matters: A discussion about NIDS would only be complete by touching on the concept of trust and its relationship with digital resilience. “With its tagline ‘One ID. Many Opportunities,’ NIDS has emerged as a vehicle through which to engineer a new mode of connectedness, solidarity, partnership, and trust between the state, citizens, and the diaspora.” However, trust in Jamaica’s two major political parties hovers at approximately 22.5%, which begs the question of how to address a “digital trust deficit effectively.”
Between the lines: Concerns about the new National Identity and Registration Act (2021) include the lack of data minimization and the failure to expand upon protections for vulnerable groups. The distrust of NIDS, in particular, ties into a “deeper historiography that tethers distrust of the state to the embodied violation of poor and Black Jamaican bodies.”

How Google’s Ad Business Funds Disinformation

What happened: A ProPublica investigation has found that Google has been placing ads on global websites in Europe, Latin America, and Africa that spread false claims on various topics such as COVID-19, climate change, and elections. The ad revenue is worth millions of dollars to those running these sites while also making money for Google. This investigation also highlighted that ads from Google are more likely to appear on misleading websites in languages other than English.
Why it matters: Google focuses on English-language enforcement and invests in oversight based on three key concerns: PR, regulatory scrutiny, and revenue. Though Google has committed $300 million to fight misinformation and support fact-checkers, “its core ad business provides critical revenue that ensures the publication of falsehoods remains profitable.”
Between the lines: An interesting point in this article is that disinformation in less developed democracies can cause even more damage than in countries with more developed democracies. This investigation has shown unequal enforcement across languages and disparity across and within regions. Google must be held accountable for its decision to make revenue globally.

📖 From our Living Dictionary:

What is an example of fairness concerns in the use of AI systems?

👇 Learn more about why it matters in AI Ethics via our Living Dictionary.

Explore the Living Dictionary!

🌐 From elsewhere on the web:

Scaling AI: Here's why you should first invest in responsible AI

Our founder, Abhishek Gupta, shared his thoughts alongside Steven Mills from BCG and Kay Firth-Butterfield from WEF in this article on:

how artificial intelligence can be transformative for businesses, but increased use of the technology inevitably leads to a higher rate of AI system failures.
why companies should first invest in responsible AI, which also yields benefits in accelerating innovation and helping them become more competitive.
how a prioritization approach that begins with low-effort, high-impact areas in responsible AI can minimize risk while maximizing investment.

💡 In case you missed it:

The Challenge of Understanding What Users Want: Inconsistent Preferences and Engagement Optimization

Engagement optimization is a foundational driving force behind online platforms: to understand what users want, platforms look at what they do and choose content for them based on their past behavior. But both research and personal experience tell us that what we do is not always a reflection of what we want; we can behave myopically and browse mindlessly, behaviors that are all too familiar online. In this paper, we develop a model to investigate the consequences of engagement-optimization when users’ behaviors do not align with their underlying values.

To delve deeper, read the full summary here.

Take Action:

We’d love to hear from you, our readers, on what recent research papers caught your attention. We’re looking for ones that have been published in journals or as a part of conference proceedings.

Click to share your suggestions.

The AI Ethics Brief

Discussion about this post