The AI Ethics Brief #164: Balancing Faith and Governance
From the Vatican and Zurich to Singapore and New Zealand, this issue explores how AI governance is being tested across infrastructure, aging and elderly care, and growing market pressure.
Welcome to The AI Ethics Brief, a bi-weekly publication by the Montreal AI Ethics Institute. We publish every other Tuesday at 10 AM ET. Stay informed on the evolving world of AI ethics with key research, insightful reporting, and thoughtful commentary. Learn more at montrealethics.ai/about.
Follow MAIEI on Bluesky and LinkedIn. ❤️ Support Our Work.
In This Edition:
🔎 One Question We’re Pondering:
Can ethical AI deliver both innovation and oversight, or are we regulating ourselves out of competitiveness?
🚨 Here’s Our Take on What Happened Recently:
Can the Vatican Shape AI Ethics? Pope Leo XIV Signals Intent
Care or Codependence? The Ethics of AI in Aging Populations
When AI Overloads the Grid: From New Zealand to Spain
💭 Insights & Perspectives:
AI Policy Corner: Singapore's National AI Strategy 2.0
Responsible AI as a Business Necessity: Three Forces Driving Market Adoption - Tech Policy Press
SHADES: Assessing Multilingual Stereotypes in Large Language Models - Research Summary
📄 Article Summaries:
AI’s Biggest Secret — Creators Don’t Understand It, Experts Split - Forbes
Duolingo will replace contract workers with AI. The company is going to be ‘AI-first,’ says its CEO. - The Verge
Satya Nadella says AI is now writing 30% of Microsoft’s code but real change is still many years away - SiliconANGLE
🔎 One Question We’re Pondering:
Can ethical AI deliver both innovation and oversight, or are we regulating ourselves out of competitiveness?
At the Point Zero Forum 2025 in Zurich, a roundtable of global leaders, including regulators, central bankers, cloud providers, and civil society organizations, convened under the Chatham House Rule to explore how we might balance innovation, regulation, and ethics in the global race to govern AI.
The Montreal AI Ethics Institute (MAIEI) participated in the session, contributing insights on civic engagement, trust-building, and responsible AI deployment and procurement in high-impact sectors.
The discussion offered five key insights into how jurisdictions could navigate this complex landscape:
The global impact of the EU AI Act was a central focus. Its risk-based classification of AI systems, including outright bans on use cases like social scoring, sets a precedent in AI governance. However, concerns were raised that overly rigid compliance obligations could drive startups, talent, and capital out of the EU. Participants emphasized the need for regulatory clarity paired with agility to foster innovation.
Principles-based regulatory models were seen as a pragmatic alternative. Jurisdictions favouring sector-specific approaches, such as the UK’s vertical model, in contrast to the EU’s horizontal framework, enable regulators to tailor oversight to their mandates. Similar experimentation-driven models are emerging in Singapore and Japan, using regulatory sandboxes and iterative guidance shaped by real-world use cases.
Ethical AI is emerging as a strategic advantage. In sectors like finance and healthcare, responsible AI practices are increasingly influencing procurement decisions. Certification frameworks, such as ISO/IEC 42001, are gaining traction as signals of trust and governance maturity. With regulatory shifts, market pressures, and public expectations converging, ethical AI is moving from a “nice-to-have” to a business necessity.
Note: We explore this further in our recent Tech Policy Press op-ed, “Responsible AI as a Business Necessity: Three Forces Driving Market Adoption.”Collaborative ecosystems are essential. Public-private co-creation of policy frameworks was highlighted as key to effective governance, alongside the vital role of civil society and bottom-up public pressure in shaping responsible AI. As more people become informed about AI, engage in public discourse, and organize around workplace concerns, expectations for transparency, fairness, and avenues for redress are rising. Governance mechanisms must be rigorous yet accessible, particularly for small and medium-sized enterprises (SMEs) and startups, and responsive to these bottom-up forces. Labeling systems, akin to sustainability or data privacy certifications, could help signal trust, but must avoid becoming exclusionary or performative.
Balancing innovation with precaution remained a central theme. While regulation is essential for safeguarding rights and public trust, participants debated whether the current risk posture is overly cautious. One provocation captured the tension:
“The question isn’t whether AI should be regulated, but how to do so without stifling what makes it valuable.”
AI governance is no longer hypothetical. It is being shaped in real time by cross-border debates over values, incentives, and risk. These discussions at the Point Zero Forum suggest that ethical AI can indeed be a competitive advantage if regulation is designed with flexibility, collaboration, and global interoperability in mind.
Many thanks to the Global Finance & Technology Network (GFTN) and the Swiss State Secretariat for International Finance for organizing the forum. We look forward to continued engagement with the Point Zero Forum community.
Please share your thoughts with the MAIEI community:
🚨 Here’s Our Take on What Happened Recently
Can the Vatican Shape AI Ethics? Pope Leo XIV Signals Intent
What Happened: On May 8, 2025, Cardinal Robert Francis Prevost was elected the new Bishop of Rome, taking the name Pope Leo XIV. In his first address to the College of Cardinals, he named artificial intelligence as one of the defining challenges of our time. Citing Pope Leo XIII, who led the Catholic Church through the upheaval of the Industrial Revolution, as inspiration, Pope Leo XIV emphasized the need for ethical reflection as society navigates another transformative era. Notably, he holds a Bachelor of Science in Mathematics from Villanova University (1977) and received an honorary Doctor of Humanities, honoris causa, from the same institution in 2014.
📌 MAIEI’s Take and Why It Matters:
The election of Pope Leo XIV may mark a significant moment for the global AI ethics and responsible AI communities. His remarks suggest an unusual depth of engagement with AI, not just as a technical or economic force, but as a moral and societal one. His decision to draw parallels between the Industrial Revolution and today's AI transformation is especially noteworthy, as it places justice, labour, and human dignity at the centre of his message.
That Pope Leo XIV is a mathematics graduate is not incidental. In the AI space, there is a persistent belief that mathematically rooted systems are objective and neutral. But as we’ve argued at MAIEI, algorithms often encode and amplify existing societal biases and power dynamics embedded in their training data.
“People have started to realize, well, these [AI] systems aren't infallible. The belief in their objectivity, because they are mathematically rooted— there is no such thing. They're simply a reflection, perhaps an amplification, of existing structures, systemic and otherwise, within society.”
— The late Abhishek Gupta, Founder, Montreal AI Ethics Institute
A mathematically literate leader with a pastoral commitment to human dignity may be uniquely positioned to challenge the myth of AI neutrality, recognizing that fairness in algorithmic systems is not a technical outcome, but a collective societal decision.
While the Vatican’s influence differs from that of lawmakers or regulators, Pope Leo XIV’s voice could lend moral clarity to global debates on disinformation, automation, labour displacement, and algorithmic accountability. His engagement may offer a bridge between theological ethics and civic AI governance, one rooted in tradition, but responsive to the ethical imperatives of our time.
Care or Codependence? The Ethics of AI in Aging Populations
What Happened: The Aging and Disability Resource Center (ADRC) of Dunn County, Wisconsin, is piloting a program to provide 250 ElliQ elderly assistance robots to seniors over the age of 60. Similar to Alexa or Siri, ElliQ is an AI-based virtual assistant that speaks to and performs tasks for its users. However, ElliQ differs from other AI assistants as it is specifically designed to benefit seniors through combating loneliness, providing health reminders, and initiating companionship with users. In the ADRC pilot program funded by the Inclusa Foundation, qualifying seniors receive the ElliQ assistant for free for a year in exchange for completing quarterly surveys.
📌 MAIEI’s Take and Why It Matters:
This pilot reflects a broader global shift toward deploying AI tools to support aging populations. In 2022, the New York State Office for the Aging (NYSOFA) launched a similar program, releasing approximately 900 ElliQ devices to New York seniors. Other states have also implemented ElliQ programs, including Florida and Washington. Moreover, PARO, a seal companion designed in Japan, has helped elderly patients with dementia, anxiety, and loneliness. The PARO seal contains a variety of sensors which utilize AI technology to process its environment and respond accordingly. Unlike ElliQ, the PARO seal mimics an animal, not a person, and therefore does not speak to its users.
The growing presence of these technologies is a matter of controversy. On the one hand, ElliQ can give medication reminders, facilitate convenient communication with friends and family, and decrease loneliness. In fact, the NYSOFA pilot program demonstrated a 95% decrease in loneliness for ElliQ users. However, they also raise serious concerns. Chief among them are data privacy and the risks of anthropomorphizing AI companions (which we also discussed in The AI Ethics Brief #163), which may lead to further isolation. The ElliQ FAQ page fails to disclose the level of user data protection, omitting the aggregate, anonymized sharing of data with third parties outlined in their privacy policy. Meanwhile, some users have described ElliQ as a “best friend” or preferable to human interaction, highlighting ethical issues around false intimacy and emotional dependency.
In our view, AI has a role to play in elderly care, but only under stronger data governance and protection mechanisms, as well as safeguards against emotional over-attachment. Rather than replace human connection, AI-backed devices like ElliQ could serve as tools to enable it. Devices could arrange virtual game nights or museum tours between app users, or connect them to local community events to attend with friends or family. Functions such as appointment reminders and video calls are useful, but conversations should ideally happen with people, not programs.
When AI Overloads the Grid: From New Zealand to Spain
What Happened: The invisible backbone of our AI revolution has suddenly become glaringly visible. As data centres devour electricity at unprecedented rates, the physical infrastructure of our power grids is buckling under pressure that nobody quite anticipated. In New Zealand, Vector's sobering forecast of a 60% surge in data centre electricity consumption over the coming decade has forced uncomfortable questions about priorities and planning. Meanwhile, across the Atlantic, Spain and Portugal's recent nationwide blackout on April 28th wasn't just an inconvenience; it was a stark warning. The cascading power incidents that preceded the total failure, as documented by Reuters, have exposed the fragility of systems we've taken for granted. Throughout Europe, policymakers find themselves caught in an insoluble balancing act: should they pursue AI dominance or prioritize energy stability? This tension isn't merely technical, it's the physical manifestation of digital ambitions colliding with material constraints.
📌 MAIEI’s Take and Why It Matters:
The ethical dimensions of this collision run deeper than most tech debates. When hospitals compete with server farms for reliable power during shortages, who decides which lights stay on? The question of justice in allocation becomes worryingly concrete. AI’s energy appetite risks derailing national climate commitments, even as those same governments enshrine ambitious sustainability targets into law. More troubling still is how ordinary households quietly shoulder the financial burden of grid expansions primarily serving private tech interests, with minimal democratic oversight. Such dual power imbalance, both electrical and political, risks cementing global inequality as regions with unstable infrastructure become permanent AI backwaters.
Did we miss anything? Let us know in the comments below.
💭 Insights & Perspectives:
📌 Editor’s Note:
The Insights & Perspectives summarized below reflect the widening spectrum of Responsible AI governance, from national strategies and organizational adoption to multilingual fairness in model development.
Singapore’s National AI Strategy 2.0 presents a pragmatic roadmap grounded in cross-border collaboration, R&D investment, and workforce development, highlighting the state's dual role as both regulator and ecosystem enabler. In parallel, our recent op-ed in Tech Policy Press outlines how top-down regulation, market pressure, and bottom-up public influence are converging to shape responsible AI as a business necessity. Yet, responsible governance also hinges on inclusive design. The SHADES dataset challenges the Western-centric defaults of many AI systems by highlighting how stereotypes propagate across 16 languages and 37 geographical regions. Its development marks a critical step toward more equitable, globally attuned AI evaluation frameworks.
Together, these perspectives highlight that AI governance is no longer limited to compliance or ethics checklists. It is increasingly about aligning systems with societal values through policies that anticipate complexity, markets that reward accountability, and datasets that reflect the full diversity of human experience.
AI Policy Corner: Singapore's National AI Strategy 2.0
By Evan Glenn. This article is part of our AI Policy Corner series, a collaboration between the Montreal AI Ethics Institute (MAIEI) and the Governance and Responsible AI Lab (GRAIL) at Purdue University. The series provides concise insights into critical AI policy developments from the local to international levels, helping our readers stay informed about the evolving landscape of AI governance.
Singapore’s National AI Strategy 2.0 lays out a comprehensive framework to “Harness AI for the Public Good, for Singapore and the World.” Divided into three systems: Activity Drivers, People & Communities, and Infrastructure & Environment, the strategy emphasizes collaboration across government, academia, and industry. Key initiatives include public funding for AI R&D, workforce development, data governance, and cross-border safety testing with partners like Japan. While general in tone, NAIS 2.0 is backed by early implementation milestones and offers a model of pragmatic, internationally-minded AI policymaking.
To dive deeper, read the full article here.
Responsible AI as a Business Necessity: Three Forces Driving Market Adoption - Tech Policy Press
In our latest op-ed published in Tech Policy Press, we explore how responsible AI is increasingly becoming a market-driven necessity. Co-authored by MAIEI’s Marianna B. Ganapini and Renjie Butalid, the piece outlines how top-down regulation, market pressure, and bottom-up public influence are converging to shape responsible AI governance.
Drawing on case studies from Microsoft and Moody’s Ratings, as well as findings from PwC’s 2024 US Responsible AI Survey and the Partnership on AI’s 2025 Guidance for Inclusive AI: Practicing Participatory Engagement, we contend that responsible AI is moving beyond voluntary principles toward operational frameworks, certification schemes, and procurement standards, emerging as both a key differentiator and a source of competitive advantage.
To dive deeper, read the full article here.
SHADES: Towards a Multilingual Assessment of Stereotypes in Large Language Models
SHADES represents the first comprehensive multilingual dataset explicitly aimed at evaluating stereotype propagation within large language models (LLMs) across diverse linguistic and cultural contexts. Developed collaboratively by an international consortium of researchers, SHADES compiles over 300 culturally-specific stereotypes, rigorously gathered and validated by native and fluent speakers across 16 languages and 37 geographical regions. The dataset aims to support more inclusive AI governance and guide policymakers, researchers, and developers in identifying and mitigating bias within multilingual systems.
To dive deeper, read the full article here.
📄 Article Summaries:
AI’s Biggest Secret — Creators Don’t Understand It, Experts Split - Forbes
Summary: Anthropic CEO Dario Amodei recently acknowledged a critical tension in AI development: even the creators of today’s frontier models often do not fully understand how they work.
People outside the field are often surprised and alarmed to learn that we do not understand how our own AI creations work. They are right to be concerned: this lack of understanding is essentially unprecedented in the history of technology.
- Dario Amodei (The Urgency of Interpretability, April 2025)
This rare moment of candor has reignited debate in the AI community over whether this interpretability gap constitutes a crisis or is simply part of the learning curve in an emerging field. Some experts argue that the opacity of AI systems poses serious risks in sectors like healthcare and finance, while others point to historical parallels in other complex technologies, where trust was earned over time through performance and oversight. Increasingly, the debate is shifting from pure technical transparency to broader calls for accountability, open science, and responsible deployment.
Why It Matters: This moment of honesty from a leading AI figure should serve as a wake-up call, not to panic, but for society to pay attention. When creators admit they cannot fully explain their own systems, the burden of caution shifts unfairly to users. Yet within this challenge lies an opportunity. Non-technical professionals play a critical role, not in decoding models, but in shaping their governance. This means engaging, learning, and leading with awareness. Trust, transparency, and accountability are no longer just engineering goals, they are business and ethical imperatives. If we are to coexist with AI systems we do not fully understand, we need stronger safeguards and a shared commitment to responsible innovation.
To dive deeper, read the full article here.
Summary: Duolingo has announced plans to become an “AI-first” company, gradually phasing out contract workers for tasks AI systems can now perform. CEO Luis von Ahn emphasized that this shift “isn’t about replacing Duos with AI. It’s about removing bottlenecks so we can do more with the outstanding Duos we already have.”
Why It Matters: Duolingo joins a growing list of companies, Shopify among them, transitioning routine operational tasks to AI as part of broader cost-efficiency and productivity strategies. While framed as a move to accelerate content creation and scale services, the shift raises questions about accountability in algorithmic decision-making and the future of work. What does it mean for companies to publicly declare themselves “AI-first”? And what signals does this send to existing and prospective employees, especially in creative, linguistic, or human-centred roles, about their value within these organizations?
To dive deeper, read the full article here.
Summary: During a fireside chat with Mark Zuckerberg at LlamaCon 2025, Microsoft CEO Satya Nadella revealed that AI now generates approximately 30% of Microsoft’s internal codebase. He noted that while AI performs well in Python, it remains less effective in more complex programming languages like C++. Both Nadella and Zuckerberg emphasized that the full economic and societal impact of AI may still take “many years” to materialize.
Why It Matters: Microsoft’s announcement offers a concrete data point in the ongoing shift toward AI-assisted development, but also serves as a caution: productivity gains are not evenly distributed across domains. Nadella stressed that AI’s potential hinges not just on technical capacity, but on rethinking how humans and machines collaborate. As AI tools increasingly take on high-volume, low-judgment tasks (e.g. routine coding tasks, automating technical workflows), the strategic value of human input will shift toward oversight, critical thinking, and contextual judgment, core pillars of ethical AI deployment. Rather than eliminating the need for human input altogether, AI raises the bar on where that input matters most.
To dive deeper, read the full article here.
❤️ Support Our Work
Help us keep The AI Ethics Brief free and accessible for everyone by becoming a paid subscriber on Substack or making a donation at montrealethics.ai/donate. Your support sustains our mission of democratizing AI ethics literacy and honours Abhishek Gupta’s legacy.
For corporate partnerships or larger contributions, please contact us at support@montrealethics.ai
✅ Take Action:
Have an article, research paper, or news item we should feature? Leave us a comment below — we’d love to hear from you!
Great article! Though I believe comparing data centre growth against essential infrastructure like hospitals creates a false dichotomy, particularly with the emergence of microgrids that can generate and store electricity locally. Data centres are/can be required to install additional generation capacity, particularly renewable sources like solar, and co-locate these with battery storage systems (see examples in Dublin, Microsoft in San Jose, Verrus' work). Their growth can also be used as fuel for local authorities to demand further investments from data centre developers (e.g. using waste heat for other purposes like heat district networks)
https://github.com/Camaron-FosterAI
I too am trying to think through these puzzles and wanted to share.