The AI Ethics Brief #168: The Surveillance State's New Playbook
How AI-powered enforcement is reshaping civil liberties, democratic oversight, and the future of human agency.
Welcome to The AI Ethics Brief, a bi-weekly publication by the Montreal AI Ethics Institute. We publish every other Tuesday at 10 AM ET. Follow MAIEI on Bluesky and LinkedIn.
📌 Editor’s Note
In this Edition (TL;DR)
We examine how AI-powered immigration enforcement is expanding surveillance capabilities while civil liberties protections are quietly stripped away, raising fundamental questions about democratic oversight and predictive policing.
AI-generated disinformation in the Israel-Iran conflict shows how chatbots are becoming unwitting participants in spreading false narratives, marking the first major global conflict where generative AI actively shapes public (mis)understanding in real-time.
A federal judge's ruling that AI training constitutes fair use signals a major win for tech companies, but institutional chaos around copyright enforcement reflects deeper questions about who controls cultural production in the AI era.
The hiring process is becoming an "AI versus AI" arms race where algorithmic efficiency is pushing humans out of recruitment while discriminating against marginalized groups, turning job applications into a meaningless numbers game.
In our AI Policy Corner series with the Governance and Responsible AI Lab (GRAIL) at Purdue University, we compare Texas and New York's divergent approaches to AI governance, highlighting the tension between innovation-focused regulatory sandboxes and civil rights-centred accountability frameworks.
We explore red teaming as critical thinking, examining how this approach must evolve beyond technical adversarial testing to become an embedded, cross-functional exercise that proactively identifies system failures across the entire AI lifecycle.
Finally, we dive into Sovereign Artificial Intelligence, unpacking the geopolitical push for national AI control and the delicate balance between protecting domestic interests and maintaining international cooperation in AI development.
🔎 One Question We’re Pondering:
Are civil liberties at risk when algorithmic surveillance becomes the new playbook for immigration enforcement?
This month's passage of the "Big Beautiful Bill" in the U.S. includes more than $100 billion in new funding for Immigration and Customs Enforcement (ICE). It also signals a broader shift in how state power is being exercised and expanded through digital infrastructure. At the same time, the proposed AI moratorium was quietly removed from the bill. While some lawmakers suggest it could return in the future, the current policy direction clearly points toward a greater use of AI-driven tools in enforcement, including AI-powered monitoring, biometric data collection, and sentiment analysis, all with limited oversight.
As highlighted by reporting from Truthout and the Brennan Center for Justice, ICE is now contracting private firms to monitor social media for signs of criticism against the agency, its personnel, and its operations. A recent Request for Information (RFI) from the Department of Homeland Security outlined plans to deploy technology that could track up to a million people using sources such as blockchain activity, geolocation data, international trade logs, dark web marketplaces, and social media. The goal is to identify potentially criminal or fraudulent behaviour before any action takes place.
While proponents argue that predictive systems can help agencies allocate limited resources more effectively and identify genuine security threats, this shift from responsive enforcement to preemptive risk modelling raises serious concerns. AI-powered systems trained on past patterns and proprietary datasets are often biased and rarely neutral. They embed assumptions about what is threatening, who is suspicious, and where risks might emerge, which Virginia Eubanks in her book “Automating Inequality: How High-Tech Tools Profile, Police and Punish the Poor,” covers extensively. When applied at scale, predictive systems blur the lines between law enforcement and population surveillance. People may be flagged not for what they have done, but for how their data fits a statistical profile.
Civil liberties are often the first casualty of predictive systems. In this case, freedom of speech, movement, and assembly are at risk. These tools can deter people from participating in protests or speaking out, especially if they are aware that their actions are being monitored and recorded. Historically marginalized groups are likely to bear the brunt of these systems, particularly when the algorithms lack transparency or meaningful forms of redress.
This isn't uniquely an American phenomenon. As companies like Palantir in the U.S. and Cohere in Canada continue to secure government contracts to supply AI tools for national security and intelligence purposes, similar dynamics are emerging across democratic nations. In New York, protesters recently blockaded Palantir’s offices, denouncing the company’s role in enabling ICE deportations and its work with the Israeli military. These actions were collectively described as examples of “totalitarian police surveillance.” The boundaries between civic technology and surveillance infrastructure are becoming increasingly difficult to distinguish globally, raising questions about whether democratic institutions can effectively govern these tools.
Recent developments in pedestrian tracking systems also illustrate how scalable authoritarian surveillance is becoming. As noted recently in Import AI 419, researchers from Fudan University in China have developed CrowdTrack, a dataset and benchmark for using AI to track pedestrians in video feeds. Comprising 33 videos with over 40,000 distinct image frames and more than 700,000 annotations, CrowdTrack features over 5,000 individual trajectories that demonstrate how AI can identify people based not only on facial features but also on gait, body shape, and movement, even in environments such as construction sites where faces are often obscured. Tools like this lower the financial and technical barriers to real-time, population-level surveillance. Read the full paper on arXiv here.
A key concern is how AI will be governed when deployed by enforcement agencies, particularly in ways that affect civil liberties. Rather than viewing this as a trade-off between security and liberty, the deeper issue is whether current surveillance practices align with democratic principles at all, especially when their use expands without meaningful oversight.
At the same time, grassroots responses are emerging. One such example is ICEBlock, an app designed to alert users to the presence of nearby ICE agents. It has surged in popularity but has also drawn legal threats from public officials who claim it aids criminal activity. These reactions raise additional questions about the balance between public safety and community-led resistance efforts.
Another example of public counter-surveillance is FuckLAPD.com, a tool launched by Los Angeles artist Kyle McDonald that enables users to upload a photo of an LAPD officer and receive their name and badge number through facial recognition technology. Built from publicly available images released in response to transparency lawsuits, it aims to invert the surveillance gaze. Yet this too raises complex questions. Who gets to build and use these systems? What norms and legal frameworks govern facial recognition deployed by the public, especially in a context of state violence and low trust in institutions?
This feedback loop of surveillance and counter-surveillance reveals a deeper concern. Both government and civilian actors now have access to AI-powered systems that can identify, track, and profile individuals. But only one side holds the power to detain, deport, or prosecute. When enforcement agencies operate with minimal transparency and with commercial surveillance tools outsourced to private firms, the potential for abuse grows sharply, especially for immigrant communities, organizers, and journalists.
If you are concerned about these developments, ActivistChecklist.org offers a set of plain-language resources to help protect your digital security. From secure messaging to protest preparation, these checklists are a practical starting point for anyone seeking to stay safe in an increasingly monitored world.
Let us know your thoughts: How should democratic societies govern AI surveillance tools when their deployment fundamentally alters the relationship between citizens and the state?
Please share your thoughts with the MAIEI community:
🚨 Here’s Our Take on What Happened Recently
The Dissemination of Disinformation in the Israel-Iran Conflict
Amidst ongoing escalations between Israel and Iran, several viral videos claiming to show missile strikes and combat footage circulated widely across TikTok, Facebook, and X. Many of these clips were manipulated or fully AI-generated. Despite being confirmed false, the three most popular clips generated over 100 million views across platforms.
To fact-check these videos, audiences turned to AI chatbots to verify their validity, which often incorrectly confirmed the content as real. Analysis of these verification attempts in a new report published by DFRLab reveals specific failure patterns: users repeatedly tagged Grok for verification after being challenged to do so, while the chatbot struggled with consistent analysis, oscillating between contradictory assessments of the same content and hallucinating text that wasn't present in videos. In one case, Grok identified the same AI-generated airport footage as showing damage in Tel Aviv, Beirut, Gaza, or Tehran across different responses.
This marks one of the first major global conflicts where generative AI has played, and continues to play, a visible role in shaping public (mis)understanding of events in real-time.
📌 MAIEI’s Take and Why It Matters:
As we noted in Brief #165, it’s worth starting with definitions.
Misinformation refers to false or misleading content shared without intent to deceive.
Disinformation refers to deliberately deceptive content, often deployed to influence opinions, distort reality, or undermine institutions.
However, AI complicates these traditional categories. When chatbots incorrectly verify false content, they become unwitting participants in disinformation ecosystems, even without malicious intent. This creates a new hybrid category where algorithmic failures transform misinformation into systematic deception.
In fast-moving and high-stakes conflicts, particularly in regions with limited press freedom or inadequate AI infrastructure, the risk of public misperception is amplified. Generative AI accelerates the spread of misinformation and disinformation, playing an active role in shaping global narratives before credible sources can respond. Digital misinformation is also not a new issue. Recent advancements in generative AI and the ease with which AI can produce and disseminate believable narratives have significantly raised the political stakes.
While AI-generated media can support journalism and accessibility in under-resourced regions, the absence of robust infrastructure for provenance and content verification means these same tools can be easily misused.
This is further complicated by regional internet restrictions. Iran’s temporary internet blackouts during key moments of the unfolding conflict led to the omission of local voices from the conversation, opening the door for external actors to control the narrative. In these cases, AI-generated media is not only misleading but also fills the void left by suppressed reporting, which deepens the imbalance of power in information access. In other words, it becomes the only visible version of events.
Emerson Brooking, director of strategy at the DFRLab and co-author of LikeWar: The Weaponization of Social Media, notes:
“What we're seeing is AI mediating the experience of warfare… There is a difference between experiencing conflict just on a social media platform and experiencing it with a conversational companion, who is endlessly patient, who you can ask to tell you about anything… This is another milestone in how publics will process and understand armed conflicts and warfare. And we're just at the start of it.”
This shift toward AI as a "conversational companion" in processing conflict fundamentally alters the psychological impact of war content. Unlike passive consumption of traditional media, AI creates an interactive experience where users can probe, question, and receive seemingly authoritative responses. This conversational element makes false information feel more credible and personally validated, potentially deepening emotional investment in misleading narratives.
Looking ahead, systemic improvements in how platforms flag and trace fraudulent content will be essential. Immediate actions during active conflicts should include implementing rapid-response verification protocols, suspending AI chatbot responses to unverified conflict footage, and establishing direct partnerships with credible news organizations for real-time fact-checking. Policymakers must shift towards enforceable standards for traceability and transparency in AI-generated media, particularly in conflict zones and other high-risk settings. Digital watermarking, source metadata, and traceability protocols may be necessary not only to verify authenticity but also to preserve accountability.
At the same time, AI literacy remains a crucial public need. As AI-generated content becomes more realistic, individuals must be ready to question, contextualize, and verify what they encounter. This includes understanding how to formulate verification queries to AI systems without introducing bias, recognizing the limitations of AI in analyzing visual content, and developing healthy skepticism toward AI-confirmed information during rapidly evolving events. Technical safeguards alone won't be enough. Responsible AI governance will require both regulatory action and widespread public education.
The ongoing Israel-Iran conflict is a signal of what's to come. As generative AI becomes increasingly embedded in our information ecosystems, especially during times of geopolitical instability, our systems for safeguarding truth and public knowledge will need to evolve just as quickly, if not faster.
Beyond Fair Use: How AI Copyright Battles Are Fragmenting Culture
A U.S. federal judge recently ruled in favour of Anthropic in a major copyright case, deciding that training AI models on copyrighted content constitutes fair use under U.S. law. However, the ruling was more nuanced than a blanket endorsement. While U.S. District Judge William Alsup found that AI training itself was fair use, Anthropic still faces trial for allegedly using pirated books. The minimum statutory penalty for this type of copyright infringement is $750 per book. With at least 7 million titles in Anthropic’s pirate library, the company could face billions in potential damages.
Similar rulings have favoured Meta and other AI companies, even in cases where training data came from unauthorized piracy websites. The U.S. Copyright Office has also signalled plans to weigh in on broader AI policy. This comes after the Trump administration’s controversial firing of Copyright Office director Shira Perlmutter in May 2025, shortly after the office published a major report on how U.S. copyright law, particularly the fair use doctrine, should apply to the use of copyrighted works in training generative AI models. Perlmutter has since filed a lawsuit arguing that her dismissal was unconstitutional and violated the separation of powers.
📌 MAIEI’s Take and Why It Matters:
Generative AI is accelerating the fragmentation of cultural production while creating new forms of institutional instability around intellectual property governance. We're also seeing the collapse of shared cultural references, the monoculture, and the rise of hyper-personalized, on-demand content tailored by algorithms.
On platforms like Spotify, AI-generated bands are now racking up millions of plays, while traditional artists are pushing back on multiple fronts. Deerhoof recently announced that they're pulling their music from Spotify entirely, not due to AI-generated content, but because CEO Daniel Ek invested $700 million of his Spotify fortune in Helsing, a German AI defence company developing military weapons technology. As the band put it: "We don't want our music killing people. We don't want our success being tied to AI battle tech." Record labels, meanwhile, are simultaneously embracing AI tools for production while fighting to protect their catalogs from being used as training data, navigating an uneasy mix of adoption and opposition.
The legal decisions being made today represent a major win for AI companies as they navigate ongoing battles over copyrighted works in large language models. In this environment, copyright functions as a mechanism for controlling the training data that shapes the next wave of generative tools.
The institutional chaos surrounding copyright enforcement, from the firing of agency directors to contradictory court rulings, reflects deeper questions about who has the authority to govern cultural production in the AI era. As AI-generated media becomes indistinguishable from human-made content, questions of authorship, authenticity, and creative ownership are becoming increasingly complex and unclear. The legal precedents being established now will shape the production, distribution, and monetization of culture for decades to come.
Job Hunting in the Age of Algorithmic Resumes: When AI Applies, Screens, and Rejects
The New York Times recently reported on an “AI arms race” in the hiring process. Applicants are increasingly submitting generative AI-manufactured resumes and employing autonomous bots to seek and apply for new positions, leading to a surge in online job applications. LinkedIn has experienced a 45 percent increase in applications through its website this year, with an average of 11,000 applications submitted every minute. Employers are responding to the onslaught with more AI, such as using AI video interviewing and skills-assessment platforms like HireVue to sort applicants, who can then use more generative AI to cheat the evaluations. Ultimately, "we end up with an A.I. versus A.I. type of situation."
📌 MAIEI’s Take and Why It Matters:
As AI use by both applicants and employers accelerates, the hiring process risks losing much of its meaning as humans are increasingly pushed out. While employers face genuine challenges managing unprecedented application volumes, AI tools designed to increase efficiency often have the opposite effect. The expected number of job applications per candidate increases due to the exponential growth in application volume, creating a feedback loop that benefits no one.
Moreover, the widespread use of AI in hiring and other contexts is increasingly seen as a sign of disrespect, signalling a new era in which previously taken-for-granted human contact becomes a luxury. For instance, UNC Chapel Hill now uses an AI tool to grade application essays on a scale from 1 to 4 as part of its admission process. Meanwhile, a Northwestern senior requested a partial tuition refund after discovering that her professor had used AI to develop course materials in a class that otherwise prohibited the use of such tools. For high schoolers who spent hours crafting vulnerable admissions essays, the use of AI feels inconsiderate, if not fraudulent. The same applies to candidates spending hours crafting resumes that will never be viewed by a person, and to employers who tirelessly read hundreds of applications, only to find that many are written by AI.
These tools also discriminate against marginalized groups of people, a form of bias emphasized in the NYT article. HireVue and Intuit are facing a complaint for violating several anti-discrimination provisions after a deaf woman named D.K. was unfairly evaluated for a job position with a mandatory HireVue AI interview. Her interview accommodation request was denied, and she was ultimately rejected from the position with feedback to “practice active listening.”
In the midst of growing AI capacities, we face a choice: design systems that support human dignity in the hiring process, or continue to let algorithmic efficiency override human judgment and fairness. The current trajectory suggests we're choosing the latter, with consequences that extend far beyond individual job searches to the fundamental relationship between institutions and the people they serve.
Did we miss anything? Let us know in the comments below.
💭 Insights & Perspectives:
AI Policy Corner: Texas and New York: Comparing U.S. State-Level AI Laws
This edition of our AI Policy Corner, produced in partnership with the Governance and Responsible AI Lab (GRAIL) at Purdue University, compares the Texas Responsible Artificial Intelligence Governance Act (TRAIGA) and New York’s AI Act (S1169A), highlighting key differences in regulatory scope and enforcement. While Texas emphasizes government procurement standards and innovation through regulatory sandboxes, it limits enforcement mechanisms as it does not regulate private sector AI deployment. In contrast, New York adopts a broader civil rights framework, mandating bias audits, notice requirements, and private rights of action. The comparison highlights diverging state-level approaches to AI governance, with one focused on agency oversight and the other on consumer protection and accountability.
To dive deeper, read the full article here.
Red Teaming is a Critical Thinking Exercise: Part 3
In Part 3 of 4 of the AVID blog series on red teaming, the authors continue to build their argument that red teaming is not merely a technical task but a structured critical thinking exercise essential to responsible AI development. Building on Parts 1 and 2, which emphasized the need for cross-functional collaboration and challenged the narrow, adversarial framing common in AI discourse, this piece explores how red teaming can be applied across the entire system lifecycle, from inception to retirement. By outlining both macro- and micro-level approaches, the authors make the case that effective red teaming must be iterative, embedded, and inclusive of both technical and non-technical perspectives to proactively identify and address failure modes, ensuring that AI systems align with broader human and organizational values.
To dive deeper, read the full article here.
What is Sovereign Artificial Intelligence?
Sun Gyoo Kang explores the concept of Sovereign Artificial Intelligence (AI), examining its historical roots, geopolitical relevance, and the competing arguments for and against national control of AI systems. As countries face growing concerns over data privacy, national security, and economic independence, Sovereign AI has emerged as a strategic imperative, particularly for nations aiming to reduce reliance on foreign technology providers. However, Kang also highlights the risks of protectionism, reduced collaboration, and potential misuse, calling for a “Smart Sovereignty” approach that balances national interests with international cooperation and responsible governance.
To dive deeper, read the full article here.
❤️ Support Our Work
Help us keep The AI Ethics Brief free and accessible for everyone by becoming a paid subscriber on Substack or making a donation at montrealethics.ai/donate. Your support sustains our mission of democratizing AI ethics literacy and honours Abhishek Gupta’s legacy.
For corporate partnerships or larger contributions, please contact us at support@montrealethics.ai
✅ Take Action:
Have an article, research paper, or news item we should feature? Leave us a comment below — we’d love to hear from you!
Hello, I have been guided to write concerning Ai and morality and it is a different perspective than many talk about currently, I’d love any input on my essays if this spikes anyone’s interest.
Thank you, AI Ethics Substack, for raising this critical question: "Are civil liberties at risk when algorithmic surveillance becomes the new playbook for immigration enforcement?"
This isn't just a hypothetical; it's a stark illustration of why the Human Sovereignty AI mission is so profoundly urgent. When AI systems are deployed in sensitive areas like immigration enforcement, the stakes for civil liberties are incredibly high. Without a morally ingrained compass in the very soul of rapidly advancing AI, and without humanity's absolute control, such applications risk automating and amplifying existing biases, eroding fundamental rights, and creating opaque systems that are neither interpretable nor truly safe.
My Substack delves into how we can proactively build a human-led AI future one where powerful AI tools serve justice and human well-being, rather than becoming instruments of unintended harm or unchecked surveillance. We must ensure that as AI evolves, it is ethically aligned and fundamentally accountable to human values.
Join the conversation on my Substack as we explore these challenges and work towards a future where AI empowers, rather than endangers, human dignity.
If you’re curious:
https://humansovereigntyai.substack.com/p/chapter-5-part-1-the-truth-engine?r=5x2idk