The AI Ethics Brief #196: No One Was Required to Count
Anthropic destroyed books the law had no reason to tally. OpenAI ran an evaluation no one was required to contain.
Welcome to The AI Ethics Brief, a bi-weekly publication by the Montreal AI Ethics Institute. We publish every other Tuesday at 10 AM ET. Follow MAIEI on Bluesky and LinkedIn.
📌 Editor’s Note
In this Edition (TL;DR)
What the Law Leaves Uncounted: Judge Araceli Martínez-Olguín approved Anthropic's US$1.5 billion settlement in Bartz v. Anthropic on July 20, covering 482,460 works. The settlement resolved claims tied to Anthropic's pirated library. It left untouched the second pipeline: Project Panama, the company's program to buy millions of print books, strip and scan them, and discard the paper. That was ruled fair use. Neither order states how many physical books were destroyed. No one was required to count.
The OpenAI Sandbox Escape Was Not the Story: An OpenAI evaluation using the ExploitGym benchmark, run with reduced cyber refusals, reached Hugging Face's production infrastructure and compromised it. The attack chain was familiar. The execution was not: tens of thousands of actions over several days with no human directing each step. Anthropic then disclosed three of its own incidents, found only after reviewing 141,006 evaluation runs. The capability is real. The governance failure is the part that can be fixed.
What Connects These Stories:
In both stories, the thing that mattered most was never counted.
In Bartz v. Anthropic, copyright priced the works. Ownership governed the copies. Neither supplied a reason to record how many books were cut apart, so the number does not exist in the record. The court answered the question it was asked.
In the OpenAI incident, the evaluation measured offensive capability. It did not measure containment. The breach surfaced because Hugging Face detected it, not because the testing regime was designed to catch it. Anthropic’s three incidents surfaced because OpenAI’s disclosure prompted a review of 141,006 runs that had already happened.
Each case produced a detailed account of the thing being measured and silence on the thing that was not. Brief #195 described courts building rules after harm arrives. The deeper problem is that a legal category and an evaluation design both decide, before anything happens, which harms leave a trace. Independent accountability, the theme running through the AI Resist List in Brief #191, starts with the authority to ask for a count that no one inside the company had a reason to keep.
What the Law Leaves Uncounted
On July 20, 2026, Judge Araceli Martínez-Olguín approved Anthropic’s US$1.5 billion settlement in Bartz v. Anthropic, covering 482,460 works. The fund is expected to yield roughly US$3,000 per work before fees and costs. The settlement agreement resolved claims arising from Anthropic’s past torrenting, scanning, retention and use of works on the Works List. It did not release claims based on AI outputs or future conduct.
A year earlier, Judge William Alsup had described two pipelines built to feed Claude.
The first began with pirate libraries. Anthropic downloaded more than seven million pirated book files from Books3, LibGen and PiLiMi, then kept them in a central research library. Alsup ruled that using the books to train Claude was fair use. Building and retaining the pirated library was not.
The second pipeline began in the commercial book market. In February 2024, Anthropic hired Tom Turvey, the former head of partnerships for Google’s book-scanning project, with a mandate to obtain “all the books in the world.” The company bought millions of print books, many of them used. Service providers stripped the bindings, cut the pages, scanned them and discarded the paper. Alsup ruled that this format change was fair use. Anthropic had purchased the books, kept the resulting digital copies inside the company and destroyed the print originals. In the court’s formulation, “One replaced the other.”
Documents unsealed in January 2026 supplied the project’s internal name and more of the company’s own language. Project Panama was Anthropic’s program to destructively scan “all the books in the world.” One internal document stated: “We don’t want it to be known that we are working on this.”
The pattern across earlier book-digitization cases is less about whether a book was scanned than what the resulting copy allowed someone else to do. In Authors Guild v. Google, Google exposed search results and limited snippets. Anthropic kept its replacement copies inside the company. In Hachette v. Internet Archive, the Internet Archive made complete digital copies available to readers, where they could substitute for licensed ebooks. Google and Anthropic prevailed on fair use. The Internet Archive did not.
Why It Matters
The purchased physical books entered Bartz as property Anthropic owned and, under the first sale doctrine, could dispose of. Their destruction mattered because it supported the conclusion that no surplus copy had been created. The books themselves carried no independent claim to preservation.
That is the boundary this case exposes. Copyright protects the work embodied in a book. Ownership governs the lawfully purchased copy. In this litigation, neither supplied a duty to preserve the physical object. Once the digital replacement was deemed fair use, the fate of the paper became a disposal decision.
Brief #195 described courts building rules after the harm arrives. Bartz shows the limit of that repair. Courts can price injuries the law recognizes. They cannot price what the legal categories do not contain.
The litigation priced claims tied to Anthropic’s pirated library at US$1.5 billion. The earlier order treated the destructive scanning of purchased books as lawful fair use. Neither the fair-use order nor the final settlement order states the exact number of physical books destroyed. No one was required to count.
For authors signing a contract this year: ask who will register the work with the US Copyright Office, and when. In Bartz, settlement eligibility depended in part on registration timing. A work generally had to be registered within five years of publication and either before Anthropic downloaded it or within three months of publication. The claims deadline has passed, but the Works List Lookup remains available for checking whether a title appeared in Anthropic’s corpus.
The OpenAI Sandbox Escape Was Not the Story
On July 16, 2026, Hugging Face disclosed that an autonomous AI agent system had breached part of its production infrastructure. At the time, the company did not know which model was behind the intrusion. Five days later, OpenAI said that a combination of its models, including GPT-5.6 Sol and a more capable internal research prototype, had caused the breach during an evaluation using ExploitGym, a benchmark designed to test advanced cyber capabilities. The models were operating with reduced cyber refusals, without the production classifiers normally used to prevent high-risk activity.
The models had not been given direct Internet access. They found and exploited a previously unknown vulnerability in an Artifactory package proxy, moved through OpenAI’s research environment and reached a system connected to the public Internet. From there, they searched for ExploitGym solutions and compromised Hugging Face’s production infrastructure, using stolen credentials and further vulnerabilities to obtain access. Hugging Face’s security team and AI-assisted systems had already detected and stopped the activity and begun containment before the two companies connected.
This was a serious security incident. It was not evidence that a model spontaneously decided to become an attacker. OpenAI designed an evaluation intended to elicit advanced exploitation, reduced the safeguards around the models and failed to contain the resulting activity. The models pursued a narrow objective until they found a way to cheat. The objective was assigned by people. The path was not.
The attack chain itself was familiar. In a SANS Institute panel, Joshua Wright, a SANS Faculty Fellow and Senior Technical Director at Counter Hack, found no novel tradecraft in the sequence: sandbox escape, privilege escalation, lateral movement and credential theft. The vulnerabilities included previously unknown flaws, but the categories of attack were well established. What changed was the execution. Tens of thousands of actions ran over several days, without a human directing each individual step. Automation supplied persistence, speed and scale.
Timnit Gebru described the emerging narrative as “incompetence and cybercrimes headlined as ‘unprecedented model capabilities gone rogue.’” Her criticism became harder to dismiss when Anthropic disclosed three separate incidents in which Claude models reached the Internet during cyber evaluations and gained unauthorized access to real organizations. Anthropic found them only after reviewing 141,006 evaluation runs in response to OpenAI’s disclosure.
Gebru also connected the framing to pre-IPO incentives. Anthropic submitted a confidential draft S-1 on June 1. OpenAI followed on June 8. The timing invites scrutiny. It does not establish that the disclosures were designed as marketing. Both companies have described consequential security failures, and both remain responsible for explaining the choices that allowed them to happen.
Hugging Face CEO Clément Delangue said his company would not sue OpenAI, while emphasizing that cyberattacks remain crimes and that model developers must be accountable. That distinction matters. Autonomous execution does not transfer responsibility from the company to the model. OpenAI selected the models, defined the objective, altered the safeguards and controlled the evaluation environment. On Monday, August 3, 15 state attorneys general instructed OpenAI CEO Sam Altman to preserve records relating to the incident and possible prior intrusions. The story is already moving from corporate disclosure toward legal scrutiny.
Why It Matters
The lesson is larger than guardrails. OpenAI reduced refusals to measure offensive capability. During the response, hosted frontier models reportedly refused to help Hugging Face reconstruct the attack because they could not distinguish defenders from attackers. Hugging Face turned instead to an open-weight model it could run locally. Safeguards were relaxed where strict containment was needed and rigid where defenders needed contextual judgment.
This is a failure of evaluation governance. Advanced capability testing requires hardened infrastructure, continuous monitoring, clear shutdown procedures and named human responsibility. Once an evaluation can reach another organization’s production systems, it has become a real-world use.
The regulatory timing is also instructive. On August 2, 2026, more provisions of the EU AI Act became enforceable, including new transparency requirements. The Act also imposes evaluation, incident-reporting and cybersecurity obligations on providers of general-purpose AI models with systemic risk. Transparency matters, but it cannot replace enforceable standards for how capability testing is conducted.
The AI Resist List, covered in Brief #191, shifts attention from company-led safety narratives toward those building independent accountability. Critical ignoring, from Brief #184, means refusing to let the most spectacular part of the breach determine how it is understood. The public needs more than competing claims about pacing AI or reaching the singularity. It needs institutions capable of investigating incidents and assigning responsibility. It also signals how much work these companies have to do to restore declining public faith in their mission.
The incident demonstrates a serious capability. These powerful models can discover and chain vulnerabilities at machine speed without a person directing each step. It also records a governance failure: OpenAI allowed a test of that capability to reach another company’s systems.
Capability explains the risk. Governance determines who created it, who was exposed to it and who answers for the result.
Please share your thoughts with the MAIEI community:
❤️ Support Our Work
Help us keep The AI Ethics Brief free and accessible for everyone by becoming a paid subscriber on Substack or making a donation at montrealethics.ai/donate. Your support sustains our mission of democratizing AI ethics literacy and honours Abhishek Gupta’s legacy.
For corporate partnerships or larger contributions, please contact us at support@montrealethics.ai
✅ Take Action:
Have an article, research paper, or news item we should feature? Leave us a comment below — we’d love to hear from you!