AI Safety & Alignment

Today's Stories

1. 1

   Judge Rules AI Training on Books Legal, Fines Anthropic $1.5B for Piracy

   A federal judge ruled that Anthropic's use of copyrighted books to train its AI models was lawful under copyright law, but ordered the company to pay $1.5 billion for obtaining those books from illegal online shadow libraries rather than licensed sources. The ruling treats AI training like reading rather than copying—a distinction that favors AI companies. Copyright law has not been updated since 1976, leaving judges to interpret decades-old guidelines for novel technologies. The case hinges on whether AI training counts as "fair use," a legal doctrine that allows copyrighted material to be used without permission if the purpose is transformative and non-competing.

   Most AI companies remain in pending litigation over copyright issues, so no definitive legal standard exists yet. Courts are inconsistent: one judge found that training on Thomson Reuters' content to build a competing platform was not fair use, while the Anthropic ruling suggests that training is permissible if not directly competitive. Future rulings may overturn or reinforce these decisions.

2. 2

   AI may make scientists work more, not better, study finds

   Researchers from Princeton, the University of Washington, and other institutions modeled how AI tools affect scientific productivity using optimal foraging theory from behavioral ecology. The model simulates research projects in two phases—evaluating idea viability and then choosing to abandon or advance—and tracks what happens when AI speeds up different stages. The study challenges the assumption that time savings from AI lead to better research. In two out of three scenarios, researchers become more productive by volume but sacrifice thoroughness: when AI helps evaluate early ideas, scientists pick only the most promising but spend less time perfecting them; when it speeds up publishing, weaker projects become worth pursuing, flooding the system with shallower papers. Only when AI accelerates the voluntary deep-dive phase—extra experiments, careful analysis—does quality improve. The authors argue this dynamic is already visible in practice: OpenAI's field report found coding speedups of up to 60× shifted bottlenecks to validation and maintenance, and a METR study found experienced developers using AI actually took 19 percent longer to finish tasks despite feeling 24 percent faster.

   The peer review system is already straining under the load. Arxiv has introduced tougher penalties, threatening a one-year submission ban for hallucinated sources or AI meta-commentary left in papers, after Sakana AI's "AI Scientist-v2" pushed a fully AI-generated paper with citation errors through an ICLR workshop. The paper argues institutional responses must be discipline-specific, since AI's effect depends on which research phase gets accelerated.

3. 3

   Snowflake expands CoCo AI guardrails to costs, access, and external tools

   Snowflake made per-user AI credit quotas generally available across CoCo (its enterprise AI assistant) and introduced three new governance layers set to launch soon: organization-wide policy (MDM), team-level agent profiles, and restricted session scope (RSS). These controls let administrators set daily and monthly limits, define which external systems agents can reach, and constrain what SQL an agent can execute based on the user's role. Before these controls, teams often restricted CoCo to limited groups or manual approval processes to manage risk, limiting productivity gains. Now administrators can write security-backed access policies—quotas make spend bounded, managed settings enforce policy consistently across all surfaces, RSS makes an agent's data reach explicit, and the Cortex AI Gateway logs and pre-approves external tool calls. Each directly answers a security reviewer's question about risk.

   The three new capabilities are generally available soon (exact date not specified). Per-user quotas are available today in Snowflake accounts; administrators can set limits via the "Cost controls for CoCo" documentation. External MCP (Model Context Protocol) tool access now flows through Tools by Cortex AI Gateway, built on technology from Snowflake's Natoma acquisition.

4. 4

   Americans' trust in AI and its leaders plummets to historic lows

   A Pew Research study found that 52% of Americans are now more concerned than excited about AI's growing presence in daily life, up sharply from 37% in 2021. A CNBC Generation Lab poll of 1,088 respondents found that a majority distrusted all nine prominent industry leaders surveyed — Microsoft CEO Satya Nadella at 65% distrust, Palantir CEO Alex Karp at 81%, and Anthropic's Dario Amodei and Alphabet's Sundar Pichai both at around 75%. For the first time, a majority of Americans under 30 (55%) are more concerned than excited about AI. Only 9% of all Americans feel more excited than concerned. The distrust extends beyond the technology itself to those building it, even as these leaders shape AI's development and deployment.

   In Europe, the picture differs: 55% of EU citizens expect AI to shape daily life positively over the next 20 years, and 62% view AI's presence at work favorably. However, 84% of Europeans agree that AI and robots need careful management, and 66% of the broader public believe AI will ultimately destroy more jobs than it creates.

5. 5

   Anthropic's AI safety focus misses decentralizing reality, critic argues

   A detailed critique published on Hacker News challenges Anthropic CEO Dario Amodei's claim that AI is "structurally" centralizing power. The author argues that AI actually decentralizes power by making it impossible for moderators to distinguish human users from bot spam, enabling mass automated abuse that has forced many websites offline. Anthropic has positioned itself around preventing misuse of frontier AI models—particularly open-source ones—through safety controls and regulation. The critique contends this framing obscures the real problem: AI tools already enable harmful behavior (bot spam, slop-posting) through existing closed models like Claude, so restricting open-source models does not address what users actually do with AI today. This suggests the company's regulatory push may reflect competitive interest rather than genuine safety concern.

   The author observes that CAPTCHAs can no longer distinguish humans from AI, and that anything hard enough to block AI also blocks legitimate users—forcing a choice between blocking machines or accepting automated abuse. The debate hinges on whether AI's real structural effect is centralizing (as Anthropic claims) or decentralizing (as the critic argues), which shapes what policy responses would actually matter.