AI Safety & Alignment

Today's Stories

1. 1

   Academic article examines 'successionism'—the fringe idea that humanity should not continue existing—as a debate topic in Silicon Valley motivated by AI risk concerns.

   Andrew Critch named the idea of humanity not continuing to exist 'successionism,' framing it as a position motivated by concerns about the speed and power of AI development. The article notes that this debate has been present in Silicon Valley, with examples cited going back to at least 2013. The article frames this as a question 'you'd hope was all too obvious'—whether humanity should continue existing—yet describes it as a topic of 'lively debate' in Silicon Valley. This suggests that AI risk concerns have pushed some technology leaders to seriously entertain ideas that would otherwise seem beyond the bounds of mainstream discussion.

   The article is sourced from a philosophy/alignment blog (No Set Gauge, republished on LessWrong) rather than reporting breaking news; it offers a conceptual examination of how AI risk anxiety has shifted the boundaries of acceptable debate in tech circles, without predicting near-term policy or business changes.

2. 2

   Japanese railways are deploying AI safety systems at level crossings to detect hazards like stalled vehicles and trapped pedestrians, with government support encouraging wider adoption.

   Kintetsu Railway began full-scale operation of an AI-equipped camera system at a crossing in Kyoto in May, after about a year of demonstration tests at two locations. The system automatically detects people and vehicles on tracks, marks them with colored outlines (reddish purple when danger is imminent), and triggers emergency alerts to nearby trains and railway departments. Nagoya Railroad has introduced similar systems at about 50 crossings and is also testing technology to prevent vehicles from entering crossings during road congestion. During Kintetsu's roughly 80 days of test data review, the system identified seven cases in which people remained trapped inside crossings or could not exit immediately—situations that company officials say could accumulate into serious accidents if left undetected. The government has started offering financial support to encourage adoption, signaling that railway operators view AI detection as an effective tool for crossing safety.

   The AI system automatically activates an emergency notification button when danger is detected, rather than relying on manual reporting by staff. Nagoya Railroad's ongoing research into preventing vehicle entry during traffic congestion suggests the technology may expand beyond pedestrian detection.

3. 3

   An article questions whether e/acc (effective accelerationism) is a substantial intellectual movement or largely a fringe phenomenon with few notable adherents and unclear philosophical foundations.

   The author observes that e/acc, a cultural movement sometimes presented as a major counterpoint to AI safety concerns, appears to have very few prominent representatives—with Beff Jezos cited as perhaps the most visible figure—and lacks clearly articulated intellectual tenets. Because e/acc is being treated in documentaries and safety discussions as a significant contrary perspective, there is value in examining whether it actually represents a substantial viewpoint with coherent arguments or is a smaller movement that may not warrant the attention it receives in these high-profile forums.

   The author suggests that the apparent counterarguments to AI risk raised in these discussions remain incomplete or unclear in the article body, leaving the reader to assess whether e/acc offers a genuinely alternative framework or merely presents itself as one.

4. 4

   Philosophy essay argues AI extinction risks should be understood as genocide-like harm, not abstract thought experiments.

   A writer on LessWrong contends that people often dismiss existential risk from AI by reasoning that if everyone dies simultaneously, there is no one left to suffer—but argues this framing misses the point by treating the catastrophe as an abstract hypothetical rather than a concrete harm. The essay suggests that the popular image of AI-caused extinction as a sudden, simultaneous event obscures the true nature of such a scenario. By reframing the risk as comparable to genocide rather than a thought experiment, the author aims to make the stakes feel more morally urgent and less surreal to readers who might otherwise discount the concern.

   The piece challenges a common objection to AI safety advocacy—the claim that death of all humans is inconsequential because no one survives to experience unhappiness—and proposes that deeper consideration reveals this view unlikely to hold up under scrutiny.

5. 5

   AI safety researchers surveyed on continual learning report mixed views on risks and promising approaches, offering a snapshot of expert opinion on how AI systems might adapt and learn over time.

   Researchers conducting a continual learning study surveyed AI safety experts with questions about the arguments, risks, forecasts, and proposed solutions related to continual learning for language model agents. The post summarizes the survey results and also provides an overview of forecasts made by other experts who did not participate. Continual learning—the ability of AI systems to keep learning and adapting after deployment—is a topic that safety researchers are actively thinking through. Understanding how experts view the risks and promise of different approaches helps clarify which angles are most worth investigating as AI systems become more capable.

   The post is part of a sequence on the implications of continual learning for LLM agents, suggesting this is an early-stage research area where expert consensus and concerns are still being formed.

6. 6

   This article argues against waiting to pause AI development, asserting that immediate action is more realistic than betting on a perfectly-timed future pause.

   The author challenges the common argument that AI should be paused later rather than now, questioning the assumption that a pause can be initiated precisely when needed and that advanced AI during a pause would aid safety research. The piece suggests that the reasoning often cited for delaying a pause—that waiting preserves the option to pause later, that public backlash after a pause might harm AI safety efforts, or that current models are not yet dangerous—may be flawed. The author implies that the mechanical and political difficulty of actually executing a pause on command may mean waiting is riskier than believed.

   The author does not fully develop the alternative argument in the provided excerpt, indicating further reasoning about why immediate action might be preferable is incomplete in this extract.