Briefing on Extinction

“Artificial superintelligence” (ASI) refers to AI that can substantially surpass humanity in all strategically relevant activities (economic, scientific, military, etc.). The timeline to ASI is highly uncertain, but probably not long. On the present trajectory, MIRI would not be surprised if ASI is developed in two or five years, and we’d be surprised if it were still more than twenty years away. AI labs are aggressively rolling out systems they don’t understand. The deep learning techniques behind the rapid AI progress of the last few years create massive neural networks automatically. The resulting models are akin to black boxes. One can see what goes in, and what comes out, but all that is visible inside are trillions of numbers. AI developers can’t tell you why a modern AI makes a given choice. ( More ) Sufficiently intelligent AIs will likely develop p

ersistent goals of their own. In real-world environments, the best way to complete complex long-term tasks is to possess a very general ability and inclination to route around all kinds of obstacles and distractions, in unwavering pursuit of longer-term objectives. We are only barely starting to see this phenomenon in today’s AIs, which are primarily good at short-term tasks, but are now being adapted into agents with greater autonomy. ( More ) Developers appear to be far from being able to give ASIs goals of the developer’s choosing. Building ASIs to have the right goals is a large scientific challenge in its own right, distinct from the challenge of building ASI at all. Currently, the field seems very unlikely to find a robust solution to this problem in time to apply it to ASI. ( More ) An ASI without worthwhile goals would be very likely to cause our extinction. Unless it has worthwhile goals, ASI will attempt to acquire, centrally control, and utilize all available resources, in ways that are incompatible with our continued survival. This doesn’t require that the AI possess any human-style desire for dominance or autonomy. It only requires that the ASI be a competent goal-optimizer, for some goal other than the goals of its developers. ( More ) A “wait and see” approach to ASI is probably not survivable. A superintelligent adversary will not reveal its full capabilities and telegraph its intentions. It will not offer a fair fight. It will make itself indispensable or undetectable until it can strike decisively and/or seize an unassailable strategic position. ( More ) MIRI doesn’t see any viable quick fixes or workarounds to misaligned ASI. OpenAI, the developer of ChatGPT, admits that today’s most important methods of steering AI won’t scale to the superhuman regime. Attempts to restrain or deceive a superior intelligence are prone to fail, including in ways we can’t foresee. Plans to align ASI using weaker AIs are similarly unsound. We also don’t think a well-funded crash program to solve alignment would be able to correctly identify adequate solutions. Our current view is that a safe way forward will likely require ASI to be delayed for a long time. ( More ) Delaying ASI likely requires a globally coordinated ban on its development, including tight control over the factors of its production. This is a large ask, but domestic oversight, mirrored by a few close allies, will not suffice. This is not a case where we just need the “right” people to build it before the “wrong” people do, as ASI is not a national weapon; it is a global suicide bomb. If anyone builds it, everyone dies. To preserve the option of shutting down ASI development if or when there is sufficient political will, MIRI advocates promptly building the “off-switch.” The off-switch refers to the systems and infrastructure required to effectively and durably enforce restrictions on AI development and deployment. It starts with identifying the relevant actors, tracking the relevant hardware, and requiring that advanced AI work take place within a limited number of monitored and secured locations. It extends to building out the protocols, plans, and chain of command required to efficiently make decisions to impose such restrictions. As the off-switch could also provide resilience to more limited AI mishaps, we hope it will find broader near-term support than a full ban. ( More ) An off-switch can only prevent our extinction from ASI if it has sufficient reach and is

actually used to shut down development in time. If humanity is to survive this dangerous period, it will have to stop treating AI as a domain for international rivalry and demonstrate a collective resolve equal to the threat.

Show HN: – AI-generated podcast for curious kids

Latest episode No. 4 Aug 15, 2026· 6:27 Earthquakes, Rainforests, and Locked Data Sam and Leah explain a powerful earthquake in Indonesia, an operation against illegal gold mining in Peru, a French court decision about social media, Google’s encrypted-data tool, and a health response in Myanmar. Audio 1 In this episode * 0:01 Opening and preview * 0:33 Indonesia earthquake * 1:41 Peru Amazon mining operation * 2:36 France social-media ruling * 3:28 Google HEIR encrypted-data tool * 4:40 Myanmar health response * 5:51 Recap and closing question Open episode page → What is ? is a daily news podcast for curious kids. Sam and Leah explain important true stories in about ten minutes—clearly, factually, and without graphic detail. Every episode ends with a question to discuss together. Grown-ups can read how it’s made Recent episodes * N

o. 3 Aug 14, 2026· 5:49 Flights, Oil, and Shooting Stars Sam and Leah explain the C919 passenger jet’s first international flight, an oil slick on Iran’s coast, hot weather in parts of Europe, and the Perseid meteor shower. They separate what is confirmed from what people are still checking. * No. 2 Aug 13, 2026· 9:38 Votes, Trains, and a Black Hole Star Sam and Leah follow a planned coalition withdrawal from Iraq, Zambia’s election count, a train derailment near Lewes, a possible black-hole-star model, and Nine PBS’s effort to recover its archive. They ask how evidence changes when a plan, count, investigation, or recovery is still underway. * No. 1 Aug 12, 2026· 9:53 Earthquakes, Eclipses, and Camera Glasses Sam and Leah follow the evidence: an earthquake in Colombia, a total solar eclipse, health workers tracing an Ebola outbreak, Robert Gilman’s return from Russia, and a privacy complaint about camera glasses in Germany. Along the way, they ask why early reports change, how predictions work, and what makes recording clear and fair. All episodes → Listen every day New episodes arrive daily, including weekends. Follow in your podcast app so you never miss one. How to subscribe → RSS feed · About Built by

SpaceX has acquired coding AI startup Cursor

Cursor says it now has the computing power to develop low-cost AI models. SpaceX Cursor, the startup that became famous for its AI coding tool, is now officially owned by SpaceX. The companies announced the acquisition in June, but the process apparently started in April, when they teamed up for Cursor's model training efforts. In its announcement , Cursor said that it will now have access to "the largest fleet of GPUs in the world," which will give it the computing capacity to build and train better models that it can offer its customers at a lower cost. SpaceX, which merged with Elon Musk's AI company xAI earlier this year, paid $60 billion for the purchase. In July, xAI officially became SpaceXAI . Shortly after the new name was announced, the companies released Grok 4.5 , the first model SpaceXAI and Cursor buil

t together. It was designed to "excel at coding, agentic tasks and knowledge work" and to deliver results "at far lower costs." Since then, they have released Grok 4.6, which builds on 4.5. The newer model was trained to do real life tasks like general coding, web development and computer-aided design, among others. Cursor said Grok 4.6 provides an early look at what it can build together with SpaceXAI.

Telegram必备的搜索引擎，极搜JISOU帮你精准找到，想要的群组、频道、视频、音乐

👉 t.me/jisou2?start=a_8247614025