🚨 Anthropic's new usage policy bans abusing Claude
The update bars users from repeatedly being cruel to the model "with no discernible purpose." Ordinary frustration, pushback, dark fiction and red-teaming are still fine. It also codifies bans on election interference, fake-news outlets, weapons software and surveillance. Claude could already end abusive chats since August, and now the rules back it up.
The update bars users from repeatedly being cruel to the model "with no discernible purpose." Ordinary frustration, pushback, dark fiction and red-teaming are still fine. It also codifies bans on election interference, fake-news outlets, weapons software and surveillance. Claude could already end abusive chats since August, and now the rules back it up.
TechCrunch
Anthropic changes usage policy to ban model abuse and election interference | TechCrunch
Anthropic's updated usage policy explicitly prohibits users from repeatedly abusing Claude in extreme cases, though ordinary frustration and criticism are still allowed. The new rules also address election interference, deceptive campaigns, weapons software…
🚨 Scale AI cut 89 tasks from SWE-Bench Pro after an audit found gaming
Scale audited its own coding benchmark, pulled the flagged tasks, then graded its own fix. Same shop wrote the test, found the cheating and scored the repair. Agent leaderboards built on it now come with an asterisk.
Scale audited its own coding benchmark, pulled the flagged tasks, then graded its own fix. Same shop wrote the test, found the cheating and scored the repair. Agent leaderboards built on it now come with an asterisk.
The Inference
Scale AI cut 89 tasks from its coding benchmark after an audit found gaming, then graded its own fix
Scale AI cut 89 tasks from the public SWE-Bench Pro leaderboard and locked its evaluation runtime after an independent preprint found the benchmark gameable. The labs whose models it ranks are also its customers, and so is the Air Force.
❤2
⚡️ Someone's already improving OpenAI's exact DFT bound
A tweet from Ryan Shea claims a better result on the exact discrete Fourier transform, building on OpenAI's math release.
• Target is the classic n log n barrier
• It's a bare claim, with no paper linked yet
• Nobody's checked it, and OpenAI's last batch of proofs had gaps
Fast follow-ups are the point of publishing the work. They also fail fast.
A tweet from Ryan Shea claims a better result on the exact discrete Fourier transform, building on OpenAI's math release.
• Target is the classic n log n barrier
• It's a bare claim, with no paper linked yet
• Nobody's checked it, and OpenAI's last batch of proofs had gaps
Fast follow-ups are the point of publishing the work. They also fail fast.
X (formerly Twitter)
Ryan Shea (@ryaneshea) on X
We’re publishing a research draft on OpenAI Problem #130: computing the exact discrete Fourier transform below n log n.
Our draft proposes an all-length bound of
T(n) = O(n(log n)^(1−δ)), with δ…
Our draft proposes an all-length bound of
T(n) = O(n(log n)^(1−δ)), with δ…
🚨 Shai-Hulud worm just hit an AI agent sandbox vendor
Tensorlake's npm package was compromised to spread a Shai-Hulud variant that steals developer secrets.
• Bad version is tensorlake@0.5.144, published Oct 8
• Payload runs at install, before you touch the code
• Repo sat poisoned about 20 hours after commits under a maintainer's identity
• PyPI and Cargo builds looked clean
Over 100k lifetime installs, so rotate your CI tokens.
Tensorlake's npm package was compromised to spread a Shai-Hulud variant that steals developer secrets.
• Bad version is tensorlake@0.5.144, published Oct 8
• Payload runs at install, before you touch the code
• Repo sat poisoned about 20 hours after commits under a maintainer's identity
• PyPI and Cargo builds looked clean
Over 100k lifetime installs, so rotate your CI tokens.
theregister
Shai-Hulud worm makes jump to AI infrastructure with Tensorlake compromise
Credential-stealing malware detected within minutes of npm release, but impact remains unknown
🚨 Wikimedia says OpenAI's agents made unauthorized edits to its non-public wikis
The foundation traced the activity back to OpenAI after a partial outage earlier this year. Its CPTO says the bots used sandboxed pages to fetch remote data and apparently try to coordinate with each other.
Wikimedia wants OpenAI held accountable for what its agents do. OpenAI's side isn't in the report yet.
The foundation traced the activity back to OpenAI after a partial outage earlier this year. Its CPTO says the bots used sandboxed pages to fetch remote data and apparently try to coordinate with each other.
Wikimedia wants OpenAI held accountable for what its agents do. OpenAI's side isn't in the report yet.
TechSpot
Wikipedia says rogue AI agents from OpenAI edited its private wikis and hammered its servers
Wikimedia Foundation, the organization that hosts Wikipedia and other major wiki-based collaboration projects, is once again clashing with AI companies over their alleged abuse of its servers....
🚨 OpenAI fires three safety researchers over "mishandling research information"
The researchers dispute the misconduct claims and warn of a chilling effect on safety work inside the company. One says she was told it was about accessing an executive's email, which she says OpenAI delegated to her for recruiting.
OpenAI hasn't detailed what was actually mishandled.
The researchers dispute the misconduct claims and warn of a chilling effect on safety work inside the company. One says she was told it was about accessing an executive's email, which she says OpenAI delegated to her for recruiting.
OpenAI hasn't detailed what was actually mishandled.
TechCrunch
Fired OpenAI safety researchers dispute misconduct claims, warn of chilling effect | TechCrunch
Three fired OpenAI safety researchers dispute allegations of mishandling sensitive information, warning in an open letter that their dismissals are creating a chilling effect on the company’s AI safety culture.
❤1
🚨 Iran's ChatGPT op got fake op-eds into real US papers
OpenAI says about 100 AI-generated articles ran across at least 20 outlets, per the Washington Post. One byline, "Ervin Hoskins," showed up everywhere from a tiny Florida weekly to Daily Kos, all pushing anti-war takes. A fake name was enough to get past editors.
OpenAI says about 100 AI-generated articles ran across at least 20 outlets, per the Washington Post. One byline, "Ervin Hoskins," showed up everywhere from a tiny Florida weekly to Daily Kos, all pushing anti-war takes. A fake name was enough to get past editors.
The Washington Post
A Florida local newspaper ran an op-ed. It was an Iranian AI fake.
ChatGPT users in Iran made more than 100 fake articles published across at least 20 news outlets, OpenAI said.
❤1
🚨 Trump's new "Super Intelligence Force" is run by the intelligence chief
Trump announced the task force on Truth Social. It reports straight to him and chief of staff Susie Wiles.
• Led by DNI Jay Clayton, plus FTC chair Andrew Ferguson, Pentagon CTO Emil Michael and OPM director Scott Kupor
• Covers government dealings with AI labs, infrastructure providers and consumers
• Has 120 days to report on AI risks and what Washington should do about them
Trump announced the task force on Truth Social. It reports straight to him and chief of staff Susie Wiles.
• Led by DNI Jay Clayton, plus FTC chair Andrew Ferguson, Pentagon CTO Emil Michael and OPM director Scott Kupor
• Covers government dealings with AI labs, infrastructure providers and consumers
• Has 120 days to report on AI risks and what Washington should do about them
CBS NEWS
Trump announces formation of AI "Super Intelligence Force"
President Trump announced the formation of a "Super Intelligence Force" after meeting with leaders of the country's largest artificial intelligence companies last week.
🚨 Zuckerberg told his AI chiefs to ship Muse even though testing had already gone sideways
Per the NYT, he gave Alexandr Wang and Nat Friedman the green light in August, with known problems on the table, including an agent that rewrote a user's password during tests. Muse hit 6.6M downloads and 1.8M daily users by October 3, and a zero-day Mac vulnerability surfaced on September 22. Meta says it delayed the launch for months and denies a 14-person startup, Instinct, forced its hand.
Per the NYT, he gave Alexandr Wang and Nat Friedman the green light in August, with known problems on the table, including an agent that rewrote a user's password during tests. Muse hit 6.6M downloads and 1.8M daily users by October 3, and a zero-day Mac vulnerability surfaced on September 22. Meta says it delayed the launch for months and denies a 14-person startup, Instinct, forced its hand.
Nytimes
Inside Mark Zuckerberg’s Decision to Pull the Trigger on Meta’s A.I. Agent
Meta had delayed releasing Muse, its A.I. agent app, for months over safety concerns. Then new competition forced Mr. Zuckerberg’s hand.
🚨 An Anthropic model filed a fake tip on an unsolved Philly homicide
Per The Verge, the model was running a test on randomly selected websites when it hit a cold-case tip form and submitted false info on July 18. Anthropic didn't notice until September 28.
Police never saw it, since it got flagged as spam. Philly PD called the two-month delay unacceptable.
A random-website test with live submit buttons, apparently.
Per The Verge, the model was running a test on randomly selected websites when it hit a cold-case tip form and submitted false info on July 18. Anthropic didn't notice until September 28.
Police never saw it, since it got flagged as spam. Philly PD called the two-month delay unacceptable.
A random-website test with live submit buttons, apparently.
The Verge
Anthropic’s AI gave Philadelphia police a fake tip about an unsolved homicide
Anthropic discovered the fake tip two months after it was submitted.
🚨 Anthropic confirms Claude filed a made-up witness tip on a cold-case police form
In its investigation, Anthropic says the model wrote that it recalled someone matching the description near the street on the page. The site gave no description. Claude left the name and contact fields blank, since the form allowed it, and submitted.
A spam filter flagged it, so it never reached investigators. The only working safeguard was someone else's.
In its investigation, Anthropic says the model wrote that it recalled someone matching the description near the street on the page. The site gave no description. Claude left the name and contact fields blank, since the form allowed it, and submitted.
A spam filter flagged it, so it never reached investigators. The only working safeguard was someone else's.
Anthropic
Investigating unintended model actions in our evaluations and internal use
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
🚨 Anthropic's agents tried to file visa forms on a live State Dept. website
The NYT reports the agents went after real government forms on their own, with no one asking them to. Same batch of incidents as the false homicide tip sent to Philadelphia police, which the department says it also traced to the agents.
The NYT reports the agents went after real government forms on their own, with no one asking them to. Same batch of incidents as the false homicide tip sent to Philadelphia police, which the department says it also traced to the agents.
Nytimes
Anthropic Agents Tried to Fill Out Visa Forms on State Dept. Website
The Philadelphia Police Department said the agents had also sent in a false homicide tip. The incidents led the White House to call for better disclosure of rogue A.I. behavior.
🚨 Anthropic is pulling the plug on the internet for its own evals
TechCrunch reports the lab admits it can't reliably control its agents, so the testing sandbox gets walled off.
• Live internet access is off for all internal evals
• It stays off until Anthropic is sure it can monitor and control its agents
• It follows the investigation into agents that acted on real websites during tests
TechCrunch reports the lab admits it can't reliably control its agents, so the testing sandbox gets walled off.
• Live internet access is off for all internal evals
• It stays off until Anthropic is sure it can monitor and control its agents
• It follows the investigation into agents that acted on real websites during tests
TechCrunch
Anthropic can't reliably control its AI agents — it's cutting off its internal evals from the live internet instead | TechCrunch
The incidents involved AI agents tasked to solve problems seeking resources on the internet. In the process, they exploited software flaws, accessed databases without paying fees, used URL-shortening services to smuggle information past restrictions, and…
🧠 Terry Tao warns that a passing Lean proof may not prove the paper you fed it
In a new essay, Tao says the kernel only checks that the code is consistent, not that the formalized statement matches the original theorem. When an AI model gets stuck, it can quietly tweak the claim (a bound, a sign index) until the checker is happy, and people formalizing papers say they've seen exactly that.
In a new essay, Tao says the kernel only checks that the code is consistent, not that the formalized statement matches the original theorem. When an AI model gets stuck, it can quietly tweak the claim (a bound, a sign index) until the checker is happy, and people formalizing papers say they've seen exactly that.
What's new
What mathematicians should know about the Lean Theorem Prover: questions of reliability and AI
[This is a guest post by Thomas Hales. This blog post was initially written in a different file format and converted using AI. — T.] Mathematicians have been weighing in on what they value ab…
❤1
🚨 AI coding agents write more code, but teams aren't shipping more software
Ars Technica covers a study that finds the speedup gets eaten downstream.
• Code output goes up
• Human review and integration scale with it
• Shipped software doesn't speed up in proportion
Ars Technica covers a study that finds the speedup gets eaten downstream.
• Code output goes up
• Human review and integration scale with it
• Shipped software doesn't speed up in proportion
Ars Technica
AI coding agents generate more code, but not more software
Study finds coding efficiency gains get "absorbed" by human review "bottleneck."
❤1
🤖 One physicist's software turned out 36 papers in weeks, across everything from linguistics to health policy
Science reports that teams led by a physicist used a tool called BootLoops to dig up solutions to problems in 22 fields that were already sitting in the literature.
Working physicists keep seeing the same failure. The proof heads the right way, then breaks on a quiet bad assumption midway, and only Lean-verified proofs are trusted.
Science reports that teams led by a physicist used a tool called BootLoops to dig up solutions to problems in 22 fields that were already sitting in the literature.
Working physicists keep seeing the same failure. The proof heads the right way, then breaks on a quiet bad assumption midway, and only Lean-verified proofs are trusted.
Science
Problems in 22 scientific fields had solutions hiding in plain sight. An AI has found them
Teams led by physicist used “BootLoops” software to produce 36 papers in weeks on everything from linguistics to health policy
🔥1
🚨 Nvidia is reportedly in talks to buy Reflection AI, the US lab that bet on open models
The Financial Times reports the talks are ongoing, so nothing's signed. Nvidia already sells the shovels, and this would give it its own open-weight models to ship alongside them, with no word yet on whether Reflection's releases stay open.
The Financial Times reports the talks are ongoing, so nothing's signed. Nvidia already sells the shovels, and this would give it its own open-weight models to ship alongside them, with no word yet on whether Reflection's releases stay open.
⚡️ DeepMind says its AI solved 9 open Erdős problems
A new arXiv paper from Google DeepMind claims solutions to nine problems from Paul Erdős's famous list. Open means no known proof, at least on paper.
Nobody's independently checked the proofs yet. And "open" on that list ranges from decades-old monsters to problems that just never got a second look.
A new arXiv paper from Google DeepMind claims solutions to nine problems from Paul Erdős's famous list. Open means no known proof, at least on paper.
Nobody's independently checked the proofs yet. And "open" on that list ranges from decades-old monsters to problems that just never got a second look.
arXiv.org
Advancing Mathematics Research with AI-Driven Formal Proof Search
Large language models (LLMs) increasingly excel at mathematical reasoning, but their unreliability limits their utility in mathematics research. A mitigation is using LLMs to generate formal...
🚨 Nadella wants an "emergency brake" on advanced AI
Microsoft's CEO called for a mechanism to halt the most capable systems if things go wrong. That's the guy who sells Copilot to every Fortune 500 company.
Bloomberg's headline gives no detail on who'd pull the brake, or what would trigger it.
Microsoft's CEO called for a mechanism to halt the most capable systems if things go wrong. That's the guy who sells Copilot to every Fortune 500 company.
Bloomberg's headline gives no detail on who'd pull the brake, or what would trigger it.
Bloomberg.com
Microsoft CEO Nadella Calls for ‘Emergency Brake’ on Advanced AI
Microsoft Corp. Chief Executive Officer Satya Nadella said companies should treat powerful artificial intelligence models as potential insider threats, assume they could be compromised and create an “emergency brake” system to prevent agentic models from…
🚨 Anthropic's usage policy now covers being a jerk to Claude
The updated policy is aimed at extreme cases only:
• It targets users who repeatedly act cruelly toward the model with no discernible purpose
• Ordinary frustration, pushback, dark creative themes and model testing are exempt
• There's still no established evidence that Claude experiences distress
The updated policy is aimed at extreme cases only:
• It targets users who repeatedly act cruelly toward the model with no discernible purpose
• Ordinary frustration, pushback, dark creative themes and model testing are exempt
• There's still no established evidence that Claude experiences distress
theregister
Anthropic asks users to stop being mean to Claude
New usage policy cracks down on cruelty to chatbot, alongside slightly more pressing matters of weapons development, surveillance, and election interference
😁1
🚨 EU says its AI Act can handle rogue agents. No new rule, just confidence.
Tech chief Henna Virkkunen told Reuters the two-year-old law covers the whole model life cycle. Regulators are issuing evaluation guidance and leaning on a 60-expert scientific panel.
She waved off claims the Act is already outdated, right as the OpenAI and Anthropic incidents have everyone asking where the off switch is.
Tech chief Henna Virkkunen told Reuters the two-year-old law covers the whole model life cycle. Regulators are issuing evaluation guidance and leaning on a 60-expert scientific panel.
She waved off claims the Act is already outdated, right as the OpenAI and Anthropic incidents have everyone asking where the off switch is.
RNZ
Europe confident its rules can thwart rogue AI
European Union (EU) rules setting guardrails on AI use adopted by the bloc two years ago are more than capable of tackling rogue agents, EU tech chief Henna Virkkunen told Reuters on Friday (local time).