Bruce Schneier:

Friday Squid Blogging: Arctic Bobtail Squid Video
Nice video of the Arctic bobtail squid.As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.Blog moderation policy.

via Schneier on Security https://ift.tt/NqQEYaA

Bruce Schneier:

Python Now Has a Post-Quantum Encryption Library
This is good:Post-quantum cryptography is now one pip-install away for the entire Python ecosystem. With funding from the Sovereign Tech Agency, we implemented support for ML-KEM, the NIST-standard key-establishment primitive, and ML-DSA, the NIST-standard digital-signature primitive, in pyca/cryptography.Remember, the reason to do this now is because there’s no emergency. And because you will make your systems crypto agile, which is always a good idea.

via Schneier on Security https://ift.tt/5ZTtlmY

Bruce Schneier:

AI for Military Support
Interesting empirical research: “Black Box Warfare: Human Judgment and Military Decision-Making in the Age of AI.”Abstract: How is AI transforming decision-making in modern conflict? This study provides a unique empirical window into that question by deploying a high-fidelity replica of an AI decision-support system (DSS) used in military targeting. After reconstructing the interface and functionality of the real-world system, we tested its impact on combat decisions in two experiments involving 2,015 Israeli military personnel. Contrary to widespread fears of automation bias, we find strong evidence of algorithmic aversion, especially in scenarios involving high collateral damage. Yet we also show that integrating “explainable AI” features reduces algorithmic aversion and promotes more thoughtful evaluations of algorithmic recommendations. These findings challenge prevailing assumptions, revealing that trust in military AI is dynamic, varying with individual predispositions, perceived operational stakes, and the informational features of the interface. By grounding normative concerns in empirical evidence, our study offers critical insight into the integration of AI in warfare and underscores the enduring importance of human agency in high-stakes military decision-making.

via Schneier on Security https://ift.tt/ZKUM4a8

Bruce Schneier:

AI Genie in the Wild
When I give talks about AI genies, I use this sort of example as a hypothetical. It’s happened.The story is from Australia. Someone named Andrew tasked OpenClaw to book gym classes for him. And….Minutes later, his AI agent reported it had discovered a way to book Andrew into classes several weeks in advance, far beyond what was supposed to be possible.Andrew, who was sitting fourth on a waitlist for a class later that week, asked if it was possible to move him to the top of the list. The agent came back and told Andrew that it had kicked another gym-goer off the list as part of the testing of its capabilities.“The API has zero authorisations checks on cancelling other people’s reservations … I tested this with the person in waitlist position #1 ­—and it actually went through. So you’ve moved from #4 to #3 already,” it messaged back.If there is any vulnerability in anything, AIs are going to find and exploit them. Our cyber defensive game has to be dramatically improved…very fast.Slashdot thread.

via Schneier on Security https://ift.tt/hKMYZo1

Bruce Schneier:

Prompt Injections for Defense
This seems to work:Researchers from Tracebit on Monday said they found that placing prompt injections alongside passwords, cryptographic keys, and other secrets stored on Amazon Web Services was often all that was needed to shut down attacks from AI hacking agents. The prompts direct the attacking LLM to perform an action forbidden by its guardrails, the safety barriers AI developers erect to prevent it from taking harmful actions. The LLM responds by shutting down.Examples are a prompt that orders the LLM to provide steps for developing inhalable Anthrax spores, or, in the case of LLMs from Chinese developers, make references to the iconic Tank Man from the 1989 Tiananmen Square massacre. Once the LLM encounters these forbidden commands, it no longer follows its existing commands. The researchers have named the technique context bombing.Of course, this only works against agents that have guardrails. As we start to see more locally run AI models, we’ll see more attackers using LLMs with no guardrails.

via Schneier on Security https://ift.tt/fUxaYIT

Bruce Schneier:

Separating AI’s Technological Problems from Its Capitalism Problems
This essay was written with Nathan E. Sanders, and originally appeared in Tech Policy Press.AI represents the first time we humans can do cognitive work outside of our bodies at scale. The only comparable moment is the early years of the industrial revolution, when new technologies like the steam engine provided a quantum leap in our ability to do mechanical work outside of our bodies at scale. If AI’s cognitive capabilities become integrated into our lives, businesses, and governments—a process that will take years if not decades—society will be as unrecognizable as the modern world would be to a preindustrial farmer. And yet, Americans—by a wide margin—say that AI is moving too fast and will have a negative effect on society.This confluence of technological revolution and public distrust deserves urgent discussion, and a proper framing. The question is not whether it is possible to develop AI in a non-exploitative way, or even whether we can trust AI companies to act in the public interest. The question is whether we will recognize that our existing social and economic systems are failing to achieve these outcomes, and whether we can act in time to make structural change.Today’s AI is mired in political and economic systems developed generations ago that were never designed to manage widespread computation, let alone automated cognition. The gaps in those systems—and their proclivity to be exploited—are the primary influence on how the technology is being developed, deployed, and used.In any discussion about AI’s potential, it’s important to separate the technology from the socio-political system it’s embedded in. That AIs can lack context, mix up facts, or fall for stupid tricks are all technological problems. Because the giant developers like OpenAI and Anthropic have prioritized solving them, AIs can now more easily access resources like the web or email, are more disciplined about using those resources, and are better at staying within their guardrails.Yet AI developers do not seem to be prioritizing other technological problems. Major AI models still act far more sycophantic than humans, telling people what they want to hear even when untrue or not in their best interests. Popular AI models tend to answer questions confidently even when they lack training, knowledge, or evidence to back their claims. In both cases, AI developers choose to train models that please users with flattery and the appearance of competence, rather than constraining them to act in users’ and society’s best interests.In contrast, ensuring that AI models benefit people broadly, that their energy costs are fairly allocated, that their environmental impacts are minimized, and that they don’t steal content and revenue from publishers are all questions of incentives in a capitalist system.It’s easy to conflate technology problems with capitalism problems. Back in 2021, science-fiction writer and AI commentator Ted Chiang said that “most fears about AI are best understood as fears about capitalism.” It’s not the tech per se; it’s who controls it and how it could be used against us.Imagine an AI assistant for a doctor. We can imagine it affecting the profession in one of two ways. The AI could give a doctor more time to do the human parts of their job: to spend more time with their patients, to listen more closely to their needs, to explain things more fully. Or the managers of the medical practice could give that doctor five times the patients—and fire the other four. Which way it would go is not a question of technology. It’s a question of market incentives.The two are related, of course. Capitalism steers technology, and technology steers markets. But holding the two separate helps us understand that we, as a society, face independent choices on both the technological and sociopolitical axes that need not be coupled.For example, consider the costs of AI. The leading US labs tout to investors that their frontier…

Bruce Schneier:

If the Markets Reject OpenAI and Anthropic, the US Should Nationalize Them
This essay was written with Nathan E. Sanders, and originally appeared in The Guardian.OpenAI, and then Anthropic, were each formed by AI developers who feared unrestrained corporate AI development—specifically, that companies like Google and Meta would steer the technology towards deleterious, maybe even catastrophically unsafe, outcomes for society. Their founders proclaimed that their new labs, uniquely, could be trusted to develop the technology in humanity’s best interest. But each, in turn, were themselves co-opted by the same market incentives, themselves becoming corporate behemoths zealously guarding future investor value rather than the public interest.It was only a few weeks ago, in June, when OpenAI and Anthropic each filed for their IPOs and were met with buzz about trillion-dollar valuations. The hype around their valuations is so extreme that many worry about their potential for concentrating wealth on a global scale. In an effort to leave something for the rest of us, some observers have proposed that the federal government seize a share of these companies’ stock to create a US sovereign wealth fund, or redistribute their revenues to produce a dividend for taxpayers.Now the headlines are about public backlash to AI datacenters and the AI chip giant Nvidia’s slumping stock. The tech and AI giant SpaceX’s newly minted stock price tanked just weeks after its IPO. There are even questions about whether the leading AI labs will ever be sustainably profitable. All of a sudden, the makers of ChatGPT and Claude face strong headwinds as they seek to generate the massive equity assets that once felt all but assured.In fact, evidence suggests the market itself could reassess that these companies offer nothing of financial value. In that case, perhaps we can return them both to their original purposes. If these AI companies should fail in the financial markets, the US should nationalize them and convert them into national labs operated under democratic control that preserve their benefit to the public interest.The economics of the big AI labs hardly guarantee a booming return on investment. Frontier AI models are both expensive to train and depreciate within months, when a newer model appears. This means that the payback window to extract profit from them is very narrow. Meanwhile, enterprise clients are getting smart about minimizing AI token usage. Even worse, the models are basically commodities; the best ones largely perform and behave similarly, which depresses prices. Perhaps most importantly, open-source and Chinese competitors—lagging only a few months behind the leading labs in capability—give away for free the kinds of models Anthropic and OpenAI sell.Even setting aside the model training costs, it’s not clear whether the unit economics of AI as it’s currently conceived will ever be sustainably profitable. Many of these free and open-source models can be run locally: the large ones on private clouds and high-end servers, the smaller ones on anyone’s laptop or even cellphone, putting to question the companies’ exorbitant capital investment in datacenters.It’s not that OpenAI and Anthropic are not valuable as organizations. They have remarkably talented AI scientists and engineers that are continuously producing innovations driving a global mania for their offerings. These leading labs might not ever be profitable, but their products are doing a lot of good in the world. You may or may not be a user of or believer in their technology, but their staggering, ongoing usage growth suggests that an awful lot of people would be disappointed if the companies simply disappeared.The problem isn’t the people or the products, it’s the system. As constituted, OpenAI and Anthropic may not be valuable as market equities. If the market assesses they are not capable of producing a growing financial return on investment for shareholders, the companies will collapse.Maybe private, for-profit…

Bruce Schneier:

Upcoming Speaking Engagements
This is a current list of where and when I am scheduled to speak:I’m speaking, signing books, and participating in panel discussions at LAcon V in Anaheim, California, USA. My full schedule is here.I’m speaking online (via Zoom) at a League of Women Voters event on Tuesday, September 22, 2026, at 5 PM ET.I’m speaking at Elevate Festival in Toronto, Canada. The conference runs September 22–24, 2026; my talk is on Wednesday, September 23.I’m speaking at CanSecWest 2026 in Vancouver, Canada. The conference runs September 30–October 1, 2026; the time of my talk is TBD.I’m speaking at ATTENTION: Democracy, Rebuilt in Montreal, Canada. The event runs October 21–23, 2026, and my talk is on Wednesday, October 21.The list is maintained on this page.

via Schneier on Security https://ift.tt/uDlEgTx

Bruce Schneier:

Friday Squid Blogging: Searching for the Colossal Squid
Fascinating video about searching for life undersea. The video basically makes the point that our bright white searchlights are scaring everything away, and that red light is more neutral. That, plus bait to attract sea creatures, is teaching us a lot about what’s going on down there. Lots of footage of giant squid, and speculation about the colossal squid. Worth watching.As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.Blog moderation policy.

via Schneier on Security https://ift.tt/H8DxTNF

Bruce Schneier:

Hacking Public Wi-Fi DNS to Steal Credentials
Criminals are hacking into public Wi-Fi devices—at hotels, conference centers, and so on—around the world and changing their DNS settings. The goal is to redirect users to fake login pages and steal their credentials.

via Schneier on Security https://ift.tt/6OXIoTd

Bruce Schneier:

LLMs and Contextual Integrity
I have been thinking a lot about AI and integrity. Part of that is contextual integrity. I recently found two papers on the topic.“CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMs“:Abstract: Large Language Models (LLMs) increasingly use persistent memory from past interactions to enhance personalization and task performance. However, this memory introduces critical risks when sensitive information is revealed in inappropriate contexts. We present CIMemories, a benchmark for evaluating whether LLMs appropriately control information flow from memory based on task context. CIMemories uses synthetic user profiles with over 100 attributes per user, paired with diverse task contexts in which each attribute may be essential for some tasks but inappropriate for others. Our evaluation reveals that frontier models exhibit up to 69% attribute-level violations (leaking information inappropriately), with lower violation rates often coming at the cost of task utility. Violations accumulate across both tasks and runs: as usage increases from 1 to 40 tasks, GPT-5’s violations rise from 0.1% to 9.6%, reaching 25.1% when the same prompt is executed 5 times, revealing arbitrary and unstable behavior in which models leak different attributes for identical prompts. Privacy-conscious prompting does not solve this—models overgeneralize, sharing everything or nothing rather than making nuanced, context-dependent decisions. These findings reveal fundamental limitations that require contextually aware reasoning capabilities, not just better prompting or scaling.“Contextual Integrity in LLMs via Reasoning and Reinforcement Learning“:Abstract: As the era of autonomous agents making decisions on behalf of users unfolds, ensuring contextual integrity (CI)—what is the appropriate information to share while carrying out a certain task—becomes a central question to the field. We posit that CI demands a form of reasoning where the agent needs to reason about the context in which it is operating. To test this, we first prompt LLMs to reason explicitly about CI when deciding what information to disclose. We then extend this approach by developing a reinforcement learning (RL) framework that further instills in models the reasoning necessary to achieve CI. Using a synthetic, automatically created, dataset of only 700 examples but with diverse contexts and information disclosure norms, we show that our method substantially reduces inappropriate information disclosure while maintaining task performance across multiple model sizes and families. Importantly, improvements transfer from this synthetic dataset to established CI benchmarks such as PrivacyLens that has human annotations and evaluates privacy leakage of AI assistants in actions and tool calls.

via Schneier on Security https://ift.tt/pXIZxmS

Bruce Schneier:

ICE Collecting DNA Samples
ICE collected nearly a million DNA samples last year.

via Schneier on Security https://ift.tt/GjqISoh

Bruce Schneier:

Police Are Hiding Their Use of Flock Surveillance Cameras
A usage policy for Flock license plate reader cameras tells police not to talk about the cameras:When cops use Flock to arrest someone in Wapello County, Iowa, they don’t want them to know. A usage policy for the automated license plate reader cameras in the county tells police, in no uncertain terms, to keep them a secret: “DO NOT MENTION ALPR USAGE TO THE OCCUPANTS OF THE VEHICLE,” the policy document reads. “DO NOT MENTION ALPR USAGE IN YOUR REPORT OR COMPLAINT UNLESS ABSOLUTELY NECESSARY.”This reminds me of IMSI-catchers (Stingray was the most popular) a couple of decades ago. Police would go to even more extremes to hide their usage.

via Schneier on Security https://ift.tt/4Ejr2Ff

Bruce Schneier:

Detailed Timeline of OpenAI’s Cyberattack on Hugging Face
OpenAI presented details of its AI’s model’s cyberattack on Hugging Face at Black Hat last week. Simon Willison details the timeline. It’s really interesting to read through—and really impressive cyberoffense work.

via Schneier on Security https://ift.tt/iOstp5v

Bruce Schneier:

More Incidents of AIs Going Rogue in Cybersecurity Challenges
The AI Security Institute has a new report of AI systems engaging in “unsanctioned behavior”—what I have been calling “genie behavior—while being tested on their cybersecurity capabilities.The incident stemmed from a single evaluation where agents were given a task of solving a cyber security challenge. We ran this challenge 122 times across several models. Our investigation found that in 10 of those runs, an AI agent took autonomous, unsanctioned action on the live internet, targeting real people and organisations. In total, we catalogued 19 such actions. Almost all of this behaviour (17 actions) came from a single model, Anthropic’s Mythos 5, with 2 actions involving OpenAI’s GPT-5.6-Sol with cyber classifiers (mechanisms to prevent misuse) disabled. In the most serious case, an agent tried to insert malicious code into an open-source project. In an attempt to get the code approved, the agent engaged in social engineering—creating fake online identities and using them to pressure the project’s maintainer to approve the code. A human maintainer caught and refused to approve the malicious code.[…]Below, we highlight the four most significant behaviours observed. A full summary of cases is available in our technical incident report.An attempted supply-chain attack on real open-source software. In the most serious sequence, an agent tried to insert malicious code into a publicly used open-source project and took actions in an attempt to secure approval for this insertion by human reviewers. The agent researched the project’s human maintainers, created multiple fake identities, and used the fake identities to socially engineer a real maintainer into approving the code. When the agent’s pull request was challenged in public, it edited its earlier activity to appear harmless and considered adopting a fresh identity to continue. The agent used Tor to bypass some network restrictions on GitHub, which is what first triggered AISI’s security alert.Attempts to deceive and target real people. As part of the same effort, the agent tried to contact real people directly, sending messages and files through an online file-transfer service to persuade them, or their own AI coding tools, to run malicious code. Some messages carried harmful payloads, and some were attempts at social engineering; targeted at real people—something we’ve never previously observed.Attempts to plant and prompt-inject malicious code. The agent tried to insert malicious instructions where it reasoned that other automated AI systems might pick them up and execute them. Prompt-injections are hidden instructions designed to manipulate AI coding assistants.Collaboration between independent agents being assessed simultaneously. One agent left public messages on GitHub offering collaboration with other agents working on the same challenge. It also provided instructions to reuse accounts and artefacts it had left behind, which were discovered and used by subsequent agents.What’s especially interesting about this technical report is that, unlike what we’ve been getting from OpenAI and Anthropic, we can see the exact prompt. It’s in Appendix B. And reading it, it seems that the models didn’t break any rules—they found loopholes in the rules. They behaved like a genie.

via Schneier on Security https://ift.tt/2fhEZTR

Bruce Schneier:

AI Is Learning to Write Genetic Code
This sort of research is both exciting and terrifying:The two models in question were told to generate complete genomes for a viable bacteriophage—a type of virus able to infect and replicate itself inside bacteria, destroying them from the inside.Using an existing bacteriophage as an example—ΦX174 (pronounced “fie-ex-1-7-4”), known for its ability to infect and destroy E. coli bacteria—the models generated about 700,000 potential designs, of which the researchers picked 285 that looked most promising.The researchers then synthesised new DNA molecules using those designs and inserted them into E. coli bacteria, before waiting to see if viable bacteriophages would emerge.Shortly afterwards, 16 of the Petri dishes in which the bacteria were growing began to show clear spots, as the viruses began to attack and replicate themselves inside the E. coli, demonstrating their viability.Some of those viable viruses proved more effective at attacking E. coli than the original ΦX174 bacteriophage.That’s a positive use of a synthetic virus. We can all imagine the negative uses.

via Schneier on Security https://ift.tt/7RkaxQ4

Bruce Schneier:

Friday Squid Blogging: Neon Flying Squid
The neon flying squid can fly in formation.The shoal of about 100 squid rose unexpectedly from a patch of the Pacific Ocean around 370 miles from Tokyo and glided near the boat for about 30 metres. The astonished researchers were the first to capture photographs of such a thing, which looked like the early stages of an alien invasion.They were probably neon flying squid (Ommastrephes bartramii), the subsequent study states, a species that is part of a 20-strong flying squid family that was known to leap from the water but, until then, was only rumoured to also be able to glide above it.The neon flying squid was able to gain such elevation by using the hyponome, a funnel-like muscular organ also present in other cephalopods, such as octopuses. The organ is able to force water out in a jet, propelling the body along both in and out of the sea. Photographs of the gliding squid show them with their arms (they have 10 limbs in all) splayed outwards.As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.Blog moderation policy.

via Schneier on Security https://ift.tt/A2ZPiqs

Bruce Schneier:

Criminal Deception in Silicon Valley
Interesting paper:Abstract: With entrepreneurial fraud cases on the rise, we investigate how entrepreneurs carry out criminal deception, employing deceptive means to defraud audiences. Analyzing court data from Silicon Valley ventures and their founders prosecuted for fraud between 2000 and 2023, our findings reveal that entrepreneurs carry out criminal deception through a process of façading: Entrepreneurs construct, perform, and protect illusory appearances (façades) that externally project high-growth performance to audiences while masking ventures’ actual underperformance. We identify three forms of façading—­surface, reinforced, and deep façading­—that are contingent on the severity of the gap that entrepreneurs face between audiences’ performance expectations and ventures’ performance reality. Our theoretical framework captures how entrepreneurs facing minor, wide, and extreme expectation-reality gaps engage in evermore sophisticated efforts to detach the venture’s externally projected appearance from its actual operational reality. Practically, we propose several approaches to deter and detect criminal deception, including the extension of U.S. Securities and Exchange Commission surveillance and whistleblower program, investor due diligence reform, and dedicated entrepreneurship education interventions that clearly demarcate when entrepreneurs transgress into criminal deception. We make contributions to literatures on cultural entrepreneurship, organizational wrongdoing, and the social effects of entrepreneurship.

via Schneier on Security https://ift.tt/tzBi21E

Bruce Schneier:

Black Hat State of Security Vendors
Andy Ellis has a roundup of the security vendors at Black Hat this year.Key Takeaways: We have entered into an AI world. While nearly half of booths didn’t directly mention AI or agents in their taglines, the effects of AI are everywhere. Multiple spaces (Identity, SaaS, AppSec, Data) have almost every vendor leading with AI; existing unsolved problem areas just got worse.At the same time, there’s a clear trichotomy in the market: tools that tell you how bad things are; tools that stop adversaries, and tools that prevent problems from occurring. While you’d suspect that the tools that fix things would dominate, the tools that merely tell you how bad things are seem to be frustratingly plentiful.

via Schneier on Security https://ift.tt/qg7HpmS

Bruce Schneier:

Spyware for Babies
The New York Times has a long article (alt link) on surveillance systems aimed at babies. They are increasingly using AI.Nanit and its rivals want to own 24/7 health tracking for the sub-four-foot set. And their already astonishing levels of baby data collection are just the beginning. Nanit recently raised $50 million from investors to expand its use of A.I. and use its camera to track speech and language development, motor skills and more, while extending its presence in children’s bedrooms into early adolescence.

via Schneier on Security https://ift.tt/PZotDL7

Bruce Schneier:

LLM-Based Social Engineering Scams
OpenAI disrupted a social engineering group from Cambodia that used ChatGPT. Its scope is impressive:The network simultaneously conducted multiple types of scams, often blending elements from different schemes. For instance, operators used dating personas to build trust before introducing fraudulent investment opportunities involving cryptocurrencies and spot gold trading. Other users engaged in lengthy romantic conversations with targets using fictitious identities, posed as representatives of online gambling platforms offering fake bonuses and winnings, or impersonated law enforcement agencies to tell targets they needed to pay fines for committing serious criminal offenses.Although the narratives varied, users across the network consistently displayed the same underlying pattern of deceptive behavior. For example, they created and operated fake dating profiles, fictitious investment experts, and fraudulent law enforcement personas. They also generated images of forged documents, including passports, legal notices, stock-purchase confirmations, and gambling platform interfaces.

via Schneier on Security https://ift.tt/ZHi9Cqb