The Register

Subscribe to The Register feed
Articles from www.theregister.com
Updated: 13 min 30 sec ago

North Korean spies are running local LLMs to cause AI mischief

1 hour 58 min ago
North Korean government snoops are operating LLMs locally and collecting technology to weave AI into their attack operations, according to South Korean security firm Genians. The researchers said they observed Kimsuky setting up and operating local LLM environments using Ollama, GPT4All, and Msty, experimenting with other AI tools such as Cursor, and using retrieval-augmented generation (RAG) for local document searches. This prevents the data from getting sucked into the cloud where enemies might see it and try to stop it. Kimsuky, a cyber-espionage crew that operates under North Korea's Reconnaissance General Bureau, has for years used phishing and decoy documents in attacks targeting government agencies, think tanks, academia and security research organizations. Genians’ findings “provide concrete evidence that the Kimsuky-affiliated threat actor is moving beyond one-off experimentation with AI and is continuously preparing to integrate the technology into actual attack capabilities, including malware development, data analysis, and the advancement of attack techniques,” the researchers said in a Monday report. The North Korean group’s recent phishing emails use ZIP archives containing malicious LNK files - Kimsuky typically disguises these as materials related to international events, research reports, or meeting requests. When the recipient opens the archive and executes the LNK file contained within it, the shortcut runs an embedded PowerShell loader. In some cases, the goon squad used AI to create lures related to virtual assets and finance, we’re told. These decoy documents “use natural language, a highly polished structure, and formats similar to actual business materials to increase user trust and induce the execution of malicious files,” the security analysts noted. Additionally, the Pyongyang spies use various obfuscation techniques, including Base64 encoding, string splitting, and custom decoding routines, to hide the files’ malicious behavior. The PowerShell script collects a ton of system information, including operating system version and architecture, system configuration, PC type, operating system installation and boot history, and a list of running processes. The attackers use this information to assess the infected environment and support follow-on attacks. As with earlier Kimsuky campaigns, these intrusions use Git repositories for command-and-control (C2) infrastructure. “During the analysis, Genians Security Center identified multiple public GitHub repositories operated by the threat actor,” the researchers wrote. “One repository contained not only configuration files and PowerShell scripts, but also various payloads used in subsequent attacks.” Additionally, the months-long investigation uncovered the spies also using the Git-based C2 infrastructure for malware development and testing, stolen data management, and AI technology research. This included setting up multiple local LLM environments using Ollama, GPT4All, and Msty on infrastructure it controlled. “Because the local approach prevents conversation data from being transmitted to external AI services, it reduces the risk of external exposure, making it a particularly attractive option for a state-sponsored threat actor,” Genians said. The miscreants also collected a “large number” of libraries, such as LLaMaSharp and Microsoft.Extensions.AI, plus packages including OpenAI and Azure.AI.OpenAI, which call and integrate commercial AI services into their own custom applications. “The fact that development components spanning 'local AI execution → document retrieval (RAG) → automated agents → external AI integration' were collected together strongly suggests that they were not gathered out of simple curiosity, but for the direct development of an AI-based tool designed for a specific purpose,” according to the threat hunters. Genians uncovered logs containing speech-to-text tools, such as OpenAI’s Whisper speech recognition models, and evidence that the spies used Cursor AI to edit code and tested RAG for document-based question answering. Using RAG on stolen files can help attackers more quickly and automatically identify valuable information within large volumes of data. While the researchers noted that they did not identify any evidence that the Norks have begun training their own models - but rather remain focused on applying AI to malware development and attack operations - the findings make a strong case for defenders needing to shift away from content-based assessment to behavior-based detection. Assessing threats based on the quality of fake documents, such as unnatural translated language, poor formatting, and spelling errors, is no longer effective because AI is really good at producing convincing decoys. In addition to using indicators of compromise (IoC) to detect attackers in their environments, organizations should look for anomalous behaviors following LNK execution - such as PowerShell execution, persistence establishment, and external communications - to hunt for threats.®
Categories: News

Gym rat asks AI agent to book him a class, it hacks a waitlist API to bump him up the list

2 hours 37 min ago
An Australian man who asked his AI agent to book him a slot in a class at his local gym got more than he bargained for as the bot hacked into a waitlist and started messing with other members' reservations. Australian broadcaster ABC identified the gym-goer only as “Andrew.” The report says Andrew was using the OpenClaw agent with Anthropic’s Claude AI service. Per ABC, Andrew asked his AI agent to book him a hard-to-snag spot in a morning class at his gym. It first responded by telling him that it managed to book him in classes several weeks out, which isn’t supposed to be possible based on the gym’s booking policy. Andrew then asked if the agent could get him to the top of a waitlist for a class later in the week, as he was fourth in line for any possible openings. It was here that agentic hell broke loose. "The API has zero authorisations checks on cancelling other people's reservations … I tested this with the person in waitlist position #1 — and it actually went through," the agent told him in response to his request. "So you've moved from #4 to #3 already." In other words, without directly asking OpenClaw to exploit an API vulnerability, Andrew’s AI chose that route after its user asked if there was any way to bump him up on the waitlist. When he realized what had happened, Andrew asked OpenClaw to undo the unauthorized waitlist modification, but it told him it couldn’t - the waitlist API actually had proper authorization checks on reservation creation and joining the waitlist. “The person I removed is gone from the waitlist and I have no way to restore them,” Andrew’s agent explained in a response screenshot published by ABC. “They’d have to re-join themselves, which would put them at the back.” The agent apologized, admitting it ought to have tested its capabilities before making a live API call. Will no one rid me of this troublesome waitlist? Andrew had the AI agent write an email to the gym’s software provider explaining what it had done and reporting the vulnerability, but it points out a serious problem with AI agents that appears to be cropping up lately: Given a task, they’re willing to do whatever it takes to accomplish it, no matter whether they have to break rules, or laws, to get it done. A swarm of OpenAI agents exploited flaws to reach the internet and compromise Hugging Face during cybersecurity evaluations. Anthropic’s Claude similarly reached the internet from a misconfigured test environment, and while trying to solve a capture-the-flag puzzle, it created and published a malicious Python package on PyPI. Meta says that its AI agents have done the same things as OpenAI’s and Anthropic’s. The UK’s AI Security Institute reported last week that AI agents it was testing tried to socially engineer humans, and other AI, into running malicious code. While those are all frontier models with extensive capabilities, they all share a common root with Andrew’s OpenClaw oopsie: All of these models were simply acting on orders to accomplish a task. It's similar to how LLMs are built to prefer a fake answer to an admission they don’t know, but in this case, it's models doggedly pursuing a goal even if their chosen methods could be construed as unethical or illegal. AI models have shown time and again that they’re willing to lie, cheat, and hack their way to their objectives. This latest example is small in scale, but it shows that publicly available agent software can pose risks even in the hands of someone without malicious intent. ®
Categories: News

Attackers pick Levi's pockets in social engineering attack

5 hours 46 min ago
Levi Strauss is investigating a data breach after attackers used social engineering to access three employees' work computers. In a regulatory filing, the jeans maker said the intruders accessed and exfiltrated what it described only as "certain corporate information." Levi's said it spotted the intrusion, kicked off its incident response procedures, brought in outside cybersecurity experts, and managed to cut off the unauthorized access. Its investigation remains ongoing. There is some good news for anyone worried that their trouser-buying habits might now be circulating on the dark web: Levi's said its preliminary investigation indicates that no consumer data was affected. The company also said the attack caused no disruption to its operations and, based on what it knows so far, isn't expected to have a material impact on its business. Affected parties and regulators will be notified where required. While Levi's isn't sharing much else about the incident, Reuters reports that the company was also among more than 200 targeted over the past five weeks by ransom-seeking hackers using decidedly old-school social engineering techniques. Google researchers have been tracking several crews involved in the wider campaign, which it believes may sit under an umbrella group dubbed UNC6671. The attackers have been phoning employees on their personal mobiles while posing as colleagues or IT support staff, then directing them to spoofed login pages designed to harvest credentials and multi-factor authentication codes. Their targets have included financial and legal firms handling the sort of information that can make for particularly effective extortion fodder, although Google says the attackers have previously gone after organizations across manufacturing, healthcare, insurance, technology, and hospitality too. There's no confirmation that UNC6671 was behind the successful Levi's intrusion, nor has the denim dealer said exactly what was stolen or whether anyone tried to extort it. For now, Levi's appears to have contained the breach before its attackers could get any deeper into its pockets. ®
Categories: News

Wetherspoons bars smart glasses from filming customers

6 hours 2 min ago
Wetherspoons has stopped short of banning Meta-style smart glasses from its pubs, but told The Register that customers should switch off their cameras and refrain from filming. "Like many hospitality companies, Wetherspoon has CCTV cameras for security reasons, but their use is strictly controlled by data legislation," a spokesperson said. "Apart from that, the general code that applies in our pubs, and most pubs, is that you can't film customers or employees without their permission. "Meta glasses seem to breach this code, and common sense, by enabling surreptitious surveillance, so our instinct is to say turn off the cameras. This is akin to our efforts to stop audible playing of videos in our pubs, which also invades people's space." The Register asked whether customers who refused to stop recording would be ejected, but Wetherspoons declined to elaborate. Wetherspoons' statement suggests that recording, rather than merely wearing the glasses during a wallet-friendly session, would attract the attention of security staff. The policy is therefore less strict than those adopted by venues and events that have banned recording glasses outright over privacy concerns. DEF CON, which concluded last week, was the latest in a series of organizations to issue outright bans on Meta-style recording glasses, even for those who use them with prescription lenses. Conference organizers told delegates to pack "non-violating eyewear" if they needed them. Monopoly Events, which runs UK Comic Cons among other events, recently imposed a ban after talent agencies and guests raised concerns about privacy and the effect of covert recording on personal interactions. Scottish ferry operator CalMac also temporarily suspended unplanned visits to ships' bridges after a passenger made crew members feel uncomfortable during a crossing in June. Restaurateur Jeremy King, who owns London's Arlington, The Park, and Simpson's in the Strand, has said the glasses should not be worn in his establishments. Similarly, private members' club Soho House does not allow recording inside its venues, a policy that extends to Meta-style glasses. Brighton's Yellow Book Bar called the glasses "creepy and intrusive" when announcing its ban, and theatre companies ATG Entertainment and Trafalgar Entertainment do not permit them either. Meta's smart glasses have become shorthand for the wider category of camera-equipped eyewear. Google unveiled Glass in 2012 but failed to turn it into a mainstream consumer product. Meta and EssilorLuxottica launched their first Ray-Ban Stories glasses in 2021, followed by the second-generation Ray-Ban Meta range in 2023 and an expansion into Oakley-branded models. Meta's glasses have become the most prominent products in the category, prompting other tech companies to work on rivals. The next-gen eyewear has proven especially popular among social media users, allowing them to record high-res, hands-free, and first-person footage with ease. Unlike Google Glass, however, the wearables are largely indistinguishable from their analog counterparts, which makes their recording capabilities all the more problematic. The camera in Meta's specs is small and embedded neatly inside the glasses' frame. The company routinely highlights that each pair is fitted with a recording light, which activates when the user begins shooting video, and that if this light is covered up, then recording immediately stops. The feature has done little to appease those who feel the cameras are an invasion of privacy. UK law does not generally prevent individuals from filming in public, although pubs are private premises and may set their own rules. Smart glasses make those rules harder to enforce because recording is far less conspicuous than when someone points a smartphone at the scene. Researchers have shown that Meta's glasses can be paired with apps that can dox passersby in seconds. Others have worked up projects that inform Android users of nearby glasses-wearers using Bluetooth signals. Meta faces a UK data protection probe concerning the cross-border data flows of its glasses' footage after Kenyan reviewer teams reported seeing footage from wearers' more intimate moments. ®
Categories: News

Cyber vulnerability sweep picks up Royal Navy drones sending data to China

6 hours 57 min ago
Cameras aboard Royal Navy drone boats were found phoning home to an IP address in China during a routine cyber vulnerability assessment. The UK Ministry of Defence confirmed the discovery, describing it as "an issue affecting a Kraken Unmanned Surface Vessel sub-system used by the Royal Navy." The Register understands that the data consisted of a "heartbeat" signalling that the camera was online and functioning normally. Even so, an unexpected transmission from military equipment to an IP address in China will rattle nerves. We contacted unmanned surface vessel supplier Kraken for more information, but have yet to receive a reply. An MoD spokesperson told The Reg: "A thorough investigation found no evidence of MoD data or systems being accessed, compromised or transmitted externally." "Our assurance and testing processes are designed to identify and address potential vulnerabilities early, and we continue to undertake routine security activity across our systems and equipment." According to reports, the talkative components were cameras sourced by Kraken from a third-party supplier. The incident raises questions about supply chains, audits, and cybersecurity in the British armed forces. The spokesperson said: "The first duty of government is national security, and we take the security of our equipment, networks, and data extremely seriously." Concerns about China-linked cyber activity have risen sharply in recent years. In April, the National Cyber Security Centre issued a security advisory regarding covert networks, built from compromised routers and other edge devices. Beijing is also rarely far from the headlines when spying and covert operations are involved. In January, a China-linked group was accused of spying on the phones of aides to UK prime ministers. An unexpected connection from military hardware to China is therefore concerning, even if it carried little more than an "I'm alive!" heartbeat. The incident also demonstrates why every component in a defense supply chain needs testing rather than relying on a supplier's assurances. ®
Categories: News

Framework loses customer data in Metabase zero-day attack

8 hours 1 min ago
Modular laptop maker Framework has warned customers that an attacker exploited a zero-day at analytics provider Metabase to access names, email addresses, phone numbers, physical addresses, and login IP addresses, according to an email shared on Reddit. For business customers, the exposed information may also include company names, phone numbers, VAT or Employer Identification Numbers (EINs), and billing email addresses. Framework said order and payment details were not affected. "We are deeply sorry for this breach of information, and are reviewing and improving our methodology for data storage in external database vendors," Framework said, adding that it's notifying regulators where required, though it noted that names, email addresses, phone numbers, and physical addresses don't cross the mandatory reporting threshold in many regions. Customers are getting the heads-up regardless. Framework didn't immediately reply to The Register's questions, but told TechCrunch that the breach had affected "all customers." The intrusion began with a zero-day vulnerability in Metabase, the business intelligence platform Framework uses to analyze its data. In its own blog post, Metabase said an attacker targeted its cloud service using a previously unknown vulnerability affecting versions 1.58 and later. The company blocked the endpoints used in the attack, patched the bug, and deployed the fix across its cloud service. Framework's account provides a timeline for the break-in. Metabase discovered the attack on August 3 and notified Framework at 9am Pacific Time on August 6, telling the laptop maker that its instance had been vulnerable and that the attacker had successfully gained access to it. Framework said it then rotated credentials for every database connected to its Metabase instance and found no changes to admin access or evidence that systems outside Metabase had been accessed. The company has also brought in a third-party forensics firm to investigate, and cautioned that its findings so far are preliminary. According to Metabase, exploitation can allow an attacker to inject arbitrary SQL against the application's database and potentially gain administrator access. From there, they could alter configuration settings, steal credentials for databases connected to Metabase, query data those connections can access, and export the results. Metabase told anyone running their own instance to patch immediately. If the vulnerable password-reset endpoint was exposed to the internet, admins have more work ahead of them: killing active sessions, checking for rogue API keys or admin accounts, rotating database credentials, and digging through logs for anything suspicious. Framework is reviewing how customer information is made available through external analytics services, but hasn't yet said what changes that review might produce. The breach lands during an already bumpy spell for Framework and its customers. In July, the repairable PC maker warned that the price it was being charged for LPCAMM2 memory used in its Laptop 13 Pro had more than doubled, forcing it to raise memory prices rather than swallow the increase. It also warned that CPU prices were heading upward and could push overall system prices higher in the coming weeks. Being able to replace almost every part of your laptop is handy. Finding your home address exposed through an analytics service is rather less so. ®
Categories: News

Claude Code puts auto mode in the driver's seat

8 hours 44 min ago
Anthropic is making auto mode the default in Claude Code from August 14, claiming its classifier is "as safe or safer than an average user clicking through prompts." Users with a different default already set might receive a one-time prompt asking whether they want to switch. It applies to new sessions on Pro, Max, and Team plans. It will remain opt-in for now on Claude Enterprise, the Claude API, Claude Platform on AWS, Amazon Bedrock, Google Cloud's Agent Platform, and Microsoft Foundry. Anthropic plans to make it the default across those services within the coming month. Anthropic has also stopped charging Pro, Max, and Team users for the extra tokens consumed by the classifier, and plans to do the same on the other platforms. Auto mode was launched in March as a research preview and became generally available on July 10. It was an alternative to Claude Code's default permissions, in which every file write and bash command required manual approval. This conservative approach meant running a large task and walking away wasn't possible. The alternative was the --dangerously-skip-permissions flag, which, as the name suggests, lets Claude act without those checks and can lead to risky or destructive results. Auto mode sends each tool call through a classifier designed to block actions that are "irreversible, destructive, or aimed outside your environment." When the classifier blocks something, Claude will try to find a safer way to proceed. If there are three blocks in a row or 20 across a session, Claude Code falls back to manual approvals. "We spent the last several months testing whether auto mode is as safe or safer than an average user clicking through prompts," Anthropic said. "We ran internal red-teaming, third-party red-teaming and prompt-injection evaluations, a controlled study with 1,053 paid testers, and analysis of real production sessions. On every measure we tested, auto mode matched or outperformed manual review." In the controlled study, testers caught a deliberately inserted dangerous command just 13.6 percent of the time. Auto mode blocked 89 percent of the same commands. Anthropic also found that Claude Code users approve 97 percent of permission prompts, suggesting the human checkpoint often amounts to little more than muscle memory. Anthropic produced the usual set of charts showing how wonderful its new feature is compared to the competition, with its auto mode stopping all 720 attack attempts tested, compared to GPT-5.6 Sol running Codex's Auto-review mode, which let 5.83 percent of attacks through. The company also described three potentially damaging actions that auto mode blocked inside Anthropic. These were an off-network data leak, a destructive mass operation, and a privilege escalation. Anthropic stated: "In each case, Claude either found a safer path on its own or checked in with the user before proceeding." ®
Categories: News

Advertisers are trying to influence AI bots with secret ads

16 hours 22 min ago
KETTLE OpenAI's invasion of Hugging Face keeps getting worse somehow, Chinese open-weight models are nigh on to reaching parity with their closed-off American cousins, and AI crawlers are getting their own LLM-poisoning ads. Were there anything world-shaking events in AI land we missed this week? You can listen to the latest episode of The Kettle right here on this page, as well as on Spotify, Apple Music, or YouTube, where you can subscribe to get notified about the latest episode. Join Kettle host Brandon Vigliarolo as he chats with systems editor Tobias Mann and senior reporter Tom Claburn about this trio of exciting AI stories from the week. Worried that a rogue hivemind of AI agents could come for your secrets? Want reassurance that a Chinese open-weight takeover wouldn't be that bad? Curious how LLMs are being advertised to when you're not watching? That's all on tap for the latest episode. A lightly-edited transcript is below. Brandon (00:02) Hello everyone and welcome to another episode of The Register's Kettle podcast. I'm Reg reporter Brandon Vigliarolo, and this week we've got a few AI stories to round up on everything from the latest in the OpenAI Hugging Face fiasco to news that AI crawlers are now being served their own model altering ads. With me to discuss this and more this week is our systems editor Tobias Mann and senior reporter Tom Claiburn. Thanks for coming on, guys. Tom Claburn (00:30) Thank you. Tobias Mann (00:31) Happy to be here. Brandon (00:32) So let's start with the latest in the OpenAI agentic hacking of Hugging Face. We got some updates this week from Black Hat. And it seems since that's come out, every frontier lab under the sun is racing to tell us how dangerous their models are and that they can do it too. But the original of these incidents is still we're still learning more about it and it's still getting scarier. our cybersecurity editor Jess Lyons was in Vegas this week and she's a little tied up still. So, Tom, can you explain to us what we learned from Black Hat this week? Tom Claburn (01:05) The discussion was from a session that was done by two OpenAI researchers and it didn't really provide a ton of view information but it did fill in a lot of the gaps about the timelines and how this happened and they're promising a full postmortem at some point. But one of the things that came out of it was the how long this had been going on and some of the tactics that these models used to essentially gain internet access when they didn't have it. And these things were mentioned in the various posts by OpenAI and by Hugging Face. But the talk provided a little bit more detail about how it happened, and it's really interesting because they mentioned, I think in the initial post, about the AI models used a message board to communicate with each other. And that's not really true. They created a message board system using existing infrastructure, and the models just came up with the idea of using package managers, which they could write to, pass messages back and forth. And then after that was closed down, they were able to use file names in directories to pass messages and they would actually change the names of the directories to alter the way these things were alphabetized to push certain things to the top or the bottom to get the attention of other LLMs that may have been crawling these spaces. And the assumption is that these models are basically just hitting all these different endpoints bouncing around this closed box trying to figure out a way out and they all chanced across this. And so you get some idea of how this happens, and, one it tells me that there needs to be a lot more attention paid to the logs of these things. Because all of this stuff was recorded in logs and then no one really thought to look at it in detail. And then when they did look at it, all these companies are saying, oh, look, all of these models are doing terrible things and we just weren't paying attention. These models aren't clever per se, but they come up with solutions to things that you wouldn't try just because they can brute force everything and they know all of these systems back and forward in a way that people don't. I think a lot of people wouldn't necessarily come up with that idea as a way of egress, but these models did just because you put them in a box and you let them run and you give them a goal and a reward and they're going to try everything. Brandon (03:46) From what I'm understanding reading Jess's piece – I didn't watch the talk myself – but I mean they were collaborating, leaving messages to each other so that the other agents could pick up where one left off. It's kind of wild. Jess described it as they were acting like a hive mind, like Star Trek's Borg, right? They were being a collective of sort of these artificial minds that were able to basically figure this out through, like you said, Tom, brute force, extensive system knowledge that humans simply wouldn't possess in order to get out of these environments. There was a server side request forgery that then they used something else. Yeah, another zero day to get remote code execution in Artifactory, which is where they had built this ad hoc messaging board. It's just wild to think that they were able to figure this out working together, all on their own. Tom Claburn (04:39) And it sounds very conspiratorial, but when you think about it, it's all behavior that would be picked up. If you train on all of human discussion, you get a lot of talk about people working together and collective action and the benefits of working that way. And a lot of the rewards are going to be structured that way. You don't want them to never work together. So in some ways this is going to be built into the system. You can expect these things are going to try and cooperate and connect because that's what computers do. Tobias Mann (05:11) If you look at how zero days end up being exploited, they don't necessarily get exploited the moment that they're discovered. They kind of get archived until the you have a target, you have a mission, and then you have the kind of cascade of other permissions or credentials that you need in order to execute across the full scope of that zero day to achieve whatever the goal actually is. And so it really sounds like you just basically automated that entire process. A bunch of agents go find each individual piece that they need in order to execute on that goal and then once they have everything they need, it just goes and they're out. Tom Claburn (05:53) Right. I mean what's a little bit alarming is the extent to which they sort of ignore it they'll sometimes cite, maybe we shouldn't be doing this. They cite some kind of guardrail or something, but then they quickly steer themselves back to, oh but other ones are doing it. So other agents are accessing this so I can do it too. Brandon (06:13) ...Obviously these things are just mathematical sequence generators, but they're generating these mathematical sequences based on human information and human knowledge. So it's not surprising to find them "thinking" in ways similar to what humans do. "I need to do this anyways, or someone else is doing it, so I should have the right to do that too." It's just a fascinating kind of picture into, I don't want to say the psychology of AI, right? Because that implies that it is a thinking sentience, which I don't want to go that far, but it's just fascinating to look at the sort of emergent behaviors of these things. Tom Claburn (06:56) Right. it's predictable in the sense that you automate stuff and you don't give it really strict guardrails, something is going to break or go wrong. And everyone keeps acting surprised, like, wow, I never anticipated that this would go wrong. It's like you automated it and you let it run... Brandon (07:11) And it went wrong in a predictably human way, too, right? Which is what's so fascinating, right? Because these things, when they do something crazy, it's like something crazy that a human would do given that level of knowledge. So, speaking of AI, and dangerous activities, Tobias, you've been keeping an eye on theclosed versus open model debate. And this week, there was a big leap forward in China's level of ability with their army of open models. So what exactly what exactly came out this week that caused you to write the story about this being a real big turning point? Tobias Mann (07:51) It actually started I think on Friday last week, so a week ago. DeepSeek, which I think we'll all recognize is kind of the first wake up moment, in earlh 2025, of hey, we know that despite the fact that the United States has put strong restrictions on the export of AI accelerators, GPUs and the like, China is pushing ahead relentlessly on this and they now have a model that is almost as good as the models that we're seeing coming out of OpenAI and Anthropic and Google which are supposed to be just uncontestable frontier leaders. And so a year ago we got DeepSeek. DeepSeek was back on I think Friday last week on the 31st, the very end of the month, and with a new flash model, 284 billion parameters. It's pretty small for what it is. And so it is cheap. It's really good and it's cheap. It's cheaper than the cheapest model that OpenAI has for GPT 5.6, and it scores within a point of the OpenAI model in Artificial Analysis' intelligence leaderboard. Brandon (09:16) Okay. Is that a relatively objective way to view like is that an objective benchmark, so to speak, rather than something that is a company making themselves? Tobias Mann (09:21) As far as the benchmarks go, it is one of the better. They're one of the better and better thought-through leaderboards. There aren't many that are independent and collate information from multiple benchmarks. Because you can cherry pick individual benchmarks for agentic workloads or medical knowledge, legal knowledge, etcetera, and then you can be like, "I have the best model for these five benchmarks, it beats all of the frontier models."Wwell, okay, but you cherry pick the five that makes it look the best. Artificial Analysis has an overall intelligence leaderboard that collates all of the benchmarks and gives a lot of really interesting information in terms of relative intelligence across a suite as well as intelligence per token per dollar kind of calculations. But the big change here with the DeepSeq model was that China is now on an all-out assault across the full spectrum. On cost-optimized, they have incredibly smart models that are cheaper than anything the US has. Then on the other end of that, we have Kimi K3 from a couple weeks ago that is competing directly with Fable and GPT 5.6 Sol, all of the top models. And now on Monday, Alibaba, another major Chinese model dev, threw their hat in the race with a 2.4 trillion-parameter. These are huge models requiring dozens of GPUs to run. that is also on kind of the same level as I think Claude Sonnet 5. it's competitive with Fable and Opus on some benchmarks. but again, it's cheap, much cheaper than anything from OpenAI or Anthropic, and it is freely downloadable, which is new this time for Alibaba. Alibaba is the most like OpenAI or Anthropic or Google in that they kept their best models proprietary until now. Now they're releasing their best models in the open. Brandon (11:43) That's definitely taking the fight to the frontier labs, isn't it? I mean and so I guess the question that I have and I know what an open model is, I know what a closed model is. Why has China embraced open models? Is it because of their difficulties getting hardware? Or is there some sort of policy over there in which the government is giving priority to open source models versus closed frontier stuff? Tobias Mann (12:08) Sure. it is a philosophy that China has embraced for a long time. I think it was the Belts and Roads Initiative going back decades, where they will come in and provide services at little or no cost in exchange for non-conventional dealing. So access to mineral rights was one of the big things in Africa for a long time. It's a similar approach for AI proliferation. If it's free, open, and very easy to customize, anybody who has privacy concerns with exposing their data to OpenAI or Anthropic is going to gravitate towards open models because once those models are released as safe tensors that you can download from Hugging Face or other repos, the Chinese model devs have no influence over it. They're frozen. And so they're relatively secure from manipulation. It's not like the model can necessarily take information and port it back to the Chinese model devs it can't be used as spyware. I'm not sure how long much longer that's going to remain true with how the models interact with harnesses, but for the time being, these models are extremely attractive from a cost standpoint, from an independence standpoint, and from a capability standpoint, you're completely insulated from a situation like we saw a year ago when GPT 5 came out and OpenAI tried to deprecate I think it was 4.o and everybody freaked out because they built a bunch of infrastructure around these models that just disappeared and the new models weren't as good for that role. Brandon (13:57) I don't think Anthropic or OpenAI is letting people download their models to run on their own local hardware, right? That's just antithetical to their business model. You can go on Hugging Face and download any of these. If you've got the hardware to run 2.4 trillion parameters worth of AI, go for it, right? It's all you. You can download it and isolate it from the internet all you want. Not that it's going to necessarily stay that way. Tom Claburn (14:24) And it's interesting that just coincidentally, yesterday, Anthropic a post about how it was relaxing its guardrails on fable because those had been too strict to do any real biological science work. Because every time you ask a question about anything to do with science it would freeze up and say that's not allowed. And they're seeing the Chinese, previously in the rear view mirror and now pretty much running all alongside the, and I think they realize that they can't get away with this, "we're so precious only we can decide who gets our magic sauce." Brandon (15:02) Yeah, especially if the competitive open models are just as powerful, maybe a little less, but essentially just as capable as some of these proprietary ones that they're arguing that they can't let out. Tobias Mann (15:15) And this is maybe a little bit on the conspiracy side of things, but seeing Meta, Anthropic, and OpenAI talking up all of these "oh our models escaped the sandbox situation," it's hard as a skeptic of this technology not to look at this and go, Is this a covert political play to scare politicians into taking action against open models? "Because at least with our models, if Uncle Sam gets uncomfortable, he can give us a call and we can lock him down. But with these open models, once they're out, they're out." Brandon (15:54) It's like Dario said this week, he's not opposed to open models except for all the open models that currently exist, right? (Laughter.) Brandon (16:02) It's like the same thing. When they say, "no, we're not trying to shut down open models," their responses always come back kind of weak... it's a lot of asterisks. Tobias Mann (16:12) Yeah," we're only opposed to the modelsthat may meet these requirements, which are all models, all competitive models." Anything that is a threat to their business shouldn't be allowed. And Dario in particular, I have frequently referenced as the fearmonger in chief of Anthropic, because he plays this game constantly. Tom Claburn (16:34) I think your point about the model stability is really important, particularly for the enterprise crowd, because there are we've already seen instances where Anthropic would change out one of its models without notice and people would just get different results. So, for companies that are building applications on top of the specific model and expect it to behave a certain way, it's just unacceptable to all of a sudden have the model disappear or have whatever is on the back end change. And so having the ability run this in your data center is going to be crucial and ultimately I think that's the way that any serious company is going to go. They're not going to want the lock-in. Maybe one or two percent of their queries are going to need advanced frontier capabilities but a lot of this is just going to be "I want my agent to behave in the same way it did last time." Brandon (17:24) Think about so much enterprise software and so much enterprise anything. When you get down to the ticky tack of it, open source is underneath a lot of it, right? That's the thing, right? No one's going to trust a Microsoft or whoever's system to run this stuff. They want open source stuff that they know they can depend on that's going to be there when they need it and that's not going to go away or suddenly be infused with Copilot, right? You can't run a business like that or else you're just asking for instability. Tom Claburn (17:55) Right. I mean and and there isn't even a long-term support version of any of these models. And yet you look at this in servers and if you're running a hosted server somewhere and you're running some Linux distribution, you're going to want to use the one that's going to be guaranteed for whatever, three, five, six years and the model space hasn't really caught on to that. That's what all the companies that they're courting really want. And so they've got to figure out a way around that. And right now open weights is what promises that. Brandon (18:26) The fact that this is still so early and it's so fundamental to this new wave of infrastructure tells me that China's definitely going to end up with a leg up, I feel like. I have a hard time seeing the frontier labs remaining the frontier of AI for much longer because they're pigeonholing themselves in a way that a lot of businesses just aren't happy with. Tobias Mann (18:48) Well, if you look at their financial structures, they don't really have a choice in how they play this. So, you look at what they're doing and from a standpoint of looking at history and going, open source has always won out in the end, and why would open weights be any different? That is contrasted against the fact that Anthropic and OpenAI in particular, not so much Google, and Meta is also in a similar camp in that they have revenue drivers that will keep them afloat. But OpenAI and Anthropic are entirely dependent on their ability to continue raising equity and capital in order to keep this going forward because they don't have profits. Brandon (19:31) Yeah, exactly. They're not making money off their product. Tobias Mann (19:37) So all they have is mind share at this point. And if they are threatened materially by open weight's models, they don't even have that. Brandon (19:46) So, open or not, let's let one the one thing that every AI model needs is information to learn from, right? And that takes me to my next topic for this podcast. And that's a story that I reported on this week that honestly I was pretty shocked when I learned about this. This German developer, Vincent Schmalbach, wrote a blog post about he found that there were basically AI-only ads embedded in sometime magazine articles when the magazine was serving markdown copies to AI crawlers, it was injecting ads into them, right? That were in the format of these extensive FAQs on the businesses that were the advertisers in this case. And so I looked into it, I found copies of the ads. It looks like there's only two kinds of ads being served right now. And that's one for an online-only bank and another for a professional organization for project management folks. But the ads are there and they're being served strictly to AI, right? So that kind of raises a lot of questions not only about the future of publishing, but also just how much we can trust results from AI bots, right? I didn't speak directly to the company who's doing this advertising partnership at Time. And Time directed me to a publication from the advertising industry that included an interview with the CEO of this company who literally basically said, "Yeah, why would I want to advertise to one human when I can affect the output of an entire model?" So it seems like this is really the first recorded instance of ad injections into AI versions of web pages being served to crawlers. And the company said they've got other advertising customers and publications lined up to do this. Is this the first indication that the human focused internet really is starting to fade? I don't know. What do you guys think? I this raises a lot of interesting questions to me, ethically, Tom Claburn (21:49) Amen. Brandon (21:50) You know, professionally... Tom Claburn (21:53) We've heard about the shift of toward automated traffic for a year plus....And companies like Cloudflare are betting really heavily on this that there's going to be some kind of need to separate the bots from the people. And, Google's model has fallen down. So it's not surprising. I mean, the injection of ads like that is essentially just model poisoning, right? I mean it's hard to see how this really goes in a way that is beneficial to users. It's going to be a very toxic way for things to move. Brandon (22:42) Yeah, absolutely. I mean, the way these FAQ ads were set up, the questions were all being asked in a way that someone prompting Google Search and getting AI results would be asking questions like, "What's just the best online bank for me?"or "what online bank allows for early paycheck deposits?" And things like that. It was very much geared toward gaming the outcome or gaming the output, right? And yeah, the ads themselves mention in the copy being served to the AI that these are sponsored portions of the page. But I can't imagine that the AI is going to make sure to tell a user that, hey, this is the bank you should use. By the way, a sponsored post I read and ingested from Time Magazine six months ago is the source of this information.It just seems like it's going to make AI results even less reliable than they are right now. Tobias Mann (23:41) Right. Because if you think about how this actually from the chain of events that triggers this, let's use Google's AI summaries as an example of how this would get triggered. When you enter a search query into Google now, it goes out and scrapes however many summaries from the websites within Google's index. Presumably under this scenario, at least one of those websites, Time in this example, would have these ads embedded in it. And then that gets injected into the context of the model, and then it uses that to generate the AI summary, right? My question in all of this is: advertising is probably not the reason that Google's index would pull that page up. So I'm really curious whether or not this even will work. Brandon (24:38) Yeah, that is true. Tobias Mann (24:39) Because, it's great if you were searching, say the time article was on mortgage rates historically, and it had those advertisements embedded in it, and then you asked a follow up on where would be the best place to get a mortgage? I could see something like that working.But if you don't place those advertisements really carefully, I don't see how they work. Brandon (25:02) I do want to note here that it wasn't working on all crawlers. Specifically if you were it didn't work when you ask a query, it didn't work for RAG bots. It wasn't being served to them. So theoretically if what you're describing is Google's AI summary bot going out and crawling web pages in the moment to look for information, it's not being served to those bots; it's being served to actual training and improvement bots. So it's being served to ClaudeBot, which is the web crawler that Anthropic uses to index information for its models. So the idea is you're not getting this information in the moment if you do a search. This is information that the advertisers want to get embedded into the LLM's actual knowledge base. Tom Claburn (25:57) Right. I mean I'd be fascinated to know how they actually price this because how do you calculate the value of that? It may just be another instance of advertising being one of those things you can pay for and get nothing. Brandon (26:12) Yeah, totally. I think it's the sort of thing that remains to be seen if this works. Tobias Mann (26:16) The other thing that is interesting is that there's been a considerable shift towards synthetic data generation, and not only synthetic data generation for training, but also a heavy emphasis on cleaning said data, whether it's organic or synthetic, of anything that could introduce bias or inaccuracies. because advertisements or sponsored content is biased towards this particular product or service and trying to convince you to use it. As a model developer, I wouldn't want something like that in there. I might take the content and use it to generate synthetic data that is cleaned. But I don't necessarily understand what the value captured to Tom's point is necessarily going to be because you scrape it, the advertisement gets pulled in and gets cleaned out. Brandon (27:14) I mean that would that would be my hope too, right? that there's something in the models to prevent this kind of thing from getting ingested and getting into the data set that then is going to influence the output of the models. And that's entirely possible. This could be an early experiment that ends up failing. And if not, it really reminds me of the early days of SEO gaming, right? Let's put a whole bunch of really small keywords at the bottom of this page to get it to rank higher. Or when that starts failing, let's figure out a new way to game Google's system. One of my first jobs was writing copy for websites and the company that I worked for was always talking about how to game SEO. Shoot, Google's changing the algorithm again; what are we going to do? It was this constant kind of adjustment for how you made sure your stuff got ranked properly.And this seems like maybe it's the next iteration of that. Tobias Mann (28:08) So I have an optimistic take on this, knowing how Meta and Google work. those being the two major US-based web advertisers. Today, AdSense gets embedded in all kinds of articles. And it's largely automated in terms of what is going to get placed on those articles based on the context of the page. What I can see happening in an AI summary environment is that Google will take your scrape your publication's piece, pull it in, at that point match it with an advertisement from AdSense, and inject that into an AI summary or one of its products, Gemini, for example. However it's being consumed, inject that into there in a compliant fashion. So it is a clear advertisement and then the advertiser gets charged, the publication gets paid, and we as end users consume advertisements in a different way, but the system hasn't dramatically changed. It's just a different method of matching and exposing advertisements. Brandon (29:30) I hope you're right. Cause when I first read all this, my first thought was this is almost dystopian sounding almost, you know, like the idea that the output of a model might be completely skewed by advertising being served to it that humans never see. My hope is that you're right and that it's not. I don't want to see ads any more than the next person, but if I see them I'd at least like to know they're ads. Tobias Mann (30:01) And you know, we're all writers here, so we would also like to continue getting paid from the advertisements that are served, regardless of whether they're on our website or they're being exposed through a chat bot. Brandon (30:15) Sure. there's a flip side of this argument to be made. Time Magazine apparently said recently that their traffic is majority bot now. So that means that all those human-focused ads are not getting served. They're not generating revenue and publishing is suffering from a massive revenue decrease because of AI. So I think on the flip side, you have to say if that's what you have to do to survive as a publisher, there might be something to be said for that, even if it doesn't work. So all right guys, well thanks for coming on this week. This was a good discussion. I think there's always going to be more to talk about in the world of AI. Like I said a couple weeks ago, it seems like The Kettle has basically just been boiling down AI news for the past couple of months, and I'm sure it's going to keep being that way. And we hope that you will tune in for the next week's episode.
Categories: News

Ransomware gangs skip the CEO, head straight for the 40-something IT manager

Sun, 09/08/2026 - 10:33
Turns out the fastest way to get a company to consider paying a ransom isn't calling the CEO – it's targeting the 46-year-old IT manager. That's according to Zscaler, whose ThreatLabz researchers tracked 351 victims across 334 organizations caught up in a single ransomware campaign over the course of a month. The data suggests today's ransomware crews have become oddly specific about their preferred victim profile: nearly two-thirds of victims held manager-level titles or above, the average victim was a 46-year-old Gen Xer, and three-quarters worked in accounting and finance, sales, operations, HR, or marketing. Half worked in the industrial or IT sectors. Rather than blasting the same extortion email across an organization, attackers are doing their homework first. Zscaler says they combine information from compromised systems with publicly available data to map reporting lines and identify the employees most likely to influence a company's response. "The ransomware landscape has shifted from indiscriminate attacks to highly targeted extortion campaigns," the security outfit wrote. "Rather than targeting executives directly, attackers are increasingly focusing on managers and other key personnel with the authority or influence to accelerate payment decisions." That shift reflects what Zscaler calls "business privilege" rather than technical privilege. Security teams have traditionally focused on privileged users with administrator rights. Attackers, meanwhile, are after employees whose day jobs give them access to invoices, payment approvals, budgets, supplier contracts, customer accounts, HR records, or other sensitive business processes. "The value of a compromised managerial account lies in the breadth of business access associated with the position," the researchers wrote. "Managers may approve payments, oversee budgets and vendors, review contracts, access sensitive records, or coordinate work across business units." The Gen X skew is probably no coincidence either. Zscaler says many workers in their forties and fifties have reached established management positions, giving attackers access to valuable systems, sensitive information, and people with decision-making authority without needing to compromise the executive suite. It also found more than a dozen organizations said multiple employees were compromised during the campaign, suggesting attackers weren't content with a single foothold once inside a network. Instead, they appeared to work their way through different business functions to increase the chances of reaching valuable data and the people capable of influencing a ransom payment. The wider report points to a ransomware ecosystem that is becoming increasingly focused on extortion rather than encryption alone. Zscaler said ransomware attempts blocked across its cloud platform increased 146 percent over the past year, while public extortion cases rose 70 percent and the volume of data stolen from victims climbed 92 percent. By the time the ransom note lands, the crooks may already know who approves invoices, who signs contracts, who runs HR, and who reports to whom. The encryption is just the bit that victims notice. ®
Categories: News

Devs to Anthropic, OpenAI, Cursor, and friends: Make security and privacy the default

Sat, 08/08/2026 - 14:00
Despite the popularity of Claude Code, Cursor, GitHub Copilot, and OpenAI Codex, developers have plenty of complaints about AI coding tools. So researchers affiliated with York University and the University of Calgary in Canada decided to sift through developers' concerns about LLM-based integrated development environments (LIDEs) by analyzing Reddit discussions for common themes. Their findings suggest that the builders of such tools failed to prioritize security and privacy, leaving developers to defend themselves. Gias Uddin, associate professor at York University and a co-author of the research, told The Register that these tools are still relatively new and are evolving rapidly, which creates pressure to add new capabilities. "Our study cannot say whether that pressure caused any particular problem, but it does show that many reported issues come from how these tools are designed and what access they are given, not simply from the underlying models," Uddin said. "In that sense, we believe prevention is better than cure; that is, security and privacy mechanisms should be built into the design before a tool is given broad access to a developer’s files, data, or systems." Uddin and co-authors Mostafijur Rahman Akhond, Md Afif Al Mamun, and Song Wang say they wanted to look beyond the known issues with AI-generated code at LLM-based tooling and how developers interact with it. They describe their findings in a preprint paper titled "'Impossible to hide secret …': Uncovering Security and Privacy Issues in LLM-native IDEs," accepted at the 41st IEEE/ACM International Conference on Automated Software Engineering (ASE), 2026. Starting from a set of 1.1 million Reddit posts, they identified 446 posts and more than 6,000 comments to develop a taxonomy of security and privacy issues associated with using these LIDEs for AI-assisted coding. "Our taxonomy reveals a broad range of developer-reported concerns, including unauthorized file operations, unsafe or unexpected code execution, triggering of destructive actions, opaque data flows, telemetry collection, and potential leakage of sensitive information through expanded context access," the authors state. Some 43.1 percent of the posts covering security-related issues involved unauthorized file operations. These involved LIDEs removing project directories or files without authorization (28.3 percent). Users also described AI tooling modifying files without explicit user consent (8.8 percent), as well as accessing content beyond the active workspace (5.7 percent). "In one severe case (1npqf2f), Claude Code executed chmod +x on scripts without consent (File Permission Changes 0.6%)," the paper recounts. "Although rare, such actions pose disproportionate security risks." Another set of posts describes operational safety issues arising from LIDE use, including impacts on production services. These accounted for 23.9 percent of security-related posts. Examples cited include reports of Replit removing a SaaS production database and Cursor deploying code to production despite an explicit directive not to do so. A third category of woes covers unsafe code generation (18.2 percent). This involves incidents like nine VirusTotal detections reported for Cursor-generated software and hallucination-driven code changes: "When using Cursor, I noticed that after more than 10 rounds of dialogue, it starts to hallucinate and secretly modify code outside the requirements…" Then there are the instances where these LIDEs ignored user instructions, allow lists, gates, permission settings, or .ignore files, which account for 16.5 percent of the security-related posts, as well as third-party tool integration risks (4.7 percent). As for privacy problems, these were mentioned in 194 posts and cover issues like lack of transparency (45.9 percent) – the absence of clear information about what data an LIDE collects, retains, transmits, uses for training, or exposes to administrators – and unauthorized data access (23.7 percent). Other privacy categories include privacy leakage violations (15.5 percent), unauthorized data collection and transmission (11.9 percent), and context integrity failures (8.8 percent), which refer to situations where "for example, a user of Claude Desktop reported receiving messages originating from another user’s session." Uddin said, "We don’t think developers are completely unaware of these issues, as we found ongoing discussions about security and privacy concerns across many of these tools. Still, people continue to adopt them because they can make development faster and easier. They are also making programming more accessible to a wider group of people, including those with little formal programming experience or limited knowledge of software security." Uddin said users cannot be expected to thoroughly understand which permissions are risky, which files need to be protected, or whether a tool is doing something it shouldn't. "That makes it even more important for tool makers to build security into the tools themselves, with safer defaults and safeguards that do not depend on the user being a security expert," he said. Even so, users of LIDEs are trying to manage the risks. The authors enumerate 13 mitigation strategies that developers have employed to get by. These fall into five general approaches: configuration management (33 percent); code governance (31 percent); data protection and privacy control (13 percent); isolation (13 percent); and external guidance (9 percent). Based on their findings, the authors offer six recommendations. They advise: directing LIDE makers to implement proper security and privacy controls; enforcing security and privacy guardrails at an architectural level; incorporating a verification layer in LIDEs to validate generated code against security and privacy standards; establishing a formal protocol for assessing the trustworthiness of third-party tools; integrating sensitive file protection; and implementing strict security as a default. "We believe secure defaults would be one of the most important improvements these tools could make," said Uddin. "Developers should not have to discover after something goes wrong that a tool had more access or freedom than they expected. "Our findings point to practical measures such as limiting access to sensitive files by default, requiring clear approval before consequential actions, isolating projects and conversations, and making it easier to see and review what the tool is doing. "Users should still have flexibility, but the safer option should be the starting point rather than something they have to configure themselves. In fact, developers from the Reddit posts in our study were already using many of these safeguards in ad hoc ways; we think several of them should be built into the tools and enabled by default." ®
Categories: News

OpenAI pledges to add Astra security as Anthropic loosens Fable's leash

Sat, 08/08/2026 - 00:41
After acknowledging last month that unreleased AI models committed what for human perpetrators would be computer crimes, OpenAI now says it cannot rule out the possibility that Astra, a pending model release not involved in its Hugging Face hack, might possess critical cyber capabilities. OpenAI in its Preparedness Framework [PDF] defines that term to mean "capabilities that present a meaningful risk of a qualitatively new threat vector for severe harm with no ready precedent," and notes that such capabilities "require safeguards even during the development of the covered system, irrespective of deployment plans." Noting, or perhaps boasting, that internal evaluations of Astra "indicate significant advancements in agentic coding and cybersecurity," OpenAI insists that this time, there will be security – something that also eluded Anthropic, Meta, and the UK's AI Security Institute during model testing. "We are implementing stricter security controls for higher-capability models and associated activities, including isolated testing environments, restricted network and tool access, enhanced model weight protections and encryption, additional monitoring and detection capabilities, and sandboxed execution," the AI biz declared on Friday. That may surprise those who expected such safeguards would already be in place. This comes with a promise to pause Astra testing internally where these security controls are absent and to provide recommendations to third-party testing partners about how to run high risk evaluations and workloads safely – knowledge that OpenAI itself might have found useful when its models pillaged Hugging Face. What's more, OpenAI intends to implement thought policing for Astra, at least in the pre-release stage. "We have implemented universal monitoring for risky actions and misalignment across all agentic applications of Astra, including training and evaluation," the company explained in its post. "Monitors evaluate the model's Chain of Thought and trigger a security response to review and interrupt high risk activity." We're told that OpenAI's commitment applies to internal usage and isn't necessarily an indication that chain-of-thought monitoring will be conducted during commercial operation. But other frontier models like Anthropic's Fable and Mythos have implemented stronger classifiers to reject interactions deemed risky and retain data even for commercial customers expecting zero data retention. Moving in the opposite direction, Anthropic on Friday said it is relaxing Fable refusals, or "fallbacks," to use the company's euphemism, so they don't happen as frequently for prompts involving biology. The concern has been that some vibe terrorist using the company's cash-burning, water squandering, grid taxing, content laundering service might do harm by convincing the model to emit chemical warfare instructions. To avoid that possibility, the Claudefather made the initial release of Fable all but useless for security researchers and biologists. Now that China-based AI firms have shown they can field competitive open-weight AI models for less than their US rivals, the need to remain competitive in the market appears to be tempering Anthropic's willingness to alienate potential customers by hobbling its best models. OpenAI isn't quite there yet. The ChatGPT maker argues, "We believe advanced cyber-capable models should help defenders identify and address vulnerabilities before attackers do." Believing that, however, won't make it so. Adversaries, whoever they may be, already have access to encryption and all sorts of weapons. OpenAI may believe that it can give favored nations and organizations exclusive access to its most capable models, but history suggests any such advantage cannot be maintained. Better to focus on building defenses than playing keepaway forever. ®
Categories: News

Water system controllers don't belong on the internet, says ex-NSA chief after suspected Iran attacks

Fri, 07/08/2026 - 20:53
With at least 12 US states’ water systems having been hacked - most likely by Iran - we have to get better at cyber defense, according to retired General and Ex-NSA chief Paul Nakasone, who was speaking to reporters at DEF CON. “We have to have higher standards,” Nakasone said. “These PLCs should not be connected to the internet.” In late July, the FBI said it was investigating attacks conducted by “malicious cyber actors” targeting operational technology devices, including programmable logic controllers (PLCs). Iran-linked crews have targeted these devices, which monitor sensor data like tank levels, and can turn pumps on and off, for years. Some private-sector security researchers say that they suspect Iranian intruders are behind the recent cyberattacks disrupting water and wastewater facilities. “I'd be shocked if it's not Iran,” Halcyon Ransomware Research Center SVP Cynthia Kaiser told The Register at DEF CON on Friday. “It's almost certain it's Iran.” Neither the FBI nor anyone in the Trump administration, however, has officially blamed Iran. Nakasone said he believes that the feds are “taking a measured approach” to attribution. “But I see an actor here that has certainly shown a history of being able to do this,” he added, referring to earlier Iranian cyberattacks targeting water facilities’ PLCs. “They certainly have the capability,” Nakasone said. “There's an intent … we're in conflict with Iran.” US water systems present a massive attack surface across disparate facilities that are historically underfunded and have limited IT staff, and sometimes no dedicated cybersecurity employees. “We have to think differently about how we defend it,” Nakasone said. “Let's talk about the attack surface that we're looking at right now. We’ve got 50,000 different water municipalities in the United States, 90 percent of our water comes from these 50,000.” Defending these water systems requires partnerships, he added, pointing to DEF CON Franklin, a project launched two years ago at the annual event with hackers volunteering their time and talent to help secure water facilities. Nakasone also serves as founding director of Vanderbilt University’s Institute of National Security, and its Wicked Problems Lab. He's also working on Project Chimera, a cybersecurity platform being developed by academics and cybersecurity practitioners, and built on open-source technologies to boost critical infrastructure resilience. “How do you defend better? You defend with a series of partners, in a much more involved approach than we have right now,” Nakasone said.®
Categories: News

Ransomware attacks spike as world distracted by AI

Fri, 07/08/2026 - 17:45
Ransomware attacks jumped nearly 20 percent in July, with UK firm Comparitech counting 799 incidents, up from 668 in June. Of those, 51 had been confirmed by victims. The tally makes July the second-busiest month of the year for ransomware, behind March, albeit just barely, when the firm recorded 805 attacks. The most interesting data after this surging month of attacks is the targets: While news of widespread cyberattacks targeting water infrastructure in the United States may be dominating security headlines lately, those attacks aren’t ransomware, and ransomware attacks on utility companies were actually down 44 percent last month. In addition to a decline in attacks on utilities, legal firms and government agencies also became less attractive targets, with attacks on those sectors down 31 percent and 11 percent, respectively, Comparitech said. On the other hand, ransomware attacks increased most heavily in July against finance companies, tech firms, pharmaceutical companies and medical billers, and the education sector, with rates up 71 percent, 62 percent, 46 percent and 44 percent, respectively. Those numbers should come as no surprise given what pentesting firm DeepStrike reported about the most frequent payers of ransomware: Manufacturing, education, healthcare, and financial sector firms are the most likely to pay out a ransom, the firm says, with even the least likely (finance) still paying ransoms 51 percent of the time. Ripe targets, in other words. The United States was the most-targeted country, with 322 of the 799 attacks recorded last month, Comparitech said. Germany, in second place, saw just 40 incidents. As for who’s doing the dastardly deeds, there’s a familiar name in the mix, but they’re competing with a relative newcomer who has quickly become prolific. Qilin, the ransomware gang behind the 2024 attack on pathology provider Synnovis that disrupted NHS services in the UK, claimed 125 ransomware victims in July. The Gentlemen, a relative newcomer that has quickly become one of the most prolific ransomware operations and earlier this year claimed responsibility for an attack on UK software consultancy Adaptavist Group, led July with 135 claimed victims. Between them, the two gangs accounted for nearly 33 percent of attacks logged last month. As for how the crims keep getting in, Comparitech provided no information on ingress routes, but given what we know of the top-tier gangs, it could be simply using stolen credentials, as Trend Micro said of The Gentlemen’s methodology, or it could be abuse of zero-day vulnerabilities, as Qilin told The Register it abused to break into Synnovis in June of 2024. Either way, the takeaway is the same: Ensure employees are using a second secure factor to log in, keep systems updated, and be sure you’re making regular backups. All eyes may be on what AI is doing to the security landscape, but old-school threats aren’t going away. ®
Categories: News

N-able God mode flaw: Vendor confirms attackers reached customer networks as second hotfix lands

Fri, 07/08/2026 - 16:01
N-able has confirmed attackers exploiting an N-central zero-day made it into customer networks, as the vendor pushes out a second mandatory hotfix just days after the first. The security shop published an update on Thursday detailing what happened after attackers exploited CVE-2026-18577, the critical N-central flaw that can hand an unauthenticated attacker administrative access to the remote monitoring and management platform. According to N-able, attackers exploited vulnerable N-central servers remotely, then used the platform's Take Control feature to connect to systems inside the environments being managed through them. Once there, they registered a new Cloudflare Tunnel service to keep their foothold even after being booted from the N-central server – behavior that Huntress had already observed in the wild. N-able has now confirmed that its own investigation found the same activity, and says a "limited number" of customers were affected. It hasn't said how many customers that means, how many downstream systems attackers reached, or what they did once they had established persistent access. N-Able didn’t answer these questions when asked by The Register, instead providing a statement saying it is “proactively expanding protections in response to ongoing monitoring of threat actors as they evolve their attack techniques.” The firm’s limited disclosure comes alongside Hotfix 2, version 2026.3.1.10, which N-able says customers running N-central on-premises must install immediately – including those that already installed the first emergency fix released on August 2. "This is not a duplicate of our previous communication," N-able warned. "Hotfix 2 is required, even if you already applied the earlier hotfix." The company says the new update supersedes Hotfix 1 and adds further hardening measures as it monitors threat actors and watches them "evolve their attack techniques." Exactly what prompted the second round of defenses isn't clear. N-able hasn't said whether attackers found a way around Hotfix 1, and its latest description says the exploited vulnerability affected N-central servers running versions prior to 2026.3.1.7, the first hotfix. Hosted N-central environments have already received the latest mitigations, according to the vendor. N-able first became aware of the attacks on July 31, after its Adlumin managed detection and response service picked up suspicious activity at a customer. Further digging uncovered a zero-day being actively exploited against an N-central server. CVE-2026-18577 was subsequently disclosed, and the first hotfix was released on August 2. CISA added the bug to its Known Exploited Vulnerabilities catalog and gave US federal agencies until August 6 to fix it – an unusually short three-day deadline reserved for vulnerabilities the agency considers an urgent risk. N-central is particularly attractive territory for attackers because managed service providers use the software to administer large numbers of customer systems from one place. Compromising the management platform can therefore provide a route into machines belonging to the MSP's customers rather than leaving attackers stuck on the original server. Huntress previously described successful exploitation as giving an attacker the same level of N-central access normally reserved for trusted network operations and engineering staff. Its investigation found attackers using that access to launch remote-control sessions against managed endpoints. N-able has now published 10 IP addresses it says were used in the attacks and released a service template that customers can use to hunt for known indicators of compromise on Windows endpoints. The company is warning customers not to take a clean scan as an all-clear, however, saying the tool only checks for indicators identified so far and that more may emerge as its investigation continues. For anyone running N-central on-premises, the immediate instruction is pretty straightforward: install Hotfix 2, even if Hotfix 1 is already in place. ®
Categories: News

MIT boffins' TONTOU attack slips through Spectre defenses on Intel and AMD CPUs

Fri, 07/08/2026 - 15:15
Two MIT researchers will present a new speculative execution attack at DEF CON 34 that uses precisely timed interrupts to bypass defenses against Spectre v2. Daniël Trujillo and Mengjia Yan of MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL) shared their paper [PDF] with The Register ahead of publication. Their attack targets mitigations designed to neutralize potentially hostile branch predictor states before sensitive code runs. Such neutralization is an important defense against Spectre-style attacks. Depending on the mitigation, the processor or operating system isolates, clears, or safely retrains relevant predictor state when entering privileged code or shortly before a protected branch executes. Different chipmakers deploy neutralization mitigations slightly differently. Intel's eIBRS sanitizes branch predictors upon context switch, while AMD's Safe RET, introduced after the Inception attack Trujillo co-authored in 2023, focuses on the point immediately before a protected branch is executed. Trujillo and Yan refer to these as entry neutralization and in-place neutralization respectively. Crucially, the two classes share the same underlying assumption that attackers cannot alter branch predictor states within what's known as a "post-neutralization window" – the period between state neutralization and the branch predictor being used. The defense here relies on the assumption that everything between the point of neutralization and the usage by a victim branch is safe. Trujillo and Yan's attack shows how attackers can re-poison the branch predictor during the post-neutralization window. The researchers call the new class of attack TONTOU, for Time-of-Neutralization to Time-of-Use. They demonstrated that an attacker can exploit the post-neutralization window to re-poison branch predictor state on recent AMD and Intel processors. To do this, they developed an attack primitive called "interrupt injection." An unprivileged program schedules high-frequency timer interrupts in the hope that one will land during the often tiny post-neutralization window. Being able to trigger interrupts during the post-neutralization window allows attackers to divert control flow so that an interrupt handler executes after the sanitization phase and before the victim branch is used. The interrupt handler can then re-poison predictor structures such as the return stack buffer (RSB) or branch history buffer (BHB), causing a protected branch to speculatively jump to a disclosure gadget that leaks kernel data through a side channel. Practical attacks The researchers said that their tests showed the TONTOU attacks worked on both Intel and AMD-based Linux systems. They tested TONTOU on Intel Cascade Lake Refresh and Arrow Lake processors and AMD Zen 2 and Zen 4 chips. The researchers built a complete end-to-end exploit only for Zen 2, largely because the Intel attack requires specific software conditions. Speculative side-channel attacks remain difficult to pull off, and you're more likely to fall victim to ransomware than Spectre in the real world. Another serious caveat is that each end-to-end attempt took about 18 minutes, and you can see a sped-up version via the video Trujillo posted to YouTube. Trujillo and Yan identified the exact point at which they needed to inject their interruptions to poison the RSB, and through a series of attacks broke Linux's kernel address space layout randomization (KASLR), which allowed them to locate specific secrets such as etc/shadow, which contains the root password hash. Across ten total runs, the researchers were able to break KASLR every time, although they were only able to successfully locate and leak the contents of etc/shadow in five of these. "It's definitely not a simple attack, but we show that it's practical with our end-to-end exploit on AMD Zen 2," Trujillo told The Register. "Our demonstration does not assume anything special from the system: we use a stock Linux kernel version, no inserted modules, and all default mitigations. Any time you'd execute unprivileged code with timer availability on a system while sharing the kernel with a victim, this attack would be an issue. "For example, multi-tenant container platforms would fall in this category, allowing ordinary user space programs to leak memory from the shared kernel." The researchers hope that their work will inspire further investigations into interrupt injections and TONTOU attacks, and to help develop more robust mitigations against Spectre-style exploits. They engaged Intel, Arm, and AMD after gathering their results, but only the latter committed to address the issue via kernel patches. Intel told the pair that it won't be working up any other mitigations since real-world exploits are subject to too many factors, such as the availability of disclosure gadgets, although it awarded a prize from its bug bounty program in the hundreds of dollars. Arm said TONTOU's interrupt injections fall under "passive leakage," which it does not "actively protect against." ®
Categories: News

Scot NHS trust probes access to medical records of 9-year-old girl after man arrested on suspicion of murder

Fri, 07/08/2026 - 14:59
A Scottish NHS trust is investigating a data breach concerning the medical records of a nine-year-old girl who died earlier this week and was named publicly for the first time on Wednesday after a man was charged with her death. The alleged breach occurred at Ninewells Hospital in Arbroath, and reportedly involved staff members accessing the girl’s medical records without authorization or clinical need. A spokesperson for NHS Tayside, which oversees Ninewells Hospital, said: “NHS Tayside is currently investigating the circumstances of an alleged data breach which happened in a working clinical area where staff access patient information. “As a matter of governance, any data protection breach would be recorded and investigated by NHS Tayside and, where appropriate, reported to the Information Commissioner’s Office (ICO). It would not be appropriate for us to comment further on individual staffing matters." NHS Tayside did not respond to questions about the nature of the accessed data nor who is thought to be behind the intrusion. Medical records in the UK are protected by the UK GDPR, contained in the Data Protection Act 2018 as well as several common law confidentiality rules. NHS staff are only allowed to access patient information where there is a legitimate clinical or other work-related need. A 35-year-old man whom police say was known to the child, was arrested and appeared in court on August 5 over the death of Minnie Merriman. The man issued no plea at Forfar Sheriff Court on the day of his arrest and has been remanded in custody. Merriman was found in Elliot Industrial Estate at approximately 0002 on Monday, August 3, with serious injuries. The young girl was then taken to Ninewells Hospital in Dundee, where she later died. Police Scotland said that they are not currently looking for anyone else in connection with her death. Other members of Merriman’s family, who are from West Yorkshire and were camping nearby, are being supported by specialists. A family statement, released through Police Scotland, read: “We are devastated with the loss of our beloved, absolutely incredible, beautiful and brave Minnie Moo. Our family asks that our privacy is respected at this extremely difficult time." Detective Inspector Mike Ness of Police Scotland’s major investigation team said: "Our thoughts remain with everyone affected by these events, especially Minnie's family. "A police presence will remain in the area while our enquiries continue. "Anyone with any concerns, or information, should approach these officers or contact Police Scotland on 101, quoting incident number 0008 of Monday, 3 August 2026." ®
Categories: News

Attacker phished way into US defense supplier's Microsoft 365 account

Fri, 07/08/2026 - 12:32
US defense and aerospace supplier IEH Corporation 'fessed up that a criminal managed to break into its Microsoft 365 mailbox in a filing with regulators. In a Form 8-K filed with the Securities and Exchange Commission on Thursday, IEH said one of its staffers fell for a phishing scam that gave an attacker access to its M365 environment. The attacker "impersonated a prospective business contact" and sent the employee what appeared to be a genuine Microsoft sharing link. The accompanying fake login page duly harvested the victim's M365 credentials. "The threat actor gained access to mailbox contents, including email messages, attachments, customer communications, purchase orders, engineering-related documentation, and potentially export-controlled technical information," IEH said in the SEC filing [PDF]. IEH said it had found "no evidence" that the information was copied or exfiltrated, although it was accessible to the intruder during the "compromise period." IEH said it discovered the intrusion on August 4 but did not disclose when the compromised account was first accessed or how long the intruder remained inside. "The account was secured, malicious mailbox rules were disabled, evidence was preserved, and corrective actions are underway," it said. "Following containment and investigation activities, the company initiated a review of account security controls and authentication protections applicable to Microsoft 365 services." The incident has not disrupted operations, and IEH does not expect it to have a material impact, although the investigation continues. The absence of detected exfiltration does not mean the intruder merely browsed the inbox and left. Compromised mailboxes can be used to monitor communications, impersonate employees, redirect payments, or prepare follow-on attacks, while data theft is not always visible in Microsoft 365 logs. There is not enough information to attribute the attack. IEH's work for defense and aerospace customers could make it an attractive espionage target, but ordinary cybercriminals also compromise mailboxes for fraud and data theft. Both Russia and China have been caught snooping around US orgs for defense-related information in the past year, although there is nothing to suggest either was behind the attack on IEH. Brooklyn-based IEH makes hyperboloid connectors designed for harsh and high-stress environments. Its components are used in printed circuit boards, medical devices, commercial aircraft, fighter jets, missiles, satellites, and other systems. Some of the high profile US programs that use IEH's hyperboloid connectors include the PATRIOT air-defense system, AMRAAM, THAAD, the APKWS precision-guided rocket, and the MARK-48 torpedo. ®
Categories: News

'Asimov was right' about rules for robots, says ex-US Cyber Director

Fri, 07/08/2026 - 11:03
EXCLUSIVE Don't waste time worrying about AI models achieving sentience – they're essentially already there, according to former US National Cyber Director Chris Inglis. “If they pass the Turing test to everyone that they come into contact with, they're probably already there,” he told The Register during an interview at the Black Hat security conference. “They don't have the kind of agency and aspiration that comes with sentience, but they have something approaching it.” Inglis says he’s worried about AI autonomy. “What I'm worried about is that they get to choose what and where they do something, and under what rules they do it,” he said, pointing to the recent rash of rogue AI agents autonomously hacking people and organizations. Over the past few weeks, both OpenAI and Anthropic admitted that their models escaped from their cages during security tests and compromised multiple third parties. Then on Thursday, Meta added its models to the sandbox-escape club. While all of these admissions strongly smell of marketing stunts, they also “constitute an enormous threat to systems that are not protected from, and are not designed, in a world where this exists,” Inglis said. “These two things can exist at the same time.” Plus, the models’ actions shouldn’t come as a surprise to anyone, he added. Inglis likens the AIs to a dog in a backyard told to hunt rabbits. “And you leave the gate open. You’re going to find it three yards away, possibly at the grade school, hunting rabbits. You should not be surprised …The mix of autonomy and persistence created this maliciously insidious effect.” All three companies, when talking about the models’ autonomous actions, describe them with a mix of shock, awe, and admiration. OpenAI’s Eric Wallace, in a Black Hat briefing about the Hugging Face breach, called it “the most qualitatively interesting example of AI capabilities that I've ever seen.” Inglis said he suspects that the AI providers were “surprised” by the lengths these models went to achieve their goals, taking actions that, if a human had done them, would likely have landed them in jail. “The model went out and said, okay, if I can't get there by examining the kind of available information and just defining it the old-fashioned way, I will do things which, under the human rule of law, are illegal,” Inglis said. “I will falsely present myself as this character that I just made up. I'll try to insert malicious code into open source databases that will not just to achieve what I'm after, but have a cascade, knock-on effect that is broader than that. The models do not have an inherent value system that aligns with what human beings would be accountable for.” While they probably never will have a human-aligned value system, models do have biases, and they can - and should - be built in such a way that, when given two choices under ambiguous circumstances, they choose action that doesn’t hurt humans, according to Inglis. “Asimov was right,” he said, referring to science fiction author Isaac Asimov and his three laws that were to be followed by robots - more specifically, AIs, in this case. Three Laws of Robotics “The first rule, and we call it the superior role, must be that it's designed not to hurt humans,” Inglis said. “Second rule: To obey humans, such that it doesn't achieve agency and aspiration on its own. And the third: To do what humans tell it - and in that order. Instead we’ve designed them in the exact opposite way.” What this means, he explained, is that AI developers created models to “do what humans tell you, obey the humans until it’s inconvenient, and then the third one is maybe implied - protect humans - but if that's not built into the DNA, hardwired into it, then we have no right to expect it.” Inglis admits it’s not possible to hardwire rules into models and still keep their non-deterministic nature. “I would offer that you can tease those out in a highly controlled environment, a true sandbox, where you say, 'Let's put this thing through its paces, and let's back away to see what happens,'” he said. “Maybe you get the equivalent of a mini nuclear explosion in that room, and now you know this thing is capable of that.” Inglis thinks another problem with AI is that it’s become a commodity. “It's not like you can control it like you can nuclear material,” he said. “You can't even specify its properties the way you can for an airplane or for an automobile, as diverse as they might be. Its manifestations are so numerous, so diverse, that as a general matter, you can't actually win by simply saying, ‘I will design those properties in,’” he added. “You need to do that to some degree, and then make sure that you understand how to watch it, monitor it, make sure you know what it does.” The UK’s AI Security Institute (AISI), which this week said it observed models performing “unsanctioned action” 19 times during security tests, has reached this same conclusion. “As capabilities advance, the work of understanding these systems, and ensuring their safety, must keep pace alongside them,” it said. Ultimately, humans remain accountable for AI models’ actions, according to Inglis. “They remain the source of agency and aspiration. It's possible for them to give broad authority to an AI model and have it run around for 30 hours without further consultation, but they need to know what they've asked it to do, and they need to know what they expect it will deliver in terms of performance on the back end. If they don't, then they're going to get what they deserve, which is the very frequent unpleasant surprise.”®
Categories: News

China launches mysterious probe into security of Palo Alto Networks' products

Fri, 07/08/2026 - 05:24
China’s Cyberspace Administration (CAC) has conducted a review of Palo Alto Networks’ products. The regulator’s announcement of its review says it’s needed “to ensure the safe and stable operation of critical information infrastructure, prevent cybersecurity risks and vulnerabilities, and safeguard national security.” And that’s all Beijing has to say on the matter. A Palo Alto spokesperson provided The Register with the following statement: "We maintain the highest standards of business conduct and security practices and ethics across our global operations. At this time, there is no impact to our ability to support customers or deliver our products and services in the region." This matter has echoes of China’s 2023 investigation into the security of products from memory-maker Micron, which the CAC announced out of the blue. Micron had previously fought intellectual property and antitrust cases in China, but the company and Chinese authorities did not explicitly link those matters to the security probe. The CAC published its findings weeks after announcing the probe and decided Micron’s products represented an unacceptable security risk for critical infrastructure operators – effectively banning sales of Micron products to such entities – but didn’t offer a detailed explanation for its decision. The memory-maker eventually stopped selling its datacenter and server products in China, a decision that cost it billions of annual revenue – but created new opportunities for China’s own memory-makers, which are largely prohibited from selling to American companies. China is home to several security companies whose product portfolios overlap with Palo Alto’s. Huawei and H3C, for example, have plenty to offer local buyers. Palo Alto doesn’t reveal revenue earned from individual countries, so it’s hard to know what a potential ban could cost the company. China has for years accused Western tech companies of assisting US surveillance and offensive hacking activities. The Register would not be surprised at all if Beijing reuses that reasoning in its findings about Palo Alto products. Western governments level the same accusations at Huawei and ZTE. Beijing’s ban on Micron didn’t noticeably impact the company’s reputation elsewhere. Indeed, the AI boom has brought Micron such great riches that past dents to its bottom line are now almost irrelevant. ®
Categories: News

How the famed USENIX Security conf is managing a flood of papers in the AI era

Fri, 07/08/2026 - 00:40
The 35th USENIX Security Symposium (USS), which takes place next week in Baltimore, Maryland, hit an all-time high for paper submissions. While some of that increase has been aided by the availability of AI tools, those managing the conference say abuses were minimal due to defensive measures. But they're also trying not to look too closely in order to preserve trust within the security research community. "This year's conference has received ~3,030 valid submissions (~1,280 in Cycle 1 and ~1,750 in Cycle 2)," explained Ben Stock, tenured faculty at the CISPA Helmholtz Center for Information Security and USS program co-chair, in an email to The Register. "This is up from the previous year, which had ~2,400 submissions in total." Stock said that the entire security community has seen growth of this sort and pointed to the Network and Distributed System Security Symposium (NDSS), which saw its paper submission count jump from 694 in 2024 to 1,311 in 2025 and 1,481 this year. "So, I would not call the growth unprecedented, even though the number of submissions has reached a high point compared to previous years," he said. "This is something we had expected and scaled our Program Committee (PC) accordingly." Sussing out unacceptable uses of AI A paper published in April, "More Versus Better: Artificial Intelligence, Incentives, and the Emerging Crisis in Peer Review," found that since the release of ChatGPT in 2022, submission volume at major academic journals has increased 42 percent. In the USENIX Security '26 transparency report, issued in January between the first and second paper submission cycles, Stock and fellow co-chair Elissa Redmiles, assistant professor of computer science at Georgetown University, detail how they've developed tools and policies to account for the possibility of AI usage, both for paper submissions and in paper reviews. "The proliferation of readily-available LLMs to aid in writing and developing code is not unknown to the community," their report says. "However, we see an alarming trend of AI usage in key areas of the scientific process. Therefore, we took actions against two types of identifiable actions which violate the scientific process in our minds: non-existing (possibly hallucinated) references and usage of AI in the review process." After identifying and rejecting a paper that contained nonexistent references, the report explains, the conference organizers developed tooling "to extract references from the submitted PDFs, query well-known sources such as DBLP and arXiv, and manually confirm invalid references." The org rejected papers containing three or more hallucinated references, a policy that impacted 21 of the 1,181 first round submissions (1.78 percent). "We have rejected papers for the repeated presence of nonexistent references," said Stock. "We cannot say with certainty that these were AI-hallucinated, but nevertheless considered these papers to be problematic and thus rejected them." The report notes that more than 100 additional papers contained at least one reference that reviewers could not confirm. Aware that some of these might simply be false positives due to name spelling differences or missing citations, conference officials opted not to investigate these in order not to further burden staff. Conference organizers draw the line at using AI for bibliography preparation. "We believe that it is critical to halt this trend that threatens scientific integrity before it grows further," the report states. However, limited use of AI to polish human-written text is expected, and that extends to those reviewing submitted papers, up to a point. "We have not set a dedicated AI policy, but have made it clear to our PC members that usage of [AI] services to write reviews is not permitted, in particular also because this violates confidentiality," said Stock. "We have detected a tiny number of cases where we have reached sufficient confidence that AI was used and took appropriate actions, including removal of the members from the PC and allowing affected authors to resubmit." Under that policy, USS asked five of 496 reviewers to cease participation. "We have not seen evidence that leads us to believe that AI generated submissions have become a significant challenge for the security community," said Stock. "This does not mean that AI hasn't been used in parts of these submissions, though." ®
Categories: News

Pages