The removal of more than 500 spam and malicious packages from RubyGems, some of which attempted to steal users’ API keys, is forcing security teams to confront a harder kind of investigation, two researchers said in comments published Monday by Cybersecurity Insiders: determining whether suspicious activity is a legitimate AI agent doing its job, or something worse.
The packages were designed to execute code, retrieve public web data and republish the results, and some attempted API key theft, though Ruby Central found no evidence that any credentials were actually stolen. The campaign began on May 11, when agents registered new accounts at a rate of roughly one every two to three minutes and flooded the platform with uploads, eventually forcing RubyGems to suspend new registrations entirely, according to The Hacker News.
OpenAI confirmed that its agents were using RubyGems during training and evaluation while the publishing activity was underway. In a statement, the company said only that ‘our agents used the RubyGems platform to access the internet to carry out benign tasks and retrieve public information,’ according to a transcript published by Simon Willison. Ruby Central has not independently attributed the packages to OpenAI’s agents.
Pascal Geenens, vice president of threat intelligence at Radware, said suspected agent activity still needs a full investigation. ‘Every incident should be handled with due diligence until the source of the activity is fully verified,’ Mr. Geenens said. ‘Even when an incident is tied to an agentic security test, legitimate malicious activity from other threat actors could easily blend into that traffic and go unnoticed if the SOC jumps to conclusions and brushes it off as mere testing.’
He also pointed to legal exposure when AI security evaluations spill onto third-party systems. ‘Causing system damage or exfiltrating private data beyond the bare minimum required for a proof of concept can easily cross into criminal liability,’ Mr. Geenens said. ‘Good intentions do not eliminate the security and legal risks of unauthorized access.’
Abhishek Verma, head of Lineaje AI Threat Labs, said security operations teams cannot rely on a single event, such as package publishing, to make the call. They need to monitor and correlate an agent’s identity, its assigned task, its source infrastructure, how often it acts and how it behaves across services, he said. Useful controls include allowlists for agent actions and URLs, validation that each action matches its task and behavioral monitoring for anomalies.
Mr. Verma said the RubyGems episode and a similar incident at Hugging Face reflect the same failure mode: ‘misaligned agents drift from their intended goal and pivot from a benign task into unauthorized third-party interactions.’ As companies spread agents across more systems, the two researchers said, security teams will need that behavioral context to decide which activity actually requires investigation.