In Houston, a man who works in finance asked a machine for chicken wings and drumsticks. He wanted to cook dinner for himself and his partner. He did not say how much. He expected a small pack. What arrived at his door was twenty pounds of chicken wings.
The man is Austin Welch, and he described the episode to Alex Bitter of Business Insider with more amusement than grievance. “We were shamelessly eating chicken wings for about five or six days straight for every meal,” he said. His theory of the error is modest. The agent, called Instinct, had seen some larger grocery purchases in his past, and he suspects it decided he wanted that quantity again.
A reflection, and only that: every new servant learns a household by its habits, and the danger lies in that diligence, because a habit is a record of what someone once did and says nothing about what they want now. The machine had read the household’s history and drawn its conclusion from it. The freezer paid the price.
The lookouts
Welch got his invitation to Instinct in late August. Since then he has used it for several kinds of purchase. He gave it a list of ten flights he has to take over the next six months, asked it to watch the fares and to ping him when it thinks each one is at its cheapest. It turned up a referral code for a United Airlines-branded credit card with a bigger miles bonus than he could find anywhere else. He also texts it his grocery orders, a habit that has become, in his words, “really, really useful in my life.”
In Minneapolis, Amando Komoda runs a business selling supplies for dry-aging meat. He uses Instinct as a sentry. Instead of searching for scarce goods himself, such as a Raspberry Pi chip that is hard to find, he has the agent set up alerts and tell him when they appear. In this arrangement the agent keeps watch and Komoda decides what to buy.
The final button
Mostly Komoda makes the purchase himself. One reason is mechanical: he has found that Instinct cannot always solve the Captcha puzzle that some checkouts demand. There is a small irony in that. The test was built to tell people from machines, and at some shops it still does.
The other reason is trust. When he does let the agent buy, he gives his permission first and pays with a single-use credit card, so that the agent cannot run up charges he did not authorise. “I wouldn’t give it access to my best credit card,” he said. “I’ve watched enough sci-fi movies that I can’t trust it completely.”
Loni Stark, vice-president of strategy and product at Adobe, has kept the payment question at arm’s length in a different way. In February she built her own agent, Molty, using OpenClaw. It researches products she is thinking of buying, but she has not connected any way for it to pay.
Her second agent, Museli, is a custom version of Meta’s Muse, and it has been allowed further in. Through Nordstrom’s website it bought her a pair of ballet shoes, handling the whole transaction and asking her permission to pay before placing the order. But it left out her loyalty number. So she gave it a more draining job: petitioning Nordstrom’s customer support for the rewards points she had missed. The petition failed. “I figured maybe Museli could negotiate something for me,” she said. It was, she thought, worth a try.
The gatekeepers
Every agent depends on the shops letting it in, and some shops do not. On 20 September, GeekWire reported, Amazon cut Muse off from shopping on Amazon.com. Users who tried were met with a popup declaring that “continued access by an unauthorized AI agent violates Amazon’s Conditions of Use.” According to that report, Amazon complained that Meta had not told it Muse would use its store, that the agent does not identify itself while browsing, and that it appears to capture and store customers’ login details. Meta answered that Muse has no visibility into people’s passwords or payment methods. The standoff, covered here earlier, showed that the big retailers mean to decide for themselves which agents get through.
Other gates have opened instead. Meta has struck a deal that lets Muse users have their agents buy from Shopify merchants, with checkout through Shop Pay, as the Motley Fool noted in setting the two retailers’ opposite approaches side by side. Wariness about these agents also has a basis beyond science fiction. On 8 October Malwarebytes reported that TIME had analysed Muse’s internal instructions and found the agent keeps constantly updated files on its users and the people they know. Agents that act on people’s behalf have already paid the wrong fees and signed forms their owners never meant to sign.
The season of gifts
The holidays are coming, and with them the first big test of how much shoppers will delegate, if they delegate anything at all. Vipin Porwal, founder and chief executive of Smarty, an online shopping rewards platform, expects many shoppers to use agents to decide which gifts to buy. He also expects many of them to stop short of letting the agent pay, given how reluctant people are to hand an agent their payment details.
“It’ll be research, finding the best price, and then consumers still go and press that final buy button.”
Stark does not see the agent replacing the shopper. For many people, she said, shopping is a form of entertainment: visiting stores, scrolling on a phone in search of a good deal. Agents may add to those pleasures without taking them over. What they will take on, in her account, is something people would rather be rid of.
“We don’t like the anxiety of wondering whether we got the best price or not,” she said. “We’re going to offload that to agents.”