When Meta let half its employees route phone calls from its Muse AI agent through outside human operators starting September 16, those operators completed between 95% and 98% of the calls. Muse’s own AI-only calling finished a lower share, Meta told employees internally without saying how much lower, according to Reuters’ reporting on the test. Meta had just launched Muse’s calling feature publicly days earlier, pitching an assistant that could dial a hairdresser or a cable company on a user’s behalf without a person on either end of the line.
That gap between what an AI agent is marketed to do and what it reliably does on a phone is why a small cluster of consumer-agent startups now routes the hard parts of a task to a human being, as CNBC’s Jonathan Vanian reported. Marissa Mayer’s Dazzle AI, Meghan Joyce’s Duckbill, and Shivani Poddar’s Wajo AI all build agents meant to look autonomous to the end user while a contractor sits somewhere in the loop, ready to take over when the model gets stuck. “We’re relying on the frontier models to give us good suggestions and a good candidate list, but we’re checking that over to make sure that we’re really delivering the type of service that we can stand behind,” Mayer said of how Dazzle hands tasks like booking a contractor for a home renovation to people.
How the handoff works
Duckbill’s version of the mechanism is now a product. The company, founded in 2022 by Joyce (previously of Uber) and Emi Gonzalez, built a consumer assistant that pairs AI with human “copilots,” and in October launched Seaworthy, a developer API that lets any agent builder outsource the physical-world steps a model can’t complete: placing a call to book, reschedule or dispute something; making a payment by phone or check; running an errand like a pickup or return. Phone calls and a full-task “concierge” mode are live; payments and errands are in early access, according to Seaworthy’s own product page. “We work on a model that is not dissimilar from Uber, where we boil tasks down into atomic bite-sized pieces,” Joyce said. “And then we track what happens.” Seaworthy prices the work at about $5 per call on average, charged per completed task rather than per minute, and the company says the API is built on experience from 30 million prior real-world actions its contractors have already completed for Duckbill’s own consumer app.
| From | To | How |
|---|---|---|
| User request | AI agent / model | task |
| AI agent / model | Confidence check (flagged if call, judgment or glitch) | |
| Confidence check (flagged if call, judgment or glitch) | AI completes task | high confidence |
| Confidence check (flagged if call, judgment or glitch) | Human contractor (background-checked, 1099) | call / confusion / CAPTCHA |
| AI completes task | Outcome returned to user | |
| Human contractor (background-checked, 1099) | Outcome returned to user |
Based on CNBC and company statements
Mayer gave CNBC a concrete failure mode: a Dazzle user asked the agent to simultaneously handle a roof replacement, a broken window and the purchase of a glass tabletop. The model conflated the two kinds of glass and started working out whether the tabletop could be fit into the window frame. Dazzle’s system flagged the confusion and passed it to a contractor. “Within a few seconds, the human had things sorted out,” Mayer said. “Whereas an agent would have burned a ridiculous number of tokens. Or they may never have gotten to a good solution.” The economics run the other way from what the marketing implies: a human caught the error in seconds; an unsupervised model, Mayer suggested, could have burned compute indefinitely chasing a wrong premise.
Wajo AI’s agent, called Fo, takes the same approach to two different failure points: phone calls and CAPTCHA puzzles, the tests sites use specifically to block bots. Founder Shivani Poddar, who led engineering work at Google DeepMind before starting the company, told CNBC that “AI plus humans, collaborative AI, is what you need to actually get the job done,” warning that users abandon agents that can’t finish what they’re asked. Wajo has not disclosed funding but counts Vinod Khosla, Jeff Dean and Gokul Rajaram among its backers, and says Fo is now used in 107 countries and has grown tenfold since its public launch on September 28, pitching trust and a user-controlled “do not call again” list as its edge over larger rivals.
What Meta’s own test showed
Muse is the largest-scale version of this pattern, and the one with the clearest numbers and the clearest backlash. Muse topped U.S. app download charts in its first two weeks, with more than 2.5 million downloads according to Sensor Tower data cited in Reuters’ reporting. Internal posts described the human-concierge test, enabled for half of Meta’s staff, as a response to businesses that hang up once they realize they’re talking to a bot: one employee said his insurance company “kept hanging up on Muse when they hear it is AI.” The 95%-to-98% completion rate for human-handled calls came from that same internal test; the AI-only rate Meta compared it against has never been published, only described as lower.
The test did not survive contact with Meta’s own privacy standards. Employees objected that routing calls to outside contractors could expose sensitive information Meta had told users was isolated inside a secure, per-user virtual machine. One employee who asked Muse to call his internet and cable provider to negotiate a bill reported that the call transcript showed the human contractor making a racist remark. A Meta Superintelligence Labs vice president apologized and said the contractor would not work on Meta projects again, and the company rolled back the feature, calling the lack of disclosure to employees “a miss.” A Meta spokesperson, Daniel Roberts, said the company is “working with merchants to continue improving this potential calling feature, and will only roll it out when it’s ready and with the proper disclosures,” and separately called employee feedback on the test “overwhelmingly positive.” Meta told CNBC it currently does not use humans to complete Muse requests. The same gap between an agent’s advertised autonomy and the human labor propping it up runs through a recent accounting of what happens after an AI agent finishes the easy part of a task at enterprises using OpenAI’s agents, where IT staff still clean up what the model leaves behind.
None of the four companies discloses what fraction of a given user’s requests actually gets diverted to a person, which makes the headline autonomy numbers hard to check against anything. Mayer is at least explicit about the arrangement: “We’re not even trying to hide it, we think the human in the loop is a benefit,” she said, framing the current state of agent AI as a “transitionary period” until models can handle phone calls and web tasks as reliably as people. Meta’s own 95%-to-98% figure is the nearest thing to a hard comparison in this market, and it was measured on a one-week internal test with employees as subjects, against an AI baseline Meta has still not put a number on.