If you have an Amazon account, there is a page somewhere in it where Amazon tells you, in plain English, what it has concluded about you. It is called “About You,” and this week people discovered it in large numbers and began posting the results. Some of them are very specific. One user was told they have “flat buttocks.” Another learned that they use their phone in the shower to read Kindle books. Another found out, or at least found out that Amazon believes, that their husband has “skinny legs.”
The feature itself isn’t new. Amazon launched it in May, and the company’s own explainer pitches it as a way to see and edit the details that shape your recommendations. The profile draws on your searches, your purchase history, the items you’ve saved to lists, the product reviews you’ve written and your conversations with Amazon’s Alexa for Shopping assistant. Amazon says the point is to connect customers with products that better fit their needs, and you can update or delete what it has written. The example Amazon gives is nice: chat with Alexa about your cat Benson’s favourite foods and it can recommend treats in the flavours Benson likes. What went viral is the rest of the page, the part where Amazon has drawn its own conclusions about you without being told anything about Benson.
Business Insider, being a newsroom full of people who cannot resist a quiz about themselves, ran the experiment on its own staff. Ben Shimkus, who has no Amazon account and so could only collect results, found the profiles typically ran to 30 to 45-ish observations per person. Thomas Gryta, the interim corporate editor, was told he really likes beef jerky. “I’ve ordered more energy bars and vitamins than jerky,” he wrote in Slack, adding that the profile “feels like a judgy accusation from a nutritionist.” Taylor Rains, a partnership editor, got this: “used to like Ryan Reynolds, but not so much anymore.” Her reaction: “How does it know?” Deputy executive editor Taylor Berman’s account is apparently optimised for “bathroom safety,” likely because he bought a shower chair after tearing his Achilles tendon last May. (He asked that readers be told his other purchases were “very cool.”) Deputy executive editor Debbie Strong was classified as a “boy mom” on the strength of a preference for Hot Wheels. One editor was told he had a girlfriend named “Barry.”
The pattern in the newsroom results is instructive. Amazon was mostly right, by that very unscientific count, and it was very good at the things it can observe directly: where you live, what groceries you buy at Whole Foods, whether you have pets. It got worse as it moved from transactions to people. Several staffers said it miscounted their kids. Paul Squire, deputy editor of homepage and programming, shares an account with his wife, which made the profile “a bit warped”; it knew about the cat and the MacBook but listed him as having one cat. “Try again, Jeff Bezos,” he wrote, on behalf of a second cat named Sylvie.
This is the basic problem of inferring a person from a receipt. A purchase is a fact. Why you made it is not in the data. If you buy Pokémon cards, Amazon sees a person who buys Pokémon cards; it does not see that they are, as many BI staffers insisted, for the kids. A household with one login looks like one strange person with several bodies. The model’s guesses about the facts are fine; its guesses about the story behind them are where Barry comes from.
The 515-page loyalty file
Amazon is not unusual in building a file on you. In a Business Insider newsletter, Dan DeFrancesco, whose own profile said he “performs manual work or construction” (he writes a newsletter), points to a better example. In August, Wired senior writer Reece Rogers used his rights as a California resident under the state’s privacy law to ask McDonald’s for the data its loyalty app held on him, Marketplace reported. He got back 515 pages. “I had no idea that it was going to be this extensive and granular,” he told Marketplace.
The best part of the file is the forecast. According to Vice’s write-up, McDonald’s predicted Rogers would visit 2.16 times over the next six weeks, spend an average of $13.49 per order, and spend $29.15 in total. Multiply 2.16 by $13.49 and you get $29.14, so the model is at least internally consistent, give or take a rounding. It also gave him an “attrition likelihood” of zero, meaning McDonald’s was confident he was not going anywhere. This is not, by fast-food standards, a heavy user. QSR Magazine argued that a few years of transaction history running to 515 pages is fairly ordinary, and that what unsettled people was seeing the back-office language, “attrition likelihood” and the like, written out. Which is roughly what happened with Amazon, except Amazon translated its back-office language into sentences about your buttocks.
Why you are being asked to help
DeFrancesco’s real point is about where this is heading. AI companies want their agents deep inside people’s lives: email, calendars, texts, bank accounts. Customers increasingly hand that access over voluntarily, because not having to think about reordering paper towels is genuinely nice. Meanwhile the companies are spending billions of dollars on these tools and need revenue from them. Subscriptions are the obvious way, but plenty of users won’t pay. The other way is the data: an assistant that orders for you can also steer you toward particular brands.
I think the interesting thing about About You, from Amazon’s side, is the edit button. A profile that sits in a database is useful. A profile that the customer reads, laughs at, and then corrects is more useful, because now the customer has done the labelling for free. Every staffer who pointed out the wrong number of kids, or the second cat, or the nonexistent Barry, was pointing out an error that Amazon would very much like fixed. The viral screenshots are a marketing problem and a privacy story, sure. They are also an enormous crowdsourced quality-assurance exercise.
So Squire’s “Try again, Jeff Bezos” is right, in a sense. Amazon would like nothing better than for him to tell it about Sylvie.