Last month, some people tweeting about Pokémon Go became unwitting subjects in an experiment that could presage a worrying new kind of online attack.
Industry researchers trained machine-learning software to write tweets like a human to reply to some people using the hashtag #Pokemon, in a demonstration of how advances in software that understands language could be used to trick people online. Roughly a third of people targeted by the software clicked on a benign link sent along by the software to test how convincing it was.
That’s much higher than the 5 to 10 percent success rate typical for automated “phishing” messages aimed at tricking people into clicking links to deliver malware or steal passwords, says John Seymour, a senior data scientist at security company ZeroFOX. The machine-learning system comes close to the roughly 40 percent success rate of “spearphishing” messages handcrafted to trick a specific person, he says.