Jasmin Laine has hundreds of thousands of subscribers, but some videos of her discussing the Canada-US trade war contain words she says she never spoke. Laine is one of hundreds of content creators who, according to researchers at McGill University in Montreal, Canada, had their identities stolen by a network using artificial intelligence (AI) to inflame the online conversation around the crumbling ties between the North American neighbors. “There you have your face and your voice and what looks like you speaking, in some cases, to something that you actually don’t stand for,” Laine said.
The Canadian, left, and US flags flutter at the Lansdowne Port of Entry next to the Thousand Islands Bridge in Lansdowne, Ontario, on Feb. 12 last year.
Photo: Reuters
The McGill Centre for Media, Technology and Democracy study published earlier this month identified about 300 Instagram and TikTok accounts misusing authentic faces in a coordinated operation which published thousands of videos and pulled in about 20 million views. The personalities whose likenesses were appropriated ranged from Christian lifestyle influencers to political commentary accounts from both sides of the spectrum.
Synthetic, inaccurate and inflammatory online material has repeatedly targeted Canadians since the trade war began, but the researchers said the digitally altered footage of real creators made the network more difficult to detect. Before the launch of TikTok’s new deepfake detection tool on Friday, the Canadian researchers found only 4 percent of the operation’s accounts triggered the platform’s synthetic content warning. The study also reported that the network further used AI to generate the videos’ scripts, reposting those which garnered the most engagement. Aengus Bridgman, one of the report’s authors and director of the Media Ecosystem Observatory, said the network appeared to originate outside of North America. The researchers found certain accounts registered in China and saw content largely posted in the middle of the day in Asia. There was no clear evidence of the content being monetized, but Bridgman said whether the effort sought commercial gain or was solely a propaganda campaign “the idea is to polarize.” The videos displayed varying levels of accuracy — often disseminating outright falsehoods — and the accounts took inconsistent political stances. Some would blame Ottawa for the trade conflict while others pointed the finger at Washington. Canadian users appeared to be unwittingly interacting with the videos without realizing they were fake. Laine, whose content leans Conservative and often criticizes the approach of Canada’s ruling Liberal government, said she found herself defending against accusations of being paid to parrot talking points she never uttered because of the inauthentic content. The accusations levied at the real influencers was a “very good example of harm” inflicted by the operation, Bridgman said. “They have fallen hook, line and sinker for the network content,” he said. A TikTok spokesperson said that it had permanently banned the accounts mentioned in the report, along with several others, for violating its rules about misleading content. “As tactics evolve, we continue to collaborate with researchers on this industry-wide challenge and strengthen our moderation technology,” the spokesperson said. Meta was also reached out about the report, and while the company declined to comment it said it had taken down the implicated Instagram accounts. Ron Butler, a Canadian real-estate influencer who also had his identity stolen by the network, said this was not the first time he had been impersonated or misrepresented. He expressed satisfaction at the unmasking while also saying that he is resigned to the presence of deceptive online material. “You can’t stop this, because there’s too many people who can do it,” he said. “It’s too easy to do.” Laine also accepted the risk of being misrepresented online, but said she was frustrated to see the conversation about the trade war artificially inflamed. “It just flattens it into these very typical, very divisive talking points that I just find are really unproductive and unhelpful,” she said.