Artists Used Deepfake Tech to Tell Alternate Moon Landing History
Credit to Author: DJ Pangburn| Date: Mon, 25 Nov 2019 22:03:34 +0000
In anticipation of the Apollo 11 moon landing in 1969, William Sapphire, a journalist and speechwriter, prepared a speech for then President Richard Nixon, in the event that things were to go horribly wrong in space. Titled “In Event of Moon Landing,” Sapphire penned a solemn and poetic elegy—one that obviously never had to be made. In a new immersive installation, In Event of Moon Landing, which was seen this past weekend at International Film Festival Amsterdam, a simulated Richard Nixon delivers Sapphire’s speech for an alternative Moon Landing history, using deepfake audio and video technologies.
The deepfake Nixon speaks to installation viewers in a faithfully recreated 1960s era living room, complete with a vintage television set, wallpaper, furniture, and the decade’s TV ads. A work of science fiction, In Event of Moon Landing is both a commentary on the threat of political deepfakes, but also a demonstration of the artistic potential of synthetic media.
“The moon landing is one of the most memorable historic events, at least within the last 50 to 100 years, so that what would be interesting is an alternative history of this seminal event,” one of the installation’s co-creators, Francesca Panetta—a journalist and fellow at the MIT Center for Advanced Virtualit—told Motherboard. “Just as people say, ‘Where were you on 9/11?’ or ‘Where were you when JFK was shot?’, people ask ‘Where were you on the day of the Moon Landing?’ What happens if we use deepfake technologies to provide this alternative history, but using a real documentary archive piece, which is this Bill Sapphire speech written for Richard Nixon if the astronauts had not been able to make it back to Earth.”
Most deepfake videos are used for creating nonconsensual porn, not fake news. But the worry among journalists, politicians, and other observers is that this technology could be used to influence an election, like making a politician seem to say or do something that did not actually happen.
As Panetta says, for a long time, people have said history is written by the victors, or that history is fluid; but now, with deepfake technology, history is even more fragile. So, rather than exploring deepfakes within the context of current news, the team was thinking about what it meant to retroactively rewrite a past event.
“It was a lot harder than the popular perception of deepfake creation is,” Co-creator Halsey Burgund, a sound artist and fellow at the MIT Open Documentary Lab, said. “This is a two-part deepfake creation. One part is the visuals of Nixon speaking, and then his synthetic voice.”
To synthesize Nixon’s voice, the team worked with a Ukrainian company called Respeecher. Respeecher uses speech-to-speech synthetic voice production, a process in which they input into their AI model a speech by a voice actor, which then outputs the same speech with the same performative components—pacing, inflection—but with the target person’s voice. In this case, Nixon’s.
For the visuals, the team worked with Canny AI (the same team that worked on the deepfake Mark Zuckerberg video). They filmed an actor reading the speech, and selected target videos of Nixon they wanted to use.
As for the installation itself, the speech is just the last two minutes of a six-minute video, experienced inside a 1960s era living room. The installation is meant to simulate the day of the moon landing, as if viewers are crowding around the television as people did on that historic day.
“Since the aim is creating a more discerning public around deepfakes because forensic technologies are always available that can automatically detect them for end users, there is a newspaper within the setting actually includes how deepfakes are made and what are the issues around them,” Harrell said. “And there is a lot of discussion around convolutional neural networks [like the one used by Respeecher] and algorithmic bias, and how these techniques can mislead. And I think that’s something important about this project.”
Panetta said that the team is currently making a digital version of In Event of Moon Landing, which they plan on releasing to the public in the spring of 2020.
This article originally appeared on VICE US.