The Undelivered Letters VR
“The Undelivered Letters” is an immersive journalistic documentary virtual reality project that pays tribute to victims of the Taiwanese White Terror. It aims to shed light on stories that have been buried in the archives for decades, and breathing new life into the realm of history and politics through VR.
In this project, we explore how emerging media technology can advance the field of journalistic documentary and history. We have chosen to focus the project on a dark political period in Taiwan from 1945 to 1987, post WWII through the Cold War. We hope to build a new kind of interactive VR documentary to memorialize the era, and help its stories and their impact live on.
After World War II, Chiang Kai-shek and his son ruled Taiwan and imposed martial law until 1987. Chiang repressed the people who dared to question his corrupt dictatorship, or showed sympathies to communists. During this time, roughly 140,000 innocent Taiwanese were imprisoned, and over 4,000 people were executed. This era later came to be known as the “White Terror.”
During the White Terror, some imprisoned victims were able to write goodbye letters to their families before they were executed. However, the government didn’t release them until half a century later. Some of the recipients passed away and never had the chance to get the letters from their loved one. I chose three of them to present in my thesis project. Their names are 高一生 (Yi-Sheng Kao), 劉耀廷 (Yao-ting Liu), and 黃溫恭 (Wen-Gong Huang).
We believe if we want to prevent future cycles of state violations, we must revisit history and learn from past trauma. The intended audience of my project will be young generation around the world who are not familiar with the worldwide state violence in the Cold War.
Based on the research we made, we visited the jails in Taipei that once imprisoned the political victims. We used photogrammetry to recreate the jails in VR, allowing the users to immerse in the victims' last minute.
Moreover, we interviewed the victims’ families, including Wen-Gong Huang’s daughter Chun-Lan Huang, Yao-ting Liu’s daughter Men-Ni Liu, and Yi-Sheng Kao’s son Ying-Jie Kao. We even visited Wen-Gong Huang’s and Yi-Sheng Kao’s former residences, in hope to showcase the buildings within the VR experience.
For Wen-Gong Huang’s story, we recreated his daughter’s childhood memory of sitting under the big old tree in the courtyard with her families, where her father was always absent.
For Yi-Sheng Kao’s story, we represented his son’s childhood memory of seeing the Rhododendron all over the mountains around their home. It is also the scene that Yi-Sheng Kao missed most in the jail, which made him created the song "Rhododendron mountain" when he was imprisoned.
For Yao-Ting Liu’s story, we recreated a lake scene based on his daughter’s old photo shoot in Sun Moon Lake. Since their former residences were teared down, we found an old train station’s 3D model to symbolize the dislocation her families suffered due to her father’s imprisonment.
We also recreated the letters, photos and related artworks into audio file, 3d models and animations. We combined all of them in Unity and HTC Vive.
User Journey map
When the user wears the VR headset, she or he will be transported to the main scene, which is the prison cell.
While the candles are lighted, the user will be able to freely explore the private belongings, including letters and photos, of three White Terror victims. Focusing on any such letter will transport the user to a new scene, specific to a victim. The user will hear the victim’s letter, see his photos, view the victim’s hand-writing of his letter in real time and experience the victims’ last moment. By this process, the user will feel that the victim is not only a “victim,” but a father, or a husband.
As the user approaches the end of the experience, the user will be sent back to the main scene to explore another story. After the user experiences all of the stories in the prison cell, the prison door will gradually open, letting in sunshine, symbolizing that history is finally known by the people.
1. Jail with “memories”
When the user wears the VR headset, she or he will be transported to the main scene, which is the prison cell. The main scene is populated with embossments that were recreated from a White Terror victim, Wu-Jan Chen. Wu-Jan Chen made oil paintings that portrayed what he and other victims experienced in prison. We gave depth to the paintings to create embossments as part of the jail scene.
2. Animation of Hand Writing
The main visual effects in most scenes are handwriting animations. We recreate the letters’ hand writing in animation, to mimic the victims’ last moment, as they tried to reach out to their loved ones.
1. Bilingual audio recording: We recorded the letters both in English and its original languages, including Chinese and Japanese because We try to maintain the original tone, and offer a translation for English speakers.
2. Background music design: The background music in every scene has strong connections to each victim. For example, the background music of Yi-Sheng Kao's scene is recreated from Yi-Sheng Kao's song, which he created for his wife.
I. Raycase for changing scenes: We used “Raycase" to change the scenes. This means, when the users look at the letter, it will automatically transport the user to a new scene related to the letter’s writer. The reason why we used “Raycase” to change the scene is many users feel like “they are playing a game”, rather than “living inside someone’s memory” when they use handheld controllers.
2. One-way experience: After the users watch the story and return to the main scene, they won’t have the chance to go back again. It prevents the user from accidentally jumping into the same scene again. Conceptually, it means the memories and moments are fleeting.
3. Comfortable transition: We used fade in and fade out effect between scenes to make the transition comfortable and smooth. Also, it can create the feeling of immersing in memories.
VR is particularly suited to stories which have strong connection with spaces, including one significant event takes place in a defined space, or a historical event took place in a space disappeared forever. Instead of watching news or reading articles from outside, VR is a powerful way to place viewers on the scene of an event and engage the audience emotionally. Today, most VR journalism is shot in 360 video, but it’s not as compelling as constructed VR, because constructed VR allows audiences to reach out, react, and feel emotional connections to stories.
Based on the exploration, my next step is to refine the project by interviewing more victims’ families and represent thier stories in VR. I hope to build a new kind of interactive VR archive to memorialize the era, and help its stories and their impact live on.
The prototype of this project was presented in 2017 VR days. After a year’s refinement, the final version of this project had the world premirme in Taipei on May 2019. You can check the difference between the prototype and the final versin via the video of the prototype:
The showcasing of this project requires a 4 by 5 meter space. The space will be decorated as a prison cell with some images about the victims’ suffering on the walls and the ceiling. Three victims’ letters and their family photos will be on the floor with candle lights for the users to explore. The layout of the physical space will coincide with the layout within the VR experience.
Producer & Director：Nai-Chen Yang
Developer：Shawn Chen, Che-Chuan Weng
Technical Artist：Tina Lin, Shawn Chen
Lighting Design：Ping Chang, Tina Lin
Voice Actor：Ying-Jie Gao, Zhi-Yuan Ceng, Jia-Qi LI
Sound Design：Yan-Ting Xu
Short Biography Director/Creator
Nai-Chen Yang, graduated from New York University (NYU)’s Interactive Telecommunications Program (ITP), focusing on emerging media technology and art. She is driven by the conviction that good stories are powerful tools for positive social impact. To explore immersive ways of storytelling, she has created projects in the realm of 360 video production, virtual reality, augmented reality, and mixed reality. Her projects were showcased in 2017 VR days, 2018 Taipei Poetry Festival, Jing-Mei White Terror Memorial Park, and won the First prize of Sustainable and Constructive News Award in 2018. You can find the related projects in her website: https://naichenyang.com