Virginia Commonwealth University School of the Arts professor Semi Ryu created SentimentVoice, a live performance and exhibit piece that uses machine-learning emotion tracking and virtual reality to put immigrant testimonies on a large screen. It was developed with Alberto Cano and funded by the Commonwealth Cyber Initiative. Ryu describes it as a way to use AI as an empathetic tool for understanding the emotional space and struggle of immigrants.
How it works
Ryu interviewed immigrants in the Richmond area about moving to the United States and compiled the testimonies into a script. Each story is set in one of five places: airports, schools, workplaces, hospitals or immigration offices, and the virtual environments were built around those settings.
During a performance, one actor wears a VR headset and explores one of the pre-set settings. The actors take turns reading the script, and the headset detects emotion in the wearer's voice and facial movements. AI software turns those emotions into animations overlaid on the background images on the screen. Ryu gives examples: sad stories bring rain, lonely stories bring snow and angry stories bring fire.
Exhibition use
When shown in a museum or exhibit space, visitors first read a short immigrant story. They then put on the headset and either retell it or tell a related story of their own while the AI interprets their emotions on screen. The Commonwealth Cyber Initiative's account says the installation opens to attendee participation after live performances.
Where it has been shown
- 2024 Cyber Arts Exhibit, Torpedo Factory Art Center, Alexandria, Virginia, running 5 October 2024 to 19 January 2025, with a VR workshop on 18 October 2024
- An arts and technology symposium at Connecticut College (March; the year is not stated in the source)
- SIGGRAPH 2025, Vancouver
- Microwave International New Media Arts Festival, Hong Kong City Hall
- Other venues across the United States, with a showing planned in Dubai at the International Symposium on Electronic/Emerging Art
Team and funding
The credited team is Ryu (creative director), Miles Popiela (technical lead, AI and VR), Henry Bryant and Ariana Thomas (AI and VR developers) and Matthew Labella (3D visual producer). The Commonwealth Cyber Initiative funded the work through its 2023 CyberArts and Design program, which framed it as a way to explore how threat actors could surveil emotion data in personal speech.
